(this is for .Net Framework 4.7)
I'm trying to write up some extension methods to aid in creating dynamic where clauses for various entities. I started a few days ago, so there's likely a lot that I don't know and some that I probably misunderstood.
I managed to create one extension method already for filtering by 1 property which works as I expect it to (I did use reflection to get the property, couldn't get it working with an interface - well, without it executing the sql that is).
I can't seem to be able to get this one working for a lambda expression though.
Note, that the solution must not trigger sql execution. Because I was able to write up some variants that "worK', but they will trigger sql execution.
The way I work with this is that once I have the code ready, I start debugging and have the "query" in the watch. And it looks like this (notice the sql code)
Once I step over my FilterString method call, it either turns into a sql result, or I get an exception (with current code), which it shouldn't:
So here's my current code that throws the exception (currently not dealing with the "match" parameter, I am implementing an "equals" call. There will be others like, starts With, like, etc)
The exception is just one of those "type mismatch" having function cannot be passed as param to string Equals or what not.
public static IQueryable<T> FilterString<T>(this IQueryable<T> query, Match match,
string criteriaItem, Expression<Func<T, string>> getItemString)
where T : class
{
if (string.IsNullOrEmpty(criteriaItem))
{
return query;
}
var param = Expression.Parameter(typeof(T), "r");
var selector = Expression.Lambda<Func<T, string>>(getItemString, param);
Expression<Func<string, bool>> prototype = item => item == criteriaItem;
var predicate = Expression.Lambda<Func<T, bool>>(
prototype.Body.ReplaceParameter(prototype.Parameters[0], selector.Body),
selector.Parameters[0]);
return query.Where(predicate);
}
and the one that executes the sql instead of just generating it
public static IQueryable<T> FilterString<T>(this IQueryable<T> query, Match match,
string criteriaItem, Expression<Func<T, string>> getItemString)
where T : class
{
if (string.IsNullOrEmpty(criteriaItem))
{
return query;
}
var param = Expression.Parameter(typeof(T), "r");
//var value = Expression.Constant(getItemString);
var equals = typeof(string).GetMethod("Equals", new Type[] { typeof(string) });
var item = Expression.Invoke(getItemString, param);
var body = Expression.Call(Expression.Constant(criteriaItem),
equals,
item);
return query.Where(Expression.Lambda<Func<T, bool>>(body, param));
}
calling these is done like so
query = query.FilterString(match, criteria_value, (r) => r.SomeProperty.MaybeSomeOtherProp.SomeString);
query = query.FilterString(match, criteria_value, (r) => r.SomeProperty.Name);
This same extension method will be called on any number of different entities, with nay number of different properties and prop names. I guess I could make use of the reflection version I got working and passing in all the property names in some array of some sort, but that is just plain ugly.
So long story short, how can I get this working in the way I explained above, taht is: having the sql generated instead of executed?
Thank you,
Note, the "ReplaceParameter" extension method is the one from here: https://stackoverflow.com/a/39206392/630515
So, you're trying to merge your prototype item => item == criteriaItem. With a passed in string property expression, like (r) => r.SomeProperty.Name to create (r) => r.SomeProperty.Name == criteriaItem.
Expression<Func<string, bool>> prototype = item => item == criteriaItem;
var predicate = Expression.Lambda<Func<T, bool>>(
ReplacingExpressionVisitor.Replace(
prototype.Parameters[0],
getItemString.Body,
prototype.Body),
getItemString.Parameters[0]);
And I think you're trying to do it this way so that criteriaItem is bound to an sql parameter, rather than being inlined as a string constant. But your question was a little hard to follow.
Related
I would like my Web API to be able to sort its output by a string parameter such as this one:
http://myapi.com/api/people?skip=0&take=50&orderBy=lastName&descending=true.
Because I also have pagination support (with skipand take) in my API, I would like the orderBy and descending parameter to be applied to the SQL query directly, so that the correct result comes from the database.
When doing this however, the code can become very hard to manage when trying to match the parameters for orderBy with the actual properties of the classes I wish to sort by just using string comparisons.
I have found a solution which is supposed to work with LINQ to Entities and thus also with the new EF7, however when I try to compile this code using the new Core CLR, I get the following message:
Error CS1503 Argument 2: cannot convert from 'System.Linq.Expressions.Expression>' to 'string'
The code from the solution that fails is the OrderBy<T>method:
public static IOrderedQueryable<T> OrderBy<T>(this IQueryable<T> source, string propertyName)
{
return source.OrderBy(ToLambda<T>(propertyName));
}
It seems like the new Core CLR does not support this attempt. Is there another way to get the solution to work with the new CLR? If no, what other alternatives do I have to enable sorting using EF7 without resulting in countless if or switch statements to compare the input strings to the property names?
The solution from your link uses an "Expression.Convert" which most of the time doesn't work with LINQ to Entities.
Here is a working extension method:
public static IOrderedQueryable<TSource> OrderBy<TSource>(this IQueryable<TSource> source, string propertyName)
{
// LAMBDA: x => x.[PropertyName]
var parameter = Expression.Parameter(typeof(TSource), "x");
Expression property = Expression.Property(parameter, propertyName);
var lambda = Expression.Lambda(property, parameter);
// REFLECTION: source.OrderBy(x => x.Property)
var orderByMethod = typeof(Queryable).GetMethods().First(x => x.Name == "OrderBy" && x.GetParameters().Length == 2);
var orderByGeneric = orderByMethod.MakeGenericMethod(typeof(TSource), property.Type);
var result = orderByGeneric.Invoke(null, new object[] { source, lambda });
return (IOrderedQueryable<TSource>)result;
}
Disclaimer: I'm the owner of the project EF+ on GitHub.
You can find other methods to order by property name in my repository: GitHub
OrderByDescending
ThenBy
ThenByDescending
AddOrAppendOrderBy
AddOrAppendOrderByDescending
EDIT: Answer sub-question
Is it possibly to sort by navigation properties using something like
this, e.g. a property name "NavigationProperty.PropertyName"
Yes, you can either split the string and loop to create the expression with the property path or use a real expression evaluator.
Disclaimer: I'm the owner of the project Eval-Expressions.NET
This library allows you to execute all LINQ method dynamically.
See: LINQ Dynamic
var result = list.OrderByDynamic(x => "NavigationProperty.PropertyName");
Does anyone have/know of an IQueryable.OrderBy extension that takes an Expression (retrieved, for example, by Reflection)? I believe the function would look something like this:
public static IQueryable<TEntity> OrderBy<TEntity>
(this IQueryable<TEntity> source, Expression sortExpression)
Expression would be assumed to be an Expression<Func<TEntity, T>> where TEntity is the same object being sorted on, and T is a type that needs to be determined in order to create the new IQueryable.
I've found many examples of extensions that take a string, including Dynamic Linq, like this:
public static IQueryable<TEntity> OrderBy<TEntity>(
this IQueryable<TEntity> source, string sortExpression)
If it's possible to take the string and use Reflection to look up the type from the object in question, it should also be possible to take the Expression, and get the value type which is right there IN the Expression.
Following is a detailed explanation of why I'd like to have this, which you may or may not need.
I have a rather large list of complex records to sort. Because the list is so long, I prefer to have the sorting done on the database side. To handle more complex properties, I've created Expressions that provide the sorting functionality, like so:
if (model.sortExpression == "PlannedValue")
{
Expression<Func<BDopp, decimal>> sorter = BDopp.PlannedValueSorter;
if (model.sortDirection == "DESC")
opps = opps.OrderByDescending(sorter).AsQueryable();
else
opps = opps.OrderBy(sorter).AsQueryable();
}
BDOpp.PlannedValueSorter retrieves a static expression from the object which allows sorting to be done without opps are still of type IQueryable:
public static Expression<Func<BDopp, decimal>> PlannedValueSorter
{
get
{
return z => z.BudgetSchedules
.Where(s => s.Type == 1)
.Sum(s => s.Value * s.Workshare * z.valueFactor / 100 / 100);
}
}
Sorting for simple properties is done with Extension methods that use Reflection to build an expression based on the name of the property passed as a string.
This works well, but for the complex types, I still need branching logic, and I'd rather not do that. What I'd rather do is check for a static property containing the expression, and then simply apply it. I can get the expression like this:
PropertyInfo info = typeof(BDopp).GetProperty(model.sortExpression + "Sorter",
BindingFlags.Static | BindingFlags.Public);
Expression expr = (Expression)info.GetValue(null, null);
For the PlannedValue property, this gets me the expression sorted in PlannedValueSorter, which I already know works.
Update:
Various digging around has gotten me what I think might be some progress:
public static IQueryable<TEntity> OrderBy<TEntity>(this IQueryable<TEntity> source,
Expression<Func<TEntity, dynamic>> sortExpression)
{
var unary = sortExpression.Body as UnaryExpression;
Type actualExpressionType = unary.Operand.Type;
actualExpressionType is in fact the return type of the Expression (for this particular property, it's decimal).
Unfortunately I'm mostly just working by trial and error, since I haven't yet wrapped my brain around how all this works, so my attempt to update the query like so is not working:
MethodCallExpression resultExp = Expression.Call(typeof(Queryable),
"OrderBy",
new Type[] { typeof(TEntity), actualExpressionType },
source.Expression, sortExpression);
return source.Provider.CreateQuery<TEntity>(resultExp);
It compiles okay, but the following error is thrown at runtime:
No generic method 'OrderBy' on type 'System.Linq.Queryable' is
compatible with the supplied type arguments and arguments. No type
arguments should be provided if the method is non-generic.
As I understand it you have an Expression and want to order/filter by it. It might surprise you how simple it is:
Expression<Func<TEntity, T>> sortExpr = ...; //you already have this
var ordered = myQuery.OrderBy(sortExpr);
OrderBy always uses an Expression so you can directly use it. If you only have a variable of type Expression (which points to an object of type Expression<Func<TEntity, T>>) and don't know statically what T is, you can do this:
Expression sortExpr = ...; //you already have this
var ordered = myQuery.OrderBy((dynamic)sortExpr);
dynamic will figure this out at runtime using reflection. No need to do reflection yourself. All that is needed here is overload resolution (performed by the C# runtime binder).
Okay, I've got a solution:
public static IQueryable<TEntity> OrderBy<TEntity>(
this IQueryable<TEntity> source,
Expression<Func<TEntity, dynamic>> sortExpression,
bool descending)
{
var unary = sortExpression.Body as UnaryExpression;
var operand = unary.Operand;
Type actualExpressionType = operand.Type;
MethodCallExpression resultExp =
Expression.Call(typeof(Queryable),
descending? "OrderByDescending" : "OrderBy",
new Type[] { typeof(TEntity), actualExpressionType },
source.Expression,
Expression.Lambda(operand, sortExpression.Parameters));
return source.Provider.CreateQuery<TEntity>(resultExp);
}
The bool descending is to allow for the standard OrderBy and OrderByDescending overloads. Two major breakthroughs here, at least for me:
Getting the Operand out of the expression.
Using Expression.Call and Expression.Lambda to create the new expression - this allows me to use an actual "Type" variable, whereas the Expression<Func<TEntity, T>> syntax requires you to use a type that's known at compile time.
Try this:
public static IEnumerable<T> OrderBy<T>(
this IEnumerable<T> collection,
string columnName
//, SortDirection direction
)
{
ParameterExpression param = Expression.Parameter(typeof(T), "x"); // x
Expression property = Expression.Property(param, columnName); // x.ColumnName
Func<T, object> lambda = Expression.Lambda<Func<T, object>>( // x => x.ColumnName
Expression.Convert(property, typeof(object)),
param)
.Compile();
Func<IEnumerable<T>, Func<T, object>, IEnumerable<T>> expression = (c, f) => c.OrderBy(f); // here you can use OrderByDescending basing on SortDirection
IEnumerable<T> sorted = expression(collection, lambda);
return sorted;
}
Usage:
IEnumerable<T> collection = ...
IEnumerable<T> ordered = collection.OrderBy("PropName");
See Code Project and sandbox.
So I'm trying to optimize a query that does a text search on 1000s of rows and displays 100s of them. To do so, I'm trying to understand some code from Microsoft's 101 Sample Queries (found here).
Here is the code I wish to understand:
[Category("Advanced")]
[Title("Dynamic query - Where")]
[Description("This sample builds a query dynamically to filter for Customers in London.")]
public void LinqToSqlAdvanced02()
{
IQueryable<Customer> custs = db.Customers;
ParameterExpression param = Expression.Parameter(typeof(Customer), "c");
Expression right = Expression.Constant("London");
Expression left = Expression.Property(param, typeof(Customer).GetProperty("City"));
Expression filter = Expression.Equal(left, right);
Expression pred = Expression.Lambda(filter, param);
Expression expr = Expression.Call(typeof(Queryable), "Where", new Type[] { typeof(Customer) }, Expression.Constant(custs), pred);
IQueryable<Customer> query = db.Customers.AsQueryable().Provider.CreateQuery<Customer>(expr);
ObjectDumper.Write(query);
}
So...I realize this is esoteric, but why does Expression.Call() need the DbSet reference it's passed to CreateQuery as <T>?
I believe what you're asking is why the third parameter (array of types) is needed in Expression.Call because if you did it through code, you'd only have to do this:
custs.Where(pred);
The reason why the code works without the generic parameter is because of implicit typing, where the compiler translates that automatically to:
custs.Where<Customer>(pred);
After it translates it, the actual bytecode contains the call with the generics specified. When you build the Expression, you don't have all the niceties like implicit typing, so you have to specify exactly what gets called.
I am trying to do a like comparison based on an outside parameter (passed by a search form) that determines type of comparison ("%string" or "string%" or "%string%")
I was thinking in the following direction:
query = query.Where(
Entity.StringProperty.Like("SearchString", SelectedComparsionType)
)
Like method would than based on selected type return
.StartsWith() or .EndsWith() or .SubString()
My knowledge of expressions is apparently far from great, since i haven't been able to construct a method that could yield the right result (server side comparison in SQL just like with StartsWith method).
The easy way
Just use
if (comparison == ComparisonType.StartsWith)
query = query.Where(e => e.StringProperty.StartsWith("SearchString"));
else if ...
The hard way
If you want to do something like this, either make sure your LINQ provider can be told of this new method somehow, and how it translates to SQL (unlikely), or prevent your method from ever reaching the LINQ provider, and provide the provider something it understands (hard). For example, instead of
query.Where(e => CompMethod(e.StringProperty, "SearchString", comparsionType))
you can create something like
var query = source.WhereLike(e => e.StringProperty, "SearchString", comparsionType)
with the following code
public enum ComparisonType { StartsWith, EndsWith, Contains }
public static class QueryableExtensions
{
public static IQueryable<T> WhereLike<T>(
this IQueryable<T> source,
Expression<Func<T, string>> field,
string value,
SelectedComparisonType comparisonType)
{
ParameterExpression p = field.Parameters[0];
return source.Where(
Expression.Lambda<Func<T, bool>>(
Expression.Call(
field.Body,
comparisonType.ToString(),
null,
Expression.Constant(value)),
p));
}
}
You can even add additional criteria this way
var query = from e in source.WhereLike(
e => e.StringProperty, "SearchString", comparsionType)
where e.OtherProperty == 123
orderby e.StringProperty
select e;
The very, very hard way
It would (technically) be possible to rewrite the expression tree before the provider sees it, so you can use the query you had in mind in the first place, but you'd have to
create a Where(this IQueryable<EntityType> source, Expression<Func<EntityType, bool>> predicate) to intercept the Queryable.Where,
rewrite the expression tree, replacing your CompMethod, wherever it is, with one of the String methods,
call the original Queryable.Where with the rewritten expression,
and first of all, be able to follow the extension method above in the first place!
But that's probably way too complicated for what you had in mind.
Sounds like you should be wanting to use:
query = query.Where(
Entity.StringProperty.Contains("SearchString")
)
This should map to:
WHERE StringProperty LIKE '%SearchString%'
This should also work for more advanced search masks such as "Mr? Sm%th", but I haven't had to test any search strings like that myself yet.
UPDATE: Based on OPs edit
It sounds like what you are asking for is something like the following:
public enum SelectedComparsionType
{
StartsWith,
EndsWith,
Contains
}
public static bool Like(this string searchString, string searchPattern, SelectedComparsionType searchType)
{
switch (searchType)
{
case SelectedComparsionType.StartsWith:
return searchString.StartsWith(searchPattern);
case SelectedComparsionType.EndsWith:
return searchString.EndsWith(searchPattern);
case SelectedComparsionType.Contains:
default:
return searchString.Contains(searchPattern);
}
}
This would allow you to write code as you require, i.e:
query = query.Where(
Entity.StringProperty.Like("SearchString", SelectedComparsionType.StartsWith)
)
However, personally, I would replace any use of SelectedComparsionType, with a direct call to the required string function. I.e
query = query.Where(
Entity.StringProperty.StartsWith("SearchString")
)
As this will still map to a SQL 'LIKE' query.
This is exactly what I had in mind, thank you. I had something similar already written, but it didn't translate to SQL. For example, it worked if I did this directly:
Entity.StringProperty.EndsWith("SearchString");
It didn't work if I used a dedicated method:
CompMethod("BaseString","SearchString",SelectedComparsionType.EndsWith)
I think it probably has something to do with expression evaluation, i'm just not sure what.
You will be better off using Regex to solve this problem.
So, I am using the Linq entity framework. I have 2 entities: Content and Tag. They are in a many-to-many relationship with one another. Content can have many Tags and Tag can have many Contents. So I am trying to write a query to select all contents where any tags names are equal to blah
The entities both have a collection of the other entity as a property(but no IDs). This is where I am struggling. I do have a custom expression for Contains (so, whoever may help me, you can assume that I can do a "contains" for a collection). I got this expression from: http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2670710&SiteID=1
Edit 1
I ended up finding my own answer.
After reading about the PredicateBuilder, reading all of the wonderful posts that people sent to me, posting on other sites, and then reading more on Combining Predicates and Canonical Function Mapping.. oh and I picked up a bit from Calling functions in LINQ queries (some of these classes were taken from these pages).
I FINALLY have a solution!!! Though there is a piece that is a bit hacked...
Let's get the hacked piece over with :(
I had to use reflector and copy the ExpressionVisitor class that is marked as internal. I then had to make some minor changes to it, to get it to work. I had to create two exceptions (because it was newing internal exceptions. I also had to change the ReadOnlyCollection() method's return from:
return sequence.ToReadOnlyCollection<Expression>();
To:
return sequence.AsReadOnly();
I would post the class, but it is quite large and I don't want to clutter this post any more than it's already going to be. I hope that in the future that class can be removed from my library and that Microsoft will make it public. Moving on...
I added a ParameterRebinder class:
public class ParameterRebinder : ExpressionVisitor {
private readonly Dictionary<ParameterExpression, ParameterExpression> map;
public ParameterRebinder(Dictionary<ParameterExpression, ParameterExpression> map) {
this.map = map ?? new Dictionary<ParameterExpression, ParameterExpression>();
}
public static Expression ReplaceParameters(Dictionary<ParameterExpression, ParameterExpression> map, Expression exp) {
return new ParameterRebinder(map).Visit(exp);
}
internal override Expression VisitParameter(ParameterExpression p) {
ParameterExpression replacement;
if (map.TryGetValue(p, out replacement)) {
p = replacement;
}
return base.VisitParameter(p);
}
}
Then I added a ExpressionExtensions class:
public static class ExpressionExtensions {
public static Expression<T> Compose<T>(this Expression<T> first, Expression<T> second, Func<Expression, Expression, Expression> merge) {
// build parameter map (from parameters of second to parameters of first)
var map = first.Parameters.Select((f, i) => new { f, s = second.Parameters[i] }).ToDictionary(p => p.s, p => p.f);
// replace parameters in the second lambda expression with parameters from the first
var secondBody = ParameterRebinder.ReplaceParameters(map, second.Body);
// apply composition of lambda expression bodies to parameters from the first expression
return Expression.Lambda<T>(merge(first.Body, secondBody), first.Parameters);
}
public static Expression<Func<T, bool>> And<T>(this Expression<Func<T, bool>> first, Expression<Func<T, bool>> second) {
return first.Compose(second, Expression.And);
}
public static Expression<Func<T, bool>> Or<T>(this Expression<Func<T, bool>> first, Expression<Func<T, bool>> second) {
return first.Compose(second, Expression.Or);
}
}
And the last class I added was PredicateBuilder:
public static class PredicateBuilder {
public static Expression<Func<T, bool>> True<T>() { return f => true; }
public static Expression<Func<T, bool>> False<T>() { return f => false; }
}
This is my result... I was able to execute this code and get back the resulting "content" entities that have matching "tag" entities from the tags that I was searching for!
public static IList<Content> GetAllContentByTags(IList<Tag> tags) {
IQueryable<Content> contentQuery = ...
Expression<Func<Content, bool>> predicate = PredicateBuilder.False<Content>();
foreach (Tag individualTag in tags) {
Tag tagParameter = individualTag;
predicate = predicate.Or(p => p.Tags.Any(tag => tag.Name.Equals(tagParameter.Name)));
}
IQueryable<Content> resultExpressions = contentQuery.Where(predicate);
return resultExpressions.ToList();
}
Please let me know if anyone needs help with this same thing, if you would like me to send you files for this, or just need more info.
Summing it up...
contentQuery.Where(
content => content.Tags.Any(tag => tags.Any(t => t.Name == tag.Name))
);
So is that what you're expecting?
I'm a little confused.
This is what the question itself asks for:
contentQuery.Where(
content => content.Tags.Any(tag => tag.Name == "blah")
);
I'm not sure what the thought process was to get to the questioner's code, really, and I'm not entirely sure exactly what its really doing. The one thing I'm really sure of is that .AsQueryable() call is completely unnecessary -- either .Tags is already an IQueryable, or the .AsQueryable() is just going to fake it for you -- adding extra calls in where there doesn't need to be any.
The error is related to the 'tags' variable. LINQ to Entities does not support a parameter that is a collection of values. Simply calling tags.AsQueryable() -- as suggested in an ealier answer -- will not work either because the default in-memory LINQ query provider is not compatible with LINQ to Entities (or other relational providers).
As a workaround, you can manually build up the filter using the expression API (see this forum post) and apply it as follows:
var filter = BuildContainsExpression<Element, string>(e => e.Name, tags.Select(t => t.Name));
var query = source.Where(e => e.NestedValues.Any(filter));
tags.Select(testTag => testTag.Name)
Where does the tags variable gets initialized from? What is it?
NOTE: please edit the question itself, rather than replying with an answer -- this is not a discussion thread, and they can re-order themselves at any time
If you're searching for all Contents that are marked with any one of a set of tags:
IEnumerable<Tag> otherTags;
...
var query = from content in contentQuery
where content.Tags.Intersection(otherTags).Any()
select content;
It looks like you might be using LINQ To SQL, in which case it might be better if you write a stored procedure to do this one: using LINQ to do this will probably not run on SQL Server -- it's very likely it will try to pull down everything from contentQuery and fetch all the .Tags collections. I'd have to actually set up a server to check that, though.