i am using asp.net 4.5 mvc 5 with code first entity framework and have the following issue:
i have two models, "PostBody" and "PostHeader". 1 PostHeader has 1-n PostBodies.
A PostBody can be "Deleted" (Property as flag).
My extension method should give me every PostBody from an IQueryable-Object which is not deleted as an IQueryable:
public static IQueryable<TSource> GetActiveVersions<TSource>(this IQueryable<TSource> source)
where TSource : PostBody
{
return source.Where(x => x.Deleted == false);
}
this is working fine when i do somethin like this
var x = db.Bodies.GetActiveVersions().ToList();
or this
var y = db.Headers.FirstOrDefault().Bodies.AsQueryable().GetActiveVersions().ToList();
etc. - But as soon as i use my extension method as part of an expression paramater within e.g. the where method, i'm running into a NotSupportedException:
var z = db.Headers.Where(h => h.Bodies.AsQueryable().GetActiveVersions().Count() > 0).ToList();
System.NotSupportedException: LINQ to Entities does not recognize the method 'System.Linq.IQueryable`1[WebApplication5.Models.PostBody] GetActiveVersions[PostBody](System.Linq.IQueryable`1[WebApplication5.Models.PostBody])' method, and this method cannot be translated into a store expression.
what am i doing wrong? or - how can i use my extension method within the where condition?
Maybe the parser cannot resolve the extension method in the linq expression. You could try to centralize your condition and use it as a expression tree, for sample:
public static Expression<Func<TSource, bool>> GetActiveVersions()
where TSource : PostBody
{
return x => x.Deleted == false;
}
And apply this expressin in your subquery (with linq) for sample:
var z = db.Headers.Where(h => h.Bodies.AsQueryable().Any(GetActiveVersions())).ToList();
Instead using the Count() > 0, prefer using .Any() to avoid to access all records of the table.
Related
I need to filter against a DBset using a list of objects passed in as a variable to my repo.
The following worked in EF6
List<MyFilterObj> incomingList = [{A: 1, B: 2}, {A: 3, B: 4}]
return await context.MyDBset
.Where(x => incomingList.Any(v =>
v.A == x.A &&
v.B == x.B)).ToListAsync();
If I try this same approach in EF Core 5 I get the following error:
The LINQ expression (MyDBset) could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable', 'AsAsyncEnumerable', 'ToList', or 'ToListAsync'. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
I can't seperate the incoming list into seperate A and B lists as the combination is essential to the filter accuracy. Otherwise I could have used this approach
return await context.MyDBset
.Where(x => incomingListA.Contains(x.A) &&
incomingListB.Contains(x.B)).ToListAsync();
This approach does work in EF Core 5
Basically I want EF to translate my query into something like:
SELECT * FROM MyTable AS m
Where (m.A = 1 AND m.B = 2) OR (m.A = 3 AND m.B = 4)
If I loop through my incomingList adding "where" filters to a query the resultant SQL uses "AND" instead of "OR" like so:
SELECT * FROM MyTable AS m
Where (m.A = 1 AND m.B = 2) AND (m.A = 3 AND m.B = 4)
Right now my work around involves binging the result set into memory with some basic filters then running an in memory filter using the original syntax. This is obviously not ideal.
Any suggestions? Thanks
Since EF Core is not willing to translate such expression (it's a well known limitation that Contains with primitive value is the only supported operation on in-memory collection in L2E query since it directly translates to SQL IN operator), you have to do it yourself. Any is basically equivalent of Or as you mentioned, so for not so big list and top level query it could easily be done by using some predicate builder implementation to dynamically build || predicate. Or directly using the the Expression class methods as in this custom extension method from my answer to How to simplify repetitive OR condition in Where(e => e.prop1.contains() || e.prop2.contains() || ...):
public static partial class QueryableExtensions
{
public static IQueryable<T> WhereAnyMatch<T, V>(this IQueryable<T> source, IEnumerable<V> values, Expression<Func<T, V, bool>> match)
{
var parameter = match.Parameters[0];
var body = values
// the easiest way to let EF Core use parameter in the SQL query rather than literal value
.Select(value => ((Expression<Func<V>>)(() => value)).Body)
.Select(value => Expression.Invoke(match, parameter, value))
.Aggregate<Expression>(Expression.OrElse);
var predicate = Expression.Lambda<Func<T, bool>>(body, parameter);
return source.Where(predicate);
}
}
with sample usage for your scenario:
context.MyDBset
.WhereAnyMatch(incomingList, (x, v) => v.A == x.A && v.B == x.B)
Normally instead of Expression.Invoke helper utilities like this would use custom ExpressionVisitor to replace the parameters of the source expression (thus simulation a "call"), which produces more naturally looking query expression, but for EF Core it doesn't really matter since it recognizes and correctly translates invocation expressions inside the expression tree.
But just in case Expression.Invoke doesn't work, here is the parameter replacing version - it should work with all query providers. First, the helpers
public static partial class ExpressionBuilder
{
public static Expression ReplaceParameter(this Expression source, ParameterExpression parameter, Expression value)
=> new ParameterReplacer { Parameter = parameter, Value = value }.Visit(source);
class ParameterReplacer : ExpressionVisitor
{
public ParameterExpression Parameter;
public Expression Value;
protected override Expression VisitParameter(ParameterExpression node)
=> node == Parameter ? Value : node;
}
}
and then simply replace the invocation
.Select(value => Expression.Invoke(match, parameter, value))
with
.Select(value => match.Body.ReplaceParameter(match.Parameters[1], value))
because specifically here we are reusing the first parameter (var parameter = match.Parameters[0];), otherwise it should be replaced as well.
I am trying to refactor the following line of an Entity Framework query into a generalized static extension method:
dbContext.Employees
.Where(e => permissionResolver.AuthorizedUsers.Select(p => p.Id).Contains(e.Id))
.OrderBy(...)
PermissionResolver is just an instance I receive a list of IDs from to compare against a user ID stored in the current record. It compiles perfectly to a SQL statement WHERE Id IN (....).
Now I am trying to create an extension method for IQueryable<T> that I can use for any type of record, I just want to pass in a property where the owner's ID is stored in.
So that is what I came up with:
public static IQueryable<T> AuthorizedRecords<T>(this IQueryable<T> query, Expression<Func<T, Int32>> property, IPermissionResolver permissionResolver)
{
Expression<Func<T, Boolean>> idIsAuthorized = entity => permissionResolver.AuthorizedUsers.Select(e => e.Id).ToList().Contains(property.Compile()(entity));
return query.Where(idIsAuthorized);
}
I'm getting a runtime error that this expression cannot be translated into SQL.
How can I combine the property expression to the main query expression that it can be translated into SQL correctly? Is there a better way to rewrite the query expression?
property.Compile() converts the expression tree into a delegate, this delegate cannot be properly translated back to an expression tree/SQL.
You need to construct expression tree like that:
var ids = permissionResolver.AuthorizedUsers.Select(e => e.Id).ToList().AsEnumerable();
// method Enumerable.Contains<int>()
var methodContains = typeof(System.Linq.Enumerable).GetMethods()
.Where(m => m.Name == "Contains" && m.GetParameters().Length == 2)
.First()
.MakeGenericMethod(typeof(int));
var lambdaParam = property.Parameters.Single();
var lambda = Expression.Lambda(
Expression.Call(
methodContains,
Expression.Constant(ids),
property.Body),
lambdaParam
);
var predicate = (Expression<Func<T, bool>>)lambda;
return query.Where(predicate);
The short answer is you cannot use custom extension methods in entity framework queries. Under the hood entity framework parses expression tree Expression<Func<>> into sql query. It seems kind of impossible to translate any possible extension method to sql so they support limited set of Linq methods to properly translate it. Some useful information about expression trees: https://learn.microsoft.com/en-us/dotnet/csharp/programming-guide/concepts/expression-trees.
Some refactoring you can do to simplify your query is to use Any method like this:
dbContext.Employees
.Where(e => permissionResolver.AuthorizedUsers.Any(u => u.Id == e.Id))
.OrderBy(...
With the recent release of Entity Framework Core 3.0, LINQ queries are no longer evaluated on the client by default. I'm a big fan of this change, as it revealed some potentially dangerous client-side evaluation in my project that I thought was translated to SQL; however, it also made some of the helper methods that I was using to avoid crazy chains of ternaries unusable.
Has anyone manged to nest LINQ expressions for use with Entity Framework Core 3.0? Here's an example of what I'm hoping to achieve:
[Fact]
public async Task Can_use_custom_expression()
{
var dbContext = new ApplicationDbContext(new DbContextOptionsBuilder<ApplicationDbContext>().UseInMemoryDatabase("Test").Options);
dbContext.Users.Add(new ApplicationUser { FirstName = "Foo", LastName = "Bar" });
dbContext.SaveChanges();
string query = "Foo";
Expression<Func<string, string, bool>> valueCheck = (value, expected) => !string.IsNullOrEmpty(value) && value.Contains(expected);
var valueCheckFunc = valueCheck.Compile();
Expression<Func<ApplicationUser, bool>> whereExpression = (u) => valueCheckFunc(u.FirstName, query);
var user = await dbContext.Users
.Where(whereExpression)
.FirstOrDefaultAsync();
Assert.NotNull(user);
}
When I run this example, I get the following exception:
Message:
System.InvalidOperationException : The LINQ expression 'Where<ApplicationUser>(
source: DbSet<ApplicationUser>,
predicate: (a) => Invoke(__valueCheckFunc_0, a.FirstName, __query_1)
)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to either AsEnumerable(), AsAsyncEnumerable(), ToList(), or ToListAsync(). See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.
Like I said, I do not want to evaluate this expression client-side, but I would like to avoid having to chain a dozen or so !string.IsNullOrEmpty(x) && x.Contains(y)'s in a single expression. I'd love some tips on how to achieve this.
If you want your expressions to be translatable to Sql by EF you need to avoid calling delegates or methods (with some exceptions, of course). But what you want to achieve is doable by replacing the delegate invocation with its defining expression. For that you need a specialized ExpressionVisitor.
The following visitor will traverse expressions replacing delegate references within its wrapping invocations by a lambda expression body:
public class DelegateByLambda: ExpressionVisitor
{
LambdaExpression delegateReferenceExpression;
LambdaExpression lambdaExpression;
Stack<InvocationExpression> invocations;
public DelegateByLambda(LambdaExpression delegateReferenceExpression, LambdaExpression lambdaExpression)
{
this.delegateReferenceExpression = delegateReferenceExpression;
this.lambdaExpression = lambdaExpression;
this.invocations = new Stack<InvocationExpression>();
}
protected override Expression VisitParameter(ParameterExpression node)
{
var paramIndex = lambdaExpression.Parameters.IndexOf(node);
if (paramIndex >= 0)
{
InvocationExpression call = invocations.Peek();
return base.Visit(call.Arguments[paramIndex]);
}
return base.VisitParameter(node);
}
protected override Expression VisitInvocation(InvocationExpression node)
{
if (node.Expression.ToString() == delegateReferenceExpression.Body.ToString())
{
invocations.Push(node);
var result = base.Visit(lambdaExpression.Body);
invocations.Pop();
return result;
}
return base.VisitInvocation(node);
}
}
This class has no protection against attempting to replace delegate invocations by lambdas with mismatching arguments (number and types) however, the following extension method will do the trick:
public static class DelegateByLambdaExtensions
{
public static Expression<T> Replace<T, X>(this Expression<T> source, Expression<Func<X>> delegateReference, Expression<X> lambdaReference)
{
return new DelegateByLambda(delegateReference, lambdaReference).Visit(source) as Expression<T>;
}
}
So, all you need to do in your code is to call the replace extension method on the expression you want to translate passing an expression returning the delegate and desired lambda expression for expansion. Your sample should look like this:
Expression<Func<string, string, bool>> valueCheck = (value, expected) => !string.IsNullOrEmpty(value) && value.Contains(expected);
var valueCheckFunc = valueCheck.Compile();
Expression<Func<ApplicationUser, bool>> whereExpression = (u) => valueCheckFunc(u.FirstName, query);
whereExpression = whereExpression.Replace(() => valueCheckFunc, valueCheck);
var user = dbContext.Users
.Where(whereExpression)
.FirstOrDefault();
Console.WriteLine(user != null ? $"Found {user.FirstName} {user.LastName}!" : "User not found!");
A working sample can be found here. https://dotnetfiddle.net/Lun3LA
This question already has answers here:
Chaining / building LINQ query with an OR instead of AND
(2 answers)
Closed 6 years ago.
Appologies if this is a simple question; suppose I have an EF Query object with 2 methods:
public static IQueryable<Animal> FourLegged(this IQueryable<Animal> query)
{
return query.Where(r => r.NumberOfLegs == 4);
}
public static IQueryable<Animal> WithoutTail(this IQueryable<Animal> query)
{
return query.Where(r => !r.HasTail);
}
Now, in my service layer, to get animals that are four-legged AND without a tail, I can do:
_animalService.Query()
.FourLegged()
.WithoutTail();
That will result in an sql query like so:
select * from Animal where NumberOfLegs = 4 AND HasTail = 0
How do I use the 2 query methods with an OR instead? I want animals that are either 4 legged OR without a tail
select * from Animal where NumberOfLegs = 4 OR HasTail = 0
In Nhibernate I would have used a simple disjunction, but I can't find that in EF.
Thanks
Solution: I ended up using LinqKit predicates mentioned on this answer. It works quite well and I can reuse predicates too.
You can’t really do this when you already called query.Where(). The predicates there are already collected in the IQueryable and they are all combined by AND.
In order to get an OR you will have to make a single query.Where() call and pass a single expression that covers your various disjunctive predicates.
In your case, the combined predicate would look like this:
query.Where(r => (r.NumberOfLegs == 4) || (!r.HasTail))
To make that more dynamic, you essentially need to build a custom expression composition function that works like this:
Expression<Func<Animal, bool>> fourLegged = r => r.NumberOfLegs == 4;
Expression<Func<Animal, bool>> withoutTail = r => !r.HasTail;
query = query.Where(CombineDisjunctivePredicates(fourLegged, withoutTail));
So let’s write that CombineDisjunctivePredicates function:
public Expression<Func<T, bool>> CombineDisjunctivePredicates<T>(params Expression<Func<T, bool>>[] predicates)
{
Expression current = Expression.Constant(false);
ParameterExpression param = Expression.Parameter(typeof(T), "obj");
foreach (var predicate in predicates)
{
var visitor = new ReplaceExpressionVisitor(predicate.Parameters[0], param);
current = Expression.Or(current, visitor.Visit(predicate.Body));
}
return Expression.Lambda<Func<T, bool>>(current, param);
}
This basically takes a number of predicates and combines them by combining the expression bodies using the boolean OR. Since we are combining different expressions which may have different expression parameters, we also need to make sure to replace all expression parameter references in the expression bodies using a common parameter. We do this using a simple ReplaceExpressionVisitor, easily implemented like this:
public class ReplaceExpressionVisitor : ExpressionVisitor
{
private readonly Expression _original;
private readonly Expression _replacement;
public ReplaceExpressionVisitor(Expression original, Expression replacement)
{
_original = original;
_replacement = replacement;
}
public override Expression Visit(Expression node)
{
return node == _original ? _replacement : base.Visit(node);
}
}
And that’s all you need to combine the predicates. You just need to make sure to change your methods now so they don’t call query.Where themselves but return a Expression<Func<Animal, bool>> instead.
I would like my Web API to be able to sort its output by a string parameter such as this one:
http://myapi.com/api/people?skip=0&take=50&orderBy=lastName&descending=true.
Because I also have pagination support (with skipand take) in my API, I would like the orderBy and descending parameter to be applied to the SQL query directly, so that the correct result comes from the database.
When doing this however, the code can become very hard to manage when trying to match the parameters for orderBy with the actual properties of the classes I wish to sort by just using string comparisons.
I have found a solution which is supposed to work with LINQ to Entities and thus also with the new EF7, however when I try to compile this code using the new Core CLR, I get the following message:
Error CS1503 Argument 2: cannot convert from 'System.Linq.Expressions.Expression>' to 'string'
The code from the solution that fails is the OrderBy<T>method:
public static IOrderedQueryable<T> OrderBy<T>(this IQueryable<T> source, string propertyName)
{
return source.OrderBy(ToLambda<T>(propertyName));
}
It seems like the new Core CLR does not support this attempt. Is there another way to get the solution to work with the new CLR? If no, what other alternatives do I have to enable sorting using EF7 without resulting in countless if or switch statements to compare the input strings to the property names?
The solution from your link uses an "Expression.Convert" which most of the time doesn't work with LINQ to Entities.
Here is a working extension method:
public static IOrderedQueryable<TSource> OrderBy<TSource>(this IQueryable<TSource> source, string propertyName)
{
// LAMBDA: x => x.[PropertyName]
var parameter = Expression.Parameter(typeof(TSource), "x");
Expression property = Expression.Property(parameter, propertyName);
var lambda = Expression.Lambda(property, parameter);
// REFLECTION: source.OrderBy(x => x.Property)
var orderByMethod = typeof(Queryable).GetMethods().First(x => x.Name == "OrderBy" && x.GetParameters().Length == 2);
var orderByGeneric = orderByMethod.MakeGenericMethod(typeof(TSource), property.Type);
var result = orderByGeneric.Invoke(null, new object[] { source, lambda });
return (IOrderedQueryable<TSource>)result;
}
Disclaimer: I'm the owner of the project EF+ on GitHub.
You can find other methods to order by property name in my repository: GitHub
OrderByDescending
ThenBy
ThenByDescending
AddOrAppendOrderBy
AddOrAppendOrderByDescending
EDIT: Answer sub-question
Is it possibly to sort by navigation properties using something like
this, e.g. a property name "NavigationProperty.PropertyName"
Yes, you can either split the string and loop to create the expression with the property path or use a real expression evaluator.
Disclaimer: I'm the owner of the project Eval-Expressions.NET
This library allows you to execute all LINQ method dynamically.
See: LINQ Dynamic
var result = list.OrderByDynamic(x => "NavigationProperty.PropertyName");