I want to define a Func<ProductItemVendor, bool> filter expression named CompareProductItemVendorIds, which can be used throughout my application, mostly within Entity Framework/LINQ queries.
I've learned that in order to be able to use this filter in a LINQ query, I must declare it as Expression<Func<>> instead of just Func<>. I understand the reason for this, and it's easy for me to do this.
But I'm having the following problems using that expression in my queries.
First, code such as:
ProductItem.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds)
Note: ProductItem is a database entity, and its ProductItemVendors property is a navigation collection.
Produces the error:
`Instance argument: cannot convert from 'System.Collections.Generic.ICollection' to 'System.Linq.IQueryable'
And second, code such as:
var results = from v in Repository.Query<ProductItemVendor>()
where CompareProductItemVendorIds(v)
select v;
Produces the error:
'CompareProductItemVendorIds' is a 'variable' but is used like a 'method'
So I have my nice shiny new Expression<Func<>>. How can I use it in my LINQ queries?
ProductItem is already an Entity, so you can't use your Expression, you need to use Compile() to get the Func<> from your Expression<Func<>> since ProductItemVendors is no longer an IQueryable
ProductItem.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds.Compile())
You would have to use your Expression on the ProductItemVendorsContext like this:
var item = Context.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds);
You cant use an Expression inside query syntax, you need to use method sytanx
var results = from v in Repository.Query<ProductItemVendor>()
.Where(CompareProductItemVendorIds)
select v;
The first case;
ProductItemVendor productItemVendor = ProductItem.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds);
Produces the error:
`Instance argument: cannot convert from 'System.Collections.Generic.ICollection' to 'System.Linq.IQueryable'
Happens because it is an ICollection, the entity has already been loaded. Given dbContext or objectContext lazy loading vs explicit loading is implemented a bit differently, I assume you are using explicit loading though. If you change the loading to lazy the type of ProductItemVendors will be IQueryable and what you are trying will succeed.
Given the second case, the expression must be compilable to SQL, else you get a lot of weird errors, probably it's possible that that is the case here.
It's hard to give you more explicit help given the information in the question, I cannot recreate it easily. If you can create a MWE-solution and upload it somewhere I can have a look, but I'm afraid I can't help more here.
I don't think composing expressions like that is supported by default.
You could try LINQKit
Here's an example from the LINQKit page; one Expression<Func<>> inside another:
Call Invoke to call the inner expression Call Expand on the final
result. For example:
Expression<Func<Purchase,bool>> criteria1 = p => p.Price > 1000;
Expression<Func<Purchase,bool>> criteria2 = p => criteria1.Invoke (p)
|| p.Description.Contains ("a");
Console.WriteLine (criteria2.Expand().ToString());
(Invoke and Expand are extension methods in LINQKit.) Here's the output:
p => ((p.Price > 1000) || p.Description.Contains("a"))
Notice that we have a nice clean expression: the call to Invoke has
been stripped away.
Related
In one of my projects I have an ExpressionVisitor to translate provided expression into some query string. But before translating it I need to evaluate all refferences in the expression to real values. To do that I use Evaluator.PartialEval method from EntityFramework Project.
Assuming I have this query:
var page = 100;
var query = myService.AsQueryable<Product>()
//.Where(x=>x.ProductId.StartsWith(p.ProductId))
.Skip(page)
.Take(page);
var evaluatedQueryExpr = Evaluator.PartialEval(query.Expression);
As you can see I have commented Where method. In this case evaluatedQueryExpr will not contain the methods Take and Skip.
However, if I use any other method with Expression before Take or Skip everything works, Evaluator evaluates an expression correctly and return it fully.
I found out that the problem occurs in the line 80 of the Evaluator class:
return Expression.Constant(fn.DynamicInvoke(null), e.Type);
Could you explain why this happens and suggest a workaround?
Update
here is a project on github
LinqToSolrQueriable inherited from IOrderedQueryable
LinqToSolrProvider inherited from IQueryProvider including line range causing the issue
The good news are that the expression is not really reduced (Skip and Take are still there :), but is simply converted from MethodCallExpression to ConstantExpression containing the original expression:
query.Expression:
.Call System.Linq.Queryable.Take(
.Call System.Linq.Queryable.Skip(
.Constant<LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product]>(LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product]),
100),
100)
evaluatedQueryExpr:
.Constant<System.Linq.IQueryable`1[LinqToSolrTest.Product]>(LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product])
Here the debug display is giving you a wrong impression. If you take the ConstaintExpression.Value, you'll see that it's a IQueryable<Product> with Expression property being exactly the same as the original query.Expression.
The bad news are that this is not what you expect from PartialEval - in fact it doesn't do anything useful in this case (except potentially breaking your query translation logic).
So why is this happening?
The method you are using from EntityFramework.Extended library is in turn taken (as indicated in the comments) from MSDN Sample Walkthrough: Creating an IQueryable LINQ Provider. It can be noticed that the PartialEval method has two overloads - one with Func<Expression, bool> fnCanBeEvaluated parameter used to identify whether a given expression node can be part of the local function (in other words, to be partially evaluated or not), and one without such parameter (used by you) which simply calls the first passing the following predicate:
private static bool CanBeEvaluatedLocally(Expression expression)
{
return expression.NodeType != ExpressionType.Parameter;
}
The effect is that it stops evaluation of ParameterExpression type expressions and any expressions containing directly or indirectly ParameterExpression. The last should explain the behavior you are observing. When the query contains Where (and basically any LINQ operator) with parametrized lambda expression (hence parameter) before the Skip / Take calls, it would stop evaluation of the containing methods (which you can see from the above query.Expression debug view - the Where call will be inside the Skip).
Now, this overload is used by the MSDN example to evaluate a concrete nested Where method lambda expression and is not generally applicable for any type of expression like IQueryable.Expression. In fact the linked project is using the PartialEval method in a single place inside QueryCache class, and also calling the other overload passing a different predicate which in addition to ParameterExpressions stops the evaluation of any expression with result type of IQueryable.
Which I think is the solution of your problem as well:
var evaluatedQueryExpr = Evaluator.PartialEval(query.Expression,
// can't evaluate parameters or queries
e => e.NodeType != ExpressionType.Parameter &&
!typeof(IQueryable).IsAssignableFrom(e.Type)
);
what is the problem with the following code?
var results = db.Departments.Select(x => customfunction(x.DepartmentID));
results.toList();
Why this is this not caught in compile time but gives an exception in runtime?
There are many different LINQ providers that allow you to use LINQ against a variety of data sources (LINQ to Entities, LINQ to Objects, LINQ to XML among others).
Although LINQ to Entities does not know how to invoke your custom method (at least, not the providers for common databases), some LINQ providers might well understand how to execute myfunction. The compiler is not integrated with all of the many LINQ providers, so the information as to whether your custom method can be included is only available at runtime.
In fact, LINQ to Objects can execute it
var results = db.Departments
.AsEnumerable()
.Select(x => myfunction(x.DepartmentID));
The conversion of
db.Departments.Select(x => myfunction(x.DepartmentID));
to sql does not happen until very late in the process at runtime.
So at compilation time Entity Framework does not know that myFunction is not something it is not going to know how to convert to sql yet.
of course you could do
db.Departments.Select(x => (x * 2));
And EF will happily be able to translate to sql but if you try to use a custom property on Departments object that doesn't map to a field in the db or use a custom method that that the Linq data provider does not know how to translate to sql you will get the same error.
The idea is simple: .Select(x => myfunction(x.DepartmentID)) is never executed in C#. Instead, Entity Framework parses this expression and translates it to SQL.
For example, if you call .Count() in C# EF will translate this to SELECT COUNT(...)
This is all very powerful, except for a few cases such as yours.
EF analyzes your code through Expression Trees.
If you call myFunction() in an expression tree, it is impossible to "expand" that call to get the body of your function. This is a limitation of how C# is implemented. That's why EF is unable to generate sensible SQL, because it has no clue what your method does.
However, there is a reasonable alternative: LinqKit
LinqKit will allow you to write "functions" that are reusable in your expressions.
Example:
static string[] QueryCustomers (Expression<Func<Purchase, bool>> purchaseCriteria)
{
var data = new MyDataContext();
var query = from c in data.Customers.AsExpandable()
where c.Purchases.Any (purchaseCriteria.Compile())
select c.Name;
return query.ToArray();
}
OK, this is doing my head in - I'm not even sure how to search for this.
Here is the first part of my function:
var rules = context.Rules.Include(r => r.CreatedBy).Include(r => r.ModifiedBy);
IUserManager um = GetUserManager();
var currentUser = um.GetCurrent();
Can someone tell me why this works:
return rules.Where(delegate(Rule r)
{
return r.CreatedBy.CompanyID == currentUser.CompanyID;
});
but this doesn't:
return rules.Where(r => r.CreatedBy.CompanyID == currentUser.CompanyID);
It's EF Code first and CreatedBy and ModifiedBy are both virtual properties.
r.CreatedBy and currentUser are both instances of the same class (if you didn't already work this out)
What the second snippet is returning is an empty list. It's as though the eager loading isn't working and the lambda doesn't cause a lazy load.
Note: I've just discovered that if I change the first line to
var rules = context.Rules.Include(r => r.CreatedBy)
.Include(r => r.ModifiedBy).ToList()
then the lambda works. The question still stands though. Why to I have to use the ToList() or the delegate. I'm doing the same thing elsewhere in the same class and it works as I'd expect.
Thanks
This is because lambdas can be implicitly converted to delegates or to expression trees. In the first case, the delegate is converted to an expression tree because rules is IQueryable<> and overload resolution chooses Queryable.Where. When you use the anonymous function, however, that can't be converted to an expression tree, so overload resolution has to choose Enumerable.Where.
When you make rules into a List<>, that forces overload resolution to choose Enumerable.Where, because List<> does not implement IQueryable<>. You could use AsEnumerable() to achieve the same effect without the overhead of creating the list.
As to why this doesn't work when you're "doing the same thing elsewhere in the same class and it works as I'd expect," we might be able to help if you give an example of code that does work.
In reference to the answer proposed in this post, can anyone explain why LINQPad won't generate any SQL for this?
var query = Products.Select(p => new
{
Orders = p.Orders.OrderByWithDirection(x => x.PurchaseDate, true)
});
The exception is as follows:
NotSupportedException: Method 'System.Linq.IOrderedEnumerable1[LINQPad.User.Order] OrderByWithDirection[Phase,String](System.Collections.Generic.IEnumerable1[LINQPad.User.Order], System.Func`2[LINQPad.User.Order,System.String], Boolean)' has no supported translation to SQL.
The problem is not that it's part of an anonymous type - the problem is that you don't actually call OrderByWithDirection directly... instead, you build up an expression tree within the projection and that expression tree contains a reference to OrderByWithDirection, which LINQ to SQL (or EF, or whatever) doesn't know about.
I can't think of any nice fix for this other than an expression tree visitor which goes through the expression tree and replaces the call with either OrderBy or OrderByDescending based on the second argument - which would have to be a constant, of course. Even that's going to be far from trivial.
To be honest, I would probably take a low-tech approach - keep as much of the query the same as you can, and then conditionally call OrderBy or OrderByDescending at the end:
var mainQuery = ...; // Stuff which is in common
var query = direction
? mainQuery.Select(p => new { Orders = p.Orders.OrderBy(x => x.PurchaseDate) })
: mainQuery.Select(p => new { Orders = p.Orders.OrderByDescending(x => x.PurchaseDate) });
As an aside, I assume your real query has more properties in the anonymous type? There's rarely any good reason to create an anonymous type with a single property.
I have the following code to get a collection of Types from an Assembly where the Type is a Class:
Assembly assembly = Assembly.LoadFile(DLLFile);
var types = assembly.GetTypes().AsEnumerable().Where(x => x.IsClass);
This works fine and as expected. However I wanted to pull out the lambda expression to a Linq Expression variable (as later on it will be used in a parameter of this method). So I did the following:
private Expression<Func<Type, bool>> _standardFilter = (x => x.IsClass);
Assembly assembly = Assembly.LoadFile(DLLFile);
var types = assembly.GetTypes().AsEnumerable().Where(_standardFilter);
However this won't compile with the error:
System.Collections.Generic.IEnumerable<System.Type>' does not contain a
definition for 'Where' and the best extension method overload
'System.Linq.Enumerable.Where<TSource>(System.Collections.Generic.IEnumerable<TSource>, System.Func<TSource,int,bool>)' has some invalid arguments
I understand that my expression doesn't conform to the predicate System.Func<TSource,int,bool>, however the Where function has an overload that takes a predicate of System.Func<TSource,bool>, which as far as I can tell should work.
I have tried converting the result of assembly.GetTypes() (which is an array) to a List in several ways without it helping the issue.
I have also made sure that I have got all the correct using statements for this class as that seems to be an issue several people run into during my Googling.
In the past I have managed to use this same technique on a IQueryable collection, but I don't understand why this won't work when the Where function is available on the IEnumerable collection and should accept the predicate I am providing.
Thanks very much for any assistance.
Compile your expression into executable code (delegate):
var types = assembly.GetTypes().AsEnumerable().Where(_standardFilter.Compile());
As #Kirk stated, it's better not to use expression tree, if you are not going to analyze it. Simply use filter of type Func<Type, bool>.
Note that the Enumerable extension method takes a Func, not an Expression. So your argument is incorrectly typed.
This is in contrast with Entity Framework, which takes Expression<Func> rather than plain Func.
So be attentive to method signatures, they are similar, can both be called with the same lambda, but are actually different!
The reason behind that is that EF inspects the Expression and converts it to SQL code, whereas Linq to Object simply executes the predicate in memory.
You could try this instead:
private Func<Type, bool> standardFilter;
standardFilter = (type) => {
return type.IsClass;
};
var types = assembly.GetTypes().AsEnumerable().Any(x => standardFilter(x));
Then you're free to change your implementation of standardFilter. As long as it takes in a Type and returns bool for whether or not that should be included it will remain modular and accomplish what I think you're going for.