We Keep Coding

Evaluator.PartialEval reduce provided expression

In one of my projects I have an ExpressionVisitor to translate provided expression into some query string. But before translating it I need to evaluate all refferences in the expression to real values. To do that I use Evaluator.PartialEval method from EntityFramework Project.
Assuming I have this query:
var page = 100;
var query = myService.AsQueryable<Product>()
//.Where(x=>x.ProductId.StartsWith(p.ProductId))
.Skip(page)
.Take(page);
var evaluatedQueryExpr = Evaluator.PartialEval(query.Expression);
As you can see I have commented Where method. In this case evaluatedQueryExpr will not contain the methods Take and Skip.
However, if I use any other method with Expression before Take or Skip everything works, Evaluator evaluates an expression correctly and return it fully.
I found out that the problem occurs in the line 80 of the Evaluator class:
return Expression.Constant(fn.DynamicInvoke(null), e.Type);
Could you explain why this happens and suggest a workaround?
Update
here is a project on github
LinqToSolrQueriable inherited from IOrderedQueryable
LinqToSolrProvider inherited from IQueryProvider including line range causing the issue

The good news are that the expression is not really reduced (Skip and Take are still there :), but is simply converted from MethodCallExpression to ConstantExpression containing the original expression:
query.Expression:
.Call System.Linq.Queryable.Take(
.Call System.Linq.Queryable.Skip(
.Constant<LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product]>(LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product]),
100),
100)
evaluatedQueryExpr:
.Constant<System.Linq.IQueryable`1[LinqToSolrTest.Product]>(LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product])
Here the debug display is giving you a wrong impression. If you take the ConstaintExpression.Value, you'll see that it's a IQueryable<Product> with Expression property being exactly the same as the original query.Expression.
The bad news are that this is not what you expect from PartialEval - in fact it doesn't do anything useful in this case (except potentially breaking your query translation logic).
So why is this happening?
The method you are using from EntityFramework.Extended library is in turn taken (as indicated in the comments) from MSDN Sample Walkthrough: Creating an IQueryable LINQ Provider. It can be noticed that the PartialEval method has two overloads - one with Func<Expression, bool> fnCanBeEvaluated parameter used to identify whether a given expression node can be part of the local function (in other words, to be partially evaluated or not), and one without such parameter (used by you) which simply calls the first passing the following predicate:
private static bool CanBeEvaluatedLocally(Expression expression)
{
return expression.NodeType != ExpressionType.Parameter;
}
The effect is that it stops evaluation of ParameterExpression type expressions and any expressions containing directly or indirectly ParameterExpression. The last should explain the behavior you are observing. When the query contains Where (and basically any LINQ operator) with parametrized lambda expression (hence parameter) before the Skip / Take calls, it would stop evaluation of the containing methods (which you can see from the above query.Expression debug view - the Where call will be inside the Skip).
Now, this overload is used by the MSDN example to evaluate a concrete nested Where method lambda expression and is not generally applicable for any type of expression like IQueryable.Expression. In fact the linked project is using the PartialEval method in a single place inside QueryCache class, and also calling the other overload passing a different predicate which in addition to ParameterExpressions stops the evaluation of any expression with result type of IQueryable.
Which I think is the solution of your problem as well:
var evaluatedQueryExpr = Evaluator.PartialEval(query.Expression,
// can't evaluate parameters or queries
e => e.NodeType != ExpressionType.Parameter &&
!typeof(IQueryable).IsAssignableFrom(e.Type)
);

Using Expression<Func<>> in a LINQ Query

I want to define a Func<ProductItemVendor, bool> filter expression named CompareProductItemVendorIds, which can be used throughout my application, mostly within Entity Framework/LINQ queries.
I've learned that in order to be able to use this filter in a LINQ query, I must declare it as Expression<Func<>> instead of just Func<>. I understand the reason for this, and it's easy for me to do this.
But I'm having the following problems using that expression in my queries.
First, code such as:
ProductItem.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds)
Note: ProductItem is a database entity, and its ProductItemVendors property is a navigation collection.
Produces the error:
`Instance argument: cannot convert from 'System.Collections.Generic.ICollection' to 'System.Linq.IQueryable'
And second, code such as:
var results = from v in Repository.Query<ProductItemVendor>()
where CompareProductItemVendorIds(v)
select v;
Produces the error:
'CompareProductItemVendorIds' is a 'variable' but is used like a 'method'
So I have my nice shiny new Expression<Func<>>. How can I use it in my LINQ queries?

ProductItem is already an Entity, so you can't use your Expression, you need to use Compile() to get the Func<> from your Expression<Func<>> since ProductItemVendors is no longer an IQueryable
ProductItem.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds.Compile())
You would have to use your Expression on the ProductItemVendorsContext like this:
var item = Context.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds);
You cant use an Expression inside query syntax, you need to use method sytanx
var results = from v in Repository.Query<ProductItemVendor>()
.Where(CompareProductItemVendorIds)
select v;

The first case;
ProductItemVendor productItemVendor = ProductItem.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds);
Produces the error:
`Instance argument: cannot convert from 'System.Collections.Generic.ICollection' to 'System.Linq.IQueryable'
Happens because it is an ICollection, the entity has already been loaded. Given dbContext or objectContext lazy loading vs explicit loading is implemented a bit differently, I assume you are using explicit loading though. If you change the loading to lazy the type of ProductItemVendors will be IQueryable and what you are trying will succeed.
Given the second case, the expression must be compilable to SQL, else you get a lot of weird errors, probably it's possible that that is the case here.
It's hard to give you more explicit help given the information in the question, I cannot recreate it easily. If you can create a MWE-solution and upload it somewhere I can have a look, but I'm afraid I can't help more here.

I don't think composing expressions like that is supported by default.
You could try LINQKit
Here's an example from the LINQKit page; one Expression<Func<>> inside another:
Call Invoke to call the inner expression Call Expand on the final
result. For example:
Expression<Func<Purchase,bool>> criteria1 = p => p.Price > 1000;
Expression<Func<Purchase,bool>> criteria2 = p => criteria1.Invoke (p)
|| p.Description.Contains ("a");
Console.WriteLine (criteria2.Expand().ToString());
(Invoke and Expand are extension methods in LINQKit.) Here's the output:
p => ((p.Price > 1000) || p.Description.Contains("a"))
Notice that we have a nice clean expression: the call to Invoke has
been stripped away.

Error when using a Linq Expression variable instead of lambda expression directly

I have the following code to get a collection of Types from an Assembly where the Type is a Class:
Assembly assembly = Assembly.LoadFile(DLLFile);
var types = assembly.GetTypes().AsEnumerable().Where(x => x.IsClass);
This works fine and as expected. However I wanted to pull out the lambda expression to a Linq Expression variable (as later on it will be used in a parameter of this method). So I did the following:
private Expression<Func<Type, bool>> _standardFilter = (x => x.IsClass);
Assembly assembly = Assembly.LoadFile(DLLFile);
var types = assembly.GetTypes().AsEnumerable().Where(_standardFilter);
However this won't compile with the error:
System.Collections.Generic.IEnumerable<System.Type>' does not contain a
definition for 'Where' and the best extension method overload
'System.Linq.Enumerable.Where<TSource>(System.Collections.Generic.IEnumerable<TSource>, System.Func<TSource,int,bool>)' has some invalid arguments
I understand that my expression doesn't conform to the predicate System.Func<TSource,int,bool>, however the Where function has an overload that takes a predicate of System.Func<TSource,bool>, which as far as I can tell should work.
I have tried converting the result of assembly.GetTypes() (which is an array) to a List in several ways without it helping the issue.
I have also made sure that I have got all the correct using statements for this class as that seems to be an issue several people run into during my Googling.
In the past I have managed to use this same technique on a IQueryable collection, but I don't understand why this won't work when the Where function is available on the IEnumerable collection and should accept the predicate I am providing.
Thanks very much for any assistance.

Compile your expression into executable code (delegate):
var types = assembly.GetTypes().AsEnumerable().Where(_standardFilter.Compile());
As #Kirk stated, it's better not to use expression tree, if you are not going to analyze it. Simply use filter of type Func<Type, bool>.

Note that the Enumerable extension method takes a Func, not an Expression. So your argument is incorrectly typed.
This is in contrast with Entity Framework, which takes Expression<Func> rather than plain Func.
So be attentive to method signatures, they are similar, can both be called with the same lambda, but are actually different!
The reason behind that is that EF inspects the Expression and converts it to SQL code, whereas Linq to Object simply executes the predicate in memory.

You could try this instead:
private Func<Type, bool> standardFilter;
standardFilter = (type) => {
return type.IsClass;
};
var types = assembly.GetTypes().AsEnumerable().Any(x => standardFilter(x));
Then you're free to change your implementation of standardFilter. As long as it takes in a Type and returns bool for whether or not that should be included it will remain modular and accomplish what I think you're going for.

Dynamic Lambda Expression inside an Expression Query

Is it possible to use a dynamic Linq Expression inside a Query Expression?
Something like this:
from obj1 in ObjectSet1
let res = ObjectSet2.Where(* SomeExpression *)
where ...
select ...
I'm trying to build Expression.Lambda<Func<TSource, bool>> Expression as for SomeExpression.
Is it possible to to use dynamic Linq Expression within the Expression Query or do I need to build the whole Expression tree from scratch?
How can I, if any, use obj1 when I'm building SomeExpression?
Note: I'm using Entity Framework, I can't use SomeExpression.Compile() within the expression tree.

It's not only possible, it's normal. Large expression trees can be generated from smaller trees (this is what LINQ does). You only have to pass in an Expression<Func<TSource, bool>> to Where().
You can see how in my reply to this other thread here - replacing operator in Where clause Lambda with a parameter . The interesting part is in the MakeWhereLambda method.

If you're wanting it to be completely dynamic, one possible option would be to use Expression Tree Serialization:
http://expressiontree.codeplex.com/
I started to use this a while back but that task got reprioritized, so I'm not sure how suitable it is for any given task. That said, it looks pretty good...
Hope this helps.
Nate

How to create dynamic lambda based Linq expression from a string in C#?

I'm having some difficulty creating Lambda-based Linq expressions from a string. Here is my basic case using this sample object/class:
public class MockClass
{
public string CreateBy { get; set; }
}
Basically I need to convert a string like this:
string stringToConvert = “x => x.CreateBy.Equals(filter.Value, StringComparison.OrdinalIgnoreCase”;
Into a to predicate/linq expression:
System.Linq.Expressions.Expression<Func<T, bool>> or in this example
System.Linq.Expressions.Expression<Func<MockClass, bool>>
So it is equivalent to the Linq expression inside of the Where method below:
query = query.Where(x => x.CreateBy.Equals(filter.Value,
StringComparison.OrdinalIgnoreCase));
I've tried using the following helpers but can't seem to figure out how to get them to work in this type of case where I want to be able to build a linq expression from string that is not know ahead of time:
http://www.albahari.com/nutshell/predicatebuilder.aspx
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx (it’s now available as a NuGet package as well called “DynamicQuery”)

A similar question was asked here:
Is there an easy way to parse a (lambda expression) string into an Action delegate?
As I understand it, this 'Dynamic Query' is actually a framework for passing in restrictions for a Where clause without using a lambda expression.
The significance of that is that lambda expressions are not dynamic methods, they're anonymous methods. If you ever take a look in an assembly, you'll see that your lambda expressions are converted into closures with any free variables as fields. The class has a method with a signature matching yours, field variables are assigned at the point of invocation.
One good way to think about that is that it implies that your lambda expression is interpreted by the c# compiler at compile-time, and variables are resolved by instantiating an object from this class at run-time.
To demonstrate this, consider the following:
var myLambda = x => x * x
You'll notice this doesn't work. That's because, in order to create the related class/method, the compiler must know, at compile-time, the type of x.
All of this is important because the notion of a lambda expression doesn't exist at the CLR at run-time (in the same form it is in code). A string that looks like a lambda expression is exactly that...