We Keep Coding

Evaluator.PartialEval reduce provided expression

In one of my projects I have an ExpressionVisitor to translate provided expression into some query string. But before translating it I need to evaluate all refferences in the expression to real values. To do that I use Evaluator.PartialEval method from EntityFramework Project.
Assuming I have this query:
var page = 100;
var query = myService.AsQueryable<Product>()
//.Where(x=>x.ProductId.StartsWith(p.ProductId))
.Skip(page)
.Take(page);
var evaluatedQueryExpr = Evaluator.PartialEval(query.Expression);
As you can see I have commented Where method. In this case evaluatedQueryExpr will not contain the methods Take and Skip.
However, if I use any other method with Expression before Take or Skip everything works, Evaluator evaluates an expression correctly and return it fully.
I found out that the problem occurs in the line 80 of the Evaluator class:
return Expression.Constant(fn.DynamicInvoke(null), e.Type);
Could you explain why this happens and suggest a workaround?
Update
here is a project on github
LinqToSolrQueriable inherited from IOrderedQueryable
LinqToSolrProvider inherited from IQueryProvider including line range causing the issue

The good news are that the expression is not really reduced (Skip and Take are still there :), but is simply converted from MethodCallExpression to ConstantExpression containing the original expression:
query.Expression:
.Call System.Linq.Queryable.Take(
.Call System.Linq.Queryable.Skip(
.Constant<LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product]>(LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product]),
100),
100)
evaluatedQueryExpr:
.Constant<System.Linq.IQueryable`1[LinqToSolrTest.Product]>(LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product])
Here the debug display is giving you a wrong impression. If you take the ConstaintExpression.Value, you'll see that it's a IQueryable<Product> with Expression property being exactly the same as the original query.Expression.
The bad news are that this is not what you expect from PartialEval - in fact it doesn't do anything useful in this case (except potentially breaking your query translation logic).
So why is this happening?
The method you are using from EntityFramework.Extended library is in turn taken (as indicated in the comments) from MSDN Sample Walkthrough: Creating an IQueryable LINQ Provider. It can be noticed that the PartialEval method has two overloads - one with Func<Expression, bool> fnCanBeEvaluated parameter used to identify whether a given expression node can be part of the local function (in other words, to be partially evaluated or not), and one without such parameter (used by you) which simply calls the first passing the following predicate:
private static bool CanBeEvaluatedLocally(Expression expression)
{
return expression.NodeType != ExpressionType.Parameter;
}
The effect is that it stops evaluation of ParameterExpression type expressions and any expressions containing directly or indirectly ParameterExpression. The last should explain the behavior you are observing. When the query contains Where (and basically any LINQ operator) with parametrized lambda expression (hence parameter) before the Skip / Take calls, it would stop evaluation of the containing methods (which you can see from the above query.Expression debug view - the Where call will be inside the Skip).
Now, this overload is used by the MSDN example to evaluate a concrete nested Where method lambda expression and is not generally applicable for any type of expression like IQueryable.Expression. In fact the linked project is using the PartialEval method in a single place inside QueryCache class, and also calling the other overload passing a different predicate which in addition to ParameterExpressions stops the evaluation of any expression with result type of IQueryable.
Which I think is the solution of your problem as well:
var evaluatedQueryExpr = Evaluator.PartialEval(query.Expression,
// can't evaluate parameters or queries
e => e.NodeType != ExpressionType.Parameter &&
!typeof(IQueryable).IsAssignableFrom(e.Type)
);

Passing Func<> to IQueryable vs Expression<Func<>>

I'm starting to look under the LINQ hood a little more, and I'm having trouble understanding some of the LINQ extension method overloads.
For example, lets say I'm querying a DbContext using .Where(). I'm always passing a standard Func<>, as opposed to an Expression<> of said Func<>. Example query below:
var db = new MyContext();
var foo = db.products.Where(p => p.Category == "books");
Here is where I'm confused. When I look at the available method signatures, I would assume the overload I'm using above would be returning me an IEnumerable...but it's actually returning an IQueryable. How is this possible, if the IQueryable overload is expecting an Expression as opposed to just a Func? It feels like the compiler is somehow helping me out by (in this case) building the Expression for me, but I can't find a resource that explains if this is the case. Thanks!

It feels like the compiler is somehow helping me out by (in this case) building the Expression for me
That is exactly what is happening. The compiler can interpret lambdas (as long as they don't have a "statement body") as either delegates or expression trees. The Queryable.Where<T>(this IQuerayble<T>, ...) extension method takes precedence over the Enumerable.Where<T>(this IEnumerable<T>, ...) extension method, so the compiler chooses to interpret your predicate as an expression tree.

It feels like the compiler is somehow helping me out by (in this case) building the Expression for me
Correct, the compiler is compiling the lambda into an Expression rather than a Func.
I can't find a resource that explains if this is the case
From MSDN:
When a lambda expression is assigned to a variable of type Expression<TDelegate>, the compiler emits code to build an expression tree that represents the lambda expression.

You're not passing a Func in that example. You're passing an Expression. A lambda can compile down into either a delegate or an expression, based on the context around it. In this case the context around it is expecting an expression, so that is what the lambda compiles into. If you actually did pass in a Func and not a lambda (which could be either) then you would get an IEnumerable for your result, not an IQueryable.

The compiler can automatically build an expression tree for you from a lamba expression, and that is what's happening here.
http://msdn.microsoft.com/en-gb/library/bb397951.aspx

Passing a compiled Expression to Linq To NHibernate

I have a collection of compiled expressions which I combined into one expression because I want to build dynamically my linq query. See ExpressionTree Compile() method generate a stackoverflow exception for the reason I used compiled expression in order to prevent from stackoverflow because in my computer when the query contains more than 7000 expressions, it throws this exception.)
Then I use the new generated expression and pass it to my FindAll method. Problem, NHibernate is not able to execute the query and says:
unable to cast object of type 'nhibernate.hql.ast.parameter' to type 'nhibernate.hql.ast.hqlbooleanexpression'
public IList<T> FindAll(Expression<Func<T, bool>> criteria)
{
return SessionFactory.GetCurrentSession()
.QueryOver<T>()
.Where(criteria)
.List();
}
I debuged and found that nhibernate tries to convert the compiled expression to boolean expression in file HqlTreeNode (method: HqlTreeNodeExtensions.AsBooleanExpression(this HqlTreeNode node) which of course doesn't works. Then what solution should I use ?
Here what the criteria variable looks like:
(Invoke(value(System.Func`2[Something.SomeEntity,System.Boolean]) // this don't work
For information, if it wasn't compiled, it would have looked like something like this:
([someEntity].UserID == 1) // this works
Thank you.

How to create dynamic lambda based Linq expression from a string in C#?

I'm having some difficulty creating Lambda-based Linq expressions from a string. Here is my basic case using this sample object/class:
public class MockClass
{
public string CreateBy { get; set; }
}
Basically I need to convert a string like this:
string stringToConvert = “x => x.CreateBy.Equals(filter.Value, StringComparison.OrdinalIgnoreCase”;
Into a to predicate/linq expression:
System.Linq.Expressions.Expression<Func<T, bool>> or in this example
System.Linq.Expressions.Expression<Func<MockClass, bool>>
So it is equivalent to the Linq expression inside of the Where method below:
query = query.Where(x => x.CreateBy.Equals(filter.Value,
StringComparison.OrdinalIgnoreCase));
I've tried using the following helpers but can't seem to figure out how to get them to work in this type of case where I want to be able to build a linq expression from string that is not know ahead of time:
http://www.albahari.com/nutshell/predicatebuilder.aspx
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx (it’s now available as a NuGet package as well called “DynamicQuery”)

A similar question was asked here:
Is there an easy way to parse a (lambda expression) string into an Action delegate?
As I understand it, this 'Dynamic Query' is actually a framework for passing in restrictions for a Where clause without using a lambda expression.
The significance of that is that lambda expressions are not dynamic methods, they're anonymous methods. If you ever take a look in an assembly, you'll see that your lambda expressions are converted into closures with any free variables as fields. The class has a method with a signature matching yours, field variables are assigned at the point of invocation.
One good way to think about that is that it implies that your lambda expression is interpreted by the c# compiler at compile-time, and variables are resolved by instantiating an object from this class at run-time.
To demonstrate this, consider the following:
var myLambda = x => x * x
You'll notice this doesn't work. That's because, in order to create the related class/method, the compiler must know, at compile-time, the type of x.
All of this is important because the notion of a lambda expression doesn't exist at the CLR at run-time (in the same form it is in code). A string that looks like a lambda expression is exactly that...

Expression tree emits runtime code?

When you are building the expression tree at runtime there's no code
emitted. It's a way to represent .NET code at runtime...
Ok...
Now lets say I have this code :
ParameterExpression p = Expression.Parameter (typeof (string), "s");
MemberExpression stringLength = Expression.Property (p, "Length");
ConstantExpression five = Expression.Constant (5);
BinaryExpression comparison = Expression.LessThan (stringLength, five);
Expression<Func<string, bool>> lambda= Expression.Lambda<Func<string, bool>> (comparison, p);
Func<string, bool> runnable = lambda.Compile();
This code Wont be in IL ? of course it will be ! ( maybe the last line wont emit code until compile ...but the first lines I think will emit code !)
So what am i saving here ?
Ok so the first 5 lines did emit code and the last one didn't... big deal.
What am i missing ? Can you please let me see the whole picture ?

With an Expression Tree, you build a description of some code instead of the code itself.
Expression Trees should not be used in the context of writing regular code that 'shouldn't be compiled at compile time'. They should be used in more dynamic scenarios.
The expression tree you show will compile to: s.Length < 5 and you invoke the runnable with bool isStringSmallerThan5 = runnable("MyString").
The whole idea of Expression Trees is that they describe some code and can be compiled at runtime. This means that you can do the following:
BinaryExpression comparison = null;
if (lessThen)
{
comparison = Expression.LessThan(stringLength, five);
}
else
{
comparison = Expression.GreaterThan(stringLength, five);
}
Now you can change the behavior of your code at runtime!
The biggest use of Expression Trees is that they can be interpreted by a provider. For example Linq To Entities uses Expression Trees and compiles them to SQL code that can be run against the database. LinqToXml is another example of what you can do with Expression Trees.
This a nice blog post to get you started.

Expression trees are useful when you receive them in a method since they enable you to make more complex use of the expression content. If you receive a predicate in a method, you can run it against a target and check the result. If you receive the expression tree representing an expression tree, you can parse it and do something useful with it. An example is LINQ which utilizes this many places, but among one in the "Where"-methods. Catching the expression tree rather than the IL makes it relatively straight forward to translate into SQL rather than just do a full 'Select' and run predicates against the materialized result.