When you are building the expression tree at runtime there's no code
emitted. It's a way to represent .NET code at runtime...
Ok...
Now lets say I have this code :
ParameterExpression p = Expression.Parameter (typeof (string), "s");
MemberExpression stringLength = Expression.Property (p, "Length");
ConstantExpression five = Expression.Constant (5);
BinaryExpression comparison = Expression.LessThan (stringLength, five);
Expression<Func<string, bool>> lambda= Expression.Lambda<Func<string, bool>> (comparison, p);
Func<string, bool> runnable = lambda.Compile();
This code Wont be in IL ? of course it will be ! ( maybe the last line wont emit code until compile ...but the first lines I think will emit code !)
So what am i saving here ?
Ok so the first 5 lines did emit code and the last one didn't... big deal.
What am i missing ? Can you please let me see the whole picture ?
With an Expression Tree, you build a description of some code instead of the code itself.
Expression Trees should not be used in the context of writing regular code that 'shouldn't be compiled at compile time'. They should be used in more dynamic scenarios.
The expression tree you show will compile to: s.Length < 5 and you invoke the runnable with bool isStringSmallerThan5 = runnable("MyString").
The whole idea of Expression Trees is that they describe some code and can be compiled at runtime. This means that you can do the following:
BinaryExpression comparison = null;
if (lessThen)
{
comparison = Expression.LessThan(stringLength, five);
}
else
{
comparison = Expression.GreaterThan(stringLength, five);
}
Now you can change the behavior of your code at runtime!
The biggest use of Expression Trees is that they can be interpreted by a provider. For example Linq To Entities uses Expression Trees and compiles them to SQL code that can be run against the database. LinqToXml is another example of what you can do with Expression Trees.
This a nice blog post to get you started.
Expression trees are useful when you receive them in a method since they enable you to make more complex use of the expression content. If you receive a predicate in a method, you can run it against a target and check the result. If you receive the expression tree representing an expression tree, you can parse it and do something useful with it. An example is LINQ which utilizes this many places, but among one in the "Where"-methods. Catching the expression tree rather than the IL makes it relatively straight forward to translate into SQL rather than just do a full 'Select' and run predicates against the materialized result.
Related
In one of my projects I have an ExpressionVisitor to translate provided expression into some query string. But before translating it I need to evaluate all refferences in the expression to real values. To do that I use Evaluator.PartialEval method from EntityFramework Project.
Assuming I have this query:
var page = 100;
var query = myService.AsQueryable<Product>()
//.Where(x=>x.ProductId.StartsWith(p.ProductId))
.Skip(page)
.Take(page);
var evaluatedQueryExpr = Evaluator.PartialEval(query.Expression);
As you can see I have commented Where method. In this case evaluatedQueryExpr will not contain the methods Take and Skip.
However, if I use any other method with Expression before Take or Skip everything works, Evaluator evaluates an expression correctly and return it fully.
I found out that the problem occurs in the line 80 of the Evaluator class:
return Expression.Constant(fn.DynamicInvoke(null), e.Type);
Could you explain why this happens and suggest a workaround?
Update
here is a project on github
LinqToSolrQueriable inherited from IOrderedQueryable
LinqToSolrProvider inherited from IQueryProvider including line range causing the issue
The good news are that the expression is not really reduced (Skip and Take are still there :), but is simply converted from MethodCallExpression to ConstantExpression containing the original expression:
query.Expression:
.Call System.Linq.Queryable.Take(
.Call System.Linq.Queryable.Skip(
.Constant<LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product]>(LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product]),
100),
100)
evaluatedQueryExpr:
.Constant<System.Linq.IQueryable`1[LinqToSolrTest.Product]>(LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product])
Here the debug display is giving you a wrong impression. If you take the ConstaintExpression.Value, you'll see that it's a IQueryable<Product> with Expression property being exactly the same as the original query.Expression.
The bad news are that this is not what you expect from PartialEval - in fact it doesn't do anything useful in this case (except potentially breaking your query translation logic).
So why is this happening?
The method you are using from EntityFramework.Extended library is in turn taken (as indicated in the comments) from MSDN Sample Walkthrough: Creating an IQueryable LINQ Provider. It can be noticed that the PartialEval method has two overloads - one with Func<Expression, bool> fnCanBeEvaluated parameter used to identify whether a given expression node can be part of the local function (in other words, to be partially evaluated or not), and one without such parameter (used by you) which simply calls the first passing the following predicate:
private static bool CanBeEvaluatedLocally(Expression expression)
{
return expression.NodeType != ExpressionType.Parameter;
}
The effect is that it stops evaluation of ParameterExpression type expressions and any expressions containing directly or indirectly ParameterExpression. The last should explain the behavior you are observing. When the query contains Where (and basically any LINQ operator) with parametrized lambda expression (hence parameter) before the Skip / Take calls, it would stop evaluation of the containing methods (which you can see from the above query.Expression debug view - the Where call will be inside the Skip).
Now, this overload is used by the MSDN example to evaluate a concrete nested Where method lambda expression and is not generally applicable for any type of expression like IQueryable.Expression. In fact the linked project is using the PartialEval method in a single place inside QueryCache class, and also calling the other overload passing a different predicate which in addition to ParameterExpressions stops the evaluation of any expression with result type of IQueryable.
Which I think is the solution of your problem as well:
var evaluatedQueryExpr = Evaluator.PartialEval(query.Expression,
// can't evaluate parameters or queries
e => e.NodeType != ExpressionType.Parameter &&
!typeof(IQueryable).IsAssignableFrom(e.Type)
);
With entity framework, we can do :
MyContext context = ... // a normal EF context
var products = context.Products.Where(p => p.Location == "France") ;
or
var products = context.Products.Where(p => p.CategoryId == 54) ;
Which are both translate in their equivalent SQL query.
OK, but somewhere over there, there's a piece of code that handle this :
public static IEnumerable<T> Where(Func<bool, T> func) {
......
}
From that Where function, how do LINQ to SQL know what's the implementation of func ?
Obvious answer maybe, but I can't really find it.
You should really do a Go To Definition on your code. The function used for LINQ-to-SQL and for Entity Framework is
IQueryable<TSource> Where<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, bool>> predicate)
contained in the System.Linq.Queryable that uses expression trees, that are a constuct able to "describe" pieces of code. A little quote:
Expression trees represent code in a tree-like data structure, where each node is an expression, for example, a method call or a binary operation such as x < y.
For example, your first expression
var products = context.Products.Where(p => p.Location == "France");
is converted by the C# compiler to this code:
ParameterExpression par = Expression.Parameter(typeof(Product), "p");
LambdaExpression lambda = Expression.Lambda(
Expression.Equal(
Expression.Property(par, "Location"),
Expression.Constant("France")),
par);
var products = context.Products.Where(lambda);
Now... while the creation of an expression tree is quite simple, the reverse operation (de-building an expression tree and creating a query) is something VERY VERY complex. Big big headache complex. Nearly magic-level complex :-)
The problem isn't de-building the expression tree. That is easy. You use an ExpressionVisitor and you are done. The problem is merging the various layers of LINQ query and comprehending what the programmer wanted to obtain.
I'll add that IL (the "assembly" of .NET) is high level enough that it is possible to decompile it (see for example IlSpy). There is at least a library, DelegateDecompiler that is able to decompile a delegate to an Expression Tree, so even without Expression Trees, LINQ-to-SQL and EF could have used something similar and decompiled methods directly to SQL language.
I have a collection of compiled expressions which I combined into one expression because I want to build dynamically my linq query. See ExpressionTree Compile() method generate a stackoverflow exception for the reason I used compiled expression in order to prevent from stackoverflow because in my computer when the query contains more than 7000 expressions, it throws this exception.)
Then I use the new generated expression and pass it to my FindAll method. Problem, NHibernate is not able to execute the query and says:
unable to cast object of type 'nhibernate.hql.ast.parameter' to type 'nhibernate.hql.ast.hqlbooleanexpression'
public IList<T> FindAll(Expression<Func<T, bool>> criteria)
{
return SessionFactory.GetCurrentSession()
.QueryOver<T>()
.Where(criteria)
.List();
}
I debuged and found that nhibernate tries to convert the compiled expression to boolean expression in file HqlTreeNode (method: HqlTreeNodeExtensions.AsBooleanExpression(this HqlTreeNode node) which of course doesn't works. Then what solution should I use ?
Here what the criteria variable looks like:
(Invoke(value(System.Func`2[Something.SomeEntity,System.Boolean]) // this don't work
For information, if it wasn't compiled, it would have looked like something like this:
([someEntity].UserID == 1) // this works
Thank you.
Is it possible to use a dynamic Linq Expression inside a Query Expression?
Something like this:
from obj1 in ObjectSet1
let res = ObjectSet2.Where(* SomeExpression *)
where ...
select ...
I'm trying to build Expression.Lambda<Func<TSource, bool>> Expression as for SomeExpression.
Is it possible to to use dynamic Linq Expression within the Expression Query or do I need to build the whole Expression tree from scratch?
How can I, if any, use obj1 when I'm building SomeExpression?
Note: I'm using Entity Framework, I can't use SomeExpression.Compile() within the expression tree.
It's not only possible, it's normal. Large expression trees can be generated from smaller trees (this is what LINQ does). You only have to pass in an Expression<Func<TSource, bool>> to Where().
You can see how in my reply to this other thread here - replacing operator in Where clause Lambda with a parameter . The interesting part is in the MakeWhereLambda method.
If you're wanting it to be completely dynamic, one possible option would be to use Expression Tree Serialization:
http://expressiontree.codeplex.com/
I started to use this a while back but that task got reprioritized, so I'm not sure how suitable it is for any given task. That said, it looks pretty good...
Hope this helps.
Nate
I have this expression :
Expression<Func<string, bool>> f = s => s.Length < 5;
ParameterExpression p = Expression.Parameter (typeof (string), "s");
MemberExpression stringLength = Expression.Property (p, "Length");
ConstantExpression five = Expression.Constant (5);
BinaryExpression comparison = Expression.LessThan (stringLength, five);
Expression<Func<string, bool>> lambda= Expression.Lambda<Func<string, bool>> (comparison, p);
//lets : test
Func<string, bool> runnable = lambda.Compile();
Console.WriteLine (runnable ("kangaroo")); // False
Console.WriteLine (runnable ("dog")); //True
I want to ask about the .Compile()
What does it compile ? And what is the difference between the first execution vs later executions...?
Compile should be something that happens once and not happens again later ....
What / How does it help me ?
When you are building the expression tree at runtime there's no code emitted. It's a way to represent .NET code at runtime.
Once you call the .Compile method on the expression tree the actual IL code is emitted to convert this expression tree into a delegate (Func<string, bool> in your case) that you could invoke at runtime. So the code that this expression tree represents can be executed only after you compile it.
Calling Compile is an expensive operation. Basically you should be calling it once and then caching the resulting delegate that you could use to invoke the code many times.
The Expression<Func<string,bool>> is only a representation of an expression, it cannot be executed. Calling Compile() gives you a compiled delegate, a piece of code that you can call. Essentially, your program composes a small code snippet at runtime, and then call it as if it were processed by the compiler. This is what the last two lines of your code do: as you can see, the compiled snippet can analyze the length of the string that you pass in - when the length is less than five, you get a True back; when it's five or more, you get a False.
What happens on first execution of the compiled snippet is platform-dependent, and should not be detectable by programmers using the .NET platform.
Compile() takes the expression tree (which is a data representation of some logic) and converts it to IL which can then be executed directly as a delegate.
The only difference between the first execution and later executions is the possibility that Compile() won't trigger the JIT compilation from IL to native processor code. That may happen on first execution.