I completely understand the concept of expression trees, but I am having a hard time trying to find situations in which they are useful. Is there a specific instance in which expression trees can be applied? Or is it only useful as a transport mechanism for code? I feel like I am missing something here. Thanks!
Some unit test mocking frameworks make use of expression trees in order to set up strongly typed expectations/verifications. Ie:
myMock.Verify(m => m.SomeMethod(someObject)); // tells moq to verify that the method
// SomeMethod was called with
// someObject as the argument
Here, the expression is never actually executed, but the expression itself holds the interesting information. The alternative without expression trees would be
myMock.Verify("SomeMethod", someObject) // we've lost the strong typing
Or is it only useful as a transport mechanism for code?
It's useful as an execution mechanism for code. Using the interpreter pattern, expression trees can directly be interpreted. This is useful because it's very easy and fast to implement. Such interpreters are ubiquitous and used even in cases that don't seem to “interpret” anything, e.g. for printing nested structures.
Expression trees are useful when you need to access function logic in order to alter or reapply it in some way.
Linq to SQL is a good example:
//a linq to sql statement
var recs (
from rec in LinqDataContext.Table
where rec.IntField > 5
select rec );
If we didn't have expression trees this statement would have to return all the records, and then apply the C# where logic to each.
With expression trees that where rec.IntField > 5 can be parsed into SQL:
--SQL statment executed
select *
from [table]
where [table].[IntField] > 5
Related
There are two important steps to compiling a LINQ query in C#. The first is transforming LINQ query syntax into a chain of method calls, as described in section 7.16 of the C# language specification. This transformation process is specified in enough detail that a language developer can use it to implement similar query syntax on a new CLR language.
The second step is turning lambda expressions into expression trees, which takes place when calling a query method that returns IQueryable, but not when calling a method that returns IEnumerable. Is it ever specified how this transformation takes place, comparable to the explanation of the query syntax transformation process?
Construction of expression trees is explicitly not defined, actually. Compiler developers are free to do use whatever approach they wish, provided of course that executing the expression produces the same result as invoking the lambda.
This is a quote from the C# Language Specification:
6.5.2 Evaluation of anonymous function conversions to expression tree types
Conversion of an anonymous function to an expression tree type produces an expression tree (§4.6). More precisely, evaluation of the anonymous function conversion leads to the construction of an object structure that represents the structure of the anonymous function itself. The precise structure of the expression tree, as well as the exact process for creating it, are implementation defined.
I added the boldface at the end.
I suspect that this was deliberately left unspecified to give compiler developers the freedom to implement whatever optimizations they find helpful. A rigid specification of the expression tree would prevent that.
Is it ever specified how this transformation takes place, comparable
to the explanation of the query syntax transformation process?
As NSFW has stated, no.
On a practical, these expression trees can change from framework to framework. A real life example would be this:
We were using expression lambdas to get the property info through expression trees.
Such as void DoSomething<P, K>(P model, Expression<Func<P, K> propertySelector), and the usage DoSomething(model, m => m.Property)
The actual property interrogation was done inside DoSomething, through reflection. This is very classical, and variants of such code exists over the intenet.
Now, this is cool, it worked nicely in .NET 4.0. However, as soon as I tried 4.5, it blew up completely, as the underlying expression tree has changed.
You can be sure that Roslyn will introduce a lot new "bugs", in the sense that some client code relies on the representation how lambdas are translated to expression trees(If you really insist doing that - using Visitor class minimizes the chances of breaking).
Ensuring that the expression trees stay the same would be major task, and it would be also limiting(speed-wise for example)
This is a follow up to a question I asked earlier seen here:
Confused about passing Expression vs. Func arguments
The accepted answerer there suggests refactoring an Expression referencing local objects into something that Linq to Entities can actually execute against the backing store (in my case SQL Server)
I've spent a long time trying to come up with something that will work for what I'm doing. My original
Func<Thing, bool> whereClause
was referencing a local Dictionary object which Linq to Entities or SQL could not understand at runtime. I tried refactoring into multiple lists which faked a dictionary, and Arrays after that. Each time, I got runtime errors complaining about the context doesn't recognize things like the methods on a List, or array indexers.
Eventually I gave up and just provided an additional method which takes a Func argument for when I cannot come up with the proper Expression.
I'm not trying to find a solution to my specific problem, I'm just wondering in general if it is always possible to convert, say a
Func<Thing, bool>
to an equivalent
Expression<Func<Thing, bool>>
which can run against Linq to Entities.
Or if there are many examples of querys where you simply must pull the data into memory first.
You don't convert a Func to an expression tree - the compiler converts a lambda expression to an expression tree... and no, that's not always possible. For example, you can't convert a statement lambda to an expression tree:
Expression<Func<string, int>> valid = text => text.Length;
Expression<Func<string, int>> invalid = text => { return text.Length; };
There are various other restrictions, too.
Even when you can create an expression tree (and if you do it manually you can build ones which the C# compiler wouldn't, particularly in .NET 4) that's not the same thing as the expression tree representing something that LINQ to SQL (etc) can translate appropriately.
Jon is of course correct; you turn a lambda into an expression tree.
To expand a bit on his "various other restrictions" handwave: a lambda converted to an expression tree may not contain:
statements
expressions useful primarily for their state mutations: assignment, compound assignment, increment and decrement operators
dynamic operations of any kind
multi-dimensional array initializers
removed partial methods
base access
pointer operations of any kind
sizeof(T) except for where T is a built-in type
COM-style indexed property invocations
COM-style "optional ref" invocations
C-style variadic method invocations
optional-argument and named-argument invocations
method groups (except of course when in an invocation)
That's not an exhaustive list; there are some other weird corner cases. But that should cover the majority of them.
Ugly:
string city = null;
if (myOrder != null && myOrder.Customer != null)
city = myOrder.Customer.City;
Better (maybe monad):
var city = myOrder
.With(x => x.Customer)
.With(x => x.City)
Even better? Any reason this couldn't be written?
var city = Maybe(() => myOrder.Customer.City);
Yes, it should be possible. However, it's quite a bit more complicated than it looks on the surface to implement an expression tree re-writer correctly. Especially if you want to be able to correctly handle fields, properties, indexed properties, method calls, and other constructs that are valid in arbitrary expressions.
It may also not be the most well-performing operation since to evaluate the expression you have to dynamically compile the expression tree into a lambda function each time.
There's an implementation on this pattern on CodePlex. I've never personally used it, so I can't say how well implemented it is, or whether it handles all of the cases I've described.
An alternative to creating an expression tree re-writer, is to write Maybe() to accept a lambda function (rather than an expression tree) and catch any ArgumentNullException thrown, returning default(T) in those cases. It rubs many people the wrong way to use exceptions for flow control in this way ... but it's certainly an easier implementation to get right. I personally avoid it myself since it can mask null reference errors within methods called as part of the expression, which is not desirable.
I recently implemented some monads in C# (including a basic expression tree parser inspired by a Bartosz Milewski article).
Take a look if you're interested: https://github.com/htoma/monads/blob/master/expressionMonad/expressionMonad/Program.cs
Some points that come to my mind:
.Solutions work fine for memory objects, but run into trouble with EF as these static calls cannot be converted to run against persistent storage (that is, SQL DB). This limits the scope of application somewhat heavily.
I will practically always want to know whether the chain did produce valid result. Hence, I will have one conditional block if(city == null) in any case.
Any current solution other than "the ugly" involves Expressions.
Hence, my choice would be something like
var property = ( () => myOrder.Customer.City );
city = HasValue(property) ? property.Invoke() : "unknown";
HasValue(Expression e) walks through the LINQ expression tree recursively until it either reaches end (returning true) or encounters null-valued property (returning false). The implementation should be simple, use MethodInfo Member of MemberExpression class to parse the AST. One could also implement getter this way as Brian suggested, but I like above better because HasValue always returns bool. Further:
Member invokations can be handled, too.
Evaluation could be made as myOrder.HasValue(x => x.Customer.City) but that brings some complications.
Simpler answer if the objects are cheap to create and you want to avoid null checks:
myOrder.NewIfNull().Customer.NewIfNull().City;
This will return either null or some initial value you set in the constructor or field initializer for City. NewIfNull isn't built-in, but it's real easy:
public static T NewIfNull<T>(this T input) where T:new()
{
return input ?? new T();
}
I know that my implementation of Maybe (as per CodeProject article) carries a cost, but I'm sure it's nothing compared to the idea of getting an Expression<T> involved there. Basically you're talking Reflection all the way. I wouldn't mind if it was pre-compiled, Roslyn-style, but we aren't there yet.
I'd argue that my implementation's advantage goes way beyond the mythical ?. operator. The ability to write a whole algorithm using a chain such as this means that you can inject your own creations (such as If, Do, etc.) and provide your own specialized logic.
I realize this is more complicated than what you're trying to do here, but it doesn't look like we're going to get a null-coalescing dot operator in C#5.
I'm trying to mimic the LINQ Where extension method for my ADO.NET DAL methods.
Bascially, my aim is to have a single method that I can call. Such as:
Product p = Dal.GetProduct(x => x.ProductId == 32);
Product p2 = Dal.GetProduct(x => x.ProductName.Contains("Soap"));
I then want to dissect those Predicates and send the filter options to parameters in an ADO.NET Stored Procedure call.
Any comments greatly appreciated.
As #Daniel points out, this is far from simple. The solution outline is to let GetProduct take an argument of type Expression<Func<Product, bool>>. You then have to traverse the parse-tree of this expression, generating the correct SQL for functions known and also decide how to handle unknown functions. There are basically two options for that:
Throw an error (as linq-to-sql does).
Skip it in the translation and then apply it on the returned result. The performance impact of this can of course be huge if a lot of data is retreived just to be filtered out.
It would be a fun exercise to do it - but I can hardly see a way to justify it in the real world, when there are already linq2sql, linq2entities and linq2NHibernate that does the job.
In addition to Anders's answer, I just want to mention that you can analyze the expression tree by using an expression visitor. To do that, you can inherit the ExpressionVisitor class (it's new in .NET 4, but you can find a 3.5 implementation in LinqKit) and override the methods you want to analyze each node.
You might also be interested in those links :
Building a LINQ Provider
Walkthrough: Creating an IQueryable LINQ Provider
I have no idea if this is possible ... but it would be cool. the question is whether it is possible but then a bit of an example if possible.
I am not sure what method signature you would use to pass the lambda expression into.
Eg the method IList<Group> GetGroups()
How would you modify that to be able to pass a lambda expression into it?
The next question is how would you code a lambda expression to return all Group objects where for example
where .deleted == false or
where .DateAdded > aDate
Yeah, I want the moon on a stick ;)
Thanks in advance.
(edit I am thinking this is a bit ill-conceived actually because of the data access layer that would actually fetch the data ... but assume that you are querying some object collection over the service and don't have to worry about the dal).
You could declare the GetGroups method with a parameter of type Expression<Func<Group, bool>>, which represents a predicate that the group must match in order to be returned :
IList<Group> GetGroups(Expression<Func<Group, bool>> predicateExpression);
The trouble is, expressions can't be serialized, so you couldn't send it to the WCF service... However, you might find a way to do it with the Expression Tree Serialization project.
You could
define a simple query language that your back-end service understands
the web service exposes a method that takes a string in this query language
write a client-side conversion system that takes in an IQueryable full of expression trees, and translates that into the query language
now the client can either write queries in your query language directly, or can write LINQ queries which your translator turns into your query language
hey, you just invent LINQ-To-Tortoise!
Matt Warren has seventeen blog articles about how to do this sort of thing.
I think that RIA services do what you want, but I do not know the magic behind it.
You could pass a "predicate": A Func<Group, bool>, which returns true or false for a given Group. Since you can pass a lambda where a Func is expected, this could be something like:
var fooGroups = GetGroups(g => g.Name.StartsWith("Foo"));