A lambda expression is an anonymous method, which under the covers is a delegate so I can do something like this:
delegate bool Foo(int x);
Foo bar = x => x == 1;
Passing this delegate to an Enumerable extension method makes perfect sense, as the typical expected argument is a Func, which is shorthand for a delegate:
public static IEnumerable<TSource> Where<TSource>(this IEnumerable<TSource> source, Func<TSource, bool> predicate);
However, I am unclear about how it is possible to pass in the delegate to a Queryable extension method like this one:
public static IQueryable<TSource> Where<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, bool>> predicate);
This method expects an Expression<TDelegate> argument, but it is perfectly legal to pass in a lambda expression. What is the mechanism that coerces the lambda expression into Expression<TDelegate> so that it may be consumed?
I am familiar with the fact that Queryable methods build out expression trees for parsing by providers, I'm just curious about this one aspect that isn't immediately obvious to me.
UPDATE
I'm becoming less ignorant about my ignorance. Lambda expressions aren't delegates, but can be used to create either delegates or expressions:
Expression<Func<int, bool>> foo = c => c == 1;
Does the compiler infer the type based on the context? I'm guessing that must be the case, as this isn't legal:
var foo = c => c == 1;
This is described in the specification:
4.6 Expression Tree Types
If a conversion exists from a lambda expression to a delegate type D,
a conversion also exists to the expression tree type Expression<D>.
Whereas the conversion of a lambda expression to a delegate type
generates a delegate that references executable code for the lambda
expression, conversion to an expression tree type creates an
expression tree representation of the lambda expression. Expression
trees are efficient in-memory data representations of lambda
expressionsand make the structure of the lambda expressiontransparent
and explicit
So there is a conversion from a lambda to a compatible expression tree type, and the compiler emits the equivalent expression tree instead of creating a delegate.
Quite simply you can't.
However, to make IQueryable methods useful, VS2008 and above include a clever compiler trick. That a lambda expression that is a single statement may be assignable to both a delegate, and an Expression<TDelegate>. The compiler normally will hoist the expression and make a method.
But for an assignment to an Expression<TDelegate> it breaks down the statements into their syntactic meaning and turns that into an expression tree.
e.g.
Func<int,int> func = x=>x*x;
Expression<Func<int,int>> expression = x=>x*x;
The first one will probably be turned into a static method with a garbled name something like::
private static int <B012>SomeMethod(int x){
return x*x;
}
Where as the second statement will be transformed into something like::
ParameterExpression paramX = Expression.Parameter(typeof(int));
Expression<Func<int,int>> expression = Expression.Lambda<Func<int,int>>(
Expression.Multiply(paramX,paramX),paramX);
But you can not do::
expression = func;
This is not valid, as func is a delegate. You can do this though::
func=expression.Compile()
Which compiles the expressions into a func.
**Note the suggested transformations may not be a 100% correct.
The reason they did this was to allow LINQ-to-Objects (Basically Map/Reduce from other language) to share the same friendly syntax as LINQ-To-Providers. So you can write a statement that means the same thing but can change where the filtering and transformation happens.
GetEmployees().Where(e=>e.LastName=="Smith")
Can read the same, but could in theory be describing doing the filtering on this box, or an the database, or parsing an xml file or any number of various things.
I believe this has to do with how the query is built on an IQueryable. The method requires an expression tree because it can be looked inside and structured to match a more optimized (potentially) query or map more closely to the underlying data source. So simply passing in a Func would allow only for execution, whereas an Expression<Func> allows for expression trees which can be observed and executed.
Also, probably more closely answering your exact question, check this SO post out.
Related
I've read this answer and understood from it the specific case it highlights, which is when you have a lambda inside another lambda and you don't want to accidentally have the inner lambda also compile with the outer one. When the outer one is compiled, you want the inner lambda expression to remain an expression tree. There, yes, it makes sense quoting the inner lambda expression.
But that's about it, I believe. Is there any other use case for quoting a lambda expression?
And if there isn't, why do all the LINQ operators, i.e. the extensions on IQueryable<T> that are declared in the Queryable class quote the predicates or lambdas they receive as arguments when they package that information in the MethodCallExpression.
I tried an example (and a few others over the last couple of days) and it doesn't seem to make any sense to quote a lambda in this case.
Here's a method call expression to a method that expects a lambda expression (and not a delegate instance) as its only parameter.
I then compile the MethodCallExpression by wrapping it inside a lambda.
But that doesn't compile the inner LambdaExpression (the argument to the GimmeExpression method) as well. It leaves the inner lambda expression as an expression tree and does not make a delegate instance of it.
In fact, it works well without quoting it.
And if I do quote the argument, it breaks and gives me an error indicating that I am passing in the wrong type of argument to the GimmeExpression method.
What's the deal? What's this quoting all about?
private static void TestMethodCallCompilation()
{
var methodInfo = typeof(Program).GetMethod("GimmeExpression",
BindingFlags.NonPublic | BindingFlags.Static);
var lambdaExpression = Expression.Lambda<Func<bool>>(Expression.Constant(true));
var methodCallExpression = Expression.Call(null, methodInfo, lambdaExpression);
var wrapperLambda = Expression.Lambda(methodCallExpression);
wrapperLambda.Compile().DynamicInvoke();
}
private static void GimmeExpression(Expression<Func<bool>> exp)
{
Console.WriteLine(exp.GetType());
Console.WriteLine("Compiling and executing expression...");
Console.WriteLine(exp.Compile().Invoke());
}
You have to pass the argument as a ConstantExpression:
private static void TestMethodCallCompilation()
{
var methodInfo = typeof(Program).GetMethod("GimmeExpression",
BindingFlags.NonPublic | BindingFlags.Static);
var lambdaExpression = Expression.Lambda<Func<bool>>(Expression.Constant(true));
var methodCallExpression =
Expression.Call(null, methodInfo, Expression.Constant(lambdaExpression));
var wrapperLambda = Expression.Lambda(methodCallExpression);
wrapperLambda.Compile().DynamicInvoke();
}
private static void GimmeExpression(Expression<Func<bool>> exp)
{
Console.WriteLine(exp.GetType());
Console.WriteLine("Compiling and executing expression...");
Console.WriteLine(exp.Compile().Invoke());
}
The reason should be pretty obvious - you're passing a constant value, so it has to be a ConstantExpression. By passing the expression directly, you're explicitly saying "and get the value of exp from this complicated expression tree". And since that expression tree doesn't actually return a value of Expression<Func<bool>>, you get an error.
The way IQueryable works doesn't really have much to do with this. The extension methods on IQueryable have to preserve all information about the expressions - including the types and references of the ParameterExpressions and similar. This is because they don't actually do anything - they just build the expression tree. The real work happens when you call queryable.Provider.Execute(expression). Basically, this is how the polymorphism is preserved even though we're doing composition, rather than inheritance (/interface implementation). But it does mean that the IQueryable extension methods themselves cannot do any shortcuts - they don't know anything about the way the IQueryProvider is actually going to interpret the query, so they can't throw anything away.
The most important benefit you get from this, though, is that you can compose the queries and subqueries. Consider a query like this:
from item in dataSource
where item.SomeRelatedItem.Where(subItem => subItem.SomeValue == 42).Count() > 2
select item;
Now, this is translated to something like this:
dataSource.Where(item => item.SomeRelatedItem.Where(subItem => subItem.SomeValue == 42).Count() > 2);
The outer query is pretty obvious - we'll get a Where with the given predicate. The inner query, however, is actually going to be a Call to Where, taking the actual predicate as an argument.
By making sure that actual invocations of the Where method are actually translated into a Call of the Where method, both of these cases become the same, and your LINQProvider is that one bit simpler :)
I've actually written LINQ providers that don't implement IQueryable, and which actually have some useful logic in the methods like Where. It's a lot simpler and more efficient, but has the drawback described above - the only way to handle subqueries would be to manually Invoke the Call expressions to get the "real" predicate expression. Yikes - that's quite an overhead for a simple LINQ query!
And of course, it helps you compose different queryable providers, although I haven't actually seen (m)any examples of using two completely different providers in a single query.
As for the difference between Expression.Constant and Expression.Quote themselves, they seem rather similar. The crucial difference is that Expression.Constant will treat any closures as actual constants, rather than closures. Expression.Quote on the other hand, will preserve the "closure-ness" of the closures. Why? Because the closure objects themselves are also passed as Expression.Constant :) And since IQueryable trees are doing lambdas of lambdas of lambdas of [...], you really don't want to lose the closure semantics at any point.
I'm working on LINQ to XML queries and have used anonymous functions as well as lambda expressions. A quick example would be the select method over IEnumerables.
I understand that LINQ queries are deferred execution, which is somewhat similar to the concept lazy evaluation, but this question came to mind when VS2012's quick watch cannot handle statements with lambda expressions.
Are Lambda Expressions type-safe in C#?
I couldn't find a direct answer to this, or maybe it's because I do not fully understand type safety. I know OCaml and Java is type safe and Python is weakly typed, another way I can think of this is if the language is type safe, then lambda expressions within that language are no special. There is ambiguity in strong/weak typing but here I refer to it as if lambda expressions with erroneous types will pass through the compiler and allowed to execute at runtime. If errors exist that throw exceptions are they only caught at run-time?
When are they checked? Compile-time or Run-time
As an example, OCaml types are checked at compile time, and will not execute until the types are resolved. Whereas Python is less strict and is a dynamic language, in which it will compile and execute even with type error, only catching the errors at run time. How does C# handle lambda expressions in this sense?
Some related research I've done before asking this question:
How are Java lambdas compiled
This blog posts says LINQ is type-safe
Tutorial on using lambda expressions from CodeProject
Difference between C# Anonymous functions and Lambda Expressions
In C# exist two types of Lambda Expression:
A lambda expression is an anonymous function that you can use to create delegates or expression tree types.
The fist type of lambda expression is synctatic sugar for an anonymous function:
Func<int, int> myFunc = x => x + 1;
is totally equivalent to:
Func<int, int> myFunc = delegate(int x) { return x + 1; };
so it is clearly type safe, because it is C# code with a different makeup.
The other type of Lambda Expression is the one that generates Expression Trees:
Expression<Func<int, int>> myFunc = x => x + 1;
This is something different. This isn't compiled to "code" but is compiled to some object of type Expression that "describe" the x => x + 1 (and even describe the type of delegate)... it is compiled to:
ParameterExpression par = Expression.Parameter(typeof(int), "x");
Expression<Func<int, int>> myFunc2 = Expression.Lambda<Func<int, int>>(
Expression.Add(par, Expression.Constant(1)),
par);
Now, this code can't be executed directly. It can be converted to executable code through the .Compile() method. In general a .Compile()d expression tree is type safe, but Expression Trees aren't normally made to be simply compiled. Programs tend to manipulate them to obtain funny result. They can be used for various tasks... For example to extract the "name" of properties or "methods" without including in the code a string with the name of the property or method, or to be converted to other languages (Entity Framework/LinqToSQL convert expression trees to SQL). An Expression Tree is quite safe (it is possible to "manually build" at runtime an invalid expression, but when you do the .Compile() you'll get an exception, and expression trees accepted by the C# compiler are normally safe to be compiled), but if the expression tree is used for other things, then errors could occur, even errors connected to type safety.
I'll quote from: Why the anonymous type instance cannot accept null values returned by the entity framework query?
var l = (from s in db.Samples
let action = db.Actions.Where(x => s.SampleID == x.SampleID && x.ActionTypeID == 1).FirstOrDefault()
where s.SampleID == sampleID
select new
{
SampleID = s.SampleID,
SampleDate = action.ActionDate,
}).ToList();
Equivalent more or less to
var l = db.Samples.Select(s => new
{
s = s,
action = db.Actions.Where(x => s.SampleID == x.SampleID && x.ActionTypeID == 1).FirstOrDefault()
}).Where(x => x.s.SampleID == sampleID).Select(x => new
{
SampleID = x.s.SampleID,
SampleDate = x.action.ActionDate
}).ToList();
Here ActionDate is DateTime, and so is SampleDate. This LINQ expression will be transformed by the compiler to a big Lambda Expression, and executed by Entity Framework SQL Server-side. Now... the problem is that action could become null, and so action.ActionDate could be null (because the expression won't be executed locally there won't be a NullReferenceException), and an exception could be thrown (will be thrown) when null is put in SampleDate (an InvalidCastException I think). So while the expression is type-safe, what the library does with it causes the expression to generate non-type-safe code (an invalid cast)
Lambdas have exactly as much static type checking as any other C# code does. It's built on the same type system, and enforces all of the same compile time type checks. You can of course turn off static type checks (by, say, performing a cast) in a lambda in just the same way that you can in any other C# code.
If a lambda is complied into executable code (instead of, say, an Expression) and is run, the exact same runtime checks will be performed as if you weren't using a lambada.
In fact, if you're using lambdas compiled into executable code, it will simply be transformed into a new named method, even though it is anonymous in your original code, in one of the earlier passes of the compiler. Once transformed into a regular named method, it then goes through all of the same type checking any other code would.
Imagine that you could write a class that is something like this:
public class Foo {
public Baz DoSomething(Bar b)
{
return new Baz(b);
}
}
Clearly this is strongly typed at compile time. So now I could make a delegate declaration that is something like this:
public delegate Baz SomeDelegate(Bar b);
and then I could modify Foo and add a property:
...
public SomeDelegate MyCall { get { return DoSomething; } }
...
You need to ask youself how is it different to do this:
Bar b = new Bar();
Foo aFoo = new Foo();
var myDelegate = aFoo.MyCall;
Baz baz = myDelegate(b);
And
Bar b = new Bar();
var myDelegate = (Bar bar) => new Baz(bar);
Baz baz = myDelegate(b);
Because what happens under the hood is pretty darn close to this. A lambda expression can be implemented by creating an anonymous class with a method in it. (FWIW, before there were lambda expression in Java, I would often simulate them by using a static private inner class). Semantically, it's more complicated than this because of variables that are free/bound and how to handle that morass gracefully (hint: Java doesn't handle it), but ultimately, lambda expression in C# are syntactic sugar to give you a delegate defined inline without about as much type inference as C# can handle and delegates are strongly typed.
I'm starting to look under the LINQ hood a little more, and I'm having trouble understanding some of the LINQ extension method overloads.
For example, lets say I'm querying a DbContext using .Where(). I'm always passing a standard Func<>, as opposed to an Expression<> of said Func<>. Example query below:
var db = new MyContext();
var foo = db.products.Where(p => p.Category == "books");
Here is where I'm confused. When I look at the available method signatures, I would assume the overload I'm using above would be returning me an IEnumerable...but it's actually returning an IQueryable. How is this possible, if the IQueryable overload is expecting an Expression as opposed to just a Func? It feels like the compiler is somehow helping me out by (in this case) building the Expression for me, but I can't find a resource that explains if this is the case. Thanks!
It feels like the compiler is somehow helping me out by (in this case) building the Expression for me
That is exactly what is happening. The compiler can interpret lambdas (as long as they don't have a "statement body") as either delegates or expression trees. The Queryable.Where<T>(this IQuerayble<T>, ...) extension method takes precedence over the Enumerable.Where<T>(this IEnumerable<T>, ...) extension method, so the compiler chooses to interpret your predicate as an expression tree.
It feels like the compiler is somehow helping me out by (in this case) building the Expression for me
Correct, the compiler is compiling the lambda into an Expression rather than a Func.
I can't find a resource that explains if this is the case
From MSDN:
When a lambda expression is assigned to a variable of type Expression<TDelegate>, the compiler emits code to build an expression tree that represents the lambda expression.
You're not passing a Func in that example. You're passing an Expression. A lambda can compile down into either a delegate or an expression, based on the context around it. In this case the context around it is expecting an expression, so that is what the lambda compiles into. If you actually did pass in a Func and not a lambda (which could be either) then you would get an IEnumerable for your result, not an IQueryable.
The compiler can automatically build an expression tree for you from a lamba expression, and that is what's happening here.
http://msdn.microsoft.com/en-gb/library/bb397951.aspx
I found the following question that can combine multiple Expression<Func<T,bool>> expressions:
How to merge two C# Lambda Expressions without an invoke?
I'm wondering whether, using a similar technique, how you go about merging a .OrderBy Expression<Func<Entity, Key>> with a .Where Expression<Func<Entity, bool>> into one Expression of type, or inheriting from type, System.Linq.Expressions.Expression.
I'm making a really cut down QueryProvider-style class for taking T => T == ... Func's via public methods .Where and .OrderBy. This is with the intention that the expression this class builds gets passed to a QueryTranslator to create suitable SQL.
A QueryTranslator of this style, when called from a QueryProvider, typically takes one System.Linq.Expressions.Expression as an argument which it then translates to SQL.
I can follow through and understand the question above for merging two .Where Funcs. The following blog post was particularly useful for this, and I traced through each of the examples:
http://blogs.msdn.com/b/meek/archive/2008/05/02/linq-to-entities-combining-predicates.aspx
This CodeProject article was also useful:
http://www.codeproject.com/Articles/24255/Exploring-Lambda-Expression-in-C
When combining a .OrderBy Func with a .Where Func however the two have different generic arguments. In the case of the .Where it's a Func<Entity, bool>, and in the case of the .OrderBy it's a Func<Entity, Key>.
Merging these two Expressions, on the surface, isn't as straight forward as merging two with the same generic arguments.
The in-built Func-Expression-producing-engine (unsure of exact term) is able to combine a .Where Func with a .OrderBy Func. I'm curious about what goes on under the hood when these two expressions are merged to become one System.Linq.Expressions.Expression. Is it possible, and if so how would you go about combining an Expression<Func<Entity, bool>> with an Expression<Func<Entity,Key>> assuming the Entity is of the same type in each Expression?
public IQueryable<T> ApplyExpressions<T, TKey>(
Expression<Func<T, TKey>> sortBy,
Expression<Func<T, bool>> filterBy,
IQueryable<T> source)
{
return source.Where(filterBy).OrderBy(sortBy);
}
IQueryable<Customer> query = ApplyExpressions<Customer, int>(
c => c.Age,
c => c.Name.StartsWith("B"),
myDC.Customers)
Expression queryExpression = query.Expression; //tada
I think where this question got ambiguous is because it's kind of unanswerable.
I was trying to build one super-expression, like how the IQueryable<> expression builder works with .Where and .OrderBy expressions all in one expression.
Without a subject, however, my limited understanding of Expressions and Expression Trees seems to suggest it's not possible.
To combine a .Where expression with a .OrderBy expression into one Expression you need MethodCallExpression's to separate the two LambdaExpressions.
The MethodCallExpression needs a subject to call the Method on. This is where the IQueryable<> gets used.
I've altered my Query class to keep all the LambdaExpression's separate, and these then get passed to the QueryTranslator separately. The QT merges the output SQL into the right places.
This is as opposed to an IQueryable<> QueryProvider that analyses one super-Expression and outputs SQL into the right places based on MethodCallExpression's in the Expression.
Could somebody explain the following declaration in a way that conveys the meaning of the expression and how it would be called?
void Delete<T>(Expression<Func<T, bool>> expression) where T : class, new();
I read it as:
Delete an object of type T, by passing in a lambda expression whose parameter is an object of type T that returns a bool.
Also, can you replace
Func<T, bool> expression
with
Predicate<T> expression
This method is probably a member of a collection type, yes?
A "predicate" is any device that says "yes" or "no" to the question "is this thing a member of that set?" So a predicate for the set "integers even positive integers" would be x=> x > 0 && x % 2 == 0.
This method probably has the semantics of "delete from the collection all members of the collection that are in the set identified by the predicate".
The predicate is passed to the method in the form of an expression tree, which is a way of passing the structure of the predicate in a manner that can be analyzed at runtime and transformed. It is typically used in scenarios where the "collection" is actually a database somewhere, and the deletion request needs to be translated into a query in the database's query language and sent over the network.
The first is a method which accepts an expression tree (not necessarily created from a lambda expression tree). The expression tree represents an expression which accepts a T and returns a bool. T is constrained to be a reference type with a parameterless constructor.
As for the semantic meaning - that's up to the documentation/implementation.
It's important to distinguish between a lambda expression, which is one way of creating an expression tree, and an expression tree itself.
As for whether it could use Predicate<T> instead - maybe. It depends on what the implementation does with it. They represent the same delegate signature, certainly - but you can't convert between the two types of expression tree trivially.
this methods gets as a parameter expression tree of function that gets object with public parameter-less constructor and returns boolean.
you can read more about expression trees and their usage here:
http://msdn.microsoft.com/en-us/library/bb397951.aspx
While the method signature looks invalid to me, essentially you are passing in an expression tree (it might not be a LambdaExpression type as Expression is the abstract base class for all expression types).
The type constraints state that T must be a reference type (inherit from a class, cannot be a value type (read: struct)) and must also have a default constructor defined.
EDIT: see Jon's answer below, he corrected the signature and answered the question correctly from there, providing more information than I.