Related
I am trying to create a generic way of getting an EntityFramework object based on its own id without passing in a lambda expression as parameter to the method GetById(). For the code below the entity T is of type Message, is known to the class where GetById() is implemented and has a property MessageId along with several other properties. The MessageId name has been hard-coded in the example below as this is still experimental - extracting the id property name from T is quite easy to fix later.
I have been struggling to find a way to construct a simple LambdaExpression which has IQueryable<T> as parameter type and hope that someone would have a clue on how this could be done. The reason why I want IQueryable<T> is because my underlying channel factory provider requires this for more complex queries.
The line with var exp = Expression.Lambda<...> in the code below shows the expression function type definition which I want to end up with, but the line gives the exception:
Expression of type System.Boolean cannot be used for return type IQueryable
That's because the body has the Boolean type while my expression parameter queryParamtRet is of type IQueryable<Message>. Further, if I change the body type to be an IQueryable<Message>, I'm not able to find the property MessageId since the type is no longer type T as Message but type IQueryable<T>.
public T GetById(int id)
{
var queryParamLeft = Expression
.Parameter(typeof(System.Data.Entity.DbSet<T>), "o");
var queryParamRet = Expression
.Parameter(typeof(IQueryable<T>), "o");
var entityFrameworkType = Expression
.Parameter(typeof(T), "o");
var queryProperty = Expression
.PropertyOrField(entityFrameworkType, "MessageId");
var body = Expression
.Equal(queryProperty, Expression.Constant(id));
var exp = Expression
.Lambda<Func<System.Data.Entity.DbSet<T>, IQueryable<T>>>(
body,
queryParamRet);
var returnXml = DoWithChannel(channel
=> channel.Load(serializer.Serialize(exp)));
}
TLDR: Write out code that you want to create an expression for, then deliberately create the expression, starting with any inner expressions before combining them into the outer expression.
If you write your intended code as a function, it would look something like this
public static IQueryable<T> FilterADbSet(DbSet<T> dbSet)
{
return Queryable.Where<T>(dbSet, o => o.MessageId == 34);
}
It has one input parameter of type DbSet<T>, an output of type IQueryable<T> and it calls Queryable.Where<T> with parameters of the dbSet variable and an expression.
Working from the outside in, you first need to build the expression to pass to the where clause. You have already done that in your code.
Next you need to create a lambda expression for the where clause.
var whereClause = Expression.Equal(queryProperty, Expression.Constant(id));
var whereClauseLambda = Expression.Lambda<Func<T, bool>>(whereClause, entityFrameworkType);
Next, as the comments indicate, you need to use Expression.Call to create a body.
My end result with making your code work is below.
static Expression<Func<IQueryable<T>, IQueryable<T>>> WhereMethodExpression = v => v.Where(z => true);
static MethodInfo WhereMethod = ((MethodCallExpression)WhereMethodExpression.Body).Method;
public T GetById(int id)
{
var queryParamLeft = Expression
.Parameter(typeof(System.Data.Entity.DbSet<T>), "dbSet");
var entityFrameworkType = Expression
.Parameter(typeof(T), "entity");
var queryProperty = Expression
.PropertyOrField(entityFrameworkType, "MessageId");
var whereClause = Expression
.Equal(queryProperty, Expression.Constant(id));
var whereClauseLambda = Expression.Lambda<Func<T, bool>>(whereClause, entityFrameworkType);
var body = Expression.Call(
WhereMethod,
queryParamLeft,
whereClauseLambda
);
var exp = Expression
.Lambda<Func<System.Data.Entity.DbSet<T>, IQueryable<T>>>(
body,
queryParamLeft);
var returnXml = DoWithChannel(channel
=> channel.Load(serializer.Serialize(exp)));
}
I used an expression to fetch the MethodInfo object of Queryable.Where<T>
Your body expression needed queryParamLeft passed in. queryParamRet is not needed
I'm having a go at making a flexible exporter to export info from a SQL db accessed via LINQ to SQL, whereby the program can dynamically choose which fields to select and do all the processing server side.
The final aim is to have a simple statement like:
var result = db.Product.Select(p => selector(p));
Where selector is a dynamically created Expression Tree describing the fields to select. Currently, I have each of the db fields assigned to it's own individual selector like:
Expression<Func<Product, string>> name = p => p.Name;
Expression<Func<Product, string>> createdAt = p => p.createdAt.ToString();
At the moment, I'm taking the chosen fields and trying to make one concatenated expression out of them that returns a comma delimited string result from the select statement, something like:
Expression<
Func<
Func<Product, string>,
Func<Product, string>,
Func<Product, string>>> combinator = (a, b) => (p) => a(p) + "," + b(p);
// So I can do something like...
Expression<Func<Product, string>> aggregate = p => "";
foreach (var field in fieldsToSelect)
aggregate = combinator(aggregate, field);
// This would create...
Expression<Func<Products, string>> aggregate = p => a(p) + "," + b(p);
Once I've built up my selector with however many fields, I can execute it in the statement and all the processing is done on the server. However, I've been unable to properly create an expression to concatenate the two child functions in such a manner that the result isn't a Func that's simply executed after the round trip to fetch the results:
var joinedSelector = combinator.Compile();
Func<Products, string> result = joinedSelector(firstSelector.Compile(), secondSelector.Compile());
var query = db.Product.Select(p => result(p)).ToList();
From my limited understanding of Expression Trees, this doesn't actually result in one as the statements are compiled to normal Funcs. I've looked at Expression.Coalesce() but am not sure if it's what I'm after (think it just does "??").
I'm pretty new to this sort of stuff, and would appreciate any help.
Even if people can think of better ways to attack the original problem, I'd still quite like an explanation of how to achieve what I'm trying to do just for the sake of learning how to use Expression Trees.
So you're looking to create a Combine method that can combine the results of two selectors. To do this we'll need a method that accepts two functions with the same input and same output, and then a function accepting two instances of that output type and returning a new value.
The function will need to replace all instances of the parameters of the selectors' body with a common parameter. It will then replace the two parameters of the result function with the corresponding bodies of the different selectors. Then we just wrap all of that up in a lambda.
public static Expression<Func<T, TResult>> Combine
<T, TIntermediate1, TIntermediate2, TResult>(
this Expression<Func<T, TIntermediate1>> first,
Expression<Func<T, TIntermediate2>> second,
Expression<Func<TIntermediate1, TIntermediate2, TResult>> resultSelector)
{
var param = Expression.Parameter(typeof(T));
var body = resultSelector.Body.Replace(
resultSelector.Parameters[0],
first.Body.Replace(first.Parameters[0], param))
.Replace(
resultSelector.Parameters[1],
second.Body.Replace(second.Parameters[0], param));
return Expression.Lambda<Func<T, TResult>>(body, param);
}
This uses the following method to replace all instances of one expression with another:
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
Now we can write the following:
Expression<Func<Product, string>> name = p => p.Name;
Expression<Func<Product, string>> createdAt = p => p.createdAt.ToString();
Expression<Func<Product, string>> aggregate =
Combine(name, createdAt, (a, b) => a + "," + b);
It actually comes out a bit simpler than in your mockup, as the result selector doesn't need to know anything about how its inputs are generated, or that they're actually selectors. This solution also allows for each selector to select out different types, and for them to differ from the result, simply because there's no real cost to adding all of this when the generic arguments are just going to be inferred.
And with this in place you can even easily aggregate an arbitrary number of selectors, given the type restrictions you've put in place:
IEnumerable<Expression<Func<Product, string>>> selectors = new[]{
name,
createdAt,
name,
};
var finalSelector = selectors.Aggregate((first, second) =>
Combine(first, second, (a, b) => a + "," + b));
This would let you, for example, have a params method accepting any number of selectors (of a common input and output type) and be able to aggregate all of their results together.
I need to combine two lambda expressions, in a way that the second expression "contains" the first.
I searched a lot but did not find any clear answer...
What i am trying to do is the following :
A first expression "expression1" is passed as a parameter to a method, and is simply used to define on which field or property the second lambda must operate.
Schematically, I am trying to do the following :
// simple field selector :
Expression<Func<T, string>> expression1 = obj => obj.field;
// trying to use the field selector in 2nd expression :
Expression<Func<T, bool>> e2 = obj => [[Result of expression1]].Equals("myValue");
In other words, I would like to get :
Expression<Func<T, bool>> e2 = obj => obj.field.Equals("myValue");
I need to do it this way because it is not always the Equals() method that will be called, but many different methods.
I tried to compile expression1 to a Func<T, string> in order to invoke this Func in expression2, but as I am using this with LinQ, I get an exception because LinQ does not support Invoke node types.
So my question is : is there a way to just combine the bodies of the two expressions, and how please ?
Thanks by advance !
Well, this should do the trick:
void Main()
{
var execute = Create<TestClass>(
first => first.MyStringField, // field selector
second => second.Equals("1234") // method selector
);
Console.WriteLine(execute(new TestClass("1234"))); // true
Console.WriteLine(execute(new TestClass("4321"))); // false
}
class TestClass
{
public string MyStringField;
public TestClass(string val){
MyStringField = val;
}
}
static Func<TSource, bool> Create<TSource>(
Expression<Func<TSource, string>> fieldSelector,
Expression<Func<string, bool>> methodSelector
)
{
// todo: classical validation of fieldSelector, if necessary.
// todo: classical validation of methodSelector, if necessary.
var compiledFieldSelector = fieldSelector.Compile();
var compiledMethodSelector = methodSelector.Compile();
return T => compiledMethodSelector(compiledFieldSelector(T));
}
Note, due the nature of lambda expressions, you need to validate the field selector and method selector, otherwise it's possible to do some very weird things.
Alternatively, the next approach creates lambda that does not "compose" things, in the sense, this is better since LinqToEntities shouldn't have problems interpreting the query.
static Func<TSource, bool> Create<TSource>(
Expression<Func<TSource, string>> memberSelector,
Expression<Func<string, bool>> methodSelector
)
{
// todo: classical validation of memberSelector, if necessary.
// todo: classical validation of methodSelector, if necessary.
var memberExpression = (MemberExpression)(memberSelector.Body);
var methodCallExpression = (MethodCallExpression)(methodSelector.Body);
// input TSource => ..
var input = Expression.Parameter(typeof(TSource));
var call = Expression.Call(
Expression.MakeMemberAccess(
input,
memberExpression.Member),
methodCallExpression.Method,
methodCallExpression.Arguments);
return Expression.Lambda<Func<TSource, bool>>(call, input)
.Compile();
}
Chris Eelmaa's 2nd answer is ok, just remove the .Compile() call to avoid having the Exception with Invoke :
static Expression<Func<TSource, bool>> Create<TSource>(
Expression<Func<TSource, string>> memberSelector,
Expression<Func<string, bool>> methodSelector
)
{
// todo: classical validation of memberSelector, if necessary.
// todo: classical validation of methodSelector, if necessary.
var memberExpression = (MemberExpression)(memberSelector.Body);
var methodCallExpression = (MethodCallExpression)(methodSelector.Body);
// input TSource => ..
var input = Expression.Parameter(typeof(TSource));
var call = Expression.Call(
Expression.MakeMemberAccess(
input,
memberExpression.Member),
methodCallExpression.Method,
methodCallExpression.Arguments);
return Expression.Lambda<Func<TSource, bool>>(call, input);
}
In my case this is then used like this :
(selector is an expression like u => u.id, and
request is a IQueryable<T>, both received as arguments)
Expression<Func<T, bool>> contains = Create<T>(selector, u => u.Contains(searchExpression));
IQueryable<T> result = request.Where(contains);
Does anyone have/know of an IQueryable.OrderBy extension that takes an Expression (retrieved, for example, by Reflection)? I believe the function would look something like this:
public static IQueryable<TEntity> OrderBy<TEntity>
(this IQueryable<TEntity> source, Expression sortExpression)
Expression would be assumed to be an Expression<Func<TEntity, T>> where TEntity is the same object being sorted on, and T is a type that needs to be determined in order to create the new IQueryable.
I've found many examples of extensions that take a string, including Dynamic Linq, like this:
public static IQueryable<TEntity> OrderBy<TEntity>(
this IQueryable<TEntity> source, string sortExpression)
If it's possible to take the string and use Reflection to look up the type from the object in question, it should also be possible to take the Expression, and get the value type which is right there IN the Expression.
Following is a detailed explanation of why I'd like to have this, which you may or may not need.
I have a rather large list of complex records to sort. Because the list is so long, I prefer to have the sorting done on the database side. To handle more complex properties, I've created Expressions that provide the sorting functionality, like so:
if (model.sortExpression == "PlannedValue")
{
Expression<Func<BDopp, decimal>> sorter = BDopp.PlannedValueSorter;
if (model.sortDirection == "DESC")
opps = opps.OrderByDescending(sorter).AsQueryable();
else
opps = opps.OrderBy(sorter).AsQueryable();
}
BDOpp.PlannedValueSorter retrieves a static expression from the object which allows sorting to be done without opps are still of type IQueryable:
public static Expression<Func<BDopp, decimal>> PlannedValueSorter
{
get
{
return z => z.BudgetSchedules
.Where(s => s.Type == 1)
.Sum(s => s.Value * s.Workshare * z.valueFactor / 100 / 100);
}
}
Sorting for simple properties is done with Extension methods that use Reflection to build an expression based on the name of the property passed as a string.
This works well, but for the complex types, I still need branching logic, and I'd rather not do that. What I'd rather do is check for a static property containing the expression, and then simply apply it. I can get the expression like this:
PropertyInfo info = typeof(BDopp).GetProperty(model.sortExpression + "Sorter",
BindingFlags.Static | BindingFlags.Public);
Expression expr = (Expression)info.GetValue(null, null);
For the PlannedValue property, this gets me the expression sorted in PlannedValueSorter, which I already know works.
Update:
Various digging around has gotten me what I think might be some progress:
public static IQueryable<TEntity> OrderBy<TEntity>(this IQueryable<TEntity> source,
Expression<Func<TEntity, dynamic>> sortExpression)
{
var unary = sortExpression.Body as UnaryExpression;
Type actualExpressionType = unary.Operand.Type;
actualExpressionType is in fact the return type of the Expression (for this particular property, it's decimal).
Unfortunately I'm mostly just working by trial and error, since I haven't yet wrapped my brain around how all this works, so my attempt to update the query like so is not working:
MethodCallExpression resultExp = Expression.Call(typeof(Queryable),
"OrderBy",
new Type[] { typeof(TEntity), actualExpressionType },
source.Expression, sortExpression);
return source.Provider.CreateQuery<TEntity>(resultExp);
It compiles okay, but the following error is thrown at runtime:
No generic method 'OrderBy' on type 'System.Linq.Queryable' is
compatible with the supplied type arguments and arguments. No type
arguments should be provided if the method is non-generic.
As I understand it you have an Expression and want to order/filter by it. It might surprise you how simple it is:
Expression<Func<TEntity, T>> sortExpr = ...; //you already have this
var ordered = myQuery.OrderBy(sortExpr);
OrderBy always uses an Expression so you can directly use it. If you only have a variable of type Expression (which points to an object of type Expression<Func<TEntity, T>>) and don't know statically what T is, you can do this:
Expression sortExpr = ...; //you already have this
var ordered = myQuery.OrderBy((dynamic)sortExpr);
dynamic will figure this out at runtime using reflection. No need to do reflection yourself. All that is needed here is overload resolution (performed by the C# runtime binder).
Okay, I've got a solution:
public static IQueryable<TEntity> OrderBy<TEntity>(
this IQueryable<TEntity> source,
Expression<Func<TEntity, dynamic>> sortExpression,
bool descending)
{
var unary = sortExpression.Body as UnaryExpression;
var operand = unary.Operand;
Type actualExpressionType = operand.Type;
MethodCallExpression resultExp =
Expression.Call(typeof(Queryable),
descending? "OrderByDescending" : "OrderBy",
new Type[] { typeof(TEntity), actualExpressionType },
source.Expression,
Expression.Lambda(operand, sortExpression.Parameters));
return source.Provider.CreateQuery<TEntity>(resultExp);
}
The bool descending is to allow for the standard OrderBy and OrderByDescending overloads. Two major breakthroughs here, at least for me:
Getting the Operand out of the expression.
Using Expression.Call and Expression.Lambda to create the new expression - this allows me to use an actual "Type" variable, whereas the Expression<Func<TEntity, T>> syntax requires you to use a type that's known at compile time.
Try this:
public static IEnumerable<T> OrderBy<T>(
this IEnumerable<T> collection,
string columnName
//, SortDirection direction
)
{
ParameterExpression param = Expression.Parameter(typeof(T), "x"); // x
Expression property = Expression.Property(param, columnName); // x.ColumnName
Func<T, object> lambda = Expression.Lambda<Func<T, object>>( // x => x.ColumnName
Expression.Convert(property, typeof(object)),
param)
.Compile();
Func<IEnumerable<T>, Func<T, object>, IEnumerable<T>> expression = (c, f) => c.OrderBy(f); // here you can use OrderByDescending basing on SortDirection
IEnumerable<T> sorted = expression(collection, lambda);
return sorted;
}
Usage:
IEnumerable<T> collection = ...
IEnumerable<T> ordered = collection.OrderBy("PropName");
See Code Project and sandbox.
How can I build an expression tree when parts of the expression are passed as arguments?
E.g. what if I wanted to create expression trees like these:
IQueryable<LxUser> test1(IQueryable<LxUser> query, string foo, string bar)
{
query=query.Where(x => x.Foo.StartsWith(foo));
return query.Where(x => x.Bar.StartsWith(bar));
}
but by creating them indirectly:
IQueryable<LxUser> test2(IQueryable<LxUser> query, string foo, string bar)
{
query=testAdd(query, x => x.Foo, foo);
return testAdd(query, x => x.Bar, bar);
}
IQueryable<T> testAdd<T>(IQueryable<T> query,
Expression<Func<T, string>> select, string find)
{
// how can I combine the select expression with StartsWith?
return query.Where(x => select(x) .. y => y.StartsWith(find));
}
Result:
While the samples didn't make much sense (sorry but I was trying to keep it simple), here's the result (thanks Quartermeister).
It can be used with Linq-to-Sql to search for a string that starts-with or is equal to the findText.
public static IQueryable<T> WhereLikeOrExact<T>(IQueryable<T> query,
Expression<Func<T, string>> selectField, string findText)
{
Expression<Func<string, bool>> find;
if (string.IsNullOrEmpty(findText) || findText=="*") return query;
if (findText.EndsWith("*"))
find=x => x.StartsWith(findText.Substring(0, findText.Length-1));
else
find=x => x==findText;
var p=Expression.Parameter(typeof(T), null);
var xpr=Expression.Invoke(find, Expression.Invoke(selectField, p));
return query.Where(Expression.Lambda<Func<T, bool>>(xpr, p));
}
e.g.
var query=context.User;
query=WhereLikeOrExact(query, x => x.FirstName, find.FirstName);
query=WhereLikeOrExact(query, x => x.LastName, find.LastName);
You can use Expression.Invoke to create an expression that represents applying one expression to another, and Expression.Lambda to create a new lambda expression for the combined expression. Something like this:
IQueryable<T> testAdd<T>(IQueryable<T> query,
Expression<Func<T, string>> select, string find)
{
Expression<Func<string, bool>> startsWith = y => y.StartsWith(find);
var parameter = Expression.Parameter(typeof(T), null);
return query.Where(
Expression.Lambda<Func<T, bool>>(
Expression.Invoke(
startsWith,
Expression.Invoke(select, parameter)),
parameter));
}
The inner Expression.Invoke represents the expression select(x) and the outer one represents calling y => y.StartsWith(find) on the value returned by select(x).
You could also use Expression.Call to represent the call to StartsWith without using a second lambda:
IQueryable<T> testAdd<T>(IQueryable<T> query,
Expression<Func<T, string>> select, string find)
{
var parameter = Expression.Parameter(typeof(T), null);
return query.Where(
Expression.Lambda<Func<T, bool>>(
Expression.Call(
Expression.Invoke(select, parameter),
"StartsWith",
null,
Expression.Constant(find)),
parameter));
}
This Works:
public IQueryable<T> Add<T>(IQueryable<T> query, Expression<Func<T, string>> Selector1,
Expression<Func<T, string>> Selector2, string data1, string data2)
{
return Add(Add(query, Selector1, data1), Selector2, data2);
}
public IQueryable<T> Add<T>(IQueryable<T> query, Expression<Func<T, string>> Selector, string data)
{
var row = Expression.Parameter(typeof(T), "row");
var expression =
Expression.Call(
Expression.Invoke(Selector, row),
"StartsWith", null, Expression.Constant(data, typeof(string))
);
var lambda = Expression.Lambda<Func<T, bool>>(expression, row);
return query.Where(lambda);
}
You use it like:
IQueryable<XlUser> query = SomehowInitializeIt();
query = Add(query, x => x.Foo, y => y.Bar, "Foo", "Bar");
Usually you don't do that in the way you descirbed (using the IQueryable Interface) but you rather use Expressions like Expression<Func<TResult, T>>. Having said that, you compose higher order functions (such as where or select) into a query and pass in expressions that will "fill in" the desired functionality.
For example, consider the signature of the Enumerable.Where method:
Where<TSource>(IEnumerable<TSource>, Func<TSource, Boolean>)
The function takes a delegate as its second argument that is called on each element. The value you return from that delegate indicates to the higher order function if it shall yield the current element (include it in the result or not).
Now, let's take a look at Queryable.Where:
Queryable.Where<TSource>-Methode (IQueryable<TSource>, Expression<Func<TSource, Boolean>>)
We can observe the same pattern of a higher order function, but instead of an Func<> delegate it takes an Expression. An expression is basically a data representation of your code. Compiling that expression will give you a real (executable) delegate. The compiler does a lot of heavy lifting to build expression trees from lambdas you assign to Expression<...>. Expression trees make it possible to compile the described code against different data sources, such as a SQL Server Database.
To come back to your example, what I think you're looking for is a selector. A selector takes each input element and returns a projection of it. It's signature looks like this: Expression<Func<TResult, T>>. For example you could specify this one:
Expression<Func<int, string>> numberFormatter = (i) => i.ToString(); // projects an int into a string
To pass in a selector, your code would need to look like this:
IQueryable<T> testAdd<T>(IQueryable<T> query, Expression<Func<string, T>> selector, string find)
{
// how can I combine the select expression with StartsWith?
return query.Select(selector) // IQueryable<string> now
.Where(x => x.StartsWith(find));
}
This selector would allow you to project the input string to the desired type. I hope I got your intention corrrectly, it's hard to see what you're trying to achieve.