We Keep Coding

LINQ: How to filter two expressions with OR, while both have conditions to include filter?

I know that if (includeFilter) query = query.Where (...) gives the same result as query = query.Where (n => !includeFilter || <expression>).
But what if I have <expression1> and <expression2> and I want query.Where (n => <expression1> || <expression2>)? Except that there are includeFilter1 and includeFilter2 too.
Simply query.Where (n => (!includeFilter1 || <expression1>) || (!includeFilter2 || <expression2>)) won't work, because in case includeFilter1 is false, <expression2> will have no effect.
Update:
I just got to this solution:
```query.Where(n => !includeFilter1 || ).Union(query.Where(n => !includeFilter2 || ))```
I'm going to use it with IQueryable; is it OK? Is there a better solution?
Update 2:
I was wrong. What I needed was this:
var query1 = query.Where(n => !includeFilter1 || <expression1>)
var query2 = query.Where(n => !includeFilter2 || <expression2>)
if (includeFilter1 && includeFilter2)
query = query1.Union(query2);
else if (includeFilter1)
query = query1;
else if (includeFilter2)
query = query2;
I couldn't come up with a more elegant way.

In general .Union() is best reserved for cases where you can't represent the logic of the filter in a single query, or when the data is being drawn from different sources in each part of the query. Each .Union() call most likely results in a separate query execution, which means the query optimizer has little to no chance to produce a reasonable execution plan.
Using .Where(n => !includeFilter || <expression>) is also less that optimal on the face of it. The expression tree will close over the includeFilter variable and may not be optimized particularly well depending on the ORM you're using. Best to avoid that where possible.
There are several options for this kind of query composition. The simplest, which works fine for a couple of entries, is this:
if (includeFilter1)
{
if (includeFilter2)
query = query.Where(q => <expression1> || <expression2>);
else
query = query.Where(q => <expression1>);
}
else if (includeFilter2)
query = query.Where(q => <expression2>);
Not very elegant perhaps, but it works just fine and only takes a few lines of code to implement... if <expression1> and <expression2> are known at compile time and aren't parameters to the method.
What I think you're looking for is a more generic way to handle filter expression composition. LINQ has some rather interesting expression handling and while expressions can be scary at first, once you get the hang of them you can do some quite cool things.
For instance, here's an extension (and a supporting class) that will take two predicates on the same type and join them with an Or:
using System.Linq.Expressions
public static class ExpressionExtensions
{
public static Expression<Func<T, bool>> Or<T>(this Expression<Func<T, bool>> left, Expression<Func<T, bool>> right)
{
// Useful for chained or conditional composition:
if (left is null)
return right;
if (right is null)
return left;
// Just in case:
if (ReferenceEquals(left, right))
return left;
// change right parameter references to match left
var rightBody = ReplaceVisitor.Replace(right.Body, right.Parameters[0], left.Parameters[0]);
// build 'or' expression
var body = Expression.OrElse(left.Body, rightBody);
// Create lambda (function) for predicate
var result = Expression.Lambda<Func<T, bool>>(body, left.Parameters[0]);
return result;
}
}
// Helper class, replaces expression instances
// Used to get the right parameters in composed expressions.
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression _from;
private readonly Expression _to;
private ReplaceVisitor(Expression from, Expression to)
{
_from = from;
_to = to;
}
public override Expression Visit(Expression e)
=> ReferenceEquals(e, _from) ? _to : base.Visit(e);
public static T Replace<T>(T expression, Expression from, Expression to)
where T : Expression
{
var visitor = new ReplaceVisitor(from, to);
return (T)visitor.Visit(expression);
}
}
(You can create a similar .And() extension using the AndAlso expression.)
Using that you can directly merge your filter expresions with an Or like so:
public IQueryable<T> ApplyFilters<T>(IQueryable<T> query,
bool useFilter1, Expression<Func<T, bool>> predicate1,
bool useFilter2, Expression<Func<T, bool>> predicate2
)
{
Expression<Func<T, bool>> predicate = null;
if (useFilter1)
predicate = predicate.Or(predicate1);
if (useFilter2)
predicate = predicate.Or(predicate2);
return predicate is null ? query : query.Where(predicate);
}
Or perhaps this:
public IQueryable<T> AnyOf<T>(IQueryable<T> query, params Expression<Func<T, bool>>[] filters)
{
Expression<Func<T, bool>> predicate = null;
foreach (var filter in filters)
predicate = predicate.Or(filter);
return predicate is null ? query : query.Where(predicate);
}
Try it out, see if it fits your use case.

I think you need
query.Where(n => (!includeFilter1 || <expression1>) &&
(!includeFilter2 || <expression2>))

Use lambda expression in another lambda expression

I need to combine two lambda expressions, in a way that the second expression "contains" the first.
I searched a lot but did not find any clear answer...
What i am trying to do is the following :
A first expression "expression1" is passed as a parameter to a method, and is simply used to define on which field or property the second lambda must operate.
Schematically, I am trying to do the following :
// simple field selector :
Expression<Func<T, string>> expression1 = obj => obj.field;
// trying to use the field selector in 2nd expression :
Expression<Func<T, bool>> e2 = obj => [[Result of expression1]].Equals("myValue");
In other words, I would like to get :
Expression<Func<T, bool>> e2 = obj => obj.field.Equals("myValue");
I need to do it this way because it is not always the Equals() method that will be called, but many different methods.
I tried to compile expression1 to a Func<T, string> in order to invoke this Func in expression2, but as I am using this with LinQ, I get an exception because LinQ does not support Invoke node types.
So my question is : is there a way to just combine the bodies of the two expressions, and how please ?
Thanks by advance !

Well, this should do the trick:
void Main()
{
var execute = Create<TestClass>(
first => first.MyStringField, // field selector
second => second.Equals("1234") // method selector
);
Console.WriteLine(execute(new TestClass("1234"))); // true
Console.WriteLine(execute(new TestClass("4321"))); // false
}
class TestClass
{
public string MyStringField;
public TestClass(string val){
MyStringField = val;
}
}
static Func<TSource, bool> Create<TSource>(
Expression<Func<TSource, string>> fieldSelector,
Expression<Func<string, bool>> methodSelector
)
{
// todo: classical validation of fieldSelector, if necessary.
// todo: classical validation of methodSelector, if necessary.
var compiledFieldSelector = fieldSelector.Compile();
var compiledMethodSelector = methodSelector.Compile();
return T => compiledMethodSelector(compiledFieldSelector(T));
}
Note, due the nature of lambda expressions, you need to validate the field selector and method selector, otherwise it's possible to do some very weird things.
Alternatively, the next approach creates lambda that does not "compose" things, in the sense, this is better since LinqToEntities shouldn't have problems interpreting the query.
static Func<TSource, bool> Create<TSource>(
Expression<Func<TSource, string>> memberSelector,
Expression<Func<string, bool>> methodSelector
)
{
// todo: classical validation of memberSelector, if necessary.
// todo: classical validation of methodSelector, if necessary.
var memberExpression = (MemberExpression)(memberSelector.Body);
var methodCallExpression = (MethodCallExpression)(methodSelector.Body);
// input TSource => ..
var input = Expression.Parameter(typeof(TSource));
var call = Expression.Call(
Expression.MakeMemberAccess(
input,
memberExpression.Member),
methodCallExpression.Method,
methodCallExpression.Arguments);
return Expression.Lambda<Func<TSource, bool>>(call, input)
.Compile();
}

Chris Eelmaa's 2nd answer is ok, just remove the .Compile() call to avoid having the Exception with Invoke :
static Expression<Func<TSource, bool>> Create<TSource>(
Expression<Func<TSource, string>> memberSelector,
Expression<Func<string, bool>> methodSelector
)
{
// todo: classical validation of memberSelector, if necessary.
// todo: classical validation of methodSelector, if necessary.
var memberExpression = (MemberExpression)(memberSelector.Body);
var methodCallExpression = (MethodCallExpression)(methodSelector.Body);
// input TSource => ..
var input = Expression.Parameter(typeof(TSource));
var call = Expression.Call(
Expression.MakeMemberAccess(
input,
memberExpression.Member),
methodCallExpression.Method,
methodCallExpression.Arguments);
return Expression.Lambda<Func<TSource, bool>>(call, input);
}
In my case this is then used like this :
(selector is an expression like u => u.id, and
request is a IQueryable<T>, both received as arguments)
Expression<Func<T, bool>> contains = Create<T>(selector, u => u.Contains(searchExpression));
IQueryable<T> result = request.Where(contains);

Understanding Expression.Invoke() Method

I've been understanding PredicateBuilder extension methods written Joseph Albahari and I saw this Expression.Invoke and honestly I couldn't understand the reason of it in the following method :
public static Expression<Func<T, bool>> Or<T> (this Expression<Func<T, bool>>
expr1, Expression<Func<T, bool>> expr2)
{
var invokedExpr = Expression.Invoke (expr2,
expr1.Parameters.Cast<Expression> ());
return Expression.Lambda<Func<T, bool>>
(Expression.OrElse (expr1.Body, invokedExpr), expr1.Parameters);
}
Even though he explained it a little bit:
The interesting work takes place inside the And and Or methods. We
start by invoking the second expression with the first expression’s
parameters. An Invoke expression calls another lambda expression using
the given expressions as arguments. We can create the conditional
expression from the body of the first expression and the invoked
version of the second. The final step is to wrap this in a new lambda
expression.
MSDN tells me that:
Creates an InvocationExpression that applies a delegate or lambda
expression to a list of argument expressions.
and this makes sense to me little bit. So basically I don't have to pass in any arguments if I use the expression like that.
But for some reason I couldn't quite understand it. Maybe I am tired or something.
Questions:
When and in which situation does it makes sense to use InvocationExpression.
Can anyone explain how Or<T> method (or AndElse<T>) method works little better?
Update:
I was thinking about InvocationExpression when I was coming from work to home and it hinted in my mind like this:
When we invoke a method, we simple say CallMe(phoneNumber, time); and this is called method invocation. Then, InvocationExpression should be an expression that expresses CallMe(phoneNumber, time);. It is similar to LambdaExpression which express a lambda such as t => t + 2. So basically it is a method invocation that is applied to arguments (not parameters). So as invocation, it is no longer expected to need a parameters but perhaps return something since arguments are already applied to its parameters.
For more information about the code I am talking about, please visit http://www.albahari.com/nutshell/predicatebuilder.aspx

Imagine you weren't working with expressions, but with delegates. Then you could write Or like this:
public static Func<T, bool> Or<T>(this Func<T, bool> expr1, Func<T, bool> expr2)
{
return x => expr1(x) || expr2(x);
}
You create a new delegate that invokes the two delegates, combined using ||. When you rewrite this to use expressions, invoking a delegate turns into Expression.Invoke():
public static Expression<Func<T, bool>> Or<T>(
this Expression<Func<T, bool>> expr1, Expression<Func<T, bool>> expr2)
{
var parameter = Expression.Parameter(typeof(T), "x");
var invokedExpr1 = Expression.Invoke(expr1, parameter);
var invokedExpr2 = Expression.Invoke(expr2, parameter);
return Expression.Lambda<Func<T, bool>>(
Expression.OrElse(invokedExpr1, invokedExpr2), parameter);
}
The reason why the actual Or isn't written like this is (most likely) an optimization: you don't have to invoke both expressions, you can reuse the body and parameter from one of them. (But you can't reuse both of them, because they have different parameters.)

Building a custom predicate to act as a filter using a foreach loop

I need to filter a list of documents by passing them to a custom filter that I'm struggling to build dynamically using a foreach loop :
var mainPredicate = PredicateBuilder.True<Document>();
// mainPredicate is combined to other filters successfully here ...
var innerPredicate = PredicateBuilder.False<Document>();
foreach (var period in periods)
{
var p = period;
Expression<Func<Document, bool>> inPeriod =
d => d.Date >= p.DateFrom && d.Date <= p.DateTo;
innerPredicate = innerPredicate.Or(d => inPeriod.Invoke(d));
}
mainPredicate = mainPredicate.And(innerPredicate);
This last line :
documents = this.ObjectSet.AsExpandable().Where(mainPredicate).ToList();
Throws this exception :
The parameter 'd' was not bound in the specified LINQ to Entities
query expression.
Anyone knows why I'm getting this exception ? I don't understand where the 'd' parameter I am passing to the InPeriod method gets lost. I don't know what is missing for this to work. My code is the same as many other examples that work perfectly. Any additionnal theoric theoric information about invoking expressions and how it works behind the scenes are welcome.

I don't understand why you do this:
innerPredicate = innerPredicate.Or(d => inPeriod.Invoke(d));
When you could just avoid the Invoke completely, like this:
innerPredicate = innerPredicate.Or(inPeriod);
This should work perfectly fine.
BTW, I have a feeling there's a bug with LINQKit here (unless there's some documentation that suggests that it doesn't support this scenario).
When I tried this similar code:
Expression<Func<int, bool>> first = p1 => p1 > 4;
Expression<Func<int, bool>> second = p2 => p2 < 2;
// Expand is similar to AsExpandable, except it works on
// expressions, not queryables.
var composite = first.Or(d => second.Invoke(d))
.Expand();
...LINQKit generated the following composite expression:
p1 => ((p1 > 4) OrElse (d < 2)) // what on earth is d?
... which indeed has the unbound parameter d (NodeType = Parameter, Name = 'd').
Dodging the Invoke with first.Or(second).Expand() generates the perfectly sensible:
p1 => ((p1 > 4) OrElse (p1 < 2)) // much better now...

Finally, I have found a way to avoid combining multiple predicates to the main expression tree.
Given that each predicate represents a different filter and I want the final, combined filter to be a series of must-be-respected conditions, we can say that each of the predicates has to return true for the final predicate to return true.
For that to work, the predicates has to be combined with AND. So, the resulting SQL query must look like this :
predicate1 AND predicate2 AND predicate3 ...
A better way to combine these predicates with AND is to chain Where query operators to the final query, like this :
var documents = this.ObjectSet.AsExpandable()
.Where(mainPredicate)
.Where(otherPredicate)
.Where(yetAnotherPredicate)
.ToList();
The resulting SQL query will combine each of these predicates with AND. That is just what I wanted to do.
It is easier than hacking out an expression tree by myself.

I use these extension methods:
public static class Extensions
{
public static Expression<Func<T, bool>> OrElse<T>(this Expression<Func<T, bool>> source, Expression<Func<T, bool>> predicate)
{
InvocationExpression invokedExpression = Expression.Invoke(predicate, source.Parameters.Cast<Expression>());
return Expression.Lambda<Func<T, bool>>(Expression.OrElse(source.Body, invokedExpression), source.Parameters);
}
public static Expression<Func<T, bool>> AndAlso<T>(this Expression<Func<T, bool>> source, Expression<Func<T, bool>> predicate)
{
InvocationExpression invokedExpression = Expression.Invoke(predicate, source.Parameters.Cast<Expression>());
return Expression.Lambda<Func<T, bool>>(Expression.AndAlso(source.Body, invokedExpression), source.Parameters);
}
}

Combining Expressions in an Expression Tree

How can I build an expression tree when parts of the expression are passed as arguments?
E.g. what if I wanted to create expression trees like these:
IQueryable<LxUser> test1(IQueryable<LxUser> query, string foo, string bar)
{
query=query.Where(x => x.Foo.StartsWith(foo));
return query.Where(x => x.Bar.StartsWith(bar));
}
but by creating them indirectly:
IQueryable<LxUser> test2(IQueryable<LxUser> query, string foo, string bar)
{
query=testAdd(query, x => x.Foo, foo);
return testAdd(query, x => x.Bar, bar);
}
IQueryable<T> testAdd<T>(IQueryable<T> query,
Expression<Func<T, string>> select, string find)
{
// how can I combine the select expression with StartsWith?
return query.Where(x => select(x) .. y => y.StartsWith(find));
}
Result:
While the samples didn't make much sense (sorry but I was trying to keep it simple), here's the result (thanks Quartermeister).
It can be used with Linq-to-Sql to search for a string that starts-with or is equal to the findText.
public static IQueryable<T> WhereLikeOrExact<T>(IQueryable<T> query,
Expression<Func<T, string>> selectField, string findText)
{
Expression<Func<string, bool>> find;
if (string.IsNullOrEmpty(findText) || findText=="*") return query;
if (findText.EndsWith("*"))
find=x => x.StartsWith(findText.Substring(0, findText.Length-1));
else
find=x => x==findText;
var p=Expression.Parameter(typeof(T), null);
var xpr=Expression.Invoke(find, Expression.Invoke(selectField, p));
return query.Where(Expression.Lambda<Func<T, bool>>(xpr, p));
}
e.g.
var query=context.User;
query=WhereLikeOrExact(query, x => x.FirstName, find.FirstName);
query=WhereLikeOrExact(query, x => x.LastName, find.LastName);

You can use Expression.Invoke to create an expression that represents applying one expression to another, and Expression.Lambda to create a new lambda expression for the combined expression. Something like this:
IQueryable<T> testAdd<T>(IQueryable<T> query,
Expression<Func<T, string>> select, string find)
{
Expression<Func<string, bool>> startsWith = y => y.StartsWith(find);
var parameter = Expression.Parameter(typeof(T), null);
return query.Where(
Expression.Lambda<Func<T, bool>>(
Expression.Invoke(
startsWith,
Expression.Invoke(select, parameter)),
parameter));
}
The inner Expression.Invoke represents the expression select(x) and the outer one represents calling y => y.StartsWith(find) on the value returned by select(x).
You could also use Expression.Call to represent the call to StartsWith without using a second lambda:
IQueryable<T> testAdd<T>(IQueryable<T> query,
Expression<Func<T, string>> select, string find)
{
var parameter = Expression.Parameter(typeof(T), null);
return query.Where(
Expression.Lambda<Func<T, bool>>(
Expression.Call(
Expression.Invoke(select, parameter),
"StartsWith",
null,
Expression.Constant(find)),
parameter));
}

This Works:
public IQueryable<T> Add<T>(IQueryable<T> query, Expression<Func<T, string>> Selector1,
Expression<Func<T, string>> Selector2, string data1, string data2)
{
return Add(Add(query, Selector1, data1), Selector2, data2);
}
public IQueryable<T> Add<T>(IQueryable<T> query, Expression<Func<T, string>> Selector, string data)
{
var row = Expression.Parameter(typeof(T), "row");
var expression =
Expression.Call(
Expression.Invoke(Selector, row),
"StartsWith", null, Expression.Constant(data, typeof(string))
);
var lambda = Expression.Lambda<Func<T, bool>>(expression, row);
return query.Where(lambda);
}
You use it like:
IQueryable<XlUser> query = SomehowInitializeIt();
query = Add(query, x => x.Foo, y => y.Bar, "Foo", "Bar");

Usually you don't do that in the way you descirbed (using the IQueryable Interface) but you rather use Expressions like Expression<Func<TResult, T>>. Having said that, you compose higher order functions (such as where or select) into a query and pass in expressions that will "fill in" the desired functionality.
For example, consider the signature of the Enumerable.Where method:
Where<TSource>(IEnumerable<TSource>, Func<TSource, Boolean>)
The function takes a delegate as its second argument that is called on each element. The value you return from that delegate indicates to the higher order function if it shall yield the current element (include it in the result or not).
Now, let's take a look at Queryable.Where:
Queryable.Where<TSource>-Methode (IQueryable<TSource>, Expression<Func<TSource, Boolean>>)
We can observe the same pattern of a higher order function, but instead of an Func<> delegate it takes an Expression. An expression is basically a data representation of your code. Compiling that expression will give you a real (executable) delegate. The compiler does a lot of heavy lifting to build expression trees from lambdas you assign to Expression<...>. Expression trees make it possible to compile the described code against different data sources, such as a SQL Server Database.
To come back to your example, what I think you're looking for is a selector. A selector takes each input element and returns a projection of it. It's signature looks like this: Expression<Func<TResult, T>>. For example you could specify this one:
Expression<Func<int, string>> numberFormatter = (i) => i.ToString(); // projects an int into a string
To pass in a selector, your code would need to look like this:
IQueryable<T> testAdd<T>(IQueryable<T> query, Expression<Func<string, T>> selector, string find)
{
// how can I combine the select expression with StartsWith?
return query.Select(selector) // IQueryable<string> now
.Where(x => x.StartsWith(find));
}
This selector would allow you to project the input string to the desired type. I hope I got your intention corrrectly, it's hard to see what you're trying to achieve.