We Keep Coding

How to combine expressions Expression<Func<T1, T2, bool>> to a single Expression<Func<T2, bool>>

I have a list of conditions composed of two Funcs:
public Func<TConfiguration, string> ConfigurationField { get;}
public Func<TNumbering, string> NumberingField { get; }
For each condition, expression would look like this:
Expression<Func<TNumbering, TConfiguration, bool>> (n, c) => criteria.ConfigurationField(c) != criteria.NumberingField(n)
I need to chain the list of these expressions with OrElse.
I tried doing something like:
BinaryExpression expression = null;
foreach (var criteria in SelectionCriteria)
{
Expression<Func<TNumbering, TConfiguration, bool>> exp = (n, c) => criteria.ConfigurationField(c) != criteria.NumberingField(n);
expression = expression == null ? exp : Expression.OrElse(expression, exp);
}
if (expression == null) return Result.Failure("Expression not defined"));
var lambda = Expression.Lambda<Func<TConfiguration, bool>>(expression);
numberingsToRemove = numberings.Where(_ => configurations.All(lambda));
However, compiler doesn't like it, says there is no implicit conversion between Expression.Lambda<Func<TConfiguration, bool>> and Binary expression.
If I use
expression = expression == null ? Expression.OrElse(exp, exp) : Expression.OrElse(expression, exp);
I get
The binary operator OrElse is not defined for the types 'System.Func<TNumbering,TConfiguration,System.Boolean> and 'System.Func<TNumbering,TConfiguration,System.Boolean>.
I am new to building expressions, can somebody point me in the right direction how to do this?

Your Expression<Func<TNumbering, TConfiguration, bool>> is a generic type whose open generic type is Expression<TDelegate>, where TDelegate is some delegate type; in this case Func<TNumbering, TConfiguration, bool>.
Expression<TDelegate> inherits from LambdaExpression, which represents a C# (or VB.NET) lambda expression.
Just like you couldn't write the following code:
var result =
(n, c) => criteria.ConfigurationField(c) != criteria.NumberingField(n) ||
(n1, c1) => criteria.ConfigurationField(c1) != criteria.NumberingField(n1);
trying to combine two LambdaExpressions with OrElse would throw an exception at runtime.
Your code isn't even compiling, because expression is typed as BinaryExpression, meaning an expression corresponding to this:
criteria.ConfigurationField(c) != criteria.NumberingField(n) ||
criteria.ConfigurationField(c1) != criteria.NumberingField(n1)
into which you're trying to put a full Expression<TDelegate>, which includes (for example) the parameter list.
Every LambdaExpression has a Body property, which extracts from an expression corresponding to this:
(n, c) => criteria.ConfigurationField(c) != criteria.NumberingField(n)
the body of the LambdaExpression, or an expression corresponding to this:
criteria.ConfigurationField(c) != criteria.NumberingField(n)
which, in theory, you could then combine into a BinaryExpression corresponding to this:
criteria.ConfigurationField(c) != criteria.NumberingField(n) ||
criteria.ConfigurationField(c1) != criteria.NumberingField(n1)
But this wouldn't work either, because each iteration introduces multiple new parameters, all of which you'd have to pass in to the final lambda.
It's possible to work around this problem, but I would suggest first and foremost you map each element in SelectionCriteria to an expression corresponding to the criteria evaluation, using the factory methods at System.Linq.Expressions.Expression. You could then combine those expressions into a BinaryExpression which you could then wrap up in a LambdaExpression or even an Expression.
It might look something like this (making some assumptions):
class Criteria<TConfiguration, TNumbering> {
public Func<TConfiguration, string> ConfigurationField { get;}
public Func<TNumbering, string> NumberingField { get; }
}
// using static System.Linq.Expressions.Expression;
var SelectionCritera = new List<Criteria>();
/*
* populate list here
*/
var configParam = Parameter(typeof(TConfiguration));
var numberingParam = Parameter(typeof(TNumbering));
var expressions =
SelectionCriteria.Select(criteria => {
var criteriaExpr = Constant(criteria);
return NotEqual( // !=
Invoke( // ( ... )
PropertyOrField( // .ConfigurationField
criteriaExpr, // criteria
"ConfigurationField"
),
configParam // c
),
Invoke( // ( ... )
PropertyOrField( // .NumberingField
criteriaExpr, // criteria
"NumberingField"
),
numberingParam // n
)
);
})
.ToList();
if (!expressions.Any) { return Result.Failure("Expression not defined")); }
// Combine all the subexpressions using ||
var body = expressions.Aggregate((prev, next) => OrElse(prev, next));
// Create a LambdaExpression
var lmbd = Lambda<Func<TConfiguration, TNumbering, bool>>(body, configParam, numberingParam);
// Create a .NET method from the LambdaExpression
var mthd = lmbd.Compile();
// Apply the method to each config/numbering pair
var result = (
from config in configs
from numbering in numbering
select (config, numbering)
).All(x => mthd(config, numbering));

Combining two expressions (Expression<Func<T, bool>>) in a single one

I want to combile two expressions :
public Expression<Func<AnyType,bool>> BuildExpression(int id, string filtreA, string filtreB)
{
Expression<Func<AnyType, bool>> extraExpression = null;
Expression<Func<AnyType, bool>> expression = null;
expression = a => a.ID == id;
var fullExpression = expression.Body;
if (!String.IsNullOrEmpty(filtreA))
{
extraExpression = a => a.TYPE == filtreA;
fullExpression = Expression.AndAlso(fullExpression, extraExpression.Body);
}
if (!String.IsNullOrEmpty(filtreB))
{
extraExpression = a => a.TYPE == filtreB;
fullExpression = Expression.AndAlso(fullExpression, extraExpression.Body);
}
expression = Expression.Lambda<Func<AnyType, bool>>(fullExpression, expression.Parameters[0]);
}
return expression;
}
//I want my final expression to be a.ID == id && a.TYPE == filtreB for example
The problem is that I get the following error : System.InvalidOperationException. Le paramètre 'a' n'est pas dans la portée.. It only happens if enter in one of my ifs.
Does anyone know how I can handle this ? Thanks
ps: My question il similar to this post but it looks like the solution doesn't work anymore : Combining two expressions (Expression<Func<T, bool>>)

You have missed parameters replacement. It is a tricky with visitors.
Trying to simplify your life, I would suggest to use popular library LINQKit
and rewrite your expression building.
public Expression<Func<AnyType, bool>> BuildExpression(int id, string filtreA, string filtreB)
{
var predicate = PredicateBuilder.New<AnyType>(true);
predicate = predicate.And(a => a.ID == id);
if (!string.IsNullOrEmpty(filtreA))
{
predicate = predicate.And(a => a.TYPE == filtreA);
}
if (!string.IsNullOrEmpty(filtreB))
{
predicate = predicate.And(a => a.TYPE == filtreB);
}
return predicate;
}

Each parameter is a different ParameterExpression instance. This problem would be more obvious if you had defined each expression with different parameter names;
expression = a => a.ID == id;
// ...
extraExpression = b => b.TYPE == filtreA;
// ...
extraExpression = c => c.TYPE == filtreB;
Note that entity framework core 2.1 and earlier didn't care if your parameters were different, and would seem to look through .Invoke operations. But since version 3, with an internal rewrite of the expression compiler, neither of those are supported. Hence why earlier examples may not work anymore.
An expression visitor to swap parameter expressions isn't too complicated;
public class MapParameters : ExpressionVisitor
{
private readonly Dictionary<ParameterExpression, ParameterExpression> mapping;
public MapParameters(IEnumerable<ParameterExpression> before, IEnumerable<ParameterExpression> after)
{
this.mapping = new Dictionary<ParameterExpression, ParameterExpression>(
before
.Zip(after)
.Select(p => KeyValuePair.Create(p.First,p.Second))
);
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (mapping.TryGetValue(node, out var replace))
return replace;
return base.VisitParameter(node);
}
}
Expression.Lambda<Func<AnyType, bool>>(
Expression.AndAlso(
expression.Body,
new MapParameters(
extraExpression.Parameters,
expression.Parameters
).Visit(extraExpression.Body)
),
expression.Parameters);
Also note that if you are using this to build a single expression for IQueryable<T>.Where(...). You could consider calling .Where(...) twice, which would achieve the same end result.

Combining Expressions for LINQ

I have a big code chunk below, the actual code is not that important, but is included in order to demonstrate what I'm trying to do.
I have a public static Expression> GetFreeSpotCount(...)
this method returns an Expression which I would like to reuse in another method
The expression itself return a number which indicates the number of [model] in a list of [model] which fulfills certain criteria.
I want the new method to return all [model]'s where the number above is greater than 0.
below is what code I have presently, I want to avoid having to duplicate so much code
public static Expression<Func<ExamTimeSlot, int>> GetFreeSpotCountFor(List<Guid> drivingSchoolIds)
{
return ets =>
ets.Participants
- ets.Exams.Where(ex => ex.Status == ExamStatus.Pending).Count(e => !ets.ExamTimeSlotReservations.Any(r => r.DrivingSchoolId == e.BookedByDrivingSchoolId))
- ((int?)ets.ExamTimeSlotReservations.Sum(r => (ets.Exams.Where(ex => ex.Status == ExamStatus.Pending).Where(e => e.BookedByDrivingSchoolId == r.DrivingSchoolId).Count()
- r.ReservedSpots) > 0 ? (ets.Exams.Where(ex => ex.Status == ExamStatus.Pending).Where(e => e.BookedByDrivingSchoolId == r.DrivingSchoolId).Count() - r.ReservedSpots) : 0) ?? 0)
- ((int?)ets.ExamTimeSlotReservations.Sum(r => r.ReservedSpots) ?? 0)
+ ((((int?)ets.ExamTimeSlotReservations.Where(r => drivingSchoolIds.Any(id => r.DrivingSchoolId == id)).Sum(r => r.ReservedSpots) ?? 0)
- ets.Exams.Where(ex => drivingSchoolIds.Any(id => ex.BookedByDrivingSchoolId == id) && ex.Status == ExamStatus.Pending).Count()) >= 0 ?
(((int?)ets.ExamTimeSlotReservations.Where(r => drivingSchoolIds.Any(id => r.DrivingSchoolId == id)).Sum(r => r.ReservedSpots) ?? 0)
- ets.Exams.Where(ex => drivingSchoolIds.Any(id => ex.BookedByDrivingSchoolId == id) && ex.Status == ExamStatus.Pending).Count())
: 0);
}
and:
public static Expression<Func<ExamTimeSlot, bool>> GetExamTimeSlotsWithFreeSpotsFor(List<Guid> drivingSchoolIds)
{
return ets =>
(ets.Participants
- ets.Exams.Where(ex => ex.Status == ExamStatus.Pending).Count(e => !ets.ExamTimeSlotReservations.Any(r => r.DrivingSchoolId == e.BookedByDrivingSchoolId))
- ((int?)ets.ExamTimeSlotReservations.Sum(r => (ets.Exams.Where(ex => ex.Status == ExamStatus.Pending).Where(e => e.BookedByDrivingSchoolId == r.DrivingSchoolId).Count()
- r.ReservedSpots) > 0 ? (ets.Exams.Where(ex => ex.Status == ExamStatus.Pending).Where(e => e.BookedByDrivingSchoolId == r.DrivingSchoolId).Count() - r.ReservedSpots) : 0) ?? 0)
- ((int?)ets.ExamTimeSlotReservations.Sum(r => r.ReservedSpots) ?? 0)
+ ((((int?)ets.ExamTimeSlotReservations.Where(r => drivingSchoolIds.Any(id => r.DrivingSchoolId == id)).Sum(r => r.ReservedSpots) ?? 0)
- ets.Exams.Where(ex => drivingSchoolIds.Any(id => ex.BookedByDrivingSchoolId == id) && ex.Status == ExamStatus.Pending).Count()) >= 0 ?
(((int?)ets.ExamTimeSlotReservations.Where(r => drivingSchoolIds.Any(id => r.DrivingSchoolId == id)).Sum(r => r.ReservedSpots) ?? 0)
- ets.Exams.Where(ex => drivingSchoolIds.Any(id => ex.BookedByDrivingSchoolId == id) && ex.Status == ExamStatus.Pending).Count())
: 0)) > 0;
}
I wanted to do something like:
public static Expression<Func<ExamTimeSlot, bool>> GetExamTimeSlotsWithFreeSpotsFor(List<Guid> drivingSchoolIds)
{
return ets => GetFreeSpotCountFor(drivingSchoolIds) > 0;
}
I have tried using Expression.GreaterThan with the first expression, but since I need the result AND [model] I could not figure out a way to make it work.

There isn't really an easier way (without a third party library) to do this. It seems like we should easily be able to do what you've written, but we're mislead because of the syntactic sugar we rely on when creating expression trees.
public static Expression<Func<ExamTimeSlot, bool>> GetExamTimeSlotsWithFreeSpotsFor(List<Guid> drivingSchoolIds)
{
var param = Expression.Parameter(typeof(ExamTimeSlot));
//We want to create a new lambda which will invoke `GetFreeSpotCountFor` with our parameter, and then check it's greater than 0
var newBody =
Expression.GreaterThan(
//View this as GetFreeSpotCountFor(drivingSchoolIds)(param) - where param will be given to us when this lambda is invoked
Expression.Invoke(
GetFreeSpotCountFor(drivingSchoolIds),
param
),
//Pass the right-hand value (0) to the GreaterThan check
Expression.Constant(0)
);
var lambda = Expression.Lambda<Func<ExamTimeSlot, bool>>(newBody, param);
return lambda;
}
Once we've got the Expression object, we need to build the expression trees manually.
The above will wrap your lambda in a new lambda, which invokes and compares the result to 0. An alternative is to check the existing lambda's .Body, and append .GreaterThan to it, and return an entirely new lambda.
Ie:
public static Expression<Func<ExamTimeSlot, bool>> GetExamTimeSlotsWithFreeSpotsFor(List<Guid> drivingSchoolIds)
{
//This grabs the existing lambda, which we will work on
var oldLambda = GetFreeSpotCountFor(drivingSchoolIds);
var newBody =
//Invoke `GreaterThan` directly on the old lambda's Body
Expression.GreaterThan(
oldLambda.Body,
//Pass the right-hand value (0) to the GreaterThan check
Expression.Constant(0)
);
//Now, we need to pass in the old parameters, and build a new lambda.
var lambda = Expression.Lambda<Func<ExamTimeSlot, bool>>(newBody, oldLambda.Parameters);
return lambda;
}

Why do you specifically want to combine the expressions instead of combining the query calls? One simple way to do this would be to make your GetExamTimeSlotsWithFreeSpotsFor not take and return an expression but take and return a query, and add two where clauses to that query (each being the expression you stored).
public static IQueryable<ExamTimeSlot> WhereExamTimeSlotsWithFreeSpotsFor (IQueryable<ExamTimeSlot> Source, List<Guid> drivingSchoolIds)
{
var FirstFilter = GetFreeSpotCountFor(drivingSchoolIds);
var SecondFilter = GetExamTimeSlotsWithFreeSpotsFor(drivingSchoolIds);
return Source
.Where(FirstFilter)
.Where(SecondFilter);
}
That way you don't need to combine anything, let your LINQ provider combine the two where clause by himself.
Note that if you really wanted to combine the expressions manually it is possible but as far as i know it will require either a third party API or working with expression trees directly (not just enjoying the compiler magic doing it for you). It's not hard if it's just to return an expression that takes 2 expressions and tosses an and operator in between but that seems overkill for the same result.

Good way to handle NullReferenceException before C# 6.0

My code below gives me a NullReferenceException and the stack trace tells me the problem is in the Count method, so I'm pretty sure at some point foo, bar or baz is null.
My code:
IQueryable<IGrouping<string, Referral>> queryable= ...;
var dict = queryable.ToDictionary(g => g.Key.ToString(),
g => g.Count(r => r.foo.bar.baz.dummy == "Success"));
I'm wondering what's a concise way to handle null cases.
I learn that in C# 6.0 I can just do foo?.bar?.baz?.dummy, however the project I'm working on isn't C# 6.0

A solution for <6.0 would be:
.Count(r => r.foo != null &&
r.foo.bar != null &&
r.foo.bar.baz != null &&
r.foo.bar.baz.dummy == "Success")
Exactly for complex constructs like the one above the null propagation operator was introduced.
Furthermore you could also refactor the expression into a private method:
private Expression<Func<Referral, bool>> Filter(string value)
{
return r => r.foo != null &&
r.foo.bar != null &&
r.foo.bar.baz != null &&
r.foo.bar.baz.dummy == value;
}
and use it as follows:
g => g.Count(Filter("Success"))

You can use the following extension methods.
public static TResult With<TInput, TResult>(this TInput o, Func<TInput, TResult> evaluator)
where TResult : class
where TInput : class
{
return o == null ? null : evaluator(o);
}
public static TResult Return<TInput, TResult>(this TInput o, Func<TInput, TResult> evaluator, TResult failureValue)
where TInput : class
{
return o == null ? failureValue : evaluator(o);
}
Their combination gives you a nice, readable API for handling nulls:
return foo
.With(o => o.bar)
.With(o => o.baz)
.Return(o => o.dummy, null);

The problem is that the ToDictionary method isn't actually done on the queryable - instead, you get the whole collection, and do the aggregation in your application, rather than on the DB server.
So instead of using ToDictionary directly, use Select first:
IQueryable<IGrouping<string, Referral>> queryable= ...;
var dict = queryable.Select(g => new { Key = g.Key.ToString(),
Count = g.Count(r => r.foo.bar.baz.dummy == "Success") })
.ToDictionary(i => i.Key, i => i.Count);
This will make sure the aggregation is done in the database (where you don't care about those nulls) and not in the C# code (where you get a NullReferenceException).
Of course, this assumes that the queryable you're using is a DB query (or, to be more precise, a queryable that supports aggregation and has ANSI SQL-like NULL semantics). If you have a different custom queryable, it's not going to help (unless you explicitly add those NULL semantics yourself).

// if null, use null
if(objectvariable == null)
{
// throw exception
}
// if not null
if(objectvariable != null)
{
// continue
}

Reusing existing linq expressions in a compiled query

Consider the following code which provides two methods: One to return an IQueryable, and one which leverages a compiled query to very efficient return the location matching a specific ID:
public IQueryable<Location> GetAllLocations()
{
return from location in Context.Location
where location.DeletedDate == null
&& location.Field1 = false
&& location.Field2 = true
&& location.Field3 > 5
select new LocationDto
{
Id = location.Id,
Name = location.Name
}
}
private static Func<MyDataContext, int, Location> _getByIdCompiled;
public Location GetLocationById(int locationId)
{
if (_getByIdCompiled == null) // precompile the query if needed
{
_getByIdCompiled = CompiledQuery.Compile<MyDataContext, int, Location>((context, id) =>
(from location in Context.Location
where location.DeletedDate == null
&& location.Field1 = false
&& location.Field2 = true
&& location.Field3 > 5
&& location.Id == id
select new LocationDto {
Id = location.Id,
Name = location.Name
})).First());
}
// Context is a public property on the repository all of this lives in
return _getByIdCompiled(Context, locationId);
}
This is a pretty big simplification of the actual code, but I think it gets the idea accross, and it works fine. The next thing I want to do is refactor the code, so that the common bit of the expression can be reused, since it will be used in many other types of compiled queries. In other words, this expression:
from location in Context.Location
where location.DeletedDate == null
&& location.Field1 = false
&& location.Field2 = true
&& location.Field3 > 5
select new LocationDto
{
Id = location.Id,
Name = location.Name
};
How can I somehow capture this in a variable or function and reuse it in multiple compiled queries? My attempts so far have led to errors complaining about things not being translatable to SQL, Member access not allowed, etc.
Update: Another potentially better way I could have asked this question is as follows:
Consider the two compiled queries below:
_getByIdCompiled = CompiledQuery.Compile<MyDataContext, int, LocationDto>((context, id) =>
(from location in Context.Location // here
where location.DeletedDate == null // here
&& location.Field1 = false // here
&& location.Field2 = true // here
&& location.Field3 > 5 // here
&& location.Id == id
select new LocationDto { // here
Id = location.Id, // here
Name = location.Name
})).First()); // here
_getByNameCompiled = CompiledQuery.Compile<MyDataContext, int, LocationDto>((context, name) =>
(from location in Context.Location // here
where location.DeletedDate == null // here
&& location.Field1 = false // here
&& location.Field2 = true // here
&& location.Field3 > 5 // here
&& location.Name == name
select new LocationDto { // here
Id = location.Id, // here
Name = location.Name // here
})).First()); // here
All of the lines marked // here are duplicate very un-dry pieces of code. (In my code base, this actually 30+ lines of code.) How do I factor it out and make it reusable?

So, this whole thing is somewhat odd in that the Compile method needs to not only see the Expression objects passed to each query operator (Where, Select, etc.) as something it can understand, but it needs to see the whole query, including the use of all of the operators, as something it can comprehend as Expression objects. This pretty much removes more traditional query composition as an option.
This is going to get a tad messy; more so than I would really like, but I don't see a whole lot of great alternatives.
What we're going to do is create a method to construct our queries. It's going to accept a filter as a parameter, and that filter will be an Expression that represents a filter for some object.
Next we're going to define a lambda that will look almost exactly like what you would pass to Compile, but with an extra parameter. That extra parameter will be of the same type as our filter, and it will represent that actual filter. We'll use that parameter, instead of the filter, throughout the lambda. We'll then use a UseIn method to replace all instances of that third parameter in our new lambda with the filter expression that we are providing to the method.
Here is the method to construct the query:
private static Expression<Func<MyDataContext, int, IQueryable<LocationDto>>>
ConstructQuery(Expression<Func<Location, bool>> filter)
{
return filter.UseIn((MyDataContext context, int id,
Expression<Func<Location, bool>> predicate) =>
from location in context.Location.Where(predicate)
where location.DeletedDate == null
&& location.Field1 == false
&& location.Field2 == true
&& location.Field3 > 5
&& location.Id == id
select new LocationDto
{
Id = location.Id,
Name = location.Name
});
}
Here is the UseIn method:
public static Expression<Func<T3, T4, T5>>
UseIn<T1, T2, T3, T4, T5>(
this Expression<Func<T1, T2>> first,
Expression<Func<T3, T4, Expression<Func<T1, T2>>, T5>> second)
{
return Expression.Lambda<Func<T3, T4, T5>>(
second.Body.Replace(second.Parameters[2], first),
second.Parameters[0],
second.Parameters[1]);
}
(The typing is a mess here, and I can't figure out how to give the generic types meaningful names.)
The following method is used to replace all instances of one expression with another:
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
Now that we've gotten through this gory mess, comes the easy part. ConstructQuery should be able to be modified at this point to represent your real query, without much difficulty.
To call this method we simply need to provide whatever filter we want applied to this alteration of the query, such as:
var constructedQuery = ConstructQuery(location => location.Id == locationId);

Linq statements have to end in either select or group clause so you can't cut out part of the query and store it elsewhere, but if you will always be filtering by the same four conditions, you can use the lambda syntax instead, and then add any additional where clauses in new queries.
And as pointed out by #Servy, you need to call First() or FirstOrDefault() to get a single element from the query result.
IQueryable<Location> GetAllLocations()
{
return Context.Location.Where( x => x.DeletedDate == null
&& x.Field1 == false
&& x.Field2 == true
&& x.Field3 > 5
).Select( x => x );
}
Location GetLocationById( int id )
{
return ( from x in GetAllLocations()
where x.Id == id
select x ).FirstOrDefault();
}
//or with lambda syntax
Location GetLocationById( int id )
{
return GetAllLocations()
.Where( x => x.Id == id )
.Select( x => x )
.FirstOrDefault();
}