We Keep Coding

How to create a reusable 'Contains' expression for EF Core

Problem
I need to execute a partial text search, alongside other filters, via a generic repository using expressions.
State of current code
I have a generic method that returns paged results from my database (via a common repository layer).
In the following working example;
PagedRequest contains the current pagesize and page number, and is used during respective Skip / Take operations.
PagedResult contains a collection of the results, along with the total number of records.
public Task<PagedResult<Person>> GetPeopleAsync(PersonSearchParams searchParams,
PagedRequest pagedRequest = null)
{
ParameterExpression argParam = Expression.Parameter(typeof(Locum), "locum");
// start with a "true" expression so we have an expression to "AndAlso" with
var alwaysTrue = Expression.Constant(true);
var expr = Expression.Equal(alwaysTrue, alwaysTrue);
if (searchParams != null)
{
BinaryExpression propExpr;
if (searchParams.DateOfBirth.HasValue)
{
propExpr = GetExpression(searchParams.DateStart,
nameof(Incident.IncidentDate),
argParam,
ExpressionType.GreaterThanOrEqual);
expr = Expression.AndAlso(expr, propExpr);
}
if (searchParams.DateOfDeath.HasValue)
{
propExpr = GetExpression(searchParams.DateEnd,
nameof(Incident.IncidentDate),
argParam,
ExpressionType.LessThanOrEqual);
expr = Expression.AndAlso(expr, propExpr);
}
if (searchParams.BranchId.HasValue && searchParams.BranchId.Value != 0)
{
propExpr = GetExpression(searchParams.BranchId,
nameof(Incident.BranchId), argParam);
expr = Expression.AndAlso(expr, propExpr);
}
}
var lambda = Expression.Lambda<Func<Locum, bool>>(expr, argParam);
return _unitOfWork.Repository.GetAsync(filter: lambda, pagedRequest: pagedRequest);
}
This is using my static GetExpression method for Expression.Equal, Expression.GreaterThanOrEqual and Expression.LessThanOrEqual queries as follows;
private static BinaryExpression GetExpression<TValue>(TValue value,
string propName, ParameterExpression argParam, ExpressionType? exprType = null)
{
BinaryExpression propExpr;
var prop = Expression.Property(argParam, propName);
var valueConst = Expression.Constant(value, typeof(TValue));
switch (exprType)
{
case ExpressionType.GreaterThanOrEqual:
propExpr = Expression.GreaterThanOrEqual(prop, valueConst);
break;
case ExpressionType.LessThanOrEqual:
propExpr = Expression.LessThanOrEqual(prop, valueConst);
break;
case ExpressionType.Equal:
default:// assume equality
propExpr = Expression.Equal(prop, valueConst);
break;
}
return propExpr;
}
NOTE: this code is working correctly.
Problem
Using example from other SO answers I have tried the following;
Expressions
I have tried getting the contains via an Expression;
static Expression<Func<bool>> GetContainsExpression<T>(string propertyName,
string propertyValue)
{
var parameterExp = Expression.Parameter(typeof(T), "type");
var propertyExp = Expression.Property(parameterExp, propertyName);
MethodInfo method = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var someValue = Expression.Constant(propertyValue, typeof(string));
var containsMethodExp = Expression.Call(propertyExp, method, someValue);
return Expression.Lambda<Func<bool>>(containsMethodExp);
}
This has to be converted to a BinaryExpression so it can be added to the expression tree using AndAlso. I've tried to compare the Expression with a true value, but this isn't working
if (searchParams.FirstName.IsNotNullOrWhiteSpace())
{
var propExpr = GetContainsExpression<Locum>(nameof(Locum.Firstname),
searchParams.FirstName);
var binExpr = Expression.MakeBinary(ExpressionType.Equal, propExpr, propExpr);
expr = Expression.AndAlso(expr, binExpr);
}
MethodCallExpression
I also tried returning the MethodCallExpression (instead of the Lambda above), using the following;
static MethodCallExpression GetContainsMethodCallExpression<T>(string propertyName,
string propertyValue)
{
var parameterExp = Expression.Parameter(typeof(T), "type");
var propertyExp = Expression.Property(parameterExp, propertyName);
MethodInfo method = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var someValue = Expression.Constant(propertyValue, typeof(string));
var containsMethodExp = Expression.Call(propertyExp, method, someValue);
return containsMethodExp;
}
I used this as follows;
if (searchParams.FirstName.IsNotNullOrWhiteSpace())
{
var propExpr = GetContainsMethodCallExpression<Person>(nameof(Person.FirstName),
searchParams.FirstName);
var binExpr = Expression.MakeBinary(ExpressionType.Equal, propExpr, alwaysTrue);
expr = Expression.AndAlso(expr, binExpr);
}
Exceptions
These expression are passed to a generic method that pages information out of the database, and the exceptions are thrown during the first execution of the query when I Count the total matching number of record on the constructed query.
System.InvalidOperationException: 'The LINQ expression 'DbSet()
.Where(p => True && p.FirstName.Contains("123") == True)' could not be translated. Either rewrite the query in a form that can be translated, or switch to client evaluation explicitly by inserting a call to 'AsEnumerable ', 'AsAsyncEnumerable ', 'ToList ', or 'ToListAsync '. See https://go.microsoft.com/fwlink/?linkid=2101038 for more information.'
This exception is thrown on a Count method I am using in my paging code. This code is already working without any filters, and with the ExpressionType filters described at the top, so I haven't included this code as I don't believe it is relevant.
pagedResult.RowCount = query.Count();

This has to be converted to a BinaryExpression so it can be added to the expression tree using AndAlso
Negative. There is no requirement Expression.AndAlso (or Expression.OrElse) operands to be binary expressions (it would have been strange like requiring left or right operand of && or || to be always comparison operators). The only requirement is them to be bool returning expressions, hence call to string Contains is a perfectly valid operand expression.
So start by changing the type of the inner local variable from BinaryExpression to Expression:
if (searchParams != null)
{
Expression propExpr;
// ...
}
The same btw applies for the initial expression - you don't need true == true, simple
Expression expr = Expression.Constant(true); would do the same.
Now you could emit method call to string.Contains in a separate method similar to the other that you've posted (passing the ParameterExpression and building property selector expression) or inline similar to:
if (searchParams.FirstName.IsNotNullOrWhiteSpace())
{
var propExpr = Expression.Property(argParam, nameof(Person.FirstName));
var valueExpr = Expression.Constant(searchParams.FirstName);
var containsExpr = Expression.Call(
propExpr, nameof(string.Contains), Type.EmptyTypes, valueExpr);
expr = Expression.AndAlso(expr, containsExpr);
}

building OR expression from List of strings [duplicate]

This question already has answers here:
Combining two expressions (Expression<Func<T, bool>>)
(10 answers)
Closed 9 months ago.
In the code below :
Expression<Func<WorkflowTask, bool>> filterBefore = wt => true;
filterBefore = filterBefore.And(wt => wt.code == "XK");
List<string> sourceLanguages = new List<string>() { "FR", "DE", "NL" };
//HOW TO BUILD OR CONDITIONS DYNAMICALLY BASED ON SOURCE LANGUAGES LIST ?
filterBefore = filterBefore.And(wt => wt.SourceLanguages.Contains("FR") || wt.WorkflowTaskContextualInfo.SourceLanguages.Contains("DE"));
I don't know how to build dynamically the OR condition on the SourceLanguages List. That list could contain any number of values (I've hardcoded it here for the sake of example).
wt.WorkflowTaskContextualInfo.SourceLanguages is a string with comma-separated values ("FR, EN" for instance)
The And expression is defined as below :
public static Expression<Func<T, bool>> And<T>(this Expression<Func<T, bool>> expr1, Expression<Func<T, bool>> expr2)
{
var invokedExpr = Expression.Invoke(expr2, expr1.Parameters.Cast<Expression>());
return Expression.Lambda<Func<T, bool>>(Expression.AndAlso(expr1.Body, invokedExpr), expr1.Parameters);
}

LINQKit's PredicateBuilder is designed specifically to address this kind of need. But if you feel that's too much overhead, you can craft your own Expression tree with a few simple utilities, as I've described in this answer
First, a general-purpose Expression Replacer:
public class ExpressionReplacer : ExpressionVisitor
{
private readonly Func<Expression, Expression> replacer;
public ExpressionReplacer(Func<Expression, Expression> replacer)
{
this.replacer = replacer;
}
public override Expression Visit(Expression node)
{
return base.Visit(replacer(node));
}
}
Next, a simple utility method to replace one parameter's usage with another parameter in a given expression:
public static T ReplaceParameter<T>(T expr, ParameterExpression toReplace, ParameterExpression replacement)
where T : Expression
{
var replacer = new ExpressionReplacer(e => e == toReplace ? replacement : e);
return (T)replacer.Visit(expr);
}
This is necessary because the lambda parameters in two different expressions are actually different parameters, even when they have the same name. For example, if you want to end up with q => q.first.Contains(first) || q.last.Contains(last), then the q in q.last.Contains(last) must be the exact same q that's provided at the beginning of the lambda expression.
Next we need a general-purpose Join method that's capable of joining Func<T, TReturn>-style Lambda Expressions together with a given Binary Expression generator.
public static Expression<Func<T, TReturn>> Join<T, TReturn>(Func<Expression, Expression, BinaryExpression> joiner, IReadOnlyCollection<Expression<Func<T, TReturn>>> expressions)
{
if (!expressions.Any())
{
throw new ArgumentException("No expressions were provided");
}
var firstExpression = expressions.First();
var otherExpressions = expressions.Skip(1);
var firstParameter = firstExpression.Parameters.Single();
var otherExpressionsWithParameterReplaced = otherExpressions.Select(e => ReplaceParameter(e.Body, e.Parameters.Single(), firstParameter));
var bodies = new[] { firstExpression.Body }.Concat(otherExpressionsWithParameterReplaced);
var joinedBodies = bodies.Aggregate(joiner);
return Expression.Lambda<Func<T, TReturn>>(joinedBodies, firstParameter);
}
Now, applying that to your example:
Expression<Func<WorkflowTask, bool>> codeCriteria = wt => wt.code == "XK";
var langCriteria = new List<string>() { "FR", "DE", "NL" }
.Select(lang => (Expression<Func<WorkflowTask, bool>>)(wt => wt.SourceLanguages.Contains(lang)))
.ToList();
var filter = Join(Expression.And, new[] { codeCriteria, Join(Expression.Or, langCriteria)});
filter will now have the equivalent of wt => wt.code == "XK" && (wt.SourceLanguages.Contains("FR") || wt.SourceLanguages.Contains("DE") || wt.SourceLanguages.Contains("NL"))

I would put the required languages in an array or list.
var required = new string[]{ "FR", "DE" };
Then you can query with
wt => required.Any(r => wt.SourceLanguages.Contains(r))
or, the other way round
wt => wt.SourceLanguages.Any(sl => required.Contains(sl))

I did not feel like importing a whole library, and the sample given seemed a bit of a stretch so I think i found an easier solution using BinaryExpression Update(Expression left, LambdaExpression? conversion, Expression right).
The samples below accepts a list of string, and constructs a chain of OR expressions. Each OR expressions calls a entity framework's LIKE method. In the end the whole ordeal gets translated nicely to SQL - so if you're on the job of making dynamic filters like i did - this should help you out.
private static Expression<Func<TMeta, bool>> GetMetaKeyFilterPredicateExpression<TMeta>(List<string> metaKeyNames)
where TMeta : IMetaKeyValuePair
{
var parameter = Expression.Parameter(typeof(TMeta));
var property = Expression.Property(Expression.Convert(parameter, typeof(IMetaKeyValuePair)),
propertyName: nameof(IMetaKeyValuePair.MetaKey));
Expression body = null!;
BinaryExpression? predicateExpression = null;
foreach (var metaKeyName in metaKeyNames)
{
var likeExpression = Expression.Call(typeof(DbFunctionsExtensions),
nameof(DbFunctionsExtensions.Like),
null,
Expression.Constant(EF.Functions),
property,
Expression.Constant(metaKeyName)
);
predicateExpression =
predicateExpression == null
? Expression.Or(likeExpression, Expression.Constant(false))
: predicateExpression.Update(predicateExpression, null, likeExpression);
}
body = (Expression?)predicateExpression ?? Expression.Constant(true);
var expr = Expression.Lambda<Func<TMeta, bool>>(body: body, parameter);
return expr;
}
var expr = GetMetaKeyFilterPredicateExpression<TMeta>(metaKeyNames);
qMetaKeyValuePairs = qMetaKeyValuePairs.Where(expr);

Dynamic where clause to filter collection elements

I have a collection List<List<object>>, which I need to filter out based on wheater the List<object> collection contains given element. I was able to build the where clause, but i get the following Exception:
An exception of type 'System.InvalidOperationException' occurred in System.Core.dll but was not handled in user code
Additional information: variable 'x' of type 'System.Collections.Generic.List`1[System.Object]' referenced from scope '', but it is not defined
I have found similar issues and i understand where the problem is, but i need help finding the solution.
Here is my code:
protected override Expression<Func<List<object>, bool>> GetWhereClause()
{
var type = typeof(List<object>);
var parameterExpression = Expression.Parameter(type, "x");
Expression expressionBody = null;
if (Verified)
{
Expression<Func<List<object>, bool>> expression = x => x.Contains("Verified");
expressionBody = expression.Body;
}
if (GoodMatch)
{
Expression<Func<List<object>, bool>> expression = x => x.Contains("Good Match");
if (expressionBody != null)
expressionBody = Expression.Or(expressionBody, expression.Body);
else
expressionBody = expression.Body;
}
//More conditions here
if (expressionBody != null)
{
var whereClauseExp = Expression.Lambda<Func<List<object>, bool>>(expressionBody, parameterExpression);
return whereClauseExp;
}
return null;
}
Now, this method generates the desired where clause, but when i try to apply it, i get the mentioned Exception.
if (whereClause != null)
{
items = items.Where(whereClause.Compile());
//
}

I had a similar use case requiring dynamic where clauses and used Predicate Builder
Using it, you could do something like:*
private Expression<Func<List<T>, bool>> GetWhereClause<T>(T itemToFind){
var predicate = PredicateBuilder.False<List<T>>();
if(Verified) {
predicate = predicate.And(p => p.Contains(itemToFind));
}
if(GoodMatch) {
predicate = predicate.Or(p => p.Contains(itemToFind));
}
return predicate;
}

You cannot use a parameter from another expression 'as is' in a new expression. When you do:
Expression<Func<List<object>, bool>> expression = x => x.Contains("Verified");
expressionBody = expression.Body;
then you simply have the inline defined parameter x in the body. Now this parameter IS NOT the same parameter, that you defined before as
var parameterExpression = Expression.Parameter(type, "x");
even if they both have the name x, that is not enough. Expression trees have reference equality.
So to get it to work, just use a visitor, that will replace the parameter with yours. Create a visitor:
public class ParameterUpdateVisitor : ExpressionVisitor
{
private ParameterExpression _parameter;
public ParameterUpdateVisitor(ParameterExpression parameter)
{
_parameter = parameter;
}
protected override Expression VisitParameter(ParameterExpression node)
{
return _parameter;
}
}
and then in your code use it like:
Expression<Func<List<object>, bool>> expression = x => x.Contains("Verified");
var visitor = new ParameterUpdateVisitor(parameterExpression);
expressionBody = visitor.Visit(expression.Body);
Of course this for every part that comes from another expression tree.
NOTE!!! this visitor is extra simplified, just for your example. If you have expressions that might have methods that have their own parameters, then make sure to replace only the parameter that you want to!
E.g. it wont work for:
Expression<Func<List<object>, bool>> expression = x => x.Select(o => o.ToString()).Contains("Verified");
because this visitor will replace the 'o' too. If you have this case, then pass in the parameter you want to replace (e.g. x, that is expression.Parameters.First()) in the constructor too, and only replace in the overridden method if node == myOldParameter.
BTW: why do you need expression trees if you compile the thing at the end anyways?

Linq Expression from Lambda: specify parameter explicitly

I want to embed an expression tree such as
Expression<Func<MyObject, double>> expr = (o) => o.Value;
into a larger expression tree generated by a parser. However, the parameter o is already defined inside the outer expression tree. In principle I would have to search the body of expr and replace all occurences of the parameter by the instance from the parsed expression tree.
Is there a built in way to do this?
Or is there even a way to directly generate the lambda expression while specifying the instance of the parameter in advance?

You can't directly instruct the compiler to reuse your existing ParameterExpression instances, but you can definitely replace them (in effect creating new expression trees) afterwards.
The built-in ExpressionVisitor helps a lot with the heavy lifting; it's a no-op visitor that you derive from to add the required functionality. In this case you need to instruct it to replace ParameterExpression instances, so you could have this:
// Sorry for the atrocious formatting, wanted to keep it scrollbar-free
class ParameterReplacementVisitor : ExpressionVisitor
{
private readonly
IEnumerable<KeyValuePair<ParameterExpression, ParameterExpression>>
replacementMap;
public ParameterReplacementVisitor(
IEnumerable<KeyValuePair<ParameterExpression, ParameterExpression>> map)
{
this.replacementMap = map;
}
protected override Expression VisitLambda<T>(Expression<T> node)
{
return Expression.Lambda<T>(
Visit(node.Body),
node.Parameters.Select(Visit).Cast<ParameterExpression>());
}
protected override Expression VisitParameter(ParameterExpression node)
{
var replacement = this.replacementMap
.Where(p => p.Key == node)
.DefaultIfEmpty()
.First().Value;
return base.VisitParameter(replacement ?? node);
}
}
which you can use like this:
Expression<Func<int, bool>> e1 = i => true;
Expression<Func<int, bool>> e2 = j => false;
Console.WriteLine(e1.Parameters[0] == e2.Parameters[0]); // false
var replacements = new Dictionary<ParameterExpression, ParameterExpression>
{
{ e1.Parameters[0], e2.Parameters[0] }
};
var replacer = new ParameterReplacementVisitor(replacements);
var e3 = replacer.VisitAndConvert(e1, "replacing parameters");
Console.WriteLine(e3.Parameters[0] == e2.Parameters[0]); // true

List<object>.Contains Expression Tree

I would like to build an Expression that would equate to expected...
Expression<Func<ReferencedEntity, bool>> expected = (ReferencedEntity referencedEntity) => foreignKeys.Contains(referencedEntity.Id);
Expression<Func<ReferencedEntity, bool>> actual;
foreignKeys type is a List<object>
Here is what I have so far and I think it would use Expression.Call() method but not sure exactly how to go about it.
ParameterExpression entityParameter = Expression.Parameter(typeof(TReferencedEntity), "referencedEntity");
MemberExpression memberExpression = Expression.Property(entityParameter, "Id");
Expression convertExpression = Expression.Convert(memberExpression, typeof(object)); //This is becuase the memberExpression for Id returns a int.
//Expression containsExpression = Expression.Call(????
//actual = Expression.Lambda<Func<TReferencedEntity, bool>>(????, entityParameter);
Thanks for you help.

Here is the solution I couldn't have done it without Samuel's suggestion though...
/// <summary>
///
/// </summary>
/// <param name="foreignKeys"></param>
/// <returns></returns>
private Expression<Func<TReferencedEntity, bool>> BuildForeignKeysContainsPredicate(List<object> foreignKeys, string primaryKey)
{
Expression<Func<TReferencedEntity, bool>> result = default(Expression<Func<TReferencedEntity, bool>>);
try
{
ParameterExpression entityParameter = Expression.Parameter(typeof(TReferencedEntity), "referencedEntity");
ConstantExpression foreignKeysParameter = Expression.Constant(foreignKeys, typeof(List<object>));
MemberExpression memberExpression = Expression.Property(entityParameter, primaryKey);
Expression convertExpression = Expression.Convert(memberExpression, typeof(object));
MethodCallExpression containsExpression = Expression.Call(foreignKeysParameter
, "Contains", new Type[] { }, convertExpression);
result = Expression.Lambda<Func<TReferencedEntity, bool>>(containsExpression, entityParameter);
}
catch (Exception ex)
{
throw ex;
}
return result;
}

I don't know the solution, but I know how you could get it. Create a dummy function that takes in an Expression<Func<ReferencedEntity, bool>> and pass it your lambda. And using a debugger you can examine how the compiler created the expression for you.