We Keep Coding

Record 'Lenses' - expression tree for with expression

Is there a way to build an expression tree for the new with operator?
I am trying to implement a 'Lens' feature for records that will only need a selector and will auto generate the mutator
My goal is to convert from a 'selector':
Expression<Func<T, TMember>> expression (ie employee => employee.Name)
To a 'mutator':
(employee, newName) => employee with { Name = newName }
I did manage to do this for the simple case above, see my answer below, however that will not work for a nested case ie:
record Employee(string Name, int Age);
record Manager(String Name, Employee Employee);
Here I want to change ie from
manager => manager.Employee.Name
to
(manager, newEmployeeName) => manager with { Employee = manager.Employee with { Name = newEmployeeName}}
Any help ?

CalcMutator method that can deal with nested properties would look something like this
static Func<T, TMember, T> CalcMutator(Expression<Func<T, TMember>> expression)
{
var typeParam = expression.Parameters.First();
var valueParam = Expression.Parameter(typeof(TMember), "v");
var variables = new List<ParameterExpression>();
var blockExpressions = new List<Expression>();
var property = (MemberExpression)expression.Body;
Expression currentValue = valueParam;
var index = 0;
while (property != null)
{
var variable = Expression.Variable(property.Expression.Type, $"v_{index}");
variables.Add(variable);
var cloneMethod = property.Expression.Type.GetMethod("<Clone>$");
if (cloneMethod is null) throw new Exception($"CalcMutatorNo Clone method on {typeof(T)}");
var cloneCall = Expression.Call(property.Expression, cloneMethod);
var assignClonedToVariable = Expression.Assign(variable, cloneCall);
var accessVariableProperty = Expression.MakeMemberAccess(variable, property.Member);
var assignVariablePropertyValue = Expression.Assign(accessVariableProperty, currentValue);
blockExpressions.Add(assignClonedToVariable);
blockExpressions.Add(assignVariablePropertyValue);
property = property.Expression as MemberExpression;
currentValue = variable;
index++;
}
// Return root object
blockExpressions.Add(currentValue);
var block = Expression.Block(variables, blockExpressions);
var assignLambda = (Expression<Func<T, TMember, T>>)Expression.Lambda(block, typeParam, valueParam);
return assignLambda.Compile();
}
Please keep in mind that Cache implemented with ImmutableDictionary is not thread safe. If you want to ensure that the cached expressions can safely be used in multi-threaded environments, it's better to use ConcurrentDictionary for the cache instead or to apply some synchronization primitives around ImmutableDictionary.

Following the lead from #JL0PD I ended up converting:
t => t.Member (ie employee => employee.Name)
into:
(t, v) => {
var c = t.<Clone>$();
c.Member = v;
return c;
}
ie:
(employee, newName) => {
var c = employee.<Clone>$();
c.Name=newName;
return c;
}
Below is a full implemetation of a record Lens including caching of delegates
Note that this does not cover nested mutators so my question above still stands
static class RecLens<T, TMember> {
public static (Func<T, TMember> Selector, Func<T, TMember, T> Mutator) Get(Expression<Func<T, TMember>> expression) {
if (!IsExpressionValid(expression.Body)) throw new Exception($"Lens Invalid expression ({expression})");
// create unique cache key, calc same key for x=>x.p and y=>y.p
var exprStr = expression.Body.ToString();
var dotPos = exprStr.IndexOf(Type.Delimiter);
var cacheKey = typeof(T).FullName + '|' + (dotPos > 0 ? exprStr.Remove(0, exprStr.IndexOf(Type.Delimiter) + 1) : "root");
if (!Cache.TryGetValue(cacheKey, out var res)) {
res = (expression.Compile(), CalcMutator(expression));
Cache = Cache.Add(cacheKey, res);
}
return res;
}
// key: "{srcType.FullName}|{member}" , ie: "Test.Organization|DevelopmentDepartment.Manager"
static ImmutableDictionary<string, (Func<T, TMember>, Func<T, TMember, T>)> Cache = ImmutableDictionary<string, (Func<T, TMember>, Func<T, TMember, T>)>.Empty;
// create delegate: (t, v) => { var c=t.<Clone>$(); c.Member = v; return c; }
static Func<T, TMember, T> CalcMutator(Expression<Func<T, TMember>> expression) {
var result = Expression.Variable(typeof(T), "c");
var typeParam = Expression.Parameter(typeof(T), "t");
var valueParam = Expression.Parameter(typeof(TMember), "v");
var cloneMethod = typeof(T).GetMethod("<Clone>$");
if (cloneMethod is null) throw new Exception($"CalcMutatorNo Clone method on {typeof(T)}");
var cloneCall = Expression.Call(typeParam, cloneMethod);
var assignResult = Expression.Assign(result, cloneCall);
var memberInfo = (expression.Body as MemberExpression)!.Member;
var resultMemberAccess = Expression.MakeMemberAccess(result, memberInfo);
var assign = Expression.Assign(resultMemberAccess, valueParam);
var block = Expression.Block(new[] { result }, assignResult, assign, result);
var assignLambda = (Expression<Func<T, TMember, T>>)Expression.Lambda(block, typeParam, valueParam);
return assignLambda.Compile();
}
// verify that expr is a member expression of its parameter
static bool IsExpressionValid(Expression expr, bool first = true) {
if (expr is ParameterExpression) return !first;
if (expr is MemberExpression memberExpr && memberExpr.Expression is object) return IsExpressionValid(memberExpr.Expression, false);
return false;
}
}
To use:
record Employee(string Name, int Age);
var (Selector, Mutator) = RecLens<Employee, string>.Get(e => e.Name);
var dave = new Employee("Dave", 30);
var name = Selector(dave); // "Dave"
var john = Mutator(dave, "John"); // Employee("John", 30)

How to append expressions in linq?

Net core application. I have a generic repository pattern implemented. I am trying to implement some filtering functionality. I have the below code.
var param = Expression.Parameter(typeof(SiteAssessmentRequest), "x");
Expression<Func<SiteAssessmentRequest, bool>> query;
query = x => x.CreatedBy == request.Userid || x.AssignedTo == request.Userid;
Expression body = Expression.Invoke(query, param);
if (request.Client != null && request.Client.Length != 0)
{
Expression<Func<SiteAssessmentRequest, bool>> internalQuery = x => request.Client.Contains(x.Client);
body = Expression.AndAlso(Expression.Invoke(query, param), Expression.Invoke(internalQuery, param));
}
if (request.CountryId != null && request.CountryId.Length != 0)
{
Expression<Func<SiteAssessmentRequest, bool>> internalQuery = x => request.CountryId.Contains(x.CountryId);
body = Expression.AndAlso(Expression.Invoke(query, param), Expression.Invoke(internalQuery, param));
}
if (request.SiteName != null && request.SiteName.Length != 0)
{
Expression<Func<SiteAssessmentRequest, bool>> internalQuery = x => request.SiteName.Contains(x.SiteName);
body = Expression.AndAlso(Expression.Invoke(query, param), Expression.Invoke(internalQuery, param));
}
if (request.Status != null && request.Status.Length != 0)
{
Expression<Func<SiteAssessmentRequest, bool>> internalQuery = x => request.Status.Contains(x.Status);
body = Expression.AndAlso(Expression.Invoke(query, param), Expression.Invoke(internalQuery, param));
}
var lambda = Expression.Lambda<Func<SiteAssessmentRequest, bool>>(body, param);
var siteAssessmentRequest = await _siteAssessmentRequestRepository.GetAsync(lambda, null, x => x.Country).ConfigureAwait(false);
In the above code when I pass more than one parameter, for example, request. Status and request.SiteName I want to filter based on status and Sitename. When I see the query only one parameter appended in the query
{x => (Invoke(x => (Not(IsNullOrEmpty(x.CreatedBy)) AndAlso Not(IsNullOrWhiteSpace(x.CreatedBy))), x)
AndAlso Invoke(x => value(Site.V1.Implementation.GetSARByFilterAr+<>c__DisplayClass12_0)
.request.Status.Contains(x.Status), x))}
After searching so much I got below code
public static class ExpressionCombiner
{
public static Expression<Func<T, bool>> And<T>(this Expression<Func<T, bool>> exp, Expression<Func<T, bool>> newExp)
{
// get the visitor
var visitor = new ParameterUpdateVisitor(newExp.Parameters.First(), exp.Parameters.First());
// replace the parameter in the expression just created
newExp = visitor.Visit(newExp) as Expression<Func<T, bool>>;
// now you can and together the two expressions
var binExp = Expression.And(exp.Body, newExp.Body);
// and return a new lambda, that will do what you want. NOTE that the binExp has reference only to te newExp.Parameters[0] (there is only 1) parameter, and no other
return Expression.Lambda<Func<T, bool>>(binExp, newExp.Parameters);
}
class ParameterUpdateVisitor : ExpressionVisitor
{
private ParameterExpression _oldParameter;
private ParameterExpression _newParameter;
public ParameterUpdateVisitor(ParameterExpression oldParameter, ParameterExpression newParameter)
{
_oldParameter = oldParameter;
_newParameter = newParameter;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (object.ReferenceEquals(node, _oldParameter))
return _newParameter;
return base.VisitParameter(node);
}
}
}
But I am struggling to make it to work above code.
In the above query, I see only status but not site name. So I want to include multiple expressions. can someone help me regarding this? Any help would be greatly appreciated. Thanks

Let me explain core concept of LambdaExpression. LambdaExpression has 0..N parameters and Body. Body of LambdaExpression is Expression which actually we want to reuse.
Basic mistake is trying to combine lambda expressions (changed parameter names because it is how ExpressionTree works - it compares parameters by reference, not by name):
Expression<Func<Some, bool>> lambda1 = x1 => x1.Id == 10;
Expression<Func<Some, bool>> lambda2 = x2 => x2.Value == 20;
// wrong
var resultExpression = Expression.AndAlso(lambda1, lambda2);
Schematically previous wrong sample should looks like (even it will crash)
(x1 => x1.Id == 10) && (x2 => x2.Value == 20)
What we have - not desired expression but combination of "functions".
So let reuse body of LambdaExpression
// not complete
var newBody = Expression.AndAlso(lambda1.Body, lambda2.Body);
Result is more acceptable but still needs corrections:
(x1.Id == 10) && (x2.Value == 20)
Why we need correction? Because we are trying to build the following LambdaExpression
var param = Expression.Parameter(typeof(some), "e");
var newBody = Expression.AndAlso(lambda1.Body, lambda2.Body);
// still wrong
var newPredicate = Expression.Lambda<Func<Some, bool>>(newBody, param)
Result of newPredicate should looks like
e => (x1.Id == 10) && (x2.Value == 20)
As you can see we left in body parameters from previous two lambdas, which is wrong and we have to replace them with new parameter param ("e") and better to do that before combination.
I'm using my implementation of replacer, which just returns desired body.
So, let's write correct lambda!
Expression<Func<Some, bool>> lambda1 = x1 => x1.Id == 10;
Expression<Func<Some, bool>> lambda2 = x2 => x2.Value == 20;
var param = Expression.Parameter(typeof(Some), "e");
var newBody = Expression.AndAlso(
ExpressionReplacer.GetBody(lambda1, param),
ExpressionReplacer.GetBody(lambda2, param));
// hurray!
var newPredicate = Expression.Lambda<Func<Some, bool>>(newBody, param);
var query = query.Where(newPredicate);
After theses steps you will have desired result:
e => (e.Id == 10) && (e.Value == 20)
And ExpressionReplacer implementation
class ExpressionReplacer : ExpressionVisitor
{
readonly IDictionary<Expression, Expression> _replaceMap;
public ExpressionReplacer(IDictionary<Expression, Expression> replaceMap)
{
_replaceMap = replaceMap ?? throw new ArgumentNullException(nameof(replaceMap));
}
public override Expression Visit(Expression exp)
{
if (exp != null && _replaceMap.TryGetValue(exp, out var replacement))
return replacement;
return base.Visit(exp);
}
public static Expression Replace(Expression expr, Expression toReplace, Expression toExpr)
{
return new ExpressionReplacer(new Dictionary<Expression, Expression> { { toReplace, toExpr } }).Visit(expr);
}
public static Expression Replace(Expression expr, IDictionary<Expression, Expression> replaceMap)
{
return new ExpressionReplacer(replaceMap).Visit(expr);
}
public static Expression GetBody(LambdaExpression lambda, params Expression[] toReplace)
{
if (lambda.Parameters.Count != toReplace.Length)
throw new InvalidOperationException();
return new ExpressionReplacer(Enumerable.Zip(lambda.Parameters, toReplace, (f, s) => Tuple.Create(f, s))
.ToDictionary(e => (Expression)e.Item1, e => e.Item2)).Visit(lambda.Body);
}
}
If you plan to work with ExpressionTree closer, I suggest to install this VS extension, which should simplify your life: Readable Expressions

Column Mapping Using Expressions and Dictionary

I'm having trouble coding an expression for IQueryable object. I can't figure out the right way to use the expression to map a string to object properties.
this is my query object and mapping dictionary:
var query = context.Industries.AsQueryable();
var columnsMap = new Dictionary<string, Expression<Func<Industry, object>>>()
{
["name"] = v => v.Name,
["isicCode"] = v => v.IsicCode.Data,
["isicCodeTitle"] = v => v.IsicCode.Title,
["isActive"] = v => v.IsActive,
};
and I'm using columnsMap dictionary for applying Orderby to my query in an extentions class:
public static IQueryable<T> ApplyOrdering<T>(this IQueryable<T> query, IQueryObject queryObj, Dictionary<string, Expression<Func<T, object>>> columnsMap)
{
if (String.IsNullOrWhiteSpace(queryObj.SortBy) || !columnsMap.ContainsKey(queryObj.SortBy))
return query;
if (queryObj.IsSortAsc)
return query.OrderBy(columnsMap[queryObj.SortBy]);
else
return query.OrderByDescending(columnsMap[queryObj.SortBy]);
}
it's ok for OrderBy but I need to do the same thing for ApplyFiltering but for filter an IQueryable object I need a different expression for where method Expression<Func<T, bool>>
public static IQueryable<T> ApplyFiltering<T>(this IQueryable<T> query, IQueryObject queryObj, Dictionary<string, Expression<Func<T, object>>> columnsMap)
{
query.Where(columnsMap['name'] == "test Name"); //this is the problem.
return query;
}
the question is how can I use my columnsMap in ApplyFiltering method? or should I change my columnsMap for that?

It is quite simple. The only problem is when you have a value type column (an int or a bool or a DateTime for example)... An Expression<Func<Industry, object>> will introduce a boxing of the field/property to object that we have to remove. This problem is absent for strings.
// isActive is a bool
Expression<Func<Industry, object>> exp = columnsMap["isActive"];
object value = true; // You can't use "true" (string) here! isActive is a bool
// Other exammple
// Expression<Func<Industry, object>> exp = columnsMap["name"];
// object value = "Foo";
var body = exp.Body;
// Remove the boxing for value types
if (body.NodeType == ExpressionType.Convert)
{
body = ((UnaryExpression)body).Operand;
}
var eq = Expression.Equal(body, Expression.Constant(value, body.Type));
var exp2 = Expression.Lambda<Func<T, bool>>(eq, exp.Parameters);
return query.Where(exp2);
Example using string value:
Expression<Func<Industry, object>> exp = columnsMap["isActive"];
string value = "true";
// Other exammple
//Expression<Func<Industry, object>> exp = columnsMap["name"];
//string value = "Foo";
var body = exp.Body;
// Remove the boxing for value types
if (body.NodeType == ExpressionType.Convert)
{
body = ((UnaryExpression)body).Operand;
}
object value2 = value;
if (value2 != null && body.Type != value2.GetType())
{
value2 = Convert.ChangeType(value2, body.Type);
}
var eq = Expression.Equal(body, Expression.Constant(value2, body.Type));
var exp2 = Expression.Lambda<Func<Industry, bool>>(eq, exp.Parameters);
return query.Where(exp2);

Like operator or using wildcards in LINQ to Entities

I'm using LINQ 2 Entities.
Following is the problem:
string str = '%test%.doc%'
.Contains(str) // converts this into LIKE '%~%test~%.doc~%%'
Expected Conversion: LIKE '%test%.doc%'
If it was LINQ 2 SQL, I could have used SqlMethods.Like as somebody answered it in my previous question. But now as I'm using L2E not L2S, I need other solution.

The SQL method PATINDEX provides the same functionality as LIKE. Therefore, you can use the SqlFunctions.PatIndex method:
.Where(x => SqlFunctions.PatIndex("%test%.doc%", x.MySearchField) > 0)

Following on from Magnus' correct answer, here is an extension method that can be re-used, as I needed in my project.
public static class LinqExtensions
{
public static Expression<Func<T, bool>> WildCardWhere<T>(this Expression<Func<T, bool>> source, Expression<Func<T, string>> selector, string terms, char separator)
{
if (terms == null || selector == null)
return source;
foreach (string term in terms.Split(new[] { separator }, StringSplitOptions.RemoveEmptyEntries))
{
string current = term;
source = source.And(
Expression.Lambda<Func<T, bool>>(
Expression.Call(selector.Body, "Contains", null, Expression.Constant(current)),
selector.Parameters[0]
)
);
}
return source;
}
}
Usage:
var terms = "%test%.doc%";
Expression<Func<Doc, bool>> whereClause = d => d;
whereClause = whereClause.WildCardWhere(d => d.docName, terms, '%');
whereClause = whereClause.WildCardWhere(d => d.someOtherProperty, "another%string%of%terms", '%');
var result = ListOfDocs.Where(whereClause).ToList();
The extension makes use of the predicate builder at http://petemontgomery.wordpress.com/2011/02/10/a-universal-predicatebuilder/. The resulting sql does a single table scan of the table, no matter how many terms are in there. Jo Vdb has an example you could start from if you wanted an extension of iQueryable instead.

You can try use this article, where author describes how to build a LIKE statement with wildcard characters in LINQ to Entities.
EDIT: Since the original link is now dead, here is the original extension class (as per Jon Koeter in the comments) and usage example.
Extension:
public static class LinqHelper
{
//Support IQueryable (Linq to Entities)
public static IQueryable<TSource> WhereLike<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, string>> valueSelector, string value, char wildcard)
{
return source.Where(BuildLikeExpression(valueSelector, value, wildcard));
}
//Support IEnumerable (Linq to objects)
public static IEnumerable<TSource> WhereLike<TSource>(this IEnumerable<TSource> sequence, Func<TSource, string> expression, string value, char wildcard)
{
var regEx = WildcardToRegex(value, wildcard);
//Prevent multiple enumeration:
var arraySequence = sequence as TSource[] ?? sequence.ToArray();
try
{
return arraySequence.Where(item => Regex.IsMatch(expression(item), regEx));
}
catch (ArgumentNullException)
{
return arraySequence;
}
}
//Used for the IEnumerable support
private static string WildcardToRegex(string value, char wildcard)
{
return "(?i:^" + Regex.Escape(value).Replace("\\" + wildcard, "." + wildcard) + "$)";
}
//Used for the IQueryable support
private static Expression<Func<TElement, bool>> BuildLikeExpression<TElement>(Expression<Func<TElement, string>> valueSelector, string value, char wildcard)
{
if (valueSelector == null) throw new ArgumentNullException("valueSelector");
var method = GetLikeMethod(value, wildcard);
value = value.Trim(wildcard);
var body = Expression.Call(valueSelector.Body, method, Expression.Constant(value));
var parameter = valueSelector.Parameters.Single();
return Expression.Lambda<Func<TElement, bool>>(body, parameter);
}
private static MethodInfo GetLikeMethod(string value, char wildcard)
{
var methodName = "Equals";
var textLength = value.Length;
value = value.TrimEnd(wildcard);
if (textLength > value.Length)
{
methodName = "StartsWith";
textLength = value.Length;
}
value = value.TrimStart(wildcard);
if (textLength > value.Length)
{
methodName = (methodName == "StartsWith") ? "Contains" : "EndsWith";
}
var stringType = typeof(string);
return stringType.GetMethod(methodName, new[] { stringType });
}
}
Usage Example:
string strEmailToFind = "%#yahoo.com"
IQueryable<User> myUsers = entities.Users.WhereLike(u => u.EmailAddress, strEmailToFind, '%');
or, if you expect your users to be more accustomed to Windows Explorer-styled wildcards:
string strEmailToFind = "*#yahoo.com"
IQueryable<User> myUsers = entities.Users.WhereLike(u => u.EmailAddress, strEmailToFind, '*');

Use a regular expression...
The following will print out all of the files in the current directory that match test.doc* (dos wildcard style - which I believe is what you're asking for)
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.IO;
namespace RegexFileTester
{
class Program
{
static void Main(string[] args)
{
string[] _files = Directory.GetFiles(".");
var _fileMatches = from i in _files
where Regex.IsMatch(i, ".*test*.doc.*")
//where Regex.IsMatch(i, ".*cs")
select i;
foreach(var _file in _fileMatches)
{
Console.WriteLine(_file);
}
}
}
}

Split the String
var str = "%test%.doc%";
var arr = str.Split(new[]{'%'} ,StringSplitOptions.RemoveEmptyEntries);
var q = tblUsers.Select (u => u);
foreach (var item in arr)
{
var localItem = item;
q = q.Where (x => x.userName.Contains(localItem));
}

So I was trying the same thing - trying to pair down a List to return all candidates that matched a SearchTerm. I wanted it so that if a user typed "Arizona" it would return everything regardless of case that had Arizona. Also, if the user typed "Arizona Cha", it would return items like "Arizona License Change". The following worked:
private List<Certification> GetCertListBySearchString()
{
string[] searchTerms = SearchString.Split(' ');
List<Certification> allCerts = _context.Certifications.ToList();
allCerts = searchTerms.Aggregate(allCerts, (current, thisSearchString) => (from ac in current
where ac.Name.ToUpper().Contains(thisSearchString.ToUpper())
select ac).ToList());
return allCerts;
}

Why my LINQ expression do not work in LINQ to SQL?

private System.Linq.Expressions.Expression<Func<ActionLogs, bool>> GetExpression()
{
Expression<Func<ActionLogs, bool>> expr = w => w.ID != -1;
if (ActionDate != null)
{
Expression<Func<ActionLogs, bool>> byDate = w => w.DateAction == ActionDate;
var body = Expression.AndAlso(expr.Body, byDate.Body);
expr = Expression.Lambda<Func<ActionLogs, bool>>(body, expr.Parameters[0]);
}
if (ActionType != ActionTypeEnum.Empty)
{
Expression<Func<ActionLogs, bool>> byActionType = w => w.ActionTypeID == (int)ActionType;
var body = Expression.AndAlso(expr.Body, byActionType.Body);
expr = Expression.Lambda<Func<ActionLogs, bool>>(body, expr.Parameters[0]);
}
if (!String.IsNullOrWhiteSpace(AuthorLogin))
{
Expression<Func<ActionLogs, bool>> byLogin = w => w.User.LoginName == AuthorLogin;
var body = Expression.AndAlso(expr.Body, byLogin.Body);
expr = Expression.Lambda<Func<ActionLogs, bool>>(body, expr.Parameters[0]);
}
if (!String.IsNullOrWhiteSpace(AdditionalInfo))
{
Expression<Func<ActionLogs, bool>> byAdditionalInfo = w => w.DescriptionText.Contains(AdditionalInfo);
var body = Expression.AndAlso(expr.Body, byAdditionalInfo.Body);
expr = Expression.Lambda<Func<ActionLogs, bool>>(body, expr.Parameters[0]);
}
return expr;
}
This is the function that generate my expression.
and when i do this:
System.Linq.Expressions.Expression<Func<ActionLogs, bool>> expr = GetExpression();
result = blablabla.Where(expr);
It says to me that The parameter 'w' is not in scope.
So the question is, how i can generate my expression, that depends on something i need, and paste it into LINQ to SQL query?

public Expression<Func<T, bool>> Combine<T>(Expression<Func<T, Boolean>> first, Expression<Func<T, Boolean>> second)
{
var toInvoke = Expression.Invoke(second, first.Parameters.Cast<Expression>());
return (Expression.Lambda<Func<T, Boolean>>(Expression.AndAlso(first.Body, toInvoke), first.Parameters));
}
This function was very helpful. It combined two expressions and built correct tree.

Could it be that you are trying to return an Expression.Lambda<T> and then assign it to an object that is a SON of it on its hierarchy(System.Linq.Expressions.Expression<T>)
Please take a look at http://msdn.microsoft.com/en-us/library/system.linq.expressions.lambdaexpression.aspx
Hope that helps,