We Keep Coding

Record 'Lenses' - expression tree for with expression

Is there a way to build an expression tree for the new with operator?
I am trying to implement a 'Lens' feature for records that will only need a selector and will auto generate the mutator
My goal is to convert from a 'selector':
Expression<Func<T, TMember>> expression (ie employee => employee.Name)
To a 'mutator':
(employee, newName) => employee with { Name = newName }
I did manage to do this for the simple case above, see my answer below, however that will not work for a nested case ie:
record Employee(string Name, int Age);
record Manager(String Name, Employee Employee);
Here I want to change ie from
manager => manager.Employee.Name
to
(manager, newEmployeeName) => manager with { Employee = manager.Employee with { Name = newEmployeeName}}
Any help ?

CalcMutator method that can deal with nested properties would look something like this
static Func<T, TMember, T> CalcMutator(Expression<Func<T, TMember>> expression)
{
var typeParam = expression.Parameters.First();
var valueParam = Expression.Parameter(typeof(TMember), "v");
var variables = new List<ParameterExpression>();
var blockExpressions = new List<Expression>();
var property = (MemberExpression)expression.Body;
Expression currentValue = valueParam;
var index = 0;
while (property != null)
{
var variable = Expression.Variable(property.Expression.Type, $"v_{index}");
variables.Add(variable);
var cloneMethod = property.Expression.Type.GetMethod("<Clone>$");
if (cloneMethod is null) throw new Exception($"CalcMutatorNo Clone method on {typeof(T)}");
var cloneCall = Expression.Call(property.Expression, cloneMethod);
var assignClonedToVariable = Expression.Assign(variable, cloneCall);
var accessVariableProperty = Expression.MakeMemberAccess(variable, property.Member);
var assignVariablePropertyValue = Expression.Assign(accessVariableProperty, currentValue);
blockExpressions.Add(assignClonedToVariable);
blockExpressions.Add(assignVariablePropertyValue);
property = property.Expression as MemberExpression;
currentValue = variable;
index++;
}
// Return root object
blockExpressions.Add(currentValue);
var block = Expression.Block(variables, blockExpressions);
var assignLambda = (Expression<Func<T, TMember, T>>)Expression.Lambda(block, typeParam, valueParam);
return assignLambda.Compile();
}
Please keep in mind that Cache implemented with ImmutableDictionary is not thread safe. If you want to ensure that the cached expressions can safely be used in multi-threaded environments, it's better to use ConcurrentDictionary for the cache instead or to apply some synchronization primitives around ImmutableDictionary.

Following the lead from #JL0PD I ended up converting:
t => t.Member (ie employee => employee.Name)
into:
(t, v) => {
var c = t.<Clone>$();
c.Member = v;
return c;
}
ie:
(employee, newName) => {
var c = employee.<Clone>$();
c.Name=newName;
return c;
}
Below is a full implemetation of a record Lens including caching of delegates
Note that this does not cover nested mutators so my question above still stands
static class RecLens<T, TMember> {
public static (Func<T, TMember> Selector, Func<T, TMember, T> Mutator) Get(Expression<Func<T, TMember>> expression) {
if (!IsExpressionValid(expression.Body)) throw new Exception($"Lens Invalid expression ({expression})");
// create unique cache key, calc same key for x=>x.p and y=>y.p
var exprStr = expression.Body.ToString();
var dotPos = exprStr.IndexOf(Type.Delimiter);
var cacheKey = typeof(T).FullName + '|' + (dotPos > 0 ? exprStr.Remove(0, exprStr.IndexOf(Type.Delimiter) + 1) : "root");
if (!Cache.TryGetValue(cacheKey, out var res)) {
res = (expression.Compile(), CalcMutator(expression));
Cache = Cache.Add(cacheKey, res);
}
return res;
}
// key: "{srcType.FullName}|{member}" , ie: "Test.Organization|DevelopmentDepartment.Manager"
static ImmutableDictionary<string, (Func<T, TMember>, Func<T, TMember, T>)> Cache = ImmutableDictionary<string, (Func<T, TMember>, Func<T, TMember, T>)>.Empty;
// create delegate: (t, v) => { var c=t.<Clone>$(); c.Member = v; return c; }
static Func<T, TMember, T> CalcMutator(Expression<Func<T, TMember>> expression) {
var result = Expression.Variable(typeof(T), "c");
var typeParam = Expression.Parameter(typeof(T), "t");
var valueParam = Expression.Parameter(typeof(TMember), "v");
var cloneMethod = typeof(T).GetMethod("<Clone>$");
if (cloneMethod is null) throw new Exception($"CalcMutatorNo Clone method on {typeof(T)}");
var cloneCall = Expression.Call(typeParam, cloneMethod);
var assignResult = Expression.Assign(result, cloneCall);
var memberInfo = (expression.Body as MemberExpression)!.Member;
var resultMemberAccess = Expression.MakeMemberAccess(result, memberInfo);
var assign = Expression.Assign(resultMemberAccess, valueParam);
var block = Expression.Block(new[] { result }, assignResult, assign, result);
var assignLambda = (Expression<Func<T, TMember, T>>)Expression.Lambda(block, typeParam, valueParam);
return assignLambda.Compile();
}
// verify that expr is a member expression of its parameter
static bool IsExpressionValid(Expression expr, bool first = true) {
if (expr is ParameterExpression) return !first;
if (expr is MemberExpression memberExpr && memberExpr.Expression is object) return IsExpressionValid(memberExpr.Expression, false);
return false;
}
}
To use:
record Employee(string Name, int Age);
var (Selector, Mutator) = RecLens<Employee, string>.Get(e => e.Name);
var dave = new Employee("Dave", 30);
var name = Selector(dave); // "Dave"
var john = Mutator(dave, "John"); // Employee("John", 30)

Find and remove parameter declaration inside Expression.Block

I know how to replace a parameter with ExpressionVisitor but I was wondering if there's a way to remove a parameter from a Expression.Block.
Ideally I should crawl the entire Expression tree and remove the parameter every time it is declared inside a Block.
Any idea how to do that with ExpressionVisitor?

A simple class to remove local variables from a BlockExpression and replace them with whatever you want.
public class BlockVariableRemover : ExpressionVisitor
{
private readonly Dictionary<Expression, Expression> replaces = new Dictionary<Expression, Expression>();
public readonly Func<ParameterExpression, int, Expression> Replacer;
public BlockVariableRemover(Func<ParameterExpression, int, Expression> replacer)
{
Replacer = replacer;
}
protected override Expression VisitBlock(BlockExpression node)
{
var removed = new List<Expression>();
var variables = node.Variables.ToList();
for (int i = 0; i < variables.Count; i++)
{
var variable = variables[i];
var to = Replacer(variable, i);
if (to != variable)
{
removed.Add(variable);
replaces.Add(variable, to);
variables.RemoveAt(i);
i--;
}
}
if (removed.Count == 0)
{
return base.VisitBlock(node);
}
var expressions = node.Expressions.ToArray();
for (int i = 0; i < expressions.Length; i++)
{
expressions[i] = Visit(expressions[i]);
}
foreach (var rem in removed)
{
replaces.Remove(rem);
}
return Expression.Block(variables, expressions);
}
public override Expression Visit(Expression node)
{
Expression to;
if (node != null && replaces.TryGetValue(node, out to))
{
return base.Visit(to);
}
return base.Visit(node);
}
}
Use it like:
Expression<Func<int, int>> exp;
{
var var1 = Expression.Variable(typeof(int), "var1");
var var2 = Expression.Variable(typeof(long), "var2");
var par1 = Expression.Parameter(typeof(int), "par1");
var block = Expression.Block(new[] { var1, var2 }, Expression.Increment(var1));
exp = Expression.Lambda<Func<int, int>>(block, par1);
// Test
var compiled = exp.Compile();
Console.WriteLine(compiled(10));
}
// Begin replace
{
var par1 = exp.Parameters[0];
var block2 = new BlockVariableRemover(
// ix is the index of the variable,
// return x if you don't want to modify,
// return whatever you want (even Expression.Empty()) to do
// a replace
(x, ix) => ix == 0 && x.Type == typeof(int) ? par1 : x)
.Visit(exp.Body);
// Final result
var exp2 = Expression.Lambda<Func<int, int>>(block2, par1);
// Test
var compiled = exp2.Compile();
Console.WriteLine(compiled(10));
}

Evaluating complex expression tree

I have a basic rule engine that I've built in a very similar way to the route suggested here:
How to implement a rule engine?
Ive extended it based on further requirements, and now I need to evaluate complex classes eg
EvaluateRule("Transaction.IsOpen", "Equals", "true")
The code in its most basic form is:
var param = inputMessageType;
left = Expression.Property(param, memberName);
tProp = typeof(T).GetProperty(r.MemberName).PropertyType;
right = Expression.Constant(Convert.ChangeType(r.TargetValue, tProp));
return Expression.MakeBinary(tBinary, left, right);
In order to evaluate complex classes I used a method similar to here:
https://stackoverflow.com/questions/16544672/dynamically-evaluating-a-property-string-with-expressions
The problem that Im having is that when I try to evaluate the rule with a property of a class (Transaction.IsOpen), I get it with the type of the root type on the right hand side of the expression but the type of the complex object on the left hand side of the expression.
This results in the error:
System.InvalidOperationException: The binary operator Equal is not defined for the types 'System.Func`2[Transaction,System.Boolean]' and 'System.Boolean'.
How do I overcome this problem? I am no expert using Expression Trees, and many of the concepts are proving difficult to grasp when the example strays from the standard documentation.
Edit: Here is the code(Ive omitted some stuff that is environment specific so as keep focus with the problem)
public Actions EvaluateRulesFromMessage(ClientEventQueueMessage message)
{
var ruleGroups = _ruleRepository.GetRuleList();
var actions = new Actions();
foreach (var ruleGroup in ruleGroups)
{
if (message.MessageType == "UI_UPDATE")
{
// clean up json object
JObject dsPayload = (JObject.Parse(message.Payload));
var msgParams = JsonConvert.DeserializeObject<UiTransactionUpdate>(message.Payload);
msgParams.RulesCompleted = msgParams.RulesCompleted ?? new List<int>();
var conditionsMet = false;
// process the rules filtering out the rules that have already been evaluated
var filteredRules = ruleGroup.Rules.Where(item =>
!msgParams.RulesCompleted.Any(r => r.Equals(item.Id)));
foreach (var rule in filteredRules)
{
Func<UiTransactionUpdate, bool> compiledRule = CompileRule<UiTransactionUpdate>(rule, msgParams);
if (compiledRule(msgParams))
{
conditionsMet = true;
}
else
{
conditionsMet = false;
break;
}
}
if (conditionsMet)
{
actions = AddAction(message, ruleGroup);
break;
}
}
}
return actions;
}
public Func<UiTransactionUpdate, bool> CompileRule<T>(Rule r, UiTransactionUpdate msg)
{
var expression = Expression.Parameter(typeof(UiTransactionUpdate));
Expression expr = BuildExpr<UiTransactionUpdate>(r, expression, msg);
// build a lambda function UiTransactionUpdate->bool and compile it
return Expression.Lambda<Func<UiTransactionUpdate, bool>>(expr, expression).Compile();
}
static Expression Eval(object root, string propertyString, out Type tProp)
{
Type type = null;
var propertyNames = propertyString.Split('.');
ParameterExpression param = Expression.Parameter(root.GetType());
Expression property = param;
string propName = "";
foreach (var prop in propertyNames)
{
property = MemberExpression.PropertyOrField(property, prop);
type = property.Type;
propName = prop;
}
tProp = Type.GetType(type.UnderlyingSystemType.AssemblyQualifiedName);
var param2 = MemberExpression.Parameter(tProp);
var e = Expression.Lambda(property, param);
return e;
}
static Expression BuildExpr<T>(Rule r, ParameterExpression param, UiTransactionUpdate msg)
{
Expression left;
Type tProp;
string memberName = r.MemberName;
if (memberName.Contains("."))
{
left = Eval(msg, memberName, out tProp);
}
else
{
left = Expression.Property(param, memberName);
tProp = typeof(T).GetProperty(r.MemberName).PropertyType;
}
ExpressionType tBinary;
if (ExpressionType.TryParse(r.Operator, out tBinary))
{
Expression right=null;
switch (r.ValueType) ///todo: this needs to be refactored to be type independent
{
case TargetValueType.Value:
right = Expression.Constant(Convert.ChangeType(r.TargetValue, tProp));
break;
}
// use a binary operation ie true/false
return Expression.MakeBinary(tBinary, left, right);
}
else
{
var method = tProp.GetMethod(r.Operator);
var tParam = method.GetParameters()[0].ParameterType;
var right = Expression.Constant(Convert.ChangeType(r.TargetValue, tParam));
// use a method call, e.g. 'Contains' -> 'u.Tags.Contains(some_tag)'
return Expression.Call(left, method, right);
}
}

The sample code does not cover all data types used in your scenario, so it's hard to tell where it breaks exactly, but from the exception System.Func'2[Transaction,System.Boolean]' and 'System.Boolean it's obvious that on left hand you have a delegate that takes in Transaction and returns bool (Func<Transaction, bool>), and on the right hand you just have bool.
It's not possible to compare Func<Transaction, bool> to bool, but it's possible to call the function and compare its result:
Func<Transaction, bool> func = ...;
bool comparand = ...;
Transaction transaction = ...;
if (func(transaction) == comparand) { ... }
What translated to expression tree:
Expression funcExpression = ... /*LambdaExpression<Func<Transaction,bool>>*/;
Expression comparandExpression = Expression.Constant(true);
Expression transactionArg = /*e.g.*/Expression.Constant(transaction);
Expression funcResultExpression = Expression.Call(funcExpression, "Invoke", null, transactionArg);
Expression equalityTestExpression = Expression.Equal(funcResultExpression, comparandExpression);

Lambda Expression for dynamic Object

I am trying to build a Lambda Expression for a table that has been created at run time.
The Expression is build fine but when I call Compile() method I get this error
"ParameterExpression of type 'cseval.Item' cannot be used for delegate parameter of type 'System.Object'"
this is my function
public Func<dynamic, Boolean> GetWhereExp(List<WhereCondition> SearchFieldList, dynamic item)
{
ParameterExpression pe = Expression.Parameter(item.GetType(), "c");
Expression combined = null;
if (SearchFieldList != null)
{
foreach (WhereCondition fieldItem in SearchFieldList)
{
//Expression for accessing Fields name property
Expression columnNameProperty = Expression.Property(pe, fieldItem.ColumName);
//the name constant to match
Expression columnValue = Expression.Constant(fieldItem.Value);
//the first expression: PatientantLastName = ?
Expression e1 = Expression.Equal(columnNameProperty, columnValue);
if (combined == null)
{
combined = e;
}
else
{
combined = Expression.And(combined, e);
}
}
}
var result = Expression.Lambda<Func<dynamic, bool>>(combined, pe);
return result.Compile();
}

I've changed dynamic to generics, this code works for me:
public Func<T, Boolean> GetWhereExp<T>(List<WhereCondition> SearchFieldList, T item)
{
var pe = Expression.Parameter(item.GetType(), "c");
Expression combined = null;
if (SearchFieldList != null)
{
foreach (var fieldItem in SearchFieldList)
{
var columnNameProperty = Expression.Property(pe, fieldItem.ColumName);
var columnValue = Expression.Constant(fieldItem.Value);
var e1 = Expression.Equal(columnNameProperty, columnValue);
combined = combined == null ? e1 : Expression.And(combined, e1);
}
}
var result = Expression.Lambda<Func<T, bool>>(combined, pe);
return result.Compile();
}
Small remark: your method returns function, not an expression, so the name 'GetWhereExp' is slightly incorrect. If you want to return function, imho, it's better to use reflection.
UPD: I use this code to test:
var expressions = new List<WhereCondition>
{
new WhereCondition("Column1", "xxx"),
new WhereCondition("Column2", "yyy"),
};
var item = new
{
Column1 = "xxx",
Column2 = "yyy"
};
var func = LinqExpr.GetWhereExp(expressions, (dynamic)item);
Console.WriteLine(new[] {item}.Count(a => func(a)));

Parse string into int32 using expressions in C#

Basically I've wrote my own parser and I'm parsing a string into and expression and then compiling and storing to be reused later on.
For (an odd) example the type of string I'm parsing is this:-
if #name == 'max' and #legs > 5 and #ears > 5 then shoot()
The hash parts in the string are telling my parser to look at the properties on the object of type T that I pass in, if not found to look in a Dictionary that also gets passed in as extra.
I parse up the string into parts create an expression and join the expressions using methods from the PredicateBuilder.
I get the left value from where ever it may be and then the right value and turn it into an int32 or a string based on if it's wrapped in single quotes.. for now, then pass both into Expression.Equals/Expression.GreaterThan etc. dependant on the operator.
So far this works fine for equals and strings but take for example #ears which isn't a property on the object but is in the dictionary I end up with "does string equal int32" and it throws an exception.
I need to be able to parse the string from the dictionary into and int32 and then try the equal/greater/less than method.
Hopefully someone could shed some light on this as I haven't found anything yet that will do it and its driving me mad.
Here is the CreateExpression method I'm using to create an expression from the string.
private Expression<Func<T, IDictionary<string, string>, bool>> CreateExpression<T>(string condition)
{
string[] parts = condition.Split(new char[] { ' ' }, 3, StringSplitOptions.RemoveEmptyEntries);
if (parts.Length == 3)
{
var typeCache = _cache[typeof(T).FullName];
var param = Expression.Parameter(typeCache.T, "o");
var param2 = Expression.Parameter(typeof(IDictionary<string, string>), "d");
/* Convert right hand side into correct value type
*/
Expression right = null;
if (parts[2][0] == '\'' && parts[2][parts[2].Length - 1] == '\'')
right = Expression.Constant(parts[2].Trim(new char[] { '\'' }), typeof(string));
else
{
int x = 0;
right = (int.TryParse(parts[2].Trim(), out x))
? Expression.Constant(x)
: Expression.Constant(parts[2].Trim(new char[] { '\'' }), typeof(string));
}
/* Get the left hand side value from object T or IDictionary and then attempt to convert it
* into the right hand side value type
*/
Expression left = null;
var key = (parts[0][0] == '#') ? parts[0].TrimStart('#') : parts[0];
if (_cache[typeCache.T.FullName].Properties.Find(key, true) == null)
{
var m = typeof(ExpressionExtensions).GetMethod("GetValue");
left = Expression.Call(null, m, new Expression[] { param2, Expression.Constant(key) });
}
else
left = Expression.PropertyOrField(param, key);
/* Find the operator and return the correct expression
*/
if (parts[1] == "==")
{
return Expression.Lambda<Func<T, IDictionary<string, string>, bool>>(
Expression.Equal(left, right), new ParameterExpression[] { param, param2 });
}
else if (parts[1] == ">")
{
return Expression.Lambda<Func<T, IDictionary<string, string>, bool>>(
Expression.GreaterThan(left, right), new ParameterExpression[] { param, param2 });
}
}
return null;
}
And here is the method to parse the whole string into parts and build up the expression.
public Func<T, IDictionary<string, string>, bool> Parse<T>(string rule)
{
var type = typeof(T);
if (!_cache.ContainsKey(type.FullName))
_cache[type.FullName] = new TypeCacheItem()
{
T = type,
Properties = TypeDescriptor.GetProperties(type)
};
Expression<Func<T, IDictionary<string, string>, bool>> exp = null;
var actionIndex = rule.IndexOf("then");
if (rule.IndexOf("if") == 0 && actionIndex > 0)
{
var conditionStatement = rule.Substring(2, actionIndex - 2).Trim();
var actionStatement = rule.Substring(actionIndex + 4).Trim();
var startIndex = 0;
var endIndex = 0;
var conditionalOperator = "";
var lastConditionalOperator = "";
do
{
endIndex = FindConditionalOperator(conditionStatement, out conditionalOperator, startIndex);
var condition = (endIndex == -1) ? conditionStatement.Substring(startIndex) : conditionStatement.Substring(startIndex, endIndex - startIndex);
var x = CreateExpression<T>(condition.Trim());
if (x != null)
{
if (exp != null)
{
switch (lastConditionalOperator)
{
case "or":
exp = exp.Or<T>(x);
break;
case "and":
exp = exp.And<T>(x);
break;
default:
exp = x;
break;
}
}
else
exp = x;
}
lastConditionalOperator = conditionalOperator;
startIndex = endIndex + conditionalOperator.Length;
} while (endIndex > -1);
}
else
throw new ArgumentException("Rule must start with 'if' and contain 'then'.");
return (exp == null) ? null : exp.Compile();
}
My eyes are open to suggestions or advice on this but the idea of parsing that string as is to return true or false has to be,
EDIT:
I've now alter my method that retrieves the value from the dictionary to a generic one as Fahad suggested to this:-
public static T GetValue<T>(this IDictionary<string, string> dictionary, string key)
{
string x = string.Empty;
dictionary.TryGetValue(key, out x);
if (typeof(T) == typeof(int))
{
int i = 0;
if (int.TryParse(x, out i))
return (T)(i as object);
}
return default(T);
//throw new ArgumentException("Failed to convert dictionary value to type.");
}
And this works fine but I would rather return null than a default value for the type i've tried using Nullable ... where T : struct and returning null when not found or can't convert but I don't know how to use the null result in my expression.

I think you should redo your problem analysis and try another way. One way I would propose is to have a generic method that would give you the value from a IDictionary or IList etc., and use this method to wrap around your Expression's. The Expression would only want the "Type" of the object to be satisfied, if that is done correctly, then you could easily do it. If you know the "Type" then you can implement it with generic methods, otherwise, you could still use reflection and do generics.
So all you have to think is how to get the value from the dictionary or any other list through expressions.
Hope that helps.
-Fahad

You could use Expression.Call(null, typeof(int), "Parse", Type.EmptyTypes, yourTextExpression)