We Keep Coding

Record 'Lenses' - expression tree for with expression

Is there a way to build an expression tree for the new with operator?
I am trying to implement a 'Lens' feature for records that will only need a selector and will auto generate the mutator
My goal is to convert from a 'selector':
Expression<Func<T, TMember>> expression (ie employee => employee.Name)
To a 'mutator':
(employee, newName) => employee with { Name = newName }
I did manage to do this for the simple case above, see my answer below, however that will not work for a nested case ie:
record Employee(string Name, int Age);
record Manager(String Name, Employee Employee);
Here I want to change ie from
manager => manager.Employee.Name
to
(manager, newEmployeeName) => manager with { Employee = manager.Employee with { Name = newEmployeeName}}
Any help ?

CalcMutator method that can deal with nested properties would look something like this
static Func<T, TMember, T> CalcMutator(Expression<Func<T, TMember>> expression)
{
var typeParam = expression.Parameters.First();
var valueParam = Expression.Parameter(typeof(TMember), "v");
var variables = new List<ParameterExpression>();
var blockExpressions = new List<Expression>();
var property = (MemberExpression)expression.Body;
Expression currentValue = valueParam;
var index = 0;
while (property != null)
{
var variable = Expression.Variable(property.Expression.Type, $"v_{index}");
variables.Add(variable);
var cloneMethod = property.Expression.Type.GetMethod("<Clone>$");
if (cloneMethod is null) throw new Exception($"CalcMutatorNo Clone method on {typeof(T)}");
var cloneCall = Expression.Call(property.Expression, cloneMethod);
var assignClonedToVariable = Expression.Assign(variable, cloneCall);
var accessVariableProperty = Expression.MakeMemberAccess(variable, property.Member);
var assignVariablePropertyValue = Expression.Assign(accessVariableProperty, currentValue);
blockExpressions.Add(assignClonedToVariable);
blockExpressions.Add(assignVariablePropertyValue);
property = property.Expression as MemberExpression;
currentValue = variable;
index++;
}
// Return root object
blockExpressions.Add(currentValue);
var block = Expression.Block(variables, blockExpressions);
var assignLambda = (Expression<Func<T, TMember, T>>)Expression.Lambda(block, typeParam, valueParam);
return assignLambda.Compile();
}
Please keep in mind that Cache implemented with ImmutableDictionary is not thread safe. If you want to ensure that the cached expressions can safely be used in multi-threaded environments, it's better to use ConcurrentDictionary for the cache instead or to apply some synchronization primitives around ImmutableDictionary.

Following the lead from #JL0PD I ended up converting:
t => t.Member (ie employee => employee.Name)
into:
(t, v) => {
var c = t.<Clone>$();
c.Member = v;
return c;
}
ie:
(employee, newName) => {
var c = employee.<Clone>$();
c.Name=newName;
return c;
}
Below is a full implemetation of a record Lens including caching of delegates
Note that this does not cover nested mutators so my question above still stands
static class RecLens<T, TMember> {
public static (Func<T, TMember> Selector, Func<T, TMember, T> Mutator) Get(Expression<Func<T, TMember>> expression) {
if (!IsExpressionValid(expression.Body)) throw new Exception($"Lens Invalid expression ({expression})");
// create unique cache key, calc same key for x=>x.p and y=>y.p
var exprStr = expression.Body.ToString();
var dotPos = exprStr.IndexOf(Type.Delimiter);
var cacheKey = typeof(T).FullName + '|' + (dotPos > 0 ? exprStr.Remove(0, exprStr.IndexOf(Type.Delimiter) + 1) : "root");
if (!Cache.TryGetValue(cacheKey, out var res)) {
res = (expression.Compile(), CalcMutator(expression));
Cache = Cache.Add(cacheKey, res);
}
return res;
}
// key: "{srcType.FullName}|{member}" , ie: "Test.Organization|DevelopmentDepartment.Manager"
static ImmutableDictionary<string, (Func<T, TMember>, Func<T, TMember, T>)> Cache = ImmutableDictionary<string, (Func<T, TMember>, Func<T, TMember, T>)>.Empty;
// create delegate: (t, v) => { var c=t.<Clone>$(); c.Member = v; return c; }
static Func<T, TMember, T> CalcMutator(Expression<Func<T, TMember>> expression) {
var result = Expression.Variable(typeof(T), "c");
var typeParam = Expression.Parameter(typeof(T), "t");
var valueParam = Expression.Parameter(typeof(TMember), "v");
var cloneMethod = typeof(T).GetMethod("<Clone>$");
if (cloneMethod is null) throw new Exception($"CalcMutatorNo Clone method on {typeof(T)}");
var cloneCall = Expression.Call(typeParam, cloneMethod);
var assignResult = Expression.Assign(result, cloneCall);
var memberInfo = (expression.Body as MemberExpression)!.Member;
var resultMemberAccess = Expression.MakeMemberAccess(result, memberInfo);
var assign = Expression.Assign(resultMemberAccess, valueParam);
var block = Expression.Block(new[] { result }, assignResult, assign, result);
var assignLambda = (Expression<Func<T, TMember, T>>)Expression.Lambda(block, typeParam, valueParam);
return assignLambda.Compile();
}
// verify that expr is a member expression of its parameter
static bool IsExpressionValid(Expression expr, bool first = true) {
if (expr is ParameterExpression) return !first;
if (expr is MemberExpression memberExpr && memberExpr.Expression is object) return IsExpressionValid(memberExpr.Expression, false);
return false;
}
}
To use:
record Employee(string Name, int Age);
var (Selector, Mutator) = RecLens<Employee, string>.Get(e => e.Name);
var dave = new Employee("Dave", 30);
var name = Selector(dave); // "Dave"
var john = Mutator(dave, "John"); // Employee("John", 30)

How to convert string to System.Linq.Expressions.Expression

I have a grid which shows records from a table. On this grid I am using customized pagination and sorting, so I need to use customized column filtering as well.
var expression = ExpressionBuilder.Expression<EventModel>(request.Filters);
The above code snippet gets filter condition from Kendo Grid in the controller of type System.Linq.Expressions.Expression<Func<EventModel,bool>> expression,
and am converting it to string, and passing it to DAL code as shown below,
string filterExpression = ExpressionBuilder.Expression<EventModel>(request.Filters).ToString();
List<EventModel> eventModelList = new List<EventModel>();
eventModelList = eventComponent.GetEventData(request.PageSize, request.Page, searchstring, sortDirection, sortColumnName, filterExpression, ref recCount);
In the DAL I need to convert filterExpression from string to System.Linq.Expressions.Expression<Func<EventModel,bool>>
var res = eventInfo.AsQueryable().Where(filterExpression);//Gets error here
lstEventInfo = lstEventInfo.AsQueryable().Where(res);
Am getting an error can not convert from string to System.Linq.Expressions.Expression>'.
So could anyone tell me how could I convert a string to System.Linq.Expressions.Expression<System.Func<ModelClass,bool>> type in C#.

Here is a simple example how to create where dynamically.
public class Mock
{
public int Id { get; set; }
public int ForeignId { get; set; }
public decimal Total { get; set; }
}
class Program
{
static void Main(string[] args)
{
var list = new List<Mock>()
{
new Mock{
Id = 1,
ForeignId = 1,
Total = 100,
},
};
var query = list.AsQueryable();
// t
var parameter = Expression.Parameter(typeof(Mock), "t");
// t.Total
var propertyExpression = Expression.PropertyOrField(parameter, "Total");
// 100.00M
var constant = Expression.Constant(100M, typeof(decimal));
// t.Total == 100.00M
var equalExpression = Expression.Equal(propertyExpression, constant);
// t => t.Total == 100.00M
var lambda = Expression.Lambda(equalExpression, parameter);
// calls where.
var whereExpression = Expression.Call(typeof(Queryable), "Where", new[] { query.ElementType }, query.Expression, lambda);
// add where to query.
query = query.Provider.CreateQuery(whereExpression) as IQueryable<Mock>;
Console.ReadKey();
}
}
But You can use this
https://github.com/PoweredSoft/DynamicLinq
here is the Nuget package
https://www.nuget.org/packages/PoweredSoft.DynamicLinq/
There is a litle sample here how to do simple web filtering
https://github.com/PoweredSoft/DynamicLinq#how-it-can-be-used-in-a-web-api
You could adapt it to fit your filter expression model.
[HttpGet][Route("FindClients")]
public IHttpActionResult FindClients(string filterField = null, string filterValue = null,
string sortProperty = "Id", int? page = null, int pageSize = 50)
{
var ctx = new MyDbContext();
var query = ctx.Clients.AsQueryable();
if (!string.IsNullOrEmpty(filterField) && !string.IsNullOrEmpty(filterValue))
query = query.Query(t => t.Contains(filterField, filterValue)).OrderBy(sortProperty);
// count.
var clientCount = query.Count();
int? pages = null;
if (page.HasValue && pageSize > 0)
{
if (clientCount == 0)
pages = 0;
else
pages = clientCount / pageSize + (clientCount % pageSize != 0 ? 1 : 0);
}
if (page.HasValue)
query = query.Skip((page.Value-1) * pageSize).Take(pageSize);
var clients = query.ToList();
return Ok(new
{
total = clientCount,
pages = pages,
data = clients
});
}
An alternative is to use DynamicLinq
https://weblogs.asp.net/scottgu/dynamic-linq-part-1-using-the-linq-dynamic-query-library

I wrote this code snippet for converting from string to Expression.
public List<Product> Get(string filter = null)
{
var p = Expression.Parameter(typeof(Product), "x");
var e = (Expression)DynamicExpressionParser.ParseLambda(new[] { p }, null, filter);
var typedExpression = (Expression<Func<Product, bool>>)e;
var res = _productDal.GetList(typedExpression);
return res;
}
I used System.Linq.Dynamic.Core namespace for Asp.Net Core. You can use System.Linq.Dynamic namespace for Asp.Net.
If you use classic Asp.Net instead of Asp.Net Core, you should write
var e = (Expression)DynamicExpression.ParseLambda(new[] { p }, null, filter); instead of
var e = (Expression)DynamicExpressionParser.ParseLambda(new[] { p }, null, filter);
And your string parameter (which is named as a filter) should be like "(x.ProductID > 10)"
If your string parameter is different, you can use following code snippet to convert from expression to string for getting same string parameter into Get method.
public static string Select(this Grid _grid, Expression<Func<Product, bool>> filter = null)
{
//filter = {x => (x.ProductID > 1)}
BinaryExpression be = filter.Body as BinaryExpression;
//be = {(x.ProductID > 1)}
return be.ToString();
}

Lambda Expression for dynamic Object

I am trying to build a Lambda Expression for a table that has been created at run time.
The Expression is build fine but when I call Compile() method I get this error
"ParameterExpression of type 'cseval.Item' cannot be used for delegate parameter of type 'System.Object'"
this is my function
public Func<dynamic, Boolean> GetWhereExp(List<WhereCondition> SearchFieldList, dynamic item)
{
ParameterExpression pe = Expression.Parameter(item.GetType(), "c");
Expression combined = null;
if (SearchFieldList != null)
{
foreach (WhereCondition fieldItem in SearchFieldList)
{
//Expression for accessing Fields name property
Expression columnNameProperty = Expression.Property(pe, fieldItem.ColumName);
//the name constant to match
Expression columnValue = Expression.Constant(fieldItem.Value);
//the first expression: PatientantLastName = ?
Expression e1 = Expression.Equal(columnNameProperty, columnValue);
if (combined == null)
{
combined = e;
}
else
{
combined = Expression.And(combined, e);
}
}
}
var result = Expression.Lambda<Func<dynamic, bool>>(combined, pe);
return result.Compile();
}

I've changed dynamic to generics, this code works for me:
public Func<T, Boolean> GetWhereExp<T>(List<WhereCondition> SearchFieldList, T item)
{
var pe = Expression.Parameter(item.GetType(), "c");
Expression combined = null;
if (SearchFieldList != null)
{
foreach (var fieldItem in SearchFieldList)
{
var columnNameProperty = Expression.Property(pe, fieldItem.ColumName);
var columnValue = Expression.Constant(fieldItem.Value);
var e1 = Expression.Equal(columnNameProperty, columnValue);
combined = combined == null ? e1 : Expression.And(combined, e1);
}
}
var result = Expression.Lambda<Func<T, bool>>(combined, pe);
return result.Compile();
}
Small remark: your method returns function, not an expression, so the name 'GetWhereExp' is slightly incorrect. If you want to return function, imho, it's better to use reflection.
UPD: I use this code to test:
var expressions = new List<WhereCondition>
{
new WhereCondition("Column1", "xxx"),
new WhereCondition("Column2", "yyy"),
};
var item = new
{
Column1 = "xxx",
Column2 = "yyy"
};
var func = LinqExpr.GetWhereExp(expressions, (dynamic)item);
Console.WriteLine(new[] {item}.Count(a => func(a)));

Dynamically Modifying a LINQ Query Based on a Related Table

I am dynamically creating a LINQ query based on various search criteria.
As an example, let's say I am searching a Automobiles table, and I have an option to filter by ratings. I have two controls:
Compare type: [At least], [At most], [Less than], [Greater than], and [Equal].
Value: The value to compare the rating against.
So the user could, for example, select the compare type [At least] and the value 3, and my code needs to create a query that limits results to automobile ratings greater than or equal to 3.
I found a great solution given by VinayC in the question How to implement search functionality in C#/ASP.NET MVC. His DynamicWhere() method dynamically creates part of the expression that would produce the correct filter.
My problem is that my primary query type is Automobile but my ratings are in a separate table (Automobile.Ratings). How could I implement this same technique and filter on a type other than my primary query type?
Thanks for any tips.

Since the number of operations that you have is small, finite, and known at comiple time, you can simply handle it with a switch:
IQueryable<Something> query = GetQuery();
int ratingToComareWith = 1;
string operation = "Equal";
switch (operation)
{
case ("Equal"):
query = query.Where(item => item == ratingToComareWith);
break;
case ("Less Than"):
query = query.Where(item => item < ratingToComareWith);
break;
}

Here's an Entity Framework friendly alternative to expression building. Of course you will want to validate column and op to prevent SQL injection.
// User provided values
int value = 3;
string column = "Rating";
string op = "<";
// Dynamically built query
db.Database.SqlQuery<Automobile>(#"select distinct automobile.* from automobile
inner join ratings on ....
where [" + column + "] " + op + " #p0", value);

Here is a method to build conditions for nested collections or types for linq-to-entities.
Restructured for your needs:
public static Expression GetCondition(Expression parameter, object value, OperatorComparer operatorComparer, params string[] properties)
{
Expression resultExpression = null;
Expression childParameter, navigationPropertyPredicate;
Type childType = null;
if (properties.Count() > 1)
{
//build path
parameter = Expression.Property(parameter, properties[0]);
var isCollection = typeof(IEnumerable).IsAssignableFrom(parameter.Type);
//if it´s a collection we later need to use the predicate in the methodexpressioncall
if (isCollection)
{
childType = parameter.Type.GetGenericArguments()[0];
childParameter = Expression.Parameter(childType, childType.Name);
}
else
{
childParameter = parameter;
}
//skip current property and get navigation property expression recursivly
var innerProperties = properties.Skip(1).ToArray();
navigationPropertyPredicate = GetCondition(childParameter, test, innerProperties);
if (isCollection)
{
//build methodexpressioncall
var anyMethod = typeof(Enumerable).GetMethods().Single(m => m.Name == "Any" && m.GetParameters().Length == 2);
anyMethod = anyMethod.MakeGenericMethod(childType);
navigationPropertyPredicate = Expression.Call(anyMethod, parameter, navigationPropertyPredicate);
resultExpression = MakeLambda(parameter, navigationPropertyPredicate);
}
else
{
resultExpression = navigationPropertyPredicate;
}
}
else
{
var childProperty = parameter.Type.GetProperty(properties[0]);
var left = Expression.Property(parameter, childProperty);
var right = Expression.Constant(value,value.GetType());
if(!new List<OperatorComparer> {OperatorComparer.Contains,OperatorComparer.StartsWith}.Contains(operatorComparer))
{
navigationPropertyPredicate = Expression.MakeBinary((ExpressionType)operatorComparer,left, right);
}
else
{
var method = GetMethod(childProperty.PropertyType, operatorComparer); //get property by enum-name from type
navigationPropertyPredicate = Expression.Call(left, method, right);
}
resultExpression = MakeLambda(parameter, navigationPropertyPredicate);
}
return resultExpression;
}
private static MethodInfo GetMethod(Type type,OperatorComparer operatorComparer)
{
var method = type.GetMethod(Enum.GetName(typeof(OperatorComparer),operatorComparer));
return method;
}
public enum OperatorComparer
{
Equals = ExpressionType.Equal,
Contains,
StartsWith,
GreaterThan = ExpressionType.GreaterThan
....
}
private static Expression MakeLambda(Expression parameter, Expression predicate)
{
var resultParameterVisitor = new ParameterVisitor();
resultParameterVisitor.Visit(parameter);
var resultParameter = resultParameterVisitor.Parameter;
return Expression.Lambda(predicate, (ParameterExpression)resultParameter);
}
private class ParameterVisitor : ExpressionVisitor
{
public Expression Parameter
{
get;
private set;
}
protected override Expression VisitParameter(ParameterExpression node)
{
Parameter = node;
return node;
}
}
}
You could replace the params string[] with params Expression(Func(T,object)), if you want. Would need some more work to do it that way. You would need to definie nested collections with a syntax like
item => item.nestedCollection.Select(nested => nested.Property)
and rewrite the expression with the help of an expressionvisitor.

Dynamic expression tree to filter on nested collection properties

I'm using Entity Framework and building queries using navigation properties dynamically.
For most of my use cases, the following works fine:
private static MethodCallExpression GetNavigationPropertyExpression<T>(string propertyName, int test,
ParameterExpression parameter, string subParameter)
{
var navigationPropertyCollection = Expression.Property(parameter, propertyName);
var childType = navigationPropertyCollection.Type.GetGenericArguments()[0];
var anyMethod = typeof(Enumerable).GetMethods().Single(m => m.Name == "Any" && m.GetParameters().Length == 2).MakeGenericMethod(childType);
var aclAttribute = GetAclAttribute(typeof(T), propertyName);
var childProperty = aclAttribute.ChildProperty;
var propertyCollectionGenericArg = childType;
var serviceLocationsParam = Expression.Parameter(propertyCollectionGenericArg, subParameter);
var left = Expression.Property(serviceLocationsParam, childProperty);
var right = Expression.Constant(test, typeof(int));
var isEqual = Expression.Equal(left, right);
var subLambda = Expression.Lambda(isEqual, serviceLocationsParam);
var resultExpression = Expression.Call(anyMethod, navigationPropertyCollection, subLambda);
return resultExpression;
}
I use a custom AclAttribute class assigned to properties via metadata type and partial classes. For navigation properties, a ChildProperty is provided so that the expression builder knows to look deeper for the desired property.
For example: the table Services references another table called ServiceLocations. I need the LocationId value(s) from the ServiceLocations reference. This part works fine.
My problem is when there is more than 1 nested property to go through. Another example: the table ServiceCategories references Services which, again, references ServiceLocations. Like before, I need the LocationId value(s) from ServiceLocations.
I've done this manually by using two "Any" methods, combining, and returning the resulting expression, but there will be times where navigation properties won't be collections, or a child of a navigation property may be a collection. For those cases, I need some kind of recursive option. I've been trying for a while now, and coming up short.
Since I mentioned it, here's the test method I put together for my second example. This works, but I'm violating DRY. (Note: I didn't name the tables):
private static MethodCallExpression GetNestedNavigationPropertyExpression(int test, ParameterExpression rootParameter)
{
var servicesProperty = Expression.Property(rootParameter, "tblServices");
var servicesParameter = Expression.Parameter(servicesProperty.Type.GetGenericArguments()[0], "ss");
var serviceLocationsProperty = Expression.Property(servicesParameter, "tblServiceLocations");
var serviceLocationsParameter = Expression.Parameter(serviceLocationsProperty.Type.GetGenericArguments()[0], "s");
var servicesAnyMethod = typeof(Enumerable).GetMethods().Single(m => m.Name == "Any" && m.GetParameters().Length == 2).MakeGenericMethod(servicesProperty.Type.GetGenericArguments()[0]);
var serviceLocationsAnyMethod = typeof(Enumerable).GetMethods().Single(m => m.Name == "Any" && m.GetParameters().Length == 2).MakeGenericMethod(serviceLocationsProperty.Type.GetGenericArguments()[0]);
var aclAttribute = GetAclAttribute(typeof(tblService), "tblServiceLocations");
var left = Expression.Property(serviceLocationsParameter, aclAttribute.ChildProperty);
var right = Expression.Constant(test, typeof(int));
var isEqual = Expression.Equal(left, right);
var subLambda = Expression.Lambda(isEqual, serviceLocationsParameter);
var endExpression = Expression.Call(serviceLocationsAnyMethod, serviceLocationsProperty, subLambda);
var intermediaryLamba = Expression.Lambda(endExpression, servicesParameter);
var resultExpression = Expression.Call(servicesAnyMethod, servicesProperty, intermediaryLamba);
return resultExpression;
}

With this it should be possible to build your queries:
public static Expression GetNavigationPropertyExpression(Expression parameter, int test, params string[] properties)
{
Expression resultExpression = null;
Expression childParameter, navigationPropertyPredicate;
Type childType = null;
if (properties.Count() > 1)
{
//build path
parameter = Expression.Property(parameter, properties[0]);
var isCollection = typeof(IEnumerable).IsAssignableFrom(parameter.Type);
//if it´s a collection we later need to use the predicate in the methodexpressioncall
if (isCollection)
{
childType = parameter.Type.GetGenericArguments()[0];
childParameter = Expression.Parameter(childType, childType.Name);
}
else
{
childParameter = parameter;
}
//skip current property and get navigation property expression recursivly
var innerProperties = properties.Skip(1).ToArray();
navigationPropertyPredicate = GetNavigationPropertyExpression(childParameter, test, innerProperties);
if (isCollection)
{
//build methodexpressioncall
var anyMethod = typeof(Enumerable).GetMethods().Single(m => m.Name == "Any" && m.GetParameters().Length == 2);
anyMethod = anyMethod.MakeGenericMethod(childType);
navigationPropertyPredicate = Expression.Call(anyMethod, parameter, navigationPropertyPredicate);
resultExpression = MakeLambda(parameter, navigationPropertyPredicate);
}
else
{
resultExpression = navigationPropertyPredicate;
}
}
else
{
//Formerly from ACLAttribute
var childProperty = parameter.Type.GetProperty(properties[0]);
var left = Expression.Property(parameter, childProperty);
var right = Expression.Constant(test, typeof(int));
navigationPropertyPredicate = Expression.Equal(left, right);
resultExpression = MakeLambda(parameter, navigationPropertyPredicate);
}
return resultExpression;
}
private static Expression MakeLambda(Expression parameter, Expression predicate)
{
var resultParameterVisitor = new ParameterVisitor();
resultParameterVisitor.Visit(parameter);
var resultParameter = resultParameterVisitor.Parameter;
return Expression.Lambda(predicate, (ParameterExpression)resultParameter);
}
private class ParameterVisitor : ExpressionVisitor
{
public Expression Parameter
{
get;
private set;
}
protected override Expression VisitParameter(ParameterExpression node)
{
Parameter = node;
return node;
}
}
Call it like:
var parameter = Expression.Parameter(typeof(A), "A");
var expression = ExpressionBuilder.GetNavigationPropertyExpression(parameter, 8,"CollectionOfB", "CollectionOfC", "ID");