Is there a way to build an expression tree for the new with operator?
I am trying to implement a 'Lens' feature for records that will only need a selector and will auto generate the mutator
My goal is to convert from a 'selector':
Expression<Func<T, TMember>> expression (ie employee => employee.Name)
To a 'mutator':
(employee, newName) => employee with { Name = newName }
I did manage to do this for the simple case above, see my answer below, however that will not work for a nested case ie:
record Employee(string Name, int Age);
record Manager(String Name, Employee Employee);
Here I want to change ie from
manager => manager.Employee.Name
to
(manager, newEmployeeName) => manager with { Employee = manager.Employee with { Name = newEmployeeName}}
Any help ?
CalcMutator method that can deal with nested properties would look something like this
static Func<T, TMember, T> CalcMutator(Expression<Func<T, TMember>> expression)
{
var typeParam = expression.Parameters.First();
var valueParam = Expression.Parameter(typeof(TMember), "v");
var variables = new List<ParameterExpression>();
var blockExpressions = new List<Expression>();
var property = (MemberExpression)expression.Body;
Expression currentValue = valueParam;
var index = 0;
while (property != null)
{
var variable = Expression.Variable(property.Expression.Type, $"v_{index}");
variables.Add(variable);
var cloneMethod = property.Expression.Type.GetMethod("<Clone>$");
if (cloneMethod is null) throw new Exception($"CalcMutatorNo Clone method on {typeof(T)}");
var cloneCall = Expression.Call(property.Expression, cloneMethod);
var assignClonedToVariable = Expression.Assign(variable, cloneCall);
var accessVariableProperty = Expression.MakeMemberAccess(variable, property.Member);
var assignVariablePropertyValue = Expression.Assign(accessVariableProperty, currentValue);
blockExpressions.Add(assignClonedToVariable);
blockExpressions.Add(assignVariablePropertyValue);
property = property.Expression as MemberExpression;
currentValue = variable;
index++;
}
// Return root object
blockExpressions.Add(currentValue);
var block = Expression.Block(variables, blockExpressions);
var assignLambda = (Expression<Func<T, TMember, T>>)Expression.Lambda(block, typeParam, valueParam);
return assignLambda.Compile();
}
Please keep in mind that Cache implemented with ImmutableDictionary is not thread safe. If you want to ensure that the cached expressions can safely be used in multi-threaded environments, it's better to use ConcurrentDictionary for the cache instead or to apply some synchronization primitives around ImmutableDictionary.
Following the lead from #JL0PD I ended up converting:
t => t.Member (ie employee => employee.Name)
into:
(t, v) => {
var c = t.<Clone>$();
c.Member = v;
return c;
}
ie:
(employee, newName) => {
var c = employee.<Clone>$();
c.Name=newName;
return c;
}
Below is a full implemetation of a record Lens including caching of delegates
Note that this does not cover nested mutators so my question above still stands
static class RecLens<T, TMember> {
public static (Func<T, TMember> Selector, Func<T, TMember, T> Mutator) Get(Expression<Func<T, TMember>> expression) {
if (!IsExpressionValid(expression.Body)) throw new Exception($"Lens Invalid expression ({expression})");
// create unique cache key, calc same key for x=>x.p and y=>y.p
var exprStr = expression.Body.ToString();
var dotPos = exprStr.IndexOf(Type.Delimiter);
var cacheKey = typeof(T).FullName + '|' + (dotPos > 0 ? exprStr.Remove(0, exprStr.IndexOf(Type.Delimiter) + 1) : "root");
if (!Cache.TryGetValue(cacheKey, out var res)) {
res = (expression.Compile(), CalcMutator(expression));
Cache = Cache.Add(cacheKey, res);
}
return res;
}
// key: "{srcType.FullName}|{member}" , ie: "Test.Organization|DevelopmentDepartment.Manager"
static ImmutableDictionary<string, (Func<T, TMember>, Func<T, TMember, T>)> Cache = ImmutableDictionary<string, (Func<T, TMember>, Func<T, TMember, T>)>.Empty;
// create delegate: (t, v) => { var c=t.<Clone>$(); c.Member = v; return c; }
static Func<T, TMember, T> CalcMutator(Expression<Func<T, TMember>> expression) {
var result = Expression.Variable(typeof(T), "c");
var typeParam = Expression.Parameter(typeof(T), "t");
var valueParam = Expression.Parameter(typeof(TMember), "v");
var cloneMethod = typeof(T).GetMethod("<Clone>$");
if (cloneMethod is null) throw new Exception($"CalcMutatorNo Clone method on {typeof(T)}");
var cloneCall = Expression.Call(typeParam, cloneMethod);
var assignResult = Expression.Assign(result, cloneCall);
var memberInfo = (expression.Body as MemberExpression)!.Member;
var resultMemberAccess = Expression.MakeMemberAccess(result, memberInfo);
var assign = Expression.Assign(resultMemberAccess, valueParam);
var block = Expression.Block(new[] { result }, assignResult, assign, result);
var assignLambda = (Expression<Func<T, TMember, T>>)Expression.Lambda(block, typeParam, valueParam);
return assignLambda.Compile();
}
// verify that expr is a member expression of its parameter
static bool IsExpressionValid(Expression expr, bool first = true) {
if (expr is ParameterExpression) return !first;
if (expr is MemberExpression memberExpr && memberExpr.Expression is object) return IsExpressionValid(memberExpr.Expression, false);
return false;
}
}
To use:
record Employee(string Name, int Age);
var (Selector, Mutator) = RecLens<Employee, string>.Get(e => e.Name);
var dave = new Employee("Dave", 30);
var name = Selector(dave); // "Dave"
var john = Mutator(dave, "John"); // Employee("John", 30)
Related
I am trying to build a filter method for IQueryable Type,
I could achieve this for ex:
query = query.Filter(UsersListId, e => e.UserId);
public static IQueryable<T> Filter<T, TSearch>(this IQueryable<T> query, List<TSearch> list, Expression<Func<T, TSearch>> props)
{
if (list == null || list.Count == 0)
{
return query;
}
var propertyPath = props.Body.ToString().Replace(props.Parameters[0] + ".", string.Empty);
var containsMethod = typeof(List<TSearch>).GetMethod("Contains", new Type[] { typeof(TSearch) });
ConstantExpression constlist = Expression.Constant(list);
var param = Expression.Parameter(typeof(T), "Entity");
var newvalue = GetPropertyValue(param, propertyPath);
var body = Expression.Call(constlist, containsMethod, newvalue);
var exp = Expression.Lambda<Func<T, bool>>(body, param);
return query.Where(exp);
}
public static MemberExpression GetPropertyValue(ParameterExpression parExp, string propertyPath)
{
var properties = propertyPath.Split('.').ToArray();
var value = Expression.Property(parExp, properties[0]);
for (int i = 1; i < properties.Length; i++)
{
value = Expression.Property(value, properties[i]);
}
return value;
}
this code takes the propertyPath (ex: x.User.Department.Id)
and returns the value,
The problems here:
I cannot pass a nullable object if the list isn't nullable too
I am not quite sure weather it's the right way to access properties
so my question is what is the solution for these problems?
I'm having trouble coding an expression for IQueryable object. I can't figure out the right way to use the expression to map a string to object properties.
this is my query object and mapping dictionary:
var query = context.Industries.AsQueryable();
var columnsMap = new Dictionary<string, Expression<Func<Industry, object>>>()
{
["name"] = v => v.Name,
["isicCode"] = v => v.IsicCode.Data,
["isicCodeTitle"] = v => v.IsicCode.Title,
["isActive"] = v => v.IsActive,
};
and I'm using columnsMap dictionary for applying Orderby to my query in an extentions class:
public static IQueryable<T> ApplyOrdering<T>(this IQueryable<T> query, IQueryObject queryObj, Dictionary<string, Expression<Func<T, object>>> columnsMap)
{
if (String.IsNullOrWhiteSpace(queryObj.SortBy) || !columnsMap.ContainsKey(queryObj.SortBy))
return query;
if (queryObj.IsSortAsc)
return query.OrderBy(columnsMap[queryObj.SortBy]);
else
return query.OrderByDescending(columnsMap[queryObj.SortBy]);
}
it's ok for OrderBy but I need to do the same thing for ApplyFiltering but for filter an IQueryable object I need a different expression for where method Expression<Func<T, bool>>
public static IQueryable<T> ApplyFiltering<T>(this IQueryable<T> query, IQueryObject queryObj, Dictionary<string, Expression<Func<T, object>>> columnsMap)
{
query.Where(columnsMap['name'] == "test Name"); //this is the problem.
return query;
}
the question is how can I use my columnsMap in ApplyFiltering method? or should I change my columnsMap for that?
It is quite simple. The only problem is when you have a value type column (an int or a bool or a DateTime for example)... An Expression<Func<Industry, object>> will introduce a boxing of the field/property to object that we have to remove. This problem is absent for strings.
// isActive is a bool
Expression<Func<Industry, object>> exp = columnsMap["isActive"];
object value = true; // You can't use "true" (string) here! isActive is a bool
// Other exammple
// Expression<Func<Industry, object>> exp = columnsMap["name"];
// object value = "Foo";
var body = exp.Body;
// Remove the boxing for value types
if (body.NodeType == ExpressionType.Convert)
{
body = ((UnaryExpression)body).Operand;
}
var eq = Expression.Equal(body, Expression.Constant(value, body.Type));
var exp2 = Expression.Lambda<Func<T, bool>>(eq, exp.Parameters);
return query.Where(exp2);
Example using string value:
Expression<Func<Industry, object>> exp = columnsMap["isActive"];
string value = "true";
// Other exammple
//Expression<Func<Industry, object>> exp = columnsMap["name"];
//string value = "Foo";
var body = exp.Body;
// Remove the boxing for value types
if (body.NodeType == ExpressionType.Convert)
{
body = ((UnaryExpression)body).Operand;
}
object value2 = value;
if (value2 != null && body.Type != value2.GetType())
{
value2 = Convert.ChangeType(value2, body.Type);
}
var eq = Expression.Equal(body, Expression.Constant(value2, body.Type));
var exp2 = Expression.Lambda<Func<Industry, bool>>(eq, exp.Parameters);
return query.Where(exp2);
I know how to replace a parameter with ExpressionVisitor but I was wondering if there's a way to remove a parameter from a Expression.Block.
Ideally I should crawl the entire Expression tree and remove the parameter every time it is declared inside a Block.
Any idea how to do that with ExpressionVisitor?
A simple class to remove local variables from a BlockExpression and replace them with whatever you want.
public class BlockVariableRemover : ExpressionVisitor
{
private readonly Dictionary<Expression, Expression> replaces = new Dictionary<Expression, Expression>();
public readonly Func<ParameterExpression, int, Expression> Replacer;
public BlockVariableRemover(Func<ParameterExpression, int, Expression> replacer)
{
Replacer = replacer;
}
protected override Expression VisitBlock(BlockExpression node)
{
var removed = new List<Expression>();
var variables = node.Variables.ToList();
for (int i = 0; i < variables.Count; i++)
{
var variable = variables[i];
var to = Replacer(variable, i);
if (to != variable)
{
removed.Add(variable);
replaces.Add(variable, to);
variables.RemoveAt(i);
i--;
}
}
if (removed.Count == 0)
{
return base.VisitBlock(node);
}
var expressions = node.Expressions.ToArray();
for (int i = 0; i < expressions.Length; i++)
{
expressions[i] = Visit(expressions[i]);
}
foreach (var rem in removed)
{
replaces.Remove(rem);
}
return Expression.Block(variables, expressions);
}
public override Expression Visit(Expression node)
{
Expression to;
if (node != null && replaces.TryGetValue(node, out to))
{
return base.Visit(to);
}
return base.Visit(node);
}
}
Use it like:
Expression<Func<int, int>> exp;
{
var var1 = Expression.Variable(typeof(int), "var1");
var var2 = Expression.Variable(typeof(long), "var2");
var par1 = Expression.Parameter(typeof(int), "par1");
var block = Expression.Block(new[] { var1, var2 }, Expression.Increment(var1));
exp = Expression.Lambda<Func<int, int>>(block, par1);
// Test
var compiled = exp.Compile();
Console.WriteLine(compiled(10));
}
// Begin replace
{
var par1 = exp.Parameters[0];
var block2 = new BlockVariableRemover(
// ix is the index of the variable,
// return x if you don't want to modify,
// return whatever you want (even Expression.Empty()) to do
// a replace
(x, ix) => ix == 0 && x.Type == typeof(int) ? par1 : x)
.Visit(exp.Body);
// Final result
var exp2 = Expression.Lambda<Func<int, int>>(block2, par1);
// Test
var compiled = exp2.Compile();
Console.WriteLine(compiled(10));
}
I am trying to build a Lambda Expression for a table that has been created at run time.
The Expression is build fine but when I call Compile() method I get this error
"ParameterExpression of type 'cseval.Item' cannot be used for delegate parameter of type 'System.Object'"
this is my function
public Func<dynamic, Boolean> GetWhereExp(List<WhereCondition> SearchFieldList, dynamic item)
{
ParameterExpression pe = Expression.Parameter(item.GetType(), "c");
Expression combined = null;
if (SearchFieldList != null)
{
foreach (WhereCondition fieldItem in SearchFieldList)
{
//Expression for accessing Fields name property
Expression columnNameProperty = Expression.Property(pe, fieldItem.ColumName);
//the name constant to match
Expression columnValue = Expression.Constant(fieldItem.Value);
//the first expression: PatientantLastName = ?
Expression e1 = Expression.Equal(columnNameProperty, columnValue);
if (combined == null)
{
combined = e;
}
else
{
combined = Expression.And(combined, e);
}
}
}
var result = Expression.Lambda<Func<dynamic, bool>>(combined, pe);
return result.Compile();
}
I've changed dynamic to generics, this code works for me:
public Func<T, Boolean> GetWhereExp<T>(List<WhereCondition> SearchFieldList, T item)
{
var pe = Expression.Parameter(item.GetType(), "c");
Expression combined = null;
if (SearchFieldList != null)
{
foreach (var fieldItem in SearchFieldList)
{
var columnNameProperty = Expression.Property(pe, fieldItem.ColumName);
var columnValue = Expression.Constant(fieldItem.Value);
var e1 = Expression.Equal(columnNameProperty, columnValue);
combined = combined == null ? e1 : Expression.And(combined, e1);
}
}
var result = Expression.Lambda<Func<T, bool>>(combined, pe);
return result.Compile();
}
Small remark: your method returns function, not an expression, so the name 'GetWhereExp' is slightly incorrect. If you want to return function, imho, it's better to use reflection.
UPD: I use this code to test:
var expressions = new List<WhereCondition>
{
new WhereCondition("Column1", "xxx"),
new WhereCondition("Column2", "yyy"),
};
var item = new
{
Column1 = "xxx",
Column2 = "yyy"
};
var func = LinqExpr.GetWhereExp(expressions, (dynamic)item);
Console.WriteLine(new[] {item}.Count(a => func(a)));
I'm using LINQ 2 Entities.
Following is the problem:
string str = '%test%.doc%'
.Contains(str) // converts this into LIKE '%~%test~%.doc~%%'
Expected Conversion: LIKE '%test%.doc%'
If it was LINQ 2 SQL, I could have used SqlMethods.Like as somebody answered it in my previous question. But now as I'm using L2E not L2S, I need other solution.
The SQL method PATINDEX provides the same functionality as LIKE. Therefore, you can use the SqlFunctions.PatIndex method:
.Where(x => SqlFunctions.PatIndex("%test%.doc%", x.MySearchField) > 0)
Following on from Magnus' correct answer, here is an extension method that can be re-used, as I needed in my project.
public static class LinqExtensions
{
public static Expression<Func<T, bool>> WildCardWhere<T>(this Expression<Func<T, bool>> source, Expression<Func<T, string>> selector, string terms, char separator)
{
if (terms == null || selector == null)
return source;
foreach (string term in terms.Split(new[] { separator }, StringSplitOptions.RemoveEmptyEntries))
{
string current = term;
source = source.And(
Expression.Lambda<Func<T, bool>>(
Expression.Call(selector.Body, "Contains", null, Expression.Constant(current)),
selector.Parameters[0]
)
);
}
return source;
}
}
Usage:
var terms = "%test%.doc%";
Expression<Func<Doc, bool>> whereClause = d => d;
whereClause = whereClause.WildCardWhere(d => d.docName, terms, '%');
whereClause = whereClause.WildCardWhere(d => d.someOtherProperty, "another%string%of%terms", '%');
var result = ListOfDocs.Where(whereClause).ToList();
The extension makes use of the predicate builder at http://petemontgomery.wordpress.com/2011/02/10/a-universal-predicatebuilder/. The resulting sql does a single table scan of the table, no matter how many terms are in there. Jo Vdb has an example you could start from if you wanted an extension of iQueryable instead.
You can try use this article, where author describes how to build a LIKE statement with wildcard characters in LINQ to Entities.
EDIT: Since the original link is now dead, here is the original extension class (as per Jon Koeter in the comments) and usage example.
Extension:
public static class LinqHelper
{
//Support IQueryable (Linq to Entities)
public static IQueryable<TSource> WhereLike<TSource>(this IQueryable<TSource> source, Expression<Func<TSource, string>> valueSelector, string value, char wildcard)
{
return source.Where(BuildLikeExpression(valueSelector, value, wildcard));
}
//Support IEnumerable (Linq to objects)
public static IEnumerable<TSource> WhereLike<TSource>(this IEnumerable<TSource> sequence, Func<TSource, string> expression, string value, char wildcard)
{
var regEx = WildcardToRegex(value, wildcard);
//Prevent multiple enumeration:
var arraySequence = sequence as TSource[] ?? sequence.ToArray();
try
{
return arraySequence.Where(item => Regex.IsMatch(expression(item), regEx));
}
catch (ArgumentNullException)
{
return arraySequence;
}
}
//Used for the IEnumerable support
private static string WildcardToRegex(string value, char wildcard)
{
return "(?i:^" + Regex.Escape(value).Replace("\\" + wildcard, "." + wildcard) + "$)";
}
//Used for the IQueryable support
private static Expression<Func<TElement, bool>> BuildLikeExpression<TElement>(Expression<Func<TElement, string>> valueSelector, string value, char wildcard)
{
if (valueSelector == null) throw new ArgumentNullException("valueSelector");
var method = GetLikeMethod(value, wildcard);
value = value.Trim(wildcard);
var body = Expression.Call(valueSelector.Body, method, Expression.Constant(value));
var parameter = valueSelector.Parameters.Single();
return Expression.Lambda<Func<TElement, bool>>(body, parameter);
}
private static MethodInfo GetLikeMethod(string value, char wildcard)
{
var methodName = "Equals";
var textLength = value.Length;
value = value.TrimEnd(wildcard);
if (textLength > value.Length)
{
methodName = "StartsWith";
textLength = value.Length;
}
value = value.TrimStart(wildcard);
if (textLength > value.Length)
{
methodName = (methodName == "StartsWith") ? "Contains" : "EndsWith";
}
var stringType = typeof(string);
return stringType.GetMethod(methodName, new[] { stringType });
}
}
Usage Example:
string strEmailToFind = "%#yahoo.com"
IQueryable<User> myUsers = entities.Users.WhereLike(u => u.EmailAddress, strEmailToFind, '%');
or, if you expect your users to be more accustomed to Windows Explorer-styled wildcards:
string strEmailToFind = "*#yahoo.com"
IQueryable<User> myUsers = entities.Users.WhereLike(u => u.EmailAddress, strEmailToFind, '*');
Use a regular expression...
The following will print out all of the files in the current directory that match test.doc* (dos wildcard style - which I believe is what you're asking for)
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Text.RegularExpressions;
using System.IO;
namespace RegexFileTester
{
class Program
{
static void Main(string[] args)
{
string[] _files = Directory.GetFiles(".");
var _fileMatches = from i in _files
where Regex.IsMatch(i, ".*test*.doc.*")
//where Regex.IsMatch(i, ".*cs")
select i;
foreach(var _file in _fileMatches)
{
Console.WriteLine(_file);
}
}
}
}
Split the String
var str = "%test%.doc%";
var arr = str.Split(new[]{'%'} ,StringSplitOptions.RemoveEmptyEntries);
var q = tblUsers.Select (u => u);
foreach (var item in arr)
{
var localItem = item;
q = q.Where (x => x.userName.Contains(localItem));
}
So I was trying the same thing - trying to pair down a List to return all candidates that matched a SearchTerm. I wanted it so that if a user typed "Arizona" it would return everything regardless of case that had Arizona. Also, if the user typed "Arizona Cha", it would return items like "Arizona License Change". The following worked:
private List<Certification> GetCertListBySearchString()
{
string[] searchTerms = SearchString.Split(' ');
List<Certification> allCerts = _context.Certifications.ToList();
allCerts = searchTerms.Aggregate(allCerts, (current, thisSearchString) => (from ac in current
where ac.Name.ToUpper().Contains(thisSearchString.ToUpper())
select ac).ToList());
return allCerts;
}