We Keep Coding

How can I construct a where clause in C#/EF Core with the field defined at runtime?

This seems simple, but I can't decipher the LINQ required to do it. I also can't add any new dependencies.
Basically, I'm trying to make this code generic:
if (filter.matchMode.ToUpper().Equals("EQ")) {
query = query.Where(x => x.SomeField.Equals(filter.Value));
if (filter.matchMode.ToUpper().Equals("LT")) {
query = query.Where(x => x.SomeField < filter.Value);
} else [... 5 other match modes ...]
}
Now, SomeField is only one of about 5 fields that needs this functionality. And there's 5 matching operators. I could just copy/pasta the whole thing, and then deal with the debt of having to change tons of code every time a new operator or field enters the mix, but that really doesn't seem right.
Basically, I need some way of defining SomeField at runtime, so I can factor out the whole if/else tree and use it for each supported field.
I've tried going down this road, but I think I'm misunderstanding something fundamental about expressions:
var entityType = typeof(TableObject);
ParameterExpression arg = Expression.Parameter(entityType, "x");
MemberExpression amproperty = Expression.Property(arg, "SomeField");
MemberExpression property = Expression.Property(amproperty, "Equals"); // ???
// what's next, if this is even the right direction...
EDIT: a shorter version of the question might be: how can I construct the following object foo using MemberExpressions and lambdas, such that SomeField can be passed in as a string "SomeField":
Expression<Func<TableObject, bool>> foo = x => x.SomeField.Equals("FOO");
UPDATE: here's what I ended up coming up with:
private IQueryable<TableObject> processFilter(
IQueryable<TableObject> query,
FilterItem filter,
string fieldName)
{
var entityType = typeof(TableObject);
// construct the argument and access object
var propertyInfo = entityType.GetProperty(fieldName);
ParameterExpression arg = Expression.Parameter(entityType, "x");
MemberExpression access = Expression.MakeMemberAccess(arg,
typeof(TableObject).GetProperty(fieldName)
);
// translate the operation into the appropriate Expression method
Expression oprFunc;
if (filter.MatchMode.ToUpper().Equals("EQ")) {
oprfunc =
Expression.Equal(access, Expression.Constant(filter.Value));
} else if (filter.MatchMode.ToUpper().Equals("LT")) {
oprfunc =
Expression.LessThan(access, Expression.Constant(filter.IntValue));
} else {
throw new ArgumentException(
$"invalid argument: ${filter.MatchMode}"
);
}
// construct the lambda
var func = Expression.Lambda<Func<TableObject, bool>>(oprFunc, arg);
// return the new query
return query.Where(func);
}
So far, this seems to cover most of the cases. It starts to go off the rails with nullable fields and date comparisons, but it will work for what I need it to do. The part I'm still not completely sure about is Expression.MakeMemberAccess. I've seen this written many ways and I'm not sure if that's the correct way to create that expression.

Let's say you have class like this:
class SomeClass
{
public string a1 { get; set; }
public string b1 { get; set; }
public string c1 { get; set; }
}
also let's assume you have method like this:
public List<T> GetAll(Expression<Func<T, bool>> criteria )
{
return someDataLINQCapable.Where(criteria);
}
Method could be called like this:
GetAll<SomeClass>(m => m.a1 != null && m.b1=="SomethingUseful")

Dynamically creating an expression which selects an objects property

I want to be able to build up an expression dynamically, which is essentially a property selector.
I am trying to use this so I can provide a flexible search UI and then translate the selected search parameters to an Entity Framework query.
I have most of what I need thanks to another library I am using, but am missing the final part which translates my query string parameters to the appropriate expression selector the other library requires.
The library takes an argument of :
Expression<Func<TObject, TPropertyType>>
An example of how this would be coded if baked into an application would be :
Expression<Func<MyObject, int>> expression = x=> x.IntegerProperty;
However, I need to be able to generate this expression dynamically, as the important point is that all I will know is the type of object (MyObject) and the property name as a string value ("IntegerProperty"). The property value will obviously map to an property on the object which could be of any non complex type.
So essentially I think I am wanting to find a way to build up the expression dynamically which specifies the correct object property to return and where the return value is determined by that property type.
psuedo code :
string ObjectPropertyName
Type ObjectType
Type ObjectPropertyType = typeof(ObjectType).GetProperty(ObjectPropertyName).Property
Expression<Func<[ObjectType], [ObjectPropertyType]>> expression = x=> x.[ObjectPropertyName];
Update :
I have got as far as this
ParameterExpression objectParameter = Expression.Parameter(type, "x");
MemberExpression objectProperty = Expression.Property(objectParameter, "PropertyNameString");
Expression<Func<ObjectType, int>> expression = Expression.Lambda<Func<ObjectType, int>>(objectProperty, objectParameter);
But the problem I have with this is that the return type is not always an int but may be some other type.

Doing what you asked is bit tricky but not impossible. Since the property type is not known until run time so you can not declare the Expression<Func<,>> so it would be done by reflection.
public static class QueryableExtension
{
public static object Build<Tobject>(this Tobject source, string propertyName)
{
var propInfo = typeof(Tobject).GetProperty(propertyName);
var parameter = Expression.Parameter(typeof(Tobject), "x");
var property = Expression.Property(parameter, propInfo);
var delegateType = typeof(Func<,>)
.MakeGenericType(typeof(Tobject), propInfo.PropertyType);
var lambda = GetExpressionLambdaMethod()
.MakeGenericMethod(delegateType)
.Invoke(null, new object[] { property, new[] { parameter } });
return lambda;
}
public static MethodInfo GetExpressionLambdaMethod()
{
return typeof(Expression)
.GetMethods()
.Where(m => m.Name == "Lambda")
.Select(m => new
{
Method = m,
Params = m.GetParameters(),
Args = m.GetGenericArguments()
})
.Where(x => x.Params.Length == 2
&& x.Args.Length == 1
)
.Select(x => x.Method)
.First();
}
}
Usage -
var expression = testObject.Build("YourPropertyName");
Now this will build the Expression you desired with return type of property. But since we don't know about your library but I suggest you to call your library method via reflection and pass the expression wrapped under object.

As I mentioned in the comments, building expression without knowing the property type is easy (even with nested property support):
static LambdaExpression MakeSelector(Type objectType, string path)
{
var item = Expression.Parameter(objectType, "item");
var body = path.Split('.').Aggregate((Expression)item, Expression.PropertyOrField);
return Expression.Lambda(body, item);
}
But then you'll need to find a way to call your generic library method - using reflection or dynamic call.

If you have both ObjectType and ObjectPropertyType as generic type parameters, you can use the Expression class to do something like this:
public static Expression<Func<TObject, TPropertyType>> Generate<TObject, TPropertyType>(
string property_name)
{
var parameter = Expression.Parameter(typeof (TObject));
return Expression.Lambda<Func<TObject, TPropertyType>>(
Expression.Property(parameter, property_name), parameter);
}

There is old intresting library DynamicLinq. May be it will be useful for you. It extends System.Linq.Dynamic to support execution of Lambda expressions defined in a string. With use of DynamicLinq you can do somethink like:
public class IndexViewModel
{
public bool HasPassword { get; set; }
public string PhoneNumber { get; set; }
public bool TwoFactor { get; set; }
public bool BrowserRemembered { get; set; }
}
//...........
Expression<Func<IndexViewModel, bool>> ex =
System.Linq.Dynamic.DynamicExpression.ParseLambda<IndexViewModel, bool>("TwoFactor");
var model = new ReactJs.NET.Models.IndexViewModel() { TwoFactor = true };
var res = ex.Compile()(model);
// res == true
System.Diagnostics.Debug.Assert(res);

Apply a dynamically created lambda to an object instance

I have some code which dynamically create a lambda starting from strings. For example, I have a filter class like the following:
public class Criteria {
public string Property { get; set; }
public string Operator { get; set; }
public string Value { get; set; }
}
And I am able to create a lambda like x => x.Name == "Foo" starting from a Criteria instance like this
Criteria c = new Criteria() {
Property = "Name",
Operator = "equal",
Value = "Foo"
}
Supposing to have a class like
public class Receipt {
public string Name { get; set; }
public int Amount { get; set; }
[other props omitted]
public ICollection<ReceiptDetail> Details { get; set; }
}
I would like to:
Apply the lambda to any object (I know that the lambda should be created with a ParameterExpression of the Receipt class)
Getting back the boolean result of the lambda (e.g. Is the name equals to Foo?)
Apply the same logic to collections Count() method (e.g. Building a lambda which checks against receipt.Details.Count()
Is this possible?
EDIT: As per the comments, I am elaborating my needs a bit more. This code will give me the chance to answer to a requirement that I have and that says: If there is a rule specified for my object then the application should behave a bit differently. While this is a common requirement, I would like to create a code which will permit me to extend it as more rules will be added. Actually I only have 5 rule types:
Verify if the input comes in a specific day of the week
Verify if the input comes in a specific time range
Verify if the field "X" of the input is less/equal/greather than a value
Verify if the field "Y" of the input contains a value
Verify if the field "Z" of the input, which is a collection, has a count that is less/equal/greather than a value
For the first 4 points I have been able to dynamically create a lambda expression, with code like in P.Brian.Mackey answer, which I could apply, using the Specification pattern, to the object itself.
The last point needs to be implemented almost in the same way but the only difference is that the left part of the expression was a method call and not a Property (specifically the ICollection<T>.Count() method)

Here's something to get you started. There is plenty of room for improvement. Especially the ugly factory. This demo is intended to show how to use Expressions to solve the problems, not as a best practices or factory pattern demo. If anything is unclear please feel free to ask for clarification.
Usage
[Test]
public void ApplySameLogicToCollectionsCount()
{
var receipt = new Receipt();
var details = new ReceiptDetail();
var details2 = new ReceiptDetail();
receipt.Details.Add(details);
receipt.Details.Add(details2);
var result = LambdaGeneratorFactory<ICollection<ReceiptDetail>>.Run(detailsCount);
Assert.IsTrue(result(receipt.Details));
}
Factory
public static class LambdaGeneratorFactory<T>
{
//This is an ugly implementation of a Factory pattern.
//You should improve this possibly with interfaces, maybe abstract factory. I'd start with an ICriteria.
public static Predicate<T> Run(Criteria criteria)
{
if (typeof(T) == typeof (Receipt))
{
return CreateLambda(criteria);
}
else if (typeof (T) == typeof (ICollection<ReceiptDetail>))
{
return CreateLambdaWithCount(criteria);
}
return null;
}
private static Predicate<T> CreateLambda(Criteria criteria)
{
ParameterExpression pe = Expression.Parameter(typeof(T), "i");
Expression left = Expression.Property(pe, typeof(T).GetProperty(criteria.Property));
Expression right = Expression.Constant(criteria.Value);
Expression predicateBody = Expression.Equal(left, right);
var predicate = Expression.Lambda<Predicate<T>>(predicateBody, new ParameterExpression[] { pe }).Compile();
return predicate;
}
private static Predicate<T> CreateLambdaWithCount(Criteria criteria)
{
ParameterExpression pe = Expression.Parameter(typeof(T), "i");
Expression count = Expression.Property(pe, typeof(T).GetProperty("Count"));
Expression left = Expression.Call(count, typeof(Object).GetMethod("ToString"));
Expression right = Expression.Constant(criteria.Value);
Expression predicateBody = Expression.Equal(left, right);
var predicate = Expression.Lambda<Predicate<T>>(predicateBody, new ParameterExpression[] { pe }).Compile();
return predicate;
}
}
Criteria
private Criteria detailsCount = new Criteria()
{
Property = "Details",
Operator = "equal",
Value = "2"
};
Switch to ICriteria and things will b cleaner. A better factory and no need for ToString. Program to an interface.
All that being said, this code feels a bit funky. What is the point of generating functions from strings? I get the feeling this is heading towards generating C# from a grammar. I'm not convinced that will scale well. For non-trivial implementations consider lex/yacc first. You can find more details for doing this in the Pragmatic Programmer "Implementing a mini language".

Yours is a fascinating question, and I'd like to understand the requirements. I created a demo, and I'm wondering, how does the demo differ from what you're trying to accomplish? There is a working version here https://dotnetfiddle.net/AEBZ1w too.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
public class Program
{
public static void Main()
{
Criteria c = new Criteria() {
Property = "Name",
Operator = "==",
Value = "Foo" };
var queryable = (new List<Receipt>() {
new Receipt { Name = "Foo", Amount = 1 },
new Receipt { Name = "Foo", Amount = 2 },
new Receipt { Name = "Bar" }
}).AsQueryable();
var parameter = Expression.Parameter(typeof(Receipt), "x");
var property = Expression.Property(parameter, typeof(Receipt).GetProperty(c.Property));
var constant = Expression.Constant(c.Value);
var operation = Expression.Equal(property, constant);
var expression = Expression.Call(
typeof(Queryable),
"Where",
new Type[] { queryable.ElementType },
queryable.Expression,
Expression.Lambda<Func<Receipt, bool>>(operation, new ParameterExpression[] { parameter })
);
Console.WriteLine("Linq Expression: {0} \n", expression.ToString());
Console.WriteLine("Results: \n");
var results = queryable.Provider.CreateQuery<Receipt>(expression);
foreach(var r in results)
{
Console.WriteLine("{0}:{1}", r.Name, r.Amount);
}
}
}
public class Criteria
{
public string Property, Operator, Value;
}
public class ReceiptDetail
{
public string ItemName;
}
public class Receipt
{
public string Name { get; set; }
public int Amount;
public ICollection<ReceiptDetail> Details;
}
References
How to: Execute Expression Trees
How to: Use Expression Trees to Build Dynamic Queries
IQueryable Interface
Expression Methods
Expression.Property Method

You could use reflection with generics. One way to solve the problem could be an extension
public static class EnumerableExtensions
{
public static IEnumerable<T> Where<T>(this IEnumerable<T> source, Criteria c)
{
var sourceType = typeof(T);
var propertyMember = sourceType.GetProperty(c.Property);
Func<string, bool> predicate = null;
switch (c.Operator)
{
case "equal":
predicate = (v) => v == c.Value;
break;
// other operators
default:
throw new ArgumentException("Unsupported operator.");
}
return source.Where(v => predicate((string)propertyMember.GetMethod.Invoke(v, null)));
}
}
Which you could make use of in your code:
void FooBar()
{
Criteria c = new Criteria()
{
Property = "Name",
Operator = "equal",
Value = "foo"
};
var source = new Receipt[2];
source[0] = new Receipt { Name = "foo", Amount = 1 };
source[1] = new Receipt { Name = "bar", Amount = 2 };
var result = source.Where(c);
}
This is just to give you an idea. Improvements would be error handling (property not found, invalid cast, null values, etc.), refactoring to enable unit testing (e.g. inject the select "strategy") and performance (e.g. building, compiling and caching expression trees instead of reflection). This should give you enough keywords to learn about. Hope this helps.

How to store a mapping between a type A and a run-time Func<A,B> type transformation?

Runtime Mapping objects of type A into a similar type A`
List of types is discovered run-time using System.Reflection
An expression Expression<Func<object, object>> is made for each appropriate Type. The function converts object of type oldType into an object of newType.
Sidenote:
The generic type arguments for the Func<object, object> are somewhat-misleading, it feels like it should be Func<newType,oldType>, but the types are not known compile-time:
Expression<Func<object, object>> GetConvertExpression(Type oldType, Type newType)
The expressions are compiled and stored into a Dictionary<Type, Func<object, object>>
Question - How to solve the downsides of the current approach?
A dictionary lookup is necessary to convert an object. I had thought of baking in a big if/then/else or case statment into a big unified Expression, this could work well with a small number of types.
Unnecessary casting operations within the generated IL.
Argument and return type of the generated code is of type object. This basically ensures that there will be casting operations all around. Especially when a large transformation of List<oldType> to List<newType> has to be performed.
Generated code from the Expression:
object convert(object oldInst)
{
newType newInst = newType();
oldType oldInstCast = (oldType)oldInst;
newInst.field1 = oldInstCast.field1;
.......
// note: if some field of oldInst has a type in the DICTIONARY and
// it's value is not null, we have to perform the convert
// operation recursively for that field
if(oldInstCast.field3 != null)
{
// Type of the field was determined when this code was generated
// And it is known that the dictionary will contain it
Func<object, object> transformFun = dictionary[oldFieldType];
newInst.field3 = (newFieldType)transformFun(oldInstCast.field3);
}
newInst.fieldz = oldInstCast.fieldz;
return newInst;
}

EDIT: I wrote the original answer at around midnight. I think it shows. It sounds like what you need most are how to construct the delegate when the parameter and return types are only known at runtime. You get around this problem by using object. You can still do it strongly typed. The most relevant calls are:
Type delegateType = typeof(Func<,>).MakeGenericType(typeof(A), typeof(NewA));
This creates the delegate type Func<A, NewA> as for converting from A to NewA. To create an Expression<Func<A, NewA>> when A and NewA aren't necessarily known at compile time, you can use this overload of Expression.Lambda.
Previous answer:
Excellent question. I had fun trying to draw up a solution with stronger typing.
I did a mockup of this and I see no reason you can't generate expressions of type Expression<Func<TOld, TNew>> instead of Expression<Func<object, object>>. You wouldn't need casting inside of those conversion functions. However, I'm still not sure it's a better solution. Here's what I came up with so far:
The meat of the conversion/reflection/expression code is in a class I called ConverterDictionary, which builds these converters then holds them in a dictionary, and offers them up in a nicely typed way. I'm guessing that 95% of this is non-material for you since it sounds like you already figured out the reflection and expression-building portions of your problem. Everything else is 100% sample code.
Let me know what needs clarification or further explanation, or what could be more helpful.
class Program
{
static void Main(string[] args)
{
var cv = new ConverterDictionary();
var type2_converter = cv.GetConverter<Type2, Type2_New>();
var converterF = type2_converter.Compile();
var type2 = new Type2 { Field1 = "Hello", Field2 = "World" };
var type2_New = converterF(type2);
}
}
public class ConverterDictionary
{
private Dictionary<Tuple<Type, Type>, object> conversionDict = new Dictionary<Tuple<Type, Type>, object>();
public ConverterDictionary()
{
var convertPairs = Assembly.GetExecutingAssembly().GetTypes()
.Where(t => t.CustomAttributes.Any(a => a.AttributeType == typeof(ClassConvertAttribute)))
.Select(t => Tuple.Create(t, (Type)(t.CustomAttributes.First().NamedArguments[0].TypedValue.Value)))
.ToList();
foreach(var pair in convertPairs)
{
var fromType = pair.Item1;
var toType = pair.Item2;
var fieldConversions = fromType.GetFields()
.Where(f => f.CustomAttributes.Any(a => a.AttributeType == typeof(FieldConvertAttribute)))
.Select(f => Tuple.Create(f, toType.GetField((string)(f.CustomAttributes.First().NamedArguments[0].TypedValue.Value))))
.ToList();
var delegateType = typeof(Func<,>).MakeGenericType(fromType, toType);
var param1 = Expression.Parameter(fromType, "oldInst");
var returnVar = Expression.Variable(toType, "newInst");
var expr = Expression.Lambda(
delegateType,
Expression.Block(
new ParameterExpression[] { returnVar },
new Expression[]
{
Expression.Assign(
returnVar,
Expression.New(toType)
),
}.Concat(
fieldConversions.Select(fc =>
Expression.Assign(
Expression.MakeMemberAccess(
returnVar,
fc.Item2
),
Expression.MakeMemberAccess(
param1,
fc.Item1
)
)
)
).Concat(
new Expression[] { returnVar }
)
),
param1
);
conversionDict[pair] = expr;
}
}
public Expression<Func<TFrom, TTo>> GetConverter<TFrom, TTo>()
{
var key = Tuple.Create(typeof(TFrom), typeof(TTo));
if (conversionDict.ContainsKey(key))
return conversionDict[key] as Expression<Func<TFrom, TTo>>;
return null;
}
}
[ClassConvert(ToType=typeof(Type1_New))]
public class Type1
{
[FieldConvert(ToFieldName = "Field1")]
public string Field1;
}
[ClassConvert(ToType = typeof(Type2_New))]
public class Type2
{
[FieldConvert(ToFieldName="Field1")]
public string Field1;
[FieldConvert(ToFieldName = "Field2_New")]
public string Field2;
}
public class Type1_New
{
public string Field1;
}
public class Type2_New
{
public string Field1;
public string Field2_New;
}
[AttributeUsage(AttributeTargets.Class, AllowMultiple=false)]
public class ClassConvertAttribute : Attribute
{
public Type ToType { get; set; }
}
[AttributeUsage(AttributeTargets.Field, AllowMultiple = false)]
public class FieldConvertAttribute : Attribute
{
public string ToFieldName { get; set; }
}

select a property dynamically in linq

I have a property whose name is in a variable (userFilters.property), which can be of type string/enum/number.
I want to select the distinct values of that property dynamically. how do I do it.
var query = CreatePincodeQuery(useFilters);
//userFilters.property contains the property to be selected
Expression<Func<PincodeData, string>> selectExpr = null;
IList<string> list = query.Select(selectExpr)
.Distinct()
.OrderBy(selectExpr)
.ToList();
return list.;
I should be creating the expression of type Expression<Func<PincodeData, string>> selectExpr to use as part of select & orderBy block.
How do I do it?
I looked at the solution provided here, here and here, but not able to understand how to I modify those to fit my need.
EDIT
came up with below solution, as expected its not working, and how do i convert the value to string.
Func<TEntity, string> CreateNewStatement<TEntity>(string field)
{
//https://stackoverflow.com/questions/16516971/linq-dynamic-select
var xParameter = Expression.Parameter(typeof(TEntity), "o");
var property = Expression.Property(xParameter, typeof(TEntity).GetProperty(field));
var lambda = Expression.Lambda<Func<TEntity, string>>(property, xParameter);
return lambda.Compile();
}
EDIT
I changed the type from string to object, but its still failing while creating the lambda for enum types, what could be the reason

In the last case, when you changed the return type to object, you didn't convert the value type to it.
That mistake is rather common because we, C# developers, writing our own C# code, get used to relying on the C# compiler to make the value type to object conversion automatically. But, that's not the case for expressions, so you need to do the conversion explicitly.
Your particular case is solved below, but! It is very, very easy. Since you are on the way to dynamically building queries based on the end-user input your next steps might become complicated very fast (for instance, filtering with logical conditions). Once that happens, consider using Dynamic LINQ, which is very powerful for such tasks.
public class Entity
{
public string StringProp { get; }
public int IntProp { get; }
public DayOfWeek EnumProp { get; }
public Entity(string stringProp, int intProp, DayOfWeek enumProp)
{
StringProp = stringProp;
IntProp = intProp;
EnumProp = enumProp;
}
}
[Test]
public void Expressions()
{
var entities = new List<Entity>
{
new("Prop3", 3, DayOfWeek.Wednesday),
new("Prop2", 2, DayOfWeek.Tuesday),
new("Prop1", 1, DayOfWeek.Monday),
};
const string stringPropName = nameof(Entity.StringProp);
var stringStatement = CreateNewStatement<Entity>(stringPropName);
const string intPropName = nameof(Entity.IntProp);
var intStatement = CreateNewStatement<Entity>(intPropName);
const string enumPropName = nameof(Entity.EnumProp);
var enumStatement = CreateNewStatement<Entity>(enumPropName);
IList<object> listOfStrings = entities.Select(stringStatement).Distinct().OrderBy(i => i).ToList();
TestContext.WriteLine(string.Join(",", listOfStrings));
IList<object> listOfInts = entities.Select(intStatement).Distinct().OrderBy(i => i).ToList();
TestContext.WriteLine(string.Join(",", listOfInts));
IList<object> listOfEnums = entities.Select(enumStatement).Distinct().OrderBy(i => i).ToList();
TestContext.WriteLine(string.Join(",", listOfEnums));
}
private static Func<TEntity, object> CreateNewStatement<TEntity>(string fieldName)
{
var parameter = Expression.Parameter(typeof(TEntity), "o");
var propertyInfo = typeof(TEntity).GetProperty(fieldName);
if (propertyInfo == null)
throw new InvalidOperationException($"Property '{fieldName}' not found");
Expression body = Expression.Property(parameter, propertyInfo);
if (propertyInfo.PropertyType.IsValueType)
body = Expression.Convert(body, typeof(object));
var lambda = Expression.Lambda<Func<TEntity, object>>(body, parameter);
return lambda.Compile();
}