Dynamically building LINQ queries from string - c#

Here's my scenario:
There is a collection of objects where each object contains a Dictionary<string, string>. The user can build a set of queries for this collection from another app to obtain a subset by selecting a Key in the Dictionary, an operator such as > or CONTAINS, etc., and a Value. They can also balance parenthesis to create groups of queries and select AND/OR operators to combine the queries.
As an example, let's say I have a collection of Car objects and the Dictionary contains keys for Make, Model, and Year.
My app is getting these queries in the form of a string like so:
"((Make = Honda) AND (Model CONTAINS Civic)) || (Year >= 2015)"
This tells me that from the collection of Car objects that I want cars that have Dictionary keys/values of <Make, Honda> and <Model, anything that contains "Civic"> OR <Year, greater than or equal to 2015>
So, I parse these out and put them into a QueryClass containing three string fields for the Key, the Operator, and the Value. I also keep track of the operator between the queries, and if they are in a group of parentheses or not.
Currently, I have to go through each QueryClass one by one performing the query, checking what the previous operator was, if it's part of a group, etc. and combining collections over and over until it reaches the end. This is tedious and seems like an awful way to do things. If there was a way to build these LINQ queries dynamically or perform SQL statements (what these are essential) on this collection it would be better.
Here's my query class that I'm storing the parsed strings in:
class QueryClass
{
public string FieldName { get; set; }
public string Operator { get; set; }
public object Value { get; set; }
public QueryClass(string pInput)
{
var returned = RegexHelpers.SplitKeyValue(pInput); //just returns a string like "Make = Honda" into three parts
if (returned != null)
{
FieldName = returned.Item1;
Operator = returned.Item2;
Value = returned.Item3;
}
}
}
My parsing class is pretty long so I won't post the whole thing but it returns a List<object> where each element is either:
A QueryClass
"AND" or "OR"
Another List which means it's a group of queries that were grouped by parentheses, containing the two choices above.
Here's an example of the List<object> I get after parsing a string:
I then just loop through each element, determine if the value is a double or string, and execute a LINQ statement on my collection. I'm checking if the operator was "AND" or "OR" (or none if it's just one query), if it's part of a group or not, and combining the results appropriately.

Here is my implementation of converting your query into a Func. Since I wasn't sure what type was in your collection, I made an interface to represent objects that had an attributes Dictionary<string, string> and processed that.
Basically I added a method to QueryClass to convert it to an Expression. It uses a helper dictionary string->lambda that builds the appropriate comparison Expression for each operator.
Then I added a class to convert the List<object> into a Func<IItem,bool> suitable for a LINQ Where filter.
public interface IItem {
Dictionary<string, string> attributes { get; set; }
}
class QueryClass {
public string FieldName { get; set; }
public string Operator { get; set; }
public object Value { get; set; }
public QueryClass(string pInput) {
var returned = RegexHelpers.SplitKeyValue(pInput); //just returns a string like "Make = Honda" into three parts
if (returned != null) {
FieldName = returned.Item1;
Operator = returned.Item2;
Value = returned.Item3;
}
}
static MethodInfo getItemMI = typeof(Dictionary<string, string>).GetMethod("get_Item");
static Dictionary<string, Func<Expression, Expression, Expression>> opTypes = new Dictionary<string, Func<Expression, Expression, Expression>> {
{ "==", (Expression lhs, Expression rhs) => Expression.MakeBinary(ExpressionType.Equal, lhs, rhs) },
{ ">=", (Expression lhs, Expression rhs) => Expression.MakeBinary(ExpressionType.GreaterThanOrEqual, Expression.Call(lhs, typeof(String).GetMethod("CompareTo", new[] { typeof(string) }), rhs), Expression.Constant(0)) },
{ "CONTAINS", (Expression lhs, Expression rhs) => Expression.Call(lhs, typeof(String).GetMethod("Contains"), rhs) }
};
static MemberInfo attribMI = typeof(IItem).GetMember("attributes")[0];
public Expression AsExpression(ParameterExpression p) {
var dictField = Expression.MakeMemberAccess(p, attribMI);
var lhs = Expression.Call(dictField, getItemMI, Expression.Constant(FieldName));
var rhs = Expression.Constant(Value);
if (opTypes.TryGetValue(Operator, out var exprMakerFn))
return exprMakerFn(lhs, rhs);
else
throw new InvalidExpressionException($"Unrecognized operator {Operator}");
}
}
public class LinqBuilder {
static Type TItems = typeof(IItem);
static Expression BuildOneLINQ(object term, ParameterExpression parm) {
switch (term) {
case QueryClass qc: // d => d[qc.FieldName] qc.Operator qc.Value
return qc.AsExpression(parm);
case List<object> subQuery:
return BuildLINQ(subQuery, parm);
default:
throw new Exception();
}
}
static Expression BuildLINQ(List<object> query, ParameterExpression parm) {
Expression body = null;
for (int queryIndex = 0; queryIndex < query.Count; ++queryIndex) {
var term = query[queryIndex];
switch (term) {
case string op:
var rhs = BuildOneLINQ(query[++queryIndex], parm);
var eop = (op == "AND") ? ExpressionType.AndAlso : ExpressionType.OrElse;
body = Expression.MakeBinary(eop, body, rhs);
break;
default:
body = BuildOneLINQ(term, parm);
break;
}
}
return body;
}
public static Func<IItem, bool> BuildLINQ(List<object> query) {
var parm = Expression.Parameter(TItems, "i");
return Expression.Lambda<Func<IItem, bool>>(BuildLINQ(query, parm), parm).Compile();
}
}
Once you have this, you can pass in a List<object> expression and then filter your collection. Given a query q and a collection of IItems cs, you can do:
var ans = cs.Where(LinqBuilder.BuildLINQ(q));

I would approach this problem little differently, since what you are already having is List<object>, which internally contain a QueryClass containing all the relevant fields containing information, FieldName,Operator and Value, where you are aware which of the binary expressions have to be bundled in a parentheses. Important point is how can you create a Run-time Expression to take care of all kinds of scenarios.
Following is a sample to mimic your scenario:
Sample Class
public class Car
{
public string Make {get; set;}
public string Model {get; set;}
public int Year {get; set;}
}
Query
((c.Make.Equals("Honda") AndAlso c.Model.Contains("Civic")) Or (c.Year >= 2015))
Linqpad Code
void Main()
{
var cars = new List<Car>();
Expression<Func<Car,bool>> e1 = c => c.Make.Equals("Honda");
Expression<Func<Car,bool>> e2 = c => c.Model.Contains("Civic");
Expression<Func<Car,bool>> e3 = c => c.Year >= 2015;
var expr1 = Expression.AndAlso(e1.Body,e2.Body);
var expr2 = e3;
var finalExpression = Expression.Or(expr1,expr2.Body);
finalExpression.Dump();
}
Purpose
As it can be seen that I have manually constructed the Expressions and finally Dump the final expression, since in Linqpad it provides a graphical representation of how shall the Expression be constructed dynamically, overall image is too big and deep to be pasted here (you may try yourself using LinqPad), but following are the relevant details:
Create a ParameterExpression, this acts as a lambda parameter representing the Car class object (this is independent of the fields of the Query class)
var parameterExpression = Expression.Parameter(typeof(Car),"c");
Create MemberExpression to access each relevant field of the Car class, which is used in the Query (this one needs Field property of the Query class)
var makeMemberAccessExpression = Expression.MakeMemberAccess(parameterExpression, typeof(Car).GetProperty("Make"));
var modelMemberAccessExpression = Expression.MakeMemberAccess(parameterExpression, typeof(Car).GetProperty("Model"));
var yearMemberAccessExpression = Expression.MakeMemberAccess(parameterExpression, typeof(Car).GetProperty("Year"));
Completing the Expression:
a.) c => c.Make.Equals("Honda") we create as follows: (this one needs Value property of the QueryClass)
var makeConstantExpression = Expression.Constant("Honda");
var makeEqualExpression = Expression.Equal(makeMemberAccessExpression, makeConstantExpression);
b.) c.Model.Contains("Civic") can be represented as follows here we need supply the MethodInfo for the string Contains method and create a MethodCallEXpression
var modelConstantExpression = Expression.Constant("Civic");
var stringContainsMethodInfo = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var modelContainsMethodExpression = Expression.Call(modelMemberAccessExpression, stringContainsMethodInfo, modelConstantExpression);
c.) c.Year >= 2015 can simply projected as:
var yearConstantExpression = Expression.Constant(2015);
var yearGreaterThanEqualExpression = Expression.GreaterThanOrEqual(yearMemberAccessExpression, yearConstantExpression);
Combining all together to form the Composite Expression:
Expressions a.) and b.) are combined together as follows:
((c.Make.Equals("Honda") AndAlso c.Model.Contains("Civic"))
var firstExpression = Expression.AndAlso(makeEqualExpression,modelContainsMethodExpression);
Expression c.) is independent:
c.Year >= 2015
var secondExpression = yearGreaterThanEqualExpression;
Final Combined Expression and Creation a Func Delegate
// Expressions combined via Or (||)
var finalCombinedExpression = Expression.Or(firstExpression,secondExpression);
// Create Lambda Expression
var lambda = Expression.Lambda<Func<Car,bool>>(finalCombinedExpression, parameterExpression);
// Create Func delegate via Compilation
var func = lambda.Compile();
func delegate thus can be used in any of the where clause in the Linq, which expects Func<Car,bool>
Design Suggestions
Using the explanation above and values from the Query Class placeholders or directly from the Dictionary it is feasible to create any number of Expressions to be used dynamically in the code, to be compiled and used as a Func delegate
Binary expressions like Equal, GreaterThan, LessThan, LessThanOrEqual,GreaterThanOrEqual are all exposed by the Expression trees to be used directly, For method like Contains available by default you need Reflection to get theb MethodInfo, similarly it can be done for the Static methods, just that there's object expressions
All Expressions expect values to be supplied Left to Right in the correct order, it cannot be random or incorrect order, else it will fail at run-time.
In your case, since you have few queries combined in parentheses and few independent, I would recommend creating multiple List<Expression>, where each List is combined in parentheses using AndAlso or OrElse and each list can thus be combined using And / Or
By this approach you shall be able to construct very complex requirements at runtime using Linq Expressions.

You should be able to use Linq Expressions (System.Linq.Expressions) and leverage predicates to handle your filtering.
public IQueryable<Car> GetCars(Expression<Func<Car, bool>> filter)
{
return context.Cars.Where(filter);
}
That said, the challenge will be to build your predicate expressions based off of your custom QueryClass object. To handle the filter on each Dictionary you can create a method to handle each:
public Expression<Func<Car, bool>> GetModelFilter(QueryClass modelQuery)
{
return modelQuery.Operator == "CONTAINS"? car => car.Model.Contains(modelQuery.Value) : car => car.Model == modelQuery.Value;
}
Considering you have a limited amount of filters, the above may be acceptable. However, when dealing with a large set this can also be done more dynamically, using reflection or a dynamic predicate builder but for simplicity you can follow the above.
HTH

Related

How can I construct a where clause in C#/EF Core with the field defined at runtime?

This seems simple, but I can't decipher the LINQ required to do it. I also can't add any new dependencies.
Basically, I'm trying to make this code generic:
if (filter.matchMode.ToUpper().Equals("EQ")) {
query = query.Where(x => x.SomeField.Equals(filter.Value));
if (filter.matchMode.ToUpper().Equals("LT")) {
query = query.Where(x => x.SomeField < filter.Value);
} else [... 5 other match modes ...]
}
Now, SomeField is only one of about 5 fields that needs this functionality. And there's 5 matching operators. I could just copy/pasta the whole thing, and then deal with the debt of having to change tons of code every time a new operator or field enters the mix, but that really doesn't seem right.
Basically, I need some way of defining SomeField at runtime, so I can factor out the whole if/else tree and use it for each supported field.
I've tried going down this road, but I think I'm misunderstanding something fundamental about expressions:
var entityType = typeof(TableObject);
ParameterExpression arg = Expression.Parameter(entityType, "x");
MemberExpression amproperty = Expression.Property(arg, "SomeField");
MemberExpression property = Expression.Property(amproperty, "Equals"); // ???
// what's next, if this is even the right direction...
EDIT: a shorter version of the question might be: how can I construct the following object foo using MemberExpressions and lambdas, such that SomeField can be passed in as a string "SomeField":
Expression<Func<TableObject, bool>> foo = x => x.SomeField.Equals("FOO");
UPDATE: here's what I ended up coming up with:
private IQueryable<TableObject> processFilter(
IQueryable<TableObject> query,
FilterItem filter,
string fieldName)
{
var entityType = typeof(TableObject);
// construct the argument and access object
var propertyInfo = entityType.GetProperty(fieldName);
ParameterExpression arg = Expression.Parameter(entityType, "x");
MemberExpression access = Expression.MakeMemberAccess(arg,
typeof(TableObject).GetProperty(fieldName)
);
// translate the operation into the appropriate Expression method
Expression oprFunc;
if (filter.MatchMode.ToUpper().Equals("EQ")) {
oprfunc =
Expression.Equal(access, Expression.Constant(filter.Value));
} else if (filter.MatchMode.ToUpper().Equals("LT")) {
oprfunc =
Expression.LessThan(access, Expression.Constant(filter.IntValue));
} else {
throw new ArgumentException(
$"invalid argument: ${filter.MatchMode}"
);
}
// construct the lambda
var func = Expression.Lambda<Func<TableObject, bool>>(oprFunc, arg);
// return the new query
return query.Where(func);
}
So far, this seems to cover most of the cases. It starts to go off the rails with nullable fields and date comparisons, but it will work for what I need it to do. The part I'm still not completely sure about is Expression.MakeMemberAccess. I've seen this written many ways and I'm not sure if that's the correct way to create that expression.
Let's say you have class like this:
class SomeClass
{
public string a1 { get; set; }
public string b1 { get; set; }
public string c1 { get; set; }
}
also let's assume you have method like this:
public List<T> GetAll(Expression<Func<T, bool>> criteria )
{
return someDataLINQCapable.Where(criteria);
}
Method could be called like this:
GetAll<SomeClass>(m => m.a1 != null && m.b1=="SomethingUseful")

Dynamic Linq Expression in .NET Core

First
I cannot use Entity Framework for this because this is a Desktop application in which all database calls are first routed through a Web API End Point. A requirement of this application is that all traffic is HTTP traffic.
Setup
Lets say I have an IEnumerable<BaseItem> where the BaseItem is a base model class that all models inherit from.
We also dynamically instantiate and populate our collection with a dynamic class and dynamic properties that match the column names and appropriate data types of the System.DataTable that is returned from the query.
These dynamic classes also inherit from BaseItem which exposes two functions:
void SetValue(string PropertyName, object Value)
object GetValue(string PropertyName).
Issue
I need to be able to write a linq query that dynamically selects properties from the collection based on the unique constraint columns from the table that the query referenced. I have that in a separate List, where the string values match the dynamic property names on the dynamic class.
I know the solution is probably to use an Expression Tree, but all the examples I find use actual property references, and in this case, the property can only be accessed and set through the GetValue, and SetValue functions. I'm having a hard time wrapping my head around how to achieve this. Below is an example of the linq query i want, if I KNEW the fields I wanted to return.
var Result = DefaultCollection.Select(x => new { BUSINESS_UNIT = x.GetValue("BUSINESS_UNIT"), FISCAL_YEAR = x.GetValue("FISCAL_YEAR") }).Distinct();
Again, I can't write this because I don't know which fields i need to include in the select list until runtime when I can reference the UniqueConstraint List.
How can I achieve this dynamically with Linq Expression?
Start To Finish Example
You've populated an IEnumerable<BaseItem> from a System.Data.DataTable and lets say the query that you ran was SELECT FIRST_NAME, LAST_NAME, EMAIL ADDRESS FROM TABLEA.
The IEnumerable is populated from automation in which a dynamic class is generated to represent a row in the table and dynamic properties are added to the class to represent each column in the table.
Accessing the properties is via methods in BaseItem string result = item.GetValue("FIRST_NAME") and item.SetValue("EMAIL_ADDRESS", "MyEmail#Email.com');
You also queried INFORMATION_SCHEMA to get unique constraints on TABLEA and you store it in a public class ConstraintModel object that has a public List<string> Columns representing the columns that make up the unique constraint. In this case, only one column "EMAIL_ADDRESS" is in the constraint.
If the user makes changes to the IEnumerable<BaseItem> and more than one item has the same EMAIL_ADDRESS, we know that the UPDATE statement back to SQL will fail. We want to check that the unique constraint hasn't been violated.
At the point the user saves, we want to run a "Validation Check" on the in memory IEnumerable object. Ideally we would write a linq query that essentailly did a SELECT SUM(1), EMAIL_ADDRESS GROUP BY EMAIL_ADDRESS, and if any of the results had more than 1 value, we know a duplicate exists. Obviously the above syntax is SQL not Linq, but you get the idea. I can't know at compile time what fields on the BaseItem i need to select and group by, so I need to leverage the Constraint.Columns object and use it to write a dynamic linq expression. Additionally, the properties on the Dynamic Class are only exposed through the methods that they inherit from BaseItem ... GetValue(string PropertyName) and SetValue(string PropertyName, object Value);
After reading your full example it sounds very much like you do not need dynamic linq at all to do what you're trying to do, but you do need to know what the Primary Key is, why not try something like:
Update I have replaced a simple pk lookup with a func PK lookup so that composite Primary Keys can be used.
List<string> primaryKeyColumns = new List<string> {"Id"}; // or whatever else
List<string> uniques = /// ConstraintModel.Columns;
List<BaseItem> baseItems = // your baseItems
// Now, wit
Func<BaseItem, BaseItem, bool> equalPrimaryKey = (left, right) => {
foreach(var pk in primaryKeyColumns) {
if(left.GetValue(pk) != right.GetValue(pk)) {
return false;
}
}
return true;
}; // or whatever else
var violators = baseItems.Select(x => {
return baseItems.Any(z => {
foreach(var unique in uniques) {
if (z.GetValue(unique) == x.GetValue(unique)
&& !equalPrimaryKey(x, z)) {
return true;
}
}
return false;
});
}).ToArray();
This will give you an array of base items that are duplicated by any unique column
Using a minified version of my EnumerableExpressionExt class for helping build Expression trees for Enumerable expressions, and my ExpressionExt helper class:
public static class EnumerableExpressionExt {
private static Type TEnumerable = typeof(Enumerable);
private static Type TypeGenArg(this Expression e, int n) => e.Type.GetGenericArguments()[n];
public static MethodCallExpression Distinct(this Expression src) => Expression.Call(TEnumerable, "Distinct", new[] { src.TypeGenArg(0) }, src);
public static MethodCallExpression Select(this Expression src, LambdaExpression resultFne) => Expression.Call(TEnumerable, "Select", new[] { src.TypeGenArg(0), resultFne.ReturnType }, src, resultFne);
}
public static class ExpressionExt {
public static ConstantExpression AsConst<T>(this T obj) => Expression.Constant(obj, typeof(T));
public static LambdaExpression AsLambda(this Expression body) => Expression.Lambda(body);
public static LambdaExpression Lambda(this ParameterExpression p1, Expression body) => Expression.Lambda(body, p1);
public static NewExpression New(this Type t, params Expression[] vals) => Expression.New(t.GetConstructors()[0], vals, t.GetProperties());
public static ParameterExpression Param(this Type t, string pName) => Expression.Parameter(t, pName);
}
You can create custom Expression extensions to translate your specific methods:
public static class MyExpressionExt {
public static MethodCallExpression GetValue(this Expression obj, string propName) =>
Expression.Call(obj, "GetValue", null, propName.AsConst());
}
You can build your Expression tree as follows:
// BaseItem x
var xParm = typeof(BaseItem).Param("x");
// typeof(new { object BUSINESS_UNIT, object FISCAL_YEAR })
var anonType = (new { BUSINESS_UNIT = default(object), FISCAL_YEAR = default(object) }).GetType();
// new { BUSINESS_UNIT = x.GetValue("BUSINESS_UNIT"), FISCAL_YEAR = x.GetValue("FISCAL_YEAR") }
var newExpr = anonType.New(xParm.GetValue("BUSINESS_UNIT"), xParm.GetValue("FISCAL_YEAR"));
// DefaultCollection.Select(x => newExpr)
var selExpr = DefaultCollection.AsConst().Select(xParm.Lambda(newExpr));
// DefaultCollection.Select(x => newExpr).Distinct()
var distExpr = selExpr.Distinct();
And you can test it (using LINQPad's Dump method) like this:
// () => DefaultCollection.Select(x => newExpr).Distinct()
var f = distExpr.AsLambda();
var fc = f.Compile();
fc.DynamicInvoke().Dump();
LINQPad is also very helpful for outputting compiler built Expression trees and seeing how they were built, or IQueryable<T> queries Expression member trees.
NOTE: I like not having to call AsConst() so I have additional helpers to make that easier as well.

Apply a dynamically created lambda to an object instance

I have some code which dynamically create a lambda starting from strings. For example, I have a filter class like the following:
public class Criteria {
public string Property { get; set; }
public string Operator { get; set; }
public string Value { get; set; }
}
And I am able to create a lambda like x => x.Name == "Foo" starting from a Criteria instance like this
Criteria c = new Criteria() {
Property = "Name",
Operator = "equal",
Value = "Foo"
}
Supposing to have a class like
public class Receipt {
public string Name { get; set; }
public int Amount { get; set; }
[other props omitted]
public ICollection<ReceiptDetail> Details { get; set; }
}
I would like to:
Apply the lambda to any object (I know that the lambda should be created with a ParameterExpression of the Receipt class)
Getting back the boolean result of the lambda (e.g. Is the name equals to Foo?)
Apply the same logic to collections Count() method (e.g. Building a lambda which checks against receipt.Details.Count()
Is this possible?
EDIT: As per the comments, I am elaborating my needs a bit more. This code will give me the chance to answer to a requirement that I have and that says: If there is a rule specified for my object then the application should behave a bit differently. While this is a common requirement, I would like to create a code which will permit me to extend it as more rules will be added. Actually I only have 5 rule types:
Verify if the input comes in a specific day of the week
Verify if the input comes in a specific time range
Verify if the field "X" of the input is less/equal/greather than a value
Verify if the field "Y" of the input contains a value
Verify if the field "Z" of the input, which is a collection, has a count that is less/equal/greather than a value
For the first 4 points I have been able to dynamically create a lambda expression, with code like in P.Brian.Mackey answer, which I could apply, using the Specification pattern, to the object itself.
The last point needs to be implemented almost in the same way but the only difference is that the left part of the expression was a method call and not a Property (specifically the ICollection<T>.Count() method)
Here's something to get you started. There is plenty of room for improvement. Especially the ugly factory. This demo is intended to show how to use Expressions to solve the problems, not as a best practices or factory pattern demo. If anything is unclear please feel free to ask for clarification.
Usage
[Test]
public void ApplySameLogicToCollectionsCount()
{
var receipt = new Receipt();
var details = new ReceiptDetail();
var details2 = new ReceiptDetail();
receipt.Details.Add(details);
receipt.Details.Add(details2);
var result = LambdaGeneratorFactory<ICollection<ReceiptDetail>>.Run(detailsCount);
Assert.IsTrue(result(receipt.Details));
}
Factory
public static class LambdaGeneratorFactory<T>
{
//This is an ugly implementation of a Factory pattern.
//You should improve this possibly with interfaces, maybe abstract factory. I'd start with an ICriteria.
public static Predicate<T> Run(Criteria criteria)
{
if (typeof(T) == typeof (Receipt))
{
return CreateLambda(criteria);
}
else if (typeof (T) == typeof (ICollection<ReceiptDetail>))
{
return CreateLambdaWithCount(criteria);
}
return null;
}
private static Predicate<T> CreateLambda(Criteria criteria)
{
ParameterExpression pe = Expression.Parameter(typeof(T), "i");
Expression left = Expression.Property(pe, typeof(T).GetProperty(criteria.Property));
Expression right = Expression.Constant(criteria.Value);
Expression predicateBody = Expression.Equal(left, right);
var predicate = Expression.Lambda<Predicate<T>>(predicateBody, new ParameterExpression[] { pe }).Compile();
return predicate;
}
private static Predicate<T> CreateLambdaWithCount(Criteria criteria)
{
ParameterExpression pe = Expression.Parameter(typeof(T), "i");
Expression count = Expression.Property(pe, typeof(T).GetProperty("Count"));
Expression left = Expression.Call(count, typeof(Object).GetMethod("ToString"));
Expression right = Expression.Constant(criteria.Value);
Expression predicateBody = Expression.Equal(left, right);
var predicate = Expression.Lambda<Predicate<T>>(predicateBody, new ParameterExpression[] { pe }).Compile();
return predicate;
}
}
Criteria
private Criteria detailsCount = new Criteria()
{
Property = "Details",
Operator = "equal",
Value = "2"
};
Switch to ICriteria and things will b cleaner. A better factory and no need for ToString. Program to an interface.
All that being said, this code feels a bit funky. What is the point of generating functions from strings? I get the feeling this is heading towards generating C# from a grammar. I'm not convinced that will scale well. For non-trivial implementations consider lex/yacc first. You can find more details for doing this in the Pragmatic Programmer "Implementing a mini language".
Yours is a fascinating question, and I'd like to understand the requirements. I created a demo, and I'm wondering, how does the demo differ from what you're trying to accomplish? There is a working version here https://dotnetfiddle.net/AEBZ1w too.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Linq.Expressions;
public class Program
{
public static void Main()
{
Criteria c = new Criteria() {
Property = "Name",
Operator = "==",
Value = "Foo" };
var queryable = (new List<Receipt>() {
new Receipt { Name = "Foo", Amount = 1 },
new Receipt { Name = "Foo", Amount = 2 },
new Receipt { Name = "Bar" }
}).AsQueryable();
var parameter = Expression.Parameter(typeof(Receipt), "x");
var property = Expression.Property(parameter, typeof(Receipt).GetProperty(c.Property));
var constant = Expression.Constant(c.Value);
var operation = Expression.Equal(property, constant);
var expression = Expression.Call(
typeof(Queryable),
"Where",
new Type[] { queryable.ElementType },
queryable.Expression,
Expression.Lambda<Func<Receipt, bool>>(operation, new ParameterExpression[] { parameter })
);
Console.WriteLine("Linq Expression: {0} \n", expression.ToString());
Console.WriteLine("Results: \n");
var results = queryable.Provider.CreateQuery<Receipt>(expression);
foreach(var r in results)
{
Console.WriteLine("{0}:{1}", r.Name, r.Amount);
}
}
}
public class Criteria
{
public string Property, Operator, Value;
}
public class ReceiptDetail
{
public string ItemName;
}
public class Receipt
{
public string Name { get; set; }
public int Amount;
public ICollection<ReceiptDetail> Details;
}
References
How to: Execute Expression Trees
How to: Use Expression Trees to Build Dynamic Queries
IQueryable Interface
Expression Methods
Expression.Property Method
You could use reflection with generics. One way to solve the problem could be an extension
public static class EnumerableExtensions
{
public static IEnumerable<T> Where<T>(this IEnumerable<T> source, Criteria c)
{
var sourceType = typeof(T);
var propertyMember = sourceType.GetProperty(c.Property);
Func<string, bool> predicate = null;
switch (c.Operator)
{
case "equal":
predicate = (v) => v == c.Value;
break;
// other operators
default:
throw new ArgumentException("Unsupported operator.");
}
return source.Where(v => predicate((string)propertyMember.GetMethod.Invoke(v, null)));
}
}
Which you could make use of in your code:
void FooBar()
{
Criteria c = new Criteria()
{
Property = "Name",
Operator = "equal",
Value = "foo"
};
var source = new Receipt[2];
source[0] = new Receipt { Name = "foo", Amount = 1 };
source[1] = new Receipt { Name = "bar", Amount = 2 };
var result = source.Where(c);
}
This is just to give you an idea. Improvements would be error handling (property not found, invalid cast, null values, etc.), refactoring to enable unit testing (e.g. inject the select "strategy") and performance (e.g. building, compiling and caching expression trees instead of reflection). This should give you enough keywords to learn about. Hope this helps.

Creating a dynamic Linq select clause from Expressions

Let's say I have defined the following variables:
IQueryable<MyClass> myQueryable;
Dictionary<string, Expression<Func<MyClass, bool>>> extraFields;
// the dictionary is keyed by a field name
Now, I want to tack on some dynamic fields to the IQueryable, so that it returns an IQueryable<ExtendedMyClass>, where ExtendedMyClass is defined as:
class ExtendedMyClass
{
public MyClass MyObject {get; set;}
public IEnumerable<StringAndBool> ExtraFieldValues {get; set;}
}
class StringAndBool
{
public string FieldName {get; set;}
public bool IsTrue {get; set;}
}
In other words, for every value in extraFields, I want to have a value in ExtendedMyClass.ExtraFieldValues representing whether that expression evaluates to True or not for that row.
I have a feeling this should be doable in dynamic Linq and LinqKit, though I've never used that seriously before. I'm also open to other suggestions, especially if this can be done in good ol' strong-typed Linq.
I am using Linq to Entities, so the query needs to translate to SQL.
So, we'll have a lot of steps here, but each individual step should be fairly short, self-contained, reusable, and relatively understandable.
The first thing we'll do is create a method that can combine expressions. What it will do is take an expression that accepts some input and generates an intermediate value. Then it will take a second expression that accepts, as input, the same input as the first, the type of the intermediate result, and then computes a new result. It will return a new expression taking the input of the first, and returning the output of the second.
public static Expression<Func<TFirstParam, TResult>>
Combine<TFirstParam, TIntermediate, TResult>(
this Expression<Func<TFirstParam, TIntermediate>> first,
Expression<Func<TFirstParam, TIntermediate, TResult>> second)
{
var param = Expression.Parameter(typeof(TFirstParam), "param");
var newFirst = first.Body.Replace(first.Parameters[0], param);
var newSecond = second.Body.Replace(second.Parameters[0], param)
.Replace(second.Parameters[1], newFirst);
return Expression.Lambda<Func<TFirstParam, TResult>>(newSecond, param);
}
To do this we simply replace all instances of the second parameter in the second expression's body with the body of the first expression. We also need to ensure both implementations use the same parameter instance for the main parameter.
This implementation requires having a method to replace all instances of one expression with another:
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
Next we'll write a method that accepts a sequences of expressions that accept the same input and compute the same type of output. It will transform this into a single expression that accepts the same input, but computes a sequence of the output as a result, in which each item in the sequence represents the result of each of the input expressions.
This implementation is fairly straightforward; we create a new array, use the body of each expression (replacing the parameters with a consistent one) as each item in the array.
public static Expression<Func<T, IEnumerable<TResult>>> AsSequence<T, TResult>(
this IEnumerable<Expression<Func<T, TResult>>> expressions)
{
var param = Expression.Parameter(typeof(T));
var body = Expression.NewArrayInit(typeof(TResult),
expressions.Select(selector =>
selector.Body.Replace(selector.Parameters[0], param)));
return Expression.Lambda<Func<T, IEnumerable<TResult>>>(body, param);
}
Now that we have all of these general purpose helper methods out of the way, we can start working on your specific situation.
The first step here is to turn your dictionary into a sequence of expressions, each accepting a MyClass and creating a StringAndBool that represents that pair. To do this we'll use Combine on the value of the dictionary, and then use a lambda as the second expression to use it's intermediate result to compute a StringAndBool object, in addition to closing over the pair's key.
IEnumerable<Expression<Func<MyClass, StringAndBool>>> stringAndBools =
extraFields.Select(pair => pair.Value.Combine((foo, isTrue) =>
new StringAndBool()
{
FieldName = pair.Key,
IsTrue = isTrue
}));
Now we can use our AsSequence method to transform this from a sequence of selectors into a single selector that selects out a sequence:
Expression<Func<MyClass, IEnumerable<StringAndBool>>> extrafieldsSelector =
stringAndBools.AsSequence();
Now we're almost done. We now just need to use Combine on this expression to write out our lambda for selecting a MyClass into an ExtendedMyClass while using the previous generated selector for selecting out the extra fields:
var finalQuery = myQueryable.Select(
extrafieldsSelector.Combine((foo, extraFieldValues) =>
new ExtendedMyClass
{
MyObject = foo,
ExtraFieldValues = extraFieldValues,
}));
We can take this same code, remove the intermediate variable and rely on type inference to pull it down to a single statement, assuming you don't find it too unweidly:
var finalQuery = myQueryable.Select(extraFields
.Select(pair => pair.Value.Combine((foo, isTrue) =>
new StringAndBool()
{
FieldName = pair.Key,
IsTrue = isTrue
}))
.AsSequence()
.Combine((foo, extraFieldValues) =>
new ExtendedMyClass
{
MyObject = foo,
ExtraFieldValues = extraFieldValues,
}));
It's worth noting that a key advantage of this general approach is that the use of the higher level Expression methods results in code that is at least reasonably understandable, but also that can be statically verified, at compile time, to be type safe. There are a handful of general purpose, reusable, testable, verifiable, extension methods here that, once written, allows us to solve the problem purely through composition of methods and lambdas, and that doesn't require any actual expression manipulation, which is both complex, error prone, and removes all type safety. Each of these extension methods is designed in such a way that the resulting expression will always be valid, so long as the input expressions are valid, and the input expressions here are all known to be valid as they are lambda expressions, which the compiler verifies for type safety.
I think it's helpful here to take an example extraFields, imagine how would the expression that you need look like and then figure out how to actually create it.
So, if you have:
var extraFields = new Dictionary<string, Expression<Func<MyClass, bool>>>
{
{ "Foo", x => x.Foo },
{ "Bar", x => x.Bar }
};
Then you want to generate something like:
myQueryable.Select(
x => new ExtendedMyClass
{
MyObject = x,
ExtraFieldValues =
new[]
{
new StringAndBool { FieldName = "Foo", IsTrue = x.Foo },
new StringAndBool { FieldName = "Bar", IsTrue = x.Bar }
}
});
Now you can use the expression trees API and LINQKit to create this expression:
public static IQueryable<ExtendedMyClass> Extend(
IQueryable<MyClass> myQueryable,
Dictionary<string, Expression<Func<MyClass, bool>>> extraFields)
{
Func<Expression<Func<MyClass, bool>>, MyClass, bool> invoke =
LinqKit.Extensions.Invoke;
var parameter = Expression.Parameter(typeof(MyClass));
var extraFieldsExpression =
Expression.Lambda<Func<MyClass, StringAndBool[]>>(
Expression.NewArrayInit(
typeof(StringAndBool),
extraFields.Select(
field => Expression.MemberInit(
Expression.New(typeof(StringAndBool)),
new MemberBinding[]
{
Expression.Bind(
typeof(StringAndBool).GetProperty("FieldName"),
Expression.Constant(field.Key)),
Expression.Bind(
typeof(StringAndBool).GetProperty("IsTrue"),
Expression.Call(
invoke.Method,
Expression.Constant(field.Value),
parameter))
}))),
parameter);
Expression<Func<MyClass, ExtendedMyClass>> selectExpression =
x => new ExtendedMyClass
{
MyObject = x,
ExtraFieldValues = extraFieldsExpression.Invoke(x)
};
return myQueryable.Select(selectExpression.Expand());
}

Lambda Expression Tree Parsing

I am trying to use Lambda Expressions in a project to map to a third party query API. So, I'm parsing the Expression tree by hand.
If I pass in a lambda expression like:
p => p.Title == "title"
everything works.
However, if my lambda expression looks like:
p => p.Title == myaspdropdown.SelectedValue
Using the .NET debugger, I don't see the actual value of that funciton. Instead I see something like:
p => p.Title = (value(ASP.usercontrols_myaspusercontrol_ascx).myaspdropdown.SelectedValue)
What gives? And when I try to grab the right side of the expression as a string, I get (value(ASP.usercontrols_myaspusercontrol_ascx).myaspdropdown.SelectedValue) instead of the actual value. How do I get the actual value?
Remember that when you're dealing with the lambda expression as an expression tree, you don't have executable code. Rather you have a tree of expression elements, that make up the expression you wrote.
Charlie Calvert has a good post that discusses this in detail. Included is an example of using an expression visualiser for debugging expressions.
In your case, to get the value of the righthand side of the equality expression, you'll need to create a new lambda expression, compile it and then invoke it.
I've hacked together a quick example of this - hope it delivers what you need.
public class Class1
{
public string Selection { get; set; }
public void Sample()
{
Selection = "Example";
Example<Book, bool>(p => p.Title == Selection);
}
public void Example<T,TResult>(Expression<Func<T,TResult>> exp)
{
BinaryExpression equality = (BinaryExpression)exp.Body;
Debug.Assert(equality.NodeType == ExpressionType.Equal);
// Note that you need to know the type of the rhs of the equality
var accessorExpression = Expression.Lambda<Func<string>>(equality.Right);
Func<string> accessor = accessorExpression.Compile();
var value = accessor();
Debug.Assert(value == Selection);
}
}
public class Book
{
public string Title { get; set; }
}
To get the actual value, you need to apply the logic of the expression tree to whatever context you've got.
The whole point of expression trees is that they represent the logic as data rather than evaluating the expression. You'll need to work out what the lambda expression truly means. That may mean evaluating some parts of it against local data - you'll need to decide that for yourself. Expression trees are very powerful, but it's not a simple matter to parse and use them. (Ask anyone who's written a LINQ provider... Frans Bouma has bemoaned the difficulties several times.)
Just been struggling with exactly the same issue, thanks Bevan. On an extension, the following is a generic pattern you can use to extract the value (using this in my query engine).
[TestFixture]
public class TestClass
{
[Test]
public void TEst()
{
var user = new User {Id = 123};
var idToSearch = user.Id;
var query = Creator.CreateQuery<User>()
.Where(x => x.Id == idToSearch);
}
}
public class Query<T>
{
public Query<T> Where(Expression<Func<T, object>> filter)
{
var rightValue = GenericHelper.GetVariableValue(((BinaryExpression)((UnaryExpression)filter.Body).Operand).Right.Type, ((BinaryExpression)((UnaryExpression)filter.Body).Operand).Right);
Console.WriteLine(rightValue);
return this;
}
}
internal class GenericHelper
{
internal static object GetVariableValue(Type variableType, Expression expression)
{
var targetMethodInfo = typeof(InvokeGeneric).GetMethod("GetVariableValue");
var genericTargetCall = targetMethodInfo.MakeGenericMethod(variableType);
return genericTargetCall.Invoke(new InvokeGeneric(), new[] { expression });
}
}
internal class InvokeGeneric
{
public T GetVariableValue<T>(Expression expression) where T : class
{
var accessorExpression = Expression.Lambda<Func<T>>(expression);
var accessor = accessorExpression.Compile();
return accessor();
}
}
I'm not sure I understand. Where are you "seeing" that? Is that at design-time or run-time? Lambda expressions can be thought of essentially as anonymous delegates, and will operate with deferred execution. So you shouldn't expect to see the value assigned until after execution has passed that line, obviously.
I don't think that's really what you mean though... if you clarify the question a bit maybe I can help :)

Categories