We Keep Coding

Filter List based on property names decided at runtime

I'm wondering if is it possible to make the "property name" dynamic in the filter expressions
Consider scenario
List<Person> GetPerson(int countryID, int stateID, int cityID, int zip)
{
//List of person can be filtered based on below line of code
List<Person> filteredPersons= persons.FindAll(rule => rule.CountryID == countryID).ToList();
//is it possible to specify ".Country" dynamically. something like
List<Person> filteredPersons= persons.FindAll(rule => rule."propertyName"== countryID).ToList();
}

Considering your example one method you could employ is using the .Where() extension instead of FindAll() this could then allow you to build your expression manually. A quick example would be as below.
static List<Person> GetPerson(int countryID, int stateID, int cityID, int zip)
{
//create a new expression for the type of person this.
var paramExpr = Expression.Parameter(typeof(Person));
//next we create a property expression based on the property named "CountryID" (this is case sensitive)
var property = Expression.Property(paramExpr, "CountryID");
//next we create a constant express based on the country id passed in.
var constant = Expression.Constant(countryID);
//next we create an "Equals" express where property equals containt. ie. ".CountryId" = 1
var idEqualsExpr = Expression.Equal(property, constant);
//next we convert the expression into a lamba expression
var lExpr = Expression.Lambda<Func<Person, bool>>(idEqualsExpr, paramExpr);
//finally we query our dataset
return persons.AsQueryable().Where(lExpr).ToList();
}
So this looks like alot of code but what we have basically done is manually constructed the expression tree with the end result looking similar to (and functioning as)
return persons.AsQueryable().Where(p => p.CountryId = countryId);
Now we can take this forward lets say you wanted to query for multiple properties using an and\or based on the method call. Ie you could change all your "filter" paramters to be Nullable and check if a value is passed in we filter for it such as.
static List<Person> GetPerson(int? countryID = null, int? stateID = null, int? cityID = null, int? zip = null)
{
//create a new expression for the type of person this.
var paramExpr = Expression.Parameter(typeof(Person));
//var equalExpression = Expression.Empty();
BinaryExpression equalExpression = null;
if (countryID.HasValue)
{
var e = BuildExpression(paramExpr, "CountryId", countryID.Value);
if (equalExpression == null)
equalExpression = e;
else
equalExpression = Expression.And(equalExpression, e);
}
if (stateID.HasValue)
{
var e = BuildExpression(paramExpr, "StateID", stateID.Value);
if (equalExpression == null)
equalExpression = e;
else
equalExpression = Expression.And(equalExpression, e);
}
if (equalExpression == null)
{
return new List<Person>();
}
//next we convert the expression into a lamba expression
var lExpr = Expression.Lambda<Func<Person, bool>>(equalExpression, paramExpr);
//finally we query our dataset
return persons.AsQueryable().Where(lExpr).ToList();
}
static BinaryExpression BuildExpression(Expression expression, string propertyName, object value)
{
//next we create a property expression based on the property named "CountryID" (this is case sensitive)
var property = Expression.Property(expression, propertyName);
//next we create a constant express based on the country id passed in.
var constant = Expression.Constant(value);
//next we create an "Equals" express where property equals containt. ie. ".CountryId" = 1
return Expression.Equal(property, constant);
}
Now this is a bit more code but as you can see we are now accepting null values for all our parameters and building our query for additional properties.
Now you could take this further (as assuming a more generic method is required) of passing in a Dictionary<string, object> of the properties \ values to query. This can be done as an extension method on an IEnumerable<T> as listed below.
public static class LinqExtensions
{
public static IEnumerable<T> CustomParameterQuery<T>(this IEnumerable<T> entities, Dictionary<string, object> queryVars)
{
if (entities.Count() == 0 || queryVars.Count == 0)
{
return entities;
}
//create a new expression for the type of person this.
var paramExpr = Expression.Parameter(typeof(T));
BinaryExpression equalExpression = null;
foreach (var kvp in queryVars)
{
var e = BuildExpression(paramExpr, kvp.Key, kvp.Value);
if (equalExpression == null)
equalExpression = e;
else
equalExpression = Expression.And(equalExpression, e);
}
if (equalExpression == null)
{
return new T[0];
}
//next we convert the expression into a lamba expression
var lExpr = Expression.Lambda<Func<T, bool>>(equalExpression, paramExpr);
//finally we query our dataset
return entities.AsQueryable().Where(lExpr);
}
static BinaryExpression BuildExpression(Expression expression, string propertyName, object value)
{
//next we create a property expression based on the property name
var property = Expression.Property(expression, propertyName);
//next we create a constant express based on the country id passed in.
var constant = Expression.Constant(value);
//next we create an "Equals" express where property equals containt. ie. ".CountryId" = 1
return Expression.Equal(property, constant);
}
}
Now this could be easily called as:
var dict = new Dictionary<string, object>
{
{ "CountryID", 1 },
{ "StateID", 2 }
};
var e = persons.CustomParameterQuery(dict);
Now obiously this is not a perfect example, but should get you moving in the right direction. Now you "could" also support "OR" statements etc by using Expression.Or instead of Expression.And when combining expressions.
I Must add this is very error prone as it requires the Property names to be exact to the entity, you could use reflection on T and determine if the PropertyName exists and If it is in the correct casing.

Search for System.Linq.Dynamic in NuGet. That's the easiest way to start with Dynamic Linq.

Dynamically generate lambda expression with constants from variables

I've been troubleshooting an unusual issue with some EF 6 code where queries running with the Oracle.ManagedDataAccess.Client driver are sometimes taking several minutes to return results even when the underlying query executes within 2ms. An example query would be as follows:
var result = users.Where(u => u.username == varUserName).FirstOrDefault();
This query might take several minutes to return, however if I replace the query with the same thing with a constant in the lambda function, it runs instantaneously:
var result = users.Where(u => u.username == "testUsername").FirstOrDefault();
To work around this issue, I can either write parameterised SQL queries, or I can manually generate an appropriate lambda expression tree:
var userParam = Expression.Parameter(typeof(Entity.User), "user");
var userNameField = Expression.Property(userParam, "username");
var userNameConstant = Expression.Constant(varUserName, typeof(string));
var equalUserName = Expression.Equal(userNameField, userNameConstant);
var lambda = Expression.Lambda<Func<Entity.User, bool>>(equalUserName, new ParameterExpression[] { userParam });
var result = users.Where(lambda).FirstOrDefault();
Because this works, it begs the question: is there a way to easily generate lambda expression trees which result in variables being directly included as constants, instead of references to variables?
For example, something like this would be ideal:
var lambdaExpression = (u => u.username == varUserName).ReplaceVariablesWithConstants();

It can be done relatively easy with ExpressionVisitor which evaluates the ConstantExpression members like this:
public static class ExpressionUtils
{
public static Expression<TDelegate> ReplaceVariablesWithConstants<TDelegate>(this Expression<TDelegate> source)
{
return source.Update(
new ReplaceVariablesWithConstantsVisitor().Visit(source.Body),
source.Parameters);
}
class ReplaceVariablesWithConstantsVisitor : ExpressionVisitor
{
protected override Expression VisitMember(MemberExpression node)
{
var expression = Visit(node.Expression);
if (expression is ConstantExpression)
{
var variable = ((ConstantExpression)expression).Value;
var value = node.Member is FieldInfo ?
((FieldInfo)node.Member).GetValue(variable) :
((PropertyInfo)node.Member).GetValue(variable);
return Expression.Constant(value, node.Type);
}
return node.Update(expression);
}
}
}

It's kinda hard. You need to modify the ExpressionsTree with a ExpressionsVisitor. It would probably become something like:
var lambdaExpression = ReplaceVariablesWithConstants(u => u.username == varUserName);

Execution-Deferred IQueryable<T> from Dynamic Linq?

I am using Dynamic Linq to perform some queries (sorry but it's my only option). As a result, I am getting an IQueryable instead of an IQueryable<T>. In my case, I want an IQueryable<Thing> where Thing is a concrete type.
My query is as such:
public IQueryable<Thing> Foo(MyContext db)
{
var rootQuery = db.People.Where(x => x.City != null && x.State != null);
var groupedQuery = rootQuery.GroupBy("new ( it.City, it.State )", "it", new []{"City", "State"});
var finalLogicalQuery = groupedQuery.Select("new ( Count() as TotalNumber, Key.City as City, Key.State as State )");
var executionDeferredResults = finalLogicalQuery.Take(10); // IQueryable
IQueryable<Thing> executionDeferredTypedThings = ??; // <--- Help here!!!!
return executionDeferredTypedThings;
}
My Thing.cs:
public class Thing
{
public int TotalNumber { get; set; }
public string City { get; set; }
public string State { get; set; }
}
Yes, I know the exact above thing can be done without Dynamic Linq but I have some variableness going on that I've simplified out of here. I can get it to work with my variableness if my return type is simply IQueryable but I can't figure out how to convert to IQueryable<Thing> while keeping it execution-deferred and while also keeping Entity Framework happy. I do have the dynamic Select always returning something (with the correct data) that looks like a Thing. But I simply can't figure how to return the IQueryable<Thing> and could use some help there. Thanks!!
Failed Attempt 1
Based on Rex M's suggestion, I am now trying to use AutoMapper to solve this problem (although I am not committed to this approach and am willing to try other approaches). For the AutoMapper approach, I am doing it as such:
IQueryable<Thing> executionDeferredTypedThings = executionDeferredResults.ProjectTo<Thing>(); // <--- Help here!!!!
But this results in an InvalidOperationException:
Missing map from DynamicClass2 to Thing. Create using Mapper.CreateMap.
The thing is, while I have defined Thing, I have not defined DynamicClass2 and as such, I cannot map it.
Failed Attempt 2
IQueryable<Thing> executionDeferredTypedThings = db.People.Provider.CreateQuery<Thing>(executionDeferredResults.Expression);
This gives an InvalidCastException and seems to be the same underlying problem that the above AutoMapper fail hits:
Unable to cast object of type 'System.Data.Entity.Infrastructure.DbQuery'1[DynamicClass2]' to type 'System.Linq.IQueryable'1[MyDtos.Thing]'.

You can use AutoMapper's Queryable Extensions to produce an IQueryable which wraps the underlying IQueryable, thus preserving the original IQueryable's IQueryProvider and the deferred execution, but adds in a mapping/translating component to the pipeline to convert from one type to another.
There's also AutoMapper's UseAsDataSource which makes some common query extension scenarios easier.

If I understand correctly, the following extension method should do the job for you
public static class DynamicQueryableEx
{
public static IQueryable<TResult> Select<TResult>(this IQueryable source, string selector, params object[] values)
{
if (source == null) throw new ArgumentNullException("source");
if (selector == null) throw new ArgumentNullException("selector");
var dynamicLambda = System.Linq.Dynamic.DynamicExpression.ParseLambda(source.ElementType, null, selector, values);
var memberInit = dynamicLambda.Body as MemberInitExpression;
if (memberInit == null) throw new NotSupportedException();
var resultType = typeof(TResult);
var bindings = memberInit.Bindings.Cast<MemberAssignment>()
.Select(mb => Expression.Bind(
(MemberInfo)resultType.GetProperty(mb.Member.Name) ?? resultType.GetField(mb.Member.Name),
mb.Expression));
var body = Expression.MemberInit(Expression.New(resultType), bindings);
var lambda = Expression.Lambda(body, dynamicLambda.Parameters);
return source.Provider.CreateQuery<TResult>(
Expression.Call(
typeof(Queryable), "Select",
new Type[] { source.ElementType, lambda.Body.Type },
source.Expression, Expression.Quote(lambda)));
}
}
(Side note: Frankly I have no idea what values argument is for, but added it to match the corresponding DynamicQueryable.Select method signature.)
So your example will become something like this
public IQueryable<Thing> Foo(MyContext db)
{
var rootQuery = db.People.Where(x => x.City != null && x.State != null);
var groupedQuery = rootQuery.GroupBy("new ( it.City, it.State )", "it", new []{"City", "State"});
var finalLogicalQuery = groupedQuery.Select<Thing>("new ( Count() as TotalNumber, Key.City as City, Key.State as State )"); // IQueryable<Thing>
var executionDeferredTypedThings = finalLogicalQuery.Take(10);
return executionDeferredTypedThings;
}
How it works
The idea is quite simple.
The Select method implementation inside the DynamicQueryable looks something like this
public static IQueryable Select(this IQueryable source, string selector, params object[] values)
{
if (source == null) throw new ArgumentNullException("source");
if (selector == null) throw new ArgumentNullException("selector");
LambdaExpression lambda = DynamicExpression.ParseLambda(source.ElementType, null, selector, values);
return source.Provider.CreateQuery(
Expression.Call(
typeof(Queryable), "Select",
new Type[] { source.ElementType, lambda.Body.Type },
source.Expression, Expression.Quote(lambda)));
}
What it does is to dynamically create a selector expression and bind it to the source Select method. We take exactly the same approach, but after modifying the selector expression created by the DynamicExpression.ParseLambda call.
The only requirement is that the projection is using "new (...)" syntax and the names and types of the projected properties match, which I think fits in your use case.
The returned expression is something like this
(source) => new TargetClass
{
TargetProperty1 = Expression1(source),
TargetProperty2 = Expression2(source),
...
}
where TargetClass is a dynamically generated class.
All we want is to keep the source part and just replace that target class/properties with the desired class/properties.
As for the implementation, first the property assignments are converted with
var bindings = memberInit.Bindings.Cast<MemberAssignment>()
.Select(mb => Expression.Bind(
(MemberInfo)resultType.GetProperty(mb.Member.Name) ?? resultType.GetField(mb.Member.Name),
mb.Expression));
and then the new DynamicClassXXX { ... } is replaced with with
var body = Expression.MemberInit(Expression.New(resultType), bindings);

Would something like this be of benefit to you?
public static IQueryable<TEntity> GetQuery<TEntity>(this DbContext db, bool includeReferences = false) where TEntity : class
{
try
{
if (db == null)
{
return null;
}
var key = typeof(TEntity).Name;
var metaWorkspace = db.ToObjectContext().MetadataWorkspace;
var workspaceItems = metaWorkspace.GetItems<EntityType>(DataSpace.OSpace);
var workspaceItem = workspaceItems.First(f => f.FullName.Contains(key));
var navProperties = workspaceItem.NavigationProperties;
return !includeReferences
? db.Set<TEntity>()
: navProperties.Aggregate((IQueryable<TEntity>)db.Set<TEntity>(), (current, navProperty) => current.Include(navProperty.Name));
}
catch (Exception ex)
{
throw new ArgumentException("Invalid Entity Type supplied for Lookup", ex);
}
}
You may want to take a look into the Generic Search project on Github located here:
https://github.com/danielpalme/GenericSearch

There is no need for Dynamic Linq on this one.
var groupedQuery = from p in db.People
where p.City != null && p.State != null
group p by new {p.City, p.State}
into gp
select new Thing {
TotalNumber = gp.Count(),
City = gp.Key.City,
State = gp.Key.State
};
IQueryable<Thing> retQuery = groupedQuery.AsQueryable();
retQuery= retQuery.Take(10);
return retQuery;

Resolving generic lambda expression type

I have a method that calls Entity Framework functions. It is responsible for sending the "select clause", "order by clause" and possible includes (considering that I'm using lazy loading). The method looks like this:
public IEnumerable<TReturn> GetAll<TReturn, TOrderKey>(
Expression<Func<TEntity, TReturn>> selectExp,
Expression<Func<TEntity, TOrderKey>> orderbyExp,
Boolean descending,
params Expression<Func<TEntity, Object>>[] includeExps)
{
var query = DbSet.AsQueryable();
query = !descending ? query.OrderBy(orderByExp) : query.OrderByDescending(orderByExp);
if (includeExps != null)
query = includeExps.Aggregate(query, (current, exp) => current.Include(exp));
return query.Select(selectExp).ToList();
}
When I call:
_service.GetAll(i => new { i.Name}, i => i.Name, false, null);
It works fine! That is the generated SQL is the exactly as I wanted.
However, considering a real scenario (in my case I'm using asp.net mvc), I have an Action method that gets the order parameter from the client.
That is the method:
public JsonResult GetAllUsers(string sortColumn, bool sortDescending)
{
//sortColumn string must be translated in a Expression
var users = _service.GetAll(i => new { i.Name, i.Email }, i => i.Name, sortDescending, null);
//
//
}
My first attempt was to create an expression for every column, like this:
public JsonResult GetAllUsers(string sortColumn, bool sortDescending)
{
//I don't what is the Type that I should put here
//It can be anything, like: Expression<Func<User, String>>,
//Expression<Func<User, Guid>>, Expression<Func<User, Int>>
?Type? orderExp;
switch(sortColumn)
{
case "UserId":
//Expression<Func<User, Guid>>
orderExp = i => i.UserId;
break;
case "Name":
//Expression<Func<User, String>>
orderExp = i => i.Email;
break;
}
//sortColumn string must be translated in a Expression
var users = _service.GetAll(i => new { i.Name, i.Email }, orderExp, sortDescending, null);
//
//
}
I want to create an expression based on sortProperty, there's a lot of information about it over the stackoverflow, however (see the action method) the variable must be typed before the process. The GetAll method can't be called inside every "case" because it returns an anonymous type.
I can't convert all columns to Expression<Func<User, Object>> because entity framework does not support it.
Linq.Dynamic should help, but I don't want to use string parameters.

You could overload your GetAll method as follows:
public IEnumerable<TReturn> GetAll<TReturn>(
Expression<Func<TEntity, TReturn>> selectExp,
string orderColumnName,
Boolean descending,
params Expression<Func<TEntity, Object>>[] includeExps)
{
var entityType = typeof(TEntity);
var prop = entityType.GetProperty(orderColumnName);
var param = Expression.Parameter(entityType, "i");
var orderExp = Expression.Lambda(
Expression.MakeMemberAccess(param, prop), param);
// get the original GetAll method overload
var method = this.GetType().GetMethods().Where(m => m.Name == "GetAll" && m.GetGenericArguments().Length == 2);
var actualMethod = method.First().MakeGenericMethod(typeof(TReturn), prop.PropertyType);
return (IEnumerable<TReturn>)actualMethod.Invoke(this, new object[] { selectExp, orderExp, descending, includeExps });
}
Just add a few more checks for null values, otherwise invalid sortColumn names will come back haunting you.
Use it similar to your current approach:
_service.GetAll(i => new { i.Name, i.Email }, sortColumn, sortDescending, null);

Create a JOIN at the run time in a EF6 extended DbContext class

I need to create a table joins at the runtime with a given configuration.
In this case, I have the IQueryable property which is the root and I need to create the join dynamically.
Here what I have tried out.
public class MyDbContext : DbContext
{
public IQueryable<T> AsDynamicQueryable<T>() where T : class
{
var predicate = default(Func<T, bool>); // This is a Dynamically generated predicate
var query = this.Set<T>().Where(predicate).AsQueryable();
// Now here I need to append a JOIN to the above 'query'
// So far, this is what I have done.
var rootType = typeof(T);
var innerType = Type.GetType("This type takes from the configuration");
var innerExpression = this.Set(innerType).AsQueryable();
var paramOne = Expression.Parameter(rootType, "p1");
var paramTwo = Expression.Parameter(innerType, "p2");
var outerKeySelector = Expression.Property(paramOne, "property_one"); //'property_one' is a property of a first parameter which takes from the configuration
var outerKeySelectorExpression = Expression.Lambda(outerKeySelector, paramOne); // (p1)=>p1.property_one
var innerKeySelector = Expression.Property(paramTwo, "property_two"); //'property_two' is a property of a 2nd parameter which takes from the configuration
var innerKeySelectorExpression = Expression.Lambda(innerKeySelector, paramTwo); // (p2)=>p2.property_two
var resultSelector = Expression.Lambda(paramOne, paramOne, paramTwo); // (p1,p2)=>p1
var joinMethod = typeof(Queryable)
.GetMethods()
.First(m => m.Name == "Join" && m.GetParameters().Length == 5)
.MakeGenericMethod(rootType, innerType, typeof(int), rootType);
// 1st Apptempt.
// I'm not sure that I can execute the JOIN method like this.
// But anyway, this gives the below error when I try to execute via taking Count();
// "This method supports the LINQ to Entities infrastructure and is not intended to be used directly from your code."
var newQuery = (IQueryable<T>)joinMethod
.Invoke(
query,
new object[]
{
query,
innerExpression,
outerKeySelectorExpression,
innerKeySelectorExpression,
resultSelector
});
var tt = newQuery.Count(); // Here I just try to execute the expression to check whether it works before I return the Queryable.
// 2nd Attempt
// This also gives the following error when I try to execute via taking Count();
// Unable to create a constant value of type '<type name of the root(T) type>'. Only primitive types or enumeration types are supported in this context.
var joinMethodCallExpression = Expression.Call(
null,
joinMethod,
query.Expression,
innerExpression.Expression,
outerKeySelectorExpression,
innerKeySelectorExpression,
resultSelector);
var xx = this.Set<T>().AsQueryable().Provider.CreateQuery<T>(joinMethodCallExpression);
var te = xx.Count(); // Here I just try to execute the expression to check whether it works before I return the Queryable.
throw new NotImplementedException();
}
}
Highly appreciate if someone can point out the correct way of doing this.

Here is the code. I've added my comments inside the code:
public IQueryable<T> AsDynamicQueryable<T>() where T : class
{
// ERROR!!! It should be Expression<Func<T, bool>>
// GetPredicate<T>() is my method to get the predicate. You must
// put here yours. IT must return an Expression<Func<T, bool>>
Expression<Func<T, bool>> predicate = GetPredicate<T>(); // This is a Dynamically generated predicate
// ERROR!!! Don't EVER use AsQueryable(), unless you exactly know
// what you are doing. In this example, your use of AsQueryable<>()
// is hiding the fact that you are executing the Where() LOCALLY,
// because it is a Where(this IEnumerable<>, Func<>) instead of
// being a Where(this IQueryable<>, Expression<>)
// If you want an IQueryable<>, put it in a IQueryable<> variable
IQueryable<T> query = this.Set<T>().Where(predicate);
var rootType = typeof(T);
var innerType = GetAsDynamicQueryableInnerType<T>();
// Same as before! Don't use .AsQueryable(). In this case, use
// IQueryable (non-generic). Note that in this case there was
// no problem with yoru code, so AsQueryable() wasn't doing
// "damage"
IQueryable innerExpression = this.Set(innerType);
var paramOne = Expression.Parameter(rootType, "p1");
var paramTwo = Expression.Parameter(innerType, "p2");
// GetPrimaryKey() is my method to get the property to use.
// it returns a string with the name of the property
string primaryKeyRootType = GetPrimaryKey(rootType);
var outerKeySelector = Expression.Property(paramOne, primaryKeyRootType); //'property_one' is a property of a first parameter which takes from the configuration
var outerKeySelectorExpression = Expression.Lambda(outerKeySelector, paramOne); // (p1)=>p1.property_one
// GetForeignKey() is my method to get the property to use.
// it returns a string with the name of the property
var foreignKeyInnerType = GetForeignKey(innerType, rootType);
var innerKeySelector = Expression.Property(paramTwo, foreignKeyInnerType); //'property_two' is a property of a 2nd parameter which takes from the configuration
var innerKeySelectorExpression = Expression.Lambda(innerKeySelector, paramTwo); // (p2)=>p2.property_two
var resultSelector = Expression.Lambda(paramOne, paramOne, paramTwo); // (p1,p2)=>p1
// Using outerKeySelector.Type as the type of the third parameter
// here. 99% it is typeof(int), but why not make it more generic?
var joinMethod = typeof(Queryable)
.GetMethods()
.First(m => m.Name == "Join" && m.GetParameters().Length == 5)
.MakeGenericMethod(rootType, innerType, outerKeySelector.Type, rootType);
// Queryable.Join is static, so the first parameter must be null!
// Then the parameters to pass to Queryable.Join are the ones you
// where using in the 1st case.
var newQuery = (IQueryable<T>)joinMethod.Invoke(
null,
new object[]
{
query,
innerExpression,
outerKeySelectorExpression,
innerKeySelectorExpression,
resultSelector
});
return newQuery;
}
And then there is the big problem: even if it can work, you can only get back a IQueryable<T>, but the result of a Join is normally a IQueryable<T+U>. I see that you wrote resultSelector = ... (p1,p2)=>p1, but is it really what you want?