We Keep Coding

How Can I Add a Dynamically Generated Where Expression for a Navigable Object in My Linq-To-SQL Query?

Background
My client would like to have a method of sending over an array of field (string), value (string), and comparison (enum) values in order to retrieve their data.
public class QueryableFilter {
public string Name { get; set; }
public string Value { get; set; }
public QueryableFilterCompareEnum? Compare { get; set; }
}
My company and I have never attempted to do anything like this before, so it is up to my team to come up with a viable solution. This is the result of working on a solution with a week or so of research.
What Works: Part 1
I have created a service that is able to retrieve the data from our table Classroom. Retrieval of the data is done in Entity Framework Core by way of LINQ-to-SQL. The way I have written below works if one of the fields that are supplied in the filter doesn't exist for Classroom but does exist for its related Organization (the client wanted to be able to search among organization addresses as well) and has a navigatable property.
public async Task<IEnumerable<IExportClassroom>> GetClassroomsAsync(
IEnumerable<QueryableFilter> queryableFilters = null) {
var filters = queryableFilters?.ToList();
IQueryable<ClassroomEntity> classroomQuery = ClassroomEntity.All().AsNoTracking();
// The organization table may have filters searched against it
// If any are, the organization table should be inner joined to all filters are used
IQueryable<OrganizationEntity> organizationQuery = OrganizationEntity.All().AsNoTracking();
var joinOrganizationQuery = false;
// Loop through the supplied queryable filters (if any) to construct a dynamic LINQ-to-SQL queryable
if (filters?.Count > 0) {
foreach (var filter in filters) {
try {
classroomQuery = classroomQuery.BuildExpression(filter.Name, filter.Value, filter.Compare);
} catch (ArgumentException ex) {
if (ex.ParamName == "propertyName") {
organizationQuery = organizationQuery.BuildExpression(filter.Name, filter.Value, filter.Compare);
joinOrganizationQuery = true;
} else {
throw new ArgumentException(ex.Message);
}
}
}
}
// Inner join the classroom and organization queriables (if necessary)
var query = joinOrganizationQuery
? classroomQuery.Join(organizationQuery, classroom => classroom.OrgId, org => org.OrgId, (classroom, org) => classroom)
: classroomQuery;
query = query.OrderBy(x => x.ClassroomId);
IEnumerable<IExportClassroom> results = await query.Select(ClassroomMapper).ToListAsync();
return results;
}
What Works: Part 2
The BuildExpression that exists in code is something that I created as such (with room for expansion).
public static IQueryable<T> BuildExpression<T>(this IQueryable<T> source, string columnName, string value, QueryableFilterCompareEnum? compare = QueryableFilterCompareEnum.Equal) {
var param = Expression.Parameter(typeof(T));
// Get the field/column from the Entity that matches the supplied columnName value
// If the field/column does not exists on the Entity, throw an exception; There is nothing more that can be done
MemberExpression dataField;
try {
dataField = Expression.Property(param, propertyName);
} catch (ArgumentException ex) {
if (ex.ParamName == "propertyName") {
throw new ArgumentException($"Queryable selection does not have a \"{propertyName}\" field.", ex.ParamName);
} else {
throw new ArgumentException(ex.Message);
}
}
ConstantExpression constant = !string.IsNullOrWhiteSpace(value)
? Expression.Constant(value.Trim(), typeof(string))
: Expression.Constant(value, typeof(string));
BinaryExpression binary = GetBinaryExpression(dataField, constant, compare);
Expression<Func<T, bool>> lambda = (Expression<Func<T, bool>>)Expression.Lambda(binary, param)
return source.Where(lambda);
}
private static Expression GetBinaryExpression(MemberExpression member, ConstantExpression constant, QueryableFilterCompareEnum? comparisonOperation) {
switch (comparisonOperation) {
case QueryableFilterCompareEnum.NotEqual:
return Expression.Equal(member, constant);
case QueryableFilterCompareEnum.GreaterThan:
return Expression.GreaterThan(member, constant);
case QueryableFilterCompareEnum.GreaterThanOrEqual:
return Expression.GreaterThanOrEqual(member, constant);
case QueryableFilterCompareEnum.LessThan:
return Expression.LessThan(member, constant);
case QueryableFilterCompareEnum.LessThanOrEqual:
return Expression.LessThanOrEqual(member, constant);
case QueryableFilterCompareEnum.Equal:
default:
return Expression.Equal(member, constant);
}
}
}
The Problem / Getting Around to My Question
While the inner join on the Classroom and Organization works, I'd rather not have to pull in a second entity set for checking values that are navigatable. If I typed in a City as my filter name, normally I would do this:
classroomQuery = classroomQuery.Where(x => x.Organization.City == "Atlanta");
That doesn't really work here.
I have tried a couple of different methods in order to get me what I'm looking for:
A compiled function that would return Func<T, bool>, but when put through LINQ-to-SQL, the query did not include it.
I changed it to an Expression<Func<T, bool>>, but my return didn't return a bool in the way I attempted to implement it, so that didn't work.
I switched the way that I was implementing the navigation property, but none of my functions would read the value properly.
Basically, is there some way that I can implement the following in a way that LINQ-to-SQL from Entity Framework Core will work? Other options are welcome as well.
classroomQuery = classroomQuery.Where(x => x.Organization.BuildExpression(filter.Name, filter.Value, filter.Compare));
Edit 01:
When using the expression without the dynamic builder like so:
IQueryable<ClassroomEntity>classroomQuery = ClassroomEntity.Where(x => x.ClassroomId.HasValue).Where(x => x.Organization.City == "Atlanta").AsNoTracking();
The debug reads:
.Call Microsoft.EntityFrameworkCore.EntityFrameworkQueryableExtensions.AsNoTracking(.Call System.Linq.Queryable.Where(
.Call System.Linq.Queryable.Where(
.Constant<Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[ClassroomEntity]>(Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryable`1[ClassroomEntity]),
'(.Lambda #Lambda1<System.Func`2[ClassroomEntity,System.Boolean]>)),
'(.Lambda #Lambda2<System.Func`2[ClassroomEntity,System.Boolean]>)))
.Lambda #Lambda1<System.Func`2[ClassroomEntity,System.Boolean]>(ClassroomEntity $x)
{
($x.ClassroomId).HasValue
}
.Lambda #Lambda2<System.Func`2[ClassroomEntity,System.Boolean]>(ClassroomEntity $x)
{
($x.Organization).City == "Bronx"
}
I tried with the dynamic builder to get the Classroom teacher, which gave me a debug of:
.Lambda #Lambda3<System.Func`2[ClassroomEntity,System.Boolean]>(ClassroomEntity $var1)
{
$var1.LeadTeacherName == "Sharon Candelariatest"
}
Still cannot figure out how to get ($var1.Organization) as the entity I'm reading from.

If you can ask the client to supply the full dot notation expression for the property. eg "Organization.City";
dataField = (MemberExpression)propertyName.split(".")
.Aggregate(
(Expression)param,
(result,name) => Expression.Property(result, name));

If I am getting your problem statement, you want to be able to travel up the navigation property chain.
If that is indeed the case the real challenge is getting the navigation relationships from EF. And this is where EntityTypeExtensions comes in handy. GetNavigations() in particular.
You could recursively travel up your navigation properties and build property accessor expressions as you go:
private static IEnumerable<Tuple<IProperty, Expression>> GetPropertyAccessors(this IEntityType model, Expression param)
{
var result = new List<Tuple<IProperty, Expression>>();
result.AddRange(model.GetProperties()
.Where(p => !p.IsShadowProperty()) // this is your chance to ensure property is actually declared on the type before you attempt building Expression
.Select(p => new Tuple<IProperty, Expression>(p, Expression.Property(param, p.Name)))); // Tuple is a bit clunky but hopefully conveys the idea
foreach (var nav in model.GetNavigations().Where(p => p is Navigation))
{
var parentAccessor = Expression.Property(param, nav.Name); // define a starting point so following properties would hang off there
result.AddRange(GetPropertyAccessors(nav.ForeignKey.PrincipalEntityType, parentAccessor)); //recursively call ourselves to travel up the navigation hierarchy
}
return result;
}
then your BuildExpression method can probably be a bit simplified. Notice, I added DbContext as parameter:
public static IQueryable<T> BuildExpression<T>(this IQueryable<T> source, DbContext context, string columnName, string value, QueryableFilterCompareEnum? compare = QueryableFilterCompareEnum.Equal)
{
var param = Expression.Parameter(typeof(T));
// Get the field/column from the Entity that matches the supplied columnName value
// If the field/column does not exists on the Entity, throw an exception; There is nothing more that can be done
MemberExpression dataField;
try
{
var model = context.Model.FindEntityType(typeof(T)); // start with our own entity
var props = model.GetPropertyAccessors(param); // get all available field names including navigations
var reference = props.FirstOrDefault(p => RelationalPropertyExtensions.GetColumnName(p.Item1) == columnName); // find the filtered column - you might need to handle cases where column does not exist
dataField = reference.Item2 as MemberExpression; // we happen to already have correct property accessors in our Tuples
}
catch (ArgumentException)
{
throw new NotImplementedException("I think you shouldn't be getting these anymore");
}
ConstantExpression constant = !string.IsNullOrWhiteSpace(value)
? Expression.Constant(value.Trim(), typeof(string))
: Expression.Constant(value, typeof(string));
BinaryExpression binary = GetBinaryExpression(dataField, constant, compare);
Expression<Func<T, bool>> lambda = (Expression<Func<T, bool>>)Expression.Lambda(binary, param);
return source.Where(lambda);
}
and GetClassroomsAsync would look something like this:
public async Task<IEnumerable<IExportClassroom>> GetClassroomsAsync(IEnumerable<QueryableFilter> queryableFilters = null)
{
IQueryable<ClassroomEntity> classroomQuery = ClassroomEntity.All().AsNoTracking();
// Loop through the supplied queryable filters (if any) to construct a dynamic LINQ-to-SQL queryable
foreach (var filter in queryableFilters ?? new List<QueryableFilter>())
{
try
{
classroomQuery = classroomQuery.BuildExpression(_context, filter.Name, filter.Value, filter.Compare);
}
catch (ArgumentException ex)
{
// you probably should look at catching different exceptions now as joining is not required
}
}
query = classroomQuery.OrderBy(x => x.ClassroomId);
IEnumerable<IExportClassroom> results = await query.Select(ClassroomMapper).ToListAsync();
return results;
}
Testing it out
Since you didn't supply entity hierarchy, I experimented on one of my own:
public class Entity
{
public int Id { get; set; }
}
class Company: Entity
{
public string CompanyName { get; set; }
}
class Team: Entity
{
public string TeamName { get; set; }
public Company Company { get; set; }
}
class Employee: Entity
{
public string EmployeeName { get; set; }
public Team Team { get; set; }
}
// then i've got a test harness method as GetClassroomsAsync won't compile wothout your entities
class DynamicFilters<T> where T : Entity
{
private readonly DbContext _context;
public DynamicFilters(DbContext context)
{
_context = context;
}
public IEnumerable<T> Filter(IEnumerable<QueryableFilter> queryableFilters = null)
{
IQueryable<T> mainQuery = _context.Set<T>().AsQueryable().AsNoTracking();
// Loop through the supplied queryable filters (if any) to construct a dynamic LINQ-to-SQL queryable
foreach (var filter in queryableFilters ?? new List<QueryableFilter>())
{
mainQuery = mainQuery.BuildExpression(_context, filter.Name, filter.Value, filter.Compare);
}
mainQuery = mainQuery.OrderBy(x => x.Id);
return mainQuery.ToList();
}
}
// --- DbContext
class MyDbContext : DbContext
{
public DbSet<Company> Companies{ get; set; }
public DbSet<Team> Teams { get; set; }
public DbSet<Employee> Employees { get; set; }
protected override void OnConfiguring(DbContextOptionsBuilder optionsBuilder)
{
optionsBuilder.UseSqlServer("Server=.\\SQLEXPRESS;Database=test;Trusted_Connection=true");
base.OnConfiguring(optionsBuilder);
}
}
// ---
static void Main(string[] args)
{
var context = new MyDbContext();
var someTableData = new DynamicFilters<Employee>(context).Filter(new
List<QueryableFilter> {new QueryableFilter {Name = "CompanyName", Value = "Microsoft" }});
}
With the above, and a filter CompanyName = "Microsoft" EF Core 3.1 generated me the following SQL:
SELECT [e].[Id], [e].[EmployeeName], [e].[TeamId]
FROM [Employees] AS [e]
LEFT JOIN [Teams] AS [t] ON [e].[TeamId] = [t].[Id]
LEFT JOIN [Companies] AS [c] ON [t].[CompanyId] = [c].[Id]
WHERE [c].[CompanyName] = N'Microsoft'
ORDER BY [e].[Id]
This approach seems to produce desired result but has one issue: column names must be unique across all your entities. This likely can be dealt with but since I don't know much specifics of your data model I'd defer it to you.

(Disclaimer: I've written code similar to this, but I haven't actually tested the code in this answer.)
Your BuildExpression takes one query (in the form of an IQueryable<T>) and returns another query. This constrains all your filters to be applied to the property of the parameter -- x.ClassroomId -- when you actually want to apply some of them to a property of a property of the parameter -- x.Organization.City.
I would suggest a GetFilterExpression method, which produces the filter expression off of some arbitrary base expression:
private static Expression GetFilterExpression(Expression baseExpr, string columnName, string value, QueryableFilterCompareEnum? compare = QueryableFilterCompareEnum.Equal) {
MemberExpression dataField;
try {
dataField = Expression.Property(baseExpr, columnName);
} catch (ArgumentException ex) {
if (ex.ParamName == "propertyName") {
throw new ArgumentException($"Base expression type does not have a \"{propertyName}\" field.", ex.ParamName);
} else {
throw new ArgumentException(ex.Message);
}
}
if (!string.IsNullOrWhiteSpace(value)) {
value = value.Trim();
}
ConstantExpression constant = Expression.Constant(value, typeof(string));
BinaryExpression binary = GetBinaryExpression(dataField, constant, compare);
return binary;
}
Within GetClassroomsAsync, you can either build the filter expression against the original ClassroomEntity parameter, or against the returned value of the Organization property on the parameter, by passing in a different expression:
public async Task<IEnumerable<IExportClassroom>> GetClassroomsAsync(IEnumerable<QueryableFilter> queryableFilters = null) {
var filters = queryableFilters?.ToList();
var param = Expression.Parameter(typeof(ClassroomEntity));
var orgExpr = Expression.Property(param, "Organization"); // equivalent of x.Organization
IQueryable<ClassroomEntity> query = ClassroomEntity.All().AsNoTracking();
if (filters is {}) {
// Map the filters to expressions, applied to the `x` or to the `x.Organization` as appropriate
var filterExpressions = filters.Select(filter => {
try {
return GetFilterExpression(param, filter.Name, filter.Value, filter.Compare);
} catch (ArgumentException ex) {
if (ex.ParamName == "propertyName") {
return GetFilterExpression(orgExpr, filter.Name, filter.Value, filter.Compare);
} else {
throw new ArgumentException(ex.Message);
}
}
});
// LogicalCombined is shown later in the answer
query = query.Where(
Expression.Lambda<Func<ClassroomEntity, bool>>(LogicalCombined(filters))
);
}
query = query.OrderBy(x => x.ClassroomId);
IEnumerable<IExportClassroom> results = await query.Select(ClassroomMapper).ToListAsync();
return results;
}
LogicalCombined takes multiple bool-returning expressions and combines them into a single expression:
private static Expression LogicalCombined(IEnumerable<Expression> exprs, ExpressionType expressionType = ExpressionType.AndAlso) {
// ensure the expression type is a boolean operator
switch (expressionType) {
case ExpressionType.And:
case ExpressionType.AndAlso:
case ExpressionType.Or:
case ExpressionType.OrElse:
case ExpressionType.ExclusiveOr:
break;
default:
throw new ArgumentException("Invalid expression type for logically combining expressions.");
}
Expression? final = null;
foreach (var expr in exprs) {
if (final is null) {
final = expr;
continue;
}
final = Expression.MakeBinary(expressionType, final, expr);
}
return final;
}
Some suggestions:
As I've written it, GetFilterExpression is a static method. Since all the arguments (except the base expression) come from QueryableFilter, you might consider making it an instance method off of QueryableFilter.
I would also suggest changing GetBinaryExpression to use a dictionary to map from QueryableFilterCompareEnum to the built-in ExpressionType. Then, the implementation of GetBinaryExpression is just a wrapper for the built-in Expression.MakeBinary method:
private static Dictionary<QueryableFilterCompareEnum, ExpressionType> comparisonMapping = new Dictionary<QueryableFilterCompareEnum, ExpressionType> {
[QueryableFilterCompareEnum.NotEqual] = ExpressionType.NotEqual,
[QueryableFilterCompareEnum.GreaterThan] = ExpressionType.GreaterThan,
[QueryableFilterCompareEnum.GreaterThanOrEqual] = ExpressionType.GreaterThanOrEqual,
[QueryableFilterCompareEnum.LessThan] = ExpressionType.LessThan,
[QueryableFilterCompareEnum.LessThanOrEqual] = ExpressionType.LessThanOrEqual,
[QueryableFilterCompareEnum.Equal] = ExpressionType.Equal
}
private static Expression GetBinaryExpression(MemberExpression member, ConstantExpression constant, QueryableFilterCompareEnum? comparisonOperation) {
comparisonOperation = comparisonOperation ?? QueryableFilterCompareEnum.Equal;
var expressionType = comparisonMapping[comparisonOperation];
return Expression.MakeBinary(
expressionType,
member,
constant
);
}
Both GetFilterExpression and GetClassroomsAsync handle the possibility that the specified property doesn't exist on either ClassroomEntity or OrganizationEntity, by trying to construct the member-access expression and handling the thrown exception.
It might be clearer to use reflection to test if the property exists on either type or not.
More, you might consider storing a static HashSet<string> with all the valid fieldnames, and check against that.

every Parameter object property that is not null, to be added to expression predicate as a condition

I am looking for a way to dynamically build an expression, based upon the method parameters.
This is the code snippet from my service method, where I would like to build a predicate expression.
public async Task<Account> GetCustomerAccountsAsync(Parameters params)
{
var items = await _unitOfWork.Accounts.GetWhereAsync(a => a.CustomerId == params.CustomerId && ... );
...
}
GetWhereAsync is a method from the generic repository that looks like:
public async Task<IEnumerable<TEntity>> GetWhereAsync(Expression<Func<TEntity, bool>> predicate)
{
return await Context.Set<TEntity>().Where(predicate).ToListAsync();
}
And Parameters class:
public class Parameters
{
public string CustomerId { get; set; }
public string AccountId { get; set; }
public string ProductId { get; set; }
public string CurrencyId { get; set; }
}
What I would like to implement, is that every Parameter object property that is not null,
to be added to expression predicate as a condition.
For example, if CustomerId and AccountId have some values, I would like to dynamically build predicate expression
that would have functionality same as the following predicate:
var items = await _unitOfWork.Accounts.GetWhereAsync(a => a.CustomerId == params.CustomerId && a.AccountId == params.AccountId);
Appreciate any help.

You don't need to use Expressions to build something dynamically here. You can do something like this:
_unitOfWork.Accounts.Where(a =>
(params.CustomerId == null || a.CustomerId == params.CustomerId) &&
(params.AccountId == null || a.AccountId == params.AccountId) &&
(params.ProductId == null || a.ProductId == params.ProductId) &&
(params.CurrencyId == null || a.CurrencyId == params.CurrencyId)
);
This is how I've done queries before for a search form with multiple optional search parameters.

I'm currently working a lot with Expressions so I think I can help you.
I just built a custom code for you.
The code accepts that you add Properties to your filtered class (Account) without having to change the filter building code.
The code filters string Properties and use it to create the Predicate.
public Func<Account, bool> GetPredicate(Parameters parameters)
{
var stringProperties = typeof(Parameters)
.GetProperties(BindingFlags.Public | BindingFlags.Instance)
.Where(x => x.PropertyType == typeof(string));
var parameterExpression = Expression.Parameter(typeof(Account));
var notNullPropertyNameToValue = new Dictionary<string, string>();
BinaryExpression conditionExpression = null;
foreach (var stringProperty in stringProperties)
{
var propertyValue = (string)stringProperty.GetValue(parameters);
if (propertyValue != null)
{
var propertyAccessExpression = Expression.PropertyOrField(parameterExpression, stringProperty.Name);
var propertyValueExpression = Expression.Constant(propertyValue, typeof(string));
var propertyTestExpression = Expression.Equal(propertyAccessExpression, propertyValueExpression);
if (conditionExpression == null)
{
conditionExpression = propertyTestExpression;
}
else
{
conditionExpression = Expression.And(conditionExpression, propertyTestExpression);
}
}
}
//Just return a function that includes all members if no parameter is defined.
if (conditionExpression == null)
{
return (x) => true;
}
var lambdaExpression = Expression.Lambda<Func<Account, bool>>(conditionExpression, parameterExpression);
return lambdaExpression.Compile();
}
It returns a typed predicate that you can use in Linq for example.
See this example :
void Main()
{
var customers = new List<Account>()
{
new Account()
{
CustomerId = "1",
},
new Account()
{
CustomerId = "2",
}
};
var predicate = GetPredicate(new Parameters() { CustomerId = "1" });
customers.Where(predicate);
}
If any help is needed, feel free to ask !

Filter IQueryable which contains (%like%) any of the items in the List<string>

I want to filter an IQueryable/List using List without hitting the database. The IQueryable result should include all result containing any of the strings in the List and the length of the list is unspecified.
myQueryable = myQueryable.Where(filelist => filelist.Location.Contains(filterList[0]) || filelist.Location.Contains(filterList[1]) || filelist.Location.Contains(filterList[N])...);
I'm using ASP.NET Core 3 and will use the IQueryable to hit the databse in a later phase using Entity Framework.
I've tried these two codes which did not work and my IQuery works fine if i exclude my trial codes (for filtering the attribute, location).
workOrders = workOrders.Where(filelist => filterList.Contains(filelist.Location)); //Returns only exact match
workOrders = workOrders.Where(filelist => filterList.Any(filter => filelist.Location.ToUpperInvariant().Contains(filter.ToUpperInvariant()))); //Returns error
Expression<Func<Workorder, bool>> predicate = filelist => false;
foreach (var filter in filterList)
{
Expression<Func<Workorder, bool>> orPredicate = filelist =>
filter.Contains(filelist.Location);
var body = Expression.Or(predicate.Body, orPredicate.Body);
predicate = Expression.Lambda<Func<Workorder, bool>>(body,
predicate.Parameters[0]);
}
workOrders = workOrders.Where(predicate); //Returns Error
class Workorder //Database Model
{
public string SiteId { get; set; }
public string Location { get; set; }
}
List<string> filterList //List to be used as filter
{
"filterOne",
"filterTwo",
"filterN"
};
The second code i ran gives me an error which i can get nothing out of.
System.InvalidOperationException: When called from 'VisitLambda', rewriting a node of type 'System.Linq.Expressions.ParameterExpression' must return a non-null value of the same type. Alternatively, override 'VisitLambda' and change it to not visit children of this type.
Third code i ran gives me this error,
System.InvalidOperationException: Operation is not valid due to the current state of the object.
at Microsoft.EntityFrameworkCore.Relational.Query.Pipeline.RelationalQueryableMethodTranslatingExpressionVisitor.TranslateWhere(ShapedQueryExpression source, LambdaExpression predicate)

I have definitely not tested this... but I don't think you can do it in a one-liner if you pretend EF to parse it afterwards.
However, something like this should be fine:
Expression<Func<WorkOrder, bool>> predicate = filelist => false;
foreach(var filter in filterList) {
Expression<Func<WorkOrder, bool>> orPredicate = filelist => filter.Contains(filelist.Location);
var body = Expression.Or(predicate.Body, orPredicate.Body);
predicate = Expression.Lambda<Func<WorkOrder,bool>>(body, predicate.Parameters[0]);
}
And then:
myQueryable = myQueryable.Where(predicate);
We are basically just telling it:
.Where(filelist => false || filterList[0].Contains(filelist.Location) || filterList[1].Contains(filelist.Location) || ...);
Edit
Still untested on EF, but at least the syntax seems to be ok: https://dotnetfiddle.net/Htr7Zr

It took me a while but i used your example to construct an expression tree that worked.
List<filterList> filterList //List to be used as filter
{
"filterOne",
"filterTwo",
"filterN"
}
class Workorder //Database Model
{
public string SiteId { get; set; }
public string Location { get; set; }
}
static Expression<Func<T, bool>> GetExpression<T>(string propertyName, List<string> propertyValue)
{
Expression orExpression = null;
var parameterExp = Expression.Parameter(typeof(T), "type");
var propertyExp = Expression.Property(parameterExp, propertyName);
foreach (string searchTerm in propertyValue)
{
MethodInfo method = typeof(string).GetMethod("Contains", new[] { typeof(string) });
var someValue = Expression.Constant(searchTerm, typeof(string));
var containsMethodExp = Expression.Call(propertyExp, method, someValue);
if(orExpression == null) //to handle intial phase when left expression = null
{
orExpression = Expression.Call(propertyExp, method, someValue);
}else
{
orExpression = Expression.Or(orExpression, containsMethodExp);
}
return Expression.Lambda<Func<T, bool>>(orExpression, parameterExp);
}
myQueryable = myQueryable.Where(GetExpression<Workorder>(property, filterList));
This Equals:
myQueryable = myQueryable.Where(filelist => filelist.Location.Contains(filterList[0]) || filelist.Location.Contains(filterList[1]) || filelist.Location.Contains(filterList[N])...);

Linq handling variable number of OrderBy

I need to support a variable number of Orderby terms in a Linq (to Entity) statement. That is, my function will accept a list of properties on which the data should be order. The properties can have both ascending or descending sorts. What is the best way to handle constructing the Linq query?
Thanks!

You should be able to do something along these lines:
public IEnumerable<MyType> DoSomething(params Expression<Func<MyType,object>>[] properties)
{
var query = // create LINQ query that returns IQueryable<MyType>
query = query.OrderBy(properties.First());
foreach (var property in properties.Skip(1))
{
query = query.ThenBy(property);
}
}
…
var results = DoSomething(() => x.Age, () => x.Height, () => x.LastName);
You'd need to handle the case where fewer than 2 properties are specified.

Following on from Jay's answer, this can be made into a nice extension method:
public static class EnumerableExtensions
{
public static IEnumerable<T> OrderByMany<T>(this IEnumerable<T> enumerable,
params Expression<Func<T, object>>[] expressions)
{
if (expressions.Length == 1)
return enumerable.OrderBy(expressions[0].Compile());
var query = enumerable.OrderBy(expressions[0].Compile());
for (int i = 1; i < expressions.Length;i++)
{
query = query.ThenBy(expressions[i].Compile());
}
return query;
}
}
Usage becomes quite simple, given a test object:
public class Person
{
public string Name { get; set; }
public int Age { get; set; }
}
This is then possible:
var people = new Person[]
{
new Person() {Name = "John", Age = 40},
new Person() {Name = "John", Age = 20},
new Person() {Name = "Agnes", Age = 11}
};
foreach(var per in people.OrderByMany(x => x.Name, x => x.Age))
{
Console.WriteLine("{0} Age={1}",per.Name,per.Age);
}
Output:
Agnes Age=11
John Age=20
John Age=40
UPDATE
You could add another overload of the OrderByMany method to support SortOrder as well, although it gets clunky rather quickly. Personally I'd just go for the syntax
var query = from p
in people
order by Name, Age descending;
However, for the record, in C#4 at least, I would accomplish the overload using an enum & tuple.
public enum SortOrder
{
Ascending,
Descending
}
and the extra overload:
public static IEnumerable<T> OrderByMany<T>(this IEnumerable<T> enumerable,
params Tuple<Expression<Func<T, object>>,SortOrder>[] expressions)
{
var query = (expressions[0].Item2 == SortOrder.Ascending)
? enumerable.OrderBy(expressions[0].Item1.Compile())
: enumerable.OrderByDescending(expressions[0].Item1.Compile());
for (int i = 1; i < expressions.Length; i++)
{
query = expressions[i].Item2 == SortOrder.Ascending
? query.ThenBy(expressions[i].Item1.Compile())
: query.ThenByDescending(expressions[i].Item1.Compile());
}
return query;
}
Usage becomes clumsy and hard to read:
foreach (var per in people.OrderByMany(
new Tuple<Expression<Func<Person, object>>, SortOrder>(x => x.Age, SortOrder.Descending),
new Tuple<Expression<Func<Person, object>>, SortOrder>(x => x.Name, SortOrder.Ascending)))
{
Console.WriteLine("{0} Age={1}", per.Name, per.Age);
}

To sort by an arbitrary property, you need to build an expression tree to pass to OrderBy.
To sort by an arbitrary number of properties, you need to call ThenBy in a loop.

I like Jamiec's idea but I hate using Tuples because the syntax is ugly. Therefore I built a small class that encapsulates the Tuple and exposes getters for the Item1 and Item2 properties with better variable names.
Also notice that I used a default sort order of ascending so you only need to specify a SortOrder if you want to sort in descending order.
public class SortExpression<T>
{
private Tuple<Expression<Func<T, object>>, SortOrder> tuple;
public SortExpression( Expression<Func<T, object>> expression, SortOrder order =SortOrder.Ascending )
{
tuple = new Tuple<Expression<Func<T,object>>, SortOrder>(expression, order);
}
public Expression<Func<T, object>> Expression {
get { return tuple.Item1; }
}
public SortOrder Order {
get { return tuple.Item2; }
}
}
In my specific application, I have a repository base class which takes an IQueryable and converts it to a ObservableCollection. In that method I use the SortExpression class:
public ObservableCollection<T> GetCollection(params SortExpression<T>[] sortExpressions) {
var list = new ObservableCollection<T>();
var query = FindAll();
if (!sortExpressions.Any()) {
query.ToList().ForEach(list.Add);
return list;
}
var ordered = (sortExpressions[0].Order == SortOrder.Ascending)
? query.OrderBy(sortExpressions[0].Expression.Compile())
: query.OrderByDescending(sortExpressions[0].Expression.Compile());
for (var i = 1; i < sortExpressions.Length; i++) {
ordered = sortExpressions[i].Order == SortOrder.Ascending
? ordered.ThenBy(sortExpressions[i].Expression.Compile())
: ordered.ThenByDescending(sortExpressions[i].Expression.Compile());
}
ordered.ToList().ForEach(list.Add);
return list;
}
Here is the method in use:
var repository = new ContactRepository(UnitOfWork);
return repository.GetCollection(
new SortExpression<Contact>(x => x.FirstName),
new SortExpression<Contact>(x => x.LastName));

Append to an expression

I followed this thread: link text
Jason gives an example:
public static Expression<TDelegate> AndAlso<TDelegate>(this Expression<TDelegate> left, Expression<TDelegate> right)
{
return Expression.Lambda<TDelegate>(Expression.AndAlso(left, right), left.Parameters);
}
and its usage as such:
Expression<Func<Client, bool>> clientWhere = c => true;
if (filterByClientFName)
{
clientWhere = clientWhere.AndAlso(c => c.ClientFName == searchForClientFName);
}
if (filterByClientLName)
{
clientWhere = clientWhere.AndAlso(c => c.ClientLName == searchForClientLName);
}
I have a orders table and i followed the above example, changing column names, and i get the similar error that the post creator had
The binary operator AndAlso is not defined for the types 'System.Func2[Models.Order,System.Boolean]' and 'System.Func2[Models.Order,System.Boolean]'.
Anyone have any thoughts on what I am missing?
UPDATED:
Eric, I further followed what the user of the previous post was asking, here link text
The user has this
Expression<Func<Client, bool>> clientWhere = c => true;
Expression<Func<Order, bool>> orderWhere = o => true;
Expression<Func<Product, bool>> productWhere = p => true;
if (filterByClient)
{
clientWhere = c => c.ClientID == searchForClientID;
}
Now if he were to have various conditions in filterByClient, say he either has clientid and/or some other column name, how would one build the clientWhere expression?

You're attempting to build an expression tree that represents this:
c => true && c.ClientFName == searchForClientFName
You are actually building an expression tree that represents this:
c => c=> true && c => c.ClientFName == searchForClientFName
which makes no sense at all.
Now, you might naively think that this will work:
public static Expression<TDelegate> AndAlso<TDelegate>(this Expression<TDelegate> left, Expression<TDelegate> right)
{
// NOTICE: Combining BODIES:
return Expression.Lambda<TDelegate>(Expression.AndAlso(left.Body, right.Body), left.Parameters);
}
That would produce in your case something representing
c => true && c.ClientFName == searchForClientFName
Which looks right. But in fact this is fragile. Suppose you had
... d => d.City == "London" ...
... c => c.ClientName == "Fred Smith" ...
and you used this method to combine them. You'd get an object representing
c => d.City == "London" && c.ClientName == "Fred Smith"
What the heck is d doing in there?
Furthermore, parameters are matched by object identity, not by parameter name. If you do this
... c => c.City == "London" ...
... c => c.ClientName == "Fred Smith" ...
and combine them into
c => c.City == "London" && c.ClientName == "Fred Smith"
you're in the same boat; the "c" in "c.City" is a different c than the other two.
What you actually need to do is make a third parameter object, substitute it in the bodies of both lambdas for every occurence of their parameters, and then build up a new lambda expression tree from the resulting substituted bodies.
You can build a substitution engine by writing a visitor that passes over the expression tree body, rewriting it as it goes.

It was difficult for me to understand hvd's answer so I created some code to explain it in a different way. hvd should get the credit for suggesting the ExpressionVisitor. I just couldn't understand the example in the context of Linq to X type input functions I was using.
I hope this helps somebody else coming to the question from that perspective.
Also, I created the combining code as extension methods to make it a little easier to use.
using System;
using System.Collections.Generic;
using System.Linq.Expressions;
namespace ConsoleApplication3
{
class Program
{
static void Main(string[] args)
{
var combined = TryCombiningExpressions(c => c.FirstName == "Dog", c => c.LastName == "Boy");
Console.WriteLine("Dog Boy should be true: {0}", combined(new FullName { FirstName = "Dog", LastName = "Boy" }));
Console.WriteLine("Cat Boy should be false: {0}", combined(new FullName { FirstName = "Cat", LastName = "Boy" }));
Console.ReadLine();
}
public class FullName
{
public string FirstName { get; set; }
public string LastName { get; set; }
}
public static Func<FullName, bool> TryCombiningExpressions(Expression<Func<FullName, bool>> func1, Expression<Func<FullName, bool>> func2)
{
return func1.CombineWithAndAlso(func2).Compile();
}
}
public static class CombineExpressions
{
public static Expression<Func<TInput, bool>> CombineWithAndAlso<TInput>(this Expression<Func<TInput, bool>> func1, Expression<Func<TInput, bool>> func2)
{
return Expression.Lambda<Func<TInput, bool>>(
Expression.AndAlso(
func1.Body, new ExpressionParameterReplacer(func2.Parameters, func1.Parameters).Visit(func2.Body)),
func1.Parameters);
}
public static Expression<Func<TInput, bool>> CombineWithOrElse<TInput>(this Expression<Func<TInput, bool>> func1, Expression<Func<TInput, bool>> func2)
{
return Expression.Lambda<Func<TInput, bool>>(
Expression.AndAlso(
func1.Body, new ExpressionParameterReplacer(func2.Parameters, func1.Parameters).Visit(func2.Body)),
func1.Parameters);
}
private class ExpressionParameterReplacer : ExpressionVisitor
{
public ExpressionParameterReplacer(IList<ParameterExpression> fromParameters, IList<ParameterExpression> toParameters)
{
ParameterReplacements = new Dictionary<ParameterExpression, ParameterExpression>();
for (int i = 0; i != fromParameters.Count && i != toParameters.Count; i++)
ParameterReplacements.Add(fromParameters[i], toParameters[i]);
}
private IDictionary<ParameterExpression, ParameterExpression> ParameterReplacements { get; set; }
protected override Expression VisitParameter(ParameterExpression node)
{
ParameterExpression replacement;
if (ParameterReplacements.TryGetValue(node, out replacement))
node = replacement;
return base.VisitParameter(node);
}
}
}
}

If you need it i created a small fluent library to create lambda functions on the fly without directly coping with System.Linq.Expressions. And it can easily handle the kind of situation. Just to give an example:
static void Main(string[] args)
{
var firstNameCompare = ExpressionUtil.GetComparer<FullName>((a) => a.FirstName);
var lastNameCompare = ExpressionUtil.GetComparer<FullName>((a) => a.LastName);
Func<FullName, bool> combined = (a) => firstNameCompare(a, "Dog") && lastNameCompare(a, "Boy");
var toCheck = new FullName {FirstName = "Dog", LastName = "Boy"};
Console.WriteLine("Dog Boy should be true: {0}", combined(toCheck));
toCheck = new FullName {FirstName = "Cat", LastName = "Boy"};
Console.WriteLine("Cat Boy should be false: {0}", combined(toCheck));
Console.ReadLine();
}
The GetComparer methods seek for the property passed as expression and find ho to get its value, then it builds a new Expression that will handle the comparaison.
At the end the two functions are evaluated calling the "combined" function.
If you need more verifications you could use an array and iterate on it inside the "combined lambda"
The code and documentation for the library are here: Kendar Expression Builder
While the nuget package is here: Nuget Expression Builder

I tried to implement this kind of stuff. Took me a day to find out.
My solution is based on filter in a loop based on a Array of predicate.
As a note, it s totally Generic and based Reflection because the only information about class and field are String.
To make it simple, i call directly the Model class but in a project you should go by a controler who is calling the Model.
So here we go :
The Model part where T is a Generic in the class
public class DALXmlRepository<T> where T : class
{
public T GetItem(Array predicate)
{
IQueryable<T> QueryList = null;
QueryList = ObjectList.AsQueryable<T>().Where((Expression<Func<T, bool>>)predicate.GetValue(0));
for (int i = 1; i < predicate.GetLength(0); i++)
{
QueryList = QueryList.Where((Expression<Func<T, bool>>)predicate.GetValue(i));
}
if (QueryList.FirstOrDefault() == null)
throw new InvalidOperationException(this.GetType().GetGenericArguments().First().Name + " not found.");
return QueryList.FirstOrDefault();
}
}
Now the LambdaExpression Builder, it's a base one(with String type or something else) , you can improve it with more functionnality :
private static Expression BuildLambdaExpression(Type GenericArgument, string FieldName, string FieldValue)
{
LambdaExpression lambda = null;
Expression Criteria = null;
Random r = new Random();
ParameterExpression predParam = Expression.Parameter(GenericArgument, r.Next().ToString());
if (GenericArgument.GetProperty(FieldName).PropertyType == typeof(string))
{
Expression left = Expression.PropertyOrField(predParam, FieldName);
Expression LefttoUpper = Expression.Call(left, "ToUpper", null, null);
//Type du champ recherché
Type propType = GenericArgument.GetProperty(FieldName).PropertyType;
Expression right = Expression.Constant(FieldValue, propType);
Expression RighttoUpper = Expression.Call(right, "ToUpper", null, null);
Criteria = Expression.Equal(LefttoUpper, RighttoUpper);
}
else
{
Expression left = Expression.PropertyOrField(predParam, FieldName);
Type propType = GenericArgument.GetProperty(FieldName).PropertyType;
Expression right = Expression.Constant(Convert.ChangeType(FieldValue, propType), propType);
Criteria = Expression.Equal(left, right);
}
lambda = Expression.Lambda(Criteria, predParam);
return lambda;
}
Now the Calling function :
public static Hashtable GetItemWithFilter(string Entity, XMLContext contextXML, Hashtable FieldsNameToGet, Hashtable FieldFilter)
{
//Get the type
Type type = Type.GetType("JP.Model.BO." + Entity + ", JPModel");
Type CtrlCommonType = typeof(CtrlCommon<>).MakeGenericType( type );
//Making an instance DALXmlRepository<xxx> XMLInstance = new DALXmlRepository<xxx>(contextXML);
ConstructorInfo ci = CtrlCommonType.GetConstructor(new Type[] { typeof(XMLContext), typeof(String) });
IControleur DalInstance = (IControleur)ci.Invoke(new object[] { contextXML, null });
//Building the string type Expression<func<T,bool>> to init the array
Type FuncType = typeof(Func<,>).MakeGenericType( type ,typeof(bool));
Type ExpressType = typeof(Expression<>).MakeGenericType(FuncType);
Array lambda = Array.CreateInstance(ExpressType,FieldFilter.Count);
MethodInfo method = DalInstance.GetType().GetMethod("GetItem", new Type[] { lambda.GetType() });
if (method == null)
throw new InvalidOperationException("GetItem(Array) doesn't exist for " + DalInstance.GetType().GetGenericArguments().First().Name);
int j = 0;
IDictionaryEnumerator criterias = FieldFilter.GetEnumerator();
criterias.Reset();
while (criterias.MoveNext())
{
if (!String.IsNullOrEmpty(criterias.Key.ToString()))
{
lambda.SetValue(BuildLambdaExpression(type, criterias.Key.ToString(), criterias.Value.ToString()),j);
}
else
{
throw new JPException(JPException.MessageKey.CONTROLER_PARAMFIELD_EMPTY, "GetItemWithFilter", criterias.Key.ToString());
}
j++;
}
Object item = method.Invoke(DalInstance, new object[] { lambda });
}
The argument are :
String Entity : Entity class name.
XMLContext : it s the unit of work of the repository, argument i use to initialize the Model class
Hashtable FieldsNameToGet : Index/value of the list of the field i want to get back
Hashtable FieldFilter : the key/Value with FieldName/Content used to make the Lambda expression
Good Luck.