We Keep Coding

Search a term in multiple columns

I'd like to create an extension to search terms in multiple columns.
Terms are separated with space, each term must appears to at least one given column.
Here what I've done so far:
public static IQueryable<TSource> SearchIn<TSource>(this IQueryable<TSource> query,
string searchText,
Expression<Func<TSource, string>> expression,
params Expression<Func<TSource, string>>[] expressions)
{
if (string.IsNullOrWhiteSpace(searchText))
{
return query;
}
// Concat expressions
expressions = new[] { expression }.Concat(expressions).ToArray();
// Format search text
var formattedSearchText = searchText.FormatForSearch();
var searchParts = formattedSearchText.Replace('\'', ' ').Split(' ');
// Initialize expression
var pe = Expression.Parameter(typeof(TSource), "entity");
var predicateBody = default(Expression);
// Search in each expressions, put OR in between
foreach (var expr in expressions)
{
var exprBody = default(Expression);
// Search for each words, put AND in between
foreach (var searchPart in searchParts)
{
// Create property or field expression
var left = Expression.PropertyOrField(pe, ((MemberExpression)expr.Body).Member.Name);
// Create the constant expression with current word
var search = Expression.Constant(searchPart, typeof(string));
// Create the contains function
var contains = Expression.Call(left, typeof(string).GetMethod(nameof(string.Contains), new Type[] { typeof(string) }), search);
// Check if there already a predicate body
if (exprBody == null)
{
exprBody = contains;
}
else
{
exprBody = Expression.And(exprBody, contains);
}
}
if (predicateBody == null)
{
predicateBody = exprBody;
}
else
{
predicateBody = Expression.OrElse(predicateBody, exprBody);
}
}
// Build the where method expression
var whereCallExpression = Expression.Call(
typeof(Queryable),
nameof(Queryable.Where),
new Type[] { query.ElementType },
query.Expression,
Expression.Lambda<Func<TSource, bool>>(predicateBody, new ParameterExpression[] { pe }));
// Apply the condition to the query and return it
return query.Provider.CreateQuery<TSource>(whereCallExpression);
}
It works well as long as given expressions are simple:
// It works well
query.SearchIn("foo", x => x.Column1, x => x.Column2);
But it does not work when trying to navigate through navigation properties:
// Not working
query.SearchIn("foo", x => x.Nav1.Column1);
It gives me an exception.
'Column1' is not a member of type 'Nav1'.
I understand the problem but I can't find the solution to pass through Nav1.
I need help with this one.

Instead of parsing lambda expression body just call it with given parameter:
var left = Expression.Invoke(expr, pe);
However it works only in EF Core.
In EF6 you would need to get property or field of each nested member like this:
var left = expr.Body.ToString()
.Split('.')
.Skip(1) //skip the original parameter name
.Aggregate((Expression)pe, (a, c) => Expression.PropertyOrField(a, c));
It will work only for simple lambdas like:
x => x.Prop1.Nav1
If that's not enough you would need some more advanced parsing algorithm with ExpressionVisitor for example.

Filter List based on property names decided at runtime

I'm wondering if is it possible to make the "property name" dynamic in the filter expressions
Consider scenario
List<Person> GetPerson(int countryID, int stateID, int cityID, int zip)
{
//List of person can be filtered based on below line of code
List<Person> filteredPersons= persons.FindAll(rule => rule.CountryID == countryID).ToList();
//is it possible to specify ".Country" dynamically. something like
List<Person> filteredPersons= persons.FindAll(rule => rule."propertyName"== countryID).ToList();
}

Considering your example one method you could employ is using the .Where() extension instead of FindAll() this could then allow you to build your expression manually. A quick example would be as below.
static List<Person> GetPerson(int countryID, int stateID, int cityID, int zip)
{
//create a new expression for the type of person this.
var paramExpr = Expression.Parameter(typeof(Person));
//next we create a property expression based on the property named "CountryID" (this is case sensitive)
var property = Expression.Property(paramExpr, "CountryID");
//next we create a constant express based on the country id passed in.
var constant = Expression.Constant(countryID);
//next we create an "Equals" express where property equals containt. ie. ".CountryId" = 1
var idEqualsExpr = Expression.Equal(property, constant);
//next we convert the expression into a lamba expression
var lExpr = Expression.Lambda<Func<Person, bool>>(idEqualsExpr, paramExpr);
//finally we query our dataset
return persons.AsQueryable().Where(lExpr).ToList();
}
So this looks like alot of code but what we have basically done is manually constructed the expression tree with the end result looking similar to (and functioning as)
return persons.AsQueryable().Where(p => p.CountryId = countryId);
Now we can take this forward lets say you wanted to query for multiple properties using an and\or based on the method call. Ie you could change all your "filter" paramters to be Nullable and check if a value is passed in we filter for it such as.
static List<Person> GetPerson(int? countryID = null, int? stateID = null, int? cityID = null, int? zip = null)
{
//create a new expression for the type of person this.
var paramExpr = Expression.Parameter(typeof(Person));
//var equalExpression = Expression.Empty();
BinaryExpression equalExpression = null;
if (countryID.HasValue)
{
var e = BuildExpression(paramExpr, "CountryId", countryID.Value);
if (equalExpression == null)
equalExpression = e;
else
equalExpression = Expression.And(equalExpression, e);
}
if (stateID.HasValue)
{
var e = BuildExpression(paramExpr, "StateID", stateID.Value);
if (equalExpression == null)
equalExpression = e;
else
equalExpression = Expression.And(equalExpression, e);
}
if (equalExpression == null)
{
return new List<Person>();
}
//next we convert the expression into a lamba expression
var lExpr = Expression.Lambda<Func<Person, bool>>(equalExpression, paramExpr);
//finally we query our dataset
return persons.AsQueryable().Where(lExpr).ToList();
}
static BinaryExpression BuildExpression(Expression expression, string propertyName, object value)
{
//next we create a property expression based on the property named "CountryID" (this is case sensitive)
var property = Expression.Property(expression, propertyName);
//next we create a constant express based on the country id passed in.
var constant = Expression.Constant(value);
//next we create an "Equals" express where property equals containt. ie. ".CountryId" = 1
return Expression.Equal(property, constant);
}
Now this is a bit more code but as you can see we are now accepting null values for all our parameters and building our query for additional properties.
Now you could take this further (as assuming a more generic method is required) of passing in a Dictionary<string, object> of the properties \ values to query. This can be done as an extension method on an IEnumerable<T> as listed below.
public static class LinqExtensions
{
public static IEnumerable<T> CustomParameterQuery<T>(this IEnumerable<T> entities, Dictionary<string, object> queryVars)
{
if (entities.Count() == 0 || queryVars.Count == 0)
{
return entities;
}
//create a new expression for the type of person this.
var paramExpr = Expression.Parameter(typeof(T));
BinaryExpression equalExpression = null;
foreach (var kvp in queryVars)
{
var e = BuildExpression(paramExpr, kvp.Key, kvp.Value);
if (equalExpression == null)
equalExpression = e;
else
equalExpression = Expression.And(equalExpression, e);
}
if (equalExpression == null)
{
return new T[0];
}
//next we convert the expression into a lamba expression
var lExpr = Expression.Lambda<Func<T, bool>>(equalExpression, paramExpr);
//finally we query our dataset
return entities.AsQueryable().Where(lExpr);
}
static BinaryExpression BuildExpression(Expression expression, string propertyName, object value)
{
//next we create a property expression based on the property name
var property = Expression.Property(expression, propertyName);
//next we create a constant express based on the country id passed in.
var constant = Expression.Constant(value);
//next we create an "Equals" express where property equals containt. ie. ".CountryId" = 1
return Expression.Equal(property, constant);
}
}
Now this could be easily called as:
var dict = new Dictionary<string, object>
{
{ "CountryID", 1 },
{ "StateID", 2 }
};
var e = persons.CustomParameterQuery(dict);
Now obiously this is not a perfect example, but should get you moving in the right direction. Now you "could" also support "OR" statements etc by using Expression.Or instead of Expression.And when combining expressions.
I Must add this is very error prone as it requires the Property names to be exact to the entity, you could use reflection on T and determine if the PropertyName exists and If it is in the correct casing.

Search for System.Linq.Dynamic in NuGet. That's the easiest way to start with Dynamic Linq.

Building a Linq to EF query to a variable list of keywords in a variable number of columns?

I am trying to come up with a utility method to build a Linq Query or Linq Predicate to add to an Linq to EF query to do search for all terms in a list of terms in a variable number of columns.
I am trying to use PredicateBuilder to build the where clause. With one search term and a fixed list of columns it is relatively easy.
The pseudo code that I am trying to work up looks like this so far:
private static Predicate<Project> CreateDynamicSearch(IEnumerable<strings> searchableColumns, string[] searchTerms)
{
var predicate = PredicateBuilder.True<Project>();
foreach (var columnName in searchableColumns)
{
foreach (var term in searchTerms)
{
predicate = predicate.And(a => a.**columnName**.Contains(term));
}
predicate = predicate.Or(predicate);
}
return predicate;
}
The biggest issue I have is handling the expression for the columnName. Previous advice was to use an expression tree but I do not understand how that works into this scenario.
** Update **
I've taken the code as you have it after the update. It builds but when I actually make the call it errors on the Extension.Property(param,columnName); line, with the error Instance property 'Name' is not defined for type 'System.Func`2[Myclass,System.Boolean]' message. The columnName = "Name"
** Update 2 **
The way it's called:
var test = CreateDynamicSearch<Func<Project, bool>>(searchCols, searchTerms);

You can build expression for predicate yourself, in this case it's relatively easy:
private static Expression<Func<T, bool>> CreateDynamicSearch<T>(IEnumerable<string> searchableColumns, string[] searchTerms) {
// start with true, since we combine with AND
// and true AND anything is the same as just anything
var predicate = PredicateBuilder.True<T>();
foreach (var columnName in searchableColumns) {
// start with false, because we combine with OR
// and false OR anything is the same as just anything
var columnFilter = PredicateBuilder.False<T>();
foreach (var term in searchTerms) {
// a =>
var param = Expression.Parameter(typeof(T), "a");
// a => a.ColumnName
var prop = Expression.Property(param, columnName);
// a => a.ColumnName.Contains(term)
var call = Expression.Call(prop, "Contains", new Type[0], Expression.Constant(term));
columnFilter = columnFilter.Or(Expression.Lambda<Func<T, bool>>(call, param));
}
predicate = predicate.And(columnFilter);
}
return predicate;
}
In response to comment
I was just curious if there was some way you could combine the
expression created by Expression.Property(param, columnName) with the
one the compiler generates for (string s) -> s.Contains(term)
You can do that with (for example) like this:
// a =>
var param = Expression.Parameter(typeof(T), "a");
// a => a.ColumnName
var prop = Expression.Property(param, columnName);
// s => s.Contains(term)
Expression<Func<string, bool>> contains = (string s) => s.Contains(term);
// extract body - s.Contains(term)
var containsBody = (MethodCallExpression)contains.Body;
// replace "s" parameter with our property - a.ColumnName.Contains(term)
// Update accepts new target as first parameter (old target in this case is
// "s" parameter and new target is "a.ColumnName")
// and list of arguments (in this case it's "term" - we don't need to update that).
//
var call = containsBody.Update(prop, containsBody.Arguments);
columnFilter = columnFilter.Or(Expression.Lambda<Func<T, bool>>(call, param));

How to Create an Expression tree for .Where(x => x.<deep property>.Select(y => y.id).Intersect(List<int>).Any())

I'm creating a method that receives a Queryable<T> source, a string with a property name/path (could be a deep property for example "TrParent.DataTypes" to achieve this x => x.TrParent.DataTypes) and Enumerable<int> which holds the values I need to intersect.
Basically I come from the need to create the following query dynamically (I mean <DT_Det_Tr> and TrParent.DataTypes being know only at runtime, in the example DT_Det_Tr is not a type it is a class):
var _vals = new List<int>();
var res = dbContext.Set<DT_Det_Tr>()
.Where
(x => x.TrParent.DataTypes
.Select(t => t.Id)
.Intersect(_vals)
.Any()
);
Please keep in mind that the preceding query is just an example of what I need to achieve dynamically, what I really need is an expression tree that creates a predicate like the one shown above but using a dynamic type and with the deep navigation property specified within a string.
So, I'm using this function to create the expression for the deep property:
private static LambdaExpression CreateDelegateExpression<T>(out Type resultingtype, string property, string parameterName = "x")
{
var type = typeof(T);
ParameterExpression param = Expression.Parameter(type, parameterName);
Expression expr = param;
foreach (string prop in property.Split('.'))
{
PropertyInfo pi = type.GetProperty(prop);
expr = Expression.Property(expr, pi);
type = pi.PropertyType;
}
Type delegateType = typeof(Func<,>).MakeGenericType(typeof(T), type);
LambdaExpression lambda = Expression.Lambda(delegateType, expr, param);
resultingtype = type;
return lambda;
}
And here is what I have so far for my function:
public static IQueryable<T> Intersect<T>(this IQueryable<T> source, string property, IEnumerable<int> value)
{
//List of ids
var _value = Expression.Constant(value);
//Get delegate expression to the deep property and it's inner type
Type type = null;
var lambda = CreateDelegateExpression<T>(out type, property, "x");
var enumtype = type.GetGenericArguments()[0];
ParameterExpression tpe = Expression.Parameter(enumtype, "y");
Expression propExp = Expression.Property(tpe, enumtype.GetProperty("Id"));
MethodInfo innermethod = typeof(Queryable).GetMethods().Where(x => x.Name == "Select").First();
//Error on next line...
var selectCall = Expression.Call(typeof(Queryable),
"Select",
new Type[] { enumtype, typeof(long) },
lambda,
propExp);
//TODO: Add rest of logic and actually filter the source
return source;
}
In the var selectCall = line I'm getting error:
No generic method 'Select' on type 'System.Linq.Queryable' is compatible with the supplied type arguments and arguments. No type arguments should be provided if the method is non-generic.
I've read a lot here on SO and other sites but I can't get past this part, I feel I'm going to bump into more trouble when I get to the .Intersect(List<int>).Any() part so any help on that also would be grand, thanks.

After a lot of thought, investigation and attempts I came up with a solution.
First, I made a simpler version of my goal query (the static example I used in my question), so instead of:
var res = dbContext.Set<DT_Det_Tr>()
.Where
(x => x.TrParent.DataTypes
.Select(t => t.Id)
.Intersect(_vals)
.Any()
);
I made this:
var res = dbContext.Set<DT_Det_Tr>()
.Where
(x => x.TrParent.DataTypes
.Any(y => _vals.Contains(y.Id))
);
Which is a lot easier to translate to expressions (or at least it was for me) because it omits the Select call.
I got rid of the method I was using to create the deep navigation property expression and streamlined it in my Intersect function, this was because it was doing some work I don't really need here plus I needed access to some of the variables I use inside it, then I made this:
public static IQueryable<T> Intersect<T>(this IQueryable<T> source, string property, IEnumerable<int> value)
{
var type = typeof(T);
var _value = Expression.Constant(value); //List of ids
//Declare parameter for outer lambda
ParameterExpression param = Expression.Parameter(type, "x");
//Outer Lambda
Expression expr = param;
foreach (string prop in property.Split('.')) //Dig for deep property
{
PropertyInfo pi = type.GetProperty(prop);
expr = Expression.Property(expr, pi);
type = pi.PropertyType;
}
//Get deep property's type
var enumtype = type.GetGenericArguments()[0];
//Declare parameter for inner lambda
ParameterExpression tpe = Expression.Parameter(enumtype, "y");
//Inner Collection lambda logic
//Property for inner lambda
Expression propExp = Expression.Property(tpe, enumtype.GetProperty("Id"));
//Contains method call .Contains(y.Id)
var containsMethodExp = Expression.Call(typeof(Enumerable), "Contains", new[] { propExp.Type }, _value, propExp);
//Create Expression<Func<enumtype, bool>>
var innerDelegateType = typeof(Func<,>).MakeGenericType(enumtype, typeof(bool));
//Create Inner lambda y => _vals.Contains(y.Id)
var innerFunction = Expression.Lambda(innerDelegateType, containsMethodExp, tpe);
//Get Any method info
var anyMethod = typeof(Enumerable).GetMethods().Where(m => m.Name == "Any" && m.GetParameters().Length == 2).Single().MakeGenericMethod(enumtype);
//Call Any with inner function .Any(y => _vals.Contains(y.Id))
var outerFunction = Expression.Call(anyMethod, expr, innerFunction);
//Call Where
MethodCallExpression whereCallExpression = Expression.Call
(
typeof(Queryable),
"Where",
new Type[] { source.ElementType },
source.Expression,
Expression.Lambda<Func<T, bool>>(outerFunction, new ParameterExpression[] { param })
);
//Create and return query
return source.Provider.CreateQuery<T>(whereCallExpression);
}
I hope this helps anyone trying to develop a similar solution.
Working with expression trees can be very hard and frustrating at first, but it's a really powerful tool once you get the hold of it.

If you have access to the dynamic keyword from c# 4.0, you might be able to work around the problem like this:
var _vals = new List<int>();
var res = dbContext.Set<DT_Det_Tr>()
.Where(obj => { dynamic x = obj;
return x.TrParent.DataTypes
.Select(t => t.Id)
.Intersect(_vals)
.Any();
}
);
But I don't know enough about the details of the problem you want to solve to say for sure.

How to add an AND/OR expression to the following dynamic linq expression

T is a type that may or may not have a specific property, lets say 'City'. For the types that have a property named 'City', I would like to restrict the records such that only residents of the Gotham are returned and they are sorted.
public static IQueryable<T> OrderBy<T>(this IQueryable<T> source, string ordering, params object[] values)
{
var type = typeof(T);
var property = type.GetProperty(ordering);
var parameter = Expression.Parameter(type, "p");
var propertyAccess = Expression.MakeMemberAccess(parameter, property);
var orderByExp = Expression.Lambda(propertyAccess, parameter);
MethodCallExpression resultExp = Expression.Call(
typeof(Queryable),
"OrderBy",
new Type[] { type, property.PropertyType },
source.Expression,
Expression.Quote(orderByExp));
string propertyToRestrictOn = "City";
string restrictedValue = "Gotham";
var restrictedProperty = type.GetProperty(propertyToRestrictOn);
if(null ! = restrictedProperty )
{
// TODO: What to add here so than only those records are returned that have a
// property named City and the value is 'Gotham'???
}
return source.Provider.CreateQuery<T>(resultExp);
}
if possible please name/link some helpful literature here as well just in case I have to create more complex queries i.e. mix And/OR
The code was borrowed from
How do I apply OrderBy on an IQueryable using a string column name within a generic extension method?

I'm not quite sure if I have understood you correctly, but I think I was in the same situation a few months ago.
The posted code here was my solution.
I think you should be especially interested in line 24.
Edit:
PropertyInfo p = ... // I used reflection in my examply to get properties with a certain Attribute
var expressionParameter = Expression.Parameter(typeof(SomeClass), "lambda");
var parameter = new [] { expressionParameter };
var propertyAccess = Expression.Property(expressionParameter, p);
var nullCheck = Expression.NotEqual(propertyAccess, Expression.Constant(null, p.PropertyType));
var nullCheckLambda = Expression.Lambda<Func<SomeClass, Boolean>>(nullCheck, parameter);
var containsMethodInfo = typeof(String).GetMethod("Contains", new[] { typeof(String) });
var contains = Expression.Call(propertyAccess, containsMethodInfo, Expression.Constant("ell"));
var containsLambda = Expression.Lambda<Func<SomeClass, Boolean>>(contains, new[] { expressionParameter });
var predicate = Expression.Lambda<Func<SomeClass, Boolean>>(
// line 24
Expression.AndAlso(nullCheckLambda.Body, containsLambda.Body), parameter);
Console.WriteLine(predicate.ToString());

I think you're making this harder than you have to. In the first part of your code (the OrderBy()), you don't actually need to generate the whole query expression, just the lambda inside it.
And in the second part (the optional Where()) you can do pretty much the same thing, just add Expression.Equal() and Expression.Constant():
public static IQueryable<T> OrderBy<T>(this IQueryable<T> source, string ordering)
{
var type = typeof(T);
var property = type.GetProperty(ordering);
var parameter = Expression.Parameter(type, "p");
var propertyAccess = Expression.MakeMemberAccess(parameter, property);
// necessary for value types to work
var cast = Expression.Convert(propertyAccess, typeof(object));
var orderByExp = Expression.Lambda<Func<T, object>>(cast, parameter);
IQueryable<T> result = source.OrderBy(orderByExp);
string propertyToRestrictOn = "City";
string restrictedValue = "Gotham";
var restrictedProperty = type.GetProperty(propertyToRestrictOn);
if (restrictedProperty != null)
{
var restrictionParameter = Expression.Parameter(type, "p");
var restrictionPropertyAccess =
Expression.MakeMemberAccess(restrictionParameter, restrictedProperty);
var restrictionEquality =
Expression.Equal(restrictionPropertyAccess,
Expression.Constant(restrictedValue));
var whereExp =
Expression.Lambda<Func<T, bool>>(restrictionEquality, restrictionParameter);
result = result.Where(whereExp);
}
return result;
}
Also, if your method is going to do more than just ordering, I think you shouldn't call it OrderBy().

You're half-way there already. You have the ordered expression already, so you're just going to use the Queryable.OrderBy expression call in a Queryable.Where expression (or vice-versa, doesn't really matter).
if(null != restrictedProperty )
{
var notEqualExp = Expression.NotEqual(parameter,
Expression.Constant(restrictedValue, typeof(string)));
resultExp = Expression.Call(
typeof(Queryable),
"Where",
new Type[] { type },
resultExp,
Expression.Lambda(notEqualExp, parameter));
}
Haven't worked with building Expressions by hand in a while, so this is purely being done from memory. However, it should at least get you started and give you something to work with.
P.S. I would actually perform this check BEFORE the OrderBy method call. That way you end up with Queryably.Where(...).OrderBy(...) instead. But I guess if this is getting translated by the provider anyway, then shouldn't matter. However, just something I'd do to reduce any ambiguity of the generated query.