We Keep Coding

C#, Linq, Filter a list whether its properties appear on another list

I'm trying to filter a list if its element properties appear on another list.
Here is my example:
public class Detail
{
public int Id { get; set; }
public string Material { get; set; }
public string Title { get; set; }
public string Duration { get; set; }
public string FileName { get; set; }
public string FileLocation{ get; set; }
}
I have a dataList = List<Detail>() a displayList = ["Title", "Duration","FileName"] and a filterString = "test"
with a normal Linq we have:
dataList.Where(x x=> x.Title.Contains(filterString) || x.Duration.Contains(filterString) ||x.FileName.Contains(filterString))
but my task is to filter it in more programmatically way, I want to filter dataList if the Detail properties appear on displayList. Is there any way to do it

I want to filter dataList if the Detail properties appear on displayList
Not very elegant, but straightforward:
dataList.Where(x =>
displayList.Contains(nameof(Detail.Title)) && x.Title.Contains(filterString) ||
displayList.Contains(nameof(Detail.Duration)) && x.Duration.Contains(filterString) ||
...
)

The following solution requires some more coding, but it can be reused for every class and for every type that you want to check.
We won't make a solutioin for class Detail only, nor for filterStrings only,
I'll make this a generic extension methods of IEnumerable<...>, so you can intertwine it with your other LINQ methods. See extension methods demystified
First I'll give the signatures, then I write the code:
Just like most LINQ functions: you can provide a comparer, but you don't have to; in that case the default comparer is used:
public static IEnumerable<TSource> Filter<TSource, TKey>(
this IEnumerable<TSource> source,
IEnumerable<string> propertyNames,
TKey filterValue)
{
// only call the overload with a null comparer:
return Filter(source, propertyNames, filterValue, null);
}
public static IEnumerable<TSource> Filter<TSource, TKey>(
this IEnumerable<TSource> source,
IEnumerable<string> propertyNames,
TKey filterValue,
IEqualityComparer comparer)
{
// TODO: exceptions if input null
if (comparer == null) comparer = EqualityComparer<TKey>.Default;
// TODO: implement
}
First I'll translate the strings to PropertyInfos.
Convert every PropertyInfo into a PropertySelector, similar to the keySelector in GroupBy.
Convert every KeySelector and the filterValue into a predicate as used in Where
So from a sequence of strings, I have now created a sequence of predicates: put in an object of class Detail, and it will give you for every predicate whether the property equals filterValue.
I'll do this in smaller steps, so you understand what's happening:
// From the Type of TSource, get all public readable properties
// that have a name that is in propertyNames
// and that returns a type of TKey
Type type = typeof(TSource);
IEnumerable<PropertyInfo> propertyInfos = type.GetProperties()
.Where(property => property.CanRead
&& propertyNames.Contains(property.Name)
&& property.PropertyType == typeof(TKey));
So if you mentioned a propertyName that does not return the correct type (in your Detail it had to be a string), those properties are not used. If you don't want that, consider throwing an exception.
Now that we have all PropertyInfos, we can get values and compare them. The easy way is a simple foreach:
foreach (TSource sourceElement in source)
{
// does this source element match all predicates?
bool matchesAllFilterValues = propertyInfos.All(propertyInfo =>
{
TKey propertyValue = (TKey)propertyInfo.GetValue(sourceElement);
bool matchesFilterValue = comparer.Equals(propertyValue, filterValue);
return matchesFilterValue;
});
// if all filter values are matched, then this sourceElement passes the filter
if (matchesAllFilterValues)
yield return sourceElement;
}
Some people don't like to yield return, you can also continue the LINQ statements:
// From every PropertyInfo, make a keySelector, similar to GroupBy
IEnumerable<Func<TSource, TKey>> keySelectors = propertyInfos
.Select<PropertyInfo, Func<TSource, TKey>> (propertyInfo =>
x => (TKey)propertyInfo.GetValue(x));
So if we had a string "Title", it was converted into the PropertyInfo with name Title. If you put a use it to GetValue(detcal), you get the string value of detail.Title.
// From every KeySelector and the filterValue, create a Predicate, like in Where
IEnumerable<Func<TSource, bool>> predicates = keySelectors
.Select<Func<TSource, TKey>, Func<TSource, bool>>(
keySelector => sourceItem => comparer.Equals(keySelector(sourceItem), filterValue));
We had the keySelector to fetch detail.Title, this is converted to a predicate: true if detail.Title == filterValue (using the comparer)
Now that you've converted your sequence of propertyNames and your filterValue, you can return all source items that match all predicates:
IEnumerable<TSource> filteredValues = source.Where(sourceItem =>
predicates.All(predicate => predicate(sourceItem);
return filteredValues;
Of course you can put this into one big LINQ statement, but I don't believe this makes it more readable. I even think that the yield method is more readable.
Finally a usage:
string filterValue = "Test";
IEnumerable<string> propertyNames = new string[] {"Title", "FileName"};
IEnumerable<Detail> details = ...
IEnumerable<Detail> detailsWithTitleFileNameEqualsTest = details.Filter(
propertyNames,
filterValue);
Or:
IEnumerable<Detail> detailsWithTitleFileNameEqualsTest = details.Filter(
propertyNames,
filterValue,
StringComparer.OrdinalIgnoreCase);
You can even intertwine it with other LINQ statements:
var result = dbLibraryContext.Books
.Where(book => ...)
.Select(book => new Detail() {....})
.Filter(propertyNames, filterValue)
.GroupBy(detail => detail.Material);

LINQ filter combining exact matches like SQL IN and StartsWith matches

I'm having product entity:
public class Product : DomainBase
{
public virtual string Name { get; set; }
}
And there should be option to select products by filter, which contains an array of names, like:
public static IEnumerable<Product> SearchArrayQueryLinq(IEnumerable<string> names)
{
using (var session = Database.OpenSession())
{
var products = session.Query<Product>();
var result = products.Where(product => names.Any(name => product.Name.Contains(name)));
return result.ToList();
}
}
but it throws
System.NotSupportedException: Specified method is not supported.
What is right approach, to accomplish such filtering?

Without knowing more about what database you're connecting to or what library (is it RavenDB.. having done a quick Google?) then it's hard to be completely sure what the problem is.
However, what I think is happening is that you are giving an expression to the IQueryable "Where" extension method and the library is trying to turn that into search criteria to run against the db.. and failing because "Any" is not supported in nested criteria like that (again, I'm guessing).
The LINQ expressions that may or may not be translated into the database language (eg. SQL) vary by the library that performs the translation and vary by the database being talked to.
For example, the following (which is basically what you want to do) works fine with Entity Framework:
private static void Test(IEnumerable<string> names)
{
using (var context = new NORTHWNDEntities())
{
foreach (var product in context.Products.Where(product => names.Any(name => product.ProductName.Contains(name))))
{
Console.WriteLine(product.ProductName);
}
}
Console.ReadLine();
}
One easy option for you is to change your code to
public static IEnumerable<Product> SearchArrayQueryLinq(IEnumerable<string> names)
{
using (var session = Database.OpenSession())
{
var products = session.Query<Product>();
return result = products.ToList().Where(product => names.Any(name => product.Name.Contains(name)));
}
}
This should work.. however, it will get all Products from the database and perform the filtering in-memory. This is less efficient than getting the database to perform the search.
An alternative would be to generate an "Expression<Func<Product, bool>>" filter yourself that is easier for the library that you're using to translate. If, instead, of a nested "Any" criteria, you could generate a simple set of "OR" name checks then there is a better change of it working. The following will achieve that - but it's quite a lot of code. If this is something that you need to do in several places then some of the code could be made more general and reused.
private static IEnumerable<Product> SearchArrayQueryLinq(IEnumerable<string> names)
{
using (var context = new NORTHWNDEntities())
{
return context.Products.Where(GetCombinedOrFilter(names)).ToList();
}
}
private static Expression<Func<Product, bool>> GetCombinedOrFilter(IEnumerable<string> names)
{
var filter = GetNameFilter(names.First());
foreach (var name in names.Skip(1))
filter = CombineFiltersAsOr(filter, GetNameFilter(name));
return filter;
}
private static Expression<Func<Product, bool>> GetNameFilter(string name)
{
return product => product.ProductName.Contains(name);
}
private static Expression<Func<Product, bool>> CombineFiltersAsOr(Expression<Func<Product, bool>> x, Expression<Func<Product, bool>> y)
{
// Combine two separate expressions into one by combining as "Or". In order for this to work, instead of there being a parameter
// for each expression, the parameter from the first expression must be shared between them both (otherwise things will go awry
// when this is translated into a database query) - this is why ParameterRebinder.ReplaceParameters is required.
var expressionParameter = x.Parameters.Single();
return Expression.Lambda<Func<Product, bool>>(
Expression.Or(x.Body, ParameterRebinder.ReplaceParameters(y.Body, toReplace: y.Parameters.Single(), replaceWith: expressionParameter)),
expressionParameter
);
}
// Borrowed and tweaked from https://blogs.msdn.microsoft.com/meek/2008/05/02/linq-to-entities-combining-predicates/
public sealed class ParameterRebinder : ExpressionVisitor
{
public static Expression ReplaceParameters(Expression expression, ParameterExpression toReplace, ParameterExpression replaceWith)
{
return new ParameterRebinder(toReplace, replaceWith).Visit(expression);
}
private readonly ParameterExpression _toReplace, _replaceWith;
private ParameterRebinder(ParameterExpression toReplace, ParameterExpression replaceWith)
{
_toReplace = toReplace;
_replaceWith = replaceWith;
}
protected override Expression VisitParameter(ParameterExpression p)
{
if (p == _toReplace)
p = _replaceWith;
return base.VisitParameter(p);
}
}
Update: I didn't notice your nhibernate tag - whoops! Using the criteria combining methods that nhibernate has is probably easier than all this.. :) I would have commented on your answer rather than updating my own but I haven't got the requisite 50 rep yet..

You are trying to mix both kinds of conditions and applying IEnumerable methods on string properties.
Your query should look like this:
var result = products.Where(product => names.Contains(product.Name));
to find exact matches.
For a combination of exact matches and StartsWith it should look like this:
var results = products.Where(product => (names.Contains(product.Name) || names.Any(name => name.StartsWith(product.Name))));

As I did a dive into NHibenrate documentation, it contains CriteriaAPI, so I came up to this
using (var session = Database.OpenSession())
{
var products = session.CreateCriteria<Product>();
if (names == null)
{
return products.List<Product>();
}
var orClause = Expression.Disjunction();
foreach (var name in names)
{
orClause.Add(Expression.Like(nameof(Product.Name), name, MatchMode.Start));
}
products.Add(orClause);
return products.List<Product>();
}

select a property dynamically in linq

I have a property whose name is in a variable (userFilters.property), which can be of type string/enum/number.
I want to select the distinct values of that property dynamically. how do I do it.
var query = CreatePincodeQuery(useFilters);
//userFilters.property contains the property to be selected
Expression<Func<PincodeData, string>> selectExpr = null;
IList<string> list = query.Select(selectExpr)
.Distinct()
.OrderBy(selectExpr)
.ToList();
return list.;
I should be creating the expression of type Expression<Func<PincodeData, string>> selectExpr to use as part of select & orderBy block.
How do I do it?
I looked at the solution provided here, here and here, but not able to understand how to I modify those to fit my need.
EDIT
came up with below solution, as expected its not working, and how do i convert the value to string.
Func<TEntity, string> CreateNewStatement<TEntity>(string field)
{
//https://stackoverflow.com/questions/16516971/linq-dynamic-select
var xParameter = Expression.Parameter(typeof(TEntity), "o");
var property = Expression.Property(xParameter, typeof(TEntity).GetProperty(field));
var lambda = Expression.Lambda<Func<TEntity, string>>(property, xParameter);
return lambda.Compile();
}
EDIT
I changed the type from string to object, but its still failing while creating the lambda for enum types, what could be the reason

In the last case, when you changed the return type to object, you didn't convert the value type to it.
That mistake is rather common because we, C# developers, writing our own C# code, get used to relying on the C# compiler to make the value type to object conversion automatically. But, that's not the case for expressions, so you need to do the conversion explicitly.
Your particular case is solved below, but! It is very, very easy. Since you are on the way to dynamically building queries based on the end-user input your next steps might become complicated very fast (for instance, filtering with logical conditions). Once that happens, consider using Dynamic LINQ, which is very powerful for such tasks.
public class Entity
{
public string StringProp { get; }
public int IntProp { get; }
public DayOfWeek EnumProp { get; }
public Entity(string stringProp, int intProp, DayOfWeek enumProp)
{
StringProp = stringProp;
IntProp = intProp;
EnumProp = enumProp;
}
}
[Test]
public void Expressions()
{
var entities = new List<Entity>
{
new("Prop3", 3, DayOfWeek.Wednesday),
new("Prop2", 2, DayOfWeek.Tuesday),
new("Prop1", 1, DayOfWeek.Monday),
};
const string stringPropName = nameof(Entity.StringProp);
var stringStatement = CreateNewStatement<Entity>(stringPropName);
const string intPropName = nameof(Entity.IntProp);
var intStatement = CreateNewStatement<Entity>(intPropName);
const string enumPropName = nameof(Entity.EnumProp);
var enumStatement = CreateNewStatement<Entity>(enumPropName);
IList<object> listOfStrings = entities.Select(stringStatement).Distinct().OrderBy(i => i).ToList();
TestContext.WriteLine(string.Join(",", listOfStrings));
IList<object> listOfInts = entities.Select(intStatement).Distinct().OrderBy(i => i).ToList();
TestContext.WriteLine(string.Join(",", listOfInts));
IList<object> listOfEnums = entities.Select(enumStatement).Distinct().OrderBy(i => i).ToList();
TestContext.WriteLine(string.Join(",", listOfEnums));
}
private static Func<TEntity, object> CreateNewStatement<TEntity>(string fieldName)
{
var parameter = Expression.Parameter(typeof(TEntity), "o");
var propertyInfo = typeof(TEntity).GetProperty(fieldName);
if (propertyInfo == null)
throw new InvalidOperationException($"Property '{fieldName}' not found");
Expression body = Expression.Property(parameter, propertyInfo);
if (propertyInfo.PropertyType.IsValueType)
body = Expression.Convert(body, typeof(object));
var lambda = Expression.Lambda<Func<TEntity, object>>(body, parameter);
return lambda.Compile();
}

How to build a custom expression for an advanced search screen

I am building an advanced search screen and am using nHibernate to query the DB. I have already built my DataAccess layer and have built a generic method which works great - I pass in an expression to be used as a predicate and passes back a collection of an object that matches the predicate:
public object LoadByPredicate<T>(Expression<Func<T, bool>> predicate) where T : class
Example usage is:
var items = _query.LoadByPredicate<StaticTypes>(x => x.StaticName == type) as List<StaticTypes>;
This is great.
I am now building an advanced search screen on my application where a user can search by between one and 20 different options to return matching "Products". I.e. The Product class has 20 properties and they can search by a combination of Name, Reference, Description, Value etc.....
I'd like to build a mechanism that will construct an expression for me that I can pass into my "LoadByPredicate" method above.
So far I proved this will work:
var type = typeof(Model.Product);
var property = type.GetProperty("ProductMetalProductType");
var parameter = Expression.Parameter(typeof(Model.Product), "x");
Expression<Func<Model.Product, bool>> predicate =
(Expression<Func<Model.Product, bool>>)Expression.Lambda(
Expression.Equal(
Expression.MakeMemberAccess(parameter, property),
Expression.Constant(metalProdType)),
parameter);
This works great for one item the property called "ProductMetalProductType". However I can't see how I can expand this without writing a HUGE amount of code. How can I write some code where I say on the lines of "if Product Metal type is not empty" add an extra expression to refine the search predicate by that?
Or am I going down the wrong track with the way I am constructing my expression?
Thanks in advance

You can do like this:
public class Program
{
static void Main(string[] args)
{
var parameters = new Dictionary<string, object>();
parameters.Add("ProductMetalProductType", 1);
parameters.Add("IsActive", true);
parameters.Add("Name", "New cool product");
var expr = GenerateExpression<Product>(parameters);
Console.WriteLine("Result expression:");
Console.WriteLine(expr.ToString());
}
private static Expression<Func<T, bool>> GenerateExpression<T>(Dictionary<string, object> properties)
{
var type = typeof(T);
List<Expression> expressions = new List<Expression>();
var parameter = Expression.Parameter(typeof(T), "x");
foreach (var key in properties.Keys)
{
var val = properties[key];
var property = type.GetProperty(key);
var eqExpr = Expression.Equal(Expression.MakeMemberAccess(parameter, property), Expression.Constant(val));
expressions.Add(eqExpr);
}
Expression final = expressions.First();
foreach (var expression in expressions.Skip(1))
{
final = Expression.And(final, expression);
}
Expression<Func<T, bool>> predicate =
(Expression<Func<T, bool>>) Expression.Lambda(final, parameter);
return predicate;
}
}
public class Product
{
public int ProductMetalProductType { get; set; }
public bool IsActive { get; set; }
public string Name { get; set; }
}
Here is working fiddle - http://dotnetfiddle.net/t0a9yA
Basically you can fill Dictionary with needed parameters, then generate expression based on that dictionary. Dictionary key is a property name, and value is value for filtering.

Linq to SQL Where clause based on field selected at runtime

I'm trying to create a simple reusable search using LINQ to SQL.
I pass in a list of words entered in a search box. The results are then filtered based on this criteria.
private IQueryable<User> BasicNameSearch(IQueryable<User> usersToSearch, ICollection<string> individualWordsFromSearch)
{
return usersToSearch
.Where(user =>
individualWordsFromSearch.Contains(user.Forename.ToLower())
|| individualWordsFromSearch.Contains(user.Surname.ToLower()));
}
Now I want this same search functionality on a different datasource and want to dynamically select the fields to apply the search to. For instance instead of IQueryable of Users I may have an IQueryable of Cars and instead of firstname and surname the search goes off Make and Model. Basically the goal is to reuse the search logic by dynamically selecting what to search on at runtime.

You could create an extension method that will compile your string selectors together into one expression:
public static class CompileExpressions
{
public static IQueryable<T> SearchTextFieldsOr<T>(this IQueryable<T> source,
ICollection<string> wordsFromSearch, params Func<T, string>[] stringSelectors)
{
Expression<Func<T, bool>> compiledExpression = t => false;
foreach (var filter in stringSelectors)
{
compiledExpression = compiledExpression.Or(t => wordsFromSearch.Contains(filter(t)));
}
var compiled = compiledExpression.Compile();
return source.Where(t => compiled(t));
}
public static IQueryable<T> SearchTextFieldsAnd<T>(this IQueryable<T> source,
ICollection<string> wordsFromSearch, params Func<T, string>[] stringSelectors)
{
foreach (var filter in stringSelectors)
{
source = source.Where(t => wordsFromSearch.Contains(filter(t)));
}
return source;
}
//Taken from http://www.albahari.com/nutshell/predicatebuilder.aspx
public static Expression<Func<T, bool>> Or<T>(this Expression<Func<T, bool>> expr1,
Expression<Func<T, bool>> expr2)
{
var invokedExpr = Expression.Invoke(expr2, expr1.Parameters.Cast<Expression>());
return Expression.Lambda<Func<T, bool>>
(Expression.OrElse(expr1.Body, invokedExpr), expr1.Parameters);
}
}
An example of how this could be used:
public class Entity
{
public string Name { get; set; }
public string Type { get; set; }
public string Model { get; set; }
public string Colour { get; set; }
}
class Program
{
static void Main(string[] args)
{
var source = new[]{
new Entity { Colour = "Red", Model = "New", Name="Super", Type="Filter"},
new Entity { Colour = "Green", Model = "New", Name="Super", Type="Filter"},
new Entity { Colour = "Green", Model = "New", Name="Super", Type="Filter"},
new Entity { Colour = "Green", Model = "New", Name="Super", Type="Filter"},
new Entity { Colour = "Green", Model = "New", Name="Super", Type="Filter"},
new Entity { Colour = "Green", Model = "New", Name="Super", Type="Amazing"},
};
var filters = new[] {"Red", "Amazing" };
var filteredOr = source
.AsQueryable()
.SearchTextFieldsOr(filters, t => t.Colour, t => t.Type)
.ToList();
//2 records found because we're filtering on "Colour" OR "Type"
var filteredAnd = source
.AsQueryable()
.SearchTextFieldsAnd(filters, t => t.Colour, t => t.Type)
.ToList();
//1 record found because we're filtering on "Colour" AND "Type"
}
}
Because your string selector argument is of type params Func<T, string>[], you can add as many string selectors as you want to be included in your query.

Your question is similar to this thread's (not the same question though).
In a nutshell, when you write a linq-to-sql request, it only builds a System.Linq.Expression corresponding to the actual code you typed which is given to a "provider", which will translate it to sql (you can get this provider by casting your request to IQueryable, which has a Provider property).
In fact, the code forming you request will never be "executed", and is actually not even compiled to IL like delegates you would pass to Linq-to-objects functions. (which is using System.Linq.Enumerable extension methods, while linq-to-sql is using System.Linq.Queryable ones)
Thus, you may also create linq expressions "manually" (instead of letting c# compiler build it for you under the hood), and pass them to the provider, which will parse and execute them as if you created them using the regular way.
See examples in the thread given above.
edit: or you could just look at Oliver's answer, who gave you a copy-paste-run sample :)