Build a dynamic where clause over multiple properties - c#

What I have is a List<string> IndexFields which contains a list of property names.
My issue is that I need to build a where clause based on the elements in the list.
So far I have;
var sitem = List1.Where(p => (p.GetType().GetProperty(IndexFields[0])
.GetValue(p, null) as string) == "red").FirstOrDefault();
But that only allows me to specify a single property. What I need is a builder that can build based on all the names in the List<string> IndexFields list.

The most flexible way to create dynamic queries at runtime is by using the Expression API:
For example:-
var type = typeof(T);
var properties = IndexFields.Select(x => type.GetProperty(x));
// x
var paramter = Expression.Parameter(type);
// x.Foo, x.Bar, ...
var leftHandSides = properties.Select(
x => Expression.Property(parameter, x));
// "Baz"
var rightHandSide = Expression.Constant(...);
// x.Foo == "Baz", x.Bar = "Baz", ...
var equalityExpressions = leftHandSides.Select(
x => Expression.Equal(x, rightHandSide));
// x.Foo == "Baz" && x.Bar == "Baz" && ...
var aggregatedExpressions = equalityExpressions.Aggregate(
(x, y) => Expression.AndAlso(x, y));
// x => x.Foo == "Baz" && x.Bar == "Baz" && ...
var lambda = Expression.Lambda<Func<T,bool>>(
aggregatedExpressions, parameter)
var item = List1.Where(lambda).FirstOrDefault();
A huge advantage of building your queries like this is that the resulting expression can still be e.g. translated into SQL for use with Entity Framework, wheras using reflection inside the body of your lambda is really limiting.
I really do recommend taking some time to really understand the expression framework before using it, though. If you can grok what's going on, it saves you a ton of time in the long run.
You can read more at e.g:-
http://www.digitallycreated.net/Blog/37/dynamic-queries-in-entity-framework-using-expression-trees
https://stackoverflow.com/questions/1217539/net-expression-trees-tutorial
If you're looking for something quicker and dirtier, however, you can just go ahead and chain up those Where clauses inside a foreach:-
IEnumerable<T> query = List1;
foreach (var property in IndexFields)
{
// The variable "property" gets hoisted out of local context
// which messes you up if the query is being evaluated with
// delayed execution.
// If you're working in C# 6 though, you don't need to do this.
var localProperty = property;
query = query.Where(
p => (p.GetType().GetProperty(localProperty)
.GetValue(p, null) as string) == "red");
}
var sitem = query.FirstOrDefault();

You can use a PredicateBuilder for this:
var predicate = PredicateBuilder.New<string>();
if (aCondition)
{
predicate = predicate.And(s => s == "this");
}
if (bCondition)
{
predicate = predicate.And(s => s == "that");
}

you can try something like this, worked for me in linqpad
void Main() {
var listFields = new string[] { "Field1", "Field2" };
var listValues = new string[] { "value1", "value2" };
// prepare & show dummy data
var listItems = Enumerable.Range(1, 100).Select(aaIndex => new MyItem {
Name = string.Format("item{0}", aaIndex),
Field1 = string.Format("value{0}", aaIndex % 3),
Field2 = string.Format("value{0}", aaIndex % 7)
});
listItems.Dump();
// apply filtering
var filtered = listItems.Where(aaItem => Enumerable.Range(0, listFields.Length).All(aaIndex => {
var value1 = aaItem.GetType().GetProperty(listFields[aaIndex]).GetValue(aaItem, null);
var value2 = listValues[aaIndex];
if (value1 is IComparable) {
return ((IComparable)value1).CompareTo(value2) == 0;
}
return Convert.ToString(value1) == Convert.ToString(value2);
}));
filtered.Dump();
}
// Define other methods and classes here
class MyItem {
public string Name { get; set; }
public string Field1 { get; set; }
public string Field2 { get; set; }
}

The dynamic linq library worked well for stuff like this:
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
It added overloads to take the different clauses as strings ie:
var query = List1.Where("Color1=""Red"" or Color2=""Red""");
In your case you could build the string from your index fields (probably in a loop but simplified here)
var query = List1.Where(IndexFields[0] + "=""Red"" or " IndexFields[1] + "=Red");
For use, download the sample package, and then grab LinqSamples\DynamicQuery\DynamicQuery\Dynamic.cs and compile with your project.

Related

How to make an List of Objects a parameter in a Lamba using C# Expression API

I'm trying to dynamically create an predicate to pass into a linq where clause. This is for a generic method that takes two list of the same Type and and list of property names to compare.
void SomeMethod<T>(List<T> oldRecords, List<T> newRecords, List<string> propertiesOfT)
{
// dynamically build predicate for this
var notMatch = oldRecords.Where(o => !newRecords.Any(n => n.Prop1 == o.Prop1 && n.Prop2 == o.Prop2)).ToList();
// do somethind with notMatch
}
I would like to convert this:
var notMatch = oldRecords.Where(o => !newRecords.Any(n => n.Prop1 == o.Prop1 && n.Prop2 == o.Prop2)).ToList();
To achieve this:
var predicate = "n => n.Prop1 == o.Prop1 && n.Prop2 == o.Prop2"; // sudo code
var notMatch = oldRecords.Where(o => !newRecords.Any(predicate));
or this
var predicate = "o => !newRecords.Any(n => n.Prop1 == o.Prop1 && n.Prop2 == o.Prop2)" // sudo code
var notMatch = oldRecords.Where(predicate);
How do I populate newRecords when dynamically creating the Expression?
And how would I reference parameter o and parameter n in the Expresssion.
I've gotten this far:
//construct the two parameters
var o = Expression.Parameter(typeof(T), "o");
var n = Expression.Parameter(typeof(T), "n");
// How to I go about populating o with values and n with values
// from oldRecords and newRecords? or is that no neccessary
var property = Expression.Property(o, typeof(T).GetProperty("Id").Name);
var value = Expression.Constant(Convert.ChangeType("12345", typeof(T).GetProperty("Id").PropertyType), typeof(T).GetProperty("Id").PropertyType);
BinaryExpression binaryExpression = Expression.MakeBinary(ExpressionType.Equal, property, value);
Any sudo code or clue where to look to achieve this?
With reflection it's quite easy. You just to have to think about it. Here's the working version.
void SomeMethod<T>(List<T> oldRecords, List<T> newRecords, List<string> propertiesOfT)
{
// get the list of property to match
var properties = propertiesOfT.Select(prop => typeof(T).GetProperty(prop)).ToList();
// Get all old record where we don't find any matching new record where all the property equal that old record
var notMatch = oldRecords.Where(o => !newRecords.Any(n => properties.All(prop => prop.GetValue(o).Equals(prop.GetValue(n))))).ToList();
}
And here's a sample set i tried and it works
public class test
{
public int id { get; set; } = 0;
public string desc { get; set; } = "";
public test(string s, int i)
{
desc = s;id = i;
}
}
private void Main()
{
var oldRecords = new List<test>()
{
new test("test",1),
new test("test",2)
};
var newRecords = new List<test>()
{
new test("test1",1),
new test("test",2)
};
SomeMethod(oldRecords, newRecords, new List<string>() { "id", "desc" });
}

C# IQueryable use dictionary inside Where throws Unable to create a constant value of type Exception

I am using the dictionary inside the IQueryable lambda linq throws the
Unable to create a constant value of type 'System.Collections.Generic.KeyValuePair`2
Code :
Dictionary<int, int> keyValues = new Dictionary<int, int>();
IQueryable<Account> = context.Account
.Where(W => keyValues
.Where(W1 => W1.Key == S.AccountID)
.Where(W1 => W1.Value == S.Balance)
.Count() > 0);
Details:
I have the data inside the dictionary like this
AccountID Balance
1 1000
2 2000
3 3000
I want the user which have the (ID = 1 AND Balance = 1000) OR (ID = 2 AND BALANCE = 2000) OR (ID = 3 AND BALANCE = 3000)
So how can I write the lambda for it ?
Edited
Thanks #caesay, Your answer help me lots.
I want one more favor from you.
From you answer I create the expression which look like below:
private static Expression<Func<Accounting, bool>> GenerateExpression(Dictionary<int, int> lstAccountsBalance)
{
try
{
var objAccounting = Expression.Parameter(typeof(Accounting));
Expression expr = null;
const bool NOT_ALLOWED = false;
if (lstAccountsBalance != null && lstAccountsBalance.Count > 0)
{
var clauses = new List<Expression>();
foreach (var kvp in lstAccountsBalance)
{
clauses.Add(Expression.AndAlso(
Expression.Equal(Expression.Constant(kvp.Key), Expression.Property(objAccounting, nameof(Accounting.ID))),
Expression.Equal(Expression.Constant(kvp.Value), Expression.Property(objAccounting, nameof(Accounting.Balance)))
));
}
expr = clauses.First();
foreach (var e in clauses.Skip(1))
{
expr = Expression.OrElse(e, expr);
}
var notAllowedExpr = Expression.AndAlso(
Expression.Equal(Expression.Constant(NOT_ALLOWED), Expression.Property(objAccounting, nameof(Accounting.ALLOWED))),
Expression.Equal(Expression.Constant(true), Expression.Constant(true))
);
expr = Expression.And(notAllowedExpr, expr);
}
var allowedExpr = Expression.AndAlso(
Expression.Equal(Expression.Constant(!NOT_ALLOWED), Expression.Property(objAccounting, nameof(Accounting.ALLOWED))),
Expression.Equal(Expression.Property(objAccounting, nameof(Accounting.ID)), Expression.Property(objAccounting, nameof(Accounting.ID)))
);
if (expr != null)
{
expr = Expression.OrElse(allowedExpr, expr);
}
else
{
expr = allowedExpr;
}
return Expression.Lambda<Func<Accounting, bool>>(expr, objAccounting);
}
catch (Exception ex)
{
throw objEx;
}
}
After that I compiled the expression like this:
Expression<Func<Accounting, bool>> ExpressionFunctions = GenerateExpression(lstAccountsBalance);
var compiledExpression = ExpressionFunctions.Compile();
And I used like this:
.Select(S => new
{
Accounting = S.Accounts
.Join(context.AccountInfo,
objAccounts => objAccounts.ID,
objAccountInfo => objAccountInfo.ID,
(objAccounts, objAccountInfo) => new Accounting
{
ID = objAccounts.ID,
Balance = objAccountInfo.Balance,
})
.Where(W => W.ID == user.ID)
.AsQueryable()
.Where(W => compiledExpression(W))
.Select(S1 => new Accounting()
{
ID = S1.ID,
Balance = S1.Balance
})
.ToList(),
}
And it throws the exception with the message:
System.NotSupportedException: The LINQ expression node type 'Invoke'
is not supported in LINQ to Entities.
Without Compile
Without the compile it works like charm. It gives the output want I want.
Expression<Func<Accounting, bool>> ExpressionFunctions = GenerateExpression(lstAccountsBalance);
Use:
.Select(S => new
{
Accounting = S.Accounts
.Join(context.AccountInfo,
objAccounts => objAccounts.ID,
objAccountInfo => objAccountInfo.ID,
(objAccounts, objAccountInfo) => new Accounting
{
ID = objAccounts.ID,
Balance = objAccountInfo.Balance,
})
.Where(W => W.ID == user.ID)
.AsQueryable()
.Where(ExpressionFunctions)
.Select(S1 => new Accounting()
{
ID = S1.ID,
Balance = S1.Balance
})
Thank you..
Everything inside of a EF linq query needs to be compiled to an Expression tree, and then to SQL, but the dictionary is an IEnumerable so there is no way that EF could know how to compile that.
You can either build the expression tree yourself, or use the System.Linq.Dynamic nuget package to build the sql yourself.
The example using System.Linq.Dynamic, first install the nuget package and add the using to the top of your file, then there will be a Where overload that takes a string as a parameter:
context.Account.Where(
String.Join(" OR ", keyValues.Select(kvp => $"(ID = {kvp.Key} AND Balance = {kvp.Value})")));
Essentially, everything inside of the Where clause will be executed directly as SQL, so be careful to use the Where(string, params object[]) overload to paramaterize your query if accepting user input.
The expression tree (untested) approach might look like the following:
var account = Expression.Parameter(typeof(Account));
var clauses = new List<Expression>();
foreach(var kvp in keyValues)
{
clauses.Add(Expression.AndAlso(
Expression.Equal(Expression.Constant(kvp.Key), Expression.Property(account, nameof(Account.AccountID))),
Expression.Equal(Expression.Constant(kvp.Value), Expression.Property(account, nameof(Account.Balance)))
));
}
var expr = clauses.First();
foreach (var e in clauses.Skip(1))
expr = Expression.OrElse(e, expr);
context.Account.Where(Expression.Lambda<Func<Account, bool>>(expr, account));
Essentially inside the foreach loop we're creating all of the (ID = ... AND Balance = ...) and then at the end we join them all with an OR.

Rewriting a Linq query to use if / else and ditch the var

I've recently started playing with Linq (cue groans), and am trying to get the following to compile. Now, the whereclause part uses DynamicLinq, which works fine; it's the var placeholder variable that the compiler wants a real class for; unfortunately, I am using what I believe is an anonymous class, and am not sure how to take it from here. Any suggestions?
var query;
if(whereclause != string.Empty)
{
query = Directory.GetFiles(LRSettings.Default.OperatingDirectory, LRSettings.Default.FileExtension,
SearchOption.AllDirectories).AsQueryable()
.Select(Filename => new { Filename, new FileInfo(Filename)
.LastWriteTime, new FileInfo(Filename).Extension, new FileInfo(Filename).Length })
.Where(whereclause);
}
else
{
query = Directory.GetFiles(LRSettings.Default.OperatingDirectory,
LRSettings.Default.FileExtension,
SearchOption.AllDirectories)
.AsQueryable()
.Select(Filename => new { Filename, new FileInfo(Filename).LastWriteTime, new FileInfo(Filename).Extension, new FileInfo(Filename).Length });
}
Since it seems like the only difference in your query is whether or not to include a WHERE clause, you should just be able to do this:
var query = Directory.GetFiles(LRSettings.Default.OperatingDirectory, LRSettings.Default.FileExtension, SearchOption.AllDirectories)
.AsQueryable()
.Select(Filename => new { Filename, new FileInfo(Filename).LastWriteTime, new FileInfo(Filename).Extension, new FileInfo(Filename).Length });
if(whereclause != string.Empty)
{
query = query.Where(whereclause);
}
Since you're using an IEnumerable, I don't think you have to worry about pulling too much data without the Where() clause, since it doesn't get enumerated until you access query in some fashion (like binding to a form or whatnot).
Two immediate options are to use ?: or to extract the common/starting query.
For the former:
bool expr = SomeTrueOrFalseValue();
var l = new [] { 1,2,3 };
// both "true" and "false" branches unify an IEnumerable<int>
var q = expr
? l
: l.Where(x => x > 1);
// q typed as IEnumerable<int>
For the latter:
var q = l.AsEnumerable();
// q is typed as IEnumerable<int>
if (!expr) {
q = q.Where(x => x > 1);
}
// q is still typed as IEnumerable<int> - can't be changed after var
An assignment must be included in a var declaration so the variable's type can be determined.
But without var:
IEnumerable<int> q; // not practical/possible for complex types
if (expr) {
q = l;
} else {
q = l.Where(x => x > 1);
}

Assign values from one list to another using LINQ

Hello I have a little problem with assigning property values from one lists items to anothers. I know i could solve it "the old way" by iterating through both lists etc. but I am looking for more elegant solution using LINQ.
Let's start with the code ...
class SourceType
{
public int Id;
public string Name;
// other properties
}
class DestinationType
{
public int Id;
public string Name;
// other properties
}
List<SourceType> sourceList = new List<SourceType>();
sourceList.Add(new SourceType { Id = 1, Name = "1111" });
sourceList.Add(new SourceType { Id = 2, Name = "2222" });
sourceList.Add(new SourceType { Id = 3, Name = "3333" });
sourceList.Add(new SourceType { Id = 5, Name = "5555" });
List<DestinationType> destinationList = new List<DestinationType>();
destinationList.Add(new DestinationType { Id = 1, Name = null });
destinationList.Add(new DestinationType { Id = 2, Name = null });
destinationList.Add(new DestinationType { Id = 3, Name = null });
destinationList.Add(new DestinationType { Id = 4, Name = null });
I would like to achieve the following:
destinationList should be filled with Names of corresponding entries (by Id) in sourceList
destinationList should not contain entries that are not present in both lists at once (eg. Id: 4,5 should be eliminated) - something like inner join
I would like to avoid creating new destinationList with updated entries because both lists already exist and are very large,
so no "convert" or "select new".
In the end destinationList should contain:
1 "1111"
2 "2222"
3 "3333"
Is there some kind of elegant (one line Lambda? ;) solution to this using LINQ ?
Any help will be greatly appreciated! Thanks!
I would just build up a dictionary and use that:
Dictionary<int, string> map = sourceList.ToDictionary(x => x.Id, x => x.Name);
foreach (var item in destinationList)
if (map.ContainsKey(item.Id))
item.Name = map[item.Id];
destinationList.RemoveAll(x=> x.Name == null);
Hope this will your desired result. First join two list based on key(Id) and then set property value from sourceList.
var result = destinationList.Join(sourceList, d => d.Id, s => s.Id, (d, s) =>
{
d.Name = s.Name;
return d;
}).ToList();
Barring the last requirement of "avoid creating new destinationList" this should work
var newList = destinationList.Join(sourceList, d => d.Id, s => s.Id, (d, s) => s);
To take care of "avoid creating new destinationList", below can be used, which is not any different than looping thru whole list, except that it probably is less verbose.
destinationList.ForEach(d => {
var si = sourceList
.Where(s => s.Id == d.Id)
.FirstOrDefault();
d.Name = si != null ? si.Name : "";
});
destinationList.RemoveAll(d => string.IsNullOrEmpty(d.Name));
Frankly, this is the simplest:
var dictionary = sourceList.ToDictionary(x => x.Id, x => x.Name);
foreach(var item in desitnationList) {
if(dictionary.ContainsKey(item.Id)) {
item.Name = dictionary[item.Id];
}
}
destinationList = destinationList.Where(x => x.Name != null).ToList();
You could do something ugly with Join but I wouldn't bother.
I hope this will be useful for you. At the end, destinationList has the correct data, without creating any new list of any kind.
destinationList.ForEach(x =>
{
SourceType newSource = sourceList.Find(s=>s.Id == x.Id);
if (newSource == null)
{
destinationList.Remove(destinationList.Find(d => d.Id == x.Id));
}
else
{
x.Name = newSource.Name;
}
});

Using Function in Select Clause of Entity Framework Query

I would like to implement the following logic against Entity Frameworks.
var items = from item in myContext
select new {
Value1 = TweakValue(item.Value1),
Value2 = TweakValue(item.Value2)
};
protected int TweakValue(int value)
{
// Custom processing here
return value;
}
This won't work because of the call to TweakValue() in the select clause. I understand that the query is converted to SQL, and that the problem is that TweakValue() cannot be converted to SQL. My question is what is the most economical way to implement this. Do I need a second loop to convert the values?
I'm still trying to get comfortable with LINQ expressions.
The simplest way is probably to just "move" the execution to the client to perform the transformation. In this case you'd just use:
var items = myContext.Select(item => new { item.Value1, item.Value2 })
.AsEnumerable()
.Select(item => new {
Value1 = TweakValue(item.Value1),
Value2 = TweakValue(item.Value2)
});
Note that you don't have to reuse the names for Value1 and Value2 - it's just easiest to do so.
If you really want to use query expressions:
var query = from item in myContext
select new { item.Value1, item.Value2 };
var items = from item in query.AsEnumerable()
select new {
Value1 = TweakValue(item.Value1),
Value2 = TweakValue(item.Value2)
};
If you want to perform filtering first, you can get that to occur in the database by putting the filtering, ordering etc before the call to AsEnumerable(). For example:
var query = from item in myContext
where item.Foo == bar
orderby item.Something
select new { item.Value1, item.Value2 };
var items = from item in query.AsEnumerable()
select new {
Value1 = TweakValue(item.Value1),
Value2 = TweakValue(item.Value2)
};
You don't need a loop, just another projection:
var items = myContext.Select(i => new {
Value1 = item.Value1,
Value2 = item.Value2
})
.AsEnumerable()
.Select(i => new {
Value1 = TweakValue(item.Value1),
Value2 = TweakValue(item.Value2)
});
Edit: Depending on what TweakValue actually does, you can push the whole thing to the server. Riffing on your current example:
public Expression<Func<Item, ItemProjection>> TweakValue()
{
return item => new ItemProjection
{
Value1 = item.Value1,
Value2 = item.Value2 + 0 // or something else L2E can understand...
};
}
Now use it like:
var exp = TweakValue();
var items = myContext.Select(exp);
Note I'm storing exp in a variable so that L2E doesn't try to directly invoke TweakValue in the query, which would fail.
Naturally, this only works if TweakValue does stuff that L2E can do.

Categories