We Keep Coding

Dynamically construct where clause with Func<T, string> lambda - linq to entities

(this is for .Net Framework 4.7)
I'm trying to write up some extension methods to aid in creating dynamic where clauses for various entities. I started a few days ago, so there's likely a lot that I don't know and some that I probably misunderstood.
I managed to create one extension method already for filtering by 1 property which works as I expect it to (I did use reflection to get the property, couldn't get it working with an interface - well, without it executing the sql that is).
I can't seem to be able to get this one working for a lambda expression though.
Note, that the solution must not trigger sql execution. Because I was able to write up some variants that "worK', but they will trigger sql execution.
The way I work with this is that once I have the code ready, I start debugging and have the "query" in the watch. And it looks like this (notice the sql code)
Once I step over my FilterString method call, it either turns into a sql result, or I get an exception (with current code), which it shouldn't:
So here's my current code that throws the exception (currently not dealing with the "match" parameter, I am implementing an "equals" call. There will be others like, starts With, like, etc)
The exception is just one of those "type mismatch" having function cannot be passed as param to string Equals or what not.
public static IQueryable<T> FilterString<T>(this IQueryable<T> query, Match match,
string criteriaItem, Expression<Func<T, string>> getItemString)
where T : class
{
if (string.IsNullOrEmpty(criteriaItem))
{
return query;
}
var param = Expression.Parameter(typeof(T), "r");
var selector = Expression.Lambda<Func<T, string>>(getItemString, param);
Expression<Func<string, bool>> prototype = item => item == criteriaItem;
var predicate = Expression.Lambda<Func<T, bool>>(
prototype.Body.ReplaceParameter(prototype.Parameters[0], selector.Body),
selector.Parameters[0]);
return query.Where(predicate);
}
and the one that executes the sql instead of just generating it
public static IQueryable<T> FilterString<T>(this IQueryable<T> query, Match match,
string criteriaItem, Expression<Func<T, string>> getItemString)
where T : class
{
if (string.IsNullOrEmpty(criteriaItem))
{
return query;
}
var param = Expression.Parameter(typeof(T), "r");
//var value = Expression.Constant(getItemString);
var equals = typeof(string).GetMethod("Equals", new Type[] { typeof(string) });
var item = Expression.Invoke(getItemString, param);
var body = Expression.Call(Expression.Constant(criteriaItem),
equals,
item);
return query.Where(Expression.Lambda<Func<T, bool>>(body, param));
}
calling these is done like so
query = query.FilterString(match, criteria_value, (r) => r.SomeProperty.MaybeSomeOtherProp.SomeString);
query = query.FilterString(match, criteria_value, (r) => r.SomeProperty.Name);
This same extension method will be called on any number of different entities, with nay number of different properties and prop names. I guess I could make use of the reflection version I got working and passing in all the property names in some array of some sort, but that is just plain ugly.
So long story short, how can I get this working in the way I explained above, taht is: having the sql generated instead of executed?
Thank you,
Note, the "ReplaceParameter" extension method is the one from here: https://stackoverflow.com/a/39206392/630515

So, you're trying to merge your prototype item => item == criteriaItem. With a passed in string property expression, like (r) => r.SomeProperty.Name to create (r) => r.SomeProperty.Name == criteriaItem.
Expression<Func<string, bool>> prototype = item => item == criteriaItem;
var predicate = Expression.Lambda<Func<T, bool>>(
ReplacingExpressionVisitor.Replace(
prototype.Parameters[0],
getItemString.Body,
prototype.Body),
getItemString.Parameters[0]);
And I think you're trying to do it this way so that criteriaItem is bound to an sql parameter, rather than being inlined as a string constant. But your question was a little hard to follow.

Convert Method to Linq Expression for query

In our application we want to have standard methods for various conditions in our database. For instance, we have different types of transactions, and we want to create standard methods for retrieving them within other queries. However, this gives us the error:
Method '' has no supported translation to SQL
The method might look like this:
public static bool IsDividend(this TransactionLog tl)
{
return tl.SourceTypeID == (int)JobType.Dividend || tl.SourceTypeID == (int)JobType.DividendAcct;
}
To be used as such:
var dividends = ctx.TransactionLogs.Where(x => x.IsDividend());
Of course, if I copy the logic from IsDividend() into the Where clause, this works fine, but I end up duplicating this logic many places and is hard to track down if that logic changes.
I think if I would convert this to an expression like this it would work, but this is not as preferable a setup as being able to use methods:
public Expression<Func<TransactionLog, bool>> IsDividend = tl => tl.SourceTypeID == (int)JobType.Dividend || tl.SourceTypeID == (int)JobType.DividendAcct;
var dividends = ctx.TransactionLogs.Where(IsDividend);
Is there a way to force Linq to evaluate the method as an expression? Or to "transform" the method call into an expression within a linq query? Something like this:
var dividends = ctx.TransactionLogs.Where(tl => ToExpression(tl.IsDividend));
We are using Linq-to-SQL in our application.

Well having static property containing the expressions seems fine to me.
The only way to make it work with Methods would be to create a method which returns this expression, and then call it inside where:
public class TransactionLog
{
Expression<Func<TransactionLog, bool>> IsDividend() {
Expression<Func<TransactionLog, bool>> expression = tl => tl.SourceTypeID == (int)JobType.Dividend || tl.SourceTypeID == (int)JobType.DividendAcct;
return expression;
}
}
public class TransactionLogExtension
{
Expression<Func<TransactionLog, bool>> IsDividend(this TransactionLog log) {
Expression<Func<TransactionLog, bool>> expression = tl => tl.SourceTypeID == (int)JobType.Dividend || tl.SourceTypeID == (int)JobType.DividendAcct;
return expression;
}
}
and use it via
var dividends = ctx.TransactionLogs.Where(TransactionLog.IsDividend());
or as extension method
var dividends = ctx.TransactionLogs.Where(x.IsDividend());
But none of it is will work with var dividends = ctx.TransactionLogs.Where(x => x.IsDividend()); because x => x.IsDividend(); itself is an expression tree and your database provider can't translate "IsDividend" into an SQL statement.
But the other two options will at least allow you to pass in parameters (which doesn't work if you store the Expressions as instance or static properties).

I think that LINQ to SQL doesn't fully supports even common and build-in functions. (At least EF does not do it). And moreover - when it deals with user defined methods. I predict that your variant with expression will fall as well as the variant with method call unless you call it after enumeration (ToList or similar method). I suggest to keep the balance between 1) stored procedures at server, 2) common conditions hardcoded in Where clauses in C#, 3) expression trees generation in C#. All these points are relatively complex, for sure (and to my mind there is no easy and general solution).

Try using Extension Methods, like so:
public static class TransactionLogExtensions {
public static IQueryable<TransactionLog> OnlyDividends(this IQueryable<TransactionLog> tl)
{
return tl.Where(t=>t.SourceTypeID == (int)JobType.Dividend || t.SourceTypeID == (int)JobType.DividendAcct);
}
}
Call it like so:
var dividends=db.TransactionLogs.OnlyDividends();
or
var dividends=db.TransactionLogs.OnlyDividends().OrderBy(...);

For this scenario you can use Func. Linq works very good with those.
Here is the simple example of using Func in Linq query. Feel free to modify and use.
Func<int,bool> isDivident = x => 3==x;
int[] values = { 3, 7, 10 };
var result = values.Select (isDivident );

Converting Lambda Functions into Lambda Expression

I am trying to combine a multiple selection with a lambda function into an lambda expression. How do I do that? I know the last line is wrong, but giving you an idea of what I mean.
Func<Event, bool> where = null;
if (!string.IsNullOrWhiteSpace(searchToken))
where = q => q.Name.ToUpper().Contains(searchToken.ToUpper());
where += q => q.Hidden = false;
Expression<Func<Event, bool>> where1 = q => where; <-- Erroring

I suspect you want PredicateBuilder. (The source is available on that page.) You'd use it like this:
var predicate = q => !q.Hidden;
if (!string.IsNullOrWhiteSpace(searchToken))
{
predicate = predicate.And(q => q.Name.ToUpper()
.Contains(searchToken.ToUpper());
}
return predicate;
That's assuming you want to "and" the conditions - you never made that clear...
Note that that is not a good way to compare in a case-insensitive way, either. If you could tell us what's going to consume the query (e.g. LINQ to SQL, LINQ to EF) we could suggest a provider-compatible way of performing a case-insensitive query.

Look at http://msdn.microsoft.com/en-us/library/bb882637.aspx. How to use expression trees to build dynamic queries.
AFAIK when using Expression <> like that the expression must be known in compile time, because the compiler then build AST abstract syntax three and stores it as data in your Expression <> instance.

How to insert a lambda into a Linq declarative query expression

Let's say you have the following code:
string encoded="9,8,5,4,9";
// Parse the encoded string into a collection of numbers
var nums=from string s in encoded.Split(',')
select int.Parse(s);
That's easy, but what if I want to apply a lambda expression to s in the select, but still keep this as a declarative query expression, in other words:
string encoded="9,8,5,4,9";
// Parse the encoded string into a collection of numbers
var nums=from string s in encoded.Split(',')
select (s => {/* do something more complex with s and return an int */});
This of course does not compile. But, how can I get a lambda in there without switching this to fluent syntax.
Update: Thanks to guidance from StriplingWarrior, I have a convoluted but compilable solution:
var result=from string s in test.Split(',')
select ((Func<int>)
(() => {string u="1"+s+"2"; return int.Parse(u);}))();
The key is in the cast to a Func<string,int> followed by evaluation of the lambda for each iteration of the select with (s). Can anyone come up with anything simpler (i.e., without the cast to Func followed by its evaluation or perhaps something less verbose that achieves the same end result while maintaining the query expression syntax)?
Note: The lambda content above is trivial and exemplary in nature. Please don't change it.
Update 2: Yes, it's me, crazy Mike, back with an alternate (prettier?) solution to this:
public static class Lambda
{
public static U Wrap<U>(Func<U> f)
{
return f();
}
}
...
// Then in some function, in some class, in a galaxy far far away:
// Look what we can do with no casts
var res=from string s in test.Split(',')
select Lambda.Wrap(() => {string u="1"+s+"2"; return int.Parse(u);});
I think this solves the problem without the ugly cast and parenarrhea. Is something like the Lambda.Wrap generic method already present somewhere in the .NET 4.0 Framework, so that I do not have to reinvent the wheel? Not to overburden this discussion, I have moved this point into its own question: Does this "Wrap" generic method exist in .NET 4.0.

Assuming you're using LINQ to Objects, you could just use a helper method:
select DoSomethingComplex(s)
If you don't like methods, you could use a Func:
Func<string, string> f = s => { Console.WriteLine(s); return s; };
var q = from string s in new[]{"1","2"}
select f(s);
Or if you're completely hell-bent on putting it inline, you could do something like this:
from string s in new[]{"1","2"}
select ((Func<string>)(() => { Console.WriteLine(s); return s; }))()

You could simply do:
var nums = from string s in encoded.Split(',')
select (s => { DoSomething(); return aValueBasedOnS; });
The return tells the compiler the type of the resulting collection.

How about this:
var nums= (from string s in encoded.Split(',') select s).Select( W => ...);

Can anyone come up with anything
simpler?
Yes. First, you could rewrite it like this
var result = from s in encoded.Split(',')
select ((Func<int>)(() => int.Parse("1" + s + "2")))();
However, that's not really readable, particularly for a query expression. For this particular query and projection, the let keyword could be used.
var result = from s in encoded.Split(',')
let t = "1" + s + "2"
select int.Parse(t);

IEnumerable integers = encoded.Split(',').Select(s => int.Parse(s));
Edit:
IEnumerable<int> integers = from s in encoded.Split(',') select int.Parse(string.Format("1{0}2",s));

LINQ to entities - Building where clauses to test collections within a many to many relationship

So, I am using the Linq entity framework. I have 2 entities: Content and Tag. They are in a many-to-many relationship with one another. Content can have many Tags and Tag can have many Contents. So I am trying to write a query to select all contents where any tags names are equal to blah
The entities both have a collection of the other entity as a property(but no IDs). This is where I am struggling. I do have a custom expression for Contains (so, whoever may help me, you can assume that I can do a "contains" for a collection). I got this expression from: http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=2670710&SiteID=1
Edit 1
I ended up finding my own answer.

After reading about the PredicateBuilder, reading all of the wonderful posts that people sent to me, posting on other sites, and then reading more on Combining Predicates and Canonical Function Mapping.. oh and I picked up a bit from Calling functions in LINQ queries (some of these classes were taken from these pages).
I FINALLY have a solution!!! Though there is a piece that is a bit hacked...
Let's get the hacked piece over with :(
I had to use reflector and copy the ExpressionVisitor class that is marked as internal. I then had to make some minor changes to it, to get it to work. I had to create two exceptions (because it was newing internal exceptions. I also had to change the ReadOnlyCollection() method's return from:
return sequence.ToReadOnlyCollection<Expression>();
To:
return sequence.AsReadOnly();
I would post the class, but it is quite large and I don't want to clutter this post any more than it's already going to be. I hope that in the future that class can be removed from my library and that Microsoft will make it public. Moving on...
I added a ParameterRebinder class:
public class ParameterRebinder : ExpressionVisitor {
private readonly Dictionary<ParameterExpression, ParameterExpression> map;
public ParameterRebinder(Dictionary<ParameterExpression, ParameterExpression> map) {
this.map = map ?? new Dictionary<ParameterExpression, ParameterExpression>();
}
public static Expression ReplaceParameters(Dictionary<ParameterExpression, ParameterExpression> map, Expression exp) {
return new ParameterRebinder(map).Visit(exp);
}
internal override Expression VisitParameter(ParameterExpression p) {
ParameterExpression replacement;
if (map.TryGetValue(p, out replacement)) {
p = replacement;
}
return base.VisitParameter(p);
}
}
Then I added a ExpressionExtensions class:
public static class ExpressionExtensions {
public static Expression<T> Compose<T>(this Expression<T> first, Expression<T> second, Func<Expression, Expression, Expression> merge) {
// build parameter map (from parameters of second to parameters of first)
var map = first.Parameters.Select((f, i) => new { f, s = second.Parameters[i] }).ToDictionary(p => p.s, p => p.f);
// replace parameters in the second lambda expression with parameters from the first
var secondBody = ParameterRebinder.ReplaceParameters(map, second.Body);
// apply composition of lambda expression bodies to parameters from the first expression
return Expression.Lambda<T>(merge(first.Body, secondBody), first.Parameters);
}
public static Expression<Func<T, bool>> And<T>(this Expression<Func<T, bool>> first, Expression<Func<T, bool>> second) {
return first.Compose(second, Expression.And);
}
public static Expression<Func<T, bool>> Or<T>(this Expression<Func<T, bool>> first, Expression<Func<T, bool>> second) {
return first.Compose(second, Expression.Or);
}
}
And the last class I added was PredicateBuilder:
public static class PredicateBuilder {
public static Expression<Func<T, bool>> True<T>() { return f => true; }
public static Expression<Func<T, bool>> False<T>() { return f => false; }
}
This is my result... I was able to execute this code and get back the resulting "content" entities that have matching "tag" entities from the tags that I was searching for!
public static IList<Content> GetAllContentByTags(IList<Tag> tags) {
IQueryable<Content> contentQuery = ...
Expression<Func<Content, bool>> predicate = PredicateBuilder.False<Content>();
foreach (Tag individualTag in tags) {
Tag tagParameter = individualTag;
predicate = predicate.Or(p => p.Tags.Any(tag => tag.Name.Equals(tagParameter.Name)));
}
IQueryable<Content> resultExpressions = contentQuery.Where(predicate);
return resultExpressions.ToList();
}
Please let me know if anyone needs help with this same thing, if you would like me to send you files for this, or just need more info.

Summing it up...
contentQuery.Where(
content => content.Tags.Any(tag => tags.Any(t => t.Name == tag.Name))
);
So is that what you're expecting?
I'm a little confused.

This is what the question itself asks for:
contentQuery.Where(
content => content.Tags.Any(tag => tag.Name == "blah")
);
I'm not sure what the thought process was to get to the questioner's code, really, and I'm not entirely sure exactly what its really doing. The one thing I'm really sure of is that .AsQueryable() call is completely unnecessary -- either .Tags is already an IQueryable, or the .AsQueryable() is just going to fake it for you -- adding extra calls in where there doesn't need to be any.

The error is related to the 'tags' variable. LINQ to Entities does not support a parameter that is a collection of values. Simply calling tags.AsQueryable() -- as suggested in an ealier answer -- will not work either because the default in-memory LINQ query provider is not compatible with LINQ to Entities (or other relational providers).
As a workaround, you can manually build up the filter using the expression API (see this forum post) and apply it as follows:
var filter = BuildContainsExpression<Element, string>(e => e.Name, tags.Select(t => t.Name));
var query = source.Where(e => e.NestedValues.Any(filter));

tags.Select(testTag => testTag.Name)
Where does the tags variable gets initialized from? What is it?

NOTE: please edit the question itself, rather than replying with an answer -- this is not a discussion thread, and they can re-order themselves at any time
If you're searching for all Contents that are marked with any one of a set of tags:
IEnumerable<Tag> otherTags;
...
var query = from content in contentQuery
where content.Tags.Intersection(otherTags).Any()
select content;
It looks like you might be using LINQ To SQL, in which case it might be better if you write a stored procedure to do this one: using LINQ to do this will probably not run on SQL Server -- it's very likely it will try to pull down everything from contentQuery and fetch all the .Tags collections. I'd have to actually set up a server to check that, though.