We Keep Coding

Expressions are almost identic but one of them is not working

Expressions are not really identic, but should be. They differ in one slight details. I am quite new to Expressions, but I think this could be confusing even for quite experienced player. I refactored the code which processes some data to make Expression used as parameter for IQueryable.Where(). It is functionally equivalent as far as I can see.
I have here former code, which worked well, and generates perfectly functional Expression:
private Expression<Func<T, bool>> StringPropertyContains<T>(string propertyName, string value)
{
if (string.IsNullOrWhiteSpace(propertyName))
{
throw new ArgumentNullException(nameof(propertyName));
}
var param = Expression.Parameter(typeof(T));
MemberExpression member = null;
if (propertyName.Contains('/'))
{
var splittedPropertyName = propertyName.Split('/');
var propertyInfo = this.GetPropertyInfo(typeof(T), splittedPropertyName.First());
member = Expression.MakeMemberAccess(param, propertyInfo);
for (int i = 1; i < splittedPropertyName.Length; i++)
{
if (propertyInfo.PropertyType.IsInterface)
{
//specifically for IActorWithExtraDetails -> reason to refactor
if (typeof(IActor).IsAssignableFrom(propertyInfo.PropertyType) && typeof(IActor).GetProperties().FirstOrDefault(pi => pi.Name.Equals(splittedPropertyName[i], StringComparison.OrdinalIgnoreCase)) != null)
{
propertyInfo = this.GetPropertyInfo(typeof(IActor), splittedPropertyName[i]);
}
else
{
propertyInfo = this.GetPropertyInfo(propertyInfo.PropertyType, splittedPropertyName[i]);
}
}
else
{
propertyInfo = this.GetPropertyInfo(propertyInfo.PropertyType, splittedPropertyName[i]);
}
}
member = Expression.MakeMemberAccess(member, propertyInfo);
}
else
{
var propertyInfo = this.GetPropertyInfo(typeof(T), propertyName);
member = Expression.MakeMemberAccess(param, propertyInfo);
}
var constant = Expression.Constant(value, typeof(string));
var methodInfo = typeof(string).GetMethod("Contains", new Type[] { typeof(string) });
var body = Expression.Call(member, methodInfo, constant);
return Expression.Lambda<Func<T, bool>>(body, param);
}
This is how it looks in DebugView property of the IQueryable:
.Lambda #Lambda2<System.Func`2[AccessManagement.Model.Application,System.Boolean]>(AccessManagement.Model.Application $var1)
{
.Call ($var1.Name).Contains("hive")
}
Here is new, refactored code moved to own method:
private Expression<Func<T, bool>> StringPropertyContains<T>(string propertyName, string value)
{
if (string.IsNullOrWhiteSpace(propertyName))
{
throw new ArgumentNullException(nameof(propertyName));
}
var param = Expression.Parameter(typeof(T));
MemberExpression member = this.GetMemberExpression(typeof(T), propertyName.Trim('/').Split('/'));
var constant = Expression.Constant(value, typeof(string));
var methodInfo = typeof(string).GetMethod("Contains", new Type[] { typeof(string) });
var body = Expression.Call(member, methodInfo, constant);
return Expression.Lambda<Func<T, bool>>(body, param);
}
private MemberExpression GetMemberExpression(Type baseType, string[] path)
{
MemberExpression result = null;
Type type = baseType;
PropertyInfo propertyInfo = null;
foreach (string segment in path)
{
//if type is interface, just spray and pray
if (type.IsInterface)
{
propertyInfo = this.GetDescendantProperties(type)
.FirstOrDefault(pi => pi.Name.Equals(segment, StringComparison.OrdinalIgnoreCase));
}
else
{
propertyInfo = this.GetPropertyInfo(type, segment);
}
if (propertyInfo == null)
{
throw new ArgumentNullException(nameof(propertyInfo));
}
result =
result == null ?
Expression.MakeMemberAccess(Expression.Parameter(baseType), propertyInfo) :
Expression.MakeMemberAccess(result, propertyInfo);
}
return result;
}
This is how expression from refactored method looks like in DebugView:
.Lambda #Lambda2<System.Func`2[AccessManagement.Model.Application,System.Boolean]>(AccessManagement.Model.Application $var1)
{
.Call ($var2.Name).Contains("hive")
}
There is only one difference. As you can see, there is $var2 in second case, not $var1. This variable doesn't exist in whole expression tree. I have no idea why, but I'd bet that is the issue, because everything else remains same. Only other difference is that within second case Expression processing is something cachced in member.RuntimeMethodInfo.base.m_cachedData (debug view path).

In your first code snippet, you're declaring a parameter:
var param = Expression.Parameter(typeof(T));
This is used as the lambda parameter, and is used in the code here:
Expression.MakeMemberAccess(param, propertyInfo);
(this is called twice actually). So the code makes use of the parameter passed to the lambda.
In your second code snippet, you're still using param as the parameter to the lambda, but then you don't use it anywhere in the lambda body.
You're calling this instead:
Expression.MakeMemberAccess(Expression.Parameter(baseType), propertyInfo)
That Expression.Parameter(baseType) creates a second and unrelated variable, which never gets an actual value assigned to it. You should have used the param reference here.
That's where $var2 comes from. $var1 is the param reference.
Consider using the Expression.Parameter(Type, string) overload next time, which lets you name your parameters for debugging purposes. It'll be easier to reason about.

Create a Linq Tree -> Any method Expression

I'm trying to write manually this linq sentence:
IEnumerable<C> classes = this.cs
.Where(c => c.Properties.Any(p =>
p.Key.Equals("key1") &&
p.Value.Equals("v1")
)
);
As you can see I'm calling to Any extension method on a c.Properties property of C class.
Then, inside Any method I've written an And with two Equals.
I need to create this sentence using Expression builders.
Up to now, I've been able to write this:
Type entityType = typeof(T);
PropertyInfo collectionPropertyInfo = entityType.GetProperty("Properties");
if (collectionPropertyInfo == null)
throw new ArgumentException(
string.Format(
"{0} collection doesn't appear in {1}",
"Properties",
entityType.Name
)
);
Type collGenericType = collectionPropertyInfo
.GetType()
.GetGenericArguments()
.FirstOrDefault();
MemberExpression collectionMemberExpression = Expression.Property(
Expression.Parameter(entityType),
collectionPropertyInfo
);
MethodInfo anyMethod = typeof(Enumerable)
.GetMethods()
.Where(m =>
m.Name.Equals("Any") &&
m.GetParameters().Length == 2
)
.Single()
.MakeGenericMethod(collGenericType);
BinaryExpression innerConditionExpression = Expression.AndAlso(
Expression.Equal(
Expression.Property(Expression.Parameter(collGenericType, "p"), "Key"),
Expression.Constant("key1")
),
Expression.Equal(
Expression.Property(Expression.Parameter(collGenericType, "p"), "Value"),
Expression.Constant("v1")
)
);
I don't know how to write Call to Any method where instance is collectionMemberExpression and iiner condition is innerConditionExpression.

How do I build Expression Call for Any Method with generic parameter

I'm just trying make the same expression like below using Linq.Expression:
Expression<Func<Organization, bool>> expression = #org =>
#org.OrganizationFields.Any(a =>
a.CustomField.Name == field.Name &&
values.Contains(a.Value));
In this example above I have an entity called Organization and it has a property called OrganizationsFields as IEnumerable and I want to find any occurrence that match with Any parameter expression.
I just using the code below to generate expression dynamically:
string[] values = filter.GetValuesOrDefault();
ParameterExpression parameter = Expression.Parameter(typeof(T), "org");
Expression organizationFields = Expression.Property(parameter, "OrganizationFields");
MethodInfo any = typeof(Enumerable)
.GetMethods()
.FirstOrDefault(a => a.Name == "Any" && a.GetParameters().Count() == 2)
.MakeGenericMethod(typeof(OrganizationField));
Func<OrganizationField, bool> functionExpression = a =>
a.CustomField.Name == filter.Name && values.Contains(a.Value);
Expression functionParam = Expression.Constant(
functionExpression,
typeof(Func<OrganizationField, bool>));
Expression call = Expression.Call(organizationFields, any, functionParam);
return Expression.Lambda<Func<T, bool>>(call, parameter);
The problems occur when I call the method Expression.Call it throw an ArgumentExeption
Can anyone help me?
Regards

Here you go
var org = Expression.Parameter(typeof(Organization), "org");
Expression<Func<OrganizationField, bool>> predicate =
a => a.CustomField.Name == filter.Name && values.Contains(a.Value);
var body = Expression.Call(typeof(Enumerable), "Any", new[] { typeof(OrganizationField) },
Expression.PropertyOrField(org, "OrganizationFields"), predicate);
var lambda = Expression.Lambda<Func<Organization, bool>>(body, org);
The essential part (covering your post title) is the following useful Expression.Call overload
public static MethodCallExpression Call(
Type type,
string methodName,
Type[] typeArguments,
params Expression[] arguments
)
Also note that the predicate argument of Any must be passed to Expression.Call as Expression<Func<...>>.

How to Create an Expression tree for .Where(x => x.<deep property>.Select(y => y.id).Intersect(List<int>).Any())

I'm creating a method that receives a Queryable<T> source, a string with a property name/path (could be a deep property for example "TrParent.DataTypes" to achieve this x => x.TrParent.DataTypes) and Enumerable<int> which holds the values I need to intersect.
Basically I come from the need to create the following query dynamically (I mean <DT_Det_Tr> and TrParent.DataTypes being know only at runtime, in the example DT_Det_Tr is not a type it is a class):
var _vals = new List<int>();
var res = dbContext.Set<DT_Det_Tr>()
.Where
(x => x.TrParent.DataTypes
.Select(t => t.Id)
.Intersect(_vals)
.Any()
);
Please keep in mind that the preceding query is just an example of what I need to achieve dynamically, what I really need is an expression tree that creates a predicate like the one shown above but using a dynamic type and with the deep navigation property specified within a string.
So, I'm using this function to create the expression for the deep property:
private static LambdaExpression CreateDelegateExpression<T>(out Type resultingtype, string property, string parameterName = "x")
{
var type = typeof(T);
ParameterExpression param = Expression.Parameter(type, parameterName);
Expression expr = param;
foreach (string prop in property.Split('.'))
{
PropertyInfo pi = type.GetProperty(prop);
expr = Expression.Property(expr, pi);
type = pi.PropertyType;
}
Type delegateType = typeof(Func<,>).MakeGenericType(typeof(T), type);
LambdaExpression lambda = Expression.Lambda(delegateType, expr, param);
resultingtype = type;
return lambda;
}
And here is what I have so far for my function:
public static IQueryable<T> Intersect<T>(this IQueryable<T> source, string property, IEnumerable<int> value)
{
//List of ids
var _value = Expression.Constant(value);
//Get delegate expression to the deep property and it's inner type
Type type = null;
var lambda = CreateDelegateExpression<T>(out type, property, "x");
var enumtype = type.GetGenericArguments()[0];
ParameterExpression tpe = Expression.Parameter(enumtype, "y");
Expression propExp = Expression.Property(tpe, enumtype.GetProperty("Id"));
MethodInfo innermethod = typeof(Queryable).GetMethods().Where(x => x.Name == "Select").First();
//Error on next line...
var selectCall = Expression.Call(typeof(Queryable),
"Select",
new Type[] { enumtype, typeof(long) },
lambda,
propExp);
//TODO: Add rest of logic and actually filter the source
return source;
}
In the var selectCall = line I'm getting error:
No generic method 'Select' on type 'System.Linq.Queryable' is compatible with the supplied type arguments and arguments. No type arguments should be provided if the method is non-generic.
I've read a lot here on SO and other sites but I can't get past this part, I feel I'm going to bump into more trouble when I get to the .Intersect(List<int>).Any() part so any help on that also would be grand, thanks.

After a lot of thought, investigation and attempts I came up with a solution.
First, I made a simpler version of my goal query (the static example I used in my question), so instead of:
var res = dbContext.Set<DT_Det_Tr>()
.Where
(x => x.TrParent.DataTypes
.Select(t => t.Id)
.Intersect(_vals)
.Any()
);
I made this:
var res = dbContext.Set<DT_Det_Tr>()
.Where
(x => x.TrParent.DataTypes
.Any(y => _vals.Contains(y.Id))
);
Which is a lot easier to translate to expressions (or at least it was for me) because it omits the Select call.
I got rid of the method I was using to create the deep navigation property expression and streamlined it in my Intersect function, this was because it was doing some work I don't really need here plus I needed access to some of the variables I use inside it, then I made this:
public static IQueryable<T> Intersect<T>(this IQueryable<T> source, string property, IEnumerable<int> value)
{
var type = typeof(T);
var _value = Expression.Constant(value); //List of ids
//Declare parameter for outer lambda
ParameterExpression param = Expression.Parameter(type, "x");
//Outer Lambda
Expression expr = param;
foreach (string prop in property.Split('.')) //Dig for deep property
{
PropertyInfo pi = type.GetProperty(prop);
expr = Expression.Property(expr, pi);
type = pi.PropertyType;
}
//Get deep property's type
var enumtype = type.GetGenericArguments()[0];
//Declare parameter for inner lambda
ParameterExpression tpe = Expression.Parameter(enumtype, "y");
//Inner Collection lambda logic
//Property for inner lambda
Expression propExp = Expression.Property(tpe, enumtype.GetProperty("Id"));
//Contains method call .Contains(y.Id)
var containsMethodExp = Expression.Call(typeof(Enumerable), "Contains", new[] { propExp.Type }, _value, propExp);
//Create Expression<Func<enumtype, bool>>
var innerDelegateType = typeof(Func<,>).MakeGenericType(enumtype, typeof(bool));
//Create Inner lambda y => _vals.Contains(y.Id)
var innerFunction = Expression.Lambda(innerDelegateType, containsMethodExp, tpe);
//Get Any method info
var anyMethod = typeof(Enumerable).GetMethods().Where(m => m.Name == "Any" && m.GetParameters().Length == 2).Single().MakeGenericMethod(enumtype);
//Call Any with inner function .Any(y => _vals.Contains(y.Id))
var outerFunction = Expression.Call(anyMethod, expr, innerFunction);
//Call Where
MethodCallExpression whereCallExpression = Expression.Call
(
typeof(Queryable),
"Where",
new Type[] { source.ElementType },
source.Expression,
Expression.Lambda<Func<T, bool>>(outerFunction, new ParameterExpression[] { param })
);
//Create and return query
return source.Provider.CreateQuery<T>(whereCallExpression);
}
I hope this helps anyone trying to develop a similar solution.
Working with expression trees can be very hard and frustrating at first, but it's a really powerful tool once you get the hold of it.

If you have access to the dynamic keyword from c# 4.0, you might be able to work around the problem like this:
var _vals = new List<int>();
var res = dbContext.Set<DT_Det_Tr>()
.Where(obj => { dynamic x = obj;
return x.TrParent.DataTypes
.Select(t => t.Id)
.Intersect(_vals)
.Any();
}
);
But I don't know enough about the details of the problem you want to solve to say for sure.

LINQ Expression Tree Any() inside Where()

I'm trying to generate the following LINQ query:
//Query the database for all AdAccountAlerts that haven't had notifications sent out
//Then get the entity (AdAccount) the alert pertains to, and find all accounts that
//are subscribing to alerts on that entity.
var x = dataContext.Alerts.Where(a => a.NotificationsSent == null)
.OfType<AdAccountAlert>()
.ToList()
.GroupJoin(dataContext.AlertSubscriptions,
a => new Tuple<int, string>(a.AdAccountId, typeof(AdAccount).Name),
s => new Tuple<int, string>(s.EntityId, s.EntityType),
(Alert, Subscribers) => new Tuple<AdAccountAlert, IEnumerable<AlertSubscription>> (Alert, Subscribers))
.Where(s => s.Item2.Any())
.ToDictionary(kvp => (Alert)kvp.Item1, kvp => kvp.Item2.Select(s => s.Username));
Using Expression Trees (which seems to be the only way I can do this when I need to use reflection and run-time types). Note that in the real code (see below) the AdAccountAlert is actually dynamic through reflection and a for-loop.
My problem: I can generate everything up to the .Where() clause. The whereExpression method call blows up because of incompatible types. Normally I know what to put there, but the Any() method call has me confused. I've tried every type I can think of and no luck. Any help with both the .Where() and .ToDictionary() would be appreciated.
Here's what I have so far:
var alertTypes = AppDomain.CurrentDomain.GetAssemblies()
.Single(a => a.FullName.StartsWith("Alerts.Entities"))
.GetTypes()
.Where(t => typeof(Alert).IsAssignableFrom(t) && !t.IsAbstract && !t.IsInterface);
var alertSubscribers = new Dictionary<Alert, IEnumerable<string>>();
//Using tuples for joins to keep everything strongly-typed
var subscribableType = typeof(Tuple<int, string>);
var doubleTuple = Type.GetType("System.Tuple`2, mscorlib", true);
foreach (var alertType in alertTypes)
{
Type foreignKeyType = GetForeignKeyType(alertType);
if (foreignKeyType == null)
continue;
IQueryable<Alert> unnotifiedAlerts = dataContext.Alerts.Where(a => a.NotificationsSent == null);
//Generates: .OfType<alertType>()
MethodCallExpression alertsOfType = Expression.Call(typeof(Enumerable).GetMethod("OfType").MakeGenericMethod(alertType), unnotifiedAlerts.Expression);
//Generates: .ToList(), which is required for joins on Tuples
MethodCallExpression unnotifiedAlertsList = Expression.Call(typeof(Enumerable).GetMethod("ToList").MakeGenericMethod(alertType), alertsOfType);
//Generates: a => new { a.{EntityId}, EntityType = typeof(AdAccount).Name }
ParameterExpression alertParameter = Expression.Parameter(alertType, "a");
MemberExpression adAccountId = Expression.Property(alertParameter, alertType.GetProperty(alertType.GetForeignKeyId()));
NewExpression outerJoinObject = Expression.New(subscribableType.GetConstructor(new Type[] { typeof(int), typeof(string)}), adAccountId, Expression.Constant(foreignKeyType.Name));
LambdaExpression outerSelector = Expression.Lambda(outerJoinObject, alertParameter);
//Generates: s => new { s.EntityId, s.EntityType }
Type alertSubscriptionType = typeof(AlertSubscription);
ParameterExpression subscriptionParameter = Expression.Parameter(alertSubscriptionType, "s");
MemberExpression entityId = Expression.Property(subscriptionParameter, alertSubscriptionType.GetProperty("EntityId"));
MemberExpression entityType = Expression.Property(subscriptionParameter, alertSubscriptionType.GetProperty("EntityType"));
NewExpression innerJoinObject = Expression.New(subscribableType.GetConstructor(new Type[] { typeof(int), typeof(string) }), entityId, entityType);
LambdaExpression innerSelector = Expression.Lambda(innerJoinObject, subscriptionParameter);
//Generates: (Alert, Subscribers) => new Tuple<Alert, IEnumerable<AlertSubscription>>(Alert, Subscribers)
var joinResultType = doubleTuple.MakeGenericType(new Type[] { alertType, typeof(IEnumerable<AlertSubscription>) });
ParameterExpression alertTupleParameter = Expression.Parameter(alertType, "Alert");
ParameterExpression subscribersTupleParameter = Expression.Parameter(typeof(IEnumerable<AlertSubscription>), "Subscribers");
NewExpression joinResultObject = Expression.New(
joinResultType.GetConstructor(new Type[] { alertType, typeof(IEnumerable<AlertSubscription>) }),
alertTupleParameter,
subscribersTupleParameter);
LambdaExpression resultsSelector = Expression.Lambda(joinResultObject, alertTupleParameter, subscribersTupleParameter);
//Generates:
// .GroupJoin(dataContext.AlertSubscriptions,
// a => new { a.AdAccountId, typeof(AdAccount).Name },
// s => new { s.EntityId, s.EntityType },
// (Alert, Subscribers) => new Tuple<Alert, IEnumerable<AlertSubscription>>(Alert, Subscribers))
IQueryable<AlertSubscription> alertSubscriptions = dataContext.AlertSubscriptions.AsQueryable();
MethodCallExpression joinExpression = Expression.Call(typeof(Enumerable),
"GroupJoin",
new Type[]
{
alertType,
alertSubscriptions.ElementType,
outerSelector.Body.Type,
resultsSelector.ReturnType
},
unnotifiedAlertsList,
alertSubscriptions.Expression,
outerSelector,
innerSelector,
resultsSelector);
//Generates: .Where(s => s.Item2.Any())
ParameterExpression subscribersParameter = Expression.Parameter(resultsSelector.ReturnType, "s");
MemberExpression tupleSubscribers = Expression.Property(subscribersParameter, resultsSelector.ReturnType.GetProperty("Item2"));
MethodCallExpression hasSubscribers = Expression.Call(typeof(Enumerable),
"Any",
new Type[] { alertSubscriptions.ElementType },
tupleSubscribers);
LambdaExpression whereLambda = Expression.Lambda(hasSubscribers, subscriptionParameter);
MethodCallExpression whereExpression = Expression.Call(typeof(Enumerable),
"Where",
new Type[] { joinResultType },
joinExpression,
whereLambda);

Please note: Everything after and including ToList() won't work on IQueryable<T> but on IEnumerable<T>. Because of this, there is no need to create expression trees. It certainly is nothing that is interpreted by EF or similar.
If you would look at the code that is generated by the compiler for your original query, you would see that it generates expression trees only until just before the first call to ToList.
Example:
The following code:
var query = new List<int>().AsQueryable();
query.Where(x => x > 0).ToList().FirstOrDefault(x => x > 10);
Is translated by the compiler to this:
IQueryable<int> query = new List<int>().AsQueryable<int>();
IQueryable<int> arg_4D_0 = query;
ParameterExpression parameterExpression = Expression.Parameter(typeof(int), "x");
arg_4D_0.Where(Expression.Lambda<Func<int, bool>>(Expression.GreaterThan(parameterExpression, Expression.Constant(0, typeof(int))), new ParameterExpression[]
{
parameterExpression
})).ToList<int>().FirstOrDefault((int x) => x > 10);
Please note how it generates expressions for everything up to ToList. Everything after and including it are simply normal calls to extension methods.
If you don't mimick this in your code, you will actually send a call to Enumerable.ToList to the LINQ provider - which it then tries to convert to SQL and fail.

It looks like, when constructing whereLambda, your second parameter should be subscribersParameter and not subscriptionParameter. At least, that would be the cause of your exception.