Related
I've asked a specific question elsewhere, but after no response and some investigating I've got it down to something much more generic, but I'm still struggling to build an expression tree.
I'm using a third party library which does some mappings using an interface and extension methods. Those mappings are specified as an expression tree, what I want to do is build that expression tree up from string values.
The extension method signature:
public static T UpdateGraph<T>(this DbContext context, T entity, Expression<Func<IUpdateConfiguration<T>, object>> mapping = null, bool allowDelete = true) where T : class, new();
The interface IUpdateConfiguration is just a marker interface, but has the following extension methods:
public static class UpdateConfigurationExtensions
{
public static IUpdateConfiguration<T> OwnedCollection<T, T2>(this IUpdateConfiguration<T> config, Expression<Func<T, ICollection<T2>>> expression);
public static IUpdateConfiguration<T> OwnedCollection<T, T2>(this IUpdateConfiguration<T> config, Expression<Func<T, ICollection<T2>>> expression, Expression<Func<IUpdateConfiguration<T2>, object>> mapping);
public static IUpdateConfiguration<T> OwnedEntity<T, T2>(this IUpdateConfiguration<T> config, Expression<Func<T, T2>> expression);
public static IUpdateConfiguration<T> OwnedEntity<T, T2>(this IUpdateConfiguration<T> config, Expression<Func<T, T2>> expression, Expression<Func<IUpdateConfiguration<T2>, object>> mapping);
}
Using an example entity:
public class Person
{
public Car Car {get;set;}
public House House {get;set;}
}
So normal explicit usage is:
dbContext.UpdateGraph(person, mapping => mapping.OwnedEntity(p => p.House).OwnedEntity(p=> p.Car));
What I need to do is build up that mapping from a list of property names,
var props = {"Car","House"}
dbContext.UpdateGraph(person, buildExpressionFromStrings<Person>(props);
I've got so far:
static Expression<Func<IUpdateConfiguration<t>, object>> buildExpressionFromStrings<t>(IEnumerable<string> props)
{
foreach (var s in props)
{
var single = buildExpressionFromString(s);
somehow add this to chaining overall expression
}
}
static Expression<Func<IUpdateConfiguration<t>, object>> buildExpressionFromString<t>(string prop)
{
var ownedChildParam = Expression.Parameter(typeof(t));
var ownedChildExpression = Expression.PropertyOrField(ownedChildParam, prop);
var ownedChildLam = Expression.Lambda(ownedChildExpression, ownedChildParam);
// Up to here I think we've built the (o => o.Car) part of map => map.OwnedEntity(o => o.Car)
// So now we need to build the map=>map.OwnedEntity(ownedChildLam) part, by calling Expression.Call I believe, but here I'm getting confused.
}
In reality, the real-world code is more complex than this (needs to deal with recursion and child properties/mappings), but I think I can get that sorted once I get the expression built for one level. I've been tearing my hair out for over a day, trying to get this sorted... To give some context, I'm using entity framework, and some configuration to define aggregate roots.
Few things to mention. Here
mapping => mapping.OwnedEntity(p => p.House).OwnedEntity(p=> p.Car)
the parts of the lambda expression body are not lambda expressions, but just chained method call expressions, the first using the lambda expression parameter and the next using the previous result.
Second, the extension method are just static method call C# sugar, and in expression trees they must be "called" as static methods.
So, building a call to
public static IUpdateConfiguration<T> OwnedEntity<T, T2>(this IUpdateConfiguration<T> config, Expression<Func<T, T2>> expression);
could be like this
static Expression BuildConfigurationCall<T>(Expression config, string propertyName)
{
var parameter = Expression.Parameter(typeof(T), "it");
var property = Expression.Property(parameter, propertyName);
var selector = Expression.Lambda(property, parameter);
return Expression.Call(
typeof(UpdateConfigurationExtensions),
nameof(UpdateConfigurationExtensions.OwnedEntity),
new [] { typeof(T), property.Type },
config,
selector);
}
and the lambda expression in question would be:
static Expression<Func<IUpdateConfiguration<T>, object>> BuildConfigurationExpression<T>(IEnumerable<string> propertyNames)
{
var parameter = Expression.Parameter(typeof(IUpdateConfiguration<T>), "config");
var body = propertyNames.Aggregate((Expression)parameter,
(config, propertyName) => BuildConfigurationCall<T>(config, propertyName));
return Expression.Lambda<Func<IUpdateConfiguration<T>, object>>(body, parameter);
}
I use Object Builder for dynamic object creating with given props. And I use DynamicQueryable for string based expressions.
I am trying to nest an .Any() inside a .Where() clause to query a local CosmosDb emulator.
The code looks like below; where permittedStundentIds is a variable (List<long>) and a is a Document within the CosmosDb
.Where(a => permittedStudentIds.Any(sId => a.Students.Any(s => s.Id == sId)));
When I execute the query, I get the error:
Method 'Any' is not supported. ActivityId:
800000a8-0002-d600-b63f-84710c7967bb, documentdb-dotnet-sdk/1.22.0
Host/64-bit MicrosoftWindowsNT/10.0.16299.0
I have tried multiple variations to get an equivalent expression, but to no avail. The only one that worked was using a .Contains() and hard coding the student index; which is not feasible since the number of students may not be known.
.Where(a => permittedStudentIds.Contains(a.Students[0].Id));
I do understand that certain lambda extensions are not yet supported on Sql API for CosmosDb, but is there a workaround for this?
After trying out numerous combination of various lambda expressions, here is what worked out for me.
I added a StudentIds property to my DocumentModel class; redundant but used for filtering alone.
Thereafter, I OR-ed the query with .Contains(), something like this:
Expression<Func<MyDocumentModel, bool>> query = a => a.StudentIds.Contains(permittedStudentIds[0]);
foreach (var id in permittedStudentIds.Skip(1))
{
query = query.Or(a => a.StudentIds.Contains(id));
}
and then used the query like:
.Where(query);
For the query.Or() part I used the following classes:
// See: https://blogs.msdn.microsoft.com/meek/2008/05/02/linq-to-entities-combining-predicates/
public static class ExpressionExtensions
{
public static Expression<T> Compose<T>(this Expression<T> first, Expression<T> second, Func<Expression, Expression, Expression> merge)
{
// build parameter map (from parameters of second to parameters of first)
var map = first.Parameters.Select((f, i) => new { f, s = second.Parameters[i] }).ToDictionary(p => p.s, p => p.f);
// replace parameters in the second lambda expression with parameters from the first
var secondBody = ParameterVistor.ReplaceParameters(map, second.Body);
// apply composition of lambda expression bodies to parameters from the first expression
return Expression.Lambda<T>(merge(first.Body, secondBody), first.Parameters);
}
public static Expression<Func<T, bool>> And<T>(this Expression<Func<T, bool>> first, Expression<Func<T, bool>> second)
{
return first.Compose(second, Expression.AndAlso);
}
public static Expression<Func<T, bool>> Or<T>(this Expression<Func<T, bool>> first, Expression<Func<T, bool>> second)
{
return first.Compose(second, Expression.OrElse);
}
}
public class ParameterVistor : ExpressionVisitor
{
private readonly Dictionary<ParameterExpression, ParameterExpression> map;
public ParameterVistor(Dictionary<ParameterExpression, ParameterExpression> map)
{
this.map = map ?? new Dictionary<ParameterExpression, ParameterExpression>();
}
public static Expression ReplaceParameters(Dictionary<ParameterExpression, ParameterExpression> map, Expression exp)
{
return new ParameterVistor(map).Visit(exp);
}
protected override Expression VisitParameter(ParameterExpression p)
{
ParameterExpression replacement;
if (map.TryGetValue(p, out replacement))
{
p = replacement;
}
return base.VisitParameter(p);
}
}
So you have a sequence of permittedStudentIds and a Document with a sequence of Students. Every Student has an Id.
You want to know whether there are any permittedStudentsId that is also an Id of one (or more) of your Document's Students.
In other words, if permittedStudentIds has values 1, 2, you want to know if there is any Student in Document.Students with Id 1 or 2.
Why not extract the Ids of all Students, Intersect them with your permittedStudentIds and see if the result is empty or not?
var studentIds = Document.Students.Select(student => student.Id);
var intersection = studentIds.Intersect(permittedStudentIds);
var result = intersection.Any();
// TODO: make one statement.
This works if both sequences are AsQueryable, but it should also work if your Document.Students is an IQueryable and your permittedStudentIds is an IEnumerable. My best guess is that this will become an SQL contains. See Queryable.Intersect
I'm trying to achive this conversion
"Address.Street" => (p) => p.Address.Street
"Name" => (p) => p.Name
I was able to find a method to generate an order by expression using reflection but it won't work for complex sort as Address.Street since works for a single property level.
Is there a way to do this? I've seen that I compile lambda expressions but I couldn't understand how to make it work for this case.
Creating an expression is not hard, but the tricky part is how to bind it to the corresponding OrderBy(Descending) / ThenBy(Descendig) methods when you don't know the type of the property (hence the type of the selector expression result).
Here is all that encapsulated in a custom extension method:
public static partial class QueryableExtensions
{
public static IOrderedQueryable<T> OrderByMember<T>(this IQueryable<T> source, string memberPath)
{
return source.OrderByMemberUsing(memberPath, "OrderBy");
}
public static IOrderedQueryable<T> OrderByMemberDescending<T>(this IQueryable<T> source, string memberPath)
{
return source.OrderByMemberUsing(memberPath, "OrderByDescending");
}
public static IOrderedQueryable<T> ThenByMember<T>(this IOrderedQueryable<T> source, string memberPath)
{
return source.OrderByMemberUsing(memberPath, "ThenBy");
}
public static IOrderedQueryable<T> ThenByMemberDescending<T>(this IOrderedQueryable<T> source, string memberPath)
{
return source.OrderByMemberUsing(memberPath, "ThenByDescending");
}
private static IOrderedQueryable<T> OrderByMemberUsing<T>(this IQueryable<T> source, string memberPath, string method)
{
var parameter = Expression.Parameter(typeof(T), "item");
var member = memberPath.Split('.')
.Aggregate((Expression)parameter, Expression.PropertyOrField);
var keySelector = Expression.Lambda(member, parameter);
var methodCall = Expression.Call(
typeof(Queryable), method, new[] { parameter.Type, member.Type },
source.Expression, Expression.Quote(keySelector));
return (IOrderedQueryable<T>)source.Provider.CreateQuery(methodCall);
}
I need help on how to show up my 2nd extension method "OrderBy" (shown below) in my ASP.Net MVC Controller.
The 1st extension method "Where" is showing up but not "OrderBy". What do i need to do to make it show up? Or maybe my code is wrong in the 2nd extension method?
NOTE: I have already imported the namespace in my controller by adding:
using MyApplication.Models;
Here's my code for the extension methods:
namespace MyApplication.Models
{
public static class ExtensionMethods
{
public static IQueryable<T> Where<T>(this IQueryable<T> source, string columnName, string value, string filterType)
{
ParameterExpression table = Expression.Parameter(typeof(T), "x");
MemberExpression column = Expression.PropertyOrField(table, columnName);
Expression valueExpression = null;
Expression where = null;
if (column.Type.FullName.Contains("String")) {...}
if (column.Type.FullName.Contains("Int32")) {...}
if (column.Type.FullName.Contains("DateTime")){...}
var predicate = Expression.Lambda<Func<T, bool>>(where, table);
return source.Where(predicate);
}
public static IOrderedQueryable<T> OrderBy<T,TKey>(this IQueryable<T> source, string columnName)
{
ParameterExpression table = Expression.Parameter(typeof(T), "x");
Expression column = Expression.PropertyOrField(table, columnName);
var keySelector = Expression.Lambda<Func<T, TKey>>(column,table);
return source.OrderBy(keySelector);
}
}
}
Here's the code in the Controller:
using MyApplication.Models;
...
using (MyContext context = new MyContext())
{
IQueryable<Shipper> query = context.Shippers;
query = query.Where(property,value,filterType);
query = query.OrderBy(property);
}
Any help is greatly appreciated.
-Mark
EDIT:
Here's my new OrderBy extension method:
public static IOrderedQueryable<T> OrderBy<T>(this IQueryable<T> source, string columnName, bool asc)
{
var entityType = typeof(T);
var property = entityType.GetProperty(columnName);
ParameterExpression table = Expression.Parameter(entityType, "x");
Expression column = Expression.PropertyOrField(table, columnName);
string sortMethod="";
if (asc) { sortMethod = "OrderBy"; }
else { sortMethod = "OrderByDescending"; }
var keySelector = Expression.Lambda(column,table);
MethodCallExpression resultExp = Expression.Call(
typeof(Queryable),
sortMethod,
new Type[] { entityType, property.PropertyType },
source.Expression,
Expression.Quote(keySelector));
return (IOrderedQueryable<T>)source.Provider.CreateQuery<T>(resultExp);
}
The problem is that your extension method has two type parameters, but only one of them can be used in type inference by the compiler - TKey isn't mentioned at all in the normal parameter list, which is what's used in type inference.
I suspect this would find your extension method:
// Of course I don't know that it's meant to be string. You should use whatever
// type is appropriate.
query = query.OrderBy<Shipper, string>(property);
That may not be ideal in terms of usage, but at least it steers you in the right direction in terms of why it wasn't working. If you want to rely on type inference, you'll need to get rid of TKey as a type parameter, instead figuring that out - and then performing the rest of your logic - with reflection.
This question already has answers here:
Dynamic LINQ OrderBy on IEnumerable<T> / IQueryable<T>
(24 answers)
Closed 1 year ago.
What's the simplest way to code against a property in C# when I have the property name as a string? For example, I want to allow the user to order some search results by a property of their choice (using LINQ). They will choose the "order by" property in the UI - as a string value of course. Is there a way to use that string directly as a property of the linq query, without having to use conditional logic (if/else, switch) to map the strings to properties. Reflection?
Logically, this is what I'd like to do:
query = query.OrderBy(x => x."ProductId");
Update:
I did not originally specify that I'm using Linq to Entities - it appears that reflection (at least the GetProperty, GetValue approach) does not translate to L2E.
I would offer this alternative to what everyone else has posted.
System.Reflection.PropertyInfo prop = typeof(YourType).GetProperty("PropertyName");
query = query.OrderBy(x => prop.GetValue(x, null));
This avoids repeated calls to the reflection API for obtaining the property. Now the only repeated call is obtaining the value.
However
I would advocate using a PropertyDescriptor instead, as this will allow for custom TypeDescriptors to be assigned to your type, making it possible to have lightweight operations for retrieving properties and values. In the absence of a custom descriptor it will fall back to reflection anyhow.
PropertyDescriptor prop = TypeDescriptor.GetProperties(typeof(YourType)).Find("PropertyName");
query = query.OrderBy(x => prop.GetValue(x));
As for speeding it up, check out Marc Gravel's HyperDescriptor project on CodeProject. I've used this with great success; it's a life saver for high-performance data binding and dynamic property operations on business objects.
I'm a little late to the party, however, I hope this can be of some help.
The problem with using reflection is that the resulting Expression Tree will almost certainly not be supported by any Linq providers other than the internal .Net provider. This is fine for internal collections, however this will not work where the sorting is to be done at source (be that SQL, MongoDb, etc.) prior to pagination.
The code sample below provides IQueryable extention methods for OrderBy and OrderByDescending, and can be used like so:
query = query.OrderBy("ProductId");
Extension Method:
public static class IQueryableExtensions
{
public static IOrderedQueryable<T> OrderBy<T>(this IQueryable<T> source, string propertyName)
{
return source.OrderBy(ToLambda<T>(propertyName));
}
public static IOrderedQueryable<T> OrderByDescending<T>(this IQueryable<T> source, string propertyName)
{
return source.OrderByDescending(ToLambda<T>(propertyName));
}
private static Expression<Func<T, object>> ToLambda<T>(string propertyName)
{
var parameter = Expression.Parameter(typeof(T));
var property = Expression.Property(parameter, propertyName);
var propAsObject = Expression.Convert(property, typeof(object));
return Expression.Lambda<Func<T, object>>(propAsObject, parameter);
}
}
Regards, Mark.
I liked the answer from #Mark Powell, but as #ShuberFu said, it gives the error LINQ to Entities only supports casting EDM primitive or enumeration types.
Removing var propAsObject = Expression.Convert(property, typeof(object)); didn't work with properties that were value types, such as integer, as it wouldn't implicitly box the int to object.
Using Ideas from Kristofer Andersson and Marc Gravell I found a way to construct the Queryable function using the property name and have it still work with Entity Framework. I also included an optional IComparer parameter. Caution: The IComparer parameter does not work with Entity Framework and should be left out if using Linq to Sql.
The following works with Entity Framework and Linq to Sql:
query = query.OrderBy("ProductId");
And #Simon Scheurer this also works:
query = query.OrderBy("ProductCategory.CategoryId");
And if you are not using Entity Framework or Linq to Sql, this works:
query = query.OrderBy("ProductCategory", comparer);
Here is the code:
public static class IQueryableExtensions
{
public static IOrderedQueryable<T> OrderBy<T>(this IQueryable<T> query, string propertyName, IComparer<object> comparer = null)
{
return CallOrderedQueryable(query, "OrderBy", propertyName, comparer);
}
public static IOrderedQueryable<T> OrderByDescending<T>(this IQueryable<T> query, string propertyName, IComparer<object> comparer = null)
{
return CallOrderedQueryable(query, "OrderByDescending", propertyName, comparer);
}
public static IOrderedQueryable<T> ThenBy<T>(this IOrderedQueryable<T> query, string propertyName, IComparer<object> comparer = null)
{
return CallOrderedQueryable(query, "ThenBy", propertyName, comparer);
}
public static IOrderedQueryable<T> ThenByDescending<T>(this IOrderedQueryable<T> query, string propertyName, IComparer<object> comparer = null)
{
return CallOrderedQueryable(query, "ThenByDescending", propertyName, comparer);
}
/// <summary>
/// Builds the Queryable functions using a TSource property name.
/// </summary>
public static IOrderedQueryable<T> CallOrderedQueryable<T>(this IQueryable<T> query, string methodName, string propertyName,
IComparer<object> comparer = null)
{
var param = Expression.Parameter(typeof(T), "x");
var body = propertyName.Split('.').Aggregate<string, Expression>(param, Expression.PropertyOrField);
return comparer != null
? (IOrderedQueryable<T>)query.Provider.CreateQuery(
Expression.Call(
typeof(Queryable),
methodName,
new[] { typeof(T), body.Type },
query.Expression,
Expression.Lambda(body, param),
Expression.Constant(comparer)
)
)
: (IOrderedQueryable<T>)query.Provider.CreateQuery(
Expression.Call(
typeof(Queryable),
methodName,
new[] { typeof(T), body.Type },
query.Expression,
Expression.Lambda(body, param)
)
);
}
}
Yes, I don't think there's another way than Reflection.
Example:
query = query.OrderBy(x => x.GetType().GetProperty("ProductId").GetValue(x, null));
query = query.OrderBy(x => x.GetType().GetProperty("ProductId").GetValue(x, null));
Trying to recall exact syntax off the top of my head but I think that is correct.
Warning ⚠️
You just can use Reflection in case that data is in-memory. Otherwise, you will see some error like below when you work with Linq-2-EF, Linq-2-SQL, etc.
#Florin Vîrdol's comment
LINQ to Entities does not recognize the method 'System.Object
GetValue(System.Object)' method and this method cannot be translated
into a store expression.
Why 🤔
Because when you write code to provide a query to Linq query provider. It is first translated into an SQL statement and then executed on the database server.
(See image below, from https://www.tutorialsteacher.com/linq/linq-expression)
Solution ✅
By using Expression tree, you can write a generic method like this
public static IEnumerable<T> OrderDynamic<T>(IEnumerable<T> Data, string propToOrder)
{
var param = Expression.Parameter(typeof(T));
var memberAccess = Expression.Property(param, propToOrder);
var convertedMemberAccess = Expression.Convert(memberAccess, typeof(object));
var orderPredicate = Expression.Lambda<Func<T, object>>(convertedMemberAccess, param);
return Data.AsQueryable().OrderBy(orderPredicate).ToArray();
}
And use it like this
var result = OrderDynamic<Student>(yourQuery, "StudentName"); // string property
or
var result = OrderDynamic<Student>(yourQuery, "Age"); // int property
And it's also working with in-memory by converting your data into IQueryable<TElement> in your generic method return statement like this
return Data.AsQueryable().OrderBy(orderPredicate).ToArray();
See the image below to know more in-depth.
Demo on dotnetfiddle
More productive than reflection extension to dynamic order items:
public static class DynamicExtentions
{
public static object GetPropertyDynamic<Tobj>(this Tobj self, string propertyName) where Tobj : class
{
var param = Expression.Parameter(typeof(Tobj), "value");
var getter = Expression.Property(param, propertyName);
var boxer = Expression.TypeAs(getter, typeof(object));
var getPropValue = Expression.Lambda<Func<Tobj, object>>(boxer, param).Compile();
return getPropValue(self);
}
}
Example:
var ordered = items.OrderBy(x => x.GetPropertyDynamic("ProductId"));
Also you may need to cache complied lambas(e.g. in Dictionary<>)
Reflection is the answer!
typeof(YourType).GetProperty("ProductId").GetValue(theInstance);
There's lots of things you can do to cache the reflected PropertyInfo, check for bad strings, write your query comparison function, etc., but at its heart, this is what you do.
Also Dynamic Expressions can solve this problem.
You can use string-based queries through LINQ expressions that could have been dynamically constructed at run-time.
var query = query
.Where("Category.CategoryName == #0 and Orders.Count >= #1", "Book", 10)
.OrderBy("ProductId")
.Select("new(ProductName as Name, Price)");
I think we can use a powerful tool name Expression an in this case use it as an extension method as follows:
public static IOrderedQueryable<T> OrderBy<T>(this IQueryable<T> source, string ordering, bool descending)
{
var type = typeof(T);
var property = type.GetProperty(ordering);
var parameter = Expression.Parameter(type, "p");
var propertyAccess = Expression.MakeMemberAccess(parameter, property);
var orderByExp = Expression.Lambda(propertyAccess, parameter);
MethodCallExpression resultExp =
Expression.Call(typeof(Queryable), (descending ? "OrderByDescending" : "OrderBy"),
new Type[] { type, property.PropertyType }, source.Expression, Expression.Quote(orderByExp));
return (IOrderedQueryable<T>)source.Provider.CreateQuery<T>(resultExp);
}