How to get single column with anonymous method using linq expression. Here's my code and it doesn't work:
public IEnumerable<object> GetPropertyValues<T>(string propName) where T : class
{
return base.Query<T>().AsEnumerable()
.Where(x => x.GetType().GetProperty(propName).Name == propName)
.Select(x => x.GetType().GetProperty(propName).GetValue(x, null));
}
Here's the code in non generic method:
base.Query<Product>().Select(x => x.ProductName).AsEnumerable();
Thanks in advance.
This condition is incorrect, because when the property propName is missing, it crashes, rather than returning false:
.Where(x => x.GetType().GetProperty(propName).Name == propName)
If you wanted to say "the dynamic type has the property propName", a proper condition for it would look like this:
.Where(x => x.GetType().GetProperty(propName) != null)
Note that this is necessary only when some, but not all, subclasses of T have the desired property propName. If the property is present in the T itself, you could get property upfront, and do the rest of the query like this:
var theProp = typeof(T)..GetProperty(propName);
return base.Query<T>().AsEnumerable().Select(x => theProp.GetValue(x, null));
Once you call AsEnumerable() all further filtering will happen in memory rather than SQL. Thus, you could do:
var prop = typeof(T).GetProperty(propName);
return base.Query<T>().AsEnumerable().Select(t => prop.GetValue(t));
but this will select all columns of T into memory before filtering down to just one. To do the filtering in SQL, you'll need to dynamically construct an expression:
var prop = typeof(T).GetProperty(propName);
var parameter = Expression.Parameter(typeof(T));
var memberAccess = Expression.MakeMemberAccess(parameter, prop);
var selector = Expression.Lambda<Func<T, TProperty>>(memberAccess, parameter);
return base.Query<T>().Select(selector);
For an enumeration of any type of objects...
public static IEnumerable<Object> Pluck<Object>(this IEnumerable<Object> source, string propName)
{
return source
.Select(x => new {
obj: x
prop: x.GetType().GetProperty(propName)
})
.Where(x => x.prop != null)
.Select(x => x.prop.GetValue(x.obj));
};
For an enumeration of a single type of object...
public static IEnumerable<Object> Pluck<T>(this IEnumerable<T> source, string, propName)
{
var prop = typeof(T).GetProperty(propName);
return prop == null
? Enumerable.Empty<Object>()
: source.Select(x => prop.GetValue(x));
};
Related
There is a known issue with CosmosDb where if you use an ORDER BY clause it excludes documents that do not have this property defined
To work round this I'm trying to create functionality that takes a LINQ query and replaces the Order clause with a check for documents where the property is not defined so we can then run both queries and combine the results.
So:
ordersDb.Where(x => x.Name == customerName).OrderBy(x => x.CompanyName)
would become:
ordersDb.Where(x => x.Name == customerName)
.Where(x => !x.CompanyName.IsDefined()) // IsDefined is a built in CosmosDb function
Using Expression Builder I've created the following. However, I'm having issues trying to call my expression as a Where method - :
private sealed class OrderByToIsNotDefinedVisitor : ExpressionVisitor
{
protected override Expression VisitMethodCall(MethodCallExpression node)
{
if (node.Method.DeclaringType == typeof(Queryable) &&
(node.Method.Name == "OrderBy" || node.Method.Name == "OrderByDescending"))
{
// Get the IsDefined method
var methodIsDefined = typeof(TypeCheckFunctionsExtensions).GetMethod("IsDefined",
BindingFlags.Static | BindingFlags.NonPublic | BindingFlags.Public, null,
new Type[] { typeof(object) }, null);
// Apply the IsDefined method to the property that was being used for OrderBy
var isDefinedItem = Expression.Call(methodIsDefined, node.Arguments[1]);
// Alter the expression to check for !IsDefined()
var isNotDefinedItem = Expression.Not(isDefinedItem);
var entityType = node.Method.GetGenericArguments()[0];
var genericWhere = BuildGenericWhere();
var methodWhere = genericWhere.MakeGenericMethod(entityType);
var param = Expression.Parameter(entityType);
Expression newExpression =
Expression.Call(
methodWhere,
node.Arguments[0],
Expression.Lambda(
typeof(Func<,>).MakeGenericType(entityType, typeof(bool)),
isNotDefinedItem,
param));
return newExpression;
}
return base.VisitMethodCall(node);
}
}
private static MethodInfo BuildGenericWhere()
{
var genericWhereMethod = typeof(Enumerable).GetMethods(BindingFlags.Public | BindingFlags.Static)
.Where(x => x.Name == "Where" && x.GetGenericArguments().Length == 1)
.Select(x => new { Method = x, Parameters = x.GetParameters() })
.Where(x => x.Parameters.Length == 2 &&
x.Parameters[0].ParameterType.IsGenericType &&
x.Parameters[0].ParameterType.GetGenericTypeDefinition() == typeof(IEnumerable<>) &&
x.Parameters[1].ParameterType.IsGenericType &&
x.Parameters[1].ParameterType.GetGenericTypeDefinition() == typeof(Func<,>))
.Select(x => x.Method)
.Single();
return genericWhereMethod;
}
This compiles OK, but when I execute the query I get:
Microsoft.Azure.Documents.Linq.DocumentQueryException: Expression with
NodeType Lambda is not supported.
...as documentDb cannot handle the lambda clause
I've also tried replacing the lambda clause with a direct call to the where method, so the call becomes:
var updatedQueryExpression = Expression.Call(node.Arguments[0], methodWhere, isNotDefinedItem);
return updatedQueryExpression;
...however, this results in:
System.ArgumentException: Static method requires null instance,
non-static method requires non-null instance
Firstly,we are told that we can only sort with properties of document, not derived values.
So,for me,i also recommand you following the suggestion which is mentioned by #Paul in the comment: Querying all the matching data from cosmos db,then try to sort the result list and take the top elements.I believe that you already know the expression How to get first N elements of a list in C#?:
var firstFiveArrivals = myList.OrderBy(i => i.ArrivalTime).Take(5);
Since you have to get the top elements and if your matching data set is large enough, whether you use LINQ or SQL, you will encounter thresholds which could be solved by Continuation Token.
I am using Dynamic Linq to perform some queries (sorry but it's my only option). As a result, I am getting an IQueryable instead of an IQueryable<T>. In my case, I want an IQueryable<Thing> where Thing is a concrete type.
My query is as such:
public IQueryable<Thing> Foo(MyContext db)
{
var rootQuery = db.People.Where(x => x.City != null && x.State != null);
var groupedQuery = rootQuery.GroupBy("new ( it.City, it.State )", "it", new []{"City", "State"});
var finalLogicalQuery = groupedQuery.Select("new ( Count() as TotalNumber, Key.City as City, Key.State as State )");
var executionDeferredResults = finalLogicalQuery.Take(10); // IQueryable
IQueryable<Thing> executionDeferredTypedThings = ??; // <--- Help here!!!!
return executionDeferredTypedThings;
}
My Thing.cs:
public class Thing
{
public int TotalNumber { get; set; }
public string City { get; set; }
public string State { get; set; }
}
Yes, I know the exact above thing can be done without Dynamic Linq but I have some variableness going on that I've simplified out of here. I can get it to work with my variableness if my return type is simply IQueryable but I can't figure out how to convert to IQueryable<Thing> while keeping it execution-deferred and while also keeping Entity Framework happy. I do have the dynamic Select always returning something (with the correct data) that looks like a Thing. But I simply can't figure how to return the IQueryable<Thing> and could use some help there. Thanks!!
Failed Attempt 1
Based on Rex M's suggestion, I am now trying to use AutoMapper to solve this problem (although I am not committed to this approach and am willing to try other approaches). For the AutoMapper approach, I am doing it as such:
IQueryable<Thing> executionDeferredTypedThings = executionDeferredResults.ProjectTo<Thing>(); // <--- Help here!!!!
But this results in an InvalidOperationException:
Missing map from DynamicClass2 to Thing. Create using Mapper.CreateMap.
The thing is, while I have defined Thing, I have not defined DynamicClass2 and as such, I cannot map it.
Failed Attempt 2
IQueryable<Thing> executionDeferredTypedThings = db.People.Provider.CreateQuery<Thing>(executionDeferredResults.Expression);
This gives an InvalidCastException and seems to be the same underlying problem that the above AutoMapper fail hits:
Unable to cast object of type 'System.Data.Entity.Infrastructure.DbQuery'1[DynamicClass2]' to type 'System.Linq.IQueryable'1[MyDtos.Thing]'.
You can use AutoMapper's Queryable Extensions to produce an IQueryable which wraps the underlying IQueryable, thus preserving the original IQueryable's IQueryProvider and the deferred execution, but adds in a mapping/translating component to the pipeline to convert from one type to another.
There's also AutoMapper's UseAsDataSource which makes some common query extension scenarios easier.
If I understand correctly, the following extension method should do the job for you
public static class DynamicQueryableEx
{
public static IQueryable<TResult> Select<TResult>(this IQueryable source, string selector, params object[] values)
{
if (source == null) throw new ArgumentNullException("source");
if (selector == null) throw new ArgumentNullException("selector");
var dynamicLambda = System.Linq.Dynamic.DynamicExpression.ParseLambda(source.ElementType, null, selector, values);
var memberInit = dynamicLambda.Body as MemberInitExpression;
if (memberInit == null) throw new NotSupportedException();
var resultType = typeof(TResult);
var bindings = memberInit.Bindings.Cast<MemberAssignment>()
.Select(mb => Expression.Bind(
(MemberInfo)resultType.GetProperty(mb.Member.Name) ?? resultType.GetField(mb.Member.Name),
mb.Expression));
var body = Expression.MemberInit(Expression.New(resultType), bindings);
var lambda = Expression.Lambda(body, dynamicLambda.Parameters);
return source.Provider.CreateQuery<TResult>(
Expression.Call(
typeof(Queryable), "Select",
new Type[] { source.ElementType, lambda.Body.Type },
source.Expression, Expression.Quote(lambda)));
}
}
(Side note: Frankly I have no idea what values argument is for, but added it to match the corresponding DynamicQueryable.Select method signature.)
So your example will become something like this
public IQueryable<Thing> Foo(MyContext db)
{
var rootQuery = db.People.Where(x => x.City != null && x.State != null);
var groupedQuery = rootQuery.GroupBy("new ( it.City, it.State )", "it", new []{"City", "State"});
var finalLogicalQuery = groupedQuery.Select<Thing>("new ( Count() as TotalNumber, Key.City as City, Key.State as State )"); // IQueryable<Thing>
var executionDeferredTypedThings = finalLogicalQuery.Take(10);
return executionDeferredTypedThings;
}
How it works
The idea is quite simple.
The Select method implementation inside the DynamicQueryable looks something like this
public static IQueryable Select(this IQueryable source, string selector, params object[] values)
{
if (source == null) throw new ArgumentNullException("source");
if (selector == null) throw new ArgumentNullException("selector");
LambdaExpression lambda = DynamicExpression.ParseLambda(source.ElementType, null, selector, values);
return source.Provider.CreateQuery(
Expression.Call(
typeof(Queryable), "Select",
new Type[] { source.ElementType, lambda.Body.Type },
source.Expression, Expression.Quote(lambda)));
}
What it does is to dynamically create a selector expression and bind it to the source Select method. We take exactly the same approach, but after modifying the selector expression created by the DynamicExpression.ParseLambda call.
The only requirement is that the projection is using "new (...)" syntax and the names and types of the projected properties match, which I think fits in your use case.
The returned expression is something like this
(source) => new TargetClass
{
TargetProperty1 = Expression1(source),
TargetProperty2 = Expression2(source),
...
}
where TargetClass is a dynamically generated class.
All we want is to keep the source part and just replace that target class/properties with the desired class/properties.
As for the implementation, first the property assignments are converted with
var bindings = memberInit.Bindings.Cast<MemberAssignment>()
.Select(mb => Expression.Bind(
(MemberInfo)resultType.GetProperty(mb.Member.Name) ?? resultType.GetField(mb.Member.Name),
mb.Expression));
and then the new DynamicClassXXX { ... } is replaced with with
var body = Expression.MemberInit(Expression.New(resultType), bindings);
Would something like this be of benefit to you?
public static IQueryable<TEntity> GetQuery<TEntity>(this DbContext db, bool includeReferences = false) where TEntity : class
{
try
{
if (db == null)
{
return null;
}
var key = typeof(TEntity).Name;
var metaWorkspace = db.ToObjectContext().MetadataWorkspace;
var workspaceItems = metaWorkspace.GetItems<EntityType>(DataSpace.OSpace);
var workspaceItem = workspaceItems.First(f => f.FullName.Contains(key));
var navProperties = workspaceItem.NavigationProperties;
return !includeReferences
? db.Set<TEntity>()
: navProperties.Aggregate((IQueryable<TEntity>)db.Set<TEntity>(), (current, navProperty) => current.Include(navProperty.Name));
}
catch (Exception ex)
{
throw new ArgumentException("Invalid Entity Type supplied for Lookup", ex);
}
}
You may want to take a look into the Generic Search project on Github located here:
https://github.com/danielpalme/GenericSearch
There is no need for Dynamic Linq on this one.
var groupedQuery = from p in db.People
where p.City != null && p.State != null
group p by new {p.City, p.State}
into gp
select new Thing {
TotalNumber = gp.Count(),
City = gp.Key.City,
State = gp.Key.State
};
IQueryable<Thing> retQuery = groupedQuery.AsQueryable();
retQuery= retQuery.Take(10);
return retQuery;
So I had written this LINQ query using reflection, and later found out it isn't supported. What would be the best way to get the same functionality from this code?
List<Profile> profilesFromUUID = await MobileService.GetTable<Profile>().Where(p => typeof(Profile)
.GetProperty(handler.Name + "UUID").GetValue(p) == obj.uuid).ToListAsync();
Use the reflection to create the query, not in the query. Consider:
public static IQueryable<Profile> Filter(
this IQueryable<Profile> source, string name, Guid uuid)
{
// .<name>UUID
var property = typeof(Profile).GetProperty(name + "UUID");
// p
var parExp = Expression.Parameter(typeof(Profile));
// p.<name>UUID
var methodExp = Expression.Property(parExp, property);
// uuid
var constExp = Expression.Constant(uuid, typeof(Guid));
// p.<name>UUID == uuid
var binExp = Expression.Equal(methodExp, constExp);
// p => p.<name>UUID == uuid
var lambda = Expression.Lambda<Func<Profile, bool>>(binExp, parExp);
// source.Where(p => p.<name>UUID == uuid)
return source.Where(lambda);
}
This builds up the expression first (so if name was "Test" it would create the expression corresponding with p => p.TestUUID == uuid and then uses that in the call to Where.
Because this step is done first, rather than within the expression itself, there's no need for the query engine to try to translate typeof or GetProperty() into SQL (which it of course, couldn't do).
So:
var filtered = MobileService.GetTable<Profile>().Filter(handler.Name, obj.uuid);
Returns an IQueryable<Profile> with the appropriate Where attached. And so:
var profilesFromUUID = await MobileService.GetTable<Profile>().Filter(handler.Name, obj.uuid).ToListAsync();
Will as a whole first use reflection to build the query, then apply the query, then produce a list from it asynchrously and then wait for its results.
It's worth noting that since Filter() will accept any IQueryable<Profile> they can be either chained or unioned. So:
MobileService.GetTable<Profile>().Filter("A", uuid0).Filter("B", uuid1);
Is equivalent to:
from p in MobileService.GetTable<Profile>() where p.AUUID = uuid0 && p.BUUID == uuid1
And:
MobileService.GetTable<Profile>().Filter("A", uuid0).Union(
MobileService.GetTable<Profile>().Filter("B", uuid1))
Is equivalent to:
from p in MobileService.GetTable<Profile>() where p.AUUID = uuid0 || p.BUUID == uuid1
A more generalised version would be:
public static IQueryable<TSource> FilterByNamedProperty<TSource, TValue>(this IQueryable<TSource> source, string propertyName, TValue value)
{
var property = typeof(TSource).GetProperty(propertyName);
var parExp = Expression.Parameter(typeof(TSource));
var methodExp = Expression.Property(parExp, property);
var constExp = Expression.Constant(value, typeof(TValue));
var binExp = Expression.Equal(methodExp, constExp);
var lambda = Expression.Lambda<Func<TSource, bool>>(binExp, parExp);
return source.Where(lambda);
}
Then while you have to do the + "UUID" in the calling code, you can use this to do analogous queries with any IQueryable<> of any element type.
How about just compare all property name? By definition UUID would not have collision anyway. Since Profile is just a data class, the # of the property for UUID is fixed.
List<Profile> profilesFromUUID = await MobileService.GetTable<Profile>()
.Where(p =>
p.A_UUID == obj.uuid ||
p.B_UUID == obj.uuid ||
p.C_UUID == obj.uuid)
.ToListAsync();
Or add a method (extension method) for Profile like:
public static Guid GetUUIDByTableName(this Profile value, string tableName)
{
switch (tableName)
{
case "A_": return value.A_UUID;
case "B_": return value.B_UUID;
default: return Guid.Empty;
}
}
List<Profile> profilesFromUUID = await MobileService.GetTable<Profile>()
.Where(p => p.GetUUIDByTableName(handler.Name) == obj.uuid)
.ToListAsync();
I'm trying to build a LINQ expression, to filter values from a int property:
protected IQueryable<T> Some(IEnumerable<int> ids)
{
var parameter = Expression.Parameter(typeof(T), "x");
// "Col_id" (int property)
var property = Expression.Property(parameter, "Col_id");
MethodInfo method = typeof(Enumerable).
GetMethods().
Where(x => x.Name == "Contains").
Single(x => x.GetParameters().Length == 2).
MakeGenericMethod(typeof(T));
// ids = { 2, 4, 8 } etc...
var value = Expression.Constant(ids, typeof(IEnumerable<int>));
var containsMethod = Expression.Call(method, property, value); // exception
var aDelegate = Expression.Lambda<Func<T, bool>>(containsMethod, parameter);
table = myDataContext.GetTable<T>();
return table.AsQueryable().Where(aDelegate);
}
I'm trying to get something like: (x => ids.Contains(x.Col_id)), but an exception is thrown:
Expression of type 'System.Int32' cannot be used for type parameter
'System.Collections.Generic.IEnumerable'1[T] from 'Boolean
Contains[T](System.Collections.Generic.IEnumerable'1[T], T)' method
It looks to me like you've just got the arguments the wrong way round.
This:
Expression.Call(method, property, value)
means that you're calling:
Enumerable.Contains(x.Col_id, ids)
whereas you want
Enumerable.Contains(ids, x.Col_id)
So just try:
var containsMethod = Expression.Call(method, value, property);
EDIT: Next, you're building the wrong Contains type argument, I think. I suspect you want:
MethodInfo method = typeof(Enumerable).
GetMethods().
Where(x => x.Name == "Contains").
Single(x => x.GetParameters().Length == 2).
MakeGenericMethod(typeof(int));
After all, you want to call Enumerable.Contains<int>, not Enumerable.Contains<SomeType>.
I'm creating a method that receives a Queryable<T> source, a string with a property name/path (could be a deep property for example "TrParent.DataTypes" to achieve this x => x.TrParent.DataTypes) and Enumerable<int> which holds the values I need to intersect.
Basically I come from the need to create the following query dynamically (I mean <DT_Det_Tr> and TrParent.DataTypes being know only at runtime, in the example DT_Det_Tr is not a type it is a class):
var _vals = new List<int>();
var res = dbContext.Set<DT_Det_Tr>()
.Where
(x => x.TrParent.DataTypes
.Select(t => t.Id)
.Intersect(_vals)
.Any()
);
Please keep in mind that the preceding query is just an example of what I need to achieve dynamically, what I really need is an expression tree that creates a predicate like the one shown above but using a dynamic type and with the deep navigation property specified within a string.
So, I'm using this function to create the expression for the deep property:
private static LambdaExpression CreateDelegateExpression<T>(out Type resultingtype, string property, string parameterName = "x")
{
var type = typeof(T);
ParameterExpression param = Expression.Parameter(type, parameterName);
Expression expr = param;
foreach (string prop in property.Split('.'))
{
PropertyInfo pi = type.GetProperty(prop);
expr = Expression.Property(expr, pi);
type = pi.PropertyType;
}
Type delegateType = typeof(Func<,>).MakeGenericType(typeof(T), type);
LambdaExpression lambda = Expression.Lambda(delegateType, expr, param);
resultingtype = type;
return lambda;
}
And here is what I have so far for my function:
public static IQueryable<T> Intersect<T>(this IQueryable<T> source, string property, IEnumerable<int> value)
{
//List of ids
var _value = Expression.Constant(value);
//Get delegate expression to the deep property and it's inner type
Type type = null;
var lambda = CreateDelegateExpression<T>(out type, property, "x");
var enumtype = type.GetGenericArguments()[0];
ParameterExpression tpe = Expression.Parameter(enumtype, "y");
Expression propExp = Expression.Property(tpe, enumtype.GetProperty("Id"));
MethodInfo innermethod = typeof(Queryable).GetMethods().Where(x => x.Name == "Select").First();
//Error on next line...
var selectCall = Expression.Call(typeof(Queryable),
"Select",
new Type[] { enumtype, typeof(long) },
lambda,
propExp);
//TODO: Add rest of logic and actually filter the source
return source;
}
In the var selectCall = line I'm getting error:
No generic method 'Select' on type 'System.Linq.Queryable' is compatible with the supplied type arguments and arguments. No type arguments should be provided if the method is non-generic.
I've read a lot here on SO and other sites but I can't get past this part, I feel I'm going to bump into more trouble when I get to the .Intersect(List<int>).Any() part so any help on that also would be grand, thanks.
After a lot of thought, investigation and attempts I came up with a solution.
First, I made a simpler version of my goal query (the static example I used in my question), so instead of:
var res = dbContext.Set<DT_Det_Tr>()
.Where
(x => x.TrParent.DataTypes
.Select(t => t.Id)
.Intersect(_vals)
.Any()
);
I made this:
var res = dbContext.Set<DT_Det_Tr>()
.Where
(x => x.TrParent.DataTypes
.Any(y => _vals.Contains(y.Id))
);
Which is a lot easier to translate to expressions (or at least it was for me) because it omits the Select call.
I got rid of the method I was using to create the deep navigation property expression and streamlined it in my Intersect function, this was because it was doing some work I don't really need here plus I needed access to some of the variables I use inside it, then I made this:
public static IQueryable<T> Intersect<T>(this IQueryable<T> source, string property, IEnumerable<int> value)
{
var type = typeof(T);
var _value = Expression.Constant(value); //List of ids
//Declare parameter for outer lambda
ParameterExpression param = Expression.Parameter(type, "x");
//Outer Lambda
Expression expr = param;
foreach (string prop in property.Split('.')) //Dig for deep property
{
PropertyInfo pi = type.GetProperty(prop);
expr = Expression.Property(expr, pi);
type = pi.PropertyType;
}
//Get deep property's type
var enumtype = type.GetGenericArguments()[0];
//Declare parameter for inner lambda
ParameterExpression tpe = Expression.Parameter(enumtype, "y");
//Inner Collection lambda logic
//Property for inner lambda
Expression propExp = Expression.Property(tpe, enumtype.GetProperty("Id"));
//Contains method call .Contains(y.Id)
var containsMethodExp = Expression.Call(typeof(Enumerable), "Contains", new[] { propExp.Type }, _value, propExp);
//Create Expression<Func<enumtype, bool>>
var innerDelegateType = typeof(Func<,>).MakeGenericType(enumtype, typeof(bool));
//Create Inner lambda y => _vals.Contains(y.Id)
var innerFunction = Expression.Lambda(innerDelegateType, containsMethodExp, tpe);
//Get Any method info
var anyMethod = typeof(Enumerable).GetMethods().Where(m => m.Name == "Any" && m.GetParameters().Length == 2).Single().MakeGenericMethod(enumtype);
//Call Any with inner function .Any(y => _vals.Contains(y.Id))
var outerFunction = Expression.Call(anyMethod, expr, innerFunction);
//Call Where
MethodCallExpression whereCallExpression = Expression.Call
(
typeof(Queryable),
"Where",
new Type[] { source.ElementType },
source.Expression,
Expression.Lambda<Func<T, bool>>(outerFunction, new ParameterExpression[] { param })
);
//Create and return query
return source.Provider.CreateQuery<T>(whereCallExpression);
}
I hope this helps anyone trying to develop a similar solution.
Working with expression trees can be very hard and frustrating at first, but it's a really powerful tool once you get the hold of it.
If you have access to the dynamic keyword from c# 4.0, you might be able to work around the problem like this:
var _vals = new List<int>();
var res = dbContext.Set<DT_Det_Tr>()
.Where(obj => { dynamic x = obj;
return x.TrParent.DataTypes
.Select(t => t.Id)
.Intersect(_vals)
.Any();
}
);
But I don't know enough about the details of the problem you want to solve to say for sure.