I'm looking for a way to dynamically create a select list from a iQueryable object.
Concrete example, i want to do something like the following:
public void CreateSelectList(IQueryable(of EntityModel.Core.User entities), string[] columns)
{
foreach(var columnID in columns)
{
switch(columnID)
{
case "Type":
SelectList.add(e => e.UserType);
break;
case "Name":
SelectList.add(e => e.Name);
break;
etc....
}
}
var selectResult = (from u in entities select objSelectList);
}
So all properties are known, i however don't know beforehand what properties are to be selected. That will be passed via the columns parameter.
I know i'm going to run into issues with the type of the selectResult type, because when the select list is dynamic, the compiler doesn't know what the properties of the anonymous type needs to be.
If the above is not possible: The scenario I need it for is the following:
I'm trying to create a class that can be implemented to display a paged/filtered list of data. This data can be anything (depends on the implementations).The linq used is linq to entities. So they are directly linked to sql data. Now i want to only select the columns of the entities that i am actually showing in the list. Therefore i want the select to be dynamic. My entity might have a hundred properties, but if only 3 of them are shown in the list, i don't want to generate a query that selects the data of all 100 columns and then only uses 3 of them. If there is a different approach that I haven't thought of, I'm open to ideas
Edit:
Some clarifications on the contraints:
- The query needs to work with linq to entities (see question subject)
- an entity might contain 100 columns, so selecting ALL columns and then only reading the ones i need is not an option.
- The end user decides what columns to show, so the columns to select are determined at run time
- i need to create a SINGLE select, having multiple select statements means having multiple queries on the database, which i don't want
Dynamic select expression to a compile time known type can easily be build using Expression.MemberInit method with MemberBindings created using the Expression.Bind method.
Here is a custom extension method that does that:
public static class QueryableExtensions
{
public static IQueryable<TResult> Select<TResult>(this IQueryable source, string[] columns)
{
var sourceType = source.ElementType;
var resultType = typeof(TResult);
var parameter = Expression.Parameter(sourceType, "e");
var bindings = columns.Select(column => Expression.Bind(
resultType.GetProperty(column), Expression.PropertyOrField(parameter, column)));
var body = Expression.MemberInit(Expression.New(resultType), bindings);
var selector = Expression.Lambda(body, parameter);
return source.Provider.CreateQuery<TResult>(
Expression.Call(typeof(Queryable), "Select", new Type[] { sourceType, resultType },
source.Expression, Expression.Quote(selector)));
}
}
The only problem is what is the TResult type. In EF Core you can pass the entity type (like EntityModel.Core.User in your example) and it will work. In EF 6 and earlier, you need a separate non entity type because otherwise you'll get NotSupportedException - The entity or complex type cannot be constructed in a LINQ to Entities query.
UPDATE: If you want a to get rid of the string columns, I can suggest you replacing the extension method with the following class:
public class SelectList<TSource>
{
private List<MemberInfo> members = new List<MemberInfo>();
public SelectList<TSource> Add<TValue>(Expression<Func<TSource, TValue>> selector)
{
var member = ((MemberExpression)selector.Body).Member;
members.Add(member);
return this;
}
public IQueryable<TResult> Select<TResult>(IQueryable<TSource> source)
{
var sourceType = typeof(TSource);
var resultType = typeof(TResult);
var parameter = Expression.Parameter(sourceType, "e");
var bindings = members.Select(member => Expression.Bind(
resultType.GetProperty(member.Name), Expression.MakeMemberAccess(parameter, member)));
var body = Expression.MemberInit(Expression.New(resultType), bindings);
var selector = Expression.Lambda<Func<TSource, TResult>>(body, parameter);
return source.Select(selector);
}
}
with sample usage:
var selectList = new SelectList<EntityModel.Core.User>();
selectList.Add(e => e.UserType);
selectList.Add(e => e.Name);
var selectResult = selectList.Select<UserDto>(entities);
What you are going for is possible, but it's not simple. You can dynamically build EF queries using the methods and classes in the System.Linq.Expressions namespace.
See this question for a good example of how you can dynamically build your Select expression.
I believe this is what you need:
var entities = new List<User>();
entities.Add(new User { Name = "First", Type = "TypeA" });
entities.Add(new User { Name = "Second", Type = "TypeB" });
string[] columns = { "Name", "Type" };
var selectResult = new List<string>();
foreach (var columnID in columns)
{
selectResult.AddRange(entities.Select(e => e.GetType().GetProperty(columnID).GetValue(e, null).ToString()));
}
foreach (var result in selectResult)
{
Console.WriteLine(result);
}
This code outputs:
First
Second
TypeA
TypeB
UPDATE (according to comments)
// initialize alist of entities (User)
var entities = new List<User>();
entities.Add(new User { Name = "First", Type = "TypeA", SomeOtherField="abc" });
entities.Add(new User { Name = "Second", Type = "TypeB", SomeOtherField = "xyz" });
// set the wanted fields
string[] columns = { "Name", "Type" };
// create a set of properties of the User class by the set of wanted fields
var properties = typeof(User).GetProperties()
.Where(p => columns.Contains(p.Name))
.ToList();
// Get it with a single select (by use of the Dynamic object)
var selectResult = entities.Select(e =>
{
dynamic x = new ExpandoObject();
var temp = x as IDictionary<string, Object>;
foreach (var property in properties)
temp.Add(property.Name, property.GetValue(e));
return x;
});
// itterate the results
foreach (var result in selectResult)
{
Console.WriteLine(result.Name);
Console.WriteLine(result.Type);
}
This code outputs:
First
TypeA
Second
TypeB
Related
The ultimate objective is to have a piece of code that you can feed definitions of simple queries to and it generate data.
so given i can create this query
var query =
from cha in ds.Channels
select new object[]
{
cha.ChanLanguagestreamlocking,
cha.ChanMinimumtrailerduration,
from link in cha.Channelaudiolanguagelinks
select new object[]
{
link.ObjDatecreated
}
};
var data = query.ToArray();
how would i create it dynamically?
well i can get this far using a class stolen from another stack overflow question
public class SelectList<TSource>
{
private List<LambdaExpression> members = new List<LambdaExpression>();
public SelectList<TSource> Add<TValue>(Expression<Func<TSource, TValue>> selector)
{
members.Add(selector);
return this;
}
public Expression<Func<TSource, object[]>> ToDynamicColumns()
{
var parameter = Expression.Parameter(typeof(TSource), "e");
return Expression.Lambda<Func<TSource, object[]>>(
Expression.NewArrayInit(
typeof(object),
members.Select(m =>
Expression.Convert(Expression.Invoke(m, parameter), typeof(object))
)
),
parameter);
}
}
Imagine a case where the definition of which 'columns' to return is passed as a tree at runtime, this can then be mapped to predefined typessafe lambdas that define which columns to return and then the expression is created at runtime. So a hard coded version of this technique is this:
var channelColumns = new SelectList<Channel>();
channelColumns.Add(c => c.ChanLanguagestreamlocking);
channelColumns.Add(c => c.ChanMinimumtrailerduration);
var channelQuery =
ds.Channels.Select(channelColumns.ToDynamicColumns());
var bar = query.ToArray();
i.e. i can generate dynamic queries from the 'root' concept, but how do i generate the nested data.
If i do the obvious i.e. this
var audioColumns = new SelectList<Channelaudiolanguagelink>();
audioColumns.Add(a => a.ObjDatecreated);
var channelColumns = new SelectList<Channel>();
channelColumns.Add(c => c.ChanLanguagestreamlocking);
channelColumns.Add(c => c.ChanMinimumtrailerduration);
// next line causes an error
// Error CS1929 'ICollection<Channelaudiolanguagelink>' does not contain a definition for
// 'Select' and the best extension method overload
// 'Queryable.Select<Channel, object[]>(IQueryable<Channel>, Expression<Func<Channel, object[]>>)' requires a receiver of type 'IQueryable<Channel>' CSharpDb2Raw C:\Users\mark.nicholls\source\repos\scaffold2\CSharpDb2Raw\Program.cs 57 Active
channelColumns.Add(c => c.Channelaudiolanguagelinks.Select(channelColumns.ToDynamicColumns()));
var channelQuery =
ds.Channels.Select(channelColumns.ToDynamicColumns());
var bar = query.ToArray();
the error makes perfect sense.
c.Channelaudiolanguagelinks is an ICollection and so the select is looking for a Func<T,U> and I've given it an Expression<Func<T,U>>
(I don't really understand Expressions!)
Problem that you have defined method for IQueryable version of Select, so basic solution is simple - transform IEnumerable to IQueryable.
channelColumns.Add(c =>
c.Channelaudiolanguagelinks.AsQueryable().Select(audioColumns.ToDynamicColumns())
);
I have an IQueryable<T> from my DbSet in Entity Framework. I am provided a "Fuzzy Search String", named searchText, like so:
public List<T> Search<T>(string searchText)
{
using (var context = ...)
{
var baseQuery = context.Set<T>().AsQueryable();
baseQuery = baseQuery.Where(x =>
DbFunctions.Like(x.PropertyName, searchText)
|| DbFunctions.Like(x.PropertyTwo, searchText)
|| DbFunctions.Like(x.PropertyThree, searchText)
|| DbFunctio..... etc
);
return baseQuery.ToList();
}
}
But given the generic nature, I don't know what properties there are on the type. I can provide an abstract method to somebody implementing this which allows them to give me a List of Properties (or even PropertyInfo or whatever else, I can figure that out). But I don't know how to dynamically create the expression. This is what I have so far:
var baseQuery = context.Set<T>().AsQueryable();
var expression = baseQuery.Expression;
var colName = "colName"; // Or names, I can iterate.
var parameter = Expression.Parameter(typeof(T), "x");
var selector = Expression.PropertyOrField(parameter, colName);
expression = Expression.Call(typeof(DbFunctions), nameof(DbFunctions.Like),
new Type[] { baseQuery.ElementType, selector.Type },
expression, Expression.Quote(Expression.Lambda(selector, parameter)));
The problem here is... well, it doesn't work to begin with. But mainly that I'm not using the searchText anywhere in it, and don't know how to plug it in. I THINK I'm close... but have spent an inordinate amount of time on it.
Hopefully I'm getting your query logic right: if you want to build a set of LIKE conditions based on known type and list of column names, you could try something like this:
static private MethodInfo dbLikeMethod = typeof(DbFunctions).GetMethod(nameof(DbFunctions.Like), BindingFlags.Public | BindingFlags.Static, null, new Type[] {typeof(string), typeof(string)}, null); // I am targeting DbFunctions.Like(string, string). You might want another overload (or even mix them up depending on your inputs)
public List<T> Search<T>(string searchText) where T: class
{
using (var context = new ...)
{
var baseQuery = context.Set<T>().AsQueryable().Where(CreateExpression<T>(searchText));// you could probably find a more elegant way of plugging it into your query
return baseQuery.ToList();
}
}
Expression<Func<T, bool>> CreateExpression<T>(string searchText) where T : class
{
var cols = new List<string> {
"PropertyName",
"PropertyTwo" // i understand you've got a way to figure out which strings you need here
};
var parameter = Expression.Parameter(typeof(T), "x");
var dbLikeCalls = cols.Select(colName => Expression.Call(dbLikeMethod, Expression.PropertyOrField(parameter, colName), Expression.Constant(searchText))); // for convenience, generate list of DbFunctions.Like(x.<Property>, searchText) expressions here
var aggregatedCalls = dbLikeCalls.Skip(1).Aggregate((Expression)dbLikeCalls.First(), (accumulate, call) => Expression.OrElse(accumulate, call)); // aggregate the list using || operators: use first item as a seed and keep adding onto it
return Expression.Lambda<Func<T, bool>>(aggregatedCalls, parameter);
}
I have a question for you regarding the creation of a Dynamic select query in Entity Framework.
I already have a dynamic query for the select based on rights etc. But for each table I get 30+ fields that I have to parse via the .GetType().GetProperties().
Its complex and its quite costly in terms of resource due to the amount of data we have.
I have a service that tells me which fields I should select for each table. I would like to find a way to transform that into the query but I can't find something that is really dynamic.
That is not dynamic but manual:
using (var context = new StackOverflowContext())
{
var posts = context.Posts
.Where(p => p.Tags == "<sql-server>")
.Select(p => new {p.Id, p.Title});
// Do something;
}
I need to say, select only those fields but only the fields with this names.
I have the field list in a list of string but that could be changed.
Could you please help me?
Here is a .Net Fiddle code (made by msbendtsen) that allows to dynamically select columns (properties).
https://dotnetfiddle.net/3IMR1r
The sample is written for linq to objects but it should work with entity frameworks.
The key section is:
internal static IQueryable SelectProperties<T>(this IQueryable<T> queryable, IEnumerable<string> propertyNames)
{
// get propertyinfo's from original type
var properties = typeof(T).GetProperties().Where(p => propertyNames.Contains(p.Name));
// Create the x => expression
var lambdaParameterExpression = Expression.Parameter(typeof(T));
// Create the x.<propertyName>'s
var propertyExpressions = properties.Select(p => Expression.Property(lambdaParameterExpression, p));
// Creating anonymous type using dictionary of property name and property type
var anonymousType = AnonymousTypeUtils.CreateType(properties.ToDictionary(p => p.Name, p => p.PropertyType));
var anonymousTypeConstructor = anonymousType.GetConstructors().Single();
var anonymousTypeMembers = anonymousType.GetProperties().Cast<MemberInfo>().ToArray();
// Create the new {} expression using
var anonymousTypeNewExpression = Expression.New(anonymousTypeConstructor, propertyExpressions, anonymousTypeMembers);
var selectLambdaMethod = GetExpressionLambdaMethod(lambdaParameterExpression.Type, anonymousType);
var selectBodyLambdaParameters = new object[] { anonymousTypeNewExpression, new[] { lambdaParameterExpression } };
var selectBodyLambdaExpression = (LambdaExpression)selectLambdaMethod.Invoke(null, selectBodyLambdaParameters);
var selectMethod = GetQueryableSelectMethod(typeof(T), anonymousType);
var selectedQueryable = selectMethod.Invoke(null, new object[] { queryable, selectBodyLambdaExpression }) as IQueryable;
return selectedQueryable;
}
I've some problems with a LINQ query in C#.
I have in the database the same tables that have the same structure.
So, today, I've been troubling with my LINQ query.
More details, I want to join some tables using predicates.
I have a function that has two parameters.
The first parameter is some kind of Context (For example, it may be ProductContext, CarContext, CatContext and etc).
The second parameter is a List<something> that I will join with my first parameter - Context.
I do not want a set of methods.
I've added the sample:
public Element[] GetByIds( MyPredicateContext, Guid[] ids)
{
return
from id in ids
join element in MyPredicateContext on id equals element.Id
select
new Element
{
Id = element.Id,
Description = element.JobDescription,
};
}
If the query is correct, one basic issue that I can see is the return type is Element array whereas you are trying to return IEnumerable. Maybe doing a .ToArray() on the result set might solve the problem.
Why not
return MyPredicateContext.Where(element=>ids.Contains(element.Id))
.Select(e=>new Element()
{
Id = element.Id,
Description = element.JobDescription
}).ToArray();
First of all you can't create a new IQueryable from an array this will revert to pulling everything in memory and filtering there. You are working with expressions and not c# code when you do LINQ with SQL, this will only work on in memory stuff (IEnumerable).
Your query will work in SQL if you do it like this
from element in MyPredicateContext
where ids.Contains(element.Id)
select new Element
{
Id = element.Id,
Description = element.JobDescription,
}
Given that the type of IQueryable where T is an Interface or class.
The end method will look something like this
public interface IElement
{
Guid Id { get; }
string JobDescription { get; }
}
public Element[] GetByIds<T>(IQueryable<T> myPredicateContext, Guid[] ids) where T:IElement
{
return (from element in myPredicateContext
where ids.Contains(element.Id)
select new Element
{
Id = element.Id,
Description = element.JobDescription,
}).ToArray();
}
There are ways to do it with no Generics but they are a bit more advanced and will be hard to maintain.
Here is a method that will work on all T types and proper IQueryable will produce good sql just as I pointed out is a bit more advanced and you will need to lookup how expression work.
public static Element[] GetById<T, Tkey>(IQueryable<T> items,Tkey[] ids)
{
var type = typeof(T);
ParameterExpression param = Expression.Parameter(type);
var list = Expression.Constant(ids);
//The names of the properties you need to get if all models have them and are named the same and are the same type this will work
var idProp = Expression.Property(param, "Id");
var descriptionProp = Expression.Property(param, "JobDescription");
var contains = typeof(Enumerable).GetMethods().First(m => m.Name == "Contains" && m.GetParameters().Count() == 2).MakeGenericMethod(typeof(Tkey));
var where = Expression.Lambda<Func<T, bool>>(Expression.Call(contains, list, idProp), param);
return (items.
Where(where).
Select(Expression.Lambda<Func<T, Element>>(
Expression.MemberInit(
Expression.New(typeof(Element)),
Expression.Bind(typeof(Element).GetProperty("Id"), idProp),
Expression.Bind(typeof(Element).GetProperty("Description"), descriptionProp)),
param))).ToArray();
}
Call GetById(items, new Guid[] { Guid.NewGuid() })
Using EntityFramework, I can get a list of entities of a certain type using the following syntax:
List<Customer> customers = ((IQueryable<Customer>)myEntities.Customers
.Where(c => c.Surname == strSurname)
.OrderBy(c => c.Surname)).ToList<Customer>();
I can then do something like this to end up with only the data I'm interested in:
var customerSummaries = from s in customers
select new
{
s.Surname, s.FirstName, s.Address.Postcode
};
I'm given a list of strings (based on user selection) of the fields (and tables where necessary) that comprise the requested summarized data. Eg for the above 'customerSummary' the list of strings provided would be: "Surname", "FirstName", "Address.Postcode".
My question is: How do I convert that list of strings into the syntax needed to extract only the specified fields?
If that can't be done, what would be a better type (than string) for the list of columns so I can extract the right info?
I guess I need to know the type that an EF [entity|table]'s [member|column|field] is, if that makes sense.
EDIT:
I tried the suggested answer - dynamic linq - using the following syntax
string parmList = "Surname, Firstname, Address.Postcode";
var customers = myEntities.Customers.Select(parmList)
.OrderBy("Address.Postcode");
but this results in: EntitySqlException was unhandled. 'Surname' could not be resolved in the current scope or context. Make sure that all referenced variables are in scope, that required schemas are loaded, and that namespaces are referenced correctly.
So, a follow-up question. Am I using Select properly? I have only seen examples using the Where and OrderBy clauses, but I think I'm doing it right based on those.
If my Select syntax is not the problem, can anyone see what is?
Edit 2: It was upside down. This works:
string parmList = "Surname, Firstname, Address.Postcode";
var customers = myEntities.Customers
.OrderBy("Address.Postcode")
.Select(parmList);
You can use dynamic linq, check it in here. It is also available on nuget
I suggest dynamic linq as #Cuong already posted.
But for simple Select projections and as a hand-made exercise in expressions...
Using LinqRuntimeTypeBuilder from How to create LINQ Expression Tree to select an anonymous type
(see the comments there for 'why' is that necessary)
And add this...
public static IQueryable SelectDynamic(this IQueryable source, IEnumerable<string> fieldNames)
{
Dictionary<string, PropertyInfo[]> sourceProperties = new Dictionary<string, PropertyInfo[]>();
foreach (var propertyPath in fieldNames)
{
var props = propertyPath.Split('.');
var name = props.Last();
PropertyInfo[] infos;
if (sourceProperties.TryGetValue(name, out infos))
name = string.Join("", props);
sourceProperties[name] = source.ElementType.GetDeepProperty(props);
}
Type dynamicType = LinqRuntimeTypeBuilder.GetDynamicType(sourceProperties.ToDictionary(x => x.Key, x => x.Value.Last().PropertyType));
ParameterExpression sourceItem = Expression.Parameter(source.ElementType, "t");
IEnumerable<MemberBinding> bindings = dynamicType.GetFields()
.Select(p => Expression.Bind(p, sourceItem.MakePropertyExpression(sourceProperties[p.Name]))).OfType<MemberBinding>();
Expression selector = Expression.Lambda(Expression.MemberInit(
Expression.New(dynamicType.GetConstructor(Type.EmptyTypes)), bindings), sourceItem);
MethodCallExpression selectExpression = Expression.Call(typeof(Queryable), "Select", new Type[] { source.ElementType, dynamicType }, Expression.Constant(source), selector);
return Expression.Lambda(selectExpression).Compile().DynamicInvoke() as IQueryable;
}
public static PropertyInfo[] GetDeepProperty(this Type type, params string[] props)
{
List<PropertyInfo> list = new List<PropertyInfo>();
foreach (var propertyName in props)
{
var info = type.GetProperty(propertyName);
type = info.PropertyType;
list.Add(info);
}
return list.ToArray();
}
public static Expression MakePropertyExpression(this ParameterExpression sourceItem, PropertyInfo[] properties)
{
Expression property = sourceItem;
foreach (var propertyInfo in properties)
property = Expression.Property(property, propertyInfo);
return property;
}
Use it like e.g.:
public static IEnumerable<object>
SelectAsEnumerable(this IQueryable entitySet, params string[] propertyPath)
{
return entitySet.SelectDynamic(propertyPath) as IEnumerable<object>;
}
var list = db.YourEntity.SelectAsEnumerable("Name", "ID", "TestProperty.ID").ToList();
Note:
You can use similar approach to expand for OrderBy etc. - but this wasn't meant to cover all angles (or any real queries)
Deep Property Expressions from this post of mine