We Keep Coding

Dynamically generating nested queries

The ultimate objective is to have a piece of code that you can feed definitions of simple queries to and it generate data.
so given i can create this query
var query =
from cha in ds.Channels
select new object[]
{
cha.ChanLanguagestreamlocking,
cha.ChanMinimumtrailerduration,
from link in cha.Channelaudiolanguagelinks
select new object[]
{
link.ObjDatecreated
}
};
var data = query.ToArray();
how would i create it dynamically?
well i can get this far using a class stolen from another stack overflow question
public class SelectList<TSource>
{
private List<LambdaExpression> members = new List<LambdaExpression>();
public SelectList<TSource> Add<TValue>(Expression<Func<TSource, TValue>> selector)
{
members.Add(selector);
return this;
}
public Expression<Func<TSource, object[]>> ToDynamicColumns()
{
var parameter = Expression.Parameter(typeof(TSource), "e");
return Expression.Lambda<Func<TSource, object[]>>(
Expression.NewArrayInit(
typeof(object),
members.Select(m =>
Expression.Convert(Expression.Invoke(m, parameter), typeof(object))
)
),
parameter);
}
}
Imagine a case where the definition of which 'columns' to return is passed as a tree at runtime, this can then be mapped to predefined typessafe lambdas that define which columns to return and then the expression is created at runtime. So a hard coded version of this technique is this:
var channelColumns = new SelectList<Channel>();
channelColumns.Add(c => c.ChanLanguagestreamlocking);
channelColumns.Add(c => c.ChanMinimumtrailerduration);
var channelQuery =
ds.Channels.Select(channelColumns.ToDynamicColumns());
var bar = query.ToArray();
i.e. i can generate dynamic queries from the 'root' concept, but how do i generate the nested data.
If i do the obvious i.e. this
var audioColumns = new SelectList<Channelaudiolanguagelink>();
audioColumns.Add(a => a.ObjDatecreated);
var channelColumns = new SelectList<Channel>();
channelColumns.Add(c => c.ChanLanguagestreamlocking);
channelColumns.Add(c => c.ChanMinimumtrailerduration);
// next line causes an error
// Error CS1929 'ICollection<Channelaudiolanguagelink>' does not contain a definition for
// 'Select' and the best extension method overload
// 'Queryable.Select<Channel, object[]>(IQueryable<Channel>, Expression<Func<Channel, object[]>>)' requires a receiver of type 'IQueryable<Channel>' CSharpDb2Raw C:\Users\mark.nicholls\source\repos\scaffold2\CSharpDb2Raw\Program.cs 57 Active
channelColumns.Add(c => c.Channelaudiolanguagelinks.Select(channelColumns.ToDynamicColumns()));
var channelQuery =
ds.Channels.Select(channelColumns.ToDynamicColumns());
var bar = query.ToArray();
the error makes perfect sense.
c.Channelaudiolanguagelinks is an ICollection and so the select is looking for a Func<T,U> and I've given it an Expression<Func<T,U>>
(I don't really understand Expressions!)

Problem that you have defined method for IQueryable version of Select, so basic solution is simple - transform IEnumerable to IQueryable.
channelColumns.Add(c =>
c.Channelaudiolanguagelinks.AsQueryable().Select(audioColumns.ToDynamicColumns())
);

In LinqToEntities, How to pass dynamic column name to DbFunctions.Like

I have an IQueryable<T> from my DbSet in Entity Framework. I am provided a "Fuzzy Search String", named searchText, like so:
public List<T> Search<T>(string searchText)
{
using (var context = ...)
{
var baseQuery = context.Set<T>().AsQueryable();
baseQuery = baseQuery.Where(x =>
DbFunctions.Like(x.PropertyName, searchText)
|| DbFunctions.Like(x.PropertyTwo, searchText)
|| DbFunctions.Like(x.PropertyThree, searchText)
|| DbFunctio..... etc
);
return baseQuery.ToList();
}
}
But given the generic nature, I don't know what properties there are on the type. I can provide an abstract method to somebody implementing this which allows them to give me a List of Properties (or even PropertyInfo or whatever else, I can figure that out). But I don't know how to dynamically create the expression. This is what I have so far:
var baseQuery = context.Set<T>().AsQueryable();
var expression = baseQuery.Expression;
var colName = "colName"; // Or names, I can iterate.
var parameter = Expression.Parameter(typeof(T), "x");
var selector = Expression.PropertyOrField(parameter, colName);
expression = Expression.Call(typeof(DbFunctions), nameof(DbFunctions.Like),
new Type[] { baseQuery.ElementType, selector.Type },
expression, Expression.Quote(Expression.Lambda(selector, parameter)));
The problem here is... well, it doesn't work to begin with. But mainly that I'm not using the searchText anywhere in it, and don't know how to plug it in. I THINK I'm close... but have spent an inordinate amount of time on it.

Hopefully I'm getting your query logic right: if you want to build a set of LIKE conditions based on known type and list of column names, you could try something like this:
static private MethodInfo dbLikeMethod = typeof(DbFunctions).GetMethod(nameof(DbFunctions.Like), BindingFlags.Public | BindingFlags.Static, null, new Type[] {typeof(string), typeof(string)}, null); // I am targeting DbFunctions.Like(string, string). You might want another overload (or even mix them up depending on your inputs)
public List<T> Search<T>(string searchText) where T: class
{
using (var context = new ...)
{
var baseQuery = context.Set<T>().AsQueryable().Where(CreateExpression<T>(searchText));// you could probably find a more elegant way of plugging it into your query
return baseQuery.ToList();
}
}
Expression<Func<T, bool>> CreateExpression<T>(string searchText) where T : class
{
var cols = new List<string> {
"PropertyName",
"PropertyTwo" // i understand you've got a way to figure out which strings you need here
};
var parameter = Expression.Parameter(typeof(T), "x");
var dbLikeCalls = cols.Select(colName => Expression.Call(dbLikeMethod, Expression.PropertyOrField(parameter, colName), Expression.Constant(searchText))); // for convenience, generate list of DbFunctions.Like(x.<Property>, searchText) expressions here
var aggregatedCalls = dbLikeCalls.Skip(1).Aggregate((Expression)dbLikeCalls.First(), (accumulate, call) => Expression.OrElse(accumulate, call)); // aggregate the list using || operators: use first item as a seed and keep adding onto it
return Expression.Lambda<Func<T, bool>>(aggregatedCalls, parameter);
}

Dynamically build select list from linq to entities query

I'm looking for a way to dynamically create a select list from a iQueryable object.
Concrete example, i want to do something like the following:
public void CreateSelectList(IQueryable(of EntityModel.Core.User entities), string[] columns)
{
foreach(var columnID in columns)
{
switch(columnID)
{
case "Type":
SelectList.add(e => e.UserType);
break;
case "Name":
SelectList.add(e => e.Name);
break;
etc....
}
}
var selectResult = (from u in entities select objSelectList);
}
So all properties are known, i however don't know beforehand what properties are to be selected. That will be passed via the columns parameter.
I know i'm going to run into issues with the type of the selectResult type, because when the select list is dynamic, the compiler doesn't know what the properties of the anonymous type needs to be.
If the above is not possible: The scenario I need it for is the following:
I'm trying to create a class that can be implemented to display a paged/filtered list of data. This data can be anything (depends on the implementations).The linq used is linq to entities. So they are directly linked to sql data. Now i want to only select the columns of the entities that i am actually showing in the list. Therefore i want the select to be dynamic. My entity might have a hundred properties, but if only 3 of them are shown in the list, i don't want to generate a query that selects the data of all 100 columns and then only uses 3 of them. If there is a different approach that I haven't thought of, I'm open to ideas
Edit:
Some clarifications on the contraints:
- The query needs to work with linq to entities (see question subject)
- an entity might contain 100 columns, so selecting ALL columns and then only reading the ones i need is not an option.
- The end user decides what columns to show, so the columns to select are determined at run time
- i need to create a SINGLE select, having multiple select statements means having multiple queries on the database, which i don't want

Dynamic select expression to a compile time known type can easily be build using Expression.MemberInit method with MemberBindings created using the Expression.Bind method.
Here is a custom extension method that does that:
public static class QueryableExtensions
{
public static IQueryable<TResult> Select<TResult>(this IQueryable source, string[] columns)
{
var sourceType = source.ElementType;
var resultType = typeof(TResult);
var parameter = Expression.Parameter(sourceType, "e");
var bindings = columns.Select(column => Expression.Bind(
resultType.GetProperty(column), Expression.PropertyOrField(parameter, column)));
var body = Expression.MemberInit(Expression.New(resultType), bindings);
var selector = Expression.Lambda(body, parameter);
return source.Provider.CreateQuery<TResult>(
Expression.Call(typeof(Queryable), "Select", new Type[] { sourceType, resultType },
source.Expression, Expression.Quote(selector)));
}
}
The only problem is what is the TResult type. In EF Core you can pass the entity type (like EntityModel.Core.User in your example) and it will work. In EF 6 and earlier, you need a separate non entity type because otherwise you'll get NotSupportedException - The entity or complex type cannot be constructed in a LINQ to Entities query.
UPDATE: If you want a to get rid of the string columns, I can suggest you replacing the extension method with the following class:
public class SelectList<TSource>
{
private List<MemberInfo> members = new List<MemberInfo>();
public SelectList<TSource> Add<TValue>(Expression<Func<TSource, TValue>> selector)
{
var member = ((MemberExpression)selector.Body).Member;
members.Add(member);
return this;
}
public IQueryable<TResult> Select<TResult>(IQueryable<TSource> source)
{
var sourceType = typeof(TSource);
var resultType = typeof(TResult);
var parameter = Expression.Parameter(sourceType, "e");
var bindings = members.Select(member => Expression.Bind(
resultType.GetProperty(member.Name), Expression.MakeMemberAccess(parameter, member)));
var body = Expression.MemberInit(Expression.New(resultType), bindings);
var selector = Expression.Lambda<Func<TSource, TResult>>(body, parameter);
return source.Select(selector);
}
}
with sample usage:
var selectList = new SelectList<EntityModel.Core.User>();
selectList.Add(e => e.UserType);
selectList.Add(e => e.Name);
var selectResult = selectList.Select<UserDto>(entities);

What you are going for is possible, but it's not simple. You can dynamically build EF queries using the methods and classes in the System.Linq.Expressions namespace.
See this question for a good example of how you can dynamically build your Select expression.

I believe this is what you need:
var entities = new List<User>();
entities.Add(new User { Name = "First", Type = "TypeA" });
entities.Add(new User { Name = "Second", Type = "TypeB" });
string[] columns = { "Name", "Type" };
var selectResult = new List<string>();
foreach (var columnID in columns)
{
selectResult.AddRange(entities.Select(e => e.GetType().GetProperty(columnID).GetValue(e, null).ToString()));
}
foreach (var result in selectResult)
{
Console.WriteLine(result);
}
This code outputs:
First
Second
TypeA
TypeB
UPDATE (according to comments)
// initialize alist of entities (User)
var entities = new List<User>();
entities.Add(new User { Name = "First", Type = "TypeA", SomeOtherField="abc" });
entities.Add(new User { Name = "Second", Type = "TypeB", SomeOtherField = "xyz" });
// set the wanted fields
string[] columns = { "Name", "Type" };
// create a set of properties of the User class by the set of wanted fields
var properties = typeof(User).GetProperties()
.Where(p => columns.Contains(p.Name))
.ToList();
// Get it with a single select (by use of the Dynamic object)
var selectResult = entities.Select(e =>
{
dynamic x = new ExpandoObject();
var temp = x as IDictionary<string, Object>;
foreach (var property in properties)
temp.Add(property.Name, property.GetValue(e));
return x;
});
// itterate the results
foreach (var result in selectResult)
{
Console.WriteLine(result.Name);
Console.WriteLine(result.Type);
}
This code outputs:
First
TypeA
Second
TypeB

Create a JOIN at the run time in a EF6 extended DbContext class

I need to create a table joins at the runtime with a given configuration.
In this case, I have the IQueryable property which is the root and I need to create the join dynamically.
Here what I have tried out.
public class MyDbContext : DbContext
{
public IQueryable<T> AsDynamicQueryable<T>() where T : class
{
var predicate = default(Func<T, bool>); // This is a Dynamically generated predicate
var query = this.Set<T>().Where(predicate).AsQueryable();
// Now here I need to append a JOIN to the above 'query'
// So far, this is what I have done.
var rootType = typeof(T);
var innerType = Type.GetType("This type takes from the configuration");
var innerExpression = this.Set(innerType).AsQueryable();
var paramOne = Expression.Parameter(rootType, "p1");
var paramTwo = Expression.Parameter(innerType, "p2");
var outerKeySelector = Expression.Property(paramOne, "property_one"); //'property_one' is a property of a first parameter which takes from the configuration
var outerKeySelectorExpression = Expression.Lambda(outerKeySelector, paramOne); // (p1)=>p1.property_one
var innerKeySelector = Expression.Property(paramTwo, "property_two"); //'property_two' is a property of a 2nd parameter which takes from the configuration
var innerKeySelectorExpression = Expression.Lambda(innerKeySelector, paramTwo); // (p2)=>p2.property_two
var resultSelector = Expression.Lambda(paramOne, paramOne, paramTwo); // (p1,p2)=>p1
var joinMethod = typeof(Queryable)
.GetMethods()
.First(m => m.Name == "Join" && m.GetParameters().Length == 5)
.MakeGenericMethod(rootType, innerType, typeof(int), rootType);
// 1st Apptempt.
// I'm not sure that I can execute the JOIN method like this.
// But anyway, this gives the below error when I try to execute via taking Count();
// "This method supports the LINQ to Entities infrastructure and is not intended to be used directly from your code."
var newQuery = (IQueryable<T>)joinMethod
.Invoke(
query,
new object[]
{
query,
innerExpression,
outerKeySelectorExpression,
innerKeySelectorExpression,
resultSelector
});
var tt = newQuery.Count(); // Here I just try to execute the expression to check whether it works before I return the Queryable.
// 2nd Attempt
// This also gives the following error when I try to execute via taking Count();
// Unable to create a constant value of type '<type name of the root(T) type>'. Only primitive types or enumeration types are supported in this context.
var joinMethodCallExpression = Expression.Call(
null,
joinMethod,
query.Expression,
innerExpression.Expression,
outerKeySelectorExpression,
innerKeySelectorExpression,
resultSelector);
var xx = this.Set<T>().AsQueryable().Provider.CreateQuery<T>(joinMethodCallExpression);
var te = xx.Count(); // Here I just try to execute the expression to check whether it works before I return the Queryable.
throw new NotImplementedException();
}
}
Highly appreciate if someone can point out the correct way of doing this.

Here is the code. I've added my comments inside the code:
public IQueryable<T> AsDynamicQueryable<T>() where T : class
{
// ERROR!!! It should be Expression<Func<T, bool>>
// GetPredicate<T>() is my method to get the predicate. You must
// put here yours. IT must return an Expression<Func<T, bool>>
Expression<Func<T, bool>> predicate = GetPredicate<T>(); // This is a Dynamically generated predicate
// ERROR!!! Don't EVER use AsQueryable(), unless you exactly know
// what you are doing. In this example, your use of AsQueryable<>()
// is hiding the fact that you are executing the Where() LOCALLY,
// because it is a Where(this IEnumerable<>, Func<>) instead of
// being a Where(this IQueryable<>, Expression<>)
// If you want an IQueryable<>, put it in a IQueryable<> variable
IQueryable<T> query = this.Set<T>().Where(predicate);
var rootType = typeof(T);
var innerType = GetAsDynamicQueryableInnerType<T>();
// Same as before! Don't use .AsQueryable(). In this case, use
// IQueryable (non-generic). Note that in this case there was
// no problem with yoru code, so AsQueryable() wasn't doing
// "damage"
IQueryable innerExpression = this.Set(innerType);
var paramOne = Expression.Parameter(rootType, "p1");
var paramTwo = Expression.Parameter(innerType, "p2");
// GetPrimaryKey() is my method to get the property to use.
// it returns a string with the name of the property
string primaryKeyRootType = GetPrimaryKey(rootType);
var outerKeySelector = Expression.Property(paramOne, primaryKeyRootType); //'property_one' is a property of a first parameter which takes from the configuration
var outerKeySelectorExpression = Expression.Lambda(outerKeySelector, paramOne); // (p1)=>p1.property_one
// GetForeignKey() is my method to get the property to use.
// it returns a string with the name of the property
var foreignKeyInnerType = GetForeignKey(innerType, rootType);
var innerKeySelector = Expression.Property(paramTwo, foreignKeyInnerType); //'property_two' is a property of a 2nd parameter which takes from the configuration
var innerKeySelectorExpression = Expression.Lambda(innerKeySelector, paramTwo); // (p2)=>p2.property_two
var resultSelector = Expression.Lambda(paramOne, paramOne, paramTwo); // (p1,p2)=>p1
// Using outerKeySelector.Type as the type of the third parameter
// here. 99% it is typeof(int), but why not make it more generic?
var joinMethod = typeof(Queryable)
.GetMethods()
.First(m => m.Name == "Join" && m.GetParameters().Length == 5)
.MakeGenericMethod(rootType, innerType, outerKeySelector.Type, rootType);
// Queryable.Join is static, so the first parameter must be null!
// Then the parameters to pass to Queryable.Join are the ones you
// where using in the 1st case.
var newQuery = (IQueryable<T>)joinMethod.Invoke(
null,
new object[]
{
query,
innerExpression,
outerKeySelectorExpression,
innerKeySelectorExpression,
resultSelector
});
return newQuery;
}
And then there is the big problem: even if it can work, you can only get back a IQueryable<T>, but the result of a Join is normally a IQueryable<T+U>. I see that you wrote resultSelector = ... (p1,p2)=>p1, but is it really what you want?

How to produce a Subquery using non-generic Lambda

How would you translate the following generic Lambda function into a lambda expression :
context.AssociateWith<Product>(p => p.Regions.Where(r => r.Country == 'Canada')
I'm trying to create a full lambda expression without any <T> or direct call. Something like :
void AddFilter(ITable table, MetaDataMember relation)
{
var tableParam = Expression.Parameter(table.ElementType, "e");
var prop = Expression.Property(tableParam, relation.Name);
var func = typeof(Func<,>).MakeGenericType(table.ElementType, relation.type)
var exp = Expression.Lambda(func, prop, tableParam);
}
This will produce e.Regions... but I'm unable to get the Where part from there...

I know I'm very late in the game with my answer and likely this is not the exact solution you are looking for (still uses the frequently), but maybe it will help you and others building their expression:
/*
example: Session.Query.Where(m => m.Regions.Where(f => f.Name.Equals("test")))
*/
var innerItem = Expression.Parameter(typeof(MyInnerClass), "f");
var innerProperty = Expression.Property(innerItem, "Name");
var innerMethod = typeof(string).GetMethod("Equals", new[] { typeof(string) });
var innerSearchExpression = Expression.Constant(searchString, typeof(string));
var innerMethodExpression = Expression.Call(innerProperty, innerMethod, new[] { innerSearchExpression });
var innerLambda = Expression.Lambda<Func<MyInnerClass, bool>>(innerMethodExpression, innerItem);
var outerItem = Expression.Parameter(typeof(MyOuterClass), "m");
var outerProperty = Expression.Property(outerItem, info.Name);
/* calling a method extension defined in Enumerable */
var outerMethodExpression = Expression.Call(typeof(Enumerable), "Where", new[] { typeof(MyInnerClass) }, outerProperty, innerLambda);
var outerLambda = Expression.Lambda<Func<MyOuterClass, bool>>(outerMethodExpression, outerItem);
query = query.Where(outerLambda);
Based on an answer posted here: Creating a Linq expression dynamically containing a subquery.

Try this, it's not pretty but it gives you a valid expression for the whole structure. You could define the inner lambda as an expression but you would still have to compile it before you could pass it to Where(), so for the purposes of this answer it seems redundant.
Expression<Func<Product, IEnumerable<Region>>> getRegions =
p => p.Regions.Where(r => r.Country == "Canada");