We Keep Coding

Dynamically generating nested queries

The ultimate objective is to have a piece of code that you can feed definitions of simple queries to and it generate data.
so given i can create this query
var query =
from cha in ds.Channels
select new object[]
{
cha.ChanLanguagestreamlocking,
cha.ChanMinimumtrailerduration,
from link in cha.Channelaudiolanguagelinks
select new object[]
{
link.ObjDatecreated
}
};
var data = query.ToArray();
how would i create it dynamically?
well i can get this far using a class stolen from another stack overflow question
public class SelectList<TSource>
{
private List<LambdaExpression> members = new List<LambdaExpression>();
public SelectList<TSource> Add<TValue>(Expression<Func<TSource, TValue>> selector)
{
members.Add(selector);
return this;
}
public Expression<Func<TSource, object[]>> ToDynamicColumns()
{
var parameter = Expression.Parameter(typeof(TSource), "e");
return Expression.Lambda<Func<TSource, object[]>>(
Expression.NewArrayInit(
typeof(object),
members.Select(m =>
Expression.Convert(Expression.Invoke(m, parameter), typeof(object))
)
),
parameter);
}
}
Imagine a case where the definition of which 'columns' to return is passed as a tree at runtime, this can then be mapped to predefined typessafe lambdas that define which columns to return and then the expression is created at runtime. So a hard coded version of this technique is this:
var channelColumns = new SelectList<Channel>();
channelColumns.Add(c => c.ChanLanguagestreamlocking);
channelColumns.Add(c => c.ChanMinimumtrailerduration);
var channelQuery =
ds.Channels.Select(channelColumns.ToDynamicColumns());
var bar = query.ToArray();
i.e. i can generate dynamic queries from the 'root' concept, but how do i generate the nested data.
If i do the obvious i.e. this
var audioColumns = new SelectList<Channelaudiolanguagelink>();
audioColumns.Add(a => a.ObjDatecreated);
var channelColumns = new SelectList<Channel>();
channelColumns.Add(c => c.ChanLanguagestreamlocking);
channelColumns.Add(c => c.ChanMinimumtrailerduration);
// next line causes an error
// Error CS1929 'ICollection<Channelaudiolanguagelink>' does not contain a definition for
// 'Select' and the best extension method overload
// 'Queryable.Select<Channel, object[]>(IQueryable<Channel>, Expression<Func<Channel, object[]>>)' requires a receiver of type 'IQueryable<Channel>' CSharpDb2Raw C:\Users\mark.nicholls\source\repos\scaffold2\CSharpDb2Raw\Program.cs 57 Active
channelColumns.Add(c => c.Channelaudiolanguagelinks.Select(channelColumns.ToDynamicColumns()));
var channelQuery =
ds.Channels.Select(channelColumns.ToDynamicColumns());
var bar = query.ToArray();
the error makes perfect sense.
c.Channelaudiolanguagelinks is an ICollection and so the select is looking for a Func<T,U> and I've given it an Expression<Func<T,U>>
(I don't really understand Expressions!)

Problem that you have defined method for IQueryable version of Select, so basic solution is simple - transform IEnumerable to IQueryable.
channelColumns.Add(c =>
c.Channelaudiolanguagelinks.AsQueryable().Select(audioColumns.ToDynamicColumns())
);

How to consume a delegate within an expression

I'm struggling a bit to understand how to construct an expression that consumes a delegate. I'm new to expressions and embarrassingly wasn't able to create a unit test that replicated the problem I'm seeing, so hopefully the information below will be sufficient to explain the problem.
Consider the following classes:
public class Instance
{
internal Instance(string value)
{
Value = value;
}
public string Value { get; }
public Expression<Func<Container, bool>> IsContainerMatch => c => Selector(c.Item) == Value;
public Func<Item, string> Selector => i => i.Value;
}
public class Container
{
internal Container(Item item)
{
Item = item;
}
public Item Item { get; }
}
public class Item
{
internal Item(string value)
{
Value = value;
}
public string Value { get; }
}
The IsContainerMatch expression is used as an argument on a per-Instance basis for a third party method and is compiled/used at a later time. When the expression is actually called, however, I get an error that states that the variablecis referenced from scope '' but is not defined. If I'm not mistaken, this problem can be solved if I can incorporate the Selector delegate into the expression, so that the two have the same scope. (Please do correct me if I'm wrong!)
I came across this issue, but one fundamental difference I see is that the argument to my delegate is not a constant; it's determined at compile time. I haven't had much luck figuring out how to construct the expression for my scenario. Could someone provide me with a little guidance?
EDIT: This is the simplest test I can construct that fails - sorry it's not readily repeatable. The issue only occurs when I attempt to use the expression in conjunction with NHibernate; calling the func works fine when I use the method from #juharr. When I inject the expression into the .And statement, I get the error message variable c of type Container referenced from scope '', but it is not defined.
[Fact]
public void Test()
{
var instanceList = new Collection<Instance>{new Instance("test")};
var dbConfig = OracleDataClientConfiguration.Oracle10
.ConnectionString(c => c.Is("Data Source=(DESCRIPTION =(ADDRESS = (PROTOCOL = TCP)(HOST = 10.11.12.13)(PORT = 1521))(CONNECT_DATA = (SERVER = DEDICATED)(SERVICE_NAME = orcl.test.com)));Password=test;User ID=test;"))
.Driver<NHibernate.Driver.OracleManagedDataClientDriver>();
FluentConfiguration configuration = Fluently.Configure().Database(dbConfig)
.Mappings(m => m.FluentMappings.AddFromAssembly(Assembly.GetExecutingAssembly()));
var sessionFactory = configuration.BuildSessionFactory();
var session = sessionFactory.OpenSession();
var query = session.QueryOver<Container>();
foreach (var element in instanceList) query.And(c => element.Selector(c.Item) == element.Value);
var query2 = session.QueryOver<Container>();
foreach (var element in instanceList) query.And(element.IsContainerMatch);
}
EDIT #2: Note the query2 above; it is a second use case I have. The first query is the one that throws the exception, but my intent is to reuse the delegate for both the first query as well as in the expression for the second query.

Ok, lets unpack this;
public Expression<Func<Container, bool>> IsContainerMatch => c => Selector(c.Item) == Value;
IsContainerMatch is a property, returning a new instance of an expression on each get. The expression contains constant references to the Instance to access Value and Selector. This is roughly equivalent to;
public Expression<Func<Container, bool>> IsContainerMatch { get {
var inst = Expression.Constant(this);
var container = Expression.Parameter(typeof(Container), "c");
var selector = typeof(Instance).GetProperty("Selector");
var value = typeof(Instance).GetProperty("Value");
var item = typeof(Container).GetProperty("Item");
return Expression.Lambda<Func<Container, bool>>(
Expression.Equal(
Expression.Invoke(
Expression.MakeMemberAccess(inst, selector),
Expression.MakeMemberAccess(container, item)
),
Expression.MakeMemberAccess(inst, value)
),
container
);
} }
This is unlikely to be the true source of your exception. Somewhere else a new LambdaExpression has been constructed, perhaps from pieces of this Expression, with a reference to this ParameterExpression 'C'. But with a different parameter.
For example something like this might cause that Exception;
...
return Expression.Lambda<Func<Container, bool>>(
Expression.Equal(
Expression.Invoke(
Expression.MakeMemberAccess(inst, selector),
Expression.MakeMemberAccess(Expression.Parameter(typeof(Container), "X"), item)
),
Expression.MakeMemberAccess(inst, value)
),
Expression.Parameter(typeof(Container), "Y")
);
Clearly you are attempting to use a type of Expression that your 3rd party library doesn't support.
Now that you've updated the question to include NHibernate, it seems like what you're trying to achieve is;
foreach (var element in instanceList) query.And(c => c.Item.[Dynamic member] == element.Value);
So that the criteria can be evaluated efficiently by NHibernate. But since your Selector is a compiled Func, there's no way for NHibernate to look inside that method and translate this into an efficient query.

Dynamic where clause to filter collection elements

I have a collection List<List<object>>, which I need to filter out based on wheater the List<object> collection contains given element. I was able to build the where clause, but i get the following Exception:
An exception of type 'System.InvalidOperationException' occurred in System.Core.dll but was not handled in user code
Additional information: variable 'x' of type 'System.Collections.Generic.List`1[System.Object]' referenced from scope '', but it is not defined
I have found similar issues and i understand where the problem is, but i need help finding the solution.
Here is my code:
protected override Expression<Func<List<object>, bool>> GetWhereClause()
{
var type = typeof(List<object>);
var parameterExpression = Expression.Parameter(type, "x");
Expression expressionBody = null;
if (Verified)
{
Expression<Func<List<object>, bool>> expression = x => x.Contains("Verified");
expressionBody = expression.Body;
}
if (GoodMatch)
{
Expression<Func<List<object>, bool>> expression = x => x.Contains("Good Match");
if (expressionBody != null)
expressionBody = Expression.Or(expressionBody, expression.Body);
else
expressionBody = expression.Body;
}
//More conditions here
if (expressionBody != null)
{
var whereClauseExp = Expression.Lambda<Func<List<object>, bool>>(expressionBody, parameterExpression);
return whereClauseExp;
}
return null;
}
Now, this method generates the desired where clause, but when i try to apply it, i get the mentioned Exception.
if (whereClause != null)
{
items = items.Where(whereClause.Compile());
//
}

I had a similar use case requiring dynamic where clauses and used Predicate Builder
Using it, you could do something like:*
private Expression<Func<List<T>, bool>> GetWhereClause<T>(T itemToFind){
var predicate = PredicateBuilder.False<List<T>>();
if(Verified) {
predicate = predicate.And(p => p.Contains(itemToFind));
}
if(GoodMatch) {
predicate = predicate.Or(p => p.Contains(itemToFind));
}
return predicate;
}

You cannot use a parameter from another expression 'as is' in a new expression. When you do:
Expression<Func<List<object>, bool>> expression = x => x.Contains("Verified");
expressionBody = expression.Body;
then you simply have the inline defined parameter x in the body. Now this parameter IS NOT the same parameter, that you defined before as
var parameterExpression = Expression.Parameter(type, "x");
even if they both have the name x, that is not enough. Expression trees have reference equality.
So to get it to work, just use a visitor, that will replace the parameter with yours. Create a visitor:
public class ParameterUpdateVisitor : ExpressionVisitor
{
private ParameterExpression _parameter;
public ParameterUpdateVisitor(ParameterExpression parameter)
{
_parameter = parameter;
}
protected override Expression VisitParameter(ParameterExpression node)
{
return _parameter;
}
}
and then in your code use it like:
Expression<Func<List<object>, bool>> expression = x => x.Contains("Verified");
var visitor = new ParameterUpdateVisitor(parameterExpression);
expressionBody = visitor.Visit(expression.Body);
Of course this for every part that comes from another expression tree.
NOTE!!! this visitor is extra simplified, just for your example. If you have expressions that might have methods that have their own parameters, then make sure to replace only the parameter that you want to!
E.g. it wont work for:
Expression<Func<List<object>, bool>> expression = x => x.Select(o => o.ToString()).Contains("Verified");
because this visitor will replace the 'o' too. If you have this case, then pass in the parameter you want to replace (e.g. x, that is expression.Parameters.First()) in the constructor too, and only replace in the overridden method if node == myOldParameter.
BTW: why do you need expression trees if you compile the thing at the end anyways?

Dynamic Select EntityFramework and System.Linq.Dynamic

Im using the DynamicQueryable.Select() // dynamic select library
The only docu I have found so far is System.Linq.Dynamic docu
I have this query with anonymous return type which works.
var result = Context.Set<TPocoText>().Where((Expression<Func<TPocoText, bool>>) whereLambda)
.OrderByDescending(t => t.RowVersion).Skip(skip).Take(take)
.Select("new (m1,m2,Nav1) ");
and this works like select(t=> new {t.m1,t.m2,t.Nav1}) as expected
my Question
How can it do the equivalent of select(t=> new {t,t.Nav1})
i tried .Select("new (it,Nav1) ")
and .Select("new (this,Nav1) ")
the result was a parse error member not found.
Anybody know this dynamic string parsing API?
OR the equivalent Expression building syntax is also an option.
NOTE:The Nav Property ForSourceRecord is only known at runtime otherwise i would just use the normal lambda expression.

This cannot be done using anon types. The compiler just doesn't have the information needed to work, especially with the free text/string approach.
Try this...
var param = System.Linq.Expressions.Expression.Parameter(typeof(TPocoText));
var init = System.Linq.Expressions.Expression.MemberInit(
System.Linq.Expressions.Expression.New(typeof(Foo)),
new []{
System.Linq.Expressions.Expression.Bind(GetMemberInfo((Foo f) => f.Nav), System.Linq.Expressions.Expression.PropertyOrField(param, "NameOfPropertyToBindToNav")),
System.Linq.Expressions.Expression.Bind(GetMemberInfo((Foo f) => f.M1), System.Linq.Expressions.Expression.PropertyOrField(param, "M1")),
}
);
var result = Context.Set<TPocoText>().Where((Expression<Func<TPocoText, bool>>) whereLambda)
.OrderByDescending(t => t.RowVersion).Skip(skip).Take(take)
.Select(System.Linq.Expressions.Expression.Lambda<Func<TPocoText, Foo>>(init, param));
public class Foo
{
public string Nav {get;set;}
public string M1 {get;set;}
}
public static MemberInfo GetMemberInfo<T, U>(Expression<Func<T, U>> expression)
{
var member = expression.Body as MemberExpression;
if (member != null)
return member.Member;
throw new ArgumentException("Expression is not a member access", "expression");
}

C# Linq - Nested expression parameters

I have a LINQ expression which calls another expression which also calls another expression...
public static Expression<Func<Models.Order, Models.Product,decimal?>> _ExpressionOfGetCounterValue
{
get
{
var _getAmountProxy = Product._ExpressionOfGetAmount;
var _convertQuantityProxy = Product._ExpressionOfConvertQuantity;
Expression<Func<Models.Order, Models.Product, decimal?>> cv =
(order,product) => (
_getAmountProxy.Invoke(
order.Product,
_convertQuantityProxy.Invoke(order.Product,order.Quantity),
order.Product.Price));
return cv;
}
}
public static Expression<Func<Models.Product, decimal?, decimal?>> _ExpressionOfConvertQuantity
{
get
{
Expression<Func<Models.Product, decimal?, decimal?>> convertQuantity =
(product, quantity) => ModelEntities.DoubleToDecimal(
ModelEntities.DecimalToDouble(quantity.Value)
* Math.Pow(10.0, ModelEntities.DecimalToDouble(product.QuantityDecimals))
);
return convertQuantity;
}
}
When executing this, I get an error InvalidOperationException: The parameter 'order' was not bound in the specified LINQ to Entities query expression.
This exception comes from _convertQuantityProxy .Invoke(order.Product,order.Quantity)
How can I pass the original parameter "order" to this call ?
Please note that I use LinqKit / AsExpandable() function.
Thank you.

Finally I have found, I have to return
return cv.Expand();