I'm struggling a bit to understand how to construct an expression that consumes a delegate. I'm new to expressions and embarrassingly wasn't able to create a unit test that replicated the problem I'm seeing, so hopefully the information below will be sufficient to explain the problem.
Consider the following classes:
public class Instance
{
internal Instance(string value)
{
Value = value;
}
public string Value { get; }
public Expression<Func<Container, bool>> IsContainerMatch => c => Selector(c.Item) == Value;
public Func<Item, string> Selector => i => i.Value;
}
public class Container
{
internal Container(Item item)
{
Item = item;
}
public Item Item { get; }
}
public class Item
{
internal Item(string value)
{
Value = value;
}
public string Value { get; }
}
The IsContainerMatch expression is used as an argument on a per-Instance basis for a third party method and is compiled/used at a later time. When the expression is actually called, however, I get an error that states that the variablecis referenced from scope '' but is not defined. If I'm not mistaken, this problem can be solved if I can incorporate the Selector delegate into the expression, so that the two have the same scope. (Please do correct me if I'm wrong!)
I came across this issue, but one fundamental difference I see is that the argument to my delegate is not a constant; it's determined at compile time. I haven't had much luck figuring out how to construct the expression for my scenario. Could someone provide me with a little guidance?
EDIT: This is the simplest test I can construct that fails - sorry it's not readily repeatable. The issue only occurs when I attempt to use the expression in conjunction with NHibernate; calling the func works fine when I use the method from #juharr. When I inject the expression into the .And statement, I get the error message variable c of type Container referenced from scope '', but it is not defined.
[Fact]
public void Test()
{
var instanceList = new Collection<Instance>{new Instance("test")};
var dbConfig = OracleDataClientConfiguration.Oracle10
.ConnectionString(c => c.Is("Data Source=(DESCRIPTION =(ADDRESS = (PROTOCOL = TCP)(HOST = 10.11.12.13)(PORT = 1521))(CONNECT_DATA = (SERVER = DEDICATED)(SERVICE_NAME = orcl.test.com)));Password=test;User ID=test;"))
.Driver<NHibernate.Driver.OracleManagedDataClientDriver>();
FluentConfiguration configuration = Fluently.Configure().Database(dbConfig)
.Mappings(m => m.FluentMappings.AddFromAssembly(Assembly.GetExecutingAssembly()));
var sessionFactory = configuration.BuildSessionFactory();
var session = sessionFactory.OpenSession();
var query = session.QueryOver<Container>();
foreach (var element in instanceList) query.And(c => element.Selector(c.Item) == element.Value);
var query2 = session.QueryOver<Container>();
foreach (var element in instanceList) query.And(element.IsContainerMatch);
}
EDIT #2: Note the query2 above; it is a second use case I have. The first query is the one that throws the exception, but my intent is to reuse the delegate for both the first query as well as in the expression for the second query.
Ok, lets unpack this;
public Expression<Func<Container, bool>> IsContainerMatch => c => Selector(c.Item) == Value;
IsContainerMatch is a property, returning a new instance of an expression on each get. The expression contains constant references to the Instance to access Value and Selector. This is roughly equivalent to;
public Expression<Func<Container, bool>> IsContainerMatch { get {
var inst = Expression.Constant(this);
var container = Expression.Parameter(typeof(Container), "c");
var selector = typeof(Instance).GetProperty("Selector");
var value = typeof(Instance).GetProperty("Value");
var item = typeof(Container).GetProperty("Item");
return Expression.Lambda<Func<Container, bool>>(
Expression.Equal(
Expression.Invoke(
Expression.MakeMemberAccess(inst, selector),
Expression.MakeMemberAccess(container, item)
),
Expression.MakeMemberAccess(inst, value)
),
container
);
} }
This is unlikely to be the true source of your exception. Somewhere else a new LambdaExpression has been constructed, perhaps from pieces of this Expression, with a reference to this ParameterExpression 'C'. But with a different parameter.
For example something like this might cause that Exception;
...
return Expression.Lambda<Func<Container, bool>>(
Expression.Equal(
Expression.Invoke(
Expression.MakeMemberAccess(inst, selector),
Expression.MakeMemberAccess(Expression.Parameter(typeof(Container), "X"), item)
),
Expression.MakeMemberAccess(inst, value)
),
Expression.Parameter(typeof(Container), "Y")
);
Clearly you are attempting to use a type of Expression that your 3rd party library doesn't support.
Now that you've updated the question to include NHibernate, it seems like what you're trying to achieve is;
foreach (var element in instanceList) query.And(c => c.Item.[Dynamic member] == element.Value);
So that the criteria can be evaluated efficiently by NHibernate. But since your Selector is a compiled Func, there's no way for NHibernate to look inside that method and translate this into an efficient query.
Related
The ultimate objective is to have a piece of code that you can feed definitions of simple queries to and it generate data.
so given i can create this query
var query =
from cha in ds.Channels
select new object[]
{
cha.ChanLanguagestreamlocking,
cha.ChanMinimumtrailerduration,
from link in cha.Channelaudiolanguagelinks
select new object[]
{
link.ObjDatecreated
}
};
var data = query.ToArray();
how would i create it dynamically?
well i can get this far using a class stolen from another stack overflow question
public class SelectList<TSource>
{
private List<LambdaExpression> members = new List<LambdaExpression>();
public SelectList<TSource> Add<TValue>(Expression<Func<TSource, TValue>> selector)
{
members.Add(selector);
return this;
}
public Expression<Func<TSource, object[]>> ToDynamicColumns()
{
var parameter = Expression.Parameter(typeof(TSource), "e");
return Expression.Lambda<Func<TSource, object[]>>(
Expression.NewArrayInit(
typeof(object),
members.Select(m =>
Expression.Convert(Expression.Invoke(m, parameter), typeof(object))
)
),
parameter);
}
}
Imagine a case where the definition of which 'columns' to return is passed as a tree at runtime, this can then be mapped to predefined typessafe lambdas that define which columns to return and then the expression is created at runtime. So a hard coded version of this technique is this:
var channelColumns = new SelectList<Channel>();
channelColumns.Add(c => c.ChanLanguagestreamlocking);
channelColumns.Add(c => c.ChanMinimumtrailerduration);
var channelQuery =
ds.Channels.Select(channelColumns.ToDynamicColumns());
var bar = query.ToArray();
i.e. i can generate dynamic queries from the 'root' concept, but how do i generate the nested data.
If i do the obvious i.e. this
var audioColumns = new SelectList<Channelaudiolanguagelink>();
audioColumns.Add(a => a.ObjDatecreated);
var channelColumns = new SelectList<Channel>();
channelColumns.Add(c => c.ChanLanguagestreamlocking);
channelColumns.Add(c => c.ChanMinimumtrailerduration);
// next line causes an error
// Error CS1929 'ICollection<Channelaudiolanguagelink>' does not contain a definition for
// 'Select' and the best extension method overload
// 'Queryable.Select<Channel, object[]>(IQueryable<Channel>, Expression<Func<Channel, object[]>>)' requires a receiver of type 'IQueryable<Channel>' CSharpDb2Raw C:\Users\mark.nicholls\source\repos\scaffold2\CSharpDb2Raw\Program.cs 57 Active
channelColumns.Add(c => c.Channelaudiolanguagelinks.Select(channelColumns.ToDynamicColumns()));
var channelQuery =
ds.Channels.Select(channelColumns.ToDynamicColumns());
var bar = query.ToArray();
the error makes perfect sense.
c.Channelaudiolanguagelinks is an ICollection and so the select is looking for a Func<T,U> and I've given it an Expression<Func<T,U>>
(I don't really understand Expressions!)
Problem that you have defined method for IQueryable version of Select, so basic solution is simple - transform IEnumerable to IQueryable.
channelColumns.Add(c =>
c.Channelaudiolanguagelinks.AsQueryable().Select(audioColumns.ToDynamicColumns())
);
I have an IQueryable<T> from my DbSet in Entity Framework. I am provided a "Fuzzy Search String", named searchText, like so:
public List<T> Search<T>(string searchText)
{
using (var context = ...)
{
var baseQuery = context.Set<T>().AsQueryable();
baseQuery = baseQuery.Where(x =>
DbFunctions.Like(x.PropertyName, searchText)
|| DbFunctions.Like(x.PropertyTwo, searchText)
|| DbFunctions.Like(x.PropertyThree, searchText)
|| DbFunctio..... etc
);
return baseQuery.ToList();
}
}
But given the generic nature, I don't know what properties there are on the type. I can provide an abstract method to somebody implementing this which allows them to give me a List of Properties (or even PropertyInfo or whatever else, I can figure that out). But I don't know how to dynamically create the expression. This is what I have so far:
var baseQuery = context.Set<T>().AsQueryable();
var expression = baseQuery.Expression;
var colName = "colName"; // Or names, I can iterate.
var parameter = Expression.Parameter(typeof(T), "x");
var selector = Expression.PropertyOrField(parameter, colName);
expression = Expression.Call(typeof(DbFunctions), nameof(DbFunctions.Like),
new Type[] { baseQuery.ElementType, selector.Type },
expression, Expression.Quote(Expression.Lambda(selector, parameter)));
The problem here is... well, it doesn't work to begin with. But mainly that I'm not using the searchText anywhere in it, and don't know how to plug it in. I THINK I'm close... but have spent an inordinate amount of time on it.
Hopefully I'm getting your query logic right: if you want to build a set of LIKE conditions based on known type and list of column names, you could try something like this:
static private MethodInfo dbLikeMethod = typeof(DbFunctions).GetMethod(nameof(DbFunctions.Like), BindingFlags.Public | BindingFlags.Static, null, new Type[] {typeof(string), typeof(string)}, null); // I am targeting DbFunctions.Like(string, string). You might want another overload (or even mix them up depending on your inputs)
public List<T> Search<T>(string searchText) where T: class
{
using (var context = new ...)
{
var baseQuery = context.Set<T>().AsQueryable().Where(CreateExpression<T>(searchText));// you could probably find a more elegant way of plugging it into your query
return baseQuery.ToList();
}
}
Expression<Func<T, bool>> CreateExpression<T>(string searchText) where T : class
{
var cols = new List<string> {
"PropertyName",
"PropertyTwo" // i understand you've got a way to figure out which strings you need here
};
var parameter = Expression.Parameter(typeof(T), "x");
var dbLikeCalls = cols.Select(colName => Expression.Call(dbLikeMethod, Expression.PropertyOrField(parameter, colName), Expression.Constant(searchText))); // for convenience, generate list of DbFunctions.Like(x.<Property>, searchText) expressions here
var aggregatedCalls = dbLikeCalls.Skip(1).Aggregate((Expression)dbLikeCalls.First(), (accumulate, call) => Expression.OrElse(accumulate, call)); // aggregate the list using || operators: use first item as a seed and keep adding onto it
return Expression.Lambda<Func<T, bool>>(aggregatedCalls, parameter);
}
I am using Dynamic Linq to perform some queries (sorry but it's my only option). As a result, I am getting an IQueryable instead of an IQueryable<T>. In my case, I want an IQueryable<Thing> where Thing is a concrete type.
My query is as such:
public IQueryable<Thing> Foo(MyContext db)
{
var rootQuery = db.People.Where(x => x.City != null && x.State != null);
var groupedQuery = rootQuery.GroupBy("new ( it.City, it.State )", "it", new []{"City", "State"});
var finalLogicalQuery = groupedQuery.Select("new ( Count() as TotalNumber, Key.City as City, Key.State as State )");
var executionDeferredResults = finalLogicalQuery.Take(10); // IQueryable
IQueryable<Thing> executionDeferredTypedThings = ??; // <--- Help here!!!!
return executionDeferredTypedThings;
}
My Thing.cs:
public class Thing
{
public int TotalNumber { get; set; }
public string City { get; set; }
public string State { get; set; }
}
Yes, I know the exact above thing can be done without Dynamic Linq but I have some variableness going on that I've simplified out of here. I can get it to work with my variableness if my return type is simply IQueryable but I can't figure out how to convert to IQueryable<Thing> while keeping it execution-deferred and while also keeping Entity Framework happy. I do have the dynamic Select always returning something (with the correct data) that looks like a Thing. But I simply can't figure how to return the IQueryable<Thing> and could use some help there. Thanks!!
Failed Attempt 1
Based on Rex M's suggestion, I am now trying to use AutoMapper to solve this problem (although I am not committed to this approach and am willing to try other approaches). For the AutoMapper approach, I am doing it as such:
IQueryable<Thing> executionDeferredTypedThings = executionDeferredResults.ProjectTo<Thing>(); // <--- Help here!!!!
But this results in an InvalidOperationException:
Missing map from DynamicClass2 to Thing. Create using Mapper.CreateMap.
The thing is, while I have defined Thing, I have not defined DynamicClass2 and as such, I cannot map it.
Failed Attempt 2
IQueryable<Thing> executionDeferredTypedThings = db.People.Provider.CreateQuery<Thing>(executionDeferredResults.Expression);
This gives an InvalidCastException and seems to be the same underlying problem that the above AutoMapper fail hits:
Unable to cast object of type 'System.Data.Entity.Infrastructure.DbQuery'1[DynamicClass2]' to type 'System.Linq.IQueryable'1[MyDtos.Thing]'.
You can use AutoMapper's Queryable Extensions to produce an IQueryable which wraps the underlying IQueryable, thus preserving the original IQueryable's IQueryProvider and the deferred execution, but adds in a mapping/translating component to the pipeline to convert from one type to another.
There's also AutoMapper's UseAsDataSource which makes some common query extension scenarios easier.
If I understand correctly, the following extension method should do the job for you
public static class DynamicQueryableEx
{
public static IQueryable<TResult> Select<TResult>(this IQueryable source, string selector, params object[] values)
{
if (source == null) throw new ArgumentNullException("source");
if (selector == null) throw new ArgumentNullException("selector");
var dynamicLambda = System.Linq.Dynamic.DynamicExpression.ParseLambda(source.ElementType, null, selector, values);
var memberInit = dynamicLambda.Body as MemberInitExpression;
if (memberInit == null) throw new NotSupportedException();
var resultType = typeof(TResult);
var bindings = memberInit.Bindings.Cast<MemberAssignment>()
.Select(mb => Expression.Bind(
(MemberInfo)resultType.GetProperty(mb.Member.Name) ?? resultType.GetField(mb.Member.Name),
mb.Expression));
var body = Expression.MemberInit(Expression.New(resultType), bindings);
var lambda = Expression.Lambda(body, dynamicLambda.Parameters);
return source.Provider.CreateQuery<TResult>(
Expression.Call(
typeof(Queryable), "Select",
new Type[] { source.ElementType, lambda.Body.Type },
source.Expression, Expression.Quote(lambda)));
}
}
(Side note: Frankly I have no idea what values argument is for, but added it to match the corresponding DynamicQueryable.Select method signature.)
So your example will become something like this
public IQueryable<Thing> Foo(MyContext db)
{
var rootQuery = db.People.Where(x => x.City != null && x.State != null);
var groupedQuery = rootQuery.GroupBy("new ( it.City, it.State )", "it", new []{"City", "State"});
var finalLogicalQuery = groupedQuery.Select<Thing>("new ( Count() as TotalNumber, Key.City as City, Key.State as State )"); // IQueryable<Thing>
var executionDeferredTypedThings = finalLogicalQuery.Take(10);
return executionDeferredTypedThings;
}
How it works
The idea is quite simple.
The Select method implementation inside the DynamicQueryable looks something like this
public static IQueryable Select(this IQueryable source, string selector, params object[] values)
{
if (source == null) throw new ArgumentNullException("source");
if (selector == null) throw new ArgumentNullException("selector");
LambdaExpression lambda = DynamicExpression.ParseLambda(source.ElementType, null, selector, values);
return source.Provider.CreateQuery(
Expression.Call(
typeof(Queryable), "Select",
new Type[] { source.ElementType, lambda.Body.Type },
source.Expression, Expression.Quote(lambda)));
}
What it does is to dynamically create a selector expression and bind it to the source Select method. We take exactly the same approach, but after modifying the selector expression created by the DynamicExpression.ParseLambda call.
The only requirement is that the projection is using "new (...)" syntax and the names and types of the projected properties match, which I think fits in your use case.
The returned expression is something like this
(source) => new TargetClass
{
TargetProperty1 = Expression1(source),
TargetProperty2 = Expression2(source),
...
}
where TargetClass is a dynamically generated class.
All we want is to keep the source part and just replace that target class/properties with the desired class/properties.
As for the implementation, first the property assignments are converted with
var bindings = memberInit.Bindings.Cast<MemberAssignment>()
.Select(mb => Expression.Bind(
(MemberInfo)resultType.GetProperty(mb.Member.Name) ?? resultType.GetField(mb.Member.Name),
mb.Expression));
and then the new DynamicClassXXX { ... } is replaced with with
var body = Expression.MemberInit(Expression.New(resultType), bindings);
Would something like this be of benefit to you?
public static IQueryable<TEntity> GetQuery<TEntity>(this DbContext db, bool includeReferences = false) where TEntity : class
{
try
{
if (db == null)
{
return null;
}
var key = typeof(TEntity).Name;
var metaWorkspace = db.ToObjectContext().MetadataWorkspace;
var workspaceItems = metaWorkspace.GetItems<EntityType>(DataSpace.OSpace);
var workspaceItem = workspaceItems.First(f => f.FullName.Contains(key));
var navProperties = workspaceItem.NavigationProperties;
return !includeReferences
? db.Set<TEntity>()
: navProperties.Aggregate((IQueryable<TEntity>)db.Set<TEntity>(), (current, navProperty) => current.Include(navProperty.Name));
}
catch (Exception ex)
{
throw new ArgumentException("Invalid Entity Type supplied for Lookup", ex);
}
}
You may want to take a look into the Generic Search project on Github located here:
https://github.com/danielpalme/GenericSearch
There is no need for Dynamic Linq on this one.
var groupedQuery = from p in db.People
where p.City != null && p.State != null
group p by new {p.City, p.State}
into gp
select new Thing {
TotalNumber = gp.Count(),
City = gp.Key.City,
State = gp.Key.State
};
IQueryable<Thing> retQuery = groupedQuery.AsQueryable();
retQuery= retQuery.Take(10);
return retQuery;
I'm getting the error Unable to cast object of type 'System.Linq.Expressions.FieldExpression' to type 'System.Linq.Expressions.LambdaExpression' when running the below code.
The intent of this code is to allow me to filter records (Entity Framework Code First / Linq to SQL) for those containing certain strings.
NB: I'm using the third party library LinqKit: http://www.albahari.com/nutshell/predicatebuilder.aspx
FilterHelper<Country> helper = new FilterHelper<Country>();
helper.AddContains(searchAlpha2, c => c.Alpha2);
helper.AddContains(searchAlpha3, c => c.Alpha3);
helper.AddContains(searchName, c => c.Name);
IQueryable<Country> countries = db.Countries.AsExpandable().Where(helper.Predicate);
...
public class FilterHelper<T>
{
public delegate string GetColumn<T>(T item);
private Expression<Func<T,bool>> predicate;
public FilterHelper()
{
this.predicate = PredicateBuilder.True<T>();
}
public void AddContains(string searchText, GetColumn<T> getColumn)
{
if (!string.IsNullOrWhiteSpace(searchText))
predicate = predicate.And(c => getColumn(c) != null ? getColumn(c).Contains(searchText) : false);
}
public Expression<Func<T,bool>> Predicate
{
get { return this.predicate; }
}
}
Any suggestions on how I could rewrite this to avoid the above error?
NB: code also on CodeReview as my original question related to refactoring. https://codereview.stackexchange.com/questions/54888/refactor-c-linq-code-to-reduce-duplication
A FieldExpression is not a LambdaExpression. You can make one as the body of an expression. In the debugger you can try the following to explore the structure of a LambdaExpression:
Expression<Func<U,T>> f = t => t.FieldName;
then in the debugger, look at f, which is of type LambdaExpression, and the Body of LambdaExpression which is Field Expression.
I want to create lambda something like this
user => user.Address == address
but is not compiled one, I want to return LambdaExpression
If the lambda take constant like this
user => user.Age == 50
Then I can use this method
public static LambdaExpression PropertyEqual(Type tEntityType, string propertyName, object value)
{
// entity => entity.PropName == const
var itemParameter = Expression.Parameter(tEntityType, "entity");
return Expression.Lambda
(
Expression.Equal
(
Expression.Property
(
itemParameter,
propertyName
),
Expression.Constant(value) // Tried to replace this with Expression.Parameter or Expression.Variable but no luck
),
new[] { itemParameter }
);
}
How to make this method accept variable address come from the scope just outside from the lambda expression?
var addressPropertyName = "Address";
var address = new Address() {...};
var q = Repo.GetQuery().Where(PropertyEqual(typeof(User), addressPropertyName, address))
Edit: clarify my question: How build the right Expression to generate the first lambda?
Update: This is not possible because EF does not support non-scalar variable
I change the lambda to user => user.AddressId == addressId as suggested here. It just the matter how to get AddressId FK PropertyInfo from a known navigation property Address.
You can't dynamically generate a closure on a variable (you can't extend the lifetime of a variable outside its context) because this is a trick of the compiler (that rewrites your code to do it).
If you don't want a closure but you want an additional parameter then you can add an additional parameter to the expression.
You could
Expression<Func<string>> myExpr = () => address;
now you have an expression that closes around your address. Now you only have to combine the two expressions.
You'll have to change the method to:
public static LambdaExpression PropertyEqual<T>(Type tEntityType, string propertyName, Expression<Func<T>> getValue)
{
// entity => entity.PropName == const
var itemParameter = Expression.Parameter(tEntityType, "entity");
return Expression.Lambda
(
Expression.Equal
(
Expression.Property
(
itemParameter,
propertyName
),
Expression.Invoke(getValue) // You could directly use getValue.Body instead of Expression.Invoke(getValue)
),
new[] { itemParameter }
);
}