We Keep Coding

Building a parameterized EntityFramework Core Expression

Hi, I'm trying to build an Expression to get a generic entity by its primary key and getting a parameterized sql query.
Currently I can get the correct WHERE query, but it isn't parameterized.
public async Task<TDbo> Get(TKey key, Expression<Func<TEntity, TKey>> keySelector)
{
var propertyRef = keySelector.Body;
var parameter = keySelector.Parameters[0];
var constantRef = Expression.Constant(key);
var equals = Expression.Equal(propertyRef, constantRef);
var comparer = Expression.Lambda<Func<TEntity, bool>>(equals, parameter);
return await _context.Set<TDbo>().SingleOrDefaultAsync(comparer);
}
This results in the following query:
SELECT e.\"Id\", e.\"Name\" \r\n FROM \"People\" AS e\r\nWHERE e.\"Id\" = 1\r\nLIMIT 2,
instead of the wanted:
SELECT e.\"Id\", e.\"Name\" \r\n FROM \"People\" AS e\r\nWHERE e.\"Id\" = #__s_0\r\nLIMIT 2

It's because of Expression.Constant(key). Value constant expressions are not parameterized by the query translator. What you need is an expression referring to a property or field of another expression (which could be constant). That's basically what C# compiler emits for closures.
One way is to actually use the C# compiler to create lambda expression with closure and take the body:
Expression<Func<TKey>> keyValue = () => key;
var variableRef = key.Body;
(the variableRef is a replacement of yours constantRef)
Another way is to use anonymous, tuple or specific class type to create explicit closure instance and bind the corresponding property or field. For instance, with anonymous type:
var variableRef = Expression.Property(Expression.Constant(new { key }), "key");
or with System.Tuple:
var variableRef = Expression.Property(Expression.Constant(Tuple.Create(key)), "Item1");
The actual method doesn't really matter (I personally prefer the first variant with lambda) - all they will cause creating parameter by EF Core query translator.

Dynamic lambda expression for where clause

I have a dynamic list of objects on which I am using lambda expression where clause to filter items. For example, just consider that it have 3 properties, foo, bar and baz
class item // let this be current class in dynamic item list
{
bool foo;
string bar;
string baz;
}
Now if I want to filter item list where foo is false I can use following expression
var filtered = itemList.Where("!foo");
I can even filter the list by strings value as
var filtered = itemList.Where("bar==\"value\""); \\all items with bar = value
What I want to actually check is if item in list have a specific string value not null of white space. I tried following code
var filtered = itemList.Where("!String.IsNullOrWhiteSpace(baz)");
It threw an error
Expression of type 'System.Func`2[DynamicType,System.Object]' cannot
be used for parameter of type 'System.String' of method 'Boolean
IsNullOrWhiteSpace(System.String)'
Though I succeeded to get result by following query
var filtered = itemList.Where("baz!=null && baz!=\"\"");
I wanted to confirm if there is a way I can use String.IsNullOrWhiteSpace() in this query.

You can replace "!String.IsNullOrWhiteSpace(baz)" with "!(baz == null || baz.Trim() == string.Empty)" and it should work.

Have a look at System.Linq.Dynamic, there is a great example here.
You will alse need to make sure the List<T> is not List<object>, otherwise System.Linq.Dynamic will not be able to find the properties.
Here is a snippet for your example:
void Main()
{
var itemList = new List<dynamic>{ new {foo = true, bar = "a", baz = "b" }, new {foo = true, bar = (string)null, baz = "d" } };
var filtered = itemList.ToAnonymousList().Where("bar != null and bar !=\"\"");
filtered.Dump();
}
public static class EnumerableEx {
public static IList ToAnonymousList(this IEnumerable enumerable)
{
var enumerator = enumerable.GetEnumerator();
if (!enumerator.MoveNext())
throw new Exception("?? No elements??");
var value = enumerator.Current;
var returnList = (IList) typeof (List<>)
.MakeGenericType(value.GetType())
.GetConstructor(Type.EmptyTypes)
.Invoke(null);
returnList.Add(value);
while (enumerator.MoveNext())
returnList.Add(enumerator.Current);
return returnList;
}
}

I have no problem using your expression - it works fine.
I have used this with objects and entities against an EF storage.
Expression of type 'System.Func`2[DynamicType,System.Object]' cannot
be used for parameter of type 'System.String' of method 'Boolean
IsNullOrWhiteSpace(System.String)'
So looking at the error, it is stating (moving the order around):
The method IsNullOrWhiteSpace, that returns a Boolean, expects a parameter of type System.String.
But what was received was Expression of type System.Func``2[DynamicType,System.Object]'
It appears that you may have referenced an object rather than a string value for your comparison. However, without sample code for the objects you are using, and whether you are using objects, entities, or LinQ to SQL (which I haven't tried) we can only guess at what you have supplied for the expression.
Finally, what is a
'dynamic item list'?
Are you using Dynamic, or is it a normal List<Item>?

Combine property selector expression tree and value to create a predicate for EF filtering - create filter from lambda selector and value

Given a simple class with arbitrary properties (for discussion lets say Id, Name, and Description)
and given an instance of that class, I want to find matching entries in the database by specifying the property to match
I'm trying to do something in this respect similar to the AddOrUpdate method of EF, but I need the entity returned to me for further processing.
var match = new SomeClass{Name="Whatever"};
var found = Context.SomeClass.Find(x=>x.Name, match);
public static T Find<T>(this DbSet<T> set, Expression<Func<T, object>> matchOn, T matchAgainst) where T : class {
var func = matchOn.Compile();
var valueToFind = func(matchAgainst);
var combinedExpression = //matchon + "=" + valueToFind;
var found = set.FirstOrDefault(combinedExpression);
return found;
}
That gives me the value of the property in the passed in object, but I need to now combine that value with the passed in expression and pass it to the db set.
IE, the code I'm effectively trying to run is set.FirstOrDefault(x=>x.Name==valueToFind) How do I take the matchon expression (which contains x=>x.Name) and combine that with the ==valueToFind to get the x=>x.Name==valueToFind from them?
How do I build the combined expression? (I realize the "string" code above is completely wrong, but I was trying to get across what I need the function to do, but I have no idea what that syntax would look like.)
For manually coded examples, it would be easy enough just to pass in a hardcoded lambda with the value set, but my use case involves running through a collection of objects and finding the match for each one, so the value will not be known until runtime, and the method must work against arbitrary types and various properties, so I can't hardcode the property name either.

If you have a property selector, and a value to compare to, you can get an expression tree like this:
public static Func<TEntity, bool> GetComparer<TEntity,TProperty>(
Expression<Func<TEntity,TProperty>> selector, TProperty value)
{
var propertyRef = selector.Body;
var parameter = selector.Parameters[0];
var constantRef = Expression.Constant(value);
var comparer
= Expression.Lambda<Func<TEntity, bool>>
(Expression.Equal(propertyRef, constantRef), parameter)
.Compile();
return comparer;
}
Sample usage:
var comparer = GetComparer<Person, string>(p => p.Name, "John");
var persons = Person.GetPersons();
var john = persons.FirstOrDefault(comparer);

Building a common set of methods that can operate on any linq table

Problem: We make extensive use of a repository pattern to facilitate read/write operations on our datastore (MS SQL using LINQ) across multiple applications and subsections of functionality. We have series of methods that all do something similar to each other.
For example, we have the ProcessAndSortXXXXX class of methods.
private static IEnumerable<ClassErrorEntry> ProcessAndSortClassErrorLog(IQueryable<ClassErrorDb> queryable, string sortOrder)
{
var dynamic = queryable;
if (!String.IsNullOrEmpty(sortOrder.Trim()))
{
dynamic = dynamic.OrderBy(sortOrder);
}
return dynamic
.Select(l =>
new ClassErrorEntry(l.Id)
{
ClassId = l.ClassId,
Code = l.Code,
Message = l.Message,
Severity = l.Severity,
Target = l.Target
}
);
}
...and...
private static IEnumerable<ClassTimerLogEntry> ProcessAndSortClassTimerLog(IQueryable<ClassTimerDb> queryable, string sortOrder)
{
var dynamic = queryable;
if (!String.IsNullOrEmpty(sortOrder.Trim()))
{
dynamic = dynamic.OrderBy(sortOrder);
}
return dynamic
.Select(l =>
new ClassTimerLogEntry(l.Id)
{
ClassName = l.ClassName,
MethodName = l.MethodName,
StartTime = l.StartTime,
EndTime = l.EndTime,
ParentId = l.ParentId,
ExecutionOrder = l.ExecutionOrder
}
);
}
As you can tell by the code, they're all very similar until you look at the signature and then get to the the return statement where we're building out the instances of the ClassErrorEntry and ClassTimerLogEntry.
I want to build a utility method that I'll add into the base class that all of the repositories inherit from.
I want to be able to pass in arguments that can be used to instantiate the objects and pack them into the returning IEnumerable.
I found this post by ScottGu and that gets me most of what I need. It looks like this (from the sample in the documentation):
var query =
db.Customers.
Where("City = #0 and Orders.Count >= #1", "London", 10).
OrderBy("CompanyName").
Select("new(CompanyName as Name, Phone)");
Here's where I get stuck, though. I need a pointer or suggestion how I can pass in the LINQ tables and DataContext in a generic fashion so I can build out the dynamic query.
If I were to mock up the signature in pseudocode I think it would look something like this:
protected internal IEnumerable ProcessAndSort(IQueryable source, string selectClause, string whereClause, string orderByClause);
I realize that the finished signature may look different as we figure this out.
Thank you!
Update!
I now have code that works to generate an anonymous type but fails when converting to the concrete type.
public static IEnumerable<TResult> ProcessAndSort<T, TResult>(IQueryable<T> queryable,
string selector, Expression<Func<T, bool>> predicate, string sortOrder)
{
var dynamic = queryable.Where(predicate).AsQueryable();
if (!String.IsNullOrEmpty(sortOrder.Trim()))
{
dynamic = dynamic.OrderBy(sortOrder);
}
var result= dynamic.Select(selector).Cast<TResult>();
return result;
}
Here is the code that calls this method:
[TestMethod]
public void TestAnonymousClass()
{
var loggingContext = new LoggingDbDataContext(DatabaseConnectionString);
var repo = new LoggingRepository(loggingContext);
var result = repo.TestGetClassErrorLog(4407, 10, 0,
"new ( ClassId as ClassId, " +
"Code as Code, " +
"Message as Message, " +
"Severity as Severity, " +
"Target as Target )", "Target");
TestContext.WriteLine(result.ToList().Count.ToString());
}
The last line TestContext.WriteLine(result.ToList().Count.ToString()); throws the exception System.InvalidOperationException: No coercion operator is defined between types 'DynamicClass1' and 'Utilities.Logging.ClassErrorEntry'.
This chunk of code, though fails:
[TestMethod]
public void TestNamedClass()
{
var loggingContext = new LoggingDbDataContext(DatabaseConnectionString);
var repo = new LoggingRepository(loggingContext);
var result = repo.TestGetClassErrorLog(4407, 10, 0,
"new ClassErrorEntry(Id) { ClassId = ClassId, " +
"Code = Code, " +
"Message = Message, " +
"Severity = Severity, " +
"Target = Target }", "Target");
TestContext.WriteLine(result.ToList().Count.ToString());
}
This fails on a parsing error. Test method eModal.Repositories.Test.RepositoryBaseTest.TestConcreteClass threw exception:
System.Linq.Dynamic.ParseException: '(' expected, found 'ClassErrorEntry' ('Identifier') at char 19 in 'new ClassErrorEntry(Id) { ChassisAuthId = ChassisAuthId, Code = Code, Message = Message, Severity = Severity, Target = Target }'
I'm not sure that the character position is suspectas the 19th character position is a ( and the type passed into the Validate method indicates a position of 4, or the first 'C'.

I would completely advise you against making weakly typed queries just for the sake of code reuse.
Code reuse is for increasing maintainability, but weak typing can kill it, if used in a wrong way.
By writing your queries in plain text, you're effectively making the classes very hard to refactor and change, and introduce a lot of obscure dependencies.
I suggest you take a look at LinqKit that allows to combine Expressions. For example, we wrote a Paging method that splits query by pages and use it across the project with different types:
var query = CompiledQuery.Compile(
BuildFolderExpr( folder, false )
.Select( msg => selector.Invoke( msg, userId ) ) // re-use selector expression
.OrderBy( mv => mv.DateCreated, SortDirection.Descending )
.Paging() // re-use paging expression
.Expand() // LinqKit method that "injects" referenced expressions
)
public static Expression<Func<T1, T2, PagingParam, IQueryable<TItem>>> Paging<T1, T2, TItem>(
this Expression<Func<T1, T2, IQueryable<TItem>>> expr )
{
return ( T1 v1, T2 v2, PagingParam p ) => expr.Invoke( v1, v2 ).Skip( p.From ).Take( p.Count );
}
In my example, BuildMessageExpr returns a relatively simple select expression (which already depends on folder and another parameter), and different methods reuse this expression by applying filtering, ordering, getting count, further selecting with selector expression being passed as a parameter, et cetera. Once the query is created, it gets cached for future usage when parameters are similar.

It is not a direct answer to your question.
As you have said you have quite a lot code that look similar but return different types. If you will go ahead and look for generic implementation of this approach, the result may have a few hacks, you may still pass some uncomfortable SQL or check the type of the object or do some reflection kung-fu. You may still select this pass and actually someone can have a sensible idea that wouldn't look like a dirty hack.
The other option is to use a proper ORM with generic repository pattern and dependency injection(google link). Your data access layer will look much simpler and easier to maintain.

Retrieving an Expression from a property and adding it to an expression tree

I've tried to simplify this example, as the actual code I'm playing with is more complex. So while this example may seem silly, bear with me. Let's say I'm working with the AdventureWorks database and I decide I want to add a property called Blarg to the Product table that returns an expression that contains code I would like to use in several places:
public partial class Product
{
public Expression<Func<Product, string>> Blarg
{
get { return product => product.ProductModelID.HasValue ? "Blarg?" : "Blarg!"; }
}
}
What I want to do is create an expression expression tree, have it get the Expression from Product.Blarg, and group by the result. Something like this:
var productParameter = Expression.Parameter(typeof(Product), "product");
// The Problem
var groupExpression = Expression.Lambda<Func<Product, string>>(
Expression.Invoke(
Expression.Property(productParameter, "Blarg"),
productParameter),
productParameter);
using (AdventureWorksDataContext db = new AdventureWorksDataContext())
{
var result = db.Products.GroupBy(groupExpression).ToList();
// Throws ArgumentException: "The argument 'value' was the wrong type.
// Expected 'System.Delegate'.
// Actual 'System.Linq.Expressions.Expression`1[System.Func`2[LINQ_Test.Product,System.String]]'."
}
Obviously groupExpression is incorrect (see the code comment for the exception), but I'm not sure how I should be doing it. What I thought I was saying is "get the Expression from the product.Blarg, execute it, and return the string result." I guess that's not what I'm actually saying there, though. I'm still trying to figure out expression trees. Any idea how I could pull this off?

When you call Expression.Invoke, the first argument must be an existing LambdaExpression - it can't be an Expression to a LambdaExpression. Or in other words: it isn't going to evaluate Product.Blarg per row and use a different sub-expression each time.
Instead, you would retrieve this lambda first, perhaps making it static and accessing it via reflection if you only know it by name:
var lambda = (LambdaExpression) typeof(Product)
.GetProperty("Blarg").GetValue(null,null);
And pass lambda in as the argument to Expression.Invoke; here's a fully working LINQ-to-Objects example showing this (via AsQueryable()):
using System;
using System.Linq;
using System.Linq.Expressions;
public partial class Product
{
public static Expression<Func<Product, string>> Blarg
{
get { return product => product.ProductModelID.HasValue ? "Blarg?" : "Blarg!"; }
}
public int? ProductModelID { get; set; }
static void Main()
{
var lambda = (LambdaExpression)typeof(Product)
.GetProperty("Blarg").GetValue(null, null);
var productParameter = Expression.Parameter(typeof(Product), "product");
// The Problem
var groupExpression = Expression.Lambda<Func<Product, string>>(
Expression.Invoke(
lambda,
productParameter),
productParameter);
var data = new[] {
new Product { ProductModelID = 123},
new Product { ProductModelID = null},
new Product { ProductModelID = 456},
};
var qry = data.AsQueryable().GroupBy(groupExpression).ToList();
}
}

var qry = data.AsQueryable().GroupBy(Blarg).ToList();
That works, same as Marc's code.
Note: Blarg is already correct, there is no reason to 're-invoke' it.