I'm working on a generic repository for Entity Framework. To search for values I have a Find method on my repository that takes an expression. During this process I want to log the expression. I do this by getting expression.Body.
Here is my code
public async Task<TEntity> FindFirstOrDefault<TEntity>(Expression<Func<TEntity, bool>> expression) where TEntity : class
{
string expBody = expression.Body.ToString();
// Log expBody here
return await _context.Set<TEntity>().Where(expression).FirstOrDefaultAsync().ConfigureAwait(false);
}
The above code works fine when testing with a hard coded condition like this.
var blogs = await uow.Repository.FindFirstOrDefault<Blog>(x => x.BlogId == 1).ConfigureAwait(false);
When I debug that above line has expBody with a value of (x.BlogId == 1)
However if I do the comparison against a variable like this:
int id = 1;
var blogs = await uow.Repository.FindFirstOrDefault<Blog>(x => x.BlogId == id).ConfigureAwait(false);
I end up getting this for expBody:
(x.BlogId == value(TestProject.ValueService+<>c__DisplayClass4_0).intId)
How can I get the actual expression body like when I'm using a hard coded value when I'm actually referencing a variable?
Related
I'm trying to write a piece of code that is going to be used to select a value from a database. As a test I have already created this test version of the method that "works" but isn't great.
private object? FindObjectByProperty(IQueryable query, PropertyInfo primaryKeyProperty, object lookupValue)
{
object? result = null;
foreach (var value in query)
{
if (!primaryKeyProperty.GetValue(value)!.Equals(lookupValue))
continue;
result = value;
break;
}
return result;
}
The code will iterate through the query and find the first entry that matches the Equals. However, if the query contains millions of rows, as it's likely to when the IQueryable is really a DbSet (Which in this use case it is. Only I don't know what T, as it could be any of the DbSets on my EF Core data context.
What I'd like to end up with is something that looks something along these lines....
private object? FindObjectByProperty(IQueryable query, PropertyInfo primaryKeyProperty, object lookupValue)
{
return query.Where( x => x.*primaryKeyProperty* == lookupValue).FirstOrDefault();
}
The above code is pseudo code, the problems that I have is that query does not have a .Where available. Also primaryKeyProperty will need to be translated to the actual property.
The idea is that when this code is executed, ultimately executing the query, will generate a sql statement which selects a single item and returns it.
Can anyone help with solving this?
Update:
I'm working on a solution to this, so far this is what I've come up with
//using System.Linq.Dynamic.Core;
private object? FindObjectByProperty(IQueryable query, PropertyInfo primaryKeyProperty, object lookupValue)
{
var parameter = Expression.Parameter(query.ElementType);
var e1 = Expression.Equal(Expression.Property(parameter, primaryKeyProperty.Name), Expression.Constant(lookupValue));
var lambda = Expression.Lambda<Func<object, bool>>(e1, parameter);
return query.Where(lambda).FirstOrDefault();
}
This is failing on the because the func uses Object, when it really needs the real type. Trying to figure that bit out. This is getting closer.
Update #2:
Here's the answer that I needed
private object? FindObjectByProperty(IQueryable query, PropertyInfo primaryKeyProperty, object lookupValue)
{
var parameter = Expression.Parameter(query.ElementType);
var propExpr = Expression.Property(parameter, primaryKeyProperty);
var lambdaBody = Expression.Equal(propExpr, Expression.Constant(lookupValue, primaryKeyProperty.PropertyType));
var filterEFn = Expression.Lambda(lambdaBody, parameter);
return query.Where(filterEFn).FirstOrDefault();
}
The difference is that the Expression.Lambda no longer tries to define the func using generics. This is the only change that I needed to do to make the code function as I wanted.
In the test case that I was using, the T-SQL produced to lookup the value looks like this...
SELECT TOP(1) [g].[Id], [g].[Deleted], [g].[Guid], [g].[Name], [g].[ParentId]
FROM [Glossaries].[Glossaries] AS [g]
WHERE [g].[Id] = CAST(3 AS bigint)
The table [Glossaries].[Glossaries] is provided by the input query. The column name Id is provided by the primaryKeyProperty, and the number 3 is provided by the lookupValue.
This is perfect for my needs as I simply needed to select that one row and nothing else, so that I can effectively lazy load the my object property when I need it, and not before.
Also this code will be reused for many different tables.
In order to build add Where to an IQueryable where you don't have access to the actual IQueryable<T>, you need to take a step back (or up?) from calling Where at compile-time, and build the Where call at runtime, as well as the predicate lambda:
public static class DBExt {
public static IQueryable WherePropertyIs<T2>(this IQueryable src, PropertyInfo propInfo, T2 propValue) {
// return src.Where(s => s.{propInfo} == propValue)
// (T s)
var sParam = Expression.Parameter(src.ElementType, "s");
// s.propInfo
var propExpr = Expression.Property(sParam, propInfo);
// s.{propInfo} == propValue
var lambdaBody = Expression.Equal(propExpr, Expression.Constant(propValue));
// (T s) => s.{PropInfo} == propValue
var filterEFn = Expression.Lambda(lambdaBody, sParam);
var origQuery = src.Expression;
// IQueryable<Tx>.Where<Tx, Expression<Func<Tx, bool>>>()
var whereGenericMI = typeof(Queryable).GetMethods("Where", 2).Where(mi => mi.GetParameters()[1].ParameterType.GenericTypeArguments[0].GenericTypeArguments.Length == 2).First();
// IQueryable<T>.Where<T, Expression<Func<T, bool>>>()
var whereMI = whereGenericMI.MakeGenericMethod(src.ElementType);
// src.Where(s => s.{propertyInfo} == propValue)
var newQuery = Expression.Call(whereMI, origQuery, filterEFn);
return src.Provider.CreateQuery(newQuery);
}
}
I've inherited some code from a former employee that queries a mongo DB using compiled Linq expressions and the MongoRepository library (which sits on top of the MongoDB C# driver).
These were taking a long time to run - usually around 6 minutes(!) and causing problems with controller methods that used them.
So I've simplified the code and removed the call to .Compile() the lambda expression and this has seemed to solve the issue (takes <10s to run now).
My questions is: why does compiling this expression cause problems when querying mongo?
This was the gist of the original code (hacked out, so out of context sorry):
public class BaseMongoRepository<T> : MongoRepository<T, Guid> where T : IEntity<Guid> {
protected BaseMongoRepository(string connectionString) : base(connectionString) { }
protected bool IsSatisfiedBy(T entity) {
Expression<Func<T, bool>> func = x => x != null && x.ToString() == "foo"; // query was passed in, but you get the idea
var predicate = func.Compile(); // THIS LINE??
return predicate(entity);
}
public IEnumerable<T> Find() {
return base.collection.AsQueryable().Where(IsSatisfiedBy);
}
}
And I simplified it to something that just uses a regular predicate Func:
public IEnumerable<T> Find() {
return base.collection.AsQueryable().Where(x => x != null && x.ToString() == "foo");
}
Any thoughts most appreciated!
The expression can be converted by a provider to a real sql query, but a delegate can not be interpreted.
The predicate variable in this code:
Expression<Func<T, bool>> func = x => x != null && x.ToString() == "foo";
var predicate = func.Compile();
is essentially the same as:
Func<T, bool> predicate = x => x != null && x.ToString() == "foo";
When you use such a delegate, all data from the database is transferred into memory and after that the predicate is applied.
Pseudo code example:
// Using the delegate:
var data = dbContext.Users.Where(usr => IsSatisfiedBy(usr)).ToList();
// This will result in the following steps:
var userList = ExecuteQuery("SELECT * FROM Users"); // all users are fetched.
var satisfied = userList.Where(usr => IsSatisfiedBy(usr))
// Using an expression:
var data = dbContext.Users.Where(usr => usr.Name == "foo");
// This will result in the following step:
var satisfied = ExecuteQuery("SELECT * FROM Users WHERE Name = 'foo'"); // Filtered before returned to caller.
The reason for the performance problem is to record all the records of the related object and then filter it out. First you need to create the query and register from mongo db.
review for soruce code
https://github.com/fsefacan/MongoDbRepository
i have a generic repository
public TEntity GetByCondition(Func<TEntity, Boolean> where)
{
return DbSet.Where(where).FirstOrDefault<TEntity>();
}
i need to get the selected records based on condition;
public IEnumerable<ResultEntity> GetResultByParams(string _RollNo, string _Class)
{
var result = _unitOfWork.ResultRepository.GetByCondition(); // i need to check ondition of _RollNo here. Please guide me the usage.
}
// how to use the var result = _unitOfWork.ResultRepository.GetByCondition(); with condition
In general:
var result = _unitOfWork.ResultRepository.GetByCondition(res => /*your condition for "result" here*/);
By condition I mean expression that returns bool.
For example:
var result = _unitOfWork.ResultRepository.GetByCondition(res => res.RollNo == _RollNo);
It basically means: "give me all Results which have RollNo equals to _RollNo"
I recommend you to read explanations of Func from following question, it may be useful: Explanation of Func.
And, of course, msdn reference for the type of Func-s used in your case.
Currently I have a LINQ statement that Entity Framework Core throws a warning: System.InvalidOperationException: Warning as error exception for warning 'RelationalEventId.QueryClientEvaluationWarning': The LINQ expression '(Invoke(__selector_0, [x]).ToString() == __value_1)' could not be translated and will be evaluated locally.
I have turned this 'throw warning as exception' in the database context initialization to enforce writing LINQ to evaluate on the server side.
public async Task<int?> ExistsAsync<TValue>(TestModel entity, Func<TestModel, TValue> selector)
{
var value = selector(entity).ToString();
var test = await _context.TestModel.Where(x => selector(x).ToString() == value).Select(x => x.Id).FirstOrDefaultAsync();
return test;
}
The above statement is used in the following way.
var id = await _service.ExistsAsync(new TestModel {Name = name}, x => x.Name);
Is there a way to get this to translate to TSQL and evaluate on the server side? Using System.Linq.Expressions?
Edit: Follow up question to my answer below, where is good resources for learning System.Linq.Expressions? I understand basic LINQ well but when it comes to building expression tree's, I don't know where to go.
I managed to get it to work by referring to this answer.
public async Task<int?> ExistsAsync<TValue>(TestModel entity, Expression<Func<TestModel, TValue>> expression)
{
var selector = expression.Compile();
var value = selector(entity);
var predicate = Expression.Lambda<Func<Equipment, bool>>(Expression.Equal(expression.Body, Expression.Constant(value)), expression.Parameters);
var id= await _context.TestModel.Where(predicate).Select(x => x.Id).FirstOrDefaultAsync();
return id;
}
I have a Unit of Work implementation with, among others, the following method:
T Single<T>(Expression<Func<T, bool>> expression) where T : class, new();
and I call it, for instance, like this:
var person = _uow.Single<Person>(p => p.FirstName == "Sergi");
How can I verify that the Single method has been called with an argument of FirstName == "Sergi"?
I've tried the following, but to no avail:
// direct approach
session.Verify(x => x.Single<Person>(p => p.FirstName == "Sergi"));
// comparing expressions
Expression<Func<Person, bool>> expression = p => p.FirstName == "Sergi");
session.Verify(x => x
.Single(It.Is<Expression<Func<Person, bool>>>(e => e == expression));
They all result in the folowing error:
Expected invocation on the mock at least once, but was never performed
Any ideas on how that can be done?
I'm using the latest Moq from NuGet, version 4.0.10827.0
UPDATE: A Specific example
What I'm seeing is that whenever I use string literals inside the lambda, Verify works. As soon as I'm comparing variables it fails. Case in point:
// the verify
someService.GetFromType(QuestionnaireType.Objective)
session.Verify(x => x.Single<Questionnaire>(q =>
q.Type == QuestionnaireType.Objective));
// QuestionnaireType.Objective is just a constant:
const string Objective = "objective";
// the method where it's called (FAILS):
public Questionnaire GetFromType(string type)
{
// this will fail the Verify
var questionnaire = _session
.Single<Questionnaire>(q => q.Type == type);
}
// the method where it's called (PASSES):
public Questionnaire GetFromType(string type)
{
// this will pass the Verify
var questionnaire = _session
.Single<Questionnaire>(q => q.Type == QuestionnaireType.Objective);
}
How come the Verify fails as soon as I use the method parameter in the lambda expression?
What would be the proper way to write this test?
The direct approach works just fine for me:
// direct approach
session.Verify(x => x.Single<Person>(p => p.FirstName == "Sergi"));
The expression object doesn't return true for equivalent expressions so this will fail:
// comparing expressions
Expression<Func<Person, bool>> expression = p => p.FirstName == "Sergi");
session.Verify(x => x
.Single(It.Is<Expression<Func<Person, bool>>>(e => e == expression));
To understand why, run the following NUnit test:
[Test]
public void OperatorEqualEqualVerification()
{
Expression<Func<Person, bool>> expr1 = p => p.FirstName == "Sergi";
Expression<Func<Person, bool>> expr2 = p => p.FirstName == "Sergi";
Assert.IsTrue(expr1.ToString() == expr2.ToString());
Assert.IsFalse(expr1.Equals(expr2));
Assert.IsFalse(expr1 == expr2);
Assert.IsFalse(expr1.Body == expr2.Body);
Assert.IsFalse(expr1.Body.Equals(expr2.Body));
}
And as the test above indicates, comparing by the expression body will also fail, but string comparison works, so this works as well:
// even their string representations!
session.Verify(x => x
.Single(It.Is<Expression<Func<Person, bool>>>(e =>
e.ToString() == expression.ToString()));
And here's one more style of test you can add to the arsenal that also works:
[Test]
public void CallbackVerification()
{
Expression<Func<Person, bool>> actualExpression = null;
var mockUow = new Mock<IUnitOfWork>();
mockUow
.Setup(u => u.Single<Person>(It.IsAny<Expression<Func<Person, bool>>>()))
.Callback( (Expression<Func<Person,bool>> x) => actualExpression = x);
var uow = mockUow.Object;
uow.Single<Person>(p => p.FirstName == "Sergi");
Expression<Func<Person, bool>> expectedExpression = p => p.FirstName == "Sergi";
Assert.AreEqual(expectedExpression.ToString(), actualExpression.ToString());
}
As you have a number of test cases that fail that shouldn't, you likely have a different problem.
UPDATE: Per your update, consider the following setup and expressions:
string normal_type = "NORMAL";
// PersonConstants is a static class with NORMAL_TYPE defined as follows:
// public const string NORMAL_TYPE = "NORMAL";
Expression<Func<Person, bool>> expr1 = p => p.Type == normal_type;
Expression<Func<Person, bool>> expr2 = p => p.Type == PersonConstants.NORMAL_TYPE;
One expression references an instance variable of the containing method. The other represents an expression that references a const member of a static class. The two are different expressions, regardless of the values that may be assigned to the variables at runtime. If however, string normal_type is changed to const string normal_type then the expressions are again the same as each reference a const on the right hand side of the expression.
I would also like to share another approach to comparing the parameter expression to the expected expression. I searched StackOverflow for "how to compare expressions," and I was led to these articles:
how-to-test-expressions-equality
most-efficient-way-to-test-equality-of-lambda-expressions
I was then led to this Subversion repository for db4o.net. In one of their projects, namespace Db4objects.Db4o.Linq.Expressions, they include a class named ExpressionEqualityComparer. I was able to checkout this project from the repository, compile, build, and create a DLL to use in my own project.
With the ExpressionEqualityComparer, you can modify the Verify call to something like the following:
session.Verify(x => x
.Single(It.Is<Expression<Func<Person, bool>>>(e =>
new ExpressionEqualityComparer().Equals(e, expression))));
Ultimately, the ExpressionEqualityComparer and the ToString() techniques both return true in this case (with the ToString most likely being faster - speed not tested). Personally, I prefer the comparer approach since I feel it is more self-documenting and better reflects your design intent (comparing the expression objects rather a string comparison of their ToString outputs).
Note: I'm still looking for a db4o.net license file in this project, but I've not modified the code in anyway, included the copyright notice, and (since the page is publicly available) I'm assuming that's enough for now... ;-)