I have using the below code to fetch the matching data based on the where parameter and then removing them but not sure why in the log the query which gets executed shows that it is first fetching all the records from table irrespective of the where filter.
void function(Func<TEntity, bool> where)
{
IQueryable objects = DbSet.Where<TEntity>(where).AsQueryable();
foreach(var obj in objects)
DbSet.remove(obj) ;
}
Entity framework uses expression trees to be able to convert lambda expression to sql. You are taking a Func<TEntity, bool>, this causes the DbSet to be converted to a IEnumerable.Where, rather than using a IQueryable.Where, causing all items to be fetched.
You should probably replace the parameter with an Expression<Func<TSource, bool>>. Note that not all expressions can be converted to SQL, and this may result in runtime errors, so you need to be a bit careful how you write expressions.
Related
If I have a method that returns a List of Dogs, does Dapper convert the Where predicate into a SQL Where? or does it get all the dogs first and then filter the list?
public IList<Dog> GetDogsBy(Func<Dog, bool> predicate)
{
return db.Query<Dog>("SELECT * FROM Dog",null).Where(predicate).ToList();
}
It depends on what overload resolution does when resolving the Where.
If the result of the call to Query is a List<Dog> and you have a using System.Linq; directive, and the argument is a delegate, then the Where will resolve to the LINQ-to-objects version of Where. So the select query will run on the server, and then the Where will run on the client.
If the result of Query is IQueryable<Dog> and the argument is an Expression then overload resolution will choose the Where which combines with the existing query. When that query is executed, the Where runs on whatever side the query providers chooses to run it on, which is typically the server.
When it comes to SQL trnalsation, while an expression tree can be translated, a compiled method cannot. This is because once the method is compiled, it's a matter of reading IL to perform the translation, which is not exposed through the .NET API, whereas Expression Trees are explicitly designed to be navigated using metacode for the sake of translation.
Because you're passing in a Func<...> instead of an Expression<Func<...>>, you're passing that compiled method, and thus the code cannot get the expression tree, and thus the above code either will not work or will execute on your local machine.
If you were to replace the method signature with
public IList<Dog> GetDogsBy(Expression<Func<Dog, bool>> predicate)
Then it might be able to perform that translation, depending on the semantics of Dapper.
The problem description
In my application, I have a search function that builds a complex search query based on user input on top of EntityFramework DbSet object and runs it against the database, something like this:
public static IQueryable<MyEntity> ApplySearchQuery(SearchSpec search, IQueryable<MyEntity> query)
{
if (search.Condition1.HasValue)
query = query.Where(e => e.SomeProperty == search.Condition1);
if (search.Condition2.HasValue)
query = query.Where(e => e.OtherProperty == search.Condition2);
....
}
This performs pretty well on the database side. Now I have come to a point that I have a single MyEntity at hand, and I want to see if a particular SearchSpec matches the entity or not. And I need to do this for a potentially large number of SearchSpec objects, so the performance is important.
I'd also really like to avoid duplicating the expressions.
Here's what I've thought so far:
I can call Expression.Compile() on the expressions to convert them to a delegate, and then call them. But since I have a parameter (the search parameter) I need to build expressions and compile them every time, making it a very inefficient way (I suppose, correct me if I'm wrong).
I can wrap my single entity in an IQueriable using new [] { myEntity }.AsQueriable() and then evaluate the query on it. Not sure how well this will perform.
The question:
Which of the above approaches are faster?
Is any of my assumptions (about the limitations) wrong?
Is there any other way that I haven't thought about yet?
Here PredicateBuilder could work miracles for you. With PredicateBuilder you can build an Expression that can be used against an IQueryable, but, when compiled, also against objects.
You could have a method that builds and returns the Expression
using LinqKit;
Expression<Func<MyEntity, bool>> CreateExpression(SearchSpec search)
{
var predicate = PredicateBuilder.True<MyEntity>();
if (search.Condition1.HasValue)
predicate = predicate.And(e => e.SomeProperty == search.Condition1);
if (search.Condition2.HasValue)
predicate = predicate.And(e => e.OtherProperty == search.Condition2);
...
return predicate;
}
To be used as:
var predicate = CreateExpression(search);
var result = query.Where(predicate.Expand()); // will be translated into SQL.
var match = predicate.Compile()(myEntity);
Notice the Expand call. Without it, EF will fail because under the hood Invoke will be called, which can't be translated into SQL. Expand replaces these calls so that EF can convert the expression to SQL.
Given:
public EntityAddress ReadSingle(Func<EntityAddress, bool> predicate)
{
//var result = Context.CV3Address.FirstOrDefault(a => a.GUID == 1100222);
var result = Context.CV3Address.FirstOrDefault(predicate);
return result;
}
FirstOrDefault(a => a.GUID == 1100222); returns a result immediately.
FirstOrDefault(predicate); results in a timeout exception. Note that predicate = the lambda expression
My suspicion is that the latter method attempts to pull down all records which is not gonna happen with a table this large.
Why does this happen?
It happens because of the type of the predicate, which should have been instead
Expression<Func<CV3Address, bool>>
If the predicate is an expression tree (as above) and Context.CV3Address is an IQueryable<CV3Address> then EF can translate the expression tree to SQL and directly get the results from the database.
On the other hand, if the predicate is a Func<CV3Address, bool> (a delegate; a pointer to compiled code) then this cannot be translated to SQL. Therefore LINQ has no other option that to treat your repository as an IEnumerable<CV3Address>, which is a sequence that can be filtered in-memory. That has the side effect of needing to pull all records from the database in order to filter them.
If you hardcode the predicate then the compiler can treat it either as an expression tree or a delegate, and due to the type of Context.CV3Address it treats it as an expression tree.
FirstOrDefault(a => a.GUID == 1100222) creates an expression tree that uses LINQ to Entities to run the query on the DB server.
FirstOrDefault(predicate) downloads the entire table and runs the filter locally.
You need to change your method to take an expression tree:
Expression<Func<CV3Address, bool>>
I have a collection of compiled expressions which I combined into one expression because I want to build dynamically my linq query. See ExpressionTree Compile() method generate a stackoverflow exception for the reason I used compiled expression in order to prevent from stackoverflow because in my computer when the query contains more than 7000 expressions, it throws this exception.)
Then I use the new generated expression and pass it to my FindAll method. Problem, NHibernate is not able to execute the query and says:
unable to cast object of type 'nhibernate.hql.ast.parameter' to type 'nhibernate.hql.ast.hqlbooleanexpression'
public IList<T> FindAll(Expression<Func<T, bool>> criteria)
{
return SessionFactory.GetCurrentSession()
.QueryOver<T>()
.Where(criteria)
.List();
}
I debuged and found that nhibernate tries to convert the compiled expression to boolean expression in file HqlTreeNode (method: HqlTreeNodeExtensions.AsBooleanExpression(this HqlTreeNode node) which of course doesn't works. Then what solution should I use ?
Here what the criteria variable looks like:
(Invoke(value(System.Func`2[Something.SomeEntity,System.Boolean]) // this don't work
For information, if it wasn't compiled, it would have looked like something like this:
([someEntity].UserID == 1) // this works
Thank you.
I am working on an ASP.NET MVC application which uses the repository pattern with LINQ to SQL as my data source. In my repository I expose the following method:
public IEnumerable<T> Find(Expression<Func<T, bool>> where)
{
return _context.GetTable<T>().Where(where);
}
I am able to call this by saying:
repository<User>().Find(u => true);
But if I try doing (when search is null)
repository<User>().Find(u => !string.IsNullOrEmpty(search) ? u.UserName.Contains(search) : true);
I get the error:
Value cannot be null. Parameter name: text
I thought the lambda expression would execute the same thing since the value of search is null, but this is clearly not the case.
How do I fix this problem?
Because this is an expression, normal conditional operator logic (not evaluating the false expression) don't automatically apply. Since the expression is actually being translated into a SQL query expression, LINQ to SQL is evaluating the parameters of the condition, and in this case it's null. While LIKE null is valid SQL syntax, evidently LINQ to SQL doesn't appreciate being passed null. You'll have to construct two different queries, one for a null term, one for a non-null term.
Either that, or pass something like string.Empty to Contains. The only impact here will be the inability to use an index on the column, since the server will be doing a repeated conditional evaluation on every row, forcing a table scan. I'd advise using two queries without the conditional.
To add to the other answers, if you have to follow this ideal, you could simply do this instead:
repository<User>.Find(u => string.IsNullOrEmpty(search) ||
u.UserName.Contains(search));
Before blindly implementing this, if you were to do so, please read Adam's answer to learn of consequences.
Contains expects a string. You should pass string.Emtpy.
u.UserName.Contains(search)
If you try this, it'll compile but you'll get a runtime error:
string str = "Leniel";
if (str.Contains(null))
{
var num = 1;
}
Error:
Value cannot be null.
Parameter name: value