We Keep Coding

DbSet.Where in EF fetching all the data

I have using the below code to fetch the matching data based on the where parameter and then removing them but not sure why in the log the query which gets executed shows that it is first fetching all the records from table irrespective of the where filter.
void function(Func<TEntity, bool> where)
{
IQueryable objects = DbSet.Where<TEntity>(where).AsQueryable();
foreach(var obj in objects)
DbSet.remove(obj) ;
}

Entity framework uses expression trees to be able to convert lambda expression to sql. You are taking a Func<TEntity, bool>, this causes the DbSet to be converted to a IEnumerable.Where, rather than using a IQueryable.Where, causing all items to be fetched.
You should probably replace the parameter with an Expression<Func<TSource, bool>>. Note that not all expressions can be converted to SQL, and this may result in runtime errors, so you need to be a bit careful how you write expressions.

Using the same expression for EntityFramework queries and in-memory evaluation

The problem description
In my application, I have a search function that builds a complex search query based on user input on top of EntityFramework DbSet object and runs it against the database, something like this:
public static IQueryable<MyEntity> ApplySearchQuery(SearchSpec search, IQueryable<MyEntity> query)
{
if (search.Condition1.HasValue)
query = query.Where(e => e.SomeProperty == search.Condition1);
if (search.Condition2.HasValue)
query = query.Where(e => e.OtherProperty == search.Condition2);
....
}
This performs pretty well on the database side. Now I have come to a point that I have a single MyEntity at hand, and I want to see if a particular SearchSpec matches the entity or not. And I need to do this for a potentially large number of SearchSpec objects, so the performance is important.
I'd also really like to avoid duplicating the expressions.
Here's what I've thought so far:
I can call Expression.Compile() on the expressions to convert them to a delegate, and then call them. But since I have a parameter (the search parameter) I need to build expressions and compile them every time, making it a very inefficient way (I suppose, correct me if I'm wrong).
I can wrap my single entity in an IQueriable using new [] { myEntity }.AsQueriable() and then evaluate the query on it. Not sure how well this will perform.
The question:
Which of the above approaches are faster?
Is any of my assumptions (about the limitations) wrong?
Is there any other way that I haven't thought about yet?

Here PredicateBuilder could work miracles for you. With PredicateBuilder you can build an Expression that can be used against an IQueryable, but, when compiled, also against objects.
You could have a method that builds and returns the Expression
using LinqKit;
Expression<Func<MyEntity, bool>> CreateExpression(SearchSpec search)
{
var predicate = PredicateBuilder.True<MyEntity>();
if (search.Condition1.HasValue)
predicate = predicate.And(e => e.SomeProperty == search.Condition1);
if (search.Condition2.HasValue)
predicate = predicate.And(e => e.OtherProperty == search.Condition2);
...
return predicate;
}
To be used as:
var predicate = CreateExpression(search);
var result = query.Where(predicate.Expand()); // will be translated into SQL.
var match = predicate.Compile()(myEntity);
Notice the Expand call. Without it, EF will fail because under the hood Invoke will be called, which can't be translated into SQL. Expand replaces these calls so that EF can convert the expression to SQL.

Reusable predicate expressions in LINQ to Entities queries

A certain set of criteria that occurs in many different queries throughout our application has slowly grown more complex. To avoid duplication of this code, I want to split these criteria out into a method that returns the conditions as an Expression that can in turn be applied where necessary:
public Expression<Func<Invoice, bool>> GetComplexPredicate()
{
// complex predicate is returned as an Expression:
return c => ...
}
Reused as such:
var result = repository.Invoice.Where(GetComplexPredicate())
However, the statement below won't compile, since c.Invoice is just an ICollection.
var result = repository.Customer
.Where(c => c.Country == "US" && c.Invoice.Any(GetComplexPredicate()))
Is it in any way possible to use the expression like this?

There are two parts to this question:
How do I use predicate expressions on navigation properties inside a L2E query?
L2E allows the use of the AsQueryable extension method within a query. This means that I'm able to convert the ICollection to an IQueryable and apply the predicate expression. So far so good. However, it might compile, but it still won't run, since L2E won't know what to do with the predefined expression from the GetComplexPredicate method. This leads us to:
How do I combine several separate predicate expressions into one?
The enormously useful LINQKit can easily combine several predicates into one expression using PredicateBuilder. With the Expand method from LINQKit and the aforementioned AsQueryable, we can finally arrive at a statement that will both compile and run beautifully:
// build the entire predicate beforehand (PredicateBuilder + AsQueryable):
var complexPredicate = GetComplexPredicate();
var condition = PredicateBuilder.True<Customer>()
.And(c => c.Country == "US")
.And(c => c.Invoice.AsQueryable().Any(complexPredicate));
// apply criteria to query (using Expand):
var result = repository.Customer.Where(condition.Expand()).ToList();

Convert an IQueryable linq query to IEnumerable<T> cancels out linq optimized way to work?

I'm kinda newbie on .NET, and I was wondering how does linq works, since you can aply many linq queries, one after another, but none of them are really executed until they're used to transfer information or converted to list, etc.
There are 2 important ways to get a linq query, by using IQueryable<T>, which aplies the where filters directly on the Sql, and IEnumerable which get all the records and then it work with them on memory. However, let's take a look on this code:
//Linq dynamic library
IQueryable<Table> myResult = db.Categories
.Where(a => a.Name.Contains(StringName))
.OrderBy("Name")
.Skip(0)
.Take(10);
if (myResult != null)
{
return myResult.AsEnumerable();
}
else
{ return null; }
Depsite i'm using Linq dynamic library, the direct result from this query is being get on IQueryable<T>, if the query is finally being returned as IEnumerable, is the query being really filtered on the sql? or is it in memory?

It's still going to execute in the database, don't worry. Basically it's all down to which implementation of Where etc is used. While you're calling methods on IQueryable<T> - via the Queryable extension methods - it will be using expression trees. When you start to fetch from that query, it will be turned into SQL and sent to the database.
On the other hand, if you use any of those methods after you've got it as an IEnumerable<T> (in terms of the compile-time type), that will use the extension methods in Enumerable, and all of the rest of the processing would be done in-process.
As an example, consider this:
var query = db.People
.Where(x => x.Name.StartsWith("J"))
.AsEnumerable()
.Where(x => x.Age > 20);
Here AsEnumerable() just returns its input sequence, but typed as IEnumerable<T>. In this case, the database query would return only people whose name began with J - and then the age filtering would be done at the client instead.

If you return an IEnumerable<T> and then further refine the query, then the further refinement happens in memory. The part that was expressed on an IQueryable<T> will get translated to the appropiate SQL statements (for the LINQ-to-SQL case, obviously).
See Returning IEnumerable<T> vs. IQueryable<T> for a longer and more detailed answer.

Implementing IQueryable.Count

I'm working on an IQueryable provider. In my IQueryProvider I have the following code:
public TResult Execute<TResult>(Expression expression)
{
var query = GetQueryText(expression);
// Call the Web service and get the results.
var items = myWebService.Select<TResult>(query);
IQueryable<TResult> queryableItems = items.AsQueryable<TResult>();
return (TResult)queryableItems;
}
GetQueryText does all the leg work and works out the query string for the expression tree. This is all working well, so Where, OrderBy and Take are sorted. The webservice supports a count query using the following:
int count = myWebService.Count(query);
But I can't get my head round where I put this in the IQueryable or IQueryProvider.
I've basically worked from reading tutorials and open source examples, but can't seem to find one that does Count.

The answer appears simpler than I first thought. This blog post helped:
The Execute method is the entry point into your provider for actually executing query expressions. Having an explicit execute instead of just relying on IEnumerable.GetEnumerator() is important because it allows execution of expressions that do not necessarily yield sequences. For example, the query “myquery.Count()” returns a single integer. The expression tree for this query is a method call to the Count method that returns the integer. The Queryable.Count method (as well as the other aggregates and the like) use this method to execute the query ‘right now’.
I did some debugging and for a query of myContext.Where(x => x.var1 > 5) Execute is called and TResult is an IEnumerable<MyClass>
For myContext.Where(x => x.var1 > 5).Count() Execute is called and TResult is an int
So my Execute method just needs to return appropriately.