We Keep Coding

DbSet.Where in EF fetching all the data

I have using the below code to fetch the matching data based on the where parameter and then removing them but not sure why in the log the query which gets executed shows that it is first fetching all the records from table irrespective of the where filter.
void function(Func<TEntity, bool> where)
{
IQueryable objects = DbSet.Where<TEntity>(where).AsQueryable();
foreach(var obj in objects)
DbSet.remove(obj) ;
}

Entity framework uses expression trees to be able to convert lambda expression to sql. You are taking a Func<TEntity, bool>, this causes the DbSet to be converted to a IEnumerable.Where, rather than using a IQueryable.Where, causing all items to be fetched.
You should probably replace the parameter with an Expression<Func<TSource, bool>>. Note that not all expressions can be converted to SQL, and this may result in runtime errors, so you need to be a bit careful how you write expressions.

Timeout expired when using a Func<> instead of a lambda

Given:
public EntityAddress ReadSingle(Func<EntityAddress, bool> predicate)
{
//var result = Context.CV3Address.FirstOrDefault(a => a.GUID == 1100222);
var result = Context.CV3Address.FirstOrDefault(predicate);
return result;
}
FirstOrDefault(a => a.GUID == 1100222); returns a result immediately.
FirstOrDefault(predicate); results in a timeout exception. Note that predicate = the lambda expression
My suspicion is that the latter method attempts to pull down all records which is not gonna happen with a table this large.
Why does this happen?

It happens because of the type of the predicate, which should have been instead
Expression<Func<CV3Address, bool>>
If the predicate is an expression tree (as above) and Context.CV3Address is an IQueryable<CV3Address> then EF can translate the expression tree to SQL and directly get the results from the database.
On the other hand, if the predicate is a Func<CV3Address, bool> (a delegate; a pointer to compiled code) then this cannot be translated to SQL. Therefore LINQ has no other option that to treat your repository as an IEnumerable<CV3Address>, which is a sequence that can be filtered in-memory. That has the side effect of needing to pull all records from the database in order to filter them.
If you hardcode the predicate then the compiler can treat it either as an expression tree or a delegate, and due to the type of Context.CV3Address it treats it as an expression tree.

FirstOrDefault(a => a.GUID == 1100222) creates an expression tree that uses LINQ to Entities to run the query on the DB server.
FirstOrDefault(predicate) downloads the entire table and runs the filter locally.
You need to change your method to take an expression tree:
Expression<Func<CV3Address, bool>>

Combine OrderBy Expression with Where Expression

I found the following question that can combine multiple Expression<Func<T,bool>> expressions:
How to merge two C# Lambda Expressions without an invoke?
I'm wondering whether, using a similar technique, how you go about merging a .OrderBy Expression<Func<Entity, Key>> with a .Where Expression<Func<Entity, bool>> into one Expression of type, or inheriting from type, System.Linq.Expressions.Expression.
I'm making a really cut down QueryProvider-style class for taking T => T == ... Func's via public methods .Where and .OrderBy. This is with the intention that the expression this class builds gets passed to a QueryTranslator to create suitable SQL.
A QueryTranslator of this style, when called from a QueryProvider, typically takes one System.Linq.Expressions.Expression as an argument which it then translates to SQL.
I can follow through and understand the question above for merging two .Where Funcs. The following blog post was particularly useful for this, and I traced through each of the examples:
http://blogs.msdn.com/b/meek/archive/2008/05/02/linq-to-entities-combining-predicates.aspx
This CodeProject article was also useful:
http://www.codeproject.com/Articles/24255/Exploring-Lambda-Expression-in-C
When combining a .OrderBy Func with a .Where Func however the two have different generic arguments. In the case of the .Where it's a Func<Entity, bool>, and in the case of the .OrderBy it's a Func<Entity, Key>.
Merging these two Expressions, on the surface, isn't as straight forward as merging two with the same generic arguments.
The in-built Func-Expression-producing-engine (unsure of exact term) is able to combine a .Where Func with a .OrderBy Func. I'm curious about what goes on under the hood when these two expressions are merged to become one System.Linq.Expressions.Expression. Is it possible, and if so how would you go about combining an Expression<Func<Entity, bool>> with an Expression<Func<Entity,Key>> assuming the Entity is of the same type in each Expression?

public IQueryable<T> ApplyExpressions<T, TKey>(
Expression<Func<T, TKey>> sortBy,
Expression<Func<T, bool>> filterBy,
IQueryable<T> source)
{
return source.Where(filterBy).OrderBy(sortBy);
}
IQueryable<Customer> query = ApplyExpressions<Customer, int>(
c => c.Age,
c => c.Name.StartsWith("B"),
myDC.Customers)
Expression queryExpression = query.Expression; //tada

I think where this question got ambiguous is because it's kind of unanswerable.
I was trying to build one super-expression, like how the IQueryable<> expression builder works with .Where and .OrderBy expressions all in one expression.
Without a subject, however, my limited understanding of Expressions and Expression Trees seems to suggest it's not possible.
To combine a .Where expression with a .OrderBy expression into one Expression you need MethodCallExpression's to separate the two LambdaExpressions.
The MethodCallExpression needs a subject to call the Method on. This is where the IQueryable<> gets used.
I've altered my Query class to keep all the LambdaExpression's separate, and these then get passed to the QueryTranslator separately. The QT merges the output SQL into the right places.
This is as opposed to an IQueryable<> QueryProvider that analyses one super-Expression and outputs SQL into the right places based on MethodCallExpression's in the Expression.

When is ObjectQuery really an IOrderedQueryable?

Applied to entity framework, the extension methods Select() and OrderBy() both return an ObjectQuery, which is defined as:
public class ObjectQuery<T> : ObjectQuery, IOrderedQueryable<T>,
IQueryable<T>, <... more interfaces>
The return type of Select() is IQueryable<T> and that of OrderBy is IOrderedQueryable<T>. So you could say that both return the same type but in a different wrapper. Luckily so, because now we can apply ThenBy after OrderBy was called.
Now my problem.
Let's say I have this:
var query = context.Plots.Where(p => p.TrialId == 21);
This gives me an IQueryable<Plot>, which is an ObjectQuery<Plot>. But it is also an IOrderedQueryable:
var b = query is IOrderedQueryable<Plot>; // True!
But still:
var query2 = query.ThenBy(p => p.Number); // Does not compile.
// 'IQueryable<Plot>' does not contain a definition for 'ThenBy'
// and no extension method 'ThenBy' ....
When I do:
var query2 = ((IOrderedQueryable<Plot>)query).ThenBy(p => p.Number);
It compiles, but gives a runtime exception:
Expression of type 'IQueryable`1[Plot]' cannot be used for parameter of type 'IOrderedQueryable`1[Plot]' of method 'IOrderedQueryable`1[Plot] ThenBy[Plot,Nullable`1](IOrderedQueryable`1[Plot], Expressions.Expression`1[System.Func`2[Plot,System.Nullable`1[System.Int32]]])'
The cast is carried out (I checked), but the parameter of ThenBy is still seen as IQueryable (which puzzles me a bit).
Now suppose some method returns an ObjectQuery<Plot> to me as IQueryable<Plot> (like Select()). What if I want to know whether it is safe to call ThenBy on the returned object. How can I figure it out if the ObjectQuery is "real" or a "fake" IOrderedQueryable without catching exeptions?

Expression Trees are genuinely good fun! (or perhaps I'm a little bit of a freak) and will likely become useful in many a developer's future if Project Roslyn is anything to go by! =)
In your case, simple inherit from MSDN's ExpressionVisitor, and override the VisitMethodCall method in an inheriting class with something to compare m.MethodInfo with SortBy (i.e. if you're not too fussy simply check the name, if you want to be fussy use reflection to grab the actual SortBy MethodInfo to compare with.
Let me know if/what you need examples of, but honestly, after copy/pasting the ExpressionVisitor you'll probably need no more than 10 lines of non-expression-tree code ;-)
Hope that helps

Although Expression Trees are good fun, wouldn't in this case the simple solution be to use OrderBy rather than ThenBy?
OrderBy is an extension on IQueryable and returns an IOrderedQueryable.
ThenBy is an extension on IOrderedQueryable and returns an IOrderedQueryable.
So if you have a IQueryable (as in your case above, where query is an IQueryable) and you want to apply an initial ordering to it, use OrderBy. ThenBy is only intended to apply additional ordering to an already ordered query.
If you have a LINQ result of some kind, but you aren't sure if it is an IQueryable or an IOrderedQueryable and want to apply additional filtering to it, you could make two methods like:
static IOrderedQueryable<T, TKey> ApplyAdditionalOrdering<T, TKey>(this IOrderedQueryable<T, TKey> source, Expression<Func<T, TFilter>> orderBy)
{
return source.ThenBy(orderBy);
}
And
static IOrderedQueryable<T, TKey> ApplyAdditionalOrdering<T, TKey>(this IQueryable<T> source, Expression<Func<T, TFilter>> orderBy)
{
return source.OrderBy(orderBy);
}
The compiler will figure out the correct one to call based on the compile-time type of your query object.

Non-Generic AsEnumerable

How would you suggest using AsEnumerable on a non-generic IQueryable?
I cannot use the Cast<T> or OfType<T> methods to get an IQueryable<object> before calling AsEnumerable, since these methods have their own explicit translation by the underlying IQueryProvider and will break the query translation if I use them with a non-mapped entity (obviously object is not mapped).
Right now, I have my own extension method for this (below), but I'm wondering if there's a way built into the framework.
public static IEnumerable AsEnumerable(this IQueryable queryable)
{
foreach (var item in queryable)
{
yield return item;
}
}
So, with the above extension method, I can now do:
IQueryable myQuery = // some query...
// this uses the built in AsEnumerable, but breaks the IQueryable's provider because object is not mapped to the underlying datasource (and cannot be)
var result = myQuery.Cast<object>().AsEnumerable().Select(x => ....);
// this works (and uses the extension method defined above), but I'm wondering if there's a way in the framework to accomplish this
var result = myQuery.AsEnumerable().Cast<object>().Select(x => ...);

Since the interface IQueryable inherits from IEnumerable why not:
IQueryable queryable;
IEnumerable<T> = (queryable as IEnumerable).Cast<T>();
Edit
There are two Cast<> extension methods:
public static IQueryable<TResult> Cast<TResult>(this IQueryable source)
public static IEnumerable<TResult> Cast<TResult>(this IEnumerable source)
Which one is called is statically decided by the compiler. So casting an IQueryable as an IEnumerable will cause the second extension method to be called, where it will be treated as an IEnumerable.

JaredPar's answer in other words:
public static IEnumerable AsEnumerable(this IEnumerable source)
{
return source;
}
Usage:
IQueryable queryable = // ...
IEnumerable enumerable = queryable.AsEnumerable();
IEnumerable<Foo> result = enumerable.Cast<Foo>();
// ↑
// Enumerable.Cast<TResult>(this IEnumerable source)

The type IQueryable inherits from the non-generic IEnumerable so this extension method doesn't seem to serve any purpose. Why not just use it as IEnumerable directly?

To clarify, you are trying to add a "force expression evaluation" to your linq expression tree so that part of the expression tree is evaluated against the underlying provider (linq to SQL for example) and the rest is evaluated in memory (linq to objects). But you want to be able to specify the entire express tree without actually executing the query or reading any results into memory.
This is a good thing and it shows some insight into the way that Linq works. Good job.
The confusion I see here is your use of "AsEnumerable" as the method name. To me (and I think many people who thinq linq ) "AsEnumerable" is too similar to the "AsQueryable" method which is essentially a cast, not an actual evaluator. I propose you rename your method to "Evaluate". This is my Evaluate() method from my personal Linq extensions library.
/// <summary>
/// This call forces immediate evaluation of the expression tree.
/// Any earlier expressions are evaluated immediately against the underlying IQueryable (perhaps
/// a Linq to SQL provider) while any later expressions are evaluated against the resulting object
/// graph in memory.
/// This is one way to determine whether expressions get evaluated by the underlying provider or
/// by Linq to Objects in memory.
/// </summary>
public static IEnumerable<T> Evaluate<T>(this IEnumerable<T> expression)
{
foreach (var item in expression)
{
yield return item;
}
}
/// <summary>
/// This call forces immediate evaluation of the expression tree.
/// Any earlier expressions are evaluated immediately against the underlying IQueryable (perhaps
/// a Linq to SQL provider) while any later expressions are evaluated against the resulting object
/// graph in memory.
/// This is one way to determine whether expressions get evaluated by the underlying provider or
/// by Linq to Objects in memory.
/// </summary>
public static IEnumerable Evaluate(this IEnumerable expression)
{
foreach (var item in expression)
{
yield return item;
}
}
This allows you to write a query where some of the query is evaluated by SQL (for example) and the rest is evaluated in memory. A good thing.