We Keep Coding

Using the same expression for EntityFramework queries and in-memory evaluation

The problem description
In my application, I have a search function that builds a complex search query based on user input on top of EntityFramework DbSet object and runs it against the database, something like this:
public static IQueryable<MyEntity> ApplySearchQuery(SearchSpec search, IQueryable<MyEntity> query)
{
if (search.Condition1.HasValue)
query = query.Where(e => e.SomeProperty == search.Condition1);
if (search.Condition2.HasValue)
query = query.Where(e => e.OtherProperty == search.Condition2);
....
}
This performs pretty well on the database side. Now I have come to a point that I have a single MyEntity at hand, and I want to see if a particular SearchSpec matches the entity or not. And I need to do this for a potentially large number of SearchSpec objects, so the performance is important.
I'd also really like to avoid duplicating the expressions.
Here's what I've thought so far:
I can call Expression.Compile() on the expressions to convert them to a delegate, and then call them. But since I have a parameter (the search parameter) I need to build expressions and compile them every time, making it a very inefficient way (I suppose, correct me if I'm wrong).
I can wrap my single entity in an IQueriable using new [] { myEntity }.AsQueriable() and then evaluate the query on it. Not sure how well this will perform.
The question:
Which of the above approaches are faster?
Is any of my assumptions (about the limitations) wrong?
Is there any other way that I haven't thought about yet?

Here PredicateBuilder could work miracles for you. With PredicateBuilder you can build an Expression that can be used against an IQueryable, but, when compiled, also against objects.
You could have a method that builds and returns the Expression
using LinqKit;
Expression<Func<MyEntity, bool>> CreateExpression(SearchSpec search)
{
var predicate = PredicateBuilder.True<MyEntity>();
if (search.Condition1.HasValue)
predicate = predicate.And(e => e.SomeProperty == search.Condition1);
if (search.Condition2.HasValue)
predicate = predicate.And(e => e.OtherProperty == search.Condition2);
...
return predicate;
}
To be used as:
var predicate = CreateExpression(search);
var result = query.Where(predicate.Expand()); // will be translated into SQL.
var match = predicate.Compile()(myEntity);
Notice the Expand call. Without it, EF will fail because under the hood Invoke will be called, which can't be translated into SQL. Expand replaces these calls so that EF can convert the expression to SQL.

Timeout expired when using a Func<> instead of a lambda

Given:
public EntityAddress ReadSingle(Func<EntityAddress, bool> predicate)
{
//var result = Context.CV3Address.FirstOrDefault(a => a.GUID == 1100222);
var result = Context.CV3Address.FirstOrDefault(predicate);
return result;
}
FirstOrDefault(a => a.GUID == 1100222); returns a result immediately.
FirstOrDefault(predicate); results in a timeout exception. Note that predicate = the lambda expression
My suspicion is that the latter method attempts to pull down all records which is not gonna happen with a table this large.
Why does this happen?

It happens because of the type of the predicate, which should have been instead
Expression<Func<CV3Address, bool>>
If the predicate is an expression tree (as above) and Context.CV3Address is an IQueryable<CV3Address> then EF can translate the expression tree to SQL and directly get the results from the database.
On the other hand, if the predicate is a Func<CV3Address, bool> (a delegate; a pointer to compiled code) then this cannot be translated to SQL. Therefore LINQ has no other option that to treat your repository as an IEnumerable<CV3Address>, which is a sequence that can be filtered in-memory. That has the side effect of needing to pull all records from the database in order to filter them.
If you hardcode the predicate then the compiler can treat it either as an expression tree or a delegate, and due to the type of Context.CV3Address it treats it as an expression tree.

FirstOrDefault(a => a.GUID == 1100222) creates an expression tree that uses LINQ to Entities to run the query on the DB server.
FirstOrDefault(predicate) downloads the entire table and runs the filter locally.
You need to change your method to take an expression tree:
Expression<Func<CV3Address, bool>>

Custom OrderBy extension method fails while projecting anonymous type

In reference to the answer proposed in this post, can anyone explain why LINQPad won't generate any SQL for this?
var query = Products.Select(p => new
{
Orders = p.Orders.OrderByWithDirection(x => x.PurchaseDate, true)
});
The exception is as follows:
NotSupportedException: Method 'System.Linq.IOrderedEnumerable1[LINQPad.User.Order] OrderByWithDirection[Phase,String](System.Collections.Generic.IEnumerable1[LINQ‌​Pad.User.Order], System.Func`2[LINQPad.User.Order,System.String], Boolean)' has no supported translation to SQL.

The problem is not that it's part of an anonymous type - the problem is that you don't actually call OrderByWithDirection directly... instead, you build up an expression tree within the projection and that expression tree contains a reference to OrderByWithDirection, which LINQ to SQL (or EF, or whatever) doesn't know about.
I can't think of any nice fix for this other than an expression tree visitor which goes through the expression tree and replaces the call with either OrderBy or OrderByDescending based on the second argument - which would have to be a constant, of course. Even that's going to be far from trivial.
To be honest, I would probably take a low-tech approach - keep as much of the query the same as you can, and then conditionally call OrderBy or OrderByDescending at the end:
var mainQuery = ...; // Stuff which is in common
var query = direction
? mainQuery.Select(p => new { Orders = p.Orders.OrderBy(x => x.PurchaseDate) })
: mainQuery.Select(p => new { Orders = p.Orders.OrderByDescending(x => x.PurchaseDate) });
As an aside, I assume your real query has more properties in the anonymous type? There's rarely any good reason to create an anonymous type with a single property.

What is the difference between LINQ query expressions and extension methods

Below are two queries that return the same data. Other then style I am not sure which is better.
What factors influence these queries?
What are the benefits of using one style over the other?
Sample 1
var x = from s in db.Surveys
join sq in db.Survey_Questions on s.ID equals sq.Survey_ID
join q in db.Questions on sq.Question_ID equals q.ID
join qg in db.Question_Groups on q.ID equals qg.Question_ID
where s.Type_ID.Equals(typeID) & s.Type.Equals(type)
select new { question = sq.Question, status = sq.Status, grp = qg };
Sample 2
var x = db.Surveys.Where(s => s.Type_ID.Equals(typeID) & s.Type.Equals(type))
.Join(db.Survey_Questions,
s => s.ID,
sq => sq.Survey_ID,
(s, sq) => new
{
question = sq.Question,
status = sq.Status
})
.Join(db.Question_Groups,
q => q.question.ID,
qg => qg.Question_ID,
(q, qg) => new
{
question = q.question,
status = q.status,
group = qg
}).ToList();

Update: You have fixed your title, so ignore the rant.
The title of your question has nothing to do with your code samples. Your question implies that one syntax is IEnumerable and the other is IQueryable, but this is incorrect. In your samples, if db.Surveys is an IQueryable, then both your samples are using IQueryable. I will try to answer both questions.
Your two code samples are just different ways of writing the same LINQ queries (assuming they are well-written). The code in sample 1 is just shorthand for the code in sample 2. The compiler treats the code in both samples the same way. Think of the way the C# compiler will treat int? the same as Nullable<System.Int32>. Both the C# and VB.Net languages provide this shorthand query syntax. Other languages might not have this syntax and you would have to use the sample 2 syntax. In fact, other languages might not even support extension methods or lambda expressions, and you would have to use an uglier syntax yet.
Update:
To take Sander's example further, when you write this (query comprehension syntax):
var surveyNames = from s in db.Surveys select s.Name
You think the compiler turns that shorthand into this (extension methods and lambda expression):
IQueryable<string> surveryNames = db.Surveys.Select(s => s.Name);
But actually extension methods and lambda expressions are shorthand themselves. The compilers emits something like this (not exactly, but just to give an idea):
Expression<Func<Survey, string>> selector = delegate(Survey s) { return s.Name; };
IQueryable<string> surveryNames = Queryable.Select(db.Surveys, selector);
Note that Select() is just a static method in the Queryable class. If your .NET language did not support query syntax, lambdas, or extension methods, that is kinda how you would have to write the code yourself.
What are the benefits of using one style over the other?
For small queries, extension methods can be more compact:
var items = source.Where(s => s > 5);
Also, the extension method syntax can be more flexible, such as conditional where clauses:
var items = source.Where(s => s > 5);
if(smallerThanThen)
items = items.Where(s => s < 10);
if(even)
items = items.Where(s => (s % 2) == 0);
return items.OrderBy(s => s);
In addition, several methods are only available through extension method syntax (Count(), Aggregate(), Take(), Skip(), ToList(), ToArray(), etc), so if I'll use one of these, I'll usually write the whole query in this syntax to avoid mixing both syntaxes.
var floridaCount = source.Count(s => s.State == "FL");
var items = source
.Where(s => s > 5)
.Skip(5)
.Take(3)
.ToList();
On the other hand, when a query gets bigger and more complex, query comprehension syntax can be clearer, especially once you start complicating with a few let, group, join, etc.
In the end I will usually use whichever works better for each specific query.
Update: you fixed your title, so ignore the rest...
Now, about your title: With respect to LINQ, IEnumerable and IQueryable are very similar. They both have pretty much the same extension methods (Select, Where, Count, etc), with the main (only?) difference being that IEnumerable takes Func<TIn,TOut> as paremeters and IQueryable takes Expression<Func<TIn,TOut>> as parameters. You express both the same way (usually lamba expressions), but internally they are completely different.
IEnumerable is the doorway to LINQ to Objects. The LINQ to Objects extension methods can be called on any IEnumerable (arrays, lists, anything you can iterate with foreach) and the Func<TIn,TOut> is converted to IL at compile time and runs like a normal method code at run time. Note that some other LINQ providers use IEnumerable and so are actually using LINQ to Objects behind the scenes (LINQ to XML, LINQ to DataSet).
IQueryable is used by LINQ to SQL, LINQ to Entities, and other LINQ providers which need to examine your query and translate it instead of executing your code directly. IQueryable queries and their Expression<Func<TIn,TOut>>s are not compiled into IL at compile time. Instead an expression tree is created and can be examined at run time. This allows the statements to be translated into other query languages (for example T-SQL). An expression tree can be compiled into a Func<TIn,TOut> at run time and executed if desired.
An example that illustrates the difference can be found in this question where the OP wants to do part of a LINQ to SQL query in SQL Server, bring the objects into managed code, and do the rest of the query in LINQ to Objects. To achieve this all he has to do is cast the IQueryable into an IEnumerable where he wants the switch to happen.

LINQ is buzz word for a technology.
IQueryable is a .NET Interface which is used by LINQ.
Other than the style, there is no difference between the two. Use whichever style you prefer.
I prefer the first style for long statement (like that one shown here) and the second for very short statements.

The where clause in the first example is actually just syntactic sugar for the Where clause in your second method. In fact, you can write your own class that has nothing to do with Linq or IQueryable and just by having a Where method, you can use that syntactic sugar. For example:
public class MyClass
{
public MyClass Where<T>(Func<MyClass, T> predicate)
{
return new MyClass { StringProp = "Hello World" };
}
public MyClass Select<T>(Func<MyClass, T> predicate)
{
return new MyClass ();
}
public string StringProp { get; set; }
}
This is obviously a stupid example, but note that there's a Where method that just returns a new MyClass with stringprop set to Hello World. To demonstrate:
MyClass a = new MyClass();
var q = from p in a
where p.StringProp == "foo" // doesnt matter what we put here, as we're not really checking the predicate
select p;
Console.WriteLine(q.StringProp);
This will result in writing out "Hello World". Again, this example is obviously pointless, but it proves the point that the "where" syntax just looks for a Where method in your code that takes a Func.

Query expressions and extension methods are two ways to do the exact same thing. Query expressions get transformed to extension methods when compiling - they are just syntactic sugar for people who are more comfortable with SQL.
When you write this:
var surveyNames = from s in db.Surveys select s.Name;
The compiler transforms this into:
IQueryable<string> surveryNames = db.Surveys.Select(s => s.Name);
Really, I think query expressions were just created for marketing reasons - a SQL-like language construct to act as an eye-catcher when LINQ was developed, not something that offers much actual use. I find that most people just use the extension methods directly, as they result in a more unified coding style, instead of a mix of C# and SQL.

1./ Your question title does not match what you asked.
2./ Your question title does not really make sense. Linq stands for Language Integrated Query and is an umbrella term for a bunch of technologies and practices, IQueryable is an interface that is commonly used to facilitate Linq. you are comparing Apples and Oranges
3./ About your actual question, the main difference is style, for complex queries like this one, my personal preference is the 2nd version, as it clearly shows the progression of the result sets.

Your Sample1 is top level representation of Linq, it is more readable, and while compiling it'll converted to expression tree i.e your Sample2.
var x = from s in db.Surveys
join sq in db.Survey_Questions on s.ID equals sq.Survey_ID
join q in db.Questions on sq.Question_ID equals q.ID
join qg in db.Question_Groups on q.ID equals qg.Question_ID
where s.Type_ID.Equals(typeID) & s.Type.Equals(type)
select new { question = sq.Question, status = sq.Status, grp = qg };
you can try below code to get expression for written query
var exp=x.Expression;
Expressions are used when query less complicated

I think your question is better phrased like this, "What is the difference between IEnumerable<T> and IQueryable<T> with respect to LINQ"
LINQ queries return an IQueryable<T> by default. IQueryable<T> allows you to append other filters or "clauses" onto your query before you execute it.
Your LINQ query (first example) and your LINQ using method chaining (second example) produce the same result, with different syntax.
It is possible to write a LINQ query as a LINQ method chain and visa versa. It really depends on your preference.
#Lucas: The different is IEnumerable<T> does in-memory querying and IQueryable<T> does out-of-memory. Meaning, once you are in a foreach iterator, you are using IEnumerable, and when you are building your query, via either extension methods or using LINQ from o in object synatax, you are building an IQueryable<T>. The IQueryable<T> is executed as soon as you touch the Enumerator.

Another point worth mentioning is that the Linq extension methods adhere to C# language whereas the query comprehension stuff is preprocessed like is built into the compiler.
i.e you can navigate to the definition of .Select(x =>
whereas you cannot for from ... where ... select

Extension methods syntax vs query syntax

I'm trying to get a handle on if there's a good time to use standard linq keywords or linq extension methods with lambda expressions. They seems to do the same thing, just are written differently. Is it purely a matter of style?
var query = from p in Products
where p.Name.Contains("foo")
orderby c.Name
select p;
// or with extension methods:
var query = Products
.Where(p => p.Name.Contains("foo"))
.OrderBy(p => p.Name);
They're very similar with the second example being a bit more terse, but perhaps less expressive if you don't know what the => is doing.
Other than writing terse code, are there other advantages to using the extension methods as opposed to the LINQ syntax?

Honestly, sometimes it can be situational once you start using Funcs and Actions. Say you are using these three funcs:
Func<DataClasses.User, String> userName = user => user.UserName;
Func<DataClasses.User, Boolean> userIDOverTen = user => user.UserID < 10;
Func<DataClasses.User, Boolean> userIDUnderTen = user => user.UserID > 10;
As you can see the first one replaces the lamdba expression to get the user name, the second replaces a lamdba expression used to check if the ID is lower than 10, and let's face it, the third should be pretty easy to understand now.
NOTE: This is a silly example but it works.
var userList =
from user in userList
where userIDOverTen(user)
select userName;
Versus
var otherList =
userList
.Where(IDIsBelowNumber)
.Select(userName)
In this example, the second is a little less verbose since the extension method can make full use of the Func, but he Linq expression can't since it is look just for a Boolean rather than a Func that returns boolean. However, this is where it might be better to use the expression language. Say you already had a method that takes in more than just a user:
private Boolean IDIsBelowNumber(DataClasses.User user,
Int32 someNumber, Boolean doSomething)
{
return user.UserID < someNumber;
}
Note: doSomething is just there because of the where extension method being ok with a method that takes in a user and integer and returns boolean. Kind of annoying for this example.
Now if you look at the Linq query:
var completeList =
from user in userList
where IDIsBelowNumber(user, 10, true)
select userName;
You're good for it. Now the Extension Method:
var otherList =
userList
.Where(IDIsBelowNumber????)
.Select(userName)
Without a lambda expression, I really can't call that method. So now what I have to do is create a method that creates a Func based off the original method call.
private Func<DataClasses.User, Boolean> IDIsBelowNumberFunc(Int32 number)
{
return user => IDIsBelowNumber(user, number, true);
}
And then plug it in:
var otherList =
userList
.Where(IDIsBelowNumberFunc(10))
.Select(userName)
So you can see, sometimes it may just be easier to use the query approach at times.

One advantage to using LINQ extension methods (method-based queries) is that you can define custom extension methods and it will still read fine.
On the other hand, when using a LINQ query expression, the custom extension method is not in the keywords list. It will look a bit strange mixed with the other keywords.
Example
I am using a custom extension method called Into which just takes a string:
Example with query
var query = (from p in Products
where p.Name.Contains("foo")
orderby c.Name
select p).Into("MyTable");
Example with extension methods
var query = Products
.Where(p => p.Name.Contains("foo"))
.OrderBy(p => p.Name)
.Into("MyTable");
In my opinion the latter, using a method-based query, reads better when you have custom extension methods.

I think it's a good idea not to use them together and choose one and stick with it.
Mostly it's personal taste, but in the query syntax (Comprehension method) not all operators are available as was said before.
I find the Extension Methods syntax more in line with the rest of my code. I do my SQL in SQL. It's also very easy to build your expression just by adding everything on top of eachother with the extension methods.
Just my two cents.
As I cannot make comments yet I want to make one here to the answer of Programming Tool:
Why make a whole new method for the last example?? Can't you just use:
.Where(user => IDIsBelowNumber(user, 10, true))

They compile the same, and are equivalent. Personally, I prefer the lambda (extension) methods for most things, only using the statements (standard) if I'm doing LINQ to SQL or otherwise trying to emulate SQL. I find that the lambda methods flow better with code, whereas the statements are visually distracting.

I prefer the extension method syntax when I use Linq methods that have no query syntax equivalent, such as FirstOrDefault() or others like that.

I like to use the query syntax when its really a query, ie a lazy expression which evaluates on demand.
A method that looks like regular method calls (method syntax or the lambda syntax) doesn't look lazy enough, so I use that as a convention. For eg,
var query = from p in Products
where p.Name.Contains("foo")
orderby p.Name
select p;
var result = query.ToList(); //extension method syntax
If it's a not a query, I like the fluent style which looks to me consistent with other eagerly executing calls.
var nonQuery = Products.Where(p => p.Name.Contains("foo"))
.OrderBy(p => p.Name)
.ToList();
It helps me to differentiate the two styles of calls better. Of course there are situations where you will be forced to use method syntax anyway, so my convention is not very compelling enough.

One advantage of extension methods/lynda expressions is the additional operators that is offered like Skip and Take. For example, if you are creating a pagination method, being able to skip the first 10 records and take the next 10 is easy to implement.