We Keep Coding

Evaluator.PartialEval reduce provided expression

In one of my projects I have an ExpressionVisitor to translate provided expression into some query string. But before translating it I need to evaluate all refferences in the expression to real values. To do that I use Evaluator.PartialEval method from EntityFramework Project.
Assuming I have this query:
var page = 100;
var query = myService.AsQueryable<Product>()
//.Where(x=>x.ProductId.StartsWith(p.ProductId))
.Skip(page)
.Take(page);
var evaluatedQueryExpr = Evaluator.PartialEval(query.Expression);
As you can see I have commented Where method. In this case evaluatedQueryExpr will not contain the methods Take and Skip.
However, if I use any other method with Expression before Take or Skip everything works, Evaluator evaluates an expression correctly and return it fully.
I found out that the problem occurs in the line 80 of the Evaluator class:
return Expression.Constant(fn.DynamicInvoke(null), e.Type);
Could you explain why this happens and suggest a workaround?
Update
here is a project on github
LinqToSolrQueriable inherited from IOrderedQueryable
LinqToSolrProvider inherited from IQueryProvider including line range causing the issue

The good news are that the expression is not really reduced (Skip and Take are still there :), but is simply converted from MethodCallExpression to ConstantExpression containing the original expression:
query.Expression:
.Call System.Linq.Queryable.Take(
.Call System.Linq.Queryable.Skip(
.Constant<LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product]>(LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product]),
100),
100)
evaluatedQueryExpr:
.Constant<System.Linq.IQueryable`1[LinqToSolrTest.Product]>(LinqToSolr.Query.LinqToSolrQueriable`1[LinqToSolrTest.Product])
Here the debug display is giving you a wrong impression. If you take the ConstaintExpression.Value, you'll see that it's a IQueryable<Product> with Expression property being exactly the same as the original query.Expression.
The bad news are that this is not what you expect from PartialEval - in fact it doesn't do anything useful in this case (except potentially breaking your query translation logic).
So why is this happening?
The method you are using from EntityFramework.Extended library is in turn taken (as indicated in the comments) from MSDN Sample Walkthrough: Creating an IQueryable LINQ Provider. It can be noticed that the PartialEval method has two overloads - one with Func<Expression, bool> fnCanBeEvaluated parameter used to identify whether a given expression node can be part of the local function (in other words, to be partially evaluated or not), and one without such parameter (used by you) which simply calls the first passing the following predicate:
private static bool CanBeEvaluatedLocally(Expression expression)
{
return expression.NodeType != ExpressionType.Parameter;
}
The effect is that it stops evaluation of ParameterExpression type expressions and any expressions containing directly or indirectly ParameterExpression. The last should explain the behavior you are observing. When the query contains Where (and basically any LINQ operator) with parametrized lambda expression (hence parameter) before the Skip / Take calls, it would stop evaluation of the containing methods (which you can see from the above query.Expression debug view - the Where call will be inside the Skip).
Now, this overload is used by the MSDN example to evaluate a concrete nested Where method lambda expression and is not generally applicable for any type of expression like IQueryable.Expression. In fact the linked project is using the PartialEval method in a single place inside QueryCache class, and also calling the other overload passing a different predicate which in addition to ParameterExpressions stops the evaluation of any expression with result type of IQueryable.
Which I think is the solution of your problem as well:
var evaluatedQueryExpr = Evaluator.PartialEval(query.Expression,
// can't evaluate parameters or queries
e => e.NodeType != ExpressionType.Parameter &&
!typeof(IQueryable).IsAssignableFrom(e.Type)
);

how to change OrderBy direction of IQueryable

I have some function (X) which return IQueryable with "OrderBy(x => x.Name)".
I want to use X function in new code, but in this code there is parameter which determine the order (asc/desc).
I prefer not to change (X) since it used in multiple places already.
there is option to get X(), then cancel it's "OrderBy", and apply new order?
(now it throws exception, since it is like Y.OrderBy(a => a.Name).OrderByDescending(a => a.Name))
"A column has been specified more than once in the order by list. Columns in the order by list must be unique.\r\nStatement(s) could not be prepared."

use
public void x(string order = string.empty)
{
Y.OrderBy(a => a.Name);
if (order == "desc")
{
Y = Y.Reverse();
}
}

I've tried to wiggle Expression object of IOrderedQueryable that your method is actually returning (am I wrong?) but so far no luck, and I can't see the option to manipulate System.Linq.Expression or System.Linq.EnumerableQuery easy enough to use it instead of changing X() method.
IMHO, you should choose from:
create somes kind of wrapper method using the Reverse() (#Eric Lizotte answer) or OrderByDescending() - it will be less time consuming I guess but for grouping, filtering etc. you will have to create brand new function.
remove OrderBy() from X() function body, and modify your old code - returning non-ordered result will give you more flexibility and opportunity to do what you want to do with the result via linq expressions in each scenario that you'll came across.
Can you clarify for me if you're talking about "function" you have in mind a sql function that is mapped via ORM (eg. Entity Framework) in your application or just a ordinary method which is using your custom provider to return IQueryable?

Would using an extension method to query an IEnumerable be equivalent to using IQueryable?

I understand the main difference between IQueryable and IEnumerable. IQueryable executes the query in the database and returns the filtered results while IEnumerable brings the full result set into memory and executes the filtering query there.
I don't think using an extension method to query on the initial assignment of a variable will cause it to be executed in the database like IQueryable, but I just wanted to make sure.
This code will cause the full result set of People to be returned from the database, and then the filtering is done in memory:
(The People property on the context is of type DbSet)
IEnumerable<Person> people = context.People;
Person person = people.Where(x => x.FirstName == "John");
Even though I am adding the filtering below as an extension method before assigning the item to my variable, I'm assuming this code should work the same way as the code above, and bring back the full result set into memory before filtering it, right?
Person person = context.People.Where(x => x.FirstName == "John");
EDIT:
Thanks for the replies guys. I modified the code example to show what I meant (removed the IEnumerable in the second paragraph of code).
Also, to clarify, context.People is of type DbSet, which implements both IQueryable and IEnumerable. So I 'm not actually sure which .Where method is being called. IntelliSense tells me it is the IQueryable version, but can this be trusted? Is this always the case when working directly with a DbSet of a context?

IEnumerable<Person> people = context.People;
Person person = people.Where(x => x.FirstName == "John");
... will execute the IEnumerable<T>.Where method extension, which accepts a Func<TSource, bool> predicate parameter, forcing the filtering to happen in memory.
In contrast...
IEnumerable<Person> people = context.People.Where(x => x.FirstName == "John");
...will execute the IQueryable<T>.Where method extension, which accepts a Expression<Func<TSource, bool>> predicate parameter. Notice that this is an expression, not a delegate, which allows it to translate the where condition to a database condition.
So it really does make a difference which extension method you invoke.

IQueryable<T> works on expressions. It is effectively a query-builder, accumulating information about a query without doing anything... until the moment you need a value from it, when:
a query is generated from the accumulated Expression, in the target language (e.g. SQL)
the query is executed, usually on the database,
results are converted back to a C# object.
IEnumerable<T> extensions works on pre-compiled functions. When you need a value from it:
C# code in those functions is executed.
It is easy to confuse the two, because:
both have extension functions with similar names,
the lambda syntax is the same for Expressions and Functions - so you cannot tell them apart,
the use of "var" to declare variables removes the datatype (often the only clue as to which interface is being used).
IQueryable<int> a;
IEnumerable<int> b;
int x1 = a.FirstOrDefault(i => i > 10); // Expression passed in
int x2 = b.FirstOrDefault(i => i > 10); // Function passed in
Extension methods with the same name usually do the same thing (because they were written that way) but sometimes they don't.
So the answer is: No, they are not equivalent.

Using Expression<Func<>> in a LINQ Query

I want to define a Func<ProductItemVendor, bool> filter expression named CompareProductItemVendorIds, which can be used throughout my application, mostly within Entity Framework/LINQ queries.
I've learned that in order to be able to use this filter in a LINQ query, I must declare it as Expression<Func<>> instead of just Func<>. I understand the reason for this, and it's easy for me to do this.
But I'm having the following problems using that expression in my queries.
First, code such as:
ProductItem.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds)
Note: ProductItem is a database entity, and its ProductItemVendors property is a navigation collection.
Produces the error:
`Instance argument: cannot convert from 'System.Collections.Generic.ICollection' to 'System.Linq.IQueryable'
And second, code such as:
var results = from v in Repository.Query<ProductItemVendor>()
where CompareProductItemVendorIds(v)
select v;
Produces the error:
'CompareProductItemVendorIds' is a 'variable' but is used like a 'method'
So I have my nice shiny new Expression<Func<>>. How can I use it in my LINQ queries?

ProductItem is already an Entity, so you can't use your Expression, you need to use Compile() to get the Func<> from your Expression<Func<>> since ProductItemVendors is no longer an IQueryable
ProductItem.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds.Compile())
You would have to use your Expression on the ProductItemVendorsContext like this:
var item = Context.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds);
You cant use an Expression inside query syntax, you need to use method sytanx
var results = from v in Repository.Query<ProductItemVendor>()
.Where(CompareProductItemVendorIds)
select v;

The first case;
ProductItemVendor productItemVendor = ProductItem.ProductItemVendors.FirstOrDefault(CompareProductItemVendorIds);
Produces the error:
`Instance argument: cannot convert from 'System.Collections.Generic.ICollection' to 'System.Linq.IQueryable'
Happens because it is an ICollection, the entity has already been loaded. Given dbContext or objectContext lazy loading vs explicit loading is implemented a bit differently, I assume you are using explicit loading though. If you change the loading to lazy the type of ProductItemVendors will be IQueryable and what you are trying will succeed.
Given the second case, the expression must be compilable to SQL, else you get a lot of weird errors, probably it's possible that that is the case here.
It's hard to give you more explicit help given the information in the question, I cannot recreate it easily. If you can create a MWE-solution and upload it somewhere I can have a look, but I'm afraid I can't help more here.

I don't think composing expressions like that is supported by default.
You could try LINQKit
Here's an example from the LINQKit page; one Expression<Func<>> inside another:
Call Invoke to call the inner expression Call Expand on the final
result. For example:
Expression<Func<Purchase,bool>> criteria1 = p => p.Price > 1000;
Expression<Func<Purchase,bool>> criteria2 = p => criteria1.Invoke (p)
|| p.Description.Contains ("a");
Console.WriteLine (criteria2.Expand().ToString());
(Invoke and Expand are extension methods in LINQKit.) Here's the output:
p => ((p.Price > 1000) || p.Description.Contains("a"))
Notice that we have a nice clean expression: the call to Invoke has
been stripped away.

Error when using a Linq Expression variable instead of lambda expression directly

I have the following code to get a collection of Types from an Assembly where the Type is a Class:
Assembly assembly = Assembly.LoadFile(DLLFile);
var types = assembly.GetTypes().AsEnumerable().Where(x => x.IsClass);
This works fine and as expected. However I wanted to pull out the lambda expression to a Linq Expression variable (as later on it will be used in a parameter of this method). So I did the following:
private Expression<Func<Type, bool>> _standardFilter = (x => x.IsClass);
Assembly assembly = Assembly.LoadFile(DLLFile);
var types = assembly.GetTypes().AsEnumerable().Where(_standardFilter);
However this won't compile with the error:
System.Collections.Generic.IEnumerable<System.Type>' does not contain a
definition for 'Where' and the best extension method overload
'System.Linq.Enumerable.Where<TSource>(System.Collections.Generic.IEnumerable<TSource>, System.Func<TSource,int,bool>)' has some invalid arguments
I understand that my expression doesn't conform to the predicate System.Func<TSource,int,bool>, however the Where function has an overload that takes a predicate of System.Func<TSource,bool>, which as far as I can tell should work.
I have tried converting the result of assembly.GetTypes() (which is an array) to a List in several ways without it helping the issue.
I have also made sure that I have got all the correct using statements for this class as that seems to be an issue several people run into during my Googling.
In the past I have managed to use this same technique on a IQueryable collection, but I don't understand why this won't work when the Where function is available on the IEnumerable collection and should accept the predicate I am providing.
Thanks very much for any assistance.

Compile your expression into executable code (delegate):
var types = assembly.GetTypes().AsEnumerable().Where(_standardFilter.Compile());
As #Kirk stated, it's better not to use expression tree, if you are not going to analyze it. Simply use filter of type Func<Type, bool>.

Note that the Enumerable extension method takes a Func, not an Expression. So your argument is incorrectly typed.
This is in contrast with Entity Framework, which takes Expression<Func> rather than plain Func.
So be attentive to method signatures, they are similar, can both be called with the same lambda, but are actually different!
The reason behind that is that EF inspects the Expression and converts it to SQL code, whereas Linq to Object simply executes the predicate in memory.

You could try this instead:
private Func<Type, bool> standardFilter;
standardFilter = (type) => {
return type.IsClass;
};
var types = assembly.GetTypes().AsEnumerable().Any(x => standardFilter(x));
Then you're free to change your implementation of standardFilter. As long as it takes in a Type and returns bool for whether or not that should be included it will remain modular and accomplish what I think you're going for.