We Keep Coding

Lambda expression inside another lambda expression [duplicate]

I have an existing expression of type Expression<Func<T, object>>; it contains values like cust => cust.Name.
I also have a parent class with a field of type T. I need a method that accepts the above as a parameter and generates a new expression that takes the parent class (TModel) as a parameter. This will be used as an expression parameter of an MVC method.
Thus, cust => cust.Name becomes parent => parent.Customer.Name.
Likewise, cust => cust.Address.State becomes parent => parent.Customer.Address.State.
Here's my initial version:
//note: the FieldDefinition object contains the first expression
//described above, plus the MemberInfo object for the property/field
//in question
public Expression<Func<TModel, object>> ExpressionFromField<TModel>(FieldDefinition<T> field)
where TModel: BaseModel<T>
{
var param = Expression.Parameter(typeof(TModel), "t");
//Note in the next line "nameof(SelectedItem)". This is a reference
//to the property in TModel that contains the instance from which
//to retrieve the value. It is unqualified because this method
//resides within TModel.
var body = Expression.PropertyOrField(param, nameof(SelectedItem));
var member = Expression.MakeMemberAccess(body, field.Member);
return Expression.Lambda<Func<TModel, object>>(member, param);
}
The error I'm currently receiving is when I have a field with multiple parts (i.e. cust.Address.State instead of just cust.Name). I get an error on the var member line that the specified member doesn't exist--which is true, since the body at that refers to the parent's child (Customer) and not the item that contains the member (Address).
Here's what I wish I could do:
public Expression<Func<TModel, object>> ExpressionFromField<TModel>(FieldDefinition<T> field)
where TModel: BaseModel<T>
{
var param = Expression.Parameter(typeof(TModel), "t");
var body = Expression.PropertyOrField(param, nameof(SelectedItem));
var IWantThis = Expression.ApplyExpressionToField(field.Expression, body);
return Expression.Lambda<Func<TModel, object>>(IWantThis, param);
}
Any help getting to this point would be greatly appreciated.
Edit: This was marked as a possible duplicate of this question; however, the only real similarity is the solution (which is, in fact, the same). Composing expressions is not an intuitive solution to accessing nested properties via expressions (unless one's inuition is guided by certain experience, which should not be assumed). I also edited the question to note that the solution needs to be suitable for a paramter of an MVC method, which limits the possible solutions.

What you're looking for is the ability to compose expressions, just as you can compose functions:
public static Expression<Func<T, TResult>> Compose<T, TIntermediate, TResult>(
this Expression<Func<T, TIntermediate>> first,
Expression<Func<TIntermediate, TResult>> second)
{
return Expression.Lambda<Func<T, TResult>>(
second.Body.Replace(second.Parameters[0], first.Body),
first.Parameters[0]);
}
This relies on the following method to replace all instances of one expression with another:
public class ReplaceVisitor:ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression ex)
{
if(ex == from) return to;
else return base.Visit(ex);
}
}
public static Expression Replace(this Expression ex,
Expression from,
Expression to)
{
return new ReplaceVisitor(from, to).Visit(ex);
}
You can now take an expression selecting a property:
Expression<Func<Customer, object>> propertySelector = cust => cust.Name;
And an expression selecting that object from the model:
Expression<Func<CustomerModel, Customer>> modelSelector = model => model.Customer;
and compose them:
Expression<Func<Customer, object> magic = modelSelector.Compose(propertySelector);

How to convert UNION ALL to OR in Linq to SQL expression trees (Oracle ORA-00932: inconsistent datatypes: expected got CLOB)

I am working with EF6 and am using db first generated models for MSSQL and Oracle. In few places I am searching by multiple search criteria which results in UNION ALL sql generated where each query is being in it's own sub-select.
One of columns in Oracle table is CLOB and linq to sql after it wraps all selects with UNION ALL at the top of all UNIONS it calls SELECT DISTINCT
"UnionAll1"."UNIQUE_ID" AS "C1", ... which requires to compare CLOBs and fails on Oracle side.
ORA-00932: inconsistent datatypes: expected - got CLOB
Description: An unhandled exception occurred during the execution of
the current web request. Please review the stack trace for more
information about the error and where it originated in the code.
Exception Details: Oracle.ManagedDataAccess.Client.OracleException:
ORA-00932: inconsistent datatypes: expected - got CLOB
Is there a way to remove that DISTINCT statement? How can I make this work?
UPDATE
Mechanism that generates LINQ looks like this:
public static IQueryable<T> ApplySearch<T>(this IQueryable<T> queryable, SearchModel search) where T : class
{
var subQueries = new List<IQueryable<T>>();
if (search != null)
{
if (search.PolicyNumber.HasValue && typeof (IPolicyNumber).IsAssignableFrom(queryable.ElementType))
{
subQueries.Add(queryable.SearchByPolicyNumber(search));
}
if (search.UniqueId.HasValue && typeof (IUniqueId).IsAssignableFrom(queryable.ElementType))
{
subQueries.Add(queryable.SearchByUniqueId(search));
}
if (!string.IsNullOrWhiteSpace(search.PostCode) && typeof(IPostCode).IsAssignableFrom(queryable.ElementType))
{
subQueries.Add(queryable.SearchByPostCode(search));
}
}
return subQueries.DefaultIfEmpty(queryable)
.Aggregate((a, b) => a.Union(b));
}
Example of specific search method:
public static IQueryable<IRequestId> SearchByRequestId<IRequestId>(this IQueryable<IRequestId> queryable, SearchModel search)
{
var interfacesToColumnNames = new Dictionary<Type, string>
{
{typeof (IRequestId<>), "requestid"},
{typeof (IRequest_Id<>), "request_id"},
};
var paramLambda = Expression.Parameter(typeof (IRequestId));
var columnLambda = Expression.Property(paramLambda, interfacesToColumnNames.Single(o => queryable.ElementType.GetInterfaces().Any(oo => oo.Name == o.Key.Name)).Value);
var lambda = Expression.Lambda<Func<IRequestId, bool>>(
Expression.Equal(columnLambda, Expression.Convert(Expression.Constant(search.RequestId), columnLambda.Type)), paramLambda);
queryable = queryable.Where(lambda);
return queryable;
}
Example where it gets called in controller:
public ActionResult QUOTE_HOUSE()
{
var onlineDocs =
this.DatabaseManager.GetEntities<QUOTE_HOUSE>().ApplySearch(Search)
.Take(10);
return View("QUOTE_HOUSE", onlineDocs.ToList());
}

Based on the additional information from the comments, the problematic queries are produced by the following procedure:
public static IQueryable<T> ApplySearch<T>(this IQueryable<T> queryable, SearchModel search) where T : class
{
var subQueries = new List<IQueryable<T>>();
if (search != null)
{
if (search.PolicyNumber.HasValue && typeof (IPolicyNumber).IsAssignableFrom(queryable.ElementType))
{
subQueries.Add(queryable.SearchByPolicyNumber(search));
}
if (search.UniqueId.HasValue && typeof (IUniqueId).IsAssignableFrom(queryable.ElementType))
{
subQueries.Add(queryable.SearchByUniqueId(search));
}
if (!string.IsNullOrWhiteSpace(search.PostCode) && typeof(IPostCode).IsAssignableFrom(queryable.ElementType))
{
subQueries.Add(queryable.SearchByPostCode(search));
}
}
return subQueries.DefaultIfEmpty(queryable)
.Aggregate((a, b) => a.Union(b));
}
where I assume the supporting methods are something like this
public static IQueryable<T> SearchByPolicyNumber<T>(this IQueryable<T> queryable, SearchModel search) where T : class
{
return queryable.Where(x => predicate_using_PolicyNumber(x, search));
}
public static IQueryable<T> SearchByUniqueId<T>(this IQueryable<T> queryable, SearchModel search) where T : class
{
return queryable.Where(x => predicate_using_UniqueId(x, search));
}
public static IQueryable<T> SearchByPostCode<T>(this IQueryable<T> queryable, SearchModel search) where T : class
{
return queryable.Where(x => predicate_using_PostCode(x, search));
}
The problem is that EF translates the LINQ Union operator to SQL UNION ALL subquery with applied DISTINCT SELECT ... as you already found. I have no idea why it does it this way instead of simply translating it to SQL UNION, but actually there is no guarantee that it would work with such type of columns either.
The only way to solve the issue I see is to eliminate the Union operator by replacing it with a single Where with Or conditions. In order to do that, you have to slightly change your design.
First, extract the predicate part from the supporting methods:
public static class SearchPredicates
{
public static Expression<Func<T, bool>> ByPolicyNumber<T>(SearchModel search) where T : class
{
return x => predicate_using_PolicyNumber(x, search);
}
public static Expression<Func<T, bool>> ByUniqueId<T>(SearchModel search) where T : class
{
return x => predicate_using_UniqueId(x, search);
}
public static Expression<Func<T, bool>> ByPostCode<T>(SearchModel search) where T : class
{
return x => predicate_using_PostCode(x, search);
}
}
Then modify the main method like this:
public static IQueryable<T> ApplySearch<T>(this IQueryable<T> queryable, SearchModel search) where T : class
{
var predicates = new List<Expression<<Func<T, bool>>>();
if (search != null)
{
if (search.PolicyNumber.HasValue && typeof (IPolicyNumber).IsAssignableFrom(queryable.ElementType))
predicates.Add(SearchPredicates.ByPolicyNumber(search));
if (search.UniqueId.HasValue && typeof (IUniqueId).IsAssignableFrom(queryable.ElementType))
predicates.Add(SearchPredicates.ByUniqueId(search));
if (!string.IsNullOrWhiteSpace(search.PostCode) && typeof(IPostCode).IsAssignableFrom(queryable.ElementType))
predicates.Add(SearchPredicates.ByPostCode(search));
}
if (predicates.Count == 0)
return queryable;
var parameter = predicates[0].Parameters[0];
var condition = predicates[0].Body;
for (int i = 1; i < predicates.Count; i++)
condition = Expression.Or(condition, predicates[i].Body.ReplaceParameter(predicates[i].Parameters[0], parameter));
var predicate = Expression.Lambda<Func<T, bool>>(condition, parameter);
return queryable.Where(predicate);
}
You can use any EF compatible predicate builder, here I'm building the predicate manually. The helper method used is:
public static class ExpressionUtils
{
public static Expression ReplaceParameter(this Expression expression, ParameterExpression source, Expression target)
{
return new ParameterReplacer { Source = source, Target = target }.Visit(expression);
}
class ParameterReplacer : ExpressionVisitor
{
public ParameterExpression Source;
public Expression Target;
protected override Expression VisitParameter(ParameterExpression node)
{
return node == Source ? Target : base.VisitParameter(node);
}
}
}
With all that applied, hopefully the issue will be resolved.

Just figured an alternative.
return subQueries.DefaultIfEmpty(queryable)
.Aggregate((a, b) => a.Concat(b));
then after ToList() where you are actually using output .Distinct(new YourEqualityComparer())). To filter out possible duplicates.
This isn't ideal solution since requires you to manually implement IEquitable on structures, but if you are after performance it is likely to be faster. OR will require composite indexes on database while querying by two separate single column indexes then combining the data will be not require to have all possible column-index combinations on table. Another downside you are likely to use Take() before ToList() and then after filtering you might get to few records (and have to re-query).

Returning filtered IQueryable<T> with anonymous types

I have a set of Reports which I need to perform filtering on before returning the output. I would like to perform this with a single anonymous method to avoid duplicating the same code in different repositories. I'm using Entity Framework so the model types all related to the database and inherit from a base class called ReportBase.
This is what how I currently implement the filtering, each report type has to implement this method with a different context and returning a different IQueryable type.
private IQueryable<ReviewAgreement> GetFiltered(ReportFilter filter)
{
IQueryable<ReviewAgreement> reviewAgreementQueryable = Context.ReviewAgreements.Where(p => p.ClientWorkflowId == filter.ClientWorkflowId);
if (filter.AppraisalLevelId.HasValue)
{
reviewAgreementQueryable = reviewAgreementQueryable.Where(p => p.AppraisalLevelId == filter.AppraisalLevelId.Value);
}
return reviewAgreementQueryable;
}
I've been trying to implement this anonymously so I can reuse it, as in this non functional example.
public IQueryable<T> GetFiltered(ReportFilter filter)
{
IQueryable<T> reportQueryable = Context.Set<T>();
reportQueryable = reportQueryable.Where(p => p.ClientWorkflowId == filter.ClientWorkflowId);
if (filter.AppraisalLevelId.HasValue)
{
reportQueryable = reportQueryable.Where(p => p.AppraisalLevelId == filter.AppraisalLevelId.Value);
}
return reportQueryable;
}
The issue I am having is of course that the use of Where is ambiguous, so it cannot resolve p.ClientWorkflowId.
I have tried using a Func<T, TResult> delegate to pass in the filtering options this but the Where operation seems to want to return a list.
Is there actually a method I can use to achieve the effect I want?

Declare an interface that has the two ID properties that you need to perform this operation.
Ensure that your entities implement that interface.
Add a constraint to the generic argument that it implements that interface.
Note that if your base class defines both of the properties in question then you don't need an interface, and can simply constrain the type to that base class:
public IQueryable<T> GetFiltered<T>(ReportFilter filter) where T : ReportBase
{
// body unchanged
}
If you want to go down the route of accepting parameters to represent these properties than it's also possible. The first thing is that you'll need to accept expressions, not Func objects, so that the query provider can analyze them. This means changing the function signature to:
public IQueryable<T> GetFiltered<T>(ReportFilter filter,
Expression<Func<T, int>> clientIdSelector,
Expression<Func<T, int>> appraisalIdSelector)
{
Next, to turn these selectors into predicates that compare the value to an ID that we have is a bit more involved for expressions than it is for regular delegates. What we really need here is a Compose method; for delegates it's simple enough, to compose one method with another you just invoke it with the parameter being the result of the first. With expressions this means taking the body of one expression and replacing all instance of the parameter with the body of another, and then wrapping the whole thing up in a new Lambda.
public static Expression<Func<TFirstParam, TResult>>
Compose<TFirstParam, TIntermediate, TResult>(
this Expression<Func<TFirstParam, TIntermediate>> first,
Expression<Func<TIntermediate, TResult>> second)
{
var param = Expression.Parameter(typeof(TFirstParam), "param");
var newFirst = first.Body.Replace(first.Parameters[0], param);
var newSecond = second.Body.Replace(second.Parameters[0], newFirst);
return Expression.Lambda<Func<TFirstParam, TResult>>(newSecond, param);
}
This itself is dependent on the ability to replace all instances of one expression with another. To do that we'll need to use the following:
public static Expression Replace(this Expression expression,
Expression searchEx, Expression replaceEx)
{
return new ReplaceVisitor(searchEx, replaceEx).Visit(expression);
}
internal class ReplaceVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public ReplaceVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
}
Now that we have all of this in place we can actually compose our selectors with a comparison to the ID values of the filter:
IQueryable<T> reportQueryable = Context.Set<T>();
reportQueryable = reportQueryable
.Where(clientIdSelector.Compose(id => id == filter.ClientWorkflowId));
if (filter.AppraisalLevelId.HasValue)
{
reportQueryable = reportQueryable
.Where(clientIdSelector.Compose(id => id == filter.AppraisalLevelId.Value));
}
return reportQueryable;

method with a return type of "Expression"

Is it possible to create a method which returns a lambda expression? I couldn't find any proper examples.
The following syntax doesn't work, of course. It's just to visualize my idea a bit:
// Executed code
var filteredList = listWithNames.Where(GetLambdaExpression("Adam"));
// method
public Expression GetLambdaExpression(string name)
{
return listElement => listElement.Name == name;
}

You can create functions which return expressions such as this as a simple example in a predicate builder:
public static Expression<Func<T, bool>> True<T>() { return param => true; }
or this expression builder:
static Expression<T> Compose<T>(this Expression<T> first, Expression<T> second, Func<Expression, Expression, Expression> merge)
{
var map = first.Parameters
.Select((f, i) => new { f, s = second.Parameters[i] })
.ToDictionary(p => p.s, p => p.f);
var secondBody = ParameterRebinder.ReplaceParameters(map, second.Body);
return Expression.Lambda<T>(merge(first.Body, secondBody), first.Parameters);
}
in your example case You should use what Leppie has mentioned below (which I have upvoted)
<Func<TypeOflistElement,bool>>

public Expression<Func<TypeOflistElement,bool>> GetLambdaExpression(string name)
{
return listElement => listElement.Name == name;
}

You can return Func<bool, T> type like this
// Executed code
var filteredList = listWithNames.Where(GetLambdaExpression("Adam"));
// method
public Func<bool, ListElementTypeName> GetLambdaExpression(string name)
{
return listElement => listElement.Name == name;
}
But I can't understand what exactly you are trying to do with it.

You have to return Func<> since IEnumerable expects one, as in your example it would be:
public Func<String,Bool> (string name){..}

Where method for IEnumerable<T> expects delegate for Func<T, bool> method, so GetLambdaExpression() must return Func<T, bool>
Where method for IQueryable<T> expects Expression<Func<T, bool>>, so GetLambdaExpression() must return Expression<Func<T, bool>>.
Expression can be converted to delegate by invoking Compile() method.

Expressions: a get-by-ID-selector when I have the Expression to get the ID?

I have a generic base repository class for LINQ-to-Entities that can do things like selecting an entity based on the ID.
In order to subclass it, you need to implement these two:
protected override Expression<Func<User, bool>> GetByIDSelector(int id)
{
return u => u.UserID == id;
}
protected override Expression<Func<User, int>> KeySelector
{
get { return u => u.UserID; }
}
I would like to just have the KeySelector (which is used for default sort order).
My 'get by ID' method looks like this:
public virtual TDataModel GetByID(TKey id)
{
var entity = SetProvider.Set<TDataModel>().Where(GetByIdSelector(id)).SingleOrDefault();
if (entity == null)
{
throw new EntityNotFoundException<TDataModel>(id);
}
return entity;
}
Where GetByIdSelector looks like this, which should turn the KeySelector expression into one that can select by ID.
private Expression<Func<TDataModel, bool>> GetByIdSelector(TKey id)
{
return Expression.Lambda<Func<TDataModel, bool>>
(
Expression.Equal(KeySelector.Body,
Expression.Constant(id)),
KeySelector.Parameters.Single()
);
}
However, the Entity Framework throws an exception that the parameter passed in by that isn't bound to the query.
The parameter 'u' was not bound in the specified LINQ to Entities query expression.
Is there any way I can cleanly reuse the KeySelector Expression without having to rebuild the entire thing?

The reason this doesn't work is because every time you call this.KeySelector, it's creating a brand new lambda expression (hence the call to KeySelector.Parameters.Single() is not the same parameter as the parameter used for KeySelector.Body). Cache that expression into a variable as follows and it will work.
private Expression<Func<TDataModel, bool>> GetByIdSelector(TKey id)
{
var keySelector = KeySelector;
return Expression.Lambda<Func<TDataModel, bool>>(
Expression.Equal(
keySelector.Body,
Expression.Constant(id)
),
keySelector.Parameters.Single()
);
}
What I would probably do is rather fix the code in the KeySelector property to use a fixed value such that the value retrieved is always the same instance.
Example:
private static readonly Expression<Func<User, int>> _keySelector = u => u.UserID;
protected override Expression<Func<User, int>> KeySelector
{
get { return _keySelector; }
}
Then if you do that - your old code for GetByIdSelector is actually no worse than my version above.