We Keep Coding

An expression tree lambda may not contain a null propagating operator

The line price = co?.price ?? 0, in the following code gives me the above error, but if I remove ? from co.? it works fine.
I was trying to follow this MSDN example where they are using ? on line select new { person.FirstName, PetName = subpet?.Name ?? String.Empty }; So, it seems I need to understand when to use ? with ?? and when not to.
Error:
an expression tree lambda may not contain a null propagating operator
public class CustomerOrdersModelView
{
public string CustomerID { get; set; }
public int FY { get; set; }
public float? price { get; set; }
....
....
}
public async Task<IActionResult> ProductAnnualReport(string rpt)
{
var qry = from c in _context.Customers
join ord in _context.Orders
on c.CustomerID equals ord.CustomerID into co
from m in co.DefaultIfEmpty()
select new CustomerOrdersModelView
{
CustomerID = c.CustomerID,
FY = c.FY,
price = co?.price ?? 0,
....
....
};
....
....
}

The example you were quoting from uses LINQ to Objects, where the implicit lambda expressions in the query are converted into delegates... whereas you're using EF or similar, with IQueryable<T> queryies, where the lambda expressions are converted into expression trees. Expression trees don't support the null conditional operator (or tuples).
Just do it the old way:
price = co == null ? 0 : (co.price ?? 0)
(I believe the null-coalescing operator is fine in an expression tree.)

The code you link to uses List<T>. List<T> implements IEnumerable<T> but not IQueryable<T>. In that case, the projection is executed in memory and ?. works.
You're using some IQueryable<T>, which works very differently. For IQueryable<T>, a representation of the projection is created, and your LINQ provider decides what to do with it at runtime. For backwards compatibility reasons, ?. cannot be used here.
Depending on your LINQ provider, you may be able to use plain . and still not get any NullReferenceException.

Jon Skeet's answer was right, in my case I was using DateTime for my Entity class.
When I tried to use like
(a.DateProperty == null ? default : a.DateProperty.Date)
I had the error
Property 'System.DateTime Date' is not defined for type 'System.Nullable`1[System.DateTime]' (Parameter 'property')
So I needed to change DateTime? for my entity class and
(a.DateProperty == null ? default : a.DateProperty.Value.Date)

While expression tree does not support the C# 6.0 null propagating, what we can do is create a visitor that modify expression tree for safe null propagation, just like the operator does!
Here is mine:
public class NullPropagationVisitor : ExpressionVisitor
{
private readonly bool _recursive;
public NullPropagationVisitor(bool recursive)
{
_recursive = recursive;
}
protected override Expression VisitUnary(UnaryExpression propertyAccess)
{
if (propertyAccess.Operand is MemberExpression mem)
return VisitMember(mem);
if (propertyAccess.Operand is MethodCallExpression met)
return VisitMethodCall(met);
if (propertyAccess.Operand is ConditionalExpression cond)
return Expression.Condition(
test: cond.Test,
ifTrue: MakeNullable(Visit(cond.IfTrue)),
ifFalse: MakeNullable(Visit(cond.IfFalse)));
return base.VisitUnary(propertyAccess);
}
protected override Expression VisitMember(MemberExpression propertyAccess)
{
return Common(propertyAccess.Expression, propertyAccess);
}
protected override Expression VisitMethodCall(MethodCallExpression propertyAccess)
{
if (propertyAccess.Object == null)
return base.VisitMethodCall(propertyAccess);
return Common(propertyAccess.Object, propertyAccess);
}
private BlockExpression Common(Expression instance, Expression propertyAccess)
{
var safe = _recursive ? base.Visit(instance) : instance;
var caller = Expression.Variable(safe.Type, "caller");
var assign = Expression.Assign(caller, safe);
var acess = MakeNullable(new ExpressionReplacer(instance,
IsNullableStruct(instance) ? caller : RemoveNullable(caller)).Visit(propertyAccess));
var ternary = Expression.Condition(
test: Expression.Equal(caller, Expression.Constant(null)),
ifTrue: Expression.Constant(null, acess.Type),
ifFalse: acess);
return Expression.Block(
type: acess.Type,
variables: new[]
{
caller,
},
expressions: new Expression[]
{
assign,
ternary,
});
}
private static Expression MakeNullable(Expression ex)
{
if (IsNullable(ex))
return ex;
return Expression.Convert(ex, typeof(Nullable<>).MakeGenericType(ex.Type));
}
private static bool IsNullable(Expression ex)
{
return !ex.Type.IsValueType || (Nullable.GetUnderlyingType(ex.Type) != null);
}
private static bool IsNullableStruct(Expression ex)
{
return ex.Type.IsValueType && (Nullable.GetUnderlyingType(ex.Type) != null);
}
private static Expression RemoveNullable(Expression ex)
{
if (IsNullableStruct(ex))
return Expression.Convert(ex, ex.Type.GenericTypeArguments[0]);
return ex;
}
private class ExpressionReplacer : ExpressionVisitor
{
private readonly Expression _oldEx;
private readonly Expression _newEx;
internal ExpressionReplacer(Expression oldEx, Expression newEx)
{
_oldEx = oldEx;
_newEx = newEx;
}
public override Expression Visit(Expression node)
{
if (node == _oldEx)
return _newEx;
return base.Visit(node);
}
}
}
It passes on the following tests:
private static string Foo(string s) => s;
static void Main(string[] _)
{
var visitor = new NullPropagationVisitor(recursive: true);
Test1();
Test2();
Test3();
void Test1()
{
Expression<Func<string, char?>> f = s => s == "foo" ? 'X' : Foo(s).Length.ToString()[0];
var fBody = (Expression<Func<string, char?>>)visitor.Visit(f);
var fFunc = fBody.Compile();
Debug.Assert(fFunc(null) == null);
Debug.Assert(fFunc("bar") == '3');
Debug.Assert(fFunc("foo") == 'X');
}
void Test2()
{
Expression<Func<string, int>> y = s => s.Length;
var yBody = visitor.Visit(y.Body);
var yFunc = Expression.Lambda<Func<string, int?>>(
body: yBody,
parameters: y.Parameters)
.Compile();
Debug.Assert(yFunc(null) == null);
Debug.Assert(yFunc("bar") == 3);
}
void Test3()
{
Expression<Func<char?, string>> y = s => s.Value.ToString()[0].ToString();
var yBody = visitor.Visit(y.Body);
var yFunc = Expression.Lambda<Func<char?, string>>(
body: yBody,
parameters: y.Parameters)
.Compile();
Debug.Assert(yFunc(null) == null);
Debug.Assert(yFunc('A') == "A");
}
}

LINQ (.Contains()) query against whole DbSet

public List<DbSet> Get(String q = null)
{
List<DbSet> objs = new List<DbSet>();
if (!string.IsNullOrEmpty(q) && q != "undefined")
{
objs = from dealer in db.Dealers
where dealer.Contains
......(?????)
}
}
I have some 4 DbSets in my dbcontext class. I am able to run a search like this
objs = from dealer in db.Dealers
where dealer.Name.Contains(q)
However I want to be able to do something similar except do the search against all the fields in dealer, and not just name. All the fields are strings
EDIT
Okay so I am starting to think this is not the best way to do what I'm trying to achieve. I'm looking into something called "Full Text Search". Can someone either explain to me how this works in entity or give me a link to a good resource

You could write a linq extension method:
Checkout my blog post.
http://jnye.co/Posts/7/generic-iqueryable-or-search-on-multiple-properties-using-expression-trees
(classes are also in github: https://github.com/ninjanye/SearchExtensions)
public static IQueryable<T> Search<T>(this IQueryable<T> source, string searchTerm, params Expression<Func<T, string>>[] stringProperties)
{
if (String.IsNullOrEmpty(searchTerm))
{
return source;
}
var searchTermExpression = Expression.Constant(searchTerm);
//Variable to hold merged 'OR' expression
Expression orExpression = null;
//Retrieve first parameter to use accross all expressions
var singleParameter = stringProperties[0].Parameters.Single();
//Build a contains expression for each property
foreach (var stringProperty in stringProperties)
{
//Syncronise single parameter accross each property
var swappedParamExpression = SwapExpressionVisitor.Swap(stringProperty, stringProperty.Parameters.Single(), singleParameter);
//Build expression to represent x.[propertyX].Contains(searchTerm)
var containsExpression = BuildContainsExpression(swappedParamExpression, searchTermExpression);
orExpression = BuildOrExpression(orExpression, containsExpression);
}
var completeExpression = Expression.Lambda<Func<T, bool>>(orExpression, singleParameter);
return source.Where(completeExpression);
}
private static Expression BuildOrExpression(Expression existingExpression, Expression expressionToAdd)
{
if (existingExpression == null)
{
return expressionToAdd;
}
//Build 'OR' expression for each property
return Expression.OrElse(existingExpression, expressionToAdd);
}
private static MethodCallExpression BuildContainsExpression<T>(Expression<Func<T, string>> stringProperty, ConstantExpression searchTermExpression)
{
return Expression.Call(stringProperty.Body, typeof(string).GetMethod("Contains"), searchTermExpression);
}
You will also need this class:
//Create SwapVisitor to merge the parameters from each property expression into one
public class SwapVisitor : ExpressionVisitor
{
private readonly Expression from, to;
public SwapVisitor(Expression from, Expression to)
{
this.from = from;
this.to = to;
}
public override Expression Visit(Expression node)
{
return node == from ? to : base.Visit(node);
}
public static Expression Swap(Expression body, Expression from, Expression to)
{
return new SwapVisitor(from, to).Visit(body);
}
}
You can then write something like:
db.Dealers.Search(q, x => x.Field1,
x => x.Field2,
...
x => x.Field20)

Sorry, no short cuts here:
objs = from dealer in db.Dealers
where dealer.Name.Contains(q) ||
dealer.Field2.Contains(q) ||
...
dealer.Field20.Contains(q)
select dealer;
You have to specify which fields you are going to search the value in.

You can manually write conditions for all fields:
objs = from dealer in db.Dealers
where dealer.Name.Contains(q) ||
dealer.Foo.Contains(q) ||
// etc
dealer.Bar.Contains(q)
select dealer;
There is no simple way to tell Entity Framework to check all properties of entity for some condition.

Using linq to perform text-based search when number of search words can be variable [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
LINQ To SQL exception: Local sequence cannot be used in LINQ to SQL implementation of query operators except the Contains operator
I am trying the following query:
var data = (from bk in DataContext.Book
where ((searchArray.Count() == 0 || searchArray.ToList().Any(x => bk.Name.Contains(x))) ||
(searchArray.Count() == 0 || searchArray.ToList().Any(x => bk.Genre.Contains(x)))))
where searchArray is a Array containing the individual words that I want to search for, I split the string the user entered and put the results in this array. Whenever I try to run this I get the following error:
"Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator."
Can anyone tell me what I am doing wrong and what is the correct way to perform this search?
In a nutshell, I am trying to allow a user to enter a string like "Hello World" and for a query to be generated that will look for either hello or world or both. But, a user can enter any number of words.

The simplest option is probably to build the lambda expression by hand:
static class ContainsAny
{
private static readonly MethodInfo StringContains
= typeof(string).GetMethod("Contains", new[] { typeof(string) });
public static Builder<T> Words<T>(IEnumerable<string> words)
{
return new Builder<T>(words);
}
public static Builder<T> Words<T>(params string[] words)
{
return new Builder<T>(words);
}
public sealed class Builder<T>
{
private static readonly ParameterExpression Parameter
= Expression.Parameter(typeof(T), "obj");
private readonly List<Expression> _properties = new List<Expression>();
private readonly List<ConstantExpression> _words;
internal Builder(IEnumerable<string> words)
{
_words = words
.Where(word => !string.IsNullOrEmpty(word))
.Select(word => Expression.Constant(word))
.ToList();
}
public Builder<T> WithProperty(Expression<Func<T, string>> property)
{
if (_words.Count != 0)
{
_properties.Add(ReplacementVisitor.Transform(
property, property.Parameters[0], Parameter));
}
return this;
}
private Expression BuildProperty(Expression prop)
{
return _words
.Select(w => (Expression)Expression.Call(prop, StringContains, w))
.Aggregate(Expression.OrElse);
}
public Expression<Func<T, bool>> Build()
{
if (_words.Count == 0) return (T obj) => true;
var body = _properties
.Select(BuildProperty)
.Aggregate(Expression.OrElse);
return Expression.Lambda<Func<T, bool>>(body, Parameter);
}
}
private sealed class ReplacementVisitor : ExpressionVisitor
{
private ICollection<ParameterExpression> Parameters { get; set; }
private Expression Find { get; set; }
private Expression Replace { get; set; }
public static Expression Transform(
LambdaExpression source,
Expression find,
Expression replace)
{
var visitor = new ReplacementVisitor
{
Parameters = source.Parameters,
Find = find,
Replace = replace,
};
return visitor.Visit(source.Body);
}
private Expression ReplaceNode(Expression node)
{
return (node == Find) ? Replace : node;
}
protected override Expression VisitConstant(ConstantExpression node)
{
return ReplaceNode(node);
}
protected override Expression VisitBinary(BinaryExpression node)
{
var result = ReplaceNode(node);
if (result == node) result = base.VisitBinary(node);
return result;
}
protected override Expression VisitParameter(ParameterExpression node)
{
if (Parameters.Contains(node)) return ReplaceNode(node);
return Parameters.FirstOrDefault(p => p.Name == node.Name) ?? node;
}
}
}
With this code in place, you can call:
Expression<Func<Book, bool>> filter = ContainsAny
.Words<Book>(searchArray)
.WithProperty(book => book.Name)
.WithProperty(book => book.Genre)
.Build();
var data = DataContext.Book.Where(filter);
For example, if the searchArray contains { "Hello", "World" }, the generated lambda will be:
obj => (obj.Name.Contains("Hello") || obj.Name.Contains("World"))
|| (obj.Genre.Contains("Hello") || obj.Genre.Contains("World")))

If I understand what you're trying to do correctly, you should be able to condense your query down to:
from bk in DataContext.Book
where searchArray.Contains(bk.Name) || searchArray.Contains(bk.Genre)
select bk
This is basically equivalent to the SQL:
select bk.*
from Book bk
where bk.Name in (...) or bk.Genre in (...)

In your case you must combine interpreted and local queries which can hurt performance or use SQL CLR integration by creating CLR function on database.

How can i keep reference of a lambda expression that will be later passed to OrderBy or LoadWith functions?

I am trying to create a general method for accessing the DB. This will imply parameters like: page index, number of items to display per page, ordering options, load options etc.
...
public IQuerable<T> GetAll(int pageIndex, int itemsToDisplayPerPage, System.Linq.Expressions.Expression<Func<T,object>>[] orderBy, System.Linq.Expressions.Expression<Func<T,object>>[] loadOptions)
{
DataContext dc = null;
IQuerable<T> result = null;
// Make some initalizations
...
foreach(var item in orderBy)
{
result = result.OrderBy(item);
}
System.Data.Linq.DataLoadOptions loadOptions = new System.Data.Linq.DataLoadOptions();
foreach(var item in loadOptions)
{
loadOptions.LoadWith(item);
}
...
}
...
The problem is that the System.Linq.Expressions.Expression< Func< T,object > > type is not a good general representation of any lambda expression that will be passed for the both examples above.
On ordering will crash because of the object type that does not make any sense for ordering. Also on loadWith will not work. So i dont know how can i handle this problem. Any suggestions? Thank you.

Not sure about the LoadWith, as we are using linq to entities, but we successfully got orderby to work in a repository Get method. The client code looks like this:
var results = _repository.GetAll(
new GetAllCriteria()
.OrderBy(x => x.Property1)
.OrderBy(x => x.Property2)
);
We aren't using generics in repository methods just yet, that will probably come in a future refactor. But the criteria implementation looks like this:
public class GetAllCriteria
{
public Dictionary<Expression<Func<CustomType, object>>, bool> ToBeOrderedBy
{
get; private set;
}
public GetAllCriteria OrderBy(
Expression<Func<CustomType, object>> expression)
{
return OrderBy(expression, true);
}
public GetAllCriteria OrderByDescending(
Expression<Func<CustomType, object>> expression)
{
return OrderBy(expression, false);
}
private GetAllCriteria OrderBy(
Expression<Func<CustomType, object>> expression, bool isAscending)
{
if (expression != null)
{
if (ToBeOrderedBy == null)
ToBeOrderedBy = new Dictionary<Expression<Func<CustomType, object>>, bool>();
ToBeOrderedBy.Add(expression, isAscending);
}
return this;
}
}
Then, the repository orders like so:
public Collection<CustomType> GetAll(GetAllCriteria criteria)
{
var query = dbContext.CustomTypes.AsQueryable();
// some code
// apply order by
if (criteria.ToBeOrderedBy != null && criteria.ToBeOrderedBy.Count > 0)
{
var firstOrderBy = criteria.ToBeOrderedBy.First();
query = firstOrderBy.Value
? query.OrderBy(firstOrderBy.Key)
: query.OrderByDescending(firstOrderBy.Key);
query = criteria.ToBeOrderedBy.Skip(1).Aggregate(query,
(lastOrderBy, nextOrderBy) => nextOrderBy.Value
? ((IOrderedQueryable<CustomType>)lastOrderBy)
.ThenBy(nextOrderBy.Key)
: ((IOrderedQueryable<CustomType>)lastOrderBy)
.ThenByDescending(nextOrderBy.Key));
}
// some more code
var results = query.ToList();
return results;
}
If this works with linq to entities, I would imagine it should work with linq to sql.

LINQ: Sorting on many columns dynamically

I have a query of IQueryable and want to apply sorting to it dynamically, sorting can be on many columns (asc or desc). I've written the following generic function:
private IQueryable<T> ApplySorting<T,U>(IQueryable<T> query, Expression<Func<T, U>> predicate, SortOrder order)
{
if (order == SortOrder.Ascending)
{
{
return query.OrderBy<T, U>(predicate);
}
}
else
{
{
return query.OrderByDescending<T, U>(predicate);
}
}
}
SortOrder is my simple enum with 2 values: Ascending and Descending
Then I call this function in a loop, for each column that user requested sorting. However I've noticed it fails because it always sorts on the last column used, ignoring the other ones.
Then I found there's a 'ThenBy' method on IOrderedQueryable so the valid usage is:
var q = db.MyType.OrderBy(x=>x.Col1).ThenBy(y=>y.Col2); //etc.
But how can I make it generic? I tried to test if the query is IOrderedQueryable but it seems always to be true even if it's simplest var q = from x in db.MyType select x
I have no clue why it was designed like this. What's wrong with:
var q = db.MyType.OrderBy(x=>x.Col1).OrderBy(y=>y.Col2); //etc.
it's so much intuitive

You just need to check if the query is already ordered :
private IQueryable<T> ApplySorting<T,U>(IQueryable<T> query, Expression<Func<T, U>> predicate, SortOrder order)
{
var ordered = query as IOrderedQueryable<T>;
if (order == SortOrder.Ascending)
{
if (ordered != null)
return ordered.ThenBy(predicate);
return query.OrderBy(predicate);
}
else
{
if (ordered != null)
return ordered.ThenByDescending(predicate);
return query.OrderByDescending(predicate);
}
}

How about just making the first OrderBy static, then always ThenBy?
OrderColumn[] columnsToOrderby = getColumnsToOrderby();
IQueryable<T> data = getData();
if(!columnToOrderBy.Any()) { }
else
{
OrderColumn firstColumn = columnsToOrderBy[0];
IOrderedEnumerable<T> orderedData =
firstColumn.Ascending
? data.OrderBy(predicate)
: data.OrderByDescending(predicate);
for (int i = 1; i < columnsToOrderBy.Length; i++)
{
OrderColumn column = columnsToOrderBy[i];
orderedData =
column.Ascending
? orderedData.ThenBy(predicate)
: orderedData.ThenByDescending(predicate);
}
}

Total guess, but can you do something like this?
query.OrderBy(x => 1).ThenBy<T,U>(predicate)
Any syntax errors aside, the idea is to do an OrderBy() that doesn't affect anything, then do the real work in the .ThenBy() method call

I would write a wrapper around and internally use linq extension methods.
var resultList = presentList
.MyOrderBy(x => x.Something)
.MyOrderBY(y => y.SomethingElse)
.MyOrderByDesc(z => z.AnotherThing)
public IQueryable<T> MyOrderBy(IQueryable<T> prevList, Expression<Func<T, U>> predicate) {
return (prevList is IOrderedQueryable<T>)
? query.ThenBy(predicate)
: query.OrderBy(predicate);
}
public IQueryable<T> MyOrderByDesc(IQueryable<T> prevList, Expression<Func<T, U>> predicate) {
return (prevList is IOrderedQueryable<T>)
? query.ThenByDescending(predicate)
: query.OrderByDescending(predicate);
}
PS: I didn't test the code

Extension to dynamic multi-order:
public static class DynamicExtentions
{
public static IEnumerable<T> DynamicOrder<T>(this IEnumerable<T> data, string[] orderings) where T : class
{
var orderedData = data.OrderBy(x => x.GetPropertyDynamic(orderings.First()));
foreach (var nextOrder in orderings.Skip(1))
{
orderedData = orderedData.ThenBy(x => x.GetPropertyDynamic(nextOrder));
}
return orderedData;
}
public static object GetPropertyDynamic<Tobj>(this Tobj self, string propertyName) where Tobj : class
{
var param = Expression.Parameter(typeof(Tobj), "value");
var getter = Expression.Property(param, propertyName);
var boxer = Expression.TypeAs(getter, typeof(object));
var getPropValue = Expression.Lambda<Func<Tobj, object>>(boxer, param).Compile();
return getPropValue(self);
}
}
Example:
var q =(myItemsToSort.Order(["Col1","Col2"]);
Note: not sure any IQueryable provider can translate this