We Keep Coding

Lambda expression - For a Select New

I am trying to create a custom collection from an IQueryable object, where i am trying to perform a select statement but getting an error cannot convert to store expression. I am new to Lambda Expression. Kindly help me how to fix this problem.
Getting error at line c.Event.FirstUpper()
public static string FirstCharToUpper(string input)
{
if (String.IsNullOrEmpty(input))
return string.Empty;
var trimmed = input.Trim();
return trimmed.First().ToString().ToUpper() + trimmed.Substring(1);
}
public static Expression<Func<string, string>> GetFirstCaseToUpperExpression()
{
var expression = NJection.LambdaConverter.Fluent.Lambda.TransformMethodTo<Func<string, string>>()
.From(() => StringFormatter.FirstCharToUpper)
.ToLambda();
return expression;
}
Calling the Expression
return new List<LoggerModel>(
logDB.PELoggers
.Where(c => (c.SubscriberCode == SubscriberCode)).OrderByField(sortBy, ascendingOrder).Select(c => new LoggerModel()
{
DateTime = c.DateTime.Value,
Event = c.Event.FirstUpper()
})

I suppose you are using Entity Framework or a smiliar O/R mapper.
Think about what you are doing here: you are writing a LINQ query that should be executed against your database. To do this, it will translate your LINQ query into a SQL query which will then be executed against your database.
But FirstCharToUpper() is a custom method in your code. Your database does not know anything about it, so your O/R mapper's LINQ provider cannot translate it into anything meaningful in SQL, hence you get the error.
So what you need to do is to first "finish" the query against your database to have the results in-memory and after that, apply any further processing that can only be done within the boundaries of your code on that in-memory collection.
You can do this simply by inserting .AsEnumerable() in your LINQ query before you do the select with your custom expression:
logDB.PELoggers
.Where(c => (c.SubscriberCode == SubscriberCode))
.OrderByField(sortBy, ascendingOrder)
.AsEnumerable()
.Select(c => new LoggerModel()
{
DateTime = c.DateTime.Value,
Event = c.Event.FirstUpper()
})
When calling AsEnumerable(), the query against your database will be executed and the results are copied into an IEnumerable in memory. The Select() afterwards will now already be executed against the in-memory collection and not against the database anymore, thus it can use your custom FirstCharToUpper() method.
Edit based on your comments below:
Everything above is still valid, but in the comments you said your function needs to return IQueryable. In your case, what your FirstCharToUpper() method is doing is pretty simple and the LINQ-to-Entities provider does support methods like ToUpper and Substring. So I'd recommend to simply get rid of your helper method and instead write your LINQ query to do just that with methods that Entity Framework can translate to valid SQL:
logDB.PELoggers
.Where(c => (c.SubscriberCode == SubscriberCode))
.OrderByField(sortBy, ascendingOrder)
.Select(c => new LoggerModel()
{
DateTime = c.DateTime.Value,
Event = c.Event.Substring(0, 1).ToUpper()
+ c.Event.Substring(1)
})
This will result in a SQL query that will already return the content in Event with an uppercase first letter right from the database.
To also support the IsNullOrEmpty check and the Trim you are doing (both also supported by LINQ-to-Entities) I recommend to change the lambda syntax to the LINQ query syntax so you can use the let statement for the trimming, which makes the code cleaner:
from c in logDB.PELoggers
let trimmedEvent = c.Event.Trim()
where c.SubscriberCode == SubscriberCode
select new LoggerModel()
{
DateTime = c.DateTime.Value,
Event = !string.IsNullOrEmpty(trimmedEvent)
? trimmedEvent.Substring(0, 1).ToUpper()
+ trimmedEvent.Substring(1)
: string.Empty
};
In case you do not want to have this done in the LINQ query, you would need to do the uppercasing at some point later when your query against the DB has been executed, for example right in the View that will show your data. Or one option could be to apply the uppercasing in the Event property setter of your LoggerModel:
public class LoggerModel
{
// ...
private string event;
public string Event
{
get { return event; }
set { event = FirstCharToUpper(value); }
}
// ...
}
But there is no way to make custom functions work inside LINQ-to-Entities queries.

IQueryable.Any wrapper. Method cannot be translated into a store expression

I have this code:
public static bool ContainEx<T>(this IQueryable<T> query, System.Linq.Expressions.Expression<System.Func<T, bool>> expression)
{
return query.Any(expression);
}
If I use that:
return bonusesWhereSearch.WhereEx(x => userBonusesWhereSearch.ContainEx(y => y.Bonus_Id == x.Id));
I get this error message:
System.NotSupportedException: LINQ to Entities does not recognize the
method 'Boolean
ContainEx[Bonus](System.Linq.IQueryable`1[SDataEntities.Bonus],
System.Linq.Expressions.Expression`1[System.Func`2[SDataEntities.Bonus,System.Boolean]])'
method, and this method cannot be translated into a store expression.
and if I use Any:
return bonusesWhereSearch.WhereEx(x => userBonusesWhereSearch.Any(y => y.Bonus_Id == x.Id));
that does work.

Problem here is, that entity framework doesn't execute ContainEx, but tries to translate this method into SQL. And since this is custom method translation fails. If you use Any directly, it is correctly translated to SQL equivalent.

Error in a predicate with expression function

I'm using an expression function for predicate in LINQ Where method (that returns IQueryable result) which takes a class parameter:
public ActionResult FilterProfiles(FilterViewModel filter)
{
var profiles = database.Profiles
.Where(Predicate(filter))
...
}
The predicate is:
private static Expression<Func<UserProfile, bool>> Predicate(FilterViewModel f)
{
return p => (CompareFilter(p, f));
}
The problem is with the CompareFilter function:
private static bool CompareFilter(UserProfile profile, FilterViewModel filter)
{
if (filter.FirstName != null)
{
if (profile.FirstName != null)
{
if (profile.FirstName.CompareTo(filter.FirstName) != 0)
{
return false;
}
}
...
}
The exception is:
LINQ to Entities does not recognize the method 'Boolean CompareFilter(Project.Models.UserProfile, Project.Web.ViewModels.FilterViewModel)' method, and this method cannot be translated into a store expression.
Exception Details: System.NotSupportedException: LINQ to Entities does not recognize the method 'Boolean CompareFilter(Project.Models.UserProfile, Project.Web.ViewModels.FilterViewModel)' method, and this method cannot be translated into a store expression.
Thanks in advance!

The issue is that you're attempting to use a method in the lambda being translated by Entity Framework.
Entity Framework is actually using IQueryables to translate your method calls into SQL as you call them, to make querying a database seamless. This means, however, that any method you define is inherently outside the purview of Entity Framework, and it therefore cannot translate it into SQL, and thus fails. Even though you know the source, the Expression functions cannot access a compiled method as an expression tree.
A workarounf for this (and many other similar situations) is to use .AsEnumerable() which will cause further Linq calls to be the IEnumerable versions, which enumerate via foreach and thus pull our objects into memory.

Is it possible to format DB content (as string) before comparison with where clause

I'd like to be able to take a url-formatted string (e.g united-kingdom) and use it in a WHERE clause against a Country column that is not formatted in such a way (e.g. United Kingdom).
Ideally, I'd like to be able to do something like this:
db.Jobs
.Where(j => j.Country.MyStringFormattingExtension() == urlformattedstring);
I understand that this is a no go because EF would try and execute the projection on the SQL side. It gives me: "LINQ to Entities does not recognize the method 'System.String MyStringFormattingExtension(System.String)' method, and this method cannot be translated into a store expression."
It has been suggested that I return the query as an enumerable before applying the where clause, however I think that this would be pretty inefficient - returning all rows from the DB before filtering.

You can define a user defined function and import that into your database. Read this article for more details
// In SQL
CREATE FUNCTION dbo.ToFormattedString ...
// In C#
public static class EntityFunctions
{
[EdmFunction("dbo", "ToFormattedString")]
public static string ToFormattedString(this string input)
{
throw new NotSupportedException("Direct calls not supported");
}
}
var results = db.Jobs.Where(j => j.Country.ToFormattedString() == urlFormattedString);
Alternatively, you can create a view in your database that materializes the string the way you want it to be formatted then join it into your Linq query.
// In SQL
CREATE VIEW dbo.vFormattedJobs AS ...
// In C#
var results =
(from j in db.vFormattedJobs
where j.FormattedCountry == urlFormattedString
select j);

How about going the other way, and converting the URL-formatted string into the database format before using it in the query?
var dbFormattedString = urlformattedstring.ConvertToDbFormat();
var result = db.Jobs.Where(j => j.Country == dbFormattedString);
(db.Jobs is already an IEnumerable, so I suppose that the suggestion was to call ToList() on it - it would have worked, but indeed, it would have been very inefficient unless the table is very small.)

How to maintain LINQ deferred execution?

Suppose I have an IQueryable<T> expression that I'd like to encapsulate the definition of, store it and reuse it or embed it in a larger query later. For example:
IQueryable<Foo> myQuery =
from foo in blah.Foos
where foo.Bar == bar
select foo;
Now I believe that I can just keep that myQuery object around and use it like I described. But some things I'm not sure about:
How best to parameterize it? Initially I've defined this in a method and then returned the IQueryable<T> as the result of the method. This way I can define blah and bar as method arguments and I guess it just creates a new IQueryable<T> each time. Is this the best way to encapsulate the logic of an IQueryable<T>? Are there other ways?
What if my query resolves to a scalar rather than IQueryable? For instance, what if I want this query to be exactly as shown but append .Any() to just let me know if there were any results that matched? If I add the (...).Any() then the result is bool and immediately executed, right? Is there a way to utilize these Queryable operators (Any, SindleOrDefault, etc.) without immediately executing? How does LINQ-to-SQL handle this?
Edit: Part 2 is really more about trying to understand what are the limitation differences between IQueryable<T>.Where(Expression<Func<T, bool>>) vs. IQueryable<T>.Any(Expression<Func<T, bool>>). It seems as though the latter isn't as flexible when creating larger queries where the execution is to be delayed. The Where() can be appended and then other constructs can be later appended and then finally executed. Since the Any() returns a scalar value it sounds like it will immediately execute before the rest of the query can be built.

You have to be really careful about passing around IQueryables when you're using a DataContext, because once the context get's disposed you won't be able to execute on that IQueryable anymore. If you're not using a context then you might be ok, but be aware of that.
.Any() and .FirstOrDefault() are not deferred. When you call them they will cause execution to occur. However, this may not do what you think it does. For instance, in LINQ to SQL if you perform an .Any() on an IQueryable it acts as a IF EXISTS( SQL HERE ) basically.
You can chain IQueryable's along like this if you want to:
var firstQuery = from f in context.Foos
where f.Bar == bar
select f;
var secondQuery = from f in firstQuery
where f.Bar == anotherBar
orderby f.SomeDate
select f;
if (secondQuery.Any()) //immediately executes IF EXISTS( second query in SQL )
{
//causes execution on second query
//and allows you to enumerate through the results
foreach (var foo in secondQuery)
{
//do something
}
//or
//immediately executes second query in SQL with a TOP 1
//or something like that
var foo = secondQuery.FirstOrDefault();
}

A much better option than caching IQueryable objects is to cache Expression trees. All IQueryable objects have a property called Expression (I believe), which represents the current expression tree for that query.
At a later point in time, you can recreate the query by calling queryable.Provider.CreateQuery(expression), or directly on whatever the provider is (in your case a Linq2Sql Data Context).
Parameterizing these expression trees is slightly harder however, as they use ConstantExpressions to build in a value. In order to parameterize these queries, you WILL have to rebuild the query every time you want different parameters.

Any() used this way is deferred.
var q = dc.Customers.Where(c => c.Orders.Any());
Any() used this way is not deferred, but is still translated to SQL (the whole customers table is not loaded into memory).
bool result = dc.Customers.Any();
If you want a deferred Any(), do it this way:
public static class QueryableExtensions
{
public static Func<bool> DeferredAny<T>(this IQueryable<T> source)
{
return () => source.Any();
}
}
Which is called like this:
Func<bool> f = dc.Customers.DeferredAny();
bool result = f();
The downside is that this technique won't allow for sub-querying.

Create a partial application of your query inside an expression
Func[Bar,IQueryable[Blah],IQueryable[Foo]] queryMaker =
(criteria, queryable) => from foo in queryable.Foos
where foo.Bar == criteria
select foo;
and then you can use it by ...
IQueryable[Blah] blah = context.Blah;
Bar someCriteria = new Bar();
IQueryable[Foo] someFoosQuery = queryMaker(blah, someCriteria);
The query could be encapsulated within a class if you want to make it more portable / reusable.
public class FooBarQuery
{
public Bar Criteria { get; set; }
public IQueryable[Foo] GetQuery( IQueryable[Blah] queryable )
{
return from foo in queryable.Foos
where foo.Bar == Criteria
select foo;
}
}