I am using the System.Data.Entity namespace, so I can pass lambda expressions to the Linq Include method.
public ICollection<MyEntity> FindAll()
{
using (var ctx = new MyEntityContext())
{
return ctx.MyEntity.Include(x => x.SomeLazyLoadedValue).ToList();
}
}
When I'm using a Where statement in a different method, I can pass a parameter to it like so:
public ICollection<MyEntity> FindAllBy(Func<MyEntity, bool> criteria)
{
using (var ctx = new MyEntityContext())
{
return ctx.MyEntity.Where(criteria).ToList();
}
}
However, trying the same thing in an Include does not work:
public ICollection<MyEntity> FindAll(Func<MyEntity, bool> criteria)
{
using (var ctx = new MyEntityContext())
{
return ctx.MyEntity.Include(criteria).ToList();
}
}
If you try this, Visual Studio will complain that it
Cannot convert from 'System.Func<MyEntity, bool>' to 'string'
How do I pass a lambda to the Include method?
There are a few problems with your code. For instance, your FindAllBy does not do a sql WHERE query, instead it loads all the entries in your database, and then filter in-memory based on your criteria. To understand why this is like so take a look at the following:
int a = 5;
long b = 5;
Now, it's quite obvious what's happening here, but it's still quite important. The compiler reads the following code and produces two variables. One integer and one long integer, both with values set to the number 5. However, the values of these two numbers are different, even though they are set (in the source code) to the same thing. One is 32-bit, and the other is 64-bit.
Now, let's take a look at the following code:
Func<int, string> a = num => num.ToString();
Expr<Func<int, string>> b = num => num.ToString();
Here the same thing (more or less) is happening. In the first case, the C# compiler sees you want a predicate (a Func<int, string> predicate), whereas the second value is a Expr<Func<int, string>> even though the values are written the same. However, as opposed to the first example, the end result here is vastly different.
A predicate is compiled as a method on a compiler-generated class. It's compiled just as any other code, and simply allows you to remove a bunch of boilerplate. A expression on the other hand is a in-memory representation of the actual code written. In this case, for instance, the expression might look something akin to Call(int.ToString, $1). This can be read by other code and translated to for instance SQL which is then used to query your database.
Now, back to your problem. EntityFramework hands you IQueryable<T> instances, which in turn inherit IEnumerable<T>. Whenever you enumerate over the enumerable, it queries the database.
All the extension-methods that accept delegates are defined on IEnumerable and thus query your database before running the predicate. This is why you need to make sure to select the right method-overloads.
Edit (to answer comment)]
To clarify a bit more I'm going to make a few examples. Say for instance that we have a User class that cointains FirstName, LastName and Age, and the db collection is simply called db.
Expr<Func<User, bool>> olderThan10 = u => u.Age > 10;
Func<User, bool> youngerThan90 = u => u.Age < 90;
var users = db.Where(olderThan10).Where(youngerThan90);
This would result in SQL that finds all users that are older than 10, after which it would in-memory filter away everyone that was older than or equal to 90.
So passing a Func doesn't necessarily mean it queries the whole database. It just means it stops building on the query at that point, and executes it.
As for the next question, Expression<Func<T,bool>> is not a universal answer. It means "a expression that takes a T and returns a bool". In some cases, like .Include which started this whole question, you don't want to return a bool. You want to return whatever you want to include. So for instance, if we go back to our example of users, and amend a Father property on the user class which references another user, and we want to include it, in regular code we'd do
db.Include(u => u.Father);
Now. Here, u is a User, and the return value u.Father is also a user, so in this case u => u.Father is Expr<Func<User, User>> or Expr<Func<User, object>> (I don't know if entity-framework .Include accepts generic values or simply objects).
So your FindAll function should probably look like this:
public ICollection<TData> FindAll<TInclude>(Expr<Func<TData, TInclude>> include) {
using (var ctx = new TContext()) {
return ctx.T.Include(include).ToList();
}
}
Though, to be honest, this is pretty weird looking code, and it's likely that you're doing something else weird with your models given that you've (for instance) named them T and TContext. My guess is that you need to read up a bit on how generics works in C#.
Related
I have some function (X) which return IQueryable with "OrderBy(x => x.Name)".
I want to use X function in new code, but in this code there is parameter which determine the order (asc/desc).
I prefer not to change (X) since it used in multiple places already.
there is option to get X(), then cancel it's "OrderBy", and apply new order?
(now it throws exception, since it is like Y.OrderBy(a => a.Name).OrderByDescending(a => a.Name))
"A column has been specified more than once in the order by list. Columns in the order by list must be unique.\r\nStatement(s) could not be prepared."
use
public void x(string order = string.empty)
{
Y.OrderBy(a => a.Name);
if (order == "desc")
{
Y = Y.Reverse();
}
}
I've tried to wiggle Expression object of IOrderedQueryable that your method is actually returning (am I wrong?) but so far no luck, and I can't see the option to manipulate System.Linq.Expression or System.Linq.EnumerableQuery easy enough to use it instead of changing X() method.
IMHO, you should choose from:
create somes kind of wrapper method using the Reverse() (#Eric Lizotte answer) or OrderByDescending() - it will be less time consuming I guess but for grouping, filtering etc. you will have to create brand new function.
remove OrderBy() from X() function body, and modify your old code - returning non-ordered result will give you more flexibility and opportunity to do what you want to do with the result via linq expressions in each scenario that you'll came across.
Can you clarify for me if you're talking about "function" you have in mind a sql function that is mapped via ORM (eg. Entity Framework) in your application or just a ordinary method which is using your custom provider to return IQueryable?
I understand the main difference between IQueryable and IEnumerable. IQueryable executes the query in the database and returns the filtered results while IEnumerable brings the full result set into memory and executes the filtering query there.
I don't think using an extension method to query on the initial assignment of a variable will cause it to be executed in the database like IQueryable, but I just wanted to make sure.
This code will cause the full result set of People to be returned from the database, and then the filtering is done in memory:
(The People property on the context is of type DbSet)
IEnumerable<Person> people = context.People;
Person person = people.Where(x => x.FirstName == "John");
Even though I am adding the filtering below as an extension method before assigning the item to my variable, I'm assuming this code should work the same way as the code above, and bring back the full result set into memory before filtering it, right?
Person person = context.People.Where(x => x.FirstName == "John");
EDIT:
Thanks for the replies guys. I modified the code example to show what I meant (removed the IEnumerable in the second paragraph of code).
Also, to clarify, context.People is of type DbSet, which implements both IQueryable and IEnumerable. So I 'm not actually sure which .Where method is being called. IntelliSense tells me it is the IQueryable version, but can this be trusted? Is this always the case when working directly with a DbSet of a context?
IEnumerable<Person> people = context.People;
Person person = people.Where(x => x.FirstName == "John");
... will execute the IEnumerable<T>.Where method extension, which accepts a Func<TSource, bool> predicate parameter, forcing the filtering to happen in memory.
In contrast...
IEnumerable<Person> people = context.People.Where(x => x.FirstName == "John");
...will execute the IQueryable<T>.Where method extension, which accepts a Expression<Func<TSource, bool>> predicate parameter. Notice that this is an expression, not a delegate, which allows it to translate the where condition to a database condition.
So it really does make a difference which extension method you invoke.
IQueryable<T> works on expressions. It is effectively a query-builder, accumulating information about a query without doing anything... until the moment you need a value from it, when:
a query is generated from the accumulated Expression, in the target language (e.g. SQL)
the query is executed, usually on the database,
results are converted back to a C# object.
IEnumerable<T> extensions works on pre-compiled functions. When you need a value from it:
C# code in those functions is executed.
It is easy to confuse the two, because:
both have extension functions with similar names,
the lambda syntax is the same for Expressions and Functions - so you cannot tell them apart,
the use of "var" to declare variables removes the datatype (often the only clue as to which interface is being used).
IQueryable<int> a;
IEnumerable<int> b;
int x1 = a.FirstOrDefault(i => i > 10); // Expression passed in
int x2 = b.FirstOrDefault(i => i > 10); // Function passed in
Extension methods with the same name usually do the same thing (because they were written that way) but sometimes they don't.
So the answer is: No, they are not equivalent.
The problem description
In my application, I have a search function that builds a complex search query based on user input on top of EntityFramework DbSet object and runs it against the database, something like this:
public static IQueryable<MyEntity> ApplySearchQuery(SearchSpec search, IQueryable<MyEntity> query)
{
if (search.Condition1.HasValue)
query = query.Where(e => e.SomeProperty == search.Condition1);
if (search.Condition2.HasValue)
query = query.Where(e => e.OtherProperty == search.Condition2);
....
}
This performs pretty well on the database side. Now I have come to a point that I have a single MyEntity at hand, and I want to see if a particular SearchSpec matches the entity or not. And I need to do this for a potentially large number of SearchSpec objects, so the performance is important.
I'd also really like to avoid duplicating the expressions.
Here's what I've thought so far:
I can call Expression.Compile() on the expressions to convert them to a delegate, and then call them. But since I have a parameter (the search parameter) I need to build expressions and compile them every time, making it a very inefficient way (I suppose, correct me if I'm wrong).
I can wrap my single entity in an IQueriable using new [] { myEntity }.AsQueriable() and then evaluate the query on it. Not sure how well this will perform.
The question:
Which of the above approaches are faster?
Is any of my assumptions (about the limitations) wrong?
Is there any other way that I haven't thought about yet?
Here PredicateBuilder could work miracles for you. With PredicateBuilder you can build an Expression that can be used against an IQueryable, but, when compiled, also against objects.
You could have a method that builds and returns the Expression
using LinqKit;
Expression<Func<MyEntity, bool>> CreateExpression(SearchSpec search)
{
var predicate = PredicateBuilder.True<MyEntity>();
if (search.Condition1.HasValue)
predicate = predicate.And(e => e.SomeProperty == search.Condition1);
if (search.Condition2.HasValue)
predicate = predicate.And(e => e.OtherProperty == search.Condition2);
...
return predicate;
}
To be used as:
var predicate = CreateExpression(search);
var result = query.Where(predicate.Expand()); // will be translated into SQL.
var match = predicate.Compile()(myEntity);
Notice the Expand call. Without it, EF will fail because under the hood Invoke will be called, which can't be translated into SQL. Expand replaces these calls so that EF can convert the expression to SQL.
I use LINQ-SQL as my DAL, I then have a project called DB which acts as my BLL. Various applications then access the BLL to read / write data from the SQL Database.
I have these methods in my BLL for one particular table:
public IEnumerable<SystemSalesTaxList> Get_SystemSalesTaxList()
{
return from s in db.SystemSalesTaxLists
select s;
}
public SystemSalesTaxList Get_SystemSalesTaxList(string strSalesTaxID)
{
return Get_SystemSalesTaxList().Where(s => s.SalesTaxID == strSalesTaxID).FirstOrDefault();
}
public SystemSalesTaxList Get_SystemSalesTaxListByZipCode(string strZipCode)
{
return Get_SystemSalesTaxList().Where(s => s.ZipCode == strZipCode).FirstOrDefault();
}
All pretty straight forward I thought.
Get_SystemSalesTaxListByZipCode is always returning a null value though, even when it has a ZIP Code that exists in that table.
If I write the method like this, it returns the row I want:
public SystemSalesTaxList Get_SystemSalesTaxListByZipCode(string strZipCode)
{
var salesTax = from s in db.SystemSalesTaxLists
where s.ZipCode == strZipCode
select s;
return salesTax.FirstOrDefault();
}
Why does the other method not return the same, as the query should be identical ?
Note that, the overloaded Get_SystemSalesTaxList(string strSalesTaxID) returns a record just fine when I give it a valid SalesTaxID.
Is there a more efficient way to write these "helper" type classes ?
Thanks!
This is probably down to the different ways LINQ handles IEnumerable<T> and IQueryable<T>.
You have declared Get_SystemSalesTaxList as returning IEnumerable<SystemSalesTaxList>. That means that when, in your first code sample, you apply the Where operator to the results of Get_SystemSalesTaxList, it gets resolved to the Enumerable.Where extension method. (Note that what matters is the declared type. Yes, at runtime Get_SystemSalesTaxList is returning an IQueryable<SystemSalesTaxList>, but its declared type -- what the compiler sees -- is IEnumerable<SystemSalesTaxList>.) Enumerable.Where runs the specified .NET predicate over the target sequence. In this case, it iterates over all the SystemSalesTaxList objects returned by Get_SystemSalesTaxList, yielding the ones where the ZipCode property equals the specified zip code string (using the .NET String == operator).
But in your last code sample, you apply the Where operator to db.SystemSalesTaxList, which is declared as being of type IQueryable<SystemSalesTaxList>. So the Where operator in that sample gets resolved to Queryable.Where, which translates the specified predicate expression to SQL and runs it on the database.
So what's different in the zip code methods is that the first one runs the C# s.ZipCode == strZipCode test in .NET, and the second translates that into a SQL query WHERE ZipCode = 'CA 12345' (parameterised SQL really but you get the idea). Why do these give different results? Hard to be sure, but the C# == predicate is case-sensitive, and depending on your collation settings the SQL may or may not be case-sensitive. So my suspicion is that strZipCode doesn't match the database zip codes in case, but in the second version SQL Server collation is smoothing this over.
The best solution is probably to change the declaration of Get_SystemSalesTaxList to return IQueryable<SystemSalesTaxList>. The major benefit of this is that it means queries built on Get_SystemSalesTaxList will be executed database side. At the moment, your methods are pulling back EVERYTHING in the database table and filtering it client side. Changing the declaration will get your queries translated to SQL and they will run much more efficiently, and hopefully will solve your zip code issue into the bargain.
The real issue here is the use of IEnumerable<T>, which breaks "composition" of queries; this has two effects:
you are reading all (or at least, more than you need) of your table each time, even if you ask for a single row
you are running LINQ-to-Objects rules, so case-sensitivity applies
Instead, you want to be using IQueryable<T> inside your data layer, allowing you to combine multiple queries with additional Where, OrderBy, Skip, Take, etc as needed and have it build the TSQL to match (and use your db's case-sensitivity rules).
Is there a more efficient way to write these "helper" type classes ?
For more efficient (less code to debug, doesn't stream the entire table, better use of the identity-map to short-circuit additional lookups (via FirstOrDefault etc)):
public IEnumerable<SystemSalesTaxList> Get_SystemSalesTaxList()
{
return db.SystemSalesTaxLists;
}
public SystemSalesTaxList Get_SystemSalesTaxList(string salesTaxID)
{
return db.SystemSalesTaxLists.FirstOrDefault(s => s.SalesTaxID==salesTaxID);
}
public SystemSalesTaxList Get_SystemSalesTaxListByZipCode(string zipCode)
{
return db.SystemSalesTaxLists.FirstOrDefault(s => s.ZipCode == zipCode);
}
In a previous question I asked how to make "Computed properties" in a linq to sql object. The answer supplied there was sufficient for that specific case but now I've hit a similar snag in another case.
I have a database with Items that have to pass through a number of Steps. I want to have a function in my database that retrieves the Current step of the item that I can then build on. For example:
var x = db.Items.Where(item => item.Steps.CurrentStep().Completed == null);
The code to get the current step is:
Steps.OrderByDescending(step => step.Created).First();
So I tried to add an extension method to the EntitySet<Step> that returned a single Step like so:
public static OrderFlowItemStep CurrentStep(this EntitySet<OrderFlowItemStep> steps)
{
return steps.OrderByDescending(o => o.Created).First();
}
But when I try to execute the query at the top I get an error saying that the CurrentStep() function has no translation to SQL. Is there a way to add this functionality to Linq-to-SQL in any way or do I have to manually write the query every time? I tried to write the entire query out first but it's very long and if I ever change the way to get the active step of an item I have to go over all the code again.
I'm guessing that the CurrentStep() method has to return a Linq expression of some kind but I'm stuck as to how to implement it.
The problem is that CurrentStep is a normal method. Hence, the Expression contains a call to that method, and naturally SQL cannot execute arbitrary .NET methods.
You will need to represent the code as an Expression. I have one in depth example here: http://www.atrevido.net/blog/2007/09/06/Complicated+Functions+In+LINQ+To+SQL.aspx
Unfortunately, the C# 3.0 compiler has a huge omission and you cannot generate calls to Expressions. (i.e., you can't write "x => MyExpression(x)"). Working around it either requires you to write the Expression manually, or to use a delegate as a placeholder. Jomo Fisher has an interesting post about manipulating Expression trees in general.
Without actually having done it, the way I'd probably approach it is by making the CurrentStep function take the predicate you want to add ("Completed == null"). Then you can create a full Expression> predicate to hand off to Where. I'm lazy, so I'm going to do an example using String and Char (String contains Chars, just like Item contains Steps):
using System;
using System.Linq;
using System.Linq.Expressions;
class Program {
static void Main(string[] args) {
Console.WriteLine(StringPredicate(c => Char.IsDigit(c)));
var func = StringPredicate(c => Char.IsDigit(c)).Compile();
Console.WriteLine(func("h2ello"));
Console.WriteLine(func("2ello"));
}
public static Expression<Func<string,bool>> StringPredicate(Expression<Func<char,bool>> pred) {
Expression<Func<string, char>> get = s => s.First();
var p = Expression.Parameter(typeof(string), "s");
return Expression.Lambda<Func<string, bool>>(
Expression.Invoke(pred, Expression.Invoke(get, p)),
p);
}
}
So "func" is created by using StringPredicate to create an Expression. For the example, we compile it to execute it locally. In your case, you'd pass the whole predicate to "Where" so it gets translated to SQL.
The "get" expression is where you put your "extension" stuff (OrderByWhatever, First, etc.). This is then passed in to the predicate that's given to you.
Don't worry if it looks complicated; it sorta is at first. If you haven't done this kinda stuff before, it'll take a bit of time (the first time I did this kinda stuff, it took hours to get it right :|.. now it comes slightly easier). Also, as I mentioned, you can write a helper method to do this re-writing for you (so you don't directly need to use the Expression.Whatever methods), but I haven't seen any examples and haven't really needed it yet.
Check out my answer to "switch statement in linq" and see if that points you in the right direction...
The technique i demonstrate there is the one that got me past the scary "no translation to SQL" error.