In a previous question I asked how to make "Computed properties" in a linq to sql object. The answer supplied there was sufficient for that specific case but now I've hit a similar snag in another case.
I have a database with Items that have to pass through a number of Steps. I want to have a function in my database that retrieves the Current step of the item that I can then build on. For example:
var x = db.Items.Where(item => item.Steps.CurrentStep().Completed == null);
The code to get the current step is:
Steps.OrderByDescending(step => step.Created).First();
So I tried to add an extension method to the EntitySet<Step> that returned a single Step like so:
public static OrderFlowItemStep CurrentStep(this EntitySet<OrderFlowItemStep> steps)
{
return steps.OrderByDescending(o => o.Created).First();
}
But when I try to execute the query at the top I get an error saying that the CurrentStep() function has no translation to SQL. Is there a way to add this functionality to Linq-to-SQL in any way or do I have to manually write the query every time? I tried to write the entire query out first but it's very long and if I ever change the way to get the active step of an item I have to go over all the code again.
I'm guessing that the CurrentStep() method has to return a Linq expression of some kind but I'm stuck as to how to implement it.
The problem is that CurrentStep is a normal method. Hence, the Expression contains a call to that method, and naturally SQL cannot execute arbitrary .NET methods.
You will need to represent the code as an Expression. I have one in depth example here: http://www.atrevido.net/blog/2007/09/06/Complicated+Functions+In+LINQ+To+SQL.aspx
Unfortunately, the C# 3.0 compiler has a huge omission and you cannot generate calls to Expressions. (i.e., you can't write "x => MyExpression(x)"). Working around it either requires you to write the Expression manually, or to use a delegate as a placeholder. Jomo Fisher has an interesting post about manipulating Expression trees in general.
Without actually having done it, the way I'd probably approach it is by making the CurrentStep function take the predicate you want to add ("Completed == null"). Then you can create a full Expression> predicate to hand off to Where. I'm lazy, so I'm going to do an example using String and Char (String contains Chars, just like Item contains Steps):
using System;
using System.Linq;
using System.Linq.Expressions;
class Program {
static void Main(string[] args) {
Console.WriteLine(StringPredicate(c => Char.IsDigit(c)));
var func = StringPredicate(c => Char.IsDigit(c)).Compile();
Console.WriteLine(func("h2ello"));
Console.WriteLine(func("2ello"));
}
public static Expression<Func<string,bool>> StringPredicate(Expression<Func<char,bool>> pred) {
Expression<Func<string, char>> get = s => s.First();
var p = Expression.Parameter(typeof(string), "s");
return Expression.Lambda<Func<string, bool>>(
Expression.Invoke(pred, Expression.Invoke(get, p)),
p);
}
}
So "func" is created by using StringPredicate to create an Expression. For the example, we compile it to execute it locally. In your case, you'd pass the whole predicate to "Where" so it gets translated to SQL.
The "get" expression is where you put your "extension" stuff (OrderByWhatever, First, etc.). This is then passed in to the predicate that's given to you.
Don't worry if it looks complicated; it sorta is at first. If you haven't done this kinda stuff before, it'll take a bit of time (the first time I did this kinda stuff, it took hours to get it right :|.. now it comes slightly easier). Also, as I mentioned, you can write a helper method to do this re-writing for you (so you don't directly need to use the Expression.Whatever methods), but I haven't seen any examples and haven't really needed it yet.
Check out my answer to "switch statement in linq" and see if that points you in the right direction...
The technique i demonstrate there is the one that got me past the scary "no translation to SQL" error.
Related
I have some function (X) which return IQueryable with "OrderBy(x => x.Name)".
I want to use X function in new code, but in this code there is parameter which determine the order (asc/desc).
I prefer not to change (X) since it used in multiple places already.
there is option to get X(), then cancel it's "OrderBy", and apply new order?
(now it throws exception, since it is like Y.OrderBy(a => a.Name).OrderByDescending(a => a.Name))
"A column has been specified more than once in the order by list. Columns in the order by list must be unique.\r\nStatement(s) could not be prepared."
use
public void x(string order = string.empty)
{
Y.OrderBy(a => a.Name);
if (order == "desc")
{
Y = Y.Reverse();
}
}
I've tried to wiggle Expression object of IOrderedQueryable that your method is actually returning (am I wrong?) but so far no luck, and I can't see the option to manipulate System.Linq.Expression or System.Linq.EnumerableQuery easy enough to use it instead of changing X() method.
IMHO, you should choose from:
create somes kind of wrapper method using the Reverse() (#Eric Lizotte answer) or OrderByDescending() - it will be less time consuming I guess but for grouping, filtering etc. you will have to create brand new function.
remove OrderBy() from X() function body, and modify your old code - returning non-ordered result will give you more flexibility and opportunity to do what you want to do with the result via linq expressions in each scenario that you'll came across.
Can you clarify for me if you're talking about "function" you have in mind a sql function that is mapped via ORM (eg. Entity Framework) in your application or just a ordinary method which is using your custom provider to return IQueryable?
I am using the System.Data.Entity namespace, so I can pass lambda expressions to the Linq Include method.
public ICollection<MyEntity> FindAll()
{
using (var ctx = new MyEntityContext())
{
return ctx.MyEntity.Include(x => x.SomeLazyLoadedValue).ToList();
}
}
When I'm using a Where statement in a different method, I can pass a parameter to it like so:
public ICollection<MyEntity> FindAllBy(Func<MyEntity, bool> criteria)
{
using (var ctx = new MyEntityContext())
{
return ctx.MyEntity.Where(criteria).ToList();
}
}
However, trying the same thing in an Include does not work:
public ICollection<MyEntity> FindAll(Func<MyEntity, bool> criteria)
{
using (var ctx = new MyEntityContext())
{
return ctx.MyEntity.Include(criteria).ToList();
}
}
If you try this, Visual Studio will complain that it
Cannot convert from 'System.Func<MyEntity, bool>' to 'string'
How do I pass a lambda to the Include method?
There are a few problems with your code. For instance, your FindAllBy does not do a sql WHERE query, instead it loads all the entries in your database, and then filter in-memory based on your criteria. To understand why this is like so take a look at the following:
int a = 5;
long b = 5;
Now, it's quite obvious what's happening here, but it's still quite important. The compiler reads the following code and produces two variables. One integer and one long integer, both with values set to the number 5. However, the values of these two numbers are different, even though they are set (in the source code) to the same thing. One is 32-bit, and the other is 64-bit.
Now, let's take a look at the following code:
Func<int, string> a = num => num.ToString();
Expr<Func<int, string>> b = num => num.ToString();
Here the same thing (more or less) is happening. In the first case, the C# compiler sees you want a predicate (a Func<int, string> predicate), whereas the second value is a Expr<Func<int, string>> even though the values are written the same. However, as opposed to the first example, the end result here is vastly different.
A predicate is compiled as a method on a compiler-generated class. It's compiled just as any other code, and simply allows you to remove a bunch of boilerplate. A expression on the other hand is a in-memory representation of the actual code written. In this case, for instance, the expression might look something akin to Call(int.ToString, $1). This can be read by other code and translated to for instance SQL which is then used to query your database.
Now, back to your problem. EntityFramework hands you IQueryable<T> instances, which in turn inherit IEnumerable<T>. Whenever you enumerate over the enumerable, it queries the database.
All the extension-methods that accept delegates are defined on IEnumerable and thus query your database before running the predicate. This is why you need to make sure to select the right method-overloads.
Edit (to answer comment)]
To clarify a bit more I'm going to make a few examples. Say for instance that we have a User class that cointains FirstName, LastName and Age, and the db collection is simply called db.
Expr<Func<User, bool>> olderThan10 = u => u.Age > 10;
Func<User, bool> youngerThan90 = u => u.Age < 90;
var users = db.Where(olderThan10).Where(youngerThan90);
This would result in SQL that finds all users that are older than 10, after which it would in-memory filter away everyone that was older than or equal to 90.
So passing a Func doesn't necessarily mean it queries the whole database. It just means it stops building on the query at that point, and executes it.
As for the next question, Expression<Func<T,bool>> is not a universal answer. It means "a expression that takes a T and returns a bool". In some cases, like .Include which started this whole question, you don't want to return a bool. You want to return whatever you want to include. So for instance, if we go back to our example of users, and amend a Father property on the user class which references another user, and we want to include it, in regular code we'd do
db.Include(u => u.Father);
Now. Here, u is a User, and the return value u.Father is also a user, so in this case u => u.Father is Expr<Func<User, User>> or Expr<Func<User, object>> (I don't know if entity-framework .Include accepts generic values or simply objects).
So your FindAll function should probably look like this:
public ICollection<TData> FindAll<TInclude>(Expr<Func<TData, TInclude>> include) {
using (var ctx = new TContext()) {
return ctx.T.Include(include).ToList();
}
}
Though, to be honest, this is pretty weird looking code, and it's likely that you're doing something else weird with your models given that you've (for instance) named them T and TContext. My guess is that you need to read up a bit on how generics works in C#.
What happens behind the curtains when I include a function into my compiled query, like I do with DataConvert.ToThema() here to convert a table object into my custom business object:
public static class Queries
{
public static Func<MyDataContext, string, Thema> GetThemaByTitle
{
get
{
var func = CompiledQuery.Compile(
(MyDataContext db, string title) =>
(from th in elan.tbl_Thema
where th.Titel == title
select DataConvert.ToThema(th)).Single()
);
return func;
}
}
}
public static class DataConvert
{
public static Thema ToThema(tbl_Thema tblThema)
{
Thema thema = new Thema();
thema.ID = tblThema.ThemaID;
thema.Titel = tblThema.Titel;
// and some other stuff
return thema;
}
}
and call it like this
Thema th = Queries.GetThemaByTitle.Invoke(db, "someTitle");
Apparently the function is not translated in to SQL or something (how could it), but it also does not hold when I set a breakpoint there in VS2010.
It works without problems, but I don't understand how or why. What exactly happens there?
Your DataConvert.ToThema() static method is simply creating an instance of a type which has a default constructor, and setting various properties, is that correct? If so, it's not terribly different from:
(from th in elan.tbl_Thema
where th.Titel == title
select new Thema{ID=th.ThemaID, Titel=th.Titel, etc...}
).Single());
When you call Queries.GetThemaByTitle, a query is being compiled. (The way you are calling this, by the way, may or may not actually be giving you any benefits from pre-compiling). That 'Query' is actually a code expression tree, only part of which is intended to generate SQL code that is sent to the database.
Other parts of it will generate IL code which is grabbing what is returned from the database and putting it into some form for your consumption. LINQ (EF or L2S) is smart enough to be able to take your static method call and generate the IL from it to do what you want - and maybe it's doing so with an internal delegate or some such. But ultimately, it doesn't need to be (much) different from what would be generated from I substituted above.
But note that this happens regardless what the type is that you get back; somewhere, IL code is being generated that puts DB values into a CLR object. That is the other part of those expression trees.
If you want a more detailed look at those expression trees and what they involved, I'd have to dig for ya, but I'm not sure from your question if that's what you are looking for.
Let me start by pointing out, that whether you compile your query or not does not matter. You would observe the very same results even if you did not pre-compile.
Technically, as Andrew has pointed out, making this work is not that complicated. When your LINQ expression is evaluated an expression tree is constructed internally. Your function appears as a node in this expression tree. No magic here. You'll be able to write this expression both in L2S and L2E and it will compile and run fine. That is until you try to actually execute the actual SQL query against the database. This is where difference begins. L2S seems to happily execute this task, whereas L2E fails with NotSupportedException, and reporting that it does not know how to convert ToThema into store query.
So what's happening inside? In L2S, as Andrew has explained, the query compiler understands that your function can be run separately from the store query has been executed. So it emits calls to your function into the object reading pipeline (where data read from SQL is transformed to the objects that are returned as the result of your call).
Once thing Andrew was not quite right, is that it matters what's inside your static method. I don't think it does.
If you put a break point in the debugger to your function, you will see that it's called once per returned row. In the stack trace you will see "Lightweight Function", which, in reality, means that the method was emitted at run time. So this is how it works for Linq to Sql.
Linq to Entity team seemed to go different route. I do not know, what was the reasoning, why they decided to ban all InvocationExpressions from L2E queries. Perhaps these were performance reason, or may be the fact that they need to support all kind of providers, not SQL Server only, so that data readers might behave differently. Or they simply thought that most people wouldn't realize that some of those are executed per returned row and preferred to keep this option closed.
Just my thoughts. If anyone has any more insight, please chime in!
I have a method that alters an "Account" object based on the action delegate passed into it:
public static void AlterAccount(string AccountID, Action<Account> AccountAction) {
Account someAccount = accountRepository.GetAccount(AccountID);
AccountAction.Invoke(someAccount);
someAccount.Save();
}
This works as intended...
AlterAccount("Account1234", a => a.Enabled = false);
...but now what I'd like to try and do is have a method like this:
public static void AlterAccount(string AccountID, string AccountActionText) {
Account someAccount = accountRepository.GetAccount(AccountID);
Action<Account> AccountAction = MagicLibrary.ConvertMagically<Action<Account>>(AccountActionText);
AccountAction.Invoke(someAccount);
someAccount.Save();
}
It can then be used like:
AlterAccount("Account1234", "a => a.Enabled = false");
to disable account "Account1234".
I've had a look at the linq dynamic query library, which seems to do more or less what I want but for Func type delegates, and my knowledge of Expression trees etc isn't quite good enough to work out how to achieve what I want.
Is there an easy way to do what I want, or do I need to learn expressions properly and write a load of code?
(The reason I want to do this is to allow an easy way of bulk updating account objects from a powershell script where the user can specify a lambda expression to perform the changes.)
The Dynamic LINQ library is a fine choice, as it'll generate expressions you can compile to code in a lightweight fashion.
The example you provided actually produces a boolean -- so you should be able to ask for a Func and it might sort it out.
Edit: This of course is wrong, as Expressions don't have assignment in them at all.
So, another potential way is to take two lambdas. One to find the property you want, one to provide a value:
(a => a.AccountId), (a => true)
Then use reflection to set the property referenced in the first lambda with the result of the second one. Hackish, but it's still probably lightweight compared to invoking the C# compiler.
This way you don't have to do much codegen yourself - the expressions you get will contain most everything you need.
You may try this: Dynamic Lambda Expressions Using An Isolated AppDomain
It compiles a lambda expression using CodeDOM compiler. In order to dispose the in-memory assembly that gets created, the compiler runs on an isolated AppDomain. For the passing the expression through the domain boundary, it has to be serialized. Alas, Expression<> is not Serializable. So, a trick has to be used. All the details are explained in the post.
I'm the author of that component, by the way. I would like very much to hear your feedback from it.
There is no general way to parse a string into a lambda expression without a full compilation, because lambda expressions can reference things that are defined outside the lambda expression. I know of no library that handles the specific case you want. There's a long discussion of this on a thread on a C# discussion group.
The easiest way to get what you want is to compile a method at runtime. You can write a function that takes in the string "a.Enabled = true; return a;" and sticks that in the middle of a function that takes an Account as a parameter. I would use this library as a starting point, but you can also use the function mentioned on another thread.
That's easy:
Use CodeDom to generate the module containing the "surrounding class" you'll use to build the expression; this class must implement the interface known to your application
Use CodeSnippedExpression to inject the expression into its member.
Use Activator type to create the instance of this class in runtime.
Basically, you need to build the following class with CodeDom:
using System;
using MyNamespace1;
using ...
using MyNamespace[N];
namespace MyNamespace.GeneratedTypes
{
public class ExpressionContainer[M] : IHasAccountAction
{
public Action<Account> AccountAction {
get {
return [CodeSnippedExpression must be used here];
}
}
}
}
Assuming that IHasAccountAction is:
public IHasAccountAction {
public Action<Account> AccountAction { get; }
}
If this is done, you can get the expression compiled from string with ease. If you need its expression tree representation, use Expression<Action<Account>> instead of Action<Account> in generated type.
This is probably a stupid question, but here goes. I would like to be able to dynamically construct a predicate < T > from a string parsed from a database VARCHAR column, or any string, for that matter. For example, say the column in the database contained the following string:
return e.SomeStringProperty.Contains("foo");
These code/string values would be stored in the database knowing what the possible properties of the generic "e" is, and knowing that they had to return a boolean. Then, in a magical, wonderful, fantasy world, the code could execute without knowing what the predicate was, like:
string predicateCode = GetCodeFromDatabase();
var allItems = new List<SomeObject>{....};
var filteredItems = allItems.FindAll(delegate(SomeObject e) { predicateCode });
or Lambda-ized:
var filteredItems = allItems.FindAll(e => [predicateCode]);
I know it can probably never be this simple, but is there a way, maybe using Reflection.Emit, to create the delegate code dynamically from text and give it to the FindAll < T > (or any other anonymous/extension) method?
The C# and VB compilers are available from within the .NET Framework:
C# CodeDom Provider
Be aware though, that this way you end up with a separate assembly (which can only be unloaded if it's in a separate AppDomain). This approach is only feasible if you can compile all the predicates you are going to need at once. Otherwise there is too much overhead involved.
System.Reflection.Emit is a great API for dynamically emitting code for the CLR. It is, however, a bit cumbersome to use and you must learn CIL.
LINQ expression trees are an easy to use back-end (compilation to CIL) but you would have to write your own parser.
I suggest you have a look at one of the "dynamic languages" that run on the CLR (or DLR) such as IronPython. It's the most efficient way to implement this feature, if you ask me.
Check out the Dynamic Linq project it does all this and more!
http://weblogs.asp.net/scottgu/archive/2008/01/07/dynamic-linq-part-1-using-the-linq-dynamic-query-library.aspx
Great for simple stuff like user selected orderby's or where clauses
It is possible using emit, but you'd be building your own parser.
EDIT
I remember that in ScottGu's PDC keynote, he showed a feature using the CLI version of the .net framework that resembled Ruby's eval, but I can't find a URL that can corroborate this. I'm making this a commnity wiki so that anyone who has a good link can add it.
I stepped off the dynamic linq because it's limited in ways I want to search a collection, unless you prove me wrong.
My filter needs to be: in a list of orders, filter the list so that I have only the orders with in the collection of items in that order, an item with the name "coca cola".
So that will result to a method of: orders.Findall(o => o.Items.Exists(i => i.Name == "coca cola"))
In dynamic linq I didn't find any way to do that, so I started with CodeDomProvicer.
I created a new Type with a method which contains my dynamically built FindAll Method:
public static IList Filter(list, searchString)
{
// this will by dynamically built code
return orders.Findall(o => o.Items.Exists(i => i.Name == "coca cola"));
}
when I try to build this assembly:
CompilerResults results = provider.CompileAssemblyFromSource(parameters, sb.ToString());
I'm getting the error:
Invalid expression term ">"
Why isn't the compiler able to compile the predicate?