I use LINQ-SQL as my DAL, I then have a project called DB which acts as my BLL. Various applications then access the BLL to read / write data from the SQL Database.
I have these methods in my BLL for one particular table:
public IEnumerable<SystemSalesTaxList> Get_SystemSalesTaxList()
{
return from s in db.SystemSalesTaxLists
select s;
}
public SystemSalesTaxList Get_SystemSalesTaxList(string strSalesTaxID)
{
return Get_SystemSalesTaxList().Where(s => s.SalesTaxID == strSalesTaxID).FirstOrDefault();
}
public SystemSalesTaxList Get_SystemSalesTaxListByZipCode(string strZipCode)
{
return Get_SystemSalesTaxList().Where(s => s.ZipCode == strZipCode).FirstOrDefault();
}
All pretty straight forward I thought.
Get_SystemSalesTaxListByZipCode is always returning a null value though, even when it has a ZIP Code that exists in that table.
If I write the method like this, it returns the row I want:
public SystemSalesTaxList Get_SystemSalesTaxListByZipCode(string strZipCode)
{
var salesTax = from s in db.SystemSalesTaxLists
where s.ZipCode == strZipCode
select s;
return salesTax.FirstOrDefault();
}
Why does the other method not return the same, as the query should be identical ?
Note that, the overloaded Get_SystemSalesTaxList(string strSalesTaxID) returns a record just fine when I give it a valid SalesTaxID.
Is there a more efficient way to write these "helper" type classes ?
Thanks!
This is probably down to the different ways LINQ handles IEnumerable<T> and IQueryable<T>.
You have declared Get_SystemSalesTaxList as returning IEnumerable<SystemSalesTaxList>. That means that when, in your first code sample, you apply the Where operator to the results of Get_SystemSalesTaxList, it gets resolved to the Enumerable.Where extension method. (Note that what matters is the declared type. Yes, at runtime Get_SystemSalesTaxList is returning an IQueryable<SystemSalesTaxList>, but its declared type -- what the compiler sees -- is IEnumerable<SystemSalesTaxList>.) Enumerable.Where runs the specified .NET predicate over the target sequence. In this case, it iterates over all the SystemSalesTaxList objects returned by Get_SystemSalesTaxList, yielding the ones where the ZipCode property equals the specified zip code string (using the .NET String == operator).
But in your last code sample, you apply the Where operator to db.SystemSalesTaxList, which is declared as being of type IQueryable<SystemSalesTaxList>. So the Where operator in that sample gets resolved to Queryable.Where, which translates the specified predicate expression to SQL and runs it on the database.
So what's different in the zip code methods is that the first one runs the C# s.ZipCode == strZipCode test in .NET, and the second translates that into a SQL query WHERE ZipCode = 'CA 12345' (parameterised SQL really but you get the idea). Why do these give different results? Hard to be sure, but the C# == predicate is case-sensitive, and depending on your collation settings the SQL may or may not be case-sensitive. So my suspicion is that strZipCode doesn't match the database zip codes in case, but in the second version SQL Server collation is smoothing this over.
The best solution is probably to change the declaration of Get_SystemSalesTaxList to return IQueryable<SystemSalesTaxList>. The major benefit of this is that it means queries built on Get_SystemSalesTaxList will be executed database side. At the moment, your methods are pulling back EVERYTHING in the database table and filtering it client side. Changing the declaration will get your queries translated to SQL and they will run much more efficiently, and hopefully will solve your zip code issue into the bargain.
The real issue here is the use of IEnumerable<T>, which breaks "composition" of queries; this has two effects:
you are reading all (or at least, more than you need) of your table each time, even if you ask for a single row
you are running LINQ-to-Objects rules, so case-sensitivity applies
Instead, you want to be using IQueryable<T> inside your data layer, allowing you to combine multiple queries with additional Where, OrderBy, Skip, Take, etc as needed and have it build the TSQL to match (and use your db's case-sensitivity rules).
Is there a more efficient way to write these "helper" type classes ?
For more efficient (less code to debug, doesn't stream the entire table, better use of the identity-map to short-circuit additional lookups (via FirstOrDefault etc)):
public IEnumerable<SystemSalesTaxList> Get_SystemSalesTaxList()
{
return db.SystemSalesTaxLists;
}
public SystemSalesTaxList Get_SystemSalesTaxList(string salesTaxID)
{
return db.SystemSalesTaxLists.FirstOrDefault(s => s.SalesTaxID==salesTaxID);
}
public SystemSalesTaxList Get_SystemSalesTaxListByZipCode(string zipCode)
{
return db.SystemSalesTaxLists.FirstOrDefault(s => s.ZipCode == zipCode);
}
Related
I have a list of anonymous objects, that each anonymous object has the same members, and I create this list by something like this :
var listWorthies = balanceWorthies.Select(w => new
{
OwnerName = w.OwnerOnInquery,
OwnerDocDate = w.OwnerDocDate,
}).ToList();
Now I just wanted to convert the OwnerDocDate member of each dynamic object.
something like this :
var listWorthies = balanceWorthies.Select(w => new
{
OwnerName = w.OwnerOnInquery,
OwnerDocDate = ConvertDate(w.OwnerDocDate),
}).ToList();
gives me the error
LINQ to Entities does not recognize the method 'System.String ConvertDate(System.DateTime)' method, and this method cannot be translated into a store expression.
after creating the list, I tried somethings like these ones too :
foreach (dynamic Worthy in listWorthies)
{
DateTime OwnerDocDate_2 = ConvertDate(Worthy.OwnerDocDate);
Worthy.AddProperty(OwnerDocDate_2);
}
but it gives me this error :
'<>f__AnonymousType8<string,string,System.DateTime,System.DateTime,System.Guid,string>' does not contain a definition for 'AddProperty'
UPDATE
the output of function ConvertDate is String, and I want to change the type of OwnerDocDate from datetime to string.
how can I solve this?
You forgot to tell us that balanceWorthies is an IQueryable<...>, not an IEnumerable<...>.
To understand why this matters, you must understand the difference between an IQueryable<...>, and an IEnumerable<...>.
IEnumerable
An object of a class that implements IEnumerable<...>, represents a sequence. You can get the first element, and once you've got an element, you can get the next element, as long as there are elements.
At its lowest level this is done using methode GetEnumerator() / MoveNext() / Current:
IEnumerable<Cusotmer> customers = ...
IEnumerator<Customer> enumerator = customers.GetEnumerator();
while (enumerator.MoveNext())
{
// There is still a Customer in the sequence:
Customer customer = enumerator.Current;
ProcessCustomer(customer);
}
foreach will deep inside do this.
If you look at the LINQ methods, you will see that there are two groups of methods: the ones that return IEnumerable<...> and the others. The first group won't enumerate the sequence. We say that they use lazy-execution or deferred-execution. In the description of these LINQ methods, you'll find this term in the remarks section.
Concatenating methods of this group isn't expensive: the query is not executed, only the Enumerator is adjusted.
The LINQ methods that return something else than IEnumerble<...> will execute the sequence. Deep inside GetEnumerator / MoveNext / Current is called to access the elements of the source sequence one by one.
IQueryable
An object of a class that implements IQueryable<...> doesn't represent an enumerable sequence, it represents the potential to fetch an enumerable sequence.
To do this, the class holds an Expression and a Provider. The Expression holds what data must be fetched in some generic format. The Provider knows where the data must be fetched (usually a Database Management System) and what language is used to communicate with this DBMS (usually SQL).
When you ask the IQueryable to get the enumerator, the Expression is sent to the Provider, who will translate the Expression into SQL and fetch the data at the DBMS. The fetched data is presented as an IEnumerator<...>. The caller can use MoveNext() / Current to access the fetched elements one by one.
Back to your question
The Provider doesn't know your own methods. Hence it doesn't know how to translate them into SQL. In fact, there are several standard LINQ methods that are not supported by entity framework. See Supported and unsupported LINQ methods
Your compiler doesn't know how smart your Provider is, so he can't complain. You'll get the error at runtime.
The easiest way to solve your problem is by transferring the data to your local process and let your local process execute the methods as if it was an IEnumerable<...>.
This is done using the method Enumerable.AsEnumerable. As transferring data is expensive, it is wise to limit the data being transferred to a minimum before you call AsEnumerable. So first do all your Where, (Group-)Join, etc. everything that limits the amount of transferred data.
var listWorthies = balanceWorthies.Select(w => new
{
OwnerName = w.OwnerOnInquery,
OwnerDocDate = w.OwnerDocDate,
})
// OwnerDocDate has the type of balanceWorthy.OwnerDocDate
// move the data to local process, so you can ConvertDate
.AsEnumerable()
// now you can call your own methods:
.Select(fetchedItem => new
{
OwnerName = fetchedItem.OwnderName,
OwnerDocDate = ConvertDate(fetchedItem.OwnerDocDate),
});
Database management systems are extremely optimized in selecting data. It seems that ConvertDate will only translate the data into a different format. So you won't lose a lot of efficiencies.
If in other cases the method will change the amount of selected data, try to change the expression such that the DBMS can handle it, especially if it is before a Where. If the DBMS must execute your code, and you can't translate the LINQ into something that your provider supports, you'll have to write an extension method that changes the Expression. How to do that is something for a different question.
I am using the System.Data.Entity namespace, so I can pass lambda expressions to the Linq Include method.
public ICollection<MyEntity> FindAll()
{
using (var ctx = new MyEntityContext())
{
return ctx.MyEntity.Include(x => x.SomeLazyLoadedValue).ToList();
}
}
When I'm using a Where statement in a different method, I can pass a parameter to it like so:
public ICollection<MyEntity> FindAllBy(Func<MyEntity, bool> criteria)
{
using (var ctx = new MyEntityContext())
{
return ctx.MyEntity.Where(criteria).ToList();
}
}
However, trying the same thing in an Include does not work:
public ICollection<MyEntity> FindAll(Func<MyEntity, bool> criteria)
{
using (var ctx = new MyEntityContext())
{
return ctx.MyEntity.Include(criteria).ToList();
}
}
If you try this, Visual Studio will complain that it
Cannot convert from 'System.Func<MyEntity, bool>' to 'string'
How do I pass a lambda to the Include method?
There are a few problems with your code. For instance, your FindAllBy does not do a sql WHERE query, instead it loads all the entries in your database, and then filter in-memory based on your criteria. To understand why this is like so take a look at the following:
int a = 5;
long b = 5;
Now, it's quite obvious what's happening here, but it's still quite important. The compiler reads the following code and produces two variables. One integer and one long integer, both with values set to the number 5. However, the values of these two numbers are different, even though they are set (in the source code) to the same thing. One is 32-bit, and the other is 64-bit.
Now, let's take a look at the following code:
Func<int, string> a = num => num.ToString();
Expr<Func<int, string>> b = num => num.ToString();
Here the same thing (more or less) is happening. In the first case, the C# compiler sees you want a predicate (a Func<int, string> predicate), whereas the second value is a Expr<Func<int, string>> even though the values are written the same. However, as opposed to the first example, the end result here is vastly different.
A predicate is compiled as a method on a compiler-generated class. It's compiled just as any other code, and simply allows you to remove a bunch of boilerplate. A expression on the other hand is a in-memory representation of the actual code written. In this case, for instance, the expression might look something akin to Call(int.ToString, $1). This can be read by other code and translated to for instance SQL which is then used to query your database.
Now, back to your problem. EntityFramework hands you IQueryable<T> instances, which in turn inherit IEnumerable<T>. Whenever you enumerate over the enumerable, it queries the database.
All the extension-methods that accept delegates are defined on IEnumerable and thus query your database before running the predicate. This is why you need to make sure to select the right method-overloads.
Edit (to answer comment)]
To clarify a bit more I'm going to make a few examples. Say for instance that we have a User class that cointains FirstName, LastName and Age, and the db collection is simply called db.
Expr<Func<User, bool>> olderThan10 = u => u.Age > 10;
Func<User, bool> youngerThan90 = u => u.Age < 90;
var users = db.Where(olderThan10).Where(youngerThan90);
This would result in SQL that finds all users that are older than 10, after which it would in-memory filter away everyone that was older than or equal to 90.
So passing a Func doesn't necessarily mean it queries the whole database. It just means it stops building on the query at that point, and executes it.
As for the next question, Expression<Func<T,bool>> is not a universal answer. It means "a expression that takes a T and returns a bool". In some cases, like .Include which started this whole question, you don't want to return a bool. You want to return whatever you want to include. So for instance, if we go back to our example of users, and amend a Father property on the user class which references another user, and we want to include it, in regular code we'd do
db.Include(u => u.Father);
Now. Here, u is a User, and the return value u.Father is also a user, so in this case u => u.Father is Expr<Func<User, User>> or Expr<Func<User, object>> (I don't know if entity-framework .Include accepts generic values or simply objects).
So your FindAll function should probably look like this:
public ICollection<TData> FindAll<TInclude>(Expr<Func<TData, TInclude>> include) {
using (var ctx = new TContext()) {
return ctx.T.Include(include).ToList();
}
}
Though, to be honest, this is pretty weird looking code, and it's likely that you're doing something else weird with your models given that you've (for instance) named them T and TContext. My guess is that you need to read up a bit on how generics works in C#.
I am having trouble mixing c# functions with conditions in Linq-To-SQL
suppose i have a database table "things" and a local c# Function : bool isGood(thing, params)
I want to use that function to select rows from the table.
var bad = dataContext.Things.Where(t=>t.type=mytype && !isGood(t,myparams))
dataContect.Things.deleteAllOnSubmit(bad);
or
if (dataContext.Things.Any(t=>t.type=mytype && isGood(t,myparams)))
{
return false;
}
Of course this does not work, Linq has no way of translating my function into a SQL statement. So this will produce:
NotSupportedException: Method 'Boolean isGood(thing,params)' has no supported translation to SQL.
What is the best way to redesign this so that it will work?
I can split the statements and convert to list like this:
List<Things> mythings dataContext.Things.Where(t=>t.type=mytype).toList()
if (mythings.Any(t=>isGood(t,myparams)))
{
return false;
}
this works, but seems inefficient, since the whole list has to be generated in every case.
And I don't think I can do a deleteAllOnSubmit with the result
I could do a foreach over mythings instead of calling toList(), that also works. Seems inelegant though.
What other options do I have, and what would be the recommended approach here?
edit:
calling asEnumerable() seems to be another option, and seems better than toList() at least. I.e.
dataContext.Things.Where(t=>t.type=mytype).asEnumerable().Any(t=>isGood(t,myparams))
Pulling the whole list back from the database to run a local c# function on it might seem inefficient, but that what you'd have to do if your isGood() function is local.
If you can translate your isGood() function into Linq, you could apply it before the toList() call, so it would get translated into SQL and the whole list wouldn't be retrieved.
I have a question related to this previous question of mine. In an existing bit of LINQ which involves a number of joins, I'm trying to take each separate method comprising the join and convert it to a CompiledQuery.
First, the normal LINQ method:
private IQueryable<Widget> GetWidgetQuery()
{
return db.Widgets.Where(u => (!u.SomeField.HasValue || !u.SomeField.Value));
}
And here, a delegate (field) definition for a CompiledQuery along these lines:
private static readonly Func<DBDataContext, IQueryable<Widget>> GetWidgetQuery =
CompiledQuery.Compile((DBDataContext db) =>
db.Widgets.Where(u => (!u.SomeField.HasValue || !u.SomeField.Value)));
If I hover over the normal LINQ statement for the method GetWidgetQuery(), I see that it's a method as below:
(method) IQueryable<Widget> GetWidgetQuery()
However, the compiled query delegate (field) differs as follows:
(field) Func<DBDataContext, IQueryable<Widget>> GetWidgetQuery
Upon executing the latter as part of the LINQ statement, the syntax differs as follows. First, the normal LINQ's participation in the join:
var myquery =
from wxr in GetWidgetXRQuery()
join w in GetWidgetQuery() on wxr.WidgetID equals w.ID
select new DTO.WidgetList
{
...
}
And here, the invocation of the CompiledQuery in the form of the delegate:
var myquery =
from wxr in GetWidgetXRQuery()
join w in GetWidgetQuery.Invoke(myContext) on wxr.WidgetID equals w.ID
select new DTO.WidgetList
{
...
}
The former returns the expected result set; the latter, when I attempt myquery.ToList(), yields a stackoverflow exception, in part related to this limitation of .NET 3.5, I think.
Can someone please help me understand how the compiled statement existing as a field (or I guess I should say a delegate) rather than a method is killing my query? In short I know what I'm doing is wrong, but I'm not sure I understand what I misunderstand.
I tried doing roughly the same thing you're doing on EF 4, and everything seems to work fine. So it's either an EF 3.5 issue, or it has something to do with your implementation of GetWidgetXRQuery, or some combination of the two.
But the real point I'd like to make is that, as Roy Goode stated in an answer to your previous question, you lose all the advantages of a precompiled query once you extend that query in any way. By trying to perform a Join on your query, you are converting it to just a plain old query. So you might as well just use the non-compiled version which appears to work for you.
Update
Realized you were talking about LINQ to SQL. This sort of query does appear to have support in Entity Framework, but not LINQ to SQL. In .NET 4, I'm getting the following error:
An IQueryable that returns a self-referencing Constant expression is not supported.
That doesn't mean much to me, but I'm guessing that it has something to do with the way the compiled query is represented internally. I still get the same error if I evaluate the query into a variable and use that variable in the query later, so it clearly has nothing to do with the difference between a delegate and a function. I still maintain that a compiled query is not appropriate to use here. Either you need to create one big compiled query to represent the whole query you want to perform, or you need to use regular queries if you want to piece them together this way.
I just came across this same error while doing db integration testing, and to jump straight to the point without trying to explain my specific issue. Linq to Sql will create the sql query internally when using IQueryable and the moment you execute a method on that IQueryable, i.e. ToList() it executes that query on the database. So in my case I am joining to a method that returns IQueryable but is mocked to return a result, it is trying to compile that to a sql query but the IQueryable I created does not have an internal SQL query
What happens behind the curtains when I include a function into my compiled query, like I do with DataConvert.ToThema() here to convert a table object into my custom business object:
public static class Queries
{
public static Func<MyDataContext, string, Thema> GetThemaByTitle
{
get
{
var func = CompiledQuery.Compile(
(MyDataContext db, string title) =>
(from th in elan.tbl_Thema
where th.Titel == title
select DataConvert.ToThema(th)).Single()
);
return func;
}
}
}
public static class DataConvert
{
public static Thema ToThema(tbl_Thema tblThema)
{
Thema thema = new Thema();
thema.ID = tblThema.ThemaID;
thema.Titel = tblThema.Titel;
// and some other stuff
return thema;
}
}
and call it like this
Thema th = Queries.GetThemaByTitle.Invoke(db, "someTitle");
Apparently the function is not translated in to SQL or something (how could it), but it also does not hold when I set a breakpoint there in VS2010.
It works without problems, but I don't understand how or why. What exactly happens there?
Your DataConvert.ToThema() static method is simply creating an instance of a type which has a default constructor, and setting various properties, is that correct? If so, it's not terribly different from:
(from th in elan.tbl_Thema
where th.Titel == title
select new Thema{ID=th.ThemaID, Titel=th.Titel, etc...}
).Single());
When you call Queries.GetThemaByTitle, a query is being compiled. (The way you are calling this, by the way, may or may not actually be giving you any benefits from pre-compiling). That 'Query' is actually a code expression tree, only part of which is intended to generate SQL code that is sent to the database.
Other parts of it will generate IL code which is grabbing what is returned from the database and putting it into some form for your consumption. LINQ (EF or L2S) is smart enough to be able to take your static method call and generate the IL from it to do what you want - and maybe it's doing so with an internal delegate or some such. But ultimately, it doesn't need to be (much) different from what would be generated from I substituted above.
But note that this happens regardless what the type is that you get back; somewhere, IL code is being generated that puts DB values into a CLR object. That is the other part of those expression trees.
If you want a more detailed look at those expression trees and what they involved, I'd have to dig for ya, but I'm not sure from your question if that's what you are looking for.
Let me start by pointing out, that whether you compile your query or not does not matter. You would observe the very same results even if you did not pre-compile.
Technically, as Andrew has pointed out, making this work is not that complicated. When your LINQ expression is evaluated an expression tree is constructed internally. Your function appears as a node in this expression tree. No magic here. You'll be able to write this expression both in L2S and L2E and it will compile and run fine. That is until you try to actually execute the actual SQL query against the database. This is where difference begins. L2S seems to happily execute this task, whereas L2E fails with NotSupportedException, and reporting that it does not know how to convert ToThema into store query.
So what's happening inside? In L2S, as Andrew has explained, the query compiler understands that your function can be run separately from the store query has been executed. So it emits calls to your function into the object reading pipeline (where data read from SQL is transformed to the objects that are returned as the result of your call).
Once thing Andrew was not quite right, is that it matters what's inside your static method. I don't think it does.
If you put a break point in the debugger to your function, you will see that it's called once per returned row. In the stack trace you will see "Lightweight Function", which, in reality, means that the method was emitted at run time. So this is how it works for Linq to Sql.
Linq to Entity team seemed to go different route. I do not know, what was the reasoning, why they decided to ban all InvocationExpressions from L2E queries. Perhaps these were performance reason, or may be the fact that they need to support all kind of providers, not SQL Server only, so that data readers might behave differently. Or they simply thought that most people wouldn't realize that some of those are executed per returned row and preferred to keep this option closed.
Just my thoughts. If anyone has any more insight, please chime in!