Using Linq to SQL I'm writing queries that are taking advantage of the IQueryable.GroupBy method.
Even though my query involves many tables and left joins lets say for illustration that we are only working with two tables. TableA has a one to many relationship to TableB.
var queryResults = from db.TableA
.Join(db.TableB, tA => tA.ID, tB => tB.TableA_ID, (ta, tb) => tb)
.GroupBy(tb => tb.TableA);
This will give me an
<IQueryable<IGrouping<TableA, TableB>>
On the surface this seems to work however I'm worried because I'm calling the GroupBy method and passing in a reference type for the keySelector argument.
Please help me understand why this is or isn't a safe thing to do.
Hard to tell what you're trying to do, but I think you might want to use comprehension syntax here to make it easier to write/understand. In particular, do your grouping after you've done whatever joins or other operations you want to do instead of mixing them together.
var query = ... (can be chained methods, whatever)
var grouped = from row in query
group row by row.SomeProperty;
Then you'll likely have an easier time (I would think) writing and reasoning about the query.
Try this:
var queryResults =
from ta in db.TableA
join tb in db.TableB on ta.ID equals tb.TableA_ID
group tb by ta;
And yes, this is safe to do on reference types. The query generated by linq-to-sql has nothing to do with reference types - it's just SQL.
Related
I am still in the midst of converting our web app from vb to C# and this complex query, that I just finished converting and have tested and it works properly, needs refactoring.
This section could probably perform a little better if lambdas were used (I've read in numerous places that lambda expressions perform faster)
foreach (var datapersymbol in (from symId in symblist
select (
from q in ctx.HistoricalPriceData.Where(q => q.SymbolId == symId)
join x in ctx.Symbols on q.SymbolId equals x.SymbolId
orderby q.Date descending
select new {q.Date, q.LastPrice, x.Symbol1})).ToList())
As you can see I've added lambdas where I thought appropriate but Im certain they can be in other parts of this expressions as well. I'm still getting up to speed in C# so Im not an expert here.
Assuming you have a navigation property on HistoricalPriceData to Symbols called "Symbol":
foreach(var datapersymbol in ctx
.HistoricalPriceData
.Where(h=>symblist.Contains(h.SymbolId))
.OrderByDescending(h=>h.Date)
.Select(h=>new {h.Date,h.LastPrice,h.Symbol.Symbol1}))
As Rahul pointed out, there is no performance difference between lamda/method syntax and query syntax, however, this will perform slightly better because it drops the .ToList(), which will allow the loop to begin processing as soon as it gets the first record from the database rather than waiting for the entire result (and it doesn't need to create a List). I've also simplified it somewhat by using .Contains instead of a subquery as well, and switched to using navigation properties rather than a explicit join which makes it easier to read/more maintainable.
For the clarity of the code I will rather refactor it like below;
var datapersymbols = from symId in symblist
select (
from q in ctx.HistoricalPriceData.Where(q => q.SymbolId == symId)
join x in ctx.Symbols on q.SymbolId equals x.SymbolId
orderby q.Date descending
select new {q.Date, q.LastPrice, x.Symbol1});
foreach (var datapersymbol in datapersymbols)
{
}
As far as I can see, in this case, there is not much that you can gain from the C# side. You have to check the query plan for the SQL query and then add necessary indexes. The SymbolId field should be an indexed field.
I am surprised I cannot find a solution for this on the web, but wording the search terms was a bit difficult. The question I have is about generating entity SQL that only returns the needed columns in a group join using Lambda syntax.
The following is a "toy" example. I am not joining on two entities, rather on an enumerated list and an entity. And tunnelling is not an acceptable answer. I need to apply this to a much larger problem using a group join and select many.
var result1 = clientprofiles.Join(Context.Adjusters,
c => c.AdjusterId,
a => a.AdjusterId,
(c, a) => new {a.ClientAccountId}).ToList();
Using Julie Lehrman's Entity profiler, I see that the query is being generated to select every record in the rows that meet the join criteria. How do I pare it down so it only selects the ClientAccountId field in this example?
You can project a set of columns on any select from the context, so in your case you can constrain the Context.Adjusters parameter by using
Context.Adjusters.Select(a=> new { a.ClientAccountId })
to constrain the query to just the single column
I am currently working on a project leveraging EF and I am wondering if there is a more efficient or cleaner way to handle what I have below.
In SQL Server I could get the data I want by doing something like this:
SELECT tbl2.* FROM
dbo.Table1 tbl
INNER JOIN dbo.Table2 tbl2 ON tbl.Column = tbls2.Colunm
WHERE tbl.Column2 IS NULL
UNION
SELECT * FROM
dbo.Table2
WHERE Column2 = value
Very straight forward. However in LINQ I have something that looks like this:
var results1 = Repository.Select<Table>()
.Include(t => t.Table2)
.Where(t => t.Column == null);
var table2Results = results1.Select(t => t.Table2);
var results2 = Repository.Select<Table2>().Where(t => t.Column2 == "VALUE");
table2Results = table2Results.Concat(results2);
return results2.ToList();
First and foremost the return type of the method that contains this code is of type IEnumerable< Table2 > so first I get back all of the Table2 associations where a column in Table1 is null. I then have to select out my Table2 records so that I have a variable that is of type IEnumerable. The rest of the code is fairly straightforward in what it does.
This seems awfully chatty to me and, I think, there is a better way to do what I am trying to achieve. The produced SQL isn't terrible (I've omitted the column list for readability)
SELECT
[UnionAll1].*
FROM (SELECT
[Extent2].*
FROM [dbo].[Table1] AS [Extent1]
INNER JOIN [dbo].[Table2] AS [Extent2] ON [Extent1].[Column] = [Extent2].[Column]
WHERE [Extent1].[Column2] IS NULL
UNION ALL
SELECT
[Extent3].*
FROM [dbo].[Table2] AS [Extent3]
WHERE VALUE = [Extent3].[Column]) AS [UnionAll1]
So is there a cleaner / more efficient way to do what I have described? Thanks!
Well, one problem is that your results may not return the same data as your original SQL query. Union will select distinct values, Union All will select all values. First, I think your code could be made a lot clearer like so:
// Notice the lack of "Include". "Include" only states what should be returned
// *with* the original type, and is not necessary if you only need to select the
// individual property.
var firstResults = Repository.Select<Table>()
.Where(t => t.Column == null)
.Select(t => t.Table2);
var secondResults = Repository.Select<Table2>()
.Where(t => t.Column2 == "Value");
return firstResults.Union(secondResults);
If you know that it's impossible to have duplicates in this query, use Concat instead on the last line (which will produce the UNION ALL that you see in your current code) for reasons described in more detail here. If you want something similar to the original query, continue to use Union like in the example above.
It's important to remember that LINQ-to-Entities is not always going to be able to produce the SQL that you desire, since it has to handle so many cases in a generic fashion. The benefit of using EF is that it makes your code a lot more expressive, clearer, strongly typed, etc. so you should favor readability first. Then, if you actually see a performance problem when profiling, then you might want to consider alternate ways to query for the data. If you profile the two queries first, then you might not even care about the answer to this question.
I guess it should be really simple, but i cannot find how to do it.
I have a linq query, that selects one column, of type int, and i need it sorted.
var values = (from p in context.Products
where p.LockedSince == null
select Convert.ToInt32(p.SearchColumn3)).Distinct();
values = values.OrderBy(x => x);
SearchColumn3 is op type string, but i only contains integers. So i thought, converting to Int32 and ordering would definitely give me a nice 1,2,3 sorted list of values. But instead, the list stays ordered like it were strings.
199 20 201
Update:
I've done some tests with C# code and LinqPad.
LinqPad generates the following SQL:
SELECT [t2].[value]
FROM (
SELECT DISTINCT [t1].[value]
FROM (
SELECT CONVERT(Int,[t0].[SearchColumn3]) AS [value], [t0].[LockedSince], [t0].[SearchColumn3]
FROM [Product] AS [t0]
) AS [t1]
WHERE ([t1].[LockedSince] IS NULL)
) AS [t2]
ORDER BY [t2].[value]
And my SQL profiler says that my C# code generates this piece of SQL:
SELECT DISTINCT a.[SearchColumn3] AS COL1
FROM [Product] a
WHERE a.[LockedSince] IS NULL
ORDER BY a.[SearchColumn3]
So it look like C# Linq code just omits the Convert.ToInt32.
Can anyone say something useful about this?
[Disclaimer - I work at Telerik]
You can solve this problem with Telerik OpenAccess ORM too. Here is what i would suggest in this case.
var values = (from p in context.Products
where p.LockedSince == null
orderby "cast({0} as integer)".SQL<int>(p.SearchColumn3)
select "cast({0} as integer)".SQL<int>(p.SearchColumn3)).ToList().Distinct();
OpenAccess provides the SQL extension method, which gives you the ability to add some specific sql code to the generated sql statement.
We have started working on improving this behavior.
Thank you for pointing this out.
Regards
Ralph
Same answer as one my other questions, it turns out that the Linq provider i'm using, the one that comes with Telerik OpenAccess ORM does things different than the standard Linq to SQL provider! See the SQL i've posted in my opening post! I totally wasn't expecting something like this, but i seem that the Telerik OpenAccess thing still needs a lot of improvement. So be careful before you start using it. It looks nice, but it has some serious shortcomings.
I can't replicate this problem. But just make sure you're enumerating the collection when you inspect it. How are you checking the result?
values = values.OrderBy(x => x);
foreach (var v in values)
{
Console.WriteLine(v.ToString());
}
Remember, this won't change the order of the records in the database or anywhere else - only the order that you can retrieve them from the values enumeration.
Because your values variable is a result of a Linq expression, so that it doest not really have values until you calling a method such as ToList, ToArray, etc.
Get back to your example, the variable x in OrderBy method, will be treated as p.SearchColumn3 and therefore, it's a string.
To avoid that, you need to let p.SearchColumn3 become integer before OrderBy method.
You should add a let statement in to your code as below:
var values = (from p in context.Products
where p.LockedSince == null
let val = Convert.ToInt32(p.SearchColumn3)
select val).Distinct();
values = values.OrderBy(x => x);
In addition, you can combine order by statement with the first, it will be fine.
This works in LINQ-to-SQL:
var customersTest = from c in db.Customers
select new
{
Id = c.Id,
Addresses = from a in db.Addresses where c.Id.ToString() ==
a.ReferenzId select a
};
foreach (var item in customersTest)
{
Console.WriteLine(item.Id);
}
But a similar example in Entity Framework gets an error message that says basically that it can't "translate it to SQL", here is the original error message in German:
"'LINQ to Entities' erkennt die
Methode 'System.String ToString()'
nicht, und diese Methode kann nicht in
einen Speicherausdruck übersetzt
werden."
Translation:
"'LINQ to Entities' does not recognize
Method 'System.String ToString()',
this method can not be translated into
a memory expression.
Can anyone shed any light on how we could get this kind of statement to work in Entity Framework or explain why it gets this error?
Simply put: LINQ to Entities doesn't know about the conversion from your ID type to a string.
What is the type of c.ID? Is there any reason why it's one type for ID, but another for ReferenzId? If at all possible, make them the same type, at which point you won't have a problem any more. I don't know if there are other ways of performing conversions in LINQ to Entities - there may be - but aligning the types would be cleaner.
By the way, this really looks like it's a join:
var query = from c in db.Customers
join a in db.Addresses on c.Id equals a.ReferenzId into addresses
select new { Id = c.Id, Addresses = addresses };
EDIT: To respond to your comment - ToString appears in IntelliSense because the compiler has no real idea what your query is going to mean or how it will be translated. It's perfectly valid C#, and can generate a valid expression tree - it's just that EF doesn't know how to convert that expression tree into SQL.
You could try using Convert.ToString(c.Id) instead of just calling c.Id.ToString()...
Updated answer:
If you followed the link I gave you at the beginning of my answer, this missing feature has meanwhile received 75 votes and is now (finally!) implemented by Microsoft in EF 6.1. To all who participated: Thank you for voting! Your voice was heard.
For example:
var query = from e in context.Employees where e.EmployeeID.ToString() == "1" select e;
will now be translated to:
DECLARE #p0 NVarChar(1000) = '1'
SELECT [t0].[EmployeeID], [t0].[LastName], [t0].[FirstName], [t0].[Title],
[t0].[TitleOfCourtesy], [t0].[BirthDate], [t0].[HireDate], [t0].[Address],[t0].[City],
[t0].[Region], [t0].[PostalCode], [t0].[Country], [t0].[HomePhone], [t0].[Extension],
[t0].[Photo], [t0].[Notes], [t0].[ReportsTo], [t0].[PhotoPath]
FROM [Employees] AS [t0]
WHERE (CONVERT(NVarChar,[t0].[EmployeeID])) = #p0
i.e. e.EmployeeID.ToString() translates to (CONVERT(NVarChar,[t0].[EmployeeID])).
Original answer:
It makes no sense to me why Linq2EF does not translate .ToString() into a proper SQL statement, as Linq2SQL does, only the Microsoft dev team knows the reason why they did not implement it yet. :-(
But you can raise the priority to implement it if you vote for this feature by following this link.
Luckily there are also 2 workarounds available, both of them I used recently in EF queries:
I) What helped me to get around this limitation was to change the query into a list, like so:
var customersList = (from c in db.Customers
select c).ToList(); // converts to IEnumerable<T> ...
var customersTest = (from c in customersList
select new {Id=c.ID.ToString()}); // ... which allows to use .ToString()
The statement .ToList() converts to IEnumerable<T>, where .ToString() is available. Note that, Depending on the requirements, you can use .AsEnumerable() as well, which has the advantage that deferred execution is supported which is better if you have multiple linq queries depending on each other or if you're using different parameter values (many thanks to Divega for this hint!).
Afterwards you can use this query as you wish , e.g.:
var customersTest2 = from c in customersTest
select new
{
Id = c.Id,
Addresses = from a in db.Addresses where c.Id == a.ReferenzId select a
};
Of course if you need you can add more properties to the objects of customersTest as required. You can also optimize the query above, I have only used 3 steps for readability of this example.
II) For simple conversions, and if you have to reuse the generated query in further subqueries (and it needs to remain IQueryable), use SqlFunctions from System.Data.Objects.SqlClient, they will be translated into SQL queries correctly.
Example 1: Date conversion (you have to use dateparts as below shown)
var customersTest = from c in db.Customers
select new {
strDate=SqlFunctions.DateName("dd", c.EndDate)
+"."+SqlFunctions.DateName("mm", c.EndDate)
+"."+SqlFunctions.DateName("yyyy", c.EndDate)
}
Example 2: Numeric to string conversion
var customersTest = from c in db.Customers
select new {
strID=SqlFunctions.StringConvert((double)c.ID)
}
This should help you out of most situations where conversions into strings are required.
Entity Framework 6.1 RTM which was just released now support .ToString()
LINQ to Entities as far as I understand it (for v1) is very primative. In otherwords it doesn't know how to take the extension method "ToString()" and generate the SQL for it.
In LINQ to SQL, it executes the extension method "ToString()" before generating the SQL. The difference is that LINQ to Entities uses IQueryable instead of IEnumerable.
BUT, from what I remember casting should work (because casting is a data type and SQL knows about CAST()).
So
c.Id.ToString() should really be (string)c.Id
(also, make sure it is (string) and not (String)).
One of the downfalls I would say about using Lambda (in Entity Framework) to generate the SQL expression instead of pure LINQ.
Keep in mind too, that using CAST on the left side of the equals sign in SQL is a bit ill performing :-)