In sql I can do this
select 'a' as MyColumn
so, i have a query in linq with entity framework that get some data from the database, but, i need union that query with one row, and in sql I can do this:
select ... from ...
union
select 'a' as MyColumn
How can i generate this query with linq?
I tried to do this:
var query = (from ... select new {..}).Union(new List<...> { new ...() { MyColumn = 'a' } })
But i gess that Entity Framework DON'T know how to translate that in memory list to sql
I need to get an IQueryable result, not a List or other in memory Collection, because i need to join that result to other sql linq querys in the future.
This isn't possible and you shouldn't do it. Both for the same reason: Entity Framework will try to translate the whole LINQ statement into SQL, including the local list (new List<...>).
The reason why it's not possible is that EF has no way to translate C# objects into SQL constructs.
The reason why you shouldn't do it is that it's incredibly wasteful: you build the list in C# code, EF (if it could) translates it into a SQL statement, the database runs the SQL statement and converts it to a result set, EF receives the result set and converts it into the list you originally offered it.
Just to demonstrate it, I'll show what happens if you do this with a list of primitive values which EF does know how to translate into SQL:
var ints = Enumerable.Range(1,5);
var res = Products.Select(c => c.Id).Union(ints).ToList();
This produces the following SQL statement:
SELECT
[Distinct1].[C1] AS [C1]
FROM ( SELECT DISTINCT
[UnionAll5].[ProductId] AS [C1]
FROM (SELECT
[Extent1].[ProductId] AS [ProductId]
FROM [dbo].[Product] AS [Extent1]
UNION ALL
SELECT
1 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
UNION ALL
SELECT
2 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable2]
UNION ALL
SELECT
3 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable3]
UNION ALL
SELECT
4 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable4]
UNION ALL
SELECT
5 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable5]) AS [UnionAll5]
) AS [Distinct1]
As you see, for each element in the list EF generated a SingleRowTablex entry to build a "temp table" to UNION with the ids from the actual query.
Conclusion: just query what you need from the database and add to the result afterwards. It's easy enough to do that:
(from ... select new {..})
.AsEnumerable() // continue in memory
.Union(...)
Related
In documentation of Entity Framework:
https://www.entityframeworktutorial.net/querying-entity-graph-in-entity-framework.aspx
in section regarding GroupBy we can read that following code:
using (var ctx = new SchoolDBEntities())
{
var students = from s in ctx.Students
group s by s.StandardId into studentsByStandard
select studentsByStandard;
foreach (var groupItem in students)
{
Console.WriteLine(groupItem.Key);
foreach (var stud in groupItem)
{
Console.WriteLine(stud.StudentId);
}
}
}
executes internally following SQL:
SELECT
[Project2].[C1] AS [C1],
[Project2].[StandardId] AS [StandardId],
[Project2].[C2] AS [C2],
[Project2].[StudentID] AS [StudentID],
[Project2].[StudentName] AS [StudentName],
[Project2].[StandardId1] AS [StandardId1]
FROM ( SELECT
[Distinct1].[StandardId] AS [StandardId],
1 AS [C1],
[Extent2].[StudentID] AS [StudentID],
[Extent2].[StudentName] AS [StudentName],
[Extent2].[StandardId] AS [StandardId1],
CASE WHEN ([Extent2].[StudentID] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C2]
FROM (SELECT DISTINCT
[Extent1].[StandardId] AS [StandardId]
FROM [dbo].[Student] AS [Extent1] ) AS [Distinct1]
LEFT OUTER JOIN [dbo].[Student] AS [Extent2] ON ([Distinct1].[StandardId] = [Extent2]. [StandardId]) OR (([Distinct1].[StandardId] IS NULL) AND ([Extent2].[StandardId] IS NULL))
) AS [Project2]
ORDER BY [Project2].[StandardId] ASC, [Project2].[C2] ASC
go
Why there is no GroupBy clause in SQL? If there is no GroupBy clause needed, can’t we just use simple Select with OrderBy and without Joins? Can anyone explain the above query?
The bottom line is: because SQL can't return nested result sets.
Every SQL SELECT statement returns a flat list of values. LINQ is capable of returning object graphs, i.e. objects with nested objects. That's exactly what LINQ's GroupBy does.
In SQL, a GROUP BY statement only returns the grouping columns and aggregate results:
SELECT StandardId, COUNT(*)
FROM Students
GROUP BY StandardId;
The rest of the student columns is gone.
A LINQ GroupBy statement returns something like
StandardId
StudentId StudentName
1
21 "Student1"
15 "Student2"
2
48 "Student3"
91 "Student4"
17 "Student5"
Therefore, a SQL GROUP BY statement can never be the source for a LINQ GroupBy.
Entity Framework (6) knows this and it generates a SQL statement that pulls all required data from the database, adds some parts that make grouping easier, and it creates the groupings client-side.
I have to iterate through a collection of objects (let's say ID's), and execute a specific query for each of these objects. For example:
IEnumerable<int> ids = getIDs(); //[1,2,3,4...]
Right now I have this solution:
DBEntities db = new DBEntities();
var results =
from a in db.TABLEA
join b in db.TABLEB on a.id equals b.id
join c in db.TABLEC on b.oid equals c.oid
where ids.Contains(c.id)
select a;
but keep in mind that the list of IDs is smaller than the table where I am searching. That being said, the solution above seems inefficient, since I am looking for each record of my table against a smaller list, when I wanted the opposite. I also do not want to iterate through the list, and execute the query for one element at a time.
Ideally, I would want something like this:
DBEntities db = new DBEntities();
(some data structure) ids = getIDs();
var results =
from a in db.TABLEA
join b in db.TABLEB on a.id equals b.id
join c in db.TABLEC on b.oid equals c.oid
join i in ids on c.id equals i.id;
The (pseudo-)code above would iterate my elements of the list, in a single query, doing so in a single query and performing my filter by each element of the list.
Is this the way to do it? If so, what is the appropriate data structure to implement this solution? If not, which alternatives do I have?
Magnus's answer is true but not right :)
Technically you do have two options in newer versions of Entity Framework (and I discovered this by chance). Contains of course, but also Join.
Joining with a local sequence of primitive types has always been possible, but very quickly (after some tens of elements) raised a SqlException:
Some part of your SQL statement is nested too deeply. Rewrite the query or break it up into smaller queries.
EF tries to translate the local list to a temporary table in SQL. This is surprisingly non-trivial. It has to build the table by UNION-ing select statements that return 1 element each. This is what it used to look like with only 5 elements!
....
INNER JOIN (SELECT
[UnionAll3].[C1] AS [C1]
FROM (SELECT
[UnionAll2].[C1] AS [C1]
FROM (SELECT
[UnionAll1].[C1] AS [C1]
FROM (SELECT
1 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
UNION ALL
SELECT
2 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable2]) AS [UnionAll1]
UNION ALL
SELECT
3 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable3]) AS [UnionAll2]
UNION ALL
SELECT
4 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable4]) AS [UnionAll3]
UNION ALL
SELECT
5 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable5]) AS [UnionAll4] ON ....
As you see, if you look closely, the UNION statements are nested. The nesting level soon becomes too deep, which made this approach practically useless.
However, currently the SQL looks like this:
....
INNER JOIN (SELECT
1 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable1]
UNION ALL
SELECT
2 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable2]
UNION ALL
SELECT
3 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable3]
UNION ALL
SELECT
4 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable4]
UNION ALL
SELECT
5 AS [C1]
FROM ( SELECT 1 AS X ) AS [SingleRowTable5]) AS [UnionAll4] ON ....
Still not a beauty, but the nesting is replaced by chaining and the list can contain hundreds of elements.
But... (and this is why Magnus's answer is true), it doesn't perform well. A simple test with 2000 elements in the list took 2.5s with join and .25s with Contains. So there's still no practical case for joining with a local sequence.
If this is linq2Sql (or Linq2Entites) your only option is as in your example 1. You can not "join" a table with an in memory list. You have to use Contains. Which will be translated to an Where c.id IN(2,3,4,5,...) SQL query
I've written a query which should take all the rows from one table and do a subquery to a second table pulling only one value from the most recent record. In SQL Server the query takes about 15 seconds to execute and the LINQ query takes close to 2 minutes. Can someone help me with translating the SQL to LINQ, I must have done something wrong along the way.
The SQL:
SELECT a.isrunning,
worktype = (
SELECT TOP 1
w.worktype
FROM dbo.workorder w WITH (NOLOCK)
WHERE w.assetnum = a.assetnum
ORDER BY w.statusdate DESC
),
a.status,
*
FROM dbo.asset a WITH (NOLOCK)
WHERE a.assetnum IN ('list', 'of', 'asset', 'numbers')
The LINQ Query:
(
from a in db.assets
let wo = (
from w in db.workorders
where w.assetnum == a.assetnum
orderby w.statusdate descending
select w).FirstOrDefault()
where aliasStrings.Contains(a.assetnum)
select new AssetWithWorkType {
...
}
);
It is recommended to have indexes on foreign keys. Also indexes that covers filtering and ordering clauses. So I suggest you to create the following 3 indexes:
CREATE NONCLUSTERED INDEX [IDX_workorder_statusdate] ON dbo.workorder(statusdate)
CREATE NONCLUSTERED INDEX [IDX_workorder_assetnum] ON dbo.workorder(assetnum)
If assetnum column in asset table is not the primary key then additionally:
CREATE NONCLUSTERED INDEX [IDX_asset_assetnum] ON dbo.asset(assetnum)
You can create a temp table for the correlated subquery results, and then join it later. Syntax is not correct, as I dont have your table schemas or data, but the idea is the same.
CREATE TABLE #workTypes (worktype VARCHAR(X), assetnum VARCHAR(x))
INSERT INTO #workTypes
SELECT TOP 1 worktype, assetnum FROM dbo.workorder ORDER BY statusdate DESC
SELECT a.isrunning,
b.worktype,
a.status,
*
FROM dbo.asset a WITH (NOLOCK)
INNER JOIN #worktypes b
ON a.assetnum = b.assetnum
WHERE a.assetnum IN ('list', 'of', 'asset', 'numbers')
How about:
SELECT a.isrunning,
w.worktype,
cnt = count(*)
FROM dbo.asset a WITH (NOLOCK)
INNER JOIN dbo workorder w WITH (NOLOCK) on w.assetnum = a.assetnum
WHERE a.assetnum IN ('list', 'of', 'asset', 'numbers')
This would give you a count of worktypes for each asset and might allow the database server to optimize more efficiently. Also consider adding indices or using a temp table as other answers have suggested.
I'm trying to create a query similar to this:
select randomId
from myView
where ...
group by randomId
NOTE: EF doesn't support the distinct so I was thinking of going around the lack of it with the group by (or so I think)
randomId is numeric
Entity Framework V.6.0.2
This gives me the expected result in < 1 second query
When trying to do the same with EF I have been having some issues.
If I do the LINQ similar to this:
context.myView
.Where(...)
.GroupBy(mt => mt.randomId)
.Select({ Id = group.Key, Count = group.Count() } )
I will get sort of the same result but forcing a count and making the query > 6 seconds
The SQL EF generates is something like this:
SELECT
1 AS [C1],
[GroupBy1].[K1] AS [randomId],
[GroupBy1].[A1] AS [C2]
FROM (
SELECT
[Extent1].[randomId] AS [K1],
COUNT(1) AS [A1]
FROM [dbo].[myView] AS [Extent1]
WHERE (...)
GROUP BY [Extent1].[randomId]
) AS [GroupBy1]
But, if the query had the count commented out it would be back to < 1 second
If I change the Select to be like:
.Select({ Id = group.Key} )
I will get all of rows without the group by statement in the SQL query and no Distinct whatsoever:
SELECT
[Extent1].[anotherField] AS [anotherField], -- 'this field got included automatically on this query and I dont know why, it doesnt affect outcome when removed in SQL server'
[Extent1].[randomId] AS [randomId]
FROM [dbo].[myView] AS [Extent1]
WHERE (...)
Other failed attempts:
query.GroupBy(x => x.randomId).Select(group => group.FirstOrDefault());
The query that was generated is as follows:
SELECT
[Limit1].ALL FIELDS,...
FROM (SELECT
[Extent1].[randomId] AS [randomId]
FROM [dbo].[myView] AS [Extent1]
WHERE (...) AS [Project1]
OUTER APPLY (SELECT TOP (1)
[Extent2].ALL FIELDS,...
FROM [dbo].[myView] AS [Extent2]
WHERE (...) AS [Limit1] -- same as the where above
This query performed rather poorly and still managed to return all Ids for the where clause.
Does anyone have an idea on how to force the usage of the group by without an aggregating function like a count?
In SQL it works but then again I have the distinct keyword as well...
Cheers,
J
var query = from p in TableName
select new {Id = p.ColumnNameId};
var distinctItems = query.Distinct().ToList();
Here is the linq query however you should be able to write an equivalent from EF dbset too. If you have issues let me know.
Cheers!
I have a problem that I know how to solve in SQL but not with Linq to Entities.
My data looks like this:
ID GROUP TIMESTAMP
-- ----- ---------
1 A 2011-06-20
2 A 2011-06-21
3 B 2011-06-21
4 B 2011-06-22
5 B 2011-06-23
6 C 2011-06-30
I want to retrieve all the Entity objects (not just the ID) such that I am only getting the most recent record from each group. (ie. the records with ids 2, 5, 6)
In SQL I would do something like this:
SELECT * FROM my_table a
WHERE a.timestamp =
(SELECT MAX(timestamp) FROM my_table b
WHERE a.group = b.group)
(For the sake of this question you can assume that timestamp is unique within each group).
I'd like to do this query against a WCF Data Service using Linq to Entities but I can't seem to have a nested query that references the outside query like this. Can anyone help?
Possibly not as clean and efficient as the hand written version but here's what I came up with
var q = from a in db.MyEntities
where a.Timestamp == (from b in db.MyEntities
where b.Group == a.Group
select b.Timestamp).Max()
select a;
which translates into this SQL
SELECT
[Project1].[Id] AS [Id],
[Project1].[Group] AS [Group],
[Project1].[Timestamp] AS [Timestamp]
FROM ( SELECT
[Extent1].[Id] AS [Id],
[Extent1].[Group] AS [Group],
[Extent1].[Timestamp] AS [Timestamp],
[SSQTAB1].[A1] AS [C1]
FROM [MyEntities] AS [Extent1]
OUTER APPLY
(SELECT
MAX([Extent2].[Timestamp]) AS [A1]
FROM [MyEntities] AS [Extent2]
WHERE [Extent2].[Group] = [Extent1].[Group]) AS [SSQTAB1]
) AS [Project1]
WHERE [Project1].[Timestamp] = [Project1].[C1]
Hi try to use linqer that will convert your sql statements to linq query.
Linqer
Best Regards
This should work:
var query = db.my_table
.GroupBy(p=>p.group)
.Select(p=>p.OrderByDescending(q=>q.timestamp).First());
Here you go.A simple way to do.
var result = (from x in my_table
group x by x.Group into g
select new
{
g.Key,
timestamp = g.Max(x => x.TimeStamp),
g //This will return everything in g
});