This Sql query returns the expected data. I need to do the same in EF Query. I am not sure how to do it all in one EF query.
WITH cteproductactions (productkey, actionid) AS (
SELECT productkey, count(*) FROM productactions
GROUP BY productkey
HAVING count(*)>0
)
SELECT p.name,p.productkey,p.imageurl
FROM product p
INNER JOIN cteproductactions c on p.ProductKey=c.productkey
WHERE p.profileid=100
EF Query
var products = productRepo.Where(x => x.profileid=100);
var productkeys = products.Select(x => x.ProductKey).ToList();
var productActions = productActionsRepo.Where(x => productkeys.Contains(x.ProductKey));
You'll tie yourself in knots trying to write an SQL then "converting" it into LINQ or forcing EF to generate an SQL that is the same.. It's better to start from a place where you express what you want in high level (English) and write the LINQ for it; forget the SQL unless there's a real problem
The SQL as written doesn't really make sense, or need a CTE, the HAVING clause is pointless and none of the columns from the CTE are used in the output. The only purpose the CTE serves is to filter the product list down to those that have at least one productkey, so write an EF from that - "all products Where profile is is 100 and a related product key exists" - don't get bogged down in "how do I make EF do a cte?" because these SQL express the same sentiments without a CTE:
SELECT p.name,p.productkey,p.imageurl
FROM product p
INNER JOIN (SELECT DISTINCT productKey FROM productactions) c on p.ProductKey=c.productkey
WHERE p.profileid=100
SELECT DISTINCT p.name,p.productkey,p.imageurl
FROM product p
INNER JOIN productactions c on p.ProductKey=c.productkey
WHERE p.profileid=100
SELECT p.name,p.productkey,p.imageurl
FROM product p
WHERE p.profileid=100
AND EXISTS(SELECT null FROM productactions c WHERE p.ProductKey=c.productkey)
Assuming product and productactions are in a 1:M relationship connected by productKey, consider something like:
var products = productRepo
.Where(p => p.profileid==100 && p.ProductActions.Any())
.Select(p => new {p.Name, p.ProductKey, p.ImageUrl)
Main message here is "don't start from an SQL mindset and think "how can I make EF do this sql", start from a "What do I want and how can I make EF do it" - forget the SQL unless EF is generating something horrifically underperformant.
Related
I'm using Entity Framework with postgresql. A common case is that I'd like to select a function result as a part of a query, like
SELECT t.id, func1(t.id) FROM table1 t WHERE...
The problem is, that I cannot import stored procedures to entity framework from postgresql. (I get a not supported exception when trying to import them.) As a workaround, I have views of form
id, func1(id)
However, this seems to be very slow when used in joins. An example of a slow query:
SELECT t.id, f.func_result
FROM table1 t
JOIN func_view f on f.id = t.id
WHERE t.id IN (...)
About 1000 times faster version, even with a correlating subquery:
SELECT t.id,(
SELECT f.func_result
FROM func_view f
WHERE f.id = t.id
) FROM table1 t
WHERE t.id IN (...)
However, when I write this query in entity framework, it seems to use the first version. Is there a way to force EF to use a subquery? Or is there another more efficient workaround?
I have four tables in a sql server databases:
Part
-----
Id (PK)
LineId (FK)
other fields...
Line
-----
Id (PK)
ProcessId (FK)
other fields...
Process
-----
Id (PK)
ProcessTypeId (FK)
other fields...
ProcessType
-----
Id (PK)
other fields...
I am trying to use a linq query with fetch to hydrate these entities then map the result to a view model dto.
I am using two queries, one is on Part, I am applying a filter on it to narrow down the result:
var partids = s.Query<Part>()
.Where(p => p.Line.Process.ProcessType.Id == processTypeId)
.Select(p => p.Id);
I then use this query to eager load the related entities and use the first query as a subquery:
var q = s.Query<Part>()
.Fetch(p => p.Line)
.ThenFetch(l => l.Process)
.ThenFetch(pr => pr.ProcessType)
.Where(p => partids.Contains(p.Id))
.ToList();
Though this query works, I noticed that was taking a very long time to load. So, using a profiler, I looked at the generated SQL which was:
SELECT part0_.Id AS Id0_0_,
line1_.Id AS Id1_1_,
process2_.Id AS Id2_2_,
process3_.Id AS Id3_3_,
part0_.Name AS Name0_0_,
part0_.LineId AS Line3_0_0_,
line1_.Name AS Name1_1_,
line1_.ProcessId AS Proccess3_1_1_,
process2_.Name AS Name2_2_,
process2_.ProcessTypeId AS Proccess3_2_2_,
process3_.Name AS Name3_3_
FROM part part0_
LEFT OUTER JOIN Line line1_ ON part0_.LineId=line1_.Id
LEFT OUTER JOIN Process process2_ ON line1_.ProcessId=process2_.Id
LEFT OUTER JOIN ProcessType process3_ ON process2_.ProcessTypeId=process3_.Id
WHERE part0_.Id IN
( SELECT part4_.Id
FROM Part Part4_
INNER JOIN Line Line5_ ON Part4_.LineId=Line5_.Id
WHERE process2_.ProcessTypeId= 126 );
Th subquery joining back onto the main query is causing this to run extremely slow in most cases.
I would have expected the generated SQL to be this:
SELECT part0_.Id AS Id0_0_,
line1_.Id AS Id1_1_,
process2_.Id AS Id2_2_,
process3_.Id AS Id3_3_,
part0_.Name AS Name0_0_,
part0_.LineId AS Line3_0_0_,
line1_.Name AS Name1_1_,
line1_.ProcessId AS Proccess3_1_1_,
process2_.Name AS Name2_2_,
process2_.ProcessTypeId AS Proccess3_2_2_,
process3_.Name AS Name3_3_
FROM part part0_
LEFT OUTER JOIN Line line1_ ON part0_.LineId=line1_.Id
LEFT OUTER JOIN Process process2_ ON line1_.ProcessId=process2_.Id
LEFT OUTER JOIN ProcessType process3_ ON process2_.ProcessTypeId=process3_.Id
WHERE part0_.Id IN
( SELECT part4_.Id
FROM Part part4_
INNER JOIN Line Line5_ ON part4_.LineId=Line5_.Id
INNER JOIN Process Process6_ ON Line5_.LineId=Process6_.Id
WHERE Process6_.ProcessTypeId= 126 );
I am using NHibernate 4 with the linq provider for all of my queries. Am I missing something in my linq query here?
The work around I use at the moment is to hydrate the partids query with ToList and then use the list of ids from memory. However, I would like to avoid two round trips to the database in this scenario if possible.
It is not currently feasible for me to use the QueryOver or HQL apis because all of my querying a filter code uses linq.
Please help!
Unfortunately, I think the workaround that you're currently using is your best option.
This is a bug in NHibernate, it currently affects version 4 and I'm pretty sure it was there in 3.3 too.
The bug has been reported.
I would like to know how to efficiently count (SQL server side) the amount of distinct count of results for a specific range of a related entity that has a many to many relationship.
This is the current situation in entity Framework:
Table1 1<------->∞ Table2
Table2 ∞<------->∞ Table4
Table 2 and Table 4 have a many to many relationship and are linked with Table3 in SQL.
What I want is the distinct count of table4 results related to a specific range of Table1.
In LinQ to SQL the query is this:
(from dc in Table1
join vc in Table2 on dc.Table1Id equals vc.Table2Id
join vcac in Table3 on vc.Table2Id equals vcac.Table3Id
join ac in Table4 on vcac.Table3Id equals ac.Table4Id
where dc.Table1Id > 200000
group ac by ac.Table4Id into nieuw
select new { acid= nieuw.Key}).Count()
This lets SQL server return the count directly.
Because the extra table for the many to many relation ( Table3) is gone, I have had problems converting this query to L2E in query syntax. ( since I cannot join table 4 with table 2 with an inner join).
I have this in chained syntax, however, is this efficient ( does this fetch the whole list, or does it let SQLserver do the count, as I'm not sure this is an efficient way to select, Table 2 contains about 30.000 entries, I don't want it to fetch this result just to count it):
context.Table4.Where(a => a.Table2.Any(v => v.Table1Id > 200000)).Select(a => aTable4Id).Distinct().Count();
How would I go converting the Linq2SQL query into L2E in the query syntax ? Or is the chained syntax fine in this situation ?
The .Select() method uses deferred execution, meaning it won't actually run the query until you need the results. At that point in the chain it still exists only as a query definition. Then you modify with .Distinct() before getting .Count() - which does query the database using a SQL GROUP BY statement. So you should be good.
I am having an issue getting my LINQ query to output as I would like. I am unsure I am taking the right approach.
Tables:
I have two tables Contacts and Permissions with which I perform a LEFT OUTER JOIN.
Join Query:
from contact in Contacts
join permission in Permissions on contact.Id equals permission.ObjectId into permissionGrp
from p in permissionGrp.DefaultIfEmpty()
where (p==null && isAllowed) || (p!=null && /* ... further conditions */))
orderby /* ... ordering removed */
select new { contact, permission = p };
This matches the permissions to a contact where applicable, and null when no matching permission exists.
Desired
I don't wish to have duplicate contacts, I am only interested in the first Contact-Permission record. Like so:
Attempt:
So I assumed that I need to Group By my contact.Id and somehow select FirstOrDefault() on the permissions collection.
from contact in Contacts
join permission in Permissions on contact.Id equals permission.ObjectId into permissionGrp
from p in permissionGrp.DefaultIfEmpty()
where (p==null && isAllowed) || (p!=null && /* ... further conditions */))
orderby /* ... ordering removed */
group p by contact into contactPermissionsGrp
select new { contact = contactPermissionsGrp.Key, permission = contactPermissions.FirstOrDefault() };
Result:
Unfortunately this results in a NotSupportedException: Specific method is not supported.. But I am not sure if my approach is correct or a limitation of the LightSpeed ORM.
Any advise would be appreciated.
Answer question with more questions
Is your query possible in raw SQL in the database you are targeting? If the answer is No then what hope have you of replicating it in LINQ?
I personally don't think that what you are trying to achieve is possible to do via LINQ against MySQL. It would be possible in MS SQL in raw T-SQL by defining an expression and attaching a RANK() column to it then doing a query on that expression.
I feel that your available solutions are:
Find out how to write this query using raw SQL in your native dialect. Lightspeed will let you execute raw SQL and it will even (where you return enough columns) rehydrate that custom query into Entitys (however I don't think that is what you are after in this case).
Give up on reducing the "duplicates" efficiently in the database. Pull the duplicates into memory and then reduce them in memory with LINQ queryies against the IEnumerable set that you get back.
Change your database architecture so you can have a simpler query. Sometimes in situations like this I will find honour in having a column on the Contact table such as "MostSignificantPermssion". That has a number of advantages:
Simpler query to get the Contact and the ONE significant Permission record.
Makes it more obvious to other developers that one of the Permissions has a special significance.
Options!
Appendix - Example of implementing (1) in MS SQL
WITH LastUsagePerPerson AS (
SELECT
ULE.PersonId,
ULE.[Device],
ULE.[AppVersion],
ULE.[CreatedOn],
ROW_NUMBER() OVER(PARTITION BY ULE.PersonId ORDER BY ULE.CreatedOn DESC) AS rk
FROM [dbo].[UsageLogEntry] ULE
)
SELECT
[FirstName]
,[LastName]
,[EmailAddress]
,[EmailAddressUnverified]
,[MobileNumber]
,[MobileNumberUnverified]
,[LastDeviceUsed] = LastUsagePerPerson.Device
,[LastAppVersion] = LastUsagePerPerson.AppVersion
,[LastDeviceUsage] = LastUsagePerPerson.CreatedOn
,[LastLoggedInOn]
FROM [dbo].[Person] P
LEFT JOIN LastUsagePerPerson ON P.Id = LastUsagePerPerson.PersonId
WHERE rk = 1
ORDER BY [Id]
I don't know what Lightspeed can or can't. Try to simplify you LINQ Query
In EF i would do something like this.
from c in Contacts
let p = (from p in permission where p.ObjectId == c.Id select p).FirstOrDefault()
select new { ContactID = c.Id,
Name = c.Name,
Permission = p.PermissionId,
Permitted = p.Permitted};
I have understood what you are trying to get and I have solved your problem,just follow the below code what I did...
select * from contacts as a
left join permissions as b
on a.ContactId = b.ContactId
group by a.ContactId ;
I have got the requeried result using the above code which you were trying to get.Just try that,your problem will be solved.
Using EF DB first I have two entities (Supplier, Product) that have a many-to-many relationship. Entity Framework does not create an entity for the associated table (SupplierProduct) as the associated table contains only the primary keys of the strong entities.
I have been getting all Suppliers that do not supply a given product with the following query:
var q1 = context.Suppliers.Where(s=>!s.Products.Any(p=>p.Id == 1));
The SQL produced uses an EXISTS dependent subquery similar to this:
SELECT *
FROM Suppliers s
WHERE NOT EXISTS
(SELECT 1
FROM SupplierProduct sp WHERE sp.SupplierId = s.Id && sp.ProductId = 1)
Is it possible, using Linq to Entities method syntax, to produce a query that uses joins on the associated table instead?
ie:
SELECT DISTINCT s.*
FROM SupplierProduct sp
JOIN Supplier s ON s.Id = sp.SupplierId;
WHERE sp.ProductId != 1
Update
As pointed out by JoeEnos my queries above don't do the same thing. The NOT EXISTS subquery is probably the best way to go here. What if I was trying to get all suppliers who did supply a product? I would change my linq to entities query slightly to:
var q1 = context.Suppliers.Where(s => s.Products.Any(p=>p.Id == 1));
And the SQL generated would be:
SELECT *
FROM Suppliers s
WHERE EXISTS
(SELECT 1
FROM SupplierProduct sp WHERE sp.SupplierId = s.Id && sp.ProductId = 1)
Which is fine, I get the result I want. However if I was writing SQL in this case I would normally do:
SELECT s.*
FROM SupplierProduct sp
JOIN Supplier s ON s.Id = sp.SupplierId;
WHERE sp.ProductId = 1
Can my linq to entities query be changed to produce the above SQL?
To generate SQL where a join is used instead of EXISTS when selecting an entity based on its m:n association with another entity SelectMany() can be used. Eg:
var q1 = context.Suppliers.Where(s => s.Products.Any(p=>p.Id == 1));
Can be rewritten to:
var q1 = context.Products.Where(p => p.Id == 1).SelectMany(p => p.Suppliers);
Your two queries do very different things. Your Any/EXISTS query gets suppliers who do not have product 1 at all. Your JOIN query gets all suppliers who have any products other than 1, regardless of whether or not they also have product 1.
I don't think you can do what you're looking for with just a JOIN and WHERE - you can do it with an IN clause, but I think the EXISTS query is the most correct way of looking for your data.
In any case, one of the wonderful things about Entity Framework is that you don't have to worry about what gets generated - as long as the LINQ statement is ok, then it will find the best way of writing the query, and you should never have to look at it. That's especially true when you do paging and other things like that, where the LINQ is simple, but the generated SQL is horribly ugly.