I'm using Entity Framework with postgresql. A common case is that I'd like to select a function result as a part of a query, like
SELECT t.id, func1(t.id) FROM table1 t WHERE...
The problem is, that I cannot import stored procedures to entity framework from postgresql. (I get a not supported exception when trying to import them.) As a workaround, I have views of form
id, func1(id)
However, this seems to be very slow when used in joins. An example of a slow query:
SELECT t.id, f.func_result
FROM table1 t
JOIN func_view f on f.id = t.id
WHERE t.id IN (...)
About 1000 times faster version, even with a correlating subquery:
SELECT t.id,(
SELECT f.func_result
FROM func_view f
WHERE f.id = t.id
) FROM table1 t
WHERE t.id IN (...)
However, when I write this query in entity framework, it seems to use the first version. Is there a way to force EF to use a subquery? Or is there another more efficient workaround?
Related
This Sql query returns the expected data. I need to do the same in EF Query. I am not sure how to do it all in one EF query.
WITH cteproductactions (productkey, actionid) AS (
SELECT productkey, count(*) FROM productactions
GROUP BY productkey
HAVING count(*)>0
)
SELECT p.name,p.productkey,p.imageurl
FROM product p
INNER JOIN cteproductactions c on p.ProductKey=c.productkey
WHERE p.profileid=100
EF Query
var products = productRepo.Where(x => x.profileid=100);
var productkeys = products.Select(x => x.ProductKey).ToList();
var productActions = productActionsRepo.Where(x => productkeys.Contains(x.ProductKey));
You'll tie yourself in knots trying to write an SQL then "converting" it into LINQ or forcing EF to generate an SQL that is the same.. It's better to start from a place where you express what you want in high level (English) and write the LINQ for it; forget the SQL unless there's a real problem
The SQL as written doesn't really make sense, or need a CTE, the HAVING clause is pointless and none of the columns from the CTE are used in the output. The only purpose the CTE serves is to filter the product list down to those that have at least one productkey, so write an EF from that - "all products Where profile is is 100 and a related product key exists" - don't get bogged down in "how do I make EF do a cte?" because these SQL express the same sentiments without a CTE:
SELECT p.name,p.productkey,p.imageurl
FROM product p
INNER JOIN (SELECT DISTINCT productKey FROM productactions) c on p.ProductKey=c.productkey
WHERE p.profileid=100
SELECT DISTINCT p.name,p.productkey,p.imageurl
FROM product p
INNER JOIN productactions c on p.ProductKey=c.productkey
WHERE p.profileid=100
SELECT p.name,p.productkey,p.imageurl
FROM product p
WHERE p.profileid=100
AND EXISTS(SELECT null FROM productactions c WHERE p.ProductKey=c.productkey)
Assuming product and productactions are in a 1:M relationship connected by productKey, consider something like:
var products = productRepo
.Where(p => p.profileid==100 && p.ProductActions.Any())
.Select(p => new {p.Name, p.ProductKey, p.ImageUrl)
Main message here is "don't start from an SQL mindset and think "how can I make EF do this sql", start from a "What do I want and how can I make EF do it" - forget the SQL unless EF is generating something horrifically underperformant.
I'm attempting to explicitly join 3 tables using a left outer join in a linq query and am running into linq parsing issues. Performing an inner join parses correctly and returns data but using the left outer fails.
Example:
var query = from p in DatabaseContext.Products
where p.ClientID == clientID
join l in DatabaseContext.Licenses on p.ProductID equals l.ProductID into pl
from pli in pl.DefaultIfEmpty()
join a in DatabaseContext.Articles on p.ArticleID equals a.ArticleID into pa
from pai in pa.DefaultIfEmpty()
select new SomeEntityDTO
{
SomethingFromP = p.Something,
SomethingFromL = pli.Something,
SomethingFromA = pai.Something
};
As both joined tables key off of the first table, I can test each individually by removing the other join, e.g., test the query for p to l and then for p to a. These test queries function perfectly. It's also possible to remove the left outer rule and receive a proper result.
var query = from p in DatabaseContext.Products
where p.ClientID == clientID
join l in DatabaseContext.Licenses on p.ProductID equals l.ProductID
join a in DatabaseContext.Articles on p.ArticleID equals a.ArticleID
select new SomeEntityDTO
... the rest ...
Viewing the offending query in SQL Profiler (top code example) I see that the first two tables are successfully joined, e.g.:
SELECT p.Something, l.Something
FROM Products AS p
LEFT JOIN Licenses AS l ON p.ProductID = l.ProductID
WHERE p.ClientID = 5
ORDER BY p.ProductID
And, then right after this successful query, are another 2 queries (identical to each other):
SELECT a.ArticleID, a.Something, <all fields, even when not specified in query>
FROM Articles AS a
ORDER BY a.ArticleID
The outer joined 3 tables will successfully return an object, as long as I don't attempt to access a field from the "a" table. When doing that, I recieve a Null Exception error, as that table was never really joined.
As stated, removing the outer join rule brings back a successfully joined query.
I have attempted to adjust the linq query figuring that the Linq parser had an issue, but to no avail:
var query = from p in DatabaseContext.Products
from l in DatabaseContext.Licenses.Where(g => g.ProduktID == p.ProduktID).DefaultIfEmpty()
from a in DatabaseContext.Articles.Where(g => g.ArticleID == p.ArticleID).DefaultIfEmpty()
where ....
This parses to a set of CROSS APPLYs that doesn't function at all and the profiled query, when copied into a query editor window, doesn't run at all (as opposed to the 3 individual queries seen in the profiler for the first code example). I have also attempted the more complicated lambdas, which also doesn't work.
Is this an error in the Linq parser? Am I doing this completely wrong? (According to the multiple answered questions here on explicit left outer joins (as opposed to natural associations), I'm doing it correctly. But, it doesn't parse correctly. I've avoided creating the associations so I can join them without explicitly defining the join. Is that potentially required here and won't work properly without it?
Note: Each table has complex keys but I only really need to join based on single key values (the DB is part of a product I can't change).
Using. DotNet Core, EntityFramework, EntityFrameworkCore.SqlServer, etc., all version 1.0.1.
Help?
The short answer is to use EF6 instead of EFCore if you absolutely have to have complex Linq queries on your entites, even after the 1.1 release. There are still too many things missing in EFCore compared to EF6.
Roadmap here.
In my case, I kept EFCore and used the Context.Entity.FromSql(query) method in order to get the results. This allowed me to utilize EFCore for most of the EF Entities, and thereby keeping a forward-looking approach to the application, while allowing for special exceptions for complicated queries not based on an actual entity. The plan is to replace those FromSql queries as EF Core matures.
Prior to deciding on .FromSql, I also tested a query on a View and on a stored procedure. In both instances, I failed. For stored procedures, named parameters is not yet implemented, and views are not currently supported unless you attempt to trick EF into thinking the view is actually a table (which brings its own issues).
In order to access EF Core .FromSql, you need to install the following package:
Microsoft.EntityFrameworkCore.Relational
I have four tables in a sql server databases:
Part
-----
Id (PK)
LineId (FK)
other fields...
Line
-----
Id (PK)
ProcessId (FK)
other fields...
Process
-----
Id (PK)
ProcessTypeId (FK)
other fields...
ProcessType
-----
Id (PK)
other fields...
I am trying to use a linq query with fetch to hydrate these entities then map the result to a view model dto.
I am using two queries, one is on Part, I am applying a filter on it to narrow down the result:
var partids = s.Query<Part>()
.Where(p => p.Line.Process.ProcessType.Id == processTypeId)
.Select(p => p.Id);
I then use this query to eager load the related entities and use the first query as a subquery:
var q = s.Query<Part>()
.Fetch(p => p.Line)
.ThenFetch(l => l.Process)
.ThenFetch(pr => pr.ProcessType)
.Where(p => partids.Contains(p.Id))
.ToList();
Though this query works, I noticed that was taking a very long time to load. So, using a profiler, I looked at the generated SQL which was:
SELECT part0_.Id AS Id0_0_,
line1_.Id AS Id1_1_,
process2_.Id AS Id2_2_,
process3_.Id AS Id3_3_,
part0_.Name AS Name0_0_,
part0_.LineId AS Line3_0_0_,
line1_.Name AS Name1_1_,
line1_.ProcessId AS Proccess3_1_1_,
process2_.Name AS Name2_2_,
process2_.ProcessTypeId AS Proccess3_2_2_,
process3_.Name AS Name3_3_
FROM part part0_
LEFT OUTER JOIN Line line1_ ON part0_.LineId=line1_.Id
LEFT OUTER JOIN Process process2_ ON line1_.ProcessId=process2_.Id
LEFT OUTER JOIN ProcessType process3_ ON process2_.ProcessTypeId=process3_.Id
WHERE part0_.Id IN
( SELECT part4_.Id
FROM Part Part4_
INNER JOIN Line Line5_ ON Part4_.LineId=Line5_.Id
WHERE process2_.ProcessTypeId= 126 );
Th subquery joining back onto the main query is causing this to run extremely slow in most cases.
I would have expected the generated SQL to be this:
SELECT part0_.Id AS Id0_0_,
line1_.Id AS Id1_1_,
process2_.Id AS Id2_2_,
process3_.Id AS Id3_3_,
part0_.Name AS Name0_0_,
part0_.LineId AS Line3_0_0_,
line1_.Name AS Name1_1_,
line1_.ProcessId AS Proccess3_1_1_,
process2_.Name AS Name2_2_,
process2_.ProcessTypeId AS Proccess3_2_2_,
process3_.Name AS Name3_3_
FROM part part0_
LEFT OUTER JOIN Line line1_ ON part0_.LineId=line1_.Id
LEFT OUTER JOIN Process process2_ ON line1_.ProcessId=process2_.Id
LEFT OUTER JOIN ProcessType process3_ ON process2_.ProcessTypeId=process3_.Id
WHERE part0_.Id IN
( SELECT part4_.Id
FROM Part part4_
INNER JOIN Line Line5_ ON part4_.LineId=Line5_.Id
INNER JOIN Process Process6_ ON Line5_.LineId=Process6_.Id
WHERE Process6_.ProcessTypeId= 126 );
I am using NHibernate 4 with the linq provider for all of my queries. Am I missing something in my linq query here?
The work around I use at the moment is to hydrate the partids query with ToList and then use the list of ids from memory. However, I would like to avoid two round trips to the database in this scenario if possible.
It is not currently feasible for me to use the QueryOver or HQL apis because all of my querying a filter code uses linq.
Please help!
Unfortunately, I think the workaround that you're currently using is your best option.
This is a bug in NHibernate, it currently affects version 4 and I'm pretty sure it was there in 3.3 too.
The bug has been reported.
I would like to know how to efficiently count (SQL server side) the amount of distinct count of results for a specific range of a related entity that has a many to many relationship.
This is the current situation in entity Framework:
Table1 1<------->∞ Table2
Table2 ∞<------->∞ Table4
Table 2 and Table 4 have a many to many relationship and are linked with Table3 in SQL.
What I want is the distinct count of table4 results related to a specific range of Table1.
In LinQ to SQL the query is this:
(from dc in Table1
join vc in Table2 on dc.Table1Id equals vc.Table2Id
join vcac in Table3 on vc.Table2Id equals vcac.Table3Id
join ac in Table4 on vcac.Table3Id equals ac.Table4Id
where dc.Table1Id > 200000
group ac by ac.Table4Id into nieuw
select new { acid= nieuw.Key}).Count()
This lets SQL server return the count directly.
Because the extra table for the many to many relation ( Table3) is gone, I have had problems converting this query to L2E in query syntax. ( since I cannot join table 4 with table 2 with an inner join).
I have this in chained syntax, however, is this efficient ( does this fetch the whole list, or does it let SQLserver do the count, as I'm not sure this is an efficient way to select, Table 2 contains about 30.000 entries, I don't want it to fetch this result just to count it):
context.Table4.Where(a => a.Table2.Any(v => v.Table1Id > 200000)).Select(a => aTable4Id).Distinct().Count();
How would I go converting the Linq2SQL query into L2E in the query syntax ? Or is the chained syntax fine in this situation ?
The .Select() method uses deferred execution, meaning it won't actually run the query until you need the results. At that point in the chain it still exists only as a query definition. Then you modify with .Distinct() before getting .Count() - which does query the database using a SQL GROUP BY statement. So you should be good.
Using EF DB first I have two entities (Supplier, Product) that have a many-to-many relationship. Entity Framework does not create an entity for the associated table (SupplierProduct) as the associated table contains only the primary keys of the strong entities.
I have been getting all Suppliers that do not supply a given product with the following query:
var q1 = context.Suppliers.Where(s=>!s.Products.Any(p=>p.Id == 1));
The SQL produced uses an EXISTS dependent subquery similar to this:
SELECT *
FROM Suppliers s
WHERE NOT EXISTS
(SELECT 1
FROM SupplierProduct sp WHERE sp.SupplierId = s.Id && sp.ProductId = 1)
Is it possible, using Linq to Entities method syntax, to produce a query that uses joins on the associated table instead?
ie:
SELECT DISTINCT s.*
FROM SupplierProduct sp
JOIN Supplier s ON s.Id = sp.SupplierId;
WHERE sp.ProductId != 1
Update
As pointed out by JoeEnos my queries above don't do the same thing. The NOT EXISTS subquery is probably the best way to go here. What if I was trying to get all suppliers who did supply a product? I would change my linq to entities query slightly to:
var q1 = context.Suppliers.Where(s => s.Products.Any(p=>p.Id == 1));
And the SQL generated would be:
SELECT *
FROM Suppliers s
WHERE EXISTS
(SELECT 1
FROM SupplierProduct sp WHERE sp.SupplierId = s.Id && sp.ProductId = 1)
Which is fine, I get the result I want. However if I was writing SQL in this case I would normally do:
SELECT s.*
FROM SupplierProduct sp
JOIN Supplier s ON s.Id = sp.SupplierId;
WHERE sp.ProductId = 1
Can my linq to entities query be changed to produce the above SQL?
To generate SQL where a join is used instead of EXISTS when selecting an entity based on its m:n association with another entity SelectMany() can be used. Eg:
var q1 = context.Suppliers.Where(s => s.Products.Any(p=>p.Id == 1));
Can be rewritten to:
var q1 = context.Products.Where(p => p.Id == 1).SelectMany(p => p.Suppliers);
Your two queries do very different things. Your Any/EXISTS query gets suppliers who do not have product 1 at all. Your JOIN query gets all suppliers who have any products other than 1, regardless of whether or not they also have product 1.
I don't think you can do what you're looking for with just a JOIN and WHERE - you can do it with an IN clause, but I think the EXISTS query is the most correct way of looking for your data.
In any case, one of the wonderful things about Entity Framework is that you don't have to worry about what gets generated - as long as the LINQ statement is ok, then it will find the best way of writing the query, and you should never have to look at it. That's especially true when you do paging and other things like that, where the LINQ is simple, but the generated SQL is horribly ugly.