I currently have some stored procedures in T-SQL that I would like to translate to LINQ-To-Entities for greater maintainability. However, no matter how I seem to construct the LINQ query, when I inspect the generated code, it's an abhorrent terribly-performing monstrosity. I've investigated into various combinations of "let" clauses, joins, shifting around the "where" clause to both inside and out of the anonymous type selection, or even using the extension ".Where<>()" method piecemeal in the other indiciations themselves, and nothing seems to generate code anywhere as close to what I would expect or need.
The difficulties seem threefold:
The joins that I need to do are on combinations of booleans, whereas LINQ-To-Entities seems to only have join functionality for equijoins. How can I translate these joins?
I need to join on the same table multiple different times with different join/where clauses, so that I can select a different value for each record. How do I accomplish this in LINQ-To-Entities?
How do I prevent my joins on entity collections become nested messes?
The T-SQL query I'm attempting to translate is this (where the hardcoded numbers are specific hardcoded types):
SELECT Transport.*,
[Address].Street1,
Carrier1Insurance.InsuranceNumber,
Carrier2Insurance.InsuranceNumber,
Carrier3Insurance.InsuranceNumber
FROM Transport
INNER JOIN Appoint ON Transport.AppointKEY = Appoint.AppointKEY
INNER JOIN Patient ON Appoint.PatientKEY = Patient.PatientKEY
LEFT OUTER JOIN [Address] ON [Address].AddressFKEY = Patient.PatientKEY AND [Address].AddressTypeByTableKEY = 1
LEFT OUTER JOIN PatientInsurance Carrier1Insurance ON Carrier1Insurance.PatientKEY = Patient.PatientKEY AND Carrier1Insurance.CarrierKEY = 7
LEFT OUTER JOIN PatientInsurance Carrier2Insurance ON Carrier2Insurance.PatientKEY = Patient.PatientKEY AND Carrier2Insurance.CarrierKEY = 8
LEFT OUTER JOIN PatientInsurance Carrier3Insurance ON Carrier3Insurance.PatientKEY = Patient.PatientKEY AND (Carrier3Insurance.CarrierKEY <> 7 AND Carrier3Insurance.CarrierKEY = 8)
WHERE (Transport.TransportDate >= '07-01-2013' AND Transport.TransportDate <= '07-31-2013')
AND EXISTS (SELECT TOP 1 *
FROM Remit
WHERE Remit.CarrierKEY = 8
AND Remit.AppointKEY = Transport.AppointKEY
AND Remit.PaidAmt > 0)
And the latest in many, many attempts at LINQ is this:
var medicareTransportList = from transportItem in ClientEDM.Transports
join patientAddress in ClientEDM.Addresses on transportItem.Appoint.PatientKEY equals patientAddress.AddressFKEY
join carrier1Insurance in ClientEDM.PatientInsurances on transportItem.Appoint.PatientKEY equals carrier1Insurance.PatientKEY
join carrier2Insurance in ClientEDM.PatientInsurances on transportItem.Appoint.PatientKEY equals carrier2Insurance.PatientKEY
join otherInsurance in ClientEDM.PatientInsurances on transportItem.Appoint.PatientKEY equals otherInsurance.PatientKEY
where (transportItem.TransportDate > fromDate ** transportItem.TransportDate <= toDate) && transportItem.Appoint.Remits.Any(remit => remit.CarrierKEY == 0 && remit.PaidAmt > 0.00M) &&
(carrier1Insurance.CarrierKEY == 7) &&
(carrier2Insurance.CarrierKEY == 8 ) &&
(otherInsurance.CarrierKEY != 7 &&
otherInsurance.CarrierKEY != 8 ) &&
(patientAddress.AddressTypeByTableKEY == 1)
select new
{
transport = transportItem,
patient = patientAddress,
medicare = medicareInsurance,
medicaid = medicaidInsurance,
other = otherInsurance
};
The LINQ .Join() operator is equivalent to the INNER JOIN SQL operator.
For any other case, use the .GroupJoin() operator.
But do you really need to use join? In many case, using LINQ, SQL JOIN (inner or outer) can be expressed using navigation properties between entities.
Please explains your conceptual data model for a precise answer.
Related
I am trying to get my head around how to generate the equivalent of this LEFT JOIN in Linq.
It contains a subquery which gets the lowest service_id for the person in that row being joined, and then also restricts on service_code = "N" in both the subquery and the join). I just cant seem to get it to work in Linq.
SQL:-
LEFT OUTER JOIN SERVICE ON person.id_person = SERVICE.id_person
AND SERVICE.id_service = (SELECT MIN(id_service) FROM SERVICE WHERE id_person = person.id_person AND service_code = 'N')
AND SERVICE.service_code = 'N'
NB: I know how to do the left join correctly (DefaultIfEmpty() etc). Its the subquery that is the problem. How do I squeeze that subquery into the .Where clause in Linq?
This query should give you desired result and should be more performant. Also it shows how to express LEFT JOIN in alternative way.
var query =
from person in ctx.Person
from service in ctx.Service
.Where(service => service.service_code == 'N')
.Where(service => service.id_person = person.id_person)
.OrderBy(service => service.id_service)
.Take(1)
.DefaultIfEmpty()
select new
{
person,
service
};
We need to convert following query to Entity framework syntax based query but not fond any alternative for '<>' condition:
Query:
select (FirstName+' '+LastName) AS Name ,WorksNumber from [TSHumanResource]
join [TSUserProfile] on [TSUserProfile].[TSPersonID] = [TSHumanResource].[TSPersonID]
join [TSPerson] on [TSPerson].[TSPersonID] = [TSHumanResource].[TSPersonID]
where [TSUserProfile].[TSUserStatusID] = 1
and [EmployeeReference] <> ' ' and [MMSUserID] is not null order by [WorksNumber] asc
Here's what I was tring:
(from HR in oDB.TSHumanResources
join UP in oDB.TSUserProfiles on HR.TSPersonID equals UP.TSPersonID
join P in oDB.TSPersons on HR.TSPersonID equals P.TSPersonID
where UP.TSUserStatusID == 1 && HR.EmployeeReference <>
select new
{
ID = e.TSPersonID ,
ID = e.TID,
}).Take(10);
The SQL Server operator <> means not equal. In c#, the not equal operator is written like this: !=.
You have another problem in your linq query - you are currently doing an inner join instead of a left join.
A left join in LINQ is a bit cumbersome comparing to a left join in SQL - it has to go through a group join first. Your query should look more like this:
from hr in TSHumanResource
join up in TSUserProfile on hr.TSPersonID equals up.TSPersonID into upgroup
from upg in upgroup.DefaultIfEmpty()
join p in TSPerson on upg.TSPersonID equals p.TSPersonID into pgroup
from puphr in pgroup.DefaultIfEmpty()
where up.TSUserStatusID = 1
&& HR.EmployeeReference != " " // Assuming you want it different than a single space
// Other conditions here - I don't know where MMSUserID belongs to
order by hr.WorksNumber // just guessing here - I don't know if it's from hr
I am executing the following LINQ to Entities query but it is stuck and does not return response until timeout. I executed the same query on SQL Server and it return 92000 in 3 sec.
var query = (from r in WinCtx.PartsRoutings
join s in WinCtx.Tab_Processes on r.ProcessName equals s.ProcessName
join p in WinCtx.Tab_Parts on r.CustPartNum equals p.CustPartNum
select new { r}).ToList();
SQL Generated:
SELECT [ I omitted columns]
FROM [dbo].[PartsRouting] AS [Extent1]
INNER JOIN [dbo].[Tab_Processes] AS [Extent2] ON ([Extent1].[ProcessName] = [Extent2].[ProcessName]) OR (([Extent1].[ProcessName] IS NULL) AND ([Extent2].[ProcessName] IS NULL))
INNER JOIN [dbo].[Tab_Parts] AS [Extent3] ON ([Extent1].[CustPartNum] = [Extent3].[CustPartNum]) OR (([Extent1].[CustPartNum] IS NULL) AND ([Extent3].[CustPartNum] IS NULL))
PartsRouting Table has 100,000+ records, Parts = 15000+, Processes = 200.
I tried too many things found online but nothing worked for me as to how I can achieve the result with same performance of SQL.
Based on the comments, looks like the issue is caused by the additional OR with IS NULL conditions in joins generated by the EF SQL translator. They were added in EF in order to emulate the C# == operator semantics which are different from SQL = for NULL values.
You can start by turning that EF behavior off through UseDatabaseNullSemantics property (it's false by default):
WinCtx.Configuration.UseDatabaseNullSemantics = true;
Unfortunately that's not enough, because it fixes the normal comparison operators, but they simply forgot to do the same for join conditions.
In case you are using joins just for filtering (as it seems), you can replace them with LINQ Any conditions which translates to SQL EXISTS and nowadays database query optimizers are treating it the same way as if it was an inner join:
var query = (from r in WinCtx.PartsRoutings
where WinCtx.Tab_Processes.Any(s => r.ProcessName == s.ProcessName)
where WinCtx.Tab_Parts.Any(p => r.CustPartNum == p.CustPartNum)
select new { r }).ToList();
You might also consider using just select r since creating anonymous type with single property just introdeces additional memory overhead with no advantages.
Update: Looking at the latest comment, you do need fields from joined tables (that's why it's important to not omit relevant parts of the query in question). In such case, you could try the alternative join syntax with where clauses:
WinCtx.Configuration.UseDatabaseNullSemantics = true;
var query = (from r in WinCtx.PartsRoutings
from s in WinCtx.Tab_Processes where r.ProcessName == s.ProcessName
from p in WinCtx.Tab_Parts where r.CustPartNum == p.CustPartNum
select new { r, s.Foo, p.Bar }).ToList();
I have looked around and found a few posts on adding a second condition to a JOIN clause but they are always instances of having one column link to another but in my instance I need to just have a column equal a certain value. The RAW SQL I am trying to imitate is:
LEFT OUTER JOIN dbo.TimeCardHeader AS tch2 on tch1.EmployeeID = tch2.EmployeeID && tch2.WeekEndingDate = #PayPeriod
As you can see my second join condition is to match a column value from the table to a variable. I have tried the following LINQ queires but they all fail.
join leftth2 in db.TimeCardHeaders on th1.EmployeeID equals leftth2.EmployeeID AND leftth2.WeekEndingDate == PayPeriod into leftjointh2
join leftth2 in db.TimeCardHeaders on th1.EmployeeID equals leftth2.EmployeeID && leftth2.WeekEndingDate == PayPeriod into leftjointh2
join leftth2 in db.TimeCardHeaders on new
{
employeeID = th1.EmployeeID,
weekEndingDate = leftth2.WeekEndingDate
} equals new
{
employeeID = leftth2.EmployeeID,
weekEndingDate = PayPeriod
}
The first two fail saying AND and && are not valid, the last fails saying leftth2 is not in the scope of the left side.
What is the proper way of doing this?
The condition
tch2.WeekEndingDate = #PayPeriod
doesn't feel like part of the join to me - it's not comparing data in two different rows. It feels like it should be part of what you're joining on, leading to:
join leftth2 in db.TimeCardHeaders.Where(p => p.WeekEndingDate == PayPeriod)
on th1.EmployeeID equals leftth2.EmployeeID into leftjointh2
This question already exists here, but for Entity Framework to TSQL
When using the same navigation property multiple times on a select, Npgsql query results in multiple joins, one for every use of the navigation property. This result in an awful performance hit (tested)
I've read that this is a problem with EF 4, but this problem also occurs on EF 6.
I think that this is an issue with the Npgsql LINQ to SQL translator
This is the code that Npgsql generate for the same navigation property used mutliple times, obviously, only one join is needed (copied from the other questions because is exactly the same case)
LEFT OUTER JOIN [dbo].[Versions] AS [Extent4] ON [Extent1].[IDVersionReported] = [Extent4].[ID]
LEFT OUTER JOIN [dbo].[Versions] AS [Extent5] ON [Extent1].[IDVersionReported] = [Extent5].[ID]
LEFT OUTER JOIN [dbo].[Versions] AS [Extent6] ON [Extent1].[IDVersionReported] = [Extent6].[ID]
LEFT OUTER JOIN [dbo].[Versions] AS [Extent7] ON [Extent1].[IDVersionReported] = [Extent7].[ID]
Is it posible to tune PostgreSql to optimize repeated joins?
If not, which option is the best for solving this problem?
Wait until Npgsql gets fixed
Download Npgsql code and find the way to fix it
Intercept generated SQL before reaching the database, parse it, and remove duplicated joins. (Read here )
Do not use navigation properties, use LINQ joins instead
Indeed, this is a problem of Entity Framework, but I found a workaround, hope that this helps someone.
This was the original where part of the LINQ query:
from cr in Creditos
where cr.validado == 1 &&
cr.fecharegistro >= Desde &&
cr.fecharegistro <= Hasta &&
!ProductosExcluidos.Contains(cr.idproducto.Value) &&
cr.amortizaciones.Sum(am => am.importecapital - am.pagoscap - am.capcancel) > 1
//All references to the navigation property cr.numcliente
//results on a separated LEFT OUTTER JOIN between this the 'creditos' and 'clientes' tables
select new ArchivoCliente
{
RFC = cr.numcliente.rfc,
Nombres = cr.numcliente.nombres,
ApellidoPaterno = cr.numcliente.apellidopaterno,
ApellidoMaterno = cr.numcliente.apellidomaterno,
}
Note the last condition of the where, that condition does a sum of all child entities of cr, if we take out that last condition, all the duplicated LEFT OUTTER JOIN are replaced by one single JOIN, for some reason, Entity Framework doesn't like subqueries or aggregates on the where part of the query
If instead we replace the original query with this other equivalent query only a single LEFT OUTTER JOIN is generated.
(from cr in Creditos
where cr.validado == 1 &&
cr.fecharegistro >= Desde &&
cr.fecharegistro <= Hasta &&
!ProductosExcluidos.Contains(cr.idproducto.Value) &&
//Excluded aggregate function condition from the first where
//the value is now on the select and used for posterior filtering
select new ArchivoCliente
{
RFC = cr.numcliente.rfc,
Nombres = cr.numcliente.nombres,
ApellidoPaterno = cr.numcliente.apellidopaterno,
ApellidoMaterno = cr.numcliente.apellidomaterno,
SumaAmort = cr.amortizaciones.Sum(am => am.importecapital - am.pagoscap - am.capcancel)
}).Where (x => x.SumaAmort > 1);
Now instead of directly filtering on the first where statement, the aggregate result is stored as part of the projection, and then, a second where is applied to the resulting query.
This results in a much faster query, with only the necessary joins on the translated SQL statement.