In the following C# EF LINQ code, the field IcasSemester is always rendering as "yes".
public static IQueryable<EnrolmentSummaryViewModel> GetQueryable(EducorDbRepo repo, int? userId, int? semesterId)
{
return from enrolment in repo.EnrolmentQueries.GetAllBase(userId)
join t in repo.Tenants.GetAll() on enrolment.TenantId equals t.Id
join u in repo.Users.GetAll() on enrolment.UserId equals u.Id
join up in repo.UserProfiles.GetAll() on enrolment.UserId equals up.UserId
join s1 in repo.Semesters.GetAll() on enrolment.SemesterId equals s1.Id into s2
from s in s2.DefaultIfEmpty()
where (semesterId == null || enrolment.SemesterId == semesterId)
orderby enrolment.EnrolmentDate descending, enrolment.Created descending
select new EnrolmentSummaryViewModel
{
Id = enrolment.Id,
TenantId = enrolment.TenantId,
Tenant = t.Name,
UserId = enrolment.UserId,
StudentEmail = u.Email,
StudentFirstName = up.FirstName,
StudentLastName = up.LastName,
StudentFullName = up.FirstName + " " + up.LastName,
IcasSemester = enrolment.SemesterId != null ? "yes" : "no",
SemesterId = enrolment.SemesterId,
AmountPaid = enrolment.DiscountedCost,
TotalCost = enrolment.OriginalCost,
Status = enrolment.Status,
EnrolmentDate = enrolment.EnrolmentDate,
Created = enrolment.Created,
Modified = enrolment.Modified,
EnrolmentSourceId = enrolment.EnrolmentSourceId,
OverallStatusId = enrolment.OverallStatusId,
PaymentMethodId = enrolment.PaymentMethodId
};
}
The outer join is working correctly, but the conditional evaluation is getting hard-coded into the final SQL (from Sql Profiler):
exec sp_executesql N'SELECT
[Project1].[Id] AS [Id],
[Project1].[TenantId] AS [TenantId],
[Project1].[Name] AS [Name],
[Project1].[UserId] AS [UserId],
[Project1].[Email] AS [Email],
[Project1].[FirstName] AS [FirstName],
[Project1].[LastName] AS [LastName],
[Project1].[C1] AS [C1],
[Project1].[C2] AS [C2],
[Project1].[SemesterId] AS [SemesterId],
[Project1].[DiscountedCost] AS [DiscountedCost],
[Project1].[OriginalCost] AS [OriginalCost],
[Project1].[Status] AS [Status],
[Project1].[EnrolmentDate] AS [EnrolmentDate],
[Project1].[Created] AS [Created],
[Project1].[Modified] AS [Modified],
[Project1].[EnrolmentSourceId] AS [EnrolmentSourceId],
[Project1].[OverallStatusId] AS [OverallStatusId],
[Project1].[PaymentMethodId] AS [PaymentMethodId]
FROM ( SELECT
[Extent1].[Id] AS [Id],
[Extent1].[TenantId] AS [TenantId],
[Extent1].[UserId] AS [UserId],
[Extent1].[Status] AS [Status],
[Extent1].[OverallStatusId] AS [OverallStatusId],
[Extent1].[SemesterId] AS [SemesterId],
[Extent1].[EnrolmentDate] AS [EnrolmentDate],
[Extent1].[OriginalCost] AS [OriginalCost],
[Extent1].[DiscountedCost] AS [DiscountedCost],
[Extent1].[Created] AS [Created],
[Extent1].[Modified] AS [Modified],
[Extent1].[EnrolmentSourceId] AS [EnrolmentSourceId],
[Extent1].[PaymentMethodId] AS [PaymentMethodId],
[Extent2].[Name] AS [Name],
[Extent3].[Email] AS [Email],
[Extent4].[FirstName] AS [FirstName],
[Extent4].[LastName] AS [LastName],
CASE WHEN ([Extent4].[FirstName] IS NULL) THEN N'''' ELSE [Extent4].[FirstName] END + N'' '' + CASE WHEN ([Extent4].[LastName] IS NULL) THEN N'''' ELSE [Extent4].[LastName] END AS [C1],
N''yes'' AS [C2]
FROM [dbo].[Enrolments] AS [Extent1]
INNER JOIN [dbo].[Tenants] AS [Extent2] ON [Extent1].[TenantId] = [Extent2].[Id]
INNER JOIN [dbo].[Users] AS [Extent3] ON [Extent1].[UserId] = [Extent3].[Id]
INNER JOIN [dbo].[UserProfiles] AS [Extent4] ON [Extent1].[UserId] = [Extent4].[UserId]
WHERE #p__linq__0 IS NULL OR [Extent1].[SemesterId] = #p__linq__1
) AS [Project1]
ORDER BY [Project1].[EnrolmentDate] DESC, [Project1].[Created] DESC
OFFSET 10 ROWS FETCH NEXT 10 ROWS ONLY ',N'#p__linq__0 int,#p__linq__1 int',#p__linq__0=NULL,#p__linq__1=NULL
I've also tried variations of the conditional clause, such as:
IcasSemester = s != null ? "yes" : "no"
How can I get that conditional field to evaluate correctly?
The issue was in the configuration for the Enrolment entity. The following line was affecting the way the SQL was being rendered (HasRequired should have been HasOptional):
HasRequired(p => p.Semester).WithMany().HasForeignKey(x => x.SemesterId).WillCascadeOnDelete(false);
Related
I'm seeing behavior in LINQ that I just don't understand and I really want to prevent it from happening because it's causing my queries to run much slower than they should.
The following C# code produces the results I'm expecting but when I add an additional dynamic filter to it things start to go south.
resultChildrenTC = resultChildrenTC.Where(x => x.parentId != 0 && x.clientId == this.clientId);
var me = resultChildrenTC.ToList();
Good results from C# above, and what I would expect.
...
CASE WHEN (1 = [Extent12].[accept_revisions_ind]) THEN 1 ELSE 0 END AS [C14],
[Extent4].[client_id] AS [client_id]
FROM [dbo].[automation_sequence_executions] AS [Extent1]
INNER JOIN [dbo].[automation_sequence_status] AS [Extent2] ON [Extent1].[automation_sequence_status_id] = [Extent2].[id]
INNER JOIN [dbo].[automation_sequences] AS [Extent3] ON [Extent1].[automation_sequence_id] = [Extent3].[id]
INNER JOIN [dbo].[project] AS [Extent4] ON [Extent3].[project_id] = [Extent4].[id]
LEFT OUTER JOIN [dbo].[automation_sequence_execution_results] AS [Extent5] ON [Extent1].[id] = [Extent5].[auto_seq_exec_id]
LEFT OUTER JOIN [dbo].[automation_sequence_test_case_status] AS [Extent6] ON [Extent1].[runtime_case_grp_status] = [Extent6].[id]
LEFT OUTER JOIN [dbo].[users] AS [Extent7] ON [Extent1].[executed_by_id] = [Extent7].[id]
INNER JOIN [dbo].[users] AS [Extent8] ON [Extent3].[created_by_id] = [Extent8].[id]
LEFT OUTER JOIN [dbo].[users] AS [Extent9] ON [Extent3].[last_modified_by_id] = [Extent9].[id]
LEFT OUTER JOIN [dbo].[machines] AS [Extent10] ON [Extent1].[machine_id] = [Extent10].[id]
LEFT OUTER JOIN [dbo].[execution_sync_queue] AS [Extent11] ON ([Extent1].[id] = [Extent11].[execution_id]) AND (0 = [Extent11].[status])
LEFT OUTER JOIN [dbo].[execution_schedule] AS [Extent12] ON [Extent1].[schedule_id] = [Extent12].[id]
WHERE (0 <> (CASE WHEN ([Extent1].[parent_group_exec_id] IS NULL) THEN 0 ELSE [Extent1].[parent_group_exec_id] END)) AND (1 = [Extent4].[status]) AND ([Extent4].[client_id] = #p__linq__0)',N'#p__linq__0 int',#p__linq__0=0
Now once I edit the C# code from above to take in the additional filter things change...
resultChildrenTC = resultChildrenTC.Where(x => x.parentId != 0 && x.clientId == this.clientId).Where(filterExpression);
var me = resultChildrenTC.ToList();
When this runs LINQ decides to create a subquery. This is a big deal because later on I need to run an EXISTS on this query and would like to extend on this filter even more but because it's always creating this random subquery it breaks up my filtering which causes poor performance. What gives and is there a way to turn this off?
[Filter1].[client_id1] AS [client_id]
FROM (SELECT [Extent1].[id] AS [id1], [Extent1].[machine_id] AS [machine_id], [Extent1].[executed_by_id] AS [executed_by_id], [Extent1].[external_test_mgmt_id] AS [external_test_mgmt_id1], [Extent1].[schedule_id] AS [schedule_id], [Extent1].[parent_group_exec_id] AS [parent_group_exec_id], [Extent1].[patriarch_id] AS [patriarch_id], [Extent1].[runtime_case_grp_status] AS [runtime_case_grp_status], [Extent1].[execution_start_time] AS [execution_start_time], [Extent2].[id] AS [id3], [Extent2].[name] AS [name2], [Extent2].[fail_alert_ind] AS [fail_alert_ind], [Extent2].[hold_alert_ind] AS [hold_alert_ind], [Extent2].[complete_ind] AS [complete_ind], [Extent3].[id] AS [id2], [Extent3].[name] AS [name1], [Extent3].[created_by_id] AS [created_by_id], [Extent3].[last_modified_by_id] AS [last_modified_by_id], [Extent3].[case_group_ind] AS [case_group_ind], [Extent4].[name] AS [name3], [Extent4].[client_id] AS [client_id1]
FROM (SELECT [Extent1].[id] AS [id1], [Extent1].[machine_id] AS [machine_id], [Extent1].[executed_by_id] AS [executed_by_id], [Extent1].[external_test_mgmt_id] AS [external_test_mgmt_id1], [Extent1].[schedule_id] AS [schedule_id], [Extent1].[parent_group_exec_id] AS [parent_group_exec_id], [Extent1].[patriarch_id] AS [patriarch_id], [Extent1].[runtime_case_grp_status] AS [runtime_case_grp_status], [Extent1].[execution_start_time] AS [execution_start_time], [Extent2].[id] AS [id3], [Extent2].[name] AS [name2], [Extent2].[fail_alert_ind] AS [fail_alert_ind], [Extent2].[hold_alert_ind] AS [hold_alert_ind], [Extent2].[complete_ind] AS [complete_ind], [Extent3].[id] AS [id2], [Extent3].[name] AS [name1], [Extent3].[created_by_id] AS [created_by_id], [Extent3].[last_modified_by_id] AS [last_modified_by_id], [Extent3].[case_group_ind] AS [case_group_ind], [Extent4].[name] AS [name3], [Extent4].[client_id] AS [client_id1]
FROM [dbo].[automation_sequence_executions] AS [Extent1]
INNER JOIN [dbo].[automation_sequence_status] AS [Extent2] ON [Extent1].[automation_sequence_status_id] = [Extent2].[id]
INNER JOIN [dbo].[automation_sequences] AS [Extent3] ON [Extent1].[automation_sequence_id] = [Extent3].[id]
INNER JOIN [dbo].[project] AS [Extent4] ON [Extent3].[project_id] = [Extent4].[id]
WHERE (0 <> (CASE WHEN ([Extent1].[parent_group_exec_id] IS NULL) THEN 0 ELSE [Extent1].[parent_group_exec_id] END)) AND ([Extent3].[name] LIKE ''%UAT%'') AND (1 = [Extent4].[status]) ) AS [Filter1]
LEFT OUTER JOIN [dbo].[automation_sequence_execution_results] AS [Extent5] ON [Filter1].[id1] = [Extent5].[auto_seq_exec_id]
LEFT OUTER JOIN [dbo].[automation_sequence_test_case_status] AS [Extent6] ON [Filter1].[runtime_case_grp_status] = [Extent6].[id]
INNER JOIN [dbo].[users] AS [Extent7] ON [Filter1].[executed_by_id] = [Extent7].[id]
INNER JOIN [dbo].[users] AS [Extent8] ON [Filter1].[created_by_id] = [Extent8].[id]
INNER JOIN [dbo].[users] AS [Extent9] ON [Filter1].[last_modified_by_id] = [Extent9].[id]
LEFT OUTER JOIN [dbo].[machines] AS [Extent10] ON [Filter1].[machine_id] = [Extent10].[id]
LEFT OUTER JOIN [dbo].[execution_sync_queue] AS [Extent11] ON (0 = [Extent11].[status]) AND ([Filter1].[id1] = [Extent11].[execution_id])
LEFT OUTER JOIN [dbo].[execution_schedule] AS [Extent12] ON [Filter1].[schedule_id] = [Extent12].[id]
WHERE [Filter1].[client_id1] = #p__linq__0',N'#p__linq__0 int',#p__linq__0=0
UPDATE - Where clause impact
This is occurring whenever I perform a LIKE on a string field. It seems like Contains is creating this odd behavior.
Subquery:
resultChildrenTC = resultChildrenTC.Where(x => x.parentId != 0 && x.clientId == this.clientId).Where(x=>x.Name.Contains("wes"));
No subquery:
resultChildrenTC = resultChildrenTC.Where(x => x.parentId != 0 && x.clientId == this.clientId).Where(x=>x.Name == "wes");
UPDATE - Possible reasoning
I've created a number of tickets that may all be related, which I didn't know at the time.
From what I've learned about LINQ-to-entities it's a mistake to use the included FK in your queries. A more detailed view into this can be found in the following post:
LINQ - SQL Script Changing based on JOIN Order
If you already know that you would run the same query multi times, you can just cache it somehow to reduce the server's workload.
var mycatch = resultChildrenTC.Where(x => x.parentId != 0 && x.clientId == this.clientId).ToList();
var result1 = mycache .Where(filterExpression1)
var result2 = mycache .Where(filterExpression2)
var result3 = mycache .Where(filterExpression2)
I want to create this SQL query:
SELECT
a.[Seat],
b.[PlayerId],
b.[UserName],
b.[NickName],
COUNT(c.PlayerId) AS Trophy
FROM [dbo].[tbl_PlayerTableSeat] AS a
INNER JOIN [dbo].[tbl_Player] AS b ON a.[PlayerId] = b.[PlayerId]
INNER JOIN [dbo].[tbl_GameVirtualTable] AS d ON d.GameVirtualTableId = a.GameVirtualTableId
LEFT OUTER JOIN [dbo].[tbl_PlayerTableWinning] AS c ON a.[PlayerId] = c.[PlayerId] AND c.GameTableId = d.GameTableId
WHERE a.GameVirtualTableId = 36
GROUP BY a.[Seat], b.[PlayerId], b.[UserName], b.[NickName]
I have this Linq
var virtualTableSeatList = (from s in db.PlayerTableSeat
join p in db.Player on s.PlayerId equals p.PlayerId
join v in db.GameVirtualTable on s.GameVirtualTableId equals v.GameVirtualTableId
join w in db.PlayerTableWinning on new { X1 = s.PlayerId, X2 = v.GameTableId } equals new { X1 = w.PlayerId, X2 = w.GameTableId } into gj
from g in gj.DefaultIfEmpty()
where s.GameVirtualTableId == virtualGameTableId
group new { p, s } by new { p.PlayerId, s.Seat, p.NickName, p.UserName } into grp
select new VirtualTableSeatDto
{
PlayerId = grp.Key.PlayerId,
Seat = grp.Key.Seat,
NickName = grp.Key.NickName,
UserName = grp.Key.UserName,
Trophy = grp.Count()
}
).ToList();
From SQL Profiler, the Linq generates this SQL query:
exec sp_executesql N'SELECT
[GroupBy1].[K2] AS [PlayerId],
CAST( [GroupBy1].[K1] AS int) AS [C1],
[GroupBy1].[K4] AS [NickName],
[GroupBy1].[K3] AS [UserName],
[GroupBy1].[A1] AS [C2]
FROM ( SELECT
[Extent1].[Seat] AS [K1],
[Extent2].[PlayerId] AS [K2],
[Extent2].[UserName] AS [K3],
[Extent2].[NickName] AS [K4],
COUNT(1) AS [A1]
FROM [dbo].[tbl_PlayerTableSeat] AS [Extent1]
INNER JOIN [dbo].[tbl_Player] AS [Extent2] ON [Extent1].[PlayerId] = [Extent2].[PlayerId]
INNER JOIN [dbo].[tbl_GameVirtualTable] AS [Extent3] ON [Extent1].[GameVirtualTableId] = [Extent3].[GameVirtualTableId]
LEFT OUTER JOIN [dbo].[tbl_PlayerTableWinning] AS [Extent4] ON ([Extent1].[PlayerId] = [Extent4].[PlayerId]) AND ([Extent3].[GameTableId] = [Extent4].[GameTableId])
WHERE [Extent1].[GameVirtualTableId] = #p__linq__0
GROUP BY [Extent1].[Seat], [Extent2].[PlayerId], [Extent2].[UserName], [Extent2].[NickName]
) AS [GroupBy1]',N'#p__linq__0 int',#p__linq__0=36
I want to change COUNT(1) AS [A1] to COUNT([Extent4].[PlayerId]) AS [A1]
so it can return correct data.
I have no idea how to change the LinQ
Trophy = grp.Count()
so that it can count PlayerId of PlayerTableWinning instead of COUNT(1)
Updated: #Ivan Stoev
By adding the g into the group.
group new { p, s, g }
And sum the group
Trophy = grp.Sum(item => item.w != null ? 1 : 0)
It return the correct answer. However, it is using SUM instead of count. The SQL query generated is as below:
exec sp_executesql N'SELECT
[GroupBy1].[K2] AS [PlayerId],
CAST( [GroupBy1].[K1] AS int) AS [C1],
[GroupBy1].[K4] AS [NickName],
[GroupBy1].[K3] AS [UserName],
[GroupBy1].[A1] AS [C2]
FROM ( SELECT
[Filter1].[K1] AS [K1],
[Filter1].[K2] AS [K2],
[Filter1].[K3] AS [K3],
[Filter1].[K4] AS [K4],
SUM([Filter1].[A1]) AS [A1]
FROM ( SELECT
[Extent1].[Seat] AS [K1],
[Extent2].[PlayerId] AS [K2],
[Extent2].[UserName] AS [K3],
[Extent2].[NickName] AS [K4],
CASE WHEN ( NOT (([Extent4].[GameTableId] IS NULL) AND ([Extent4].[PlayerId] IS NULL) AND ([Extent4].[GameRoundId] IS NULL))) THEN 1 ELSE 0 END AS [A1]
FROM [dbo].[tbl_PlayerTableSeat] AS [Extent1]
INNER JOIN [dbo].[tbl_Player] AS [Extent2] ON [Extent1].[PlayerId] = [Extent2].[PlayerId]
INNER JOIN [dbo].[tbl_GameVirtualTable] AS [Extent3] ON [Extent1].[GameVirtualTableId] = [Extent3].[GameVirtualTableId]
LEFT OUTER JOIN [dbo].[tbl_PlayerTableWinning] AS [Extent4] ON ([Extent1].[PlayerId] = [Extent4].[PlayerId]) AND ([Extent3].[GameTableId] = [Extent4].[GameTableId])
WHERE [Extent1].[GameVirtualTableId] = #p__linq__0
) AS [Filter1]
GROUP BY [K1], [K2], [K3], [K4]
) AS [GroupBy1]',N'#p__linq__0 int',#p__linq__0=36
The only (but significant) difference between SQL COUNT(field) and COUNT(1) is that the former is excluding the NULL values, which when applied to the normally required field from the right side of a left outer join like in your case produces a different result when there are no matching records - the former returns 0 while the latter returns 1.
The "natural" LINQ equivalent would be Count(field != null), but that unfortunately is translated to a quite different SQL by the current EF query provider. So in such cases I personally use the closer equivalent expression Sum(field != null ? 1 : 0) which produces a much better SQL.
In order to apply the above to your query, you'll need an access to w inside the grouping, so change
group new { p, s }
to
group new { p, s, w }
and then use
Trophy = grp.Sum(item => item.w != null ? 1 : 0)
I have a LINQ query that gets data via Entity Framework Code First from an SQL database. This works, but it works very very slow.
This is the original query:
var tmpResult = from mdv in allMetaDataValues
where mdv.Metadata.InputType == MetadataInputType.String && mdv.Metadata.ShowInFilter && !mdv.Metadata.IsHidden && !string.IsNullOrEmpty(mdv.ValueString)
group mdv by new
{
mdv.ValueString,
mdv.Metadata
} into g
let first = g.FirstOrDefault()
select new
{
MetadataTitle = g.Key.Metadata.Title,
MetadataID = g.Key.Metadata.ID,
CollectionColor = g.Key.Metadata.Collection.Color,
CollectionID = g.Key.Metadata.Collection.ID,
MetadataValueCount = 0,
MetadataValueTitle = g.Key.ValueString,
MetadataValueID = first.ID
};
This is the generated SQL from the original query:
{SELECT
0 AS [C1],
[Project4].[Title] AS [Title],
[Project4].[ID] AS [ID],
[Extent9].[Color] AS [Color],
[Project4].[Collection_ID] AS [Collection_ID],
[Project4].[ValueString] AS [ValueString],
[Project4].[C1] AS [C2]
FROM (SELECT
[Project2].[ValueString] AS [ValueString],
[Project2].[ID] AS [ID],
[Project2].[Title] AS [Title],
[Project2].[Collection_ID] AS [Collection_ID],
(SELECT TOP (1)
[Filter4].[ID1] AS [ID]
FROM ( SELECT [Extent6].[ID] AS [ID1], [Extent6].[ValueString] AS [ValueString], [Extent7].[Collection_ID] AS [Collection_ID1], [Extent8].[ID] AS [ID2], [Extent8].[InputType] AS [InputType], [Extent8].[ShowInFilter] AS [ShowInFilter], [Extent8].[IsHidden] AS [IsHidden1]
FROM [dbo].[MetadataValue] AS [Extent6]
LEFT OUTER JOIN [dbo].[Media] AS [Extent7] ON [Extent6].[Media_ID] = [Extent7].[ID]
INNER JOIN [dbo].[Metadata] AS [Extent8] ON [Extent6].[Metadata_ID] = [Extent8].[ID]
WHERE ( NOT (([Extent6].[ValueString] IS NULL) OR (( CAST(LEN([Extent6].[ValueString]) AS int)) = 0))) AND ([Extent7].[IsHidden] <> cast(1 as bit))
) AS [Filter4]
WHERE (2 = CAST( [Filter4].[InputType] AS int)) AND ([Filter4].[ShowInFilter] = 1) AND ([Filter4].[IsHidden1] <> cast(1 as bit)) AND ([Filter4].[Collection_ID1] = #p__linq__0) AND (([Project2].[ValueString] = [Filter4].[ValueString]) OR (([Project2].[ValueString] IS NULL) AND ([Filter4].[ValueString] IS NULL))) AND (([Project2].[ID] = [Filter4].[ID2]) OR (1 = 0))) AS [C1]
FROM ( SELECT
[Distinct1].[ValueString] AS [ValueString],
[Distinct1].[ID] AS [ID],
[Distinct1].[Title] AS [Title],
[Distinct1].[Collection_ID] AS [Collection_ID]
FROM ( SELECT DISTINCT
[Filter2].[ValueString] AS [ValueString],
[Filter2].[ID3] AS [ID],
[Filter2].[InputType1] AS [InputType],
[Filter2].[Title1] AS [Title],
[Filter2].[ShowInFilter1] AS [ShowInFilter],
[Filter2].[IsHidden2] AS [IsHidden],
[Filter2].[Collection_ID2] AS [Collection_ID]
FROM ( SELECT [Filter1].[ValueString], [Filter1].[Collection_ID3], [Filter1].[IsHidden3], [Filter1].[ID3], [Filter1].[InputType1], [Filter1].[Title1], [Filter1].[ShowInFilter1], [Filter1].[IsHidden2], [Filter1].[Collection_ID2]
FROM ( SELECT [Extent1].[ValueString] AS [ValueString], [Extent2].[Collection_ID] AS [Collection_ID3], [Extent4].[IsHidden] AS [IsHidden3], [Extent5].[ID] AS [ID3], [Extent5].[InputType] AS [InputType1], [Extent5].[Title] AS [Title1], [Extent5].[ShowInFilter] AS [ShowInFilter1], [Extent5].[IsHidden] AS [IsHidden2], [Extent5].[Collection_ID] AS [Collection_ID2]
FROM [dbo].[MetadataValue] AS [Extent1]
LEFT OUTER JOIN [dbo].[Media] AS [Extent2] ON [Extent1].[Media_ID] = [Extent2].[ID]
INNER JOIN [dbo].[Metadata] AS [Extent3] ON [Extent1].[Metadata_ID] = [Extent3].[ID]
LEFT OUTER JOIN [dbo].[Metadata] AS [Extent4] ON [Extent1].[Metadata_ID] = [Extent4].[ID]
LEFT OUTER JOIN [dbo].[Metadata] AS [Extent5] ON [Extent1].[Metadata_ID] = [Extent5].[ID]
WHERE ( NOT (([Extent1].[ValueString] IS NULL) OR (( CAST(LEN([Extent1].[ValueString]) AS int)) = 0))) AND ([Extent2].[IsHidden] <> cast(1 as bit)) AND (2 = CAST( [Extent3].[InputType] AS int)) AND ([Extent3].[ShowInFilter] = 1)
) AS [Filter1]
WHERE [Filter1].[IsHidden3] <> cast(1 as bit)
) AS [Filter2]
WHERE [Filter2].[Collection_ID3] = #p__linq__0
) AS [Distinct1]
) AS [Project2] ) AS [Project4]
LEFT OUTER JOIN [dbo].[Collection] AS [Extent9] ON [Project4].[Collection_ID] = [Extent9].[ID]}
If we remove the "let first = g.FirstOrDefault()" and change "MetadataValueID = first.ID" to "MetadataValueID = 0" so that we just have a fixed ID = 0 for testing purposes, then the data loads very fast and the generated query itself is half the size compared to the original
So it seems that this part is making the query very slow:
let first = g.FirstOrDefault()
...
MetadataValueID = first.ID
};
How can this be rewritten?
If I try to rewrite the code, it is still slow:
MetadataValueID = g.Select(x => x.ID).FirstOrDefault()
or
let first = g.Select(x => x.ID).FirstOrDefault()
...
MetadataValueID = first
};
Any suggestions?
Using EF I have allways felt that it has problems efficiently translating stuff like g.Key.Metadata.Collection, so I try to join more explicitly and to include only fields, that are neccessary for your result. You can use include instead of join using repository pattern.
Then your query would look like this:
from mdv in allMetaDataValues.Include("Metadata").Include("Metadata.Collection")
where mdv.Metadata.InputType == MetadataInputType.String &&
mdv.Metadata.ShowInFilter &&
!mdv.Metadata.IsHidden &&
!string.IsNullOrEmpty(mdv.ValueString)
group mdv by new
{
MetadataID = mdv.Metadata.ID,
CollectionID = mdv.Metadata.Collection.ID,
mdv.Metadata.Title,
mdv.Metadata.Collection.Color,
mdv.ValueString
} into g
let first = g.FirstOrDefault().ID
select new
{
MetadataTitle = g.Key.Title,
MetadataID = g.Key.MetadataID,
CollectionColor = g.Key.Color,
CollectionID = g.Key.CollectionID,
MetadataValueCount = 0,
MetadataValueTitle = g.Key.ValueString,
MetadataValueID = first
}
Good tool for playing with linq is LinqPad.
The problem is also that:
let first = g.FirstOrDefault().ID
cannot be easily translated to SQL see this answer. But this rewrite simplifies the underlying query for it at least. It remains to me unclear, why you need first ID from a set without using orderby.
It could be rewriten like this:
let first = (from f in allMetaDataValues
where f.Metadata.ID == g.Key.MetadataID &&
f.ValuesString == g.Key.ValuesString select f.ID)
.FirstOrDefault()
This way you do not let EF write the query for you and you can specify exactly how to do the select.
To speed up the query you can also consider adding indexes to database according to the generated query - namely index using both colums used in where clause of this let first query.
Try the following solution.
Replace FirstOrDefault() with .Take(1). FirstOrDefault() is not lazy loaded.
var tmpResult = from mdv in allMetaDataValues
where mdv.Metadata.InputType == MetadataInputType.String && mdv.Metadata.ShowInFilter && !mdv.Metadata.IsHidden && !string.IsNullOrEmpty(mdv.ValueString)
group mdv by new
{
mdv.ValueString,
mdv.Metadata
} into g
let first = g.Take(1)
select new
{
MetadataTitle = g.Key.Metadata.Title,
MetadataID = g.Key.Metadata.ID,
CollectionColor = g.Key.Metadata.Collection.Color,
CollectionID = g.Key.Metadata.Collection.ID,
MetadataValueCount = 0,
MetadataValueTitle = g.Key.ValueString,
MetadataValueID = first.ID
};
I want to ask why Linq to Entities return more join than necessary, and way to improve them.
Here is my code:
var items = dc.WHItems
.Select(c => new testModel()
{
ID = c.ID,
Name = c.Name,
Code = c.Code,
Discontinued = c.Discontinued,
Type = c.WHItemType.Name,
Category = c.WHItemType.WHItemCategory.Name
})
.Where(c => c.Discontinued == false)
.Take(10);
Resulting SQL took from SQL Profiler
SELECT TOP (10)
[Extent1].[ID] AS [ID],
[Extent1].[Name] AS [Name],
[Extent1].[Code] AS [Code],
[Extent1].[Discontinued] AS [Discontinued],
[Extent2].[Name] AS [Name1],
[Extent4].[Name] AS [Name2]
FROM [dbo].[WHItems] AS [Extent1]
INNER JOIN [dbo].[WHItemTypes] AS [Extent2] ON [Extent1].[WHItemTypeID] = [Extent2].[ID]
LEFT OUTER JOIN [dbo].[WHItemTypes] AS [Extent3] ON [Extent1].[WHItemTypeID] = [Extent3].[ID]
LEFT OUTER JOIN [dbo].[WHItemCategories] AS [Extent4] ON [Extent3].[WHItemCategoryID] = [Extent4].[ID]
WHERE 0 = [Extent1].[Discontinued]
However using LINQPad4, resulting query satisfied me
SELECT TOP (10) [t0].[ID], [t0].[Name], [t0].[Code], [t0].[Discontinued], [t1].[Name] AS [Type], [t2].[Name] AS [Category]
FROM [WHItems] AS [t0]
INNER JOIN [WHItemTypes] AS [t1] ON [t1].[ID] = [t0].[WHItemTypeID]
INNER JOIN [WHItemCategories] AS [t2] ON [t2].[ID] = [t1].[WHItemCategoryID]
WHERE NOT ([t0].[Discontinued] = 1)
Here is another code, where I try to get items from inventories. Table WHInventories use a composite key on WHItemID and WHID, which I think explained the choice of using 'Inner Join' instead of 'Left Join' as above.
var items = dc.WHInventories
.Select(c => new testModel()
{
ID = c.WHItemID,
Name = c.WHItem.Name,
Code = c.WHItem.Code,
Discontinued = c.WHItem.Discontinued
})
.Where(c => c.Discontinued == false)
.Take(10);
Resulting queries in SQL Profiler show additional left outer join. Why would Column Code has to be retrieved from Extent3? while Column Name and Discontinued using the same Extent2
SELECT TOP (10)
[Extent1].[WHItemID] AS [WHItemID],
[Extent2].[Name] AS [Name],
[Extent3].[Code] AS [Code],
[Extent2].[Discontinued] AS [Discontinued]
FROM [dbo].[WHInventories] AS [Extent1]
INNER JOIN [dbo].[WHItems] AS [Extent2] ON [Extent1].[WHItemID] = [Extent2].[ID]
LEFT OUTER JOIN [dbo].[WHItems] AS [Extent3] ON [Extent1].[WHItemID] = [Extent3].[ID]
WHERE 0 = [Extent2].[Discontinued]
If I try explicit join which is means loosing the advantage of having 'Navigation properties'.
var items = dc.WHInventories
.Join(dc.WHItems, c => c.WHItemID, d => d.ID, (c, d) => new { c, d })
.Select(c => new testModel()
{
ID = c.c.WHItemID,
Name = c.d.Name,
Code = c.d.Code,
Discontinued = c.d.Discontinued
})
.Where(c => c.Discontinued == false)
.Take(10);
Resulting Query in SQL Profiler show what I expected. Column Code now using Extent2
SELECT TOP (10)
[Extent1].[WHItemID] AS [WHItemID],
[Extent2].[Name] AS [Name],
[Extent2].[Code] AS [Code],
[Extent2].[Discontinued] AS [Discontinued]
FROM [dbo].[WHInventories] AS [Extent1]
INNER JOIN [dbo].[WHItems] AS [Extent2] ON [Extent1].[WHItemID] = [Extent2].[ID]
WHERE 0 = [Extent2].[Discontinued]
Have tried all the various combinations from quite a few searches, and it still doesn't seem to work. I have the following query:
var r = from a in ctx.DGApprovedLinks
join h in ctx.DGHosts on a.HostID equals h.ID
join c in ctx.DGConfigs on SqlFunctions.StringConvert((double)a.ResponseCode) equals c.SubType
where c.Type == "HTTP Status"
select new
{
a.ID,
a.HostID,
h.URL,
a.SourceURL,
a.TargetURL,
c.Value,
a.ExtFlag
};
but it fails to return any results. I get the following in SQL Profiler:
SELECT
1 AS [C1],
[Extent1].[ID] AS [ID],
[Extent1].[HostID] AS [HostID],
[Extent2].[URL] AS [URL],
[Extent1].[SourceURL] AS [SourceURL],
[Extent1].[TargetURL] AS [TargetURL],
[Extent3].[Value] AS [Value],
[Extent1].[ExtFlag] AS [ExtFlag]
FROM [dbo].[DGApprovedLink] AS [Extent1]
INNER JOIN [dbo].[DGHost] AS [Extent2] ON [Extent1].[HostID] = [Extent2].[ID]
INNER JOIN [dbo].[DGConfig] AS [Extent3] ON ((STR( CAST( [Extent1].[ResponseCode] AS float))) = [Extent3].[SubType]) OR ((STR( CAST( [Extent1].[ResponseCode] AS float)) IS NULL) AND ([Extent3].[SubType] IS NULL))
WHERE N'HTTP Status' = [Extent3].[Type]
Any assistance would be much appreciated.