I'm playing around EntityFrameworkCore with WebAPI while building voting app exercise.
I want to make the code in async way where is possible.
So should I have to use the nested query in async way somehow(// Problem 1, // Problem 2)?
/* The target of the question - the query*/
var pollResults =
await _context.Polls
.Select(poll => new PollDto
{
Id = poll.Id,
Question = poll.Question,
CreatedAt = poll.CreatedAt,
Options = poll.Options
.Select(option => new OptionDto
{
Id = option.Id,
Value = option.Value,
VotesCount = option.Votes.Count() // Problem 1
})
.ToList(), // Problem 2
LastVotedAt = _context.PollVotes.Where(vote=>vote.PollId == poll.Id).Select(vote => vote.VoteDate).SingleOrDefault()
})
.ToListAsync();
/* Domain classes */
public class Poll
{
public int Id { get; set; }
public ICollection<PollOption> Options { get; set; } = new List<PollOption>();
public ICollection<PollVote> Votes { get; set; } = new List<PollVote>();
}
public class PollOption
{
public int Id { get; set; }
public string Value { get; set; }
public int PollId { get; set; }
public Poll Poll { get; set; }
public ICollection<PollVote> Votes { get; set; } = new List<PollVote>();
}
public class PollVote
{
public int Id { get; set; }
public int PollId { get; set; }
public Poll Poll { get; set; }
public int OptionId { get; set; }
public PollOption Option { get; set; }
public DateTime VoteDate { get; set; }
}
/* Dto classes */
public class PollDto
{
public int Id { get; set; }
public string Question { get; set; }
public ICollection<OptionDto> Options { get; set; } = new List<OptionDto>();
public DateTime LastVotedAt { get; set; }
}
public class OptionDto
{
public int Id { get; set; }
public string Value { get; set; }
public int VotesCount { get; set; }
}
So in not nested queries Count and SingleOrDefault would make request to the database and it should be executed in async way. But in my case the whole query is a single request.
Should I have to modify something to done the methods Count and SingleOrDefault in async way ? Or calling ToListAsync at end is enough?
I believe the answer is that 1 request to the database goes in 1 async call. But I didn't find any solution in the internet.
ToListAsync() at the end is enough. Expressions inside the query are used by EF to compose the query. They are not "executed" as SQL like they would have been as stand-alone statements against the DbSets.
For instance when I run something similar:
var parents = await context.Parents
.Select(x => new
{
x.ParentId,
x.Name,
Children = x.Children.Select(c => new { c.ChildId, c.Name }).ToList(),
ChildCount = x.Children.Count()
}).ToListAsync();
in a test and set a breakpoint with a profiler running. The statement produces a single SQL statement:
SELECT
[Project2].[ParentId] AS [ParentId],
[Project2].[Name] AS [Name],
[Project2].[C2] AS [C1],
[Project2].[C1] AS [C2],
[Project2].[ChildId] AS [ChildId],
[Project2].[Name1] AS [Name1]
FROM ( SELECT
[Project1].[ParentId] AS [ParentId],
[Project1].[Name] AS [Name],
[Extent3].[ChildId] AS [ChildId],
[Extent3].[Name] AS [Name1],
CASE WHEN ([Extent3].[ChildId] IS NULL) THEN CAST(NULL AS int) ELSE 1 END AS [C1],
[Project1].[C1] AS [C2]
FROM (SELECT
[Extent1].[ParentId] AS [ParentId],
[Extent1].[Name] AS [Name],
(SELECT
COUNT(1) AS [A1]
FROM [dbo].[Children] AS [Extent2]
WHERE [Extent1].[ParentId] = [Extent2].[ParentId]) AS [C1]
FROM [dbo].[Parents] AS [Extent1] ) AS [Project1]
LEFT OUTER JOIN [dbo].[Children] AS [Extent3] ON [Project1].[ParentId] = [Extent3].[ParentId]
) AS [Project2]
ORDER BY [Project2].[ParentId] ASC, [Project2].[C1] ASC
go
Not 3 queries that you might be concerned would block. This was when looking at the navigation properties for related records.
The bigger question I saw when looking at your example to double-check was this line:
LastVotedAt = _context.PollVotes.Where(vote=>vote.PollId == poll.Id).Select(vote => vote.VoteDate).SingleOrDefault()
As this would go back directly to the Context rather than access votes through a collection on the Poll. But I tried that as well and it too still resulted in a single query.
Children = x.Children.Select(c => new { c.ChildId, c.Name }).ToList(),
ChildCount = x.Children.Count(),
YoungestChild = context.Children.OrderBy(c=>c.BirthDate).Where(c=>c.ParentId == x.ParentId).FirstOrDefault()
In my test example I go back to the context to retrieve the Youngest child for the parent record rather than the Children navigation property. In this case it still executes as 1 query.
For questions like this I definitely recommend creating an EF experimentation sandbox project with a local database, then leverage and SQL profiler tool to watch the SQL statements being produced and when they are executed. Async is useful for queries that are expected to take a while to run, but should be used sparingly as they can lower the overall performance of the queries being run when used on every trivial query.
Related
I have the following structure: Training has many Module has many Phase has many Question.
I use the following query to get the above
Context.Trainings
.Include(x => x.Modules)
.ThenInclude(x => x.Phases)
.ThenInclude(y => y.Questions)
Question also has many Comment but that relationship is not defined as navigation property because Comment can have different type of patents. So Comment just has a ParentId that is sometimes Question and sometimes other things.
My question is how do I modify the above query to, for every Question, count the child Comment from the Context.Comments and assign it to Question.CommentCount? Kind of like a manual Include
In my head it's something like this
Context.Trainings
.Include(x => x.Modules)
.ThenInclude(x => x.Phases)
.ThenInclude(y => y.Questions.Select(x=> new Question.Question {
Name = x.Name,
Description = x.Description,
CommentCount = Context.Comments.Where(y=>y.ParentId == x.Id)
}));
But it seems you can't put projections in Include and I don't know how to think about this in another way.
With the entities set up such as ...
public class Training
{
public int Id { get; set; }
public ICollection<Module> Modules { get; set; }
}
public class Module
{
public int Id { get; set; }
public ICollection<Phase> Phases { get; set; }
}
public class Phase
{
public int Id { get; set; }
public ICollection<Question> Questions { get; set; }
}
public class Question
{
public int Id { get; set; }
[NotMapped]
public int CommentCount { get; set; }
}
public class Comment
{
public int Id { get; set; }
public int ParentId { get; set; }
}
// DbContext
public DbSet<Training> Trainings { get; set; }
public DbSet<Module> Modules { get; set; }
public DbSet<Phase> Phases { get; set; }
public DbSet<Question> Questions { get; set; }
public DbSet<Comment> Comments { get; set; }
... it can be done in a single query, but it's quite messy.
// query all nested navigations using projections with extra data
var projected = await context.Trainings
.Select(t =>
new
{
Training = t,
Modules = t.Modules.Select(m =>
new
{
Module = m,
Phases = m.Phases.Select(p =>
new
{
Phase = p,
Questions = p.Questions.Select(q =>
new
{
Question = q,
CommentCount = context.Comments.Count(c => c.ParentId == q.Id)
}
)
}
)
}
)
}
)
.ToListAsync();
// fixup by setting comment count from dto projection to "real" tracked entity
foreach (var q in projected.SelectMany(t => t.Modules).SelectMany(m => m.Phases).SelectMany(m => m.Questions))
{
q.Question.CommentCount = q.CommentCount;
}
// thanks to ef core entity tracker this will still work
var trainings = projected.Select(p => p.Training);
var totalCommentCount = trainings.SelectMany(t => t.Modules).SelectMany(m => m.Phases).SelectMany(p => p.Questions).Sum(q => q.CommentCount);
final query
SELECT [t].[Id], [t0].[Id], [t0].[TrainingId], [t0].[Id0], [t0].[ModuleId], [t0].[Id00], [t0].[PhaseId], [t0].[c]
FROM [Trainings] AS [t]
LEFT JOIN (
SELECT [m].[Id], [m].[TrainingId], [t1].[Id] AS [Id0], [t1].[ModuleId], [t1].[Id0] AS [Id00], [t1].[PhaseId], [t1].[c]
FROM [Modules] AS [m]
LEFT JOIN (
SELECT [p].[Id], [p].[ModuleId], [q].[Id] AS [Id0], [q].[PhaseId], (
SELECT COUNT(*)
FROM [Comments] AS [c]
WHERE [c].[ParentId] = [q].[Id]) AS [c]
FROM [Phases] AS [p]
LEFT JOIN [Questions] AS [q] ON [p].[Id] = [q].[PhaseId]
) AS [t1] ON [m].[Id] = [t1].[ModuleId]
) AS [t0] ON [t].[Id] = [t0].[TrainingId]
ORDER BY [t].[Id], [t0].[Id], [t0].[Id0]
As pointed out in comments, you could benefit from using TPH with real navigation collection back to comments from questions, and you should also probably use split query or multiple queries instead of joining it all up like this. But depending on use case, perhaps a single query might perform better for you.
I'm using Automapper version 10.0 with EF Core 5 and .NET 5.
It would seem that automapper causes a query that selects every property separately:
SELECT FALSE, a11."Avatar", ((
SELECT a0."Id"
FROM "AspNetUserRoles" AS a
INNER JOIN "AspNetRoles" AS a0 ON a."RoleId" = a0."Id"
WHERE a11."Id" = a."UserId"
ORDER BY a0."Order" DESC
LIMIT 1) IS NULL), (
SELECT a2."Color"
FROM "AspNetUserRoles" AS a1
INNER JOIN "AspNetRoles" AS a2 ON a1."RoleId" = a2."Id"
WHERE a11."Id" = a1."UserId"
ORDER BY a2."Order" DESC
LIMIT 1), (
SELECT a4."Id"
FROM "AspNetUserRoles" AS a3
INNER JOIN "AspNetRoles" AS a4 ON a3."RoleId" = a4."Id"
WHERE a11."Id" = a3."UserId"
ORDER BY a4."Order" DESC
LIMIT 1), (
SELECT a6."IsStaff"
FROM "AspNetUserRoles" AS a5
INNER JOIN "AspNetRoles" AS a6 ON a5."RoleId" = a6."Id"
WHERE a11."Id" = a5."UserId"
ORDER BY a6."Order" DESC
LIMIT 1), (
SELECT a8."Name"
FROM "AspNetUserRoles" AS a7
INNER JOIN "AspNetRoles" AS a8 ON a7."RoleId" = a8."Id"
WHERE a11."Id" = a7."UserId"
ORDER BY a8."Order" DESC
LIMIT 1), COALESCE((
SELECT a10."Order"
FROM "AspNetUserRoles" AS a9
INNER JOIN "AspNetRoles" AS a10 ON a9."RoleId" = a10."Id"
WHERE a11."Id" = a9."UserId"
ORDER BY a10."Order" DESC
LIMIT 1), 0), a11."Title", a11."UserName", t."Body", t."CommentsThreadId", t."DateTime", t."Id"
FROM (
SELECT c."Id", c."AuthorId", c."Body", c."CommentsThreadId", c."DateTime"
FROM "Comments" AS c
WHERE c."CommentsThreadId" = #__threadId_0
ORDER BY c."DateTime" DESC
LIMIT #__p_1 OFFSET 0
) AS t
INNER JOIN "AspNetUsers" AS a11 ON t."AuthorId" = a11."Id"
ORDER BY t."DateTime" DESC
As you can see, pretty much every single property of RoleDto is being SELECTed separately, instead of being selected just once and having its columns mapped using the AS keyword.
Mappings are done to the following DTOs, property names being true to source entities:
public class UserSimpleDto
{
public string UserName { get; set; }
public string Avatar { get; set; }
public string Title { get; set; }
public RoleDto Role { get; set; }
}
It seems the above and the below are the main culprits. OgmaUser, the source entity for the mapping of the above, contains a list of user roles, of which only one should be present in the target DTO.
OgmaUser.Roles is mapped using EF Core 5's many-to-many setup, but I did try to use an explicit join entity and the result was the exact same.
public class RoleDto
{
public long Id { get; set; }
public string Name { get; set; }
public string? Color { get; set; }
public bool IsStaff { get; set; }
public int Order { get; set; }
}
As you can see, each property of the above is being SELECT ... AS ... separately.
public class CommentDto
{
public long Id { get; set; }
public long CommentsThreadId { get; set; }
public UserSimpleDto Author { get; set; }
public DateTime DateTime { get; set; }
public string Body { get; set; }
}
I doubt this particular DTO has something to do with the result, since the issue occurs with other DTOs that contain UserSimpleDto, but I thought I should include it to present the fullest possible picture.
And the mappings are as follows:
CreateMap<OgmaUser, UserSimpleDto>()
.ForMember(
usd => usd.Role,
opts => opts.MapFrom(u => u.Roles.OrderByDescending(r => r.Order).FirstOrDefault())
);
CreateMap<OgmaRole, RoleDto>();
CreateMap<Comment, CommentDto>()
.ForMember(
cd => cd.Body,
opts => opts.MapFrom(c => Markdown.ToHtml(c.Body, null))
);
The query is generated from the following method:
public async Task<IEnumerable<CommentDto>> GetPaginated(long threadId, int page)
{
return await _context.Comments
.Where(c => c.CommentsThreadId == threadId)
.OrderByDescending(c => c.DateTime)
.ProjectTo<CommentDto>(_mapper.ConfigurationProvider)
.Skip(Math.Max(0, page - 1) * _config.CommentsPerPage)
.Take(_config.CommentsPerPage);
.AsNoTracking()
.ToListAsync();
}
Stack complains that my question is mostly code, and I suppose I do understand that. I've seen myself many people just posting their code, expecting to have it fixed, or even posting their homework and asking for it to be solved.
In this case, however, I believe that the code speaks louder than whatever explanation I could provide, so, true, non-code parts of this question are few and far between. That's the reason why I'm even writing those last paragraph, in fact, as SO will not let me submit my question otherwise.
I believe the best answer is to change UserSimpleDto to
public class UserSimpleDto
{
public string UserName { get; set; }
public string Avatar { get; set; }
public string Title { get; set; }
public IEnumerable<RoleDto> Roles { get; set; }
}
to load all user roles – there aren't all that many – and simply getting only the first one when it's just the first one that's needed. A negligible overhead in the application layer, so to speak, but the resulting query is much simpler which should negate that overhead.
Project after ToListAsync(). Don't pass IQueryable to ProjectTo method.
public async Task<IEnumerable<CommentDto>> GetPaginated(long threadId, int page)
{
return (await _context.Comments
.Where(c => c.CommentsThreadId == threadId)
.OrderByDescending(c => c.DateTime)
.Skip(Math.Max(0, page - 1) * _config.CommentsPerPage)
.Take(_config.CommentsPerPage);
.AsNoTracking()
.ToListAsync())
.ProjectTo<CommentDto>(_mapper.ConfigurationProvider);
}
I have two data models:
public class SSC {
public int Id { get; set; }
public string Name { get; set; }
//other irrelevent properties removed for brevity
public virtual ICollection<SpecReference> SpecReferences { get; set; }
}
public class SpecReference {
public int Id { get; set; }
public string Name { get; set; }
//other irrelevent properties removed for brevity
public virtual ICollection<SSC> SSCs { get; set; }
}
I have defined a many-to-many relationship between the two in my data context as:
modelBuilder.Entity<SSC>().HasMany(a => a.SpecReferences)
.WithMany(b => b.SSCs)
.Map(c => {
c.MapRightKey("SpecReference_Id");
c.MapLeftKey("SSC_Id");
c.ToTable("SpecReferenceSSCs");
});
At one point in my site, I have an advanced search page from which users can search for SSCs by a combination of any field in the SSC or any navigation field attached to the SSC. Every search is working except when a user specifies a SpecReference. The query code (abbreviated to show only relevant code) is:
var query = _dbContext.SSCs.Where(x => !x.IsDeleted);
if (designSpecificationIds != null && designSpecificationIds.Any()) {
var designSpecificationIdsHash = new HashSet<int>(designSpecificationIds);
query = query.Where(x => x.SpecReferences.Any(s => designSpecificationIdsHash.Contains(s.Id)));
}
where designSpecificationIds is an array of integers passed from the web form.
When I run the application and step through the search code, the SQL generated from the query is:
SELECT
[Extent1].[Id] AS [Id],
[Extent1].[SSCId] AS [SSCId],
[Extent1].[Name] AS [Name],
[Extent1].[SystemFunction] AS [SystemFunction],
[Extent1].[Remarks] AS [Remarks],
[Extent1].[IsSystem] AS [IsSystem],
[Extent1].[IsGrouping] AS [IsGrouping],
[Extent1].[IsConfiguration] AS [IsConfiguration],
[Extent1].[FieldTag] AS [FieldTag],
[Extent1].[ParentId] AS [ParentId],
[Extent1].[IPlantParentId] AS [IPlantParentId],
[Extent1].[BuildingId] AS [BuildingId],
[Extent1].[IsOperable] AS [IsOperable],
[Extent1].[IsAvailable] AS [IsAvailable],
[Extent1].[DutyAreaId] AS [DutyAreaId],
[Extent1].[IsDeleted] AS [IsDeleted]
FROM [dbo].[SSCs] AS [Extent1]
WHERE ([Extent1].[IsDeleted] <> 1) AND ( EXISTS (SELECT
1 AS [C1]
FROM [dbo].[SpecReferenceSSCs] AS [Extent2]
WHERE ([Extent1].[Id] = [Extent2].[SSC_Id]) AND ([Extent2].[SpecReference_Id] IN (4))
))
If I run this SQL against the database, I get the one result that I expect. However, when I examine the Linq query and attempt to enumerate the results, I get "Enumeration yielded no results." How is it possible that the SQL generated by a Linq query is returning different results than the query itself?
Edit:
To return the values, I'm using ExpressMapper to convert the Model to a DTO. The conversion is simply:
return Mapper.Map<IEnumerable<SSC>, IEnumerable<SSCIndexDto>>(query);
The DTO is:
public class SSCIndexDto {
public int Id { get; set; }
public string Name { get; set; }
//Other properties omitted for brevity
public List<SpecReferenceIndexDto> SpecReferences { get; set; }
}
Again, the mapping appears to be working correctly for all other properties, including navigation properties that are written exactly like SpecReferences. It's the query itself that's not returning any data.
I have a CashFlowView:
CREATE VIEW [dbo].[CashFlowView]
AS
WITH CTE AS
(
SELECT
ROW_NUMBER() OVER (ORDER BY RateDate) AS ID
, SUM(CASE WHEN C.CurrencyName = 'Br' THEN T.AmountMoney ELSE 0 END) AS AmountBYR
, SUM(CASE WHEN C.CurrencyName = 'Usd' THEN T.AmountMoney ELSE 0 END) AS AmountUSD
, CR.RateDate AS [DATE]
FROM Transactions AS T
INNER JOIN Accounts AS A ON A.AccountID = T.CurrentAccountID
INNER JOIN Currencies AS C ON C.CurrencyID = A.CurrencyID
RIGHT OUTER JOIN CurrencyRates AS CR ON CR.RateDate = T.ExecutionDate
GROUP BY CR.RateDate
)
SELECT
ID
, A.AmountBYR
, (SELECT SUM(B.AmountBYR) FROM CTE B WHERE B.ID<=A.ID) AS BalanceBYR
, A.AmountUSD
, (SELECT SUM(B.AmountUSD) FROM CTE B WHERE B.ID<=A.ID) AS BalanceUSD
, [Date]
FROM CTE AS A
Then I've added the Entity:
public class CashFlowView
{
[Key]
public int ID { get; set; }
public decimal AmountBYR { get; set; }
public decimal BalanceBYR { get; set; }
public decimal AmountUSD { get; set; }
public decimal BalanceUSD { get; set; }
public DateTime Date { get; set; }
}
And, as I understand, I need to add this code to my context:
public DbSet<CashFlowView> CashFlowView { get; set; }
And now I wanna to use my View:
IList<CashFlowView> listView;
using (var _db = new EconomicAppContext())
{
listView = _db.CashFlowView.ToList();
}
But listView is empty. How I may create correct mapping to View (maybe using migration) and use it?
I did it. Try to combine this article
http://www.paragon-inc.com/resources/blogs-posts/a-certain-point-of-view-part-1-ef-code-first
And use Entity Framework Power Tools to find needed result. And check connection. I've got problems with perfomance, so use dispose method carefully.
I have a fairly simple (code first) model:
Employee
[Table("vEmployee")] //note v - it's a view
public class Employee
{
[Key]
public int EmployeeNumber { get; set; }
public string FirstName { get; set; }
public string LastName { get; set; }
}
EmployeeHolidayEntitlement
[Table("tblEmployeeHolidayEntitlement")]
public class EmployeeHolidayEntitlement
{
[Key]
public int EmployeeNumber { get; set; }
public virtual Employee Employee { get; set; }
public decimal StandardEntitlement { get; set; }
//.....omitted for brevity
}
Note that EmployeeHolidayEntitlement is mapped to a table, and Employee is mapped to a view
When building my context, I do:
(not sure if this is correct!)
modelBuilder.Entity<Employee>()
.HasOptional(x => x.HolidayEntitlement)
.WithRequired(x => x.Employee);
Now, when I query, like this:
var db = new ApiContext();
var result = db.Employees.ToList();
It's very slow.
If I look in SQL profiler, I can see that instead of one statement (joining vEmployee and tblEmployeeHolidayEntitlement) I get many statements executed (one per Employee record) - for example:
First, it selects from vEmployee
SELECT
[Extent1].[id] AS [EmployeeNumber],
[Extent1].[FirstName] AS [FirstName],
[Extent1].[LastName] AS [LastName],
FROM [dbo].[vEmployee] AS [Extent1]
then one of these for each record returned
exec sp_executesql N'SELECT
[Extent1].[EmployeeNumber] AS [EmployeeNumber],
[Extent1].[StandardEntitlement] AS [StandardEntitlement]
FROM [dbo].[tblEmployeeHolidayEntitlement] AS [Extent1]
WHERE [Extent1].[EmployeeNumber] = #EntityKeyValue1',N'#EntityKeyValue1 int',#EntityKeyValue1=175219
This doesn't seem right to me -
I would of thought it should be doing something more along the lines of a LEFT JOIN like
SELECT *
FROM [dbo].[vEmployee] employee
LEFT JOIN
[dbo].[tblEmployeeHolidayEntitlement employeeEntitlement
ON
employee.id = employeeEntitlement.employeenumber
You have to use the Include method, like db.Employees.Include(e => e.HolidayEntitlement).ToList(). If you don't and you access the property you'll trigger lazy loading. That's what's happening to you.
For more information check the documentation on loading. The short of it is that if it always joined your entire object graph it'd be unacceptably slow.