I'm trying to use Include() in a query but it doesn't work as I expect.
I have a table called Bes who has this columns:
Id
Name
Pesta
CeId
OpId
The CeId and OpId are foreign keys to this tables:
Ces:
Id
Name
Ops:
Id
Name
I want to run this query (which works but doesn't fill the Ces and Ops tables)
var bes = await _context.Bes
.Include(x => x.Ops)
.Include(x => x.Ces)
.Where(x => besWithXPs.Contains(x.Pesta))
.GroupBy(x => x.Pesta)
.Select(x => x.First())
.ToListAsync();
I try to use this query without the Select() and the Ops and Ces where fill but I only want one Bes per Pesta (that's why the GroupBy and Select)
Anyone knows what is happening?
Btw I'm using Entity Framework Core with .NET Core 2
I think you should add
[JsonIgnore]
public Ces Ce { get; set; }
[JsonIgnore]
public Ods Od { get; set; }
to your model in order to let include them and then do:
var bes = await _context.Bes.AsNotracking()
.Include(x => x.Ops)
.Include(x => x.Ces)
.Where(x => besWithXPs.Contains(x.Pesta))
.GroupBy(x => x.Pesta)
.FirstOrDefaultAsync();
1 - What AsNoTracking Does:
Entity Framework exposes a number of performance tuning options to help you optimise the performance of your applications. One of these tuning options is .AsNoTracking(). This optimisation allows you to tell Entity Framework not to track the results of a query. This means that Entity Framework performs no additional processing or storage of the entities which are returned by the query. However, it also means that you can't update these entities without reattaching them to the tracking graph.
2 - Change ".Select(x => x.First()).ToListAsync();" to ".FirstOrDefaultAsync();". Use .Select only if you want to get some of the properties or if you need a new class filled with the data that you receive.
Related
I'm working with C# EF 7.0.2, Z.EntityFrameworkPlus and SQL Server. I have a database that could be grown a lot and I want to make it scalable; To do this, I figure out to work with "chunks" to avoid timeout issues.
I have 4 entities
Products
Attributes
ProductAttribute (n:m)
RequestedProducts (List of user's requests to process products)
I was trying to process the products joining the entities Products and RequestedProducts, querying them by chunks (I'm using Take to avoid use OrderBy function, cuz in the local test that increment a lot the query result time) including ProductAttribute and Attributes entities. Then, when the parallel process ends, I'm using BatchUpdate to change de "Finished" flag so that next query doesn't include processed products (I'd understood that IQueryable doesn't execute the query until is necesary)
The first run of the query it works fine, but the next iteration the entities result doesn't include the ProductAttribute entities
IQueryable<Product> products = this.context.Products
.Join(
this.context.RequestedProducts.Where(x=>!x.Finished),
p => p.Id,
pp => pp.ProductId,
(p, pp) => p)
.Include(x => x.Attributes)
.ThenInclude(x => x.Attribute)
.Include(x => x.Classifications);
IQueryable<StepProduct> queryToRun = products
.Take(productLimit);
while(queryToRun){
queryToRun
.AsParallel()
.WithDegreeOfParallelism(maxDegree)
.ForAll(product =>
{
//Here is the process products code
}
this.context.RequestedProducts
.Where(g => queryToRun.Select(q => q.ProductId).Contains(g.Id))
.Update(new RequestedProducts(){ Finished=true });
queryToRun = products
.Take(productLimit);
}
I tried to asign the full IQueryable to the queryToRun var, then I tried put the Include sentences in queryToRun and not in products, but it doesn't work. The first time all of the products entities have Attributes, but the next chunk, Attributes ever was empty.
I use Entity Framework 6 and i currently have a query with many includes which loads about 1200 entities into the dbContext. Loading the entities seems to be quite slow as the query takes almost a minute. Is there anything I can do about the performance? I have 4 such queries that take 2.5 minutes to load?
LazyLoading is enabled but for performance reasons i preload the entities.
var report = DbContext.REPORT.Single(r => r.ID == reportId);
//this query takes a bit less than 1 minute
DbContext.REPORT_ELEMENT
.Include(re => re.LAYOUT)
.Include(re => re.PAGEMASTER)
.Include(re => re.REPORT_ELEMENTS)
.Include(re => re.SUBTITLE_CONTENT)
.Include(re => re.REPORT_ELEMENT_NOTE)
.Include("SUBTITLE_CONTENT.CONTENT_ELEMENT.LANGUAGE")
.Include("TITLE_CONTENT.CONTENT_ELEMENT.LANGUAGE")
.Where(re => re.REPORT_ID == report.ID)
.Load();
Performance suggestions:
Prevent tracking. Query in read-only mode.
Prevent getting too much data in one query. Try to page it.
Prevent include. The query has too many Includes which makes performance bad.
Prevent tracking
Consider adding AsNoTracking for this makes query performance better.
Reference: https://learn.microsoft.com/en-us/ef/core/querying/tracking#no-tracking-queries
Only get the data you need
And the key reason for your slow query is it outputs too much data. Consider adding: Take(200), Skip() to take only the data you need or the current page requires. Use a pager to generate the report. This might helps a lot.
Prevent Include
Include generates SQL to select multiple tables. Which greatly increased complexity. You can only select the data you need and prevent writing the Include function.
For example, if you only want to get the last ball in the box, consider writing like this:
public class Box
{
public int Id { get; set; }
public IEnumerable<Ball> Balls { get; set; }
}
public class Ball
{
public int Id { get; set; }
public int BoxId { get; set; }
public Box Box { get; set; }
}
var boxes = await Boxes
// DO NOT Call Include(t => t.Balls) here!
.Where(somecondition)
.Select(t => new Box(){
Id = t.Id,
Balls = t.Balls.OrderByDescending(x => x.CreationTime)
.Take(1) // Only get what you need
})
.ToListAsync()
Also when we use Select we can remove .Include because it won’t have any effect here.
Disclaimer: I'm the owner of the project Entity Framework Plus
The Query IncludeOptimized feature allows to filter with include and optimize the query performance at the same time.
It usually improves the performance (split the query in smaller queries)
DbContext.REPORT_ELEMENT
.IncludeOptimized(re => re.LAYOUT)
.IncludeOptimized(re => re.PAGEMASTER)
.IncludeOptimized(re => re.REPORT_ELEMENTS)
.IncludeOptimized(re => re.SUBTITLE_CONTENT)
.IncludeOptimized(re => re.REPORT_ELEMENT_NOTE)
.IncludeOptimized(re => re.SUBTITLE_CONTENT.Select(sc => sc.CONTENT_ELEMENT)) // SelectMany?
.IncludeOptimized(re => re.SUBTITLE_CONTENT.Select(sc => sc.CONTENT_ELEMENT).Select(ce => ce.LANGUAGE)) // SelectMany?
.IncludeOptimized(re => re.TITLE_CONTENT)
.IncludeOptimized(re => re.SUBTITLE_CONTENT.Select(sc => sc.CONTENT_ELEMENT)) // SelectMany?
.IncludeOptimized(re => re.SUBTITLE_CONTENT.Select(sc => sc.CONTENT_ELEMENT).Select(ce => ce.LANGUAGE)) // SelectMany?
.Where(re => re.REPORT_ID == report.ID)
.Load();
Documentation: EF+ Query IncludeOptimized
Additionally to the advices from Anduin, I would like to add the advice to split up the Includes() in several distinct queries. EF will be able to keep track of the references between the entities within the same DBContext. As a general rule of thumb - do not use more then three Includes() in the same query. Also ensure, that you have an index in the DB for every resulting JOIN.
To be able to do so, you must expose your FK fields in the entities additionally to the navigation properties.
Your initial query would become something like this:
DbContext.LAYOUT
.Where(re => re.LAYOUT_ID == report.LAYOUT_FK)
.Load();
DbContext.PAGEMASTER
.Where(re => re.PAGEMASTERT_ID == report.PAGEMASTER_FK)
.Load();
The goal is to get the first DateTime and Last DateTime from a collection on an Entity (Foreign Key). My Entity is an organization and my collection are Invoices. I'm grouping results since Organizations unfortunately are not Unique. I'm dealing with duplicate data and cannot assume my organizations are unique so I'm grouping by a Number field on my Entity.
I'm using .NET Core 2.1.2 with Entity Framework.
I'm trying to get the following query generated from LINQ:
SELECT MIN([organization].[Id]) AS Id, MIN([organization].[Name]) AS Name,
MIN([organization].[Number]) AS Number, MIN([invoice].[Date])
AS First, MAX([invoice].[Date]) AS Last
FROM [organization]
INNER JOIN [invoice] ON [invoice].[OrganizationId] = [organization].[Id]
GROUP BY [organization].[Number], [organization].[Name]
ORDER BY [organization].[Name]
However I have no idea how to get to write the LINQ query to get it to generate this result.
I got as far as:
await _context
.Organization
.Where(z => z.Invoices.Any())
.GroupBy(organization => new
{
organization.Number,
organization.Name
})
.Select(grouping => new
{
Id = grouping.Min(organization => organization.Id),
Name = grouping.Min(organization => organization.Name),
Number= grouping.Min(organization => organization.Number),
//First = ?,
//Last = ?
})
.OrderBy(z => z.Name)
.ToListAsync();
I have no clue how to write the LINQ query in such a way that it generates the above.
I have a couple questions still:
Are the Min statements for Id, Name and Number correct ways of getting the
first element in the grouping?
Do I need a join statement or is "WHERE EXISTS" better (this got generated before I changed the code)?
Does anyone know how to finish writing the LINQ statement? Because I have to get the first and last Date from the Invoices Collection on my Organization Entity:
organization.Invoices.Min(invoice => invoice.Date)
organization.Invoices.Max(invoice => invoice.Date)
Here is the trick.
To make inner join by using collection navigation property simple use SelectMany and project all primitive properties that you need later (this is important for the current EF Core query translator). Then perform the GroupBy and project the key properties / aggregates. Finally do the ordering.
So
var query = _context
.Organization
.SelectMany(organization => organization.Invoices, (organization, invoice) => new
{
organization.Id,
organization.Number,
organization.Name,
invoice.Date
})
.GroupBy(e => new
{
e.Number,
e.Name
})
.Select(g => new
{
Id = g.Min(e => e.Id),
Name = g.Key.Name,
Number = g.Key.Number,
First = g.Min(e => e.Date),
Last = g.Max(e => e.Date),
})
.OrderBy(e => e.Name);
is translated to
SELECT MIN([organization].[Id]) AS [Id], [organization].[Name], [organization].[Number],
MIN([organization.Invoice].[Date]) AS [First], MAX([organization.Invoice].[Date]) AS [Last]
FROM [Organization] AS [organization]
INNER JOIN [Invoice] AS [organization.Invoice] ON [organization].[Id] = [organization.Invoice].[OrganizationId]
GROUP BY [organization].[Number], [organization].[Name]
ORDER BY [organization].[Name]
I've a supplier table filled with supplier objects, supplierID is the primary key here. I've to find all products supplied by these suppliers. Since there is a many-many relationship, I've a bridge table in between SupplierProducts, with supplierID and productID as a composite primary key.
I've used lambda function to get an IEnumerable<SupplierProducts> for a particular supplier. Now, I'd like to query the products table to find all products that are in IEnumerable<SupplierProducts>. I don't want to use a foreach() to populate the products table, but rather an 'in' like lambda expression.
I'm sure this must have been answered earlier, but unfortunately after trying for fifteen minutes, I've not been able to find a clear solution. I've looked into contains() and any() functions. Here is my code:
IEnumerable<SupplierProducts> supplierProducts = db.SupplierProducts.Where(w => w.SupplierID == supplierID).ToList();
IEnumerable<Products> products = db.Products.Where(w => w.ProductID.contains(supplierProducts.productID)).ToList();
You are pretty close: all you need to do is selecting IDs, and then using Contains, like this:
var supplierProductIds = db.SupplierProducts
.Where(w => w.SupplierID == supplierID)
.Select(p => p.productID)
.ToList(); // You could get better performance without ToList
IEnumerable<Products> products = db.Products
.Where(w => supplierProductIds.Contains(w.ProductID))
.ToList();
If you have constructed you model correctly you should have the virtual property for Products in Supplier class
public virtual ICollection<Product> Products {get; set;}
and this would have made your task easier:
db.Suppliers.Where(s => s.Id == supplierID).Select(s => s.Products)
How do I include a child of a child entitiy?
Ie, Jobs have Quotes which have QuoteItems
var job = db.Jobs
.Where(x => x.JobID == id)
.Include(x => x.Quotes)
.Include(x => x.Quotes.QuoteItems) // This doesn't work
.SingleOrDefault();
Just to be clearer - I'm trying to retrieve a single Job item, and it's associated Quotes (one to many) and for each Quote the associated QuoteItems (One Quote can have many QuoteItems)
The reason I'm asking is because in my Quote Index view I'm trying to show the Total of all the Quote items for each Quote by SUMming the Subtotal, but it's coming out as 0. I'm calling the Subtotal like this:
#item.QuoteItem.Sum(p => p.Subtotal)
I believe the reason I have this issue is that my Linq query above isn't retrieving the associated QuoteItems for each Quote.
To get a job and eager load all its quotes and their quoteitems, you write:
var job = db.Jobs
.Include(x => x.Quotes.Select(q => q.QuoteItems))
.Where(x => x.JobID == id)
.SingleOrDefault();
You might need SelectMany instead of Select if QuoteItems is a collection too.
Note to others; The strongly typed Include() method is an extension method so you need to include using System.Data.Entity; at the top of your file.
The method in the accepted answer doesn't work in .NET Core.
For anyone using .NET Core, while the magic string way does work, the cleaner way to do it would be ThenInclude:
var job = db.Jobs
.Where(x => x.JobID == id)
.Include(x => x.Quotes)
.ThenInclude(x => x.QuoteItems)
.SingleOrDefault();
Source: Work with data in ASP.NET Core Apps | Microsoft Learn
This will do the job (given that we are talking entity framework and you want to fetch child-entities):
var job = db.Jobs
.Include(x => x.Quotes) // include the "Job.Quotes" relation and data
.Include("Quotes.QuoteItems") // include the "Job.Quotes.QuoteItems" relation with data
.Where(x => x.JobID == id) // going on the original Job.JobID
.SingleOrDefault(); // fetches the first hit from db.
For more information about the Include statement have a look at this: https://learn.microsoft.com/en-us/dotnet/api/system.data.objects.objectquery-1.include
This answer has been getting upvotes throught the years, so I'd just like to clarify, try https://stackoverflow.com/a/24120209/691294 first. This answer is for those cases where all else fails and you have to resort to a black magic solution (i.e. using magic strings).
This did the trick for me as #flindeberg said here .
Just added checking if there are children in each parent item in the list
List<WCF.DAL.Company> companies = dbCtx.Companies.Where(x=>x.CompanyBranches.Count > 0)
.Include(c => c.CompanyBranches)
.Include("CompanyBranches.Address")
.ToList();