While debugging some performance issues I discovered that Entity framework was loading a lot of records via lazy loading (900 extra query calls ain't fast!) but I was sure I had the correct include. I've managed to get this down to quite a small test case to demonstrate the confusion I'm having, the actual use case is more complex so I don't have a lot of scope to re-work the signature of what I'm doing but hopefully this is a clear example of the issue I'm having.
Documents have Many MetaInfo rows related. I want to get all documents grouped by MetaInfo rows with a specific value, but I want all the MetaInfo rows included so I don't have to fire off a new request for all the Documents MetaInfo.
So I've got the following Query.
ctx.Configuration.LazyLoadingEnabled = false;
var DocsByCreator = ctx.Documents
.Include(d => d.MetaInfo) // Load all the metaInfo for each object
.SelectMany(d => d.MetaInfo.Where(m => m.Name == "Author") // For each Author
.Select(m => new { Doc = d, Creator = m })) // Create an object with the Author and the Document they authored.
.ToList(); // Actualize the collection
I expected this to have all the Document / Author pairs, and have all the Document MetatInfo property filled.
That's not what happens, I get the Document objects, and the Authors just fine, but the Documents MetaInfo property ONLY has MetaInfo objects with Name == "Author"
If I move the where clause out of the select many it does the same, unless I move it to after the actualisation (which while here might not be a big deal, it is in the real application as it means we're getting a huge amount more data than we want to deal with.)
After playing with a bunch of different ways to do this I think it really looks like the issue is when you do a select(...new...) as well as the where and the include. Doing the select, or the Where clause after actualisation makes the data appear the way I expected it to.
I figured it was an issue with the MetaInfo property of Document being filtered, so I rewrote it as follows to test the theory and was surprised for find that this also gives the same (I think wrong) result.
ctx.Configuration.LazyLoadingEnabled = false;
var DocsByCreator = ctx.Meta
.Where(m => m.Name == "Author")
.Include(m => m.Document.MetaInfo) // Load all the metaInfo for Document
.Select(m => new { Doc = m.Document, Creator = m })
.ToList(); // Actualize the collection
Since we're not putting the where on the Document.MetaInfo property I expected this to bypass the problem, but strangely it doesn't the documents still only appear to have "Author" MetaInfo object.
I've created a simple test project and uploaded it to github with a bunch of test cases in, as far as I can tell they should all pass, bug only the ones with premature actualisation pass.
https://github.com/Robert-Laverick/EFIncludeIssue
Anyone got any theories? Am I abusing EF / SQL in some way I'm missing? Is there anything I can do differently to get the same organisation of results? Is this a bug in EF that's just been hidden from view by the LazyLoad being on by default, and it being a bit of an odd group type operation?
This is a limitation in EF in that Includes will be ignored if the scope of the entities returned is changed from where the include was introduced.
I couldn't find the reference to this for EF6, but it is documented for EF Core. (https://learn.microsoft.com/en-us/ef/core/querying/related-data) (see "ignore includes") I suspect it is a limit in place to stop EF's SQL generation from going completely AWOL in certain scenarios.
So while var docs = context.Documents.Include(d => d.Metas) would return the metas eager loaded against the document; As soon as you .SelectMany() you are changing what EF is supposed to return, so the Include statement is ignored.
If you want to return all documents, and include a property that is their author:
var DocsByCreator = ctx.Documents
.Include(d => d.MetaInfo)
.ToList() // Materialize the documents and their Metas.
.SelectMany(d => d.MetaInfo.Where(m => m.Name == "Author") // For each Author
.Select(m => new { Doc = d, Creator = m })) // Create an object with the Author and the Document they authored.
.ToList(); // grab your collection of Doc and Author.
If you only want documents that have authors:
var DocsByCreator = ctx.Documents
.Include(d => d.MetaInfo)
.Where(d => d.MetaInfo.Any(m => m.Name == "Author")
.ToList() // Materialize the documents and their Metas.
.SelectMany(d => d.MetaInfo.Where(m => m.Name == "Author") // For each Author
.Select(m => new { Doc = d, Creator = m })) // Create an object with the Author and the Document they authored.
.ToList(); // grab your collection of Doc and Author.
This means you will want to be sure that all of your filtering logic is done above that first 'ToList() call. Alternatively you can consider resolving the Author meta after the query such as when view models are populated, or an unmapped "Author" property on Document that resolves it. Though I generally avoid unmapped properties because if their use slips into an EF query, you get a nasty error at runtime.
Edit: Based on the requirement to skip & take I would recommend utilizing view models to return data rather than returning entities. Using a view model you can instruct EF to return just the raw data you need, compose the view models with either simple filler code or utilizing Automapper which plays nicely with IQueryable and EF and can handle most deferred cases like this.
For example:
public class DocumentViewModel
{
public int DocumentId { get; set; }
public string Name { get; set; }
public ICollection<MetaViewModel> Metas { get; set; } = new List<MetaViewModel>();
[NotMapped]
public string Author // This could be update to be a Meta, or specialized view model.
{
get { return Metas.SingleOrDefault(x => x.Name == "Author")?.Value; }
}
}
public class MetaViewModel
{
public int MetaId { get; set; }
public string Name { get; set; }
public string Value { get; set; }
}
Then the query:
var viewModels = context.Documents
.Select(x => new DocumentViewModel
{
DocumentId = x.DocumentId,
Name = x.Name,
Metas = x.Metas.Select(m => new MetaViewModel
{
MetaId = m.MetaId,
Name = m.Name,
Value = m.Value
}).ToList()
}).Skip(pageNumber*pageSize)
.Take(PageSize)
.ToList();
The relationship of an "author" to a document is implied, not enforced, at the data level. This solution keeps the entity models "pure" to the data representation and lets the code handle transforming that implied relationship into exposing a document's author.
The .Select() population can be handled by Automapper using .ProjectTo<TViewModel>().
By returning view models rather than entities you can avoid issues like this where .Include() operations get invalidated, plus avoid issues due to the temptation of detaching and reattaching entities between different contexts, plus improve performance and resource usage by only selecting and transmitting the data needed, and avoiding lazy load serialization issues if you forget to disable lazy-load or unexpected #null data with it.
Related
I'm trying to load some user info in code, but EF returns null.
foreach (var user in allAutoUsers)
{
Wallet wallet = db.CabinetUsers
.Find(user.IdCabinetUser)?
.Wallets
.FirstOrDefault(x => x.TypeCurrency == currency);
}
Variable user reporting 1 wallet, but when I'm trying to get it in code above it returns null.
Are there some ways to solve this problem?
Read more into Linq expressions rather than relying on Find. You can be running into issues if your relationships between entities are not defined as virtual which would prevent EF from lazy loading them.
The issue with using .Find() is that it will return the entity if it exists, however, attempting to access any related property that the DbContext isn't already aware of will require a lazy load call. This can be easily missed if lazy loading is disabled, or the member isn't virtual, and it can be a performance issue when it is enabled.
Instead, Linq can allow you to query through the object graph directly to get what you want:
foreach (var user in allAutoUsers)
{
Wallet wallet = db.CabinetUsers
.Where(x => x.IdCabinetUser == user.IdCabinetUser)
.SelectMany(x => x.Wallets)
.SingleOrDefault(x => x.TypeCurrency == currency);
// do stuff with wallet...
}
This assumes that there will be only 1 wallet for the specified currency per user. When expecting 0 or 1, use SingleOrDefault. Use FirstOrDefault only when you are expecting 0 or many, want the "first" one and have specified an OrderBy clause to ensure the first item is predictable.
This will result in a query per user. To accomplish with 1 query for all users:
var userIds = allAutoUsers.Select(x => x.IdCabinetUser).ToList();
var userWallets = db.CabinetUsers
.Where(x => userIds.Contains(x.IdCabinetUser))
.Select(x => new
{
x.IdCabinetUser,
Wallet = x.SelectMany(x => x.Wallets)
.SingleOrDefault(x => x.TypeCurrency == currency);
}).ToList();
From this I would consider expanding the wallets SelectMany with a Select for the details in the Wallet you actually care about rather than a reference to the entire wallet entity. This has the benefit of speeding up the query, reducing memory use, and avoids the potential for lazy load calls tripping things up if Wallet references any other entities that get touched later on.
For example if you only need the IdWallet, WalletName, TypeCurrency, and Balance:
// replace this line from above...
Wallet = x.SelectMany(x => x.Wallets)
// with ...
Wallet = x.SelectMany(x => x.Wallets.Select(w => new
{
w.IdWallet,
w.WalletName,
w.TypeCurrency,
w.Ballance
}) // ...
From there you can foreach to your heart's content without extra queries:
foreach ( var userWallet in userWallets)
{
// do stuff with userWallet.Wallet and userWallet.IdCabinetUser.
}
If you want to return the wallet details to a calling method or view or such, then you cannot use an anonymous type for that ( new { } ). Instead you will need to define a simple class for the data you want to return and use Select into that. I.e. new WalletDTO { IdWallet = w.IdWallet, //... } Even if you use Entities, it is recommended to reduce these into DTOs or ViewModels rather than returning entities. Entities should not "live" longer than the DbContext that spawned them otherwise you get all kinds of nasty behaviour popping up like ObjectDisposedException and serialization exceptions.
I'm working on small app which is written in c# .net core and I'm populating one prop in a code because that information is not available in database, code looks like this:
public async Task<IEnumerable<ProductDTO>> GetData(Request request)
{
IQueryable<Product> query = _context.Products;
var products = await query.ToListAsync();
// WARNING - THIS SOLUTION LOOKS EXPENCIVE TO ME!
return MapDataAsDTO(products).Select(c =>
{
c.HasBrandStock = products.Any(cc => cc.ParentProductId == c.Id);
return c;
});
}
}
private IEnumerable<ProductDTO> MapDataAsDTO(IEnumerable<Product> products)
{
return products.Select(p => MapData(p)).ToList();
}
What is bothering me here is this code:
return MapDataAsDTO(products).Select(c =>
{
c.HasBrandStock = data.Any(cc => cc.ParentProductId == c.Id);
return c;
});
}
I've tested it on like 300k rows and it seems slow, I'm wondering is there a better solutions in this situations?
Thanks guys!
Cheers
First up, this method is loading all products, and generally that is a bad idea unless you are guaranteeing that the total number of records will remain reasonable, and the total size of those records will be reasonable. If the system can grow, add support for server-side pagination now. (Page # and Page size, leveraging Skip & Take) 300k products is not a reasonable number to be loading all data in one hit. Any way you skin this cat it will be slow, expensive, and error prone due to server load without paging. One user making a request on the server will need to have the DB server allocate for and load up 300k rows, transmit that data over the wire to the app server, which will allocate memory for those 300k rows, then transmit that data over the wire to the client who literally does not need those 300k rows at once. What do you think happens when 10 users hit this page? 100? And what happens when it's "to slow" and they start hammering the F5 key a few times. >:)
Second, async is not a silver bullet. It doesn't make queries faster, it actually makes them a bit slower. What it does do is allow your web server to be more responsive to other requests while those slower queries are running. Default to synchronous queries, get them running as efficiently as possible, then for the larger ones that are justified, switch them to asynchronous. MS made async extremely easy to implement, perhaps too easy to treat as a default. Keep it simple and synchronous to start, then re-factor methods to async as needed.
From what I can see you want to load all products into DTOs, and for products that are recognized as being a "parent" of at least one other product, you want to set their DTO's HasBrandStock to True. So given product IDs 1 and 2, where 2's parent ID is 1, the DTO for Product ID 1 would have a HasBrandStock True while Product ID 2 would have HasBrandStock = False.
One option would be to tackle this operation in 2 queries:
var parentProductIds = _context.Products
.Where(x => x.ParentProductId != null)
.Select(x => x.ParentProductId)
.Distinct()
.ToList();
var dtos = _context.Products
.Select(x => new ProductDTO
{
ProductId = x.ProductId,
ProductName = x.ProductName,
// ...
HasBrandStock = parentProductIds.Contains(x.ProductId)
}).ToList();
I'm using a manual Select here because I don't know what your MapAsDto method is actually doing. I'd highly recommend using Automapper and it's ProjectTo<T> method if you want to simplify the mapping code. Custom mapping functions can too easily hide expensive bugs like ToList calls when someone hits a scenario that EF cannot translate.
The first query gets a distinct list of just the Product IDs that are the parent ID of at least one other product. The second query maps out all products into DTOs, setting the HasBrandStock based on whether each product appears in the parentProductIds list or not.
This option will work if a relatively limited number of products are recognized as "parents". That first list can only get so big before it risks crapping out being too many items to translate into an IN clause.
The better option would be to look at your mapping. You have a ParentProductId, does a product entity have an associated ChildProducts collection?
public class Product
{
public int ProductId { get; set; }
public string ProductName { get; set; }
// ...
public virtual Product ParentProduct { get; set; }
public virtual ICollection<Product> ChildProducts { get; set; } = new List<Product>();
}
public class ProductConfiguration : EntityTypeConfiguration<Product>
{
public ProductConfiguration()
{
HasKey(x => x.ProductId);
HasOptional(x => x.ParentProduct)
.WithMany(x => x.ChildProducts)
.Map(x => x.MapKey("ParentProductId"));
}
}
This example maps the ParentProductId without exposing a field in the entity (recommended). Otherwise, if you do expose a ParentProductId, substitute the .Map(...) call with .HasForeignKey(x => x.ParentProductId).
This assumes EF6 as per your tags, if you're using EF Core then you use HasForeignKey("ParentProductId") in place of Map(...) to establish a shadow property for the FK without exposing a property. The entity configuration is a bit different with Core.
This allows your queries to leverage the relationship between parent products and any related children products. Populating the DTOs can be accomplished with one query:
var dtos = _context.Products
.Select(x => new ProductDTO
{
ProductId = x.ProductId,
ProductName = x.ProductName,
// ...
HasBrandStock = x.ChildProducts.Any()
}).ToList();
This leverages the relationship to populate your DTO and it's flag in one pass. The caveat here is that there is now a cyclical relationship between product and itself represented in the entity. This means don't feed entities to something like a serializer. That includes avoiding adding entities as members of DTOs/ViewModels.
What seemed that it should be a relatively straight-forward task has turned into something of a surprisingly complex issue. To the point that I'm starting to think that my methodology perhaps is simply out of scope with the capabilities of Linq.
What I'm trying to do is piece-together a Linq query and then invoke .Include() in order to pull-in values from a number of child entities. For example, let's say I have these entities:
public class Parent
{
public int Id { get; set; }
public string Name { get; set; }
public string Location { get; set; }
public ISet<Child> Children { get; set; }
}
public class Child
{
public int Id { get; set; }
public int ParentId { get; set; }
public Parent Parent { get; set; }
public string Name { get; set; }
}
And let's say I want to perform a query to retrieve records from Parent, where Name is some value and Location is some other value, and then include Child records, too. But for whatever reason I don't know the query values for Name and Location at the same time, so I have to take two separate queryables and join them, such:
MyDbContext C = new MyDbContext();
var queryOne = C.Parent.Where(p => p.Name == myName);
var queryTwo = C.Parent.Where(p => p.Location == myLocation);
var finalQuery = queryOne.Intersect(queryTwo);
That works fine, producing results exactly as if I had just done:
var query = C.Parent.Where(p => p.Name == myName && p.Location = myLocation);
And similarly, I can:
var finalQuery = queryOne.Union(queryTwo);
To give me results just as if I had:
var query = C.Parent.Where(p => p.Name == myName || p.Location = myLocation);
What I cannot do, however, once the Intersect() or Union() is applied, however, is then go about mapping the Child using Include(), as in:
finalQuery.Include(p => p.Children);
This code will compile, but produces results as follows:
In the case of a Union(), a result set will be produced, but no Child entities will be enumerated.
In the case of an Intersect(), a run-time error is generated upon attempt to apply Include(), as follows:
Expression of type
'System.Collections.Generic.IEnumerable`1[Microsoft.EntityFrameworkCore.Query.Internal.AnonymousObject]'
cannot be used for parameter of type
'System.Collections.Generic.IEnumerable`1[System.Object]' of method
'System.Collections.Generic.IEnumerable`1[System.Object]
Intersect[Object](System.Collections.Generic.IEnumerable`1[System.Object],
System.Collections.Generic.IEnumerable`1[System.Object])'
The thing that baffles me is that this code will work exactly as expected:
var query = C.Parent.Where(p => p.Name == myName).Where(p => p.Location == myLocation);
query.Include(p => p.Children);
I.e., with the results as desired, including the Child entities enumerated.
my methodology perhaps is simply out of scope with the capabilities of Linq
The problem is not LINQ, but EF Core query translation, and specifically the lack of Intersect / Union / Concat / Except method SQL translation, tracked by #6812 Query: Translate IQueryable.Concat/Union/Intersect/Except/etc. to server.
Shortly, such queries currently use client evaluation, which with combination of how the EF Core handles Include leads to many unexpected runtime exceptions (like your case #2) or wrong behaviors (like Ignored Includes in your case #1).
So while your approach technically perfectly makes sense, according to the EF Core team leader response
Changing this to producing a single SQL query on the server isn't currently a top priority
so this currently is not even planned for 3.0 release, although there are plans to change (rewrite) the whole query translation pipeline, which might allow implementing that as well.
For now, you have no options. You may try processing the query expression trees yourself, but that's a complicated task and you'll probably find why it is not implemented yet :) If you can convert your queries to the equivalent single query with combined Where condition, then applying Include will be fine.
P.S. Note that even now your approach technically "works" w/o Include, prefomance wise the way it is evaluated client side makes it absolutely non equivalent of the corresponding single query.
A long time has gone by, but this .Include problem still exists in EF 6. However, there is a workaround: Append every child request with .Include before intersecting/Unionizing.
MyDbContext C = new MyDbContext();
var queryOne = db.Parents.Where(p => p.Name == parent.Name).Include("Children");
var queryTwo = db.Parents.Where(p => p.Location == parent.Location).Include("Children");
var finalQuery = queryOne.Intersect(queryTwo);
As stated by #Ivan Stoev, Intersection/Union is done with after-fetched data, while .Include is ok at request time.
So, as of now, you have this one option available.
In entity framework core 2.0, I have many-many relationship between Post and Category (the binding class is PostCategory).
When the user updates a Post, the whole Post object (with its PostCategory collection) is being sent to the server, and here I want to reassign the new received Collection PostCategory (the user may change this Collection significantly by adding new categories, and removing some categories).
Simplified code I use to update that collection (I just assign completely new collection):
var post = await dbContext.Posts
.Include(p => p.PostCategories)
.ThenInclude(pc => pc.Category)
.SingleOrDefaultAsync(someId);
post.PostCategories = ... Some new collection...; // <<<
dbContext.Posts.Update(post);
await dbContext.SaveChangesAsync();
This new collection has objects with the same Id of objects in the previous collection (e.g. the user removed some (but not all) categories). Because of the, I get an exception:
System.InvalidOperationException: The instance of entity type 'PostCategory' cannot be tracked because another instance with the same key value for {'CategoryId', 'PostId'} is already being tracked.
How can I rebuild the new collection (or simply assign a new collection) efficiently without getting this exception?
UPDATE
The answer in this link seems to be related to what I want, but it is a good and efficient method? Is there any possible better approach?
UPDATE 2
I get my post (to edit overwrite its values) like this:
public async Task<Post> GetPostAsync(Guid postId)
{
return await dbContext.Posts
.Include(p => p.Writer)
.ThenInclude(u => u.Profile)
.Include(p => p.Comments)
.Include(p => p.PostCategories)
.ThenInclude(pc => pc.Category)
.Include(p => p.PostPackages)
.ThenInclude(pp => pp.Package)
//.AsNoTracking()
.SingleOrDefaultAsync(p => p.Id == postId);
}
UPDATE 3 (The code in my controller, which tries to update the post):
var writerId = User.GetUserId();
var categories = await postService.GetOrCreateCategoriesAsync(
vm.CategoryViewModels.Select(cvm => cvm.Name), writerId);
var post = await postService.GetPostAsync(vm.PostId);
post.Title = vm.PostTitle;
post.Content = vm.ContentText;
post.PostCategories = categories?.Select(c => new PostCategory { CategoryId = c.Id, PostId = post.Id }).ToArray();
await postService.UpdatePostAsync(post); // Check the implementation in Update4.
UPDATE 4:
public async Task<Post> UpdatePostAsync(Post post)
{
// Find (load from the database) the existing post
var existingPost = await dbContext.Posts
.SingleOrDefaultAsync(p => p.Id == post.Id);
// Apply primitive property modifications
dbContext.Entry(existingPost).CurrentValues.SetValues(post);
// Apply many-to-many link modifications
dbContext.Set<PostCategory>().UpdateLinks(
pc => pc.PostId, post.Id,
pc => pc.CategoryId,
post.PostCategories.Select(pc => pc.CategoryId)
);
// Apply all changes to the db
await dbContext.SaveChangesAsync();
return existingPost;
}
The main challenge when working with disconnect link entities is to detect and apply the added and deleted links. And EF Core (as of the time of writing) provides little if no help to do that.
The answer from the link is ok (the custom Except method is too heavier for what it does IMO), but it has some traps - the existing links has to be retrieved in advance using the eager / explicit loading (though with EF Core 2.1 lazy loading that might not be an issue), and the new links should have only FK properties populated - if they contain reference navigation properties, EF Core will try to create new linked entities when calling Add / AddRange.
A while ago I answered similar, but slightly different question - Generic method for updating EFCore joins. Here is the more generalized and optimized version of the custom generic extension method from the answer:
public static class EFCoreExtensions
{
public static void UpdateLinks<TLink, TFromId, TToId>(this DbSet<TLink> dbSet,
Expression<Func<TLink, TFromId>> fromIdProperty, TFromId fromId,
Expression<Func<TLink, TToId>> toIdProperty, IEnumerable<TToId> toIds)
where TLink : class, new()
{
// link => link.FromId == fromId
Expression<Func<TFromId>> fromIdVar = () => fromId;
var filter = Expression.Lambda<Func<TLink, bool>>(
Expression.Equal(fromIdProperty.Body, fromIdVar.Body),
fromIdProperty.Parameters);
var existingLinks = dbSet.AsTracking().Where(filter);
var toIdSet = new HashSet<TToId>(toIds);
if (toIdSet.Count == 0)
{
//The new set is empty - delete all existing links
dbSet.RemoveRange(existingLinks);
return;
}
// Delete the existing links which do not exist in the new set
var toIdSelector = toIdProperty.Compile();
foreach (var existingLink in existingLinks)
{
if (!toIdSet.Remove(toIdSelector(existingLink)))
dbSet.Remove(existingLink);
}
// Create new links for the remaining items in the new set
if (toIdSet.Count == 0) return;
// toId => new TLink { FromId = fromId, ToId = toId }
var toIdParam = Expression.Parameter(typeof(TToId), "toId");
var createLink = Expression.Lambda<Func<TToId, TLink>>(
Expression.MemberInit(
Expression.New(typeof(TLink)),
Expression.Bind(((MemberExpression)fromIdProperty.Body).Member, fromIdVar.Body),
Expression.Bind(((MemberExpression)toIdProperty.Body).Member, toIdParam)),
toIdParam);
dbSet.AddRange(toIdSet.Select(createLink.Compile()));
}
}
It uses a single database query to retrieve the exiting links from the database. The overhead are few dynamically built expressions and compiled delegates (in order to keep the calling code simplest as possible) and a single temporary HashSet for fast lookup. The performance affect of the expression / delegate building should be negligible, and can be cached if needed.
The idea is to pass just a single existing key for one of the linked entities and list of exiting keys for the other linked entity. So depending of which of the linked entity links you are updating, it will be called differently.
In you sample, assuming you are receiving IEnumerable<PostCategory> postCategories, the process would be something like this:
var post = await dbContext.Posts
.SingleOrDefaultAsync(someId);
dbContext.Set<PostCategory>().UpdateLinks(pc =>
pc.PostId, post.Id, pc => pc.CategoryId, postCategories.Select(pc => pc.CategoryId));
await dbContext.SaveChangesAsync();
Note that this method allows you to change the requirement and accept IEnumerable<int> postCategoryIds:
dbContext.Set<PostCategory>().UpdateLinks(pc =>
pc.PostId, post.Id, pc => pc.CategoryId, postCategoryIds);
or IEnumerable<Category> postCategories:
dbContext.Set<PostCategory>().UpdateLinks(pc =>
pc.PostId, post.Id, pc => pc.CategoryId, postCategories.Select(c => c.Id));
or similar DTOs / ViewModels.
Category posts can be updated in a similar manner, with corresponding selectors swapped.
Update: In case you a receiving a (potentially) modified Post post entity instance, the whole update procedure cold be like this:
// Find (load from the database) the existing post
var existingPost = await dbContext.Posts
.SingleOrDefaultAsync(p => p.Id == post.Id);
if (existingPost == null)
{
// Handle the invalid call
return;
}
// Apply primitive property modifications
dbContext.Entry(existingPost).CurrentValues.SetValues(post);
// Apply many-to-many link modifications
dbContext.Set<PostCategory>().UpdateLinks(pc => pc.PostId, post.Id,
pc => pc.CategoryId, post.PostCategories.Select(pc => pc.CategoryId));
// Apply all changes to the db
await dbContext.SaveChangesAsync();
Note that EF Core uses separate database query for eager loading related collecttions. Since the helper method does the same, there is no need to Include link related data when retrieving the main entity from the database.
Consider following LINQ query:
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select new
{
ItemProp1 = obj,
ItemProp2 = obj.NavProp2.Any(n => n.Active)
}).SingleOrDefault();
This runs as expected, but item.ItemProp1.NavProp1 is NULL.
As it explains here this is because of the query actually changes after using Include(). but the question is what is the solution with this situation?
Edit:
When I change the query like this, every things works fine:
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select obj).SingleOrDefault();
Regarding to this article I guess what the problem is... but the solution provided by author not working in my situation (because of using anonymous type in final select rather than entity type).
As you mentioned, Include is only effective when the final result of the query consists of the entities that should include the Include-d navigation properties.
So in this case Include has effect:
var list = _db.SampleEntity.Include(s => s.NavProp1).ToList();
The SQL query will contain a JOIN and each SampleEntity will have its NavProp1 loaded.
In this case it has no effect:
var list = _db.SampleEntity.Include(s => s.NavProp1)
.Select(s => new { s })
.ToList();
The SQL query won't even contain a JOIN, EF completely ignores the Include.
If in the latter query you want the SampleEntitys to contain their NavProp1s you can do:
var list = _db.SampleEntity
.Select(s => new { s, s.NavProp1 })
.ToList();
Now Entity Framework has fetched SampleEntitys and NavProp1 entities from the database separately, but it glues them together by a process called relationship fixup. As you see, the Include is not necessary to make this happen.
However, if Navprop1 is a collection, you'll notice that...
var navprop1 = list.First().s.Navprop1;
...will still execute a query to fetch Navprop1 by lazy loading. Why is that?
While relationship fixup does fill Navprop1 properties, it doesn't mark them as loaded. This only happens when Include loaded the properties. So now we have SampleEntity all having their Navprop1s, but you can't access them without triggering lazy loading. The only thing you can do to prevent this is
_db.Configuration.LazyLoadingEnabled = false;
var navprop1 = list.First().s.Navprop1;
(or by preventing lazy loading by disabling proxy creation or by not making Navprop1 virtual.)
Now you'll get Navprop1 without a new query.
For reference navigation properties this doesn't apply, lazy loading isn't triggered when it's enabled.
In Entity Framework core, things have changed drastically in this area. A query like _db.SampleEntity.Include(s => s.NavProp1).Select(s => new { s }) will now include NavProp1 in the end result. EF-core is smarter in looking for "Includable" entities in the end result. Therefore, we won't feel inclined to shape a query like Select(s => new { s, s.NavProp1 }) in order to populate the navigation property. Be aware though, that if we use such a query without Include, lazy loading will still be triggered when s.NavProp1 is accessed.
I know this will probably get a few laughs, but don't forget the obvious like i just did. The row in the database didn't actually have a foreign key reference! I should have checked the dam data first before thinking EF Include wasn't working! Grrr. 30 minutes of my life I won't get back.
If your model is defined properly it should work without any problems.
using System.Data.Entity;
var item = _db.SampleEntity
.Include(p => p.NavigationProperty)
.Select(p => new YourModel{
PropertyOne = p.Something,
PropertyTwo = p.NavigationProperty.Any(x => x.Active)
})
.SingleOrDefault(p => p.Something == true);
How did you find that item.ItemProp1.NavProp1 is null. EF uses proxies to load all required properties when you try to access it.
What about
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select obj).SingleOrDefault();
Assert.IsNotNull(obj.NavProp1);
Assert.IsNotNull(obj.NavProp2);
You can also try with
var item = (from obj in _db.SampleEntity.Include(s => s.NavProp1)
select new
{
ItemProp1 = obj,
NavProp1 = obj.NavProp1,
ItemProp2 = obj.NavProp2.Any(n => n.Active)
}).SingleOrDefault();
Assert.IsNotNull(item.NavProp1)
Of course I assume that you don't have any problems with EF navigation property mappings.