We Keep Coding

Entity Framework Core 6 lazy loading takes "loading data when needed" too literal

I have a simple class called Company and a corresponding service (CompanyService) to access an Azure SQL Database. A company contains Users.
public class Company
{
[Key]
public int Id { get; set; }
[Required]
[StringLength(64, MinimumLength = 5)]
public string Name { get; set; }
public ICollection<User> Users { get; set; } = new List<User>(); // removing the initialization does not make any difference
}
public class User
{
[Key]
public int Id { get; set; }
[Required]
[StringLength(64, MinimumLength = 5)]
public string Name { get; set; }
public Company Company { get; set; }
public int CompanyId { get; set; }
}
Both of these models are added in my DbContext and I can make simple CRUD operations. Anyways, I started to play around with the UI and wanted to display the users of a company. So basically, you have a table with all companies -> you select a company and click the "edit" button -> a new view opens where the properties of the company can now be updated and all corresponding users are displayed.
After implementing this, I realized, that my Company.Users list is completely empty, even though there should be 10 dummy users in it. I checked the view, where all users are displayed and see that all of them are there. I navigated to my companies page again, selected the same company and what do I see? The company's users!
So what is the problem: The user data is only loaded from my CompanyService AFTER I have accessed the user data from my UserService. I have 0 clue, why this is happening.
I access my data like this:
public async Task<Company?> GetCompanyById(int id)
{
return await this.Context.Company.Include(c => c.Users).FirstOrDefaultAsync(c => c.Id == id);
}
My database is created like this:
builder.Services.AddDbContextFactory<DatabaseContext>(options => options.UseSqlServer(builder.Configuration.GetConnectionString("DefaultConnection")));

You're not actually using lazy loading at all. For Lazy loading to work you need to declare the Company.Users collection as virtual. Even then it will only work while the company is still within scope of the DbContext it was read from. This can trip you up when passing entities around. When Lazy Loading is enabled and you attempt to access an unloaded collection after the DbContext is disposed for instance, you will get an exception.
What you are seeing instead is EF's entity tracking and reference population behaviour. When you request an entity that has relationships to other entities, you can eager load those relationships to ensure the associated entities are loaded and referenced, or leave them for EF to work out. EF will automatically associate any related entities that it happens to already be tracking, regardless of whether you tell it to eager load or not.
So for example lets say I have a Parent Child relationship. Parent ID P1 has 3 children, ID C1, C3, and C3. With lazy loading disabled if I use the following statement:
var parent = context.Parents.Single(p => p.Id == "P1");
var count = parent.Children.Count();
I will get "0". (Provided the Children collection is initialized, otherwise I'd get a NullReferenceException) If I turn on lazy loading, or I eager load like below:
var parent = context.Parents.Include(p => p.Children).Single(p => p.Id == "P1");
var count = parent.Children.Count();
in both cases I would get "3".
Now where things get interesting. If lazy loading is disabled/Children is not virtual and I do the following.
var child = context.Children.Single(c => c.Id == "C1");
var parent = context.Parents.Single(p => p.Id == "P1");
var count = parent.Children.Count();
I will get a count of "1", not "0" because I didn't eager load, not "3" because lazy loading isn't applied here. If 2 of the related children happened to be tracked, I'd get "2". Now normally you won't do something so obvious, but any earlier code that "might" have loaded a tracked reference to related data to a row you are later loading will be automatically included in that new row's relation. This can lead to confusing bugs where sometimes related data seems to be available but not other times, or not a complete set of data is available.
This is one underlying reason why you want DbContext lifespans to be as short as possible. The more entities a DbContext is tracking the longer it takes to "work out" possible relationships between tracked entities to populate references, and also the more potentially stale references might be used. (Rather than loading fresh, current data from the DB)
The best practices with EF when it comes to avoiding issues like this:
When reading information for display, use Projection rather than loading entities. I.e. load ViewModels using Select or Automapper's ProjectTo. This avoids loading tracked entities and ensures that data comes from current persisted data state. It also builds far more efficient queries than loading entities with eager loaded relationships. (Which get slow since EF is building Cartesian products behind the scenes)
If you don't need to update data but do need to load entities, use AsNoTracking. This avoids entities stuffing the tracking cache, slowing things down and causing these problems.
When updating entities, always eager load relationships that need to be updated. Ideally design systems to update child relations independent of the parent. I.e. AddChild, RemoveChild, UpdateChild rather than making changes to a children collection and trying to update them along with every other change witnin a single UpdateParent.
Ensure that DbContexts are not alive any longer than they are absolutely needed. Long-lived DbContexts collect expensive baggage.

When retrieving a set with EFCore, why do the retrieved entities not include their relationships to other objects?

While using Entity Framework Core with SQL Server I encountered an unexpected problem. I have an entity A that has a one-to-many relationship on entity B.
[Table("client")]
public class Client
{
public long ID { get; set; }
public string Name { get; set; }
public ICollection<Configuration> Configurations { get; set; } = new LinkedList<Configuration>();
}
I get a list of instances of entity A from the database like this:
public ICollection<Client> GetAllClients()
{
return _dbContext.Clients.ToList();
}
When I call this function I get a list of instances without the instances of entity B in the relationship. Why are the objects in the relationship not retrieved correctly?
I've also found out that if I add this line of code to the function the entities are retrieved as expected.
public ICollection<Client> GetAllClients()
{
var test = _dbContext.Configurations.ToList();
return _dbContext.Clients.ToList();
}
This makes no sense to me. What am I missing here?

You can use the Include method to specify related data to be included in query results (Eager loading).
Take a look in the following example:
public ICollection<Client> GetAllClients()
{
return _dbContext.Clients.Include(x => x.Configurations).ToList();
}
You can check the MSDN reference.

Related reference/collection properties must either be eagerly or explictly loaded. You generally want to eagerly load using Include:
var clients = await _context.Clients.Include(x => x.Configurations).ToListAsync();
Alternatively, you can lazy load, but that's generally a bad idea, as it can lead to N+1 query problems (i.e. you issue one query, and then a separate additional query for each item as you iterate through, which is obviously highly inefficient). Regardless, lazy loading requires two things:
The reference/collection property must have the virtual keyword. EF adds lazy loading functionality by creating a dynamic proxy of your entity class and overriding the property getter. Overriding, of course, can only be done on virtuals.
You have to explicitly add the lazy-loading services:
services.AddDbContext<MyContext>(o => o.UseLazyLoadingProxies()
.UseSqlServer(connectionString));
It works when you query the configurations as well, because EF has object-fixup. In other words, it will automatically fill related reference/collection properties, if it has previously retrieved those objects already and has them in its object cache. Otherwise, and if you do not otherwise load the relations, the property will remain null.

Linq to entities lazy loading

I have the following class generated by entity framework:
public partial class Branch
{
public short Id { get; set; }
public short CompanyId { get; set; }
public string Code { get; set; }
public string Title { get; set; }
public virtual Company Ts_Companies { get; set; }
}
I have the following method which takes all of the branches out of the database:
public Branch[] LoadBranches(int companyId, int page, int limit, string search, string sort, string sortOrder)
{
using (var dbContext = new TimeShedulerEntities())
{
var _branches = (from ct in dbContext.Branches
where ct.Title.Contains(search) || ct.Code.Contains(search)
select ct).OrderBy(c => c.Title).Skip((page - 1) * limit).Take(limit);
return _branches.ToArray();
}
}
In my model designer I see that the Lazy Loading is set to true, but when I iterate over the branches, the property Ts_Companies is null. Also I get the following exception:
An exception of type 'System.ObjectDisposedException' occurred in
EntityFramework.dll but was not handled in user code
Additional information: The ObjectContext instance has been disposed
and can no longer be used for operations that require a connection.
Am I forgetting something?

You created and disposed of the context during your function since it was inside the using statement. Each entity happens to know from which context it was created so that lazy loading is possible.
When you accessed the Ts_Companies property, the entity realized that it had not yet loaded that property since it is probably a navigation property and attempted to ask its ObjectContext (TimeShedulerEntities) to load that property. However, the context had been disposed and so that it what caused that exception.
You need to modify your query as follows to 'pre-load' the Ts_Companies:
var _branches = (from ct in dbContext.Branches.Include("Ts_Companies")
where ct.Title.Contains(search) || ct.Code.Contains(search)
select ct).OrderBy(c => c.Title).Skip((page - 1) * limit).Take(limit);
It will take possibly quite a bit longer to load depending on the size of the Ts_Companies object and how many you end up bringing back at once, but the entity will stop asking its object context to load the Ts_Companies since you would have already loaded them.
A side note: I have found that creation and disposal of object context on a per-method basis causes problems when the entities are passed outside the function. If you want to create and destroy the object context in every function, you probably want to have the function return something that is not an entity. In other words, have an object that can be constructed from an entity and has the properties you need, but don't have it reference the entity. In java these are often called Data Transfer Objects (DTOs). You lose the read-write ability of entity framework, but you don't have unexpected ObjectDisposedExceptions flying all over the place.
The problem comes when you ask an entity to be associated with another (for example, adding on entity to a ICollection property of another entity) when they come from different objectcontexts. This will cause headaches for you since you would have to manually attach the objects to the same context before performing that operation. Additionally, you lose the ability to save changes to those entities without manually attaching them to a different context.
My opinion on how I would do it:
I've found it easier to either have an object containing all of these database access functions control the lifetime of the context (i.e. have your containing object be IDisposable and during disposal, destroy the context) or simply not return entities and have the datastore be read-old, write-new essentially without any modification ability.
For example, I have my object (I will call it my data access object) with a bunch of methods for getting database objects. These methods return entities. The data access object also has a SaveChanges method which simply calls the context's SaveChanges method. The data access object contains the context in a protected property and keeps it around until the data access object itself is disposed. Nobody but the data access object is allowed to touch its context. At that point, the context is disposed by manually calling 'Dispose'. The data access object could then used inside a using statement if that is your use case.
In any case, it is probably best to avoid passing entities attached to a context outside the scope in which their context exists since entity framework keeps references to that context all over the place in the individual entities

But you didn't load your Ts_Companies, use Eager Loading instead:
var _branches = dbContext.Branches
.Where(b => b.Title.Contains(search) || b.Code.Contains(search))
.Include("Ts_Companies")
.OrderBy(c => c.Title)
.Skip((page - 1) * limit)
.Take(limit);
And I came across the same issue before System.ObjectDisposedException, in my MVC project and I didn't use using blocks,instead I define my context on class level.If I need to return and use an array (in my View) I use that context.If I need to just update some information then I have used using blocks.I hope this helps.

How to trigger lazy loading on a navigation property?

How much do you have to "touch" a navigation property to assure lazy loading of a collection?
I am using Entity Framework 5.0 with lazy loading turned on. Consider a simple class:
public class MyResource
{
string name {get;set;}
public virtual ICollection<ResourceEvent> ResourceEvents{ get; set; }
}
When I set up a "foreach" on the collection, I want to avoid individual retrieval of each object in the collection.
using(context = new MyDBContext)
{
MyResource aresource = context.MyResources.Where(a=>a.Name==myname).Single();
//now I want to lazy load the ResourceEvents collection
if(aresource.MyResources!=null) // will this load collection?
{
List<ResourceEvent> alist = aresource.MyResources.ToList();//or must I add this?
foreach(ResourceEvent re in alist)// (or in aresource.MyResources)
{
//do something
}
}
}
I know I can use Include(), but assume the MyResource object comes from somewhere else where we don't know whether collection has been retrieved or not.

You can load collections this way:
context.Entry(aresource).Collection(p => p.MyResources).Load();
For single references use Reference() instead of Collection().

Accessing the navigation property will enumerate the collection which means EF will load all the entities at that time - not one by one. This is important to know because if, let's say, you want the first entity, and you write areasource.MyResources.First(), EF will load all entity objects for that collection even though you're only planning on using one. aresource.MyResources will enumerate the collection AND THEN the First() operation will be performed.
To avoid that, you want to get the IQueryable for that navigation property and build on that. For the example I mentioned, you would do the following:
context.Entry(aresource).Collection( c => p.MyResources ).Query().First()
This statement will only retrieve the one entity from the DB and not all the entities in the navigation property collection.

Entity Framework: Why do navigation properties disappear after a group by?

I retrieve a collection with the following query:
var numbers = _betDetailItem.GetBetDetailItems().Where(betDetailItem => betDetailItem.BetDetail.Bet.DateDrawing == resultToCreate.Date && betDetailItem.BetDetail.Bet.Status == 1).Where(condition);
Right there I'm able to access my navigation properties and navigate through binded info. Note how I actually use them to filter the data.
After I group the results, the navigation properties become null.
var grouped = numbers.GroupBy(p => p.BetDetail.Bet);
//Iterate through the collection created by the Grouping
foreach (IGrouping<Bet, BetDetailItem> group in grouped)
{
var details = group.Key.BetDetails; //This is what doesn't work. BetDetails is a navigation property which was accessible in the previous query.
}
Am I doing something wrong?

You are confusing LINQ to Entities and object operations.
This is LINQ to Entities:
var numbers = _betDetailItem.GetBetDetailItems().Where(betDetailItem => betDetailItem.BetDetail.Bet.DateDrawing == resultToCreate.Date && betDetailItem.BetDetail.Bet.Status == 1).Where(condition);
So is this:
var grouped = numbers.GroupBy(p => p.BetDetail.Bet);
These are object operations:
foreach (IGrouping<Bet, BetDetailItem> group in grouped)
{
var details = group.Key.BetDetails; //This is what doesn't work. BetDetails is a navigation property which was accessible in the previous query.
}
In LINQ to Entities, there is never any need to think about loading related instances. You can always refer to any property of any object. However, at some point, you want to move out of the LINQ to Entities world and into object space, because you want to work with instances of type BetDetail instead of type IQueryable<BetDetail>. This means that the Entity Framework is now required to generate SQL to retrieve data from the database. At that point, it doesn't snow which related instances you will be accessing in your code later on. Nothing in your LINQ to Entities query forces the loading of the related Bet. So unless you do something to cause it to be loaded, like use eager loading, explicit loading, or EF 4 lazy loading, it won't be loaded.
Using lazy loading (e.g., in Entity Framework 4, or in another ORM) will make this code appear to function, but it will be unnecessarily slow, due to the large number of database queries generated. A better solution would be to use eager loading or projection. This way there will be only one DB roundtrip.

Once you do a GroupBy(), you're no longer dealing with your entities -- they have been... well, grouped, so the var in var grouped = ... is now of type IEnumerable<IGrouping<.... As such, the methods available on the items in the grouped collection are the methods of the IGrouping<> interface.
You may want to OrderBy() instead of GroupBy(), depending on your need, or you'll need to iterate on two levels: iterate over each group in grouped, and over each member within each of those.
Once you are inside of a particular IGrouping<>, you should have access to the properties for which you are looking.