We Keep Coding

EF Core is lazy loading when I try to eager load with .Include()

I'm using EF Core 2.2.6 (database first) and it seems like just having lazy loading enabled keeps me from being able to eager load. Does enabling lazy loading preclude using eager loading in any capacity?
namespace Example.Models
{
public class Lead
{
public int Id { get; set; }
public LeadOrganization LeadOrganization { get; set; }
public Lead(ExampleContext.Data.Lead dbLead)
{
Id = dbLead.Id;
LeadOrganization = new LeadOrganization(dbLead.LeadOrganization);
}
public static Lead GetLead(int id)
{
using (var db = new ExampleContext())
{
var dbLead = db.Leads
.Include(l => l.LeadOrganization)
.ThenInclude(lo => lo.LeadOrganizationAddresses)
.ThenInclude(loa => loa.AddressType)
.FirstOrDefault(l => l.Id== id);
return new Lead(dbLead);
}
}
}
}
namespace Example.Models
{
public class LeadOrganization
{
public IEnumerable<LeadOrganizationAddress> Addresses { get; set; }
public LeadOrganization(ExampleContext.Data.LeadOrganization dbLeadOrganization)
{
Addresses = dbLeadOrganization.LeadOrganizationAddresses.Select(loa => new LeadOrganizationAddress(loa));
}
}
}
namespace Example.Models
{
public class LeadOrganizationAddress
{
public AddressType AddressType { get; set; }
public LeadOrganizationAddress(ExampleContext.Data.LeadOrganizationAddress dbLeadOrganizationAddress)
{
AddressType = new AddressType(dbLeadOrganizationAddress.AddressType);
}
}
}
namespace Example.Models
{
public class AddressType
{
public short Id { get; set; }
public AddressType(ExampleContext.Data.AddressType dbAddressType)
{
Id = dbAddressType.Id;
}
}
}
The ExampleContext.Data namespace contains the EF-generated partial classes from the database. Lead, LeadOrganization, LeadOrganizationAddress, and AddressType are classes that are basically 1:1 with the partials in terms of properties, but with static methods added (yes it's weird, but it's what I have to work with).
A Lead has a LeadOrganization, which in turn has at least one LeadOrganizationAddress, which in turn has an AddressType.
When GetLead calls the Lead constructor, the data from the query has not loaded, even though it should be eager loaded. This leads to problems down the line of nested objects. When it eventually gets to the LeadOrganizationAddress constructor, the DbContext has been disposed, and thus can't lazy load the associated AddressType.
Am I misunderstanding the whole point of eager loading? I thought it would retrieve all the data upon the initial query, letting me then pass that to the constructor without issue. I shouldn't need to keep going back to the database and lazy load anything.
Can you simply not eager load if you have lazy loading enabled? Is there some other workaround like forcing it to load any proxied entities?

Ok, after investigating the issue, there is a problem with EF Core 2.x lazy loading via proxies implementation. The related tracked issues are
#15170: Eager loading include with using UseLazyLoadingProxies
#12780: Store IsLoaded flags in proxy for better lazy-loading experience
The problem is that the navigation properties are eager loaded, but LazyLoader does not know that when disposed - can't safely access context change tracker and simply is throwing exception. The relevant code can be seen here, in the very first line:
if (_disposed)
{
Logger.LazyLoadOnDisposedContextWarning(Context, entity, navigationName);
}
As I read it, it's supposed to be fixed in EF Core 3.0 when released with the following "breaking change" - Lazy-loading proxies no longer assume navigation properties are fully loaded. It also partially explains the current problem:
Old behavior
Before EF Core 3.0, once a DbContext was disposed there was no way of knowing if a given navigation property on an entity obtained from that context was fully loaded or not.
Unfortunately this doesn't help you with the current problem. The options I see are:
Wait for EF Core 3.0 release
Don't use lazy loading via proxies
Turn off the lazy loading on disposed context warning - by default it is Throw, change it to Log or Ignore, for instance:
optionsBuilder.ConfigureWarnings(warnings => warnings
.Log(CoreEventId.LazyLoadOnDisposedContextWarning)
);

I assume you use UseLazyLoadingProxies() but want to disable the lazy loading for specific Includes in your queries. This is not implemented yet:
https://github.com/aspnet/EntityFrameworkCore/issues/10787
The only thing what you can do right now:
1.) Disable the lazy-loading proxy ("default lazy loading for all properties")
2.) Then use (manual implemented) lazy-loading for specific properties, for example in one of your cases:
public class LeadOrganization
{
private ILazyLoader _lazyLoader { get; set; }
private IEnumerable<LeadOrganizationAddress> _addresses;
public LeadOrganization(ILazyLoader lazyLoader)
{
_lazyLoader = lazyLoader;
}
public IEnumerable<LeadOrganizationAddress> Addresses
{
get => _addresses;
set => _addresses = value;
}
public IEnumerable<LeadOrganizationAddress> AddressesLazy
{
get
{
_lazyLoader?.Load(this, ref _addresses);
}
set => this._addresses = value;
}
}
So for eager-loading use .Include(lo=>lo.Addresses), for lazy-loading use .Include(lo=>lo.AddressesLazy)
Edit 1
IMO lazy loading shouldn't be enabled per default for all properties - this could impact the performance of your whole implementation. So the solution above is one alternative in cases where lazy loading brings you advantages. I also would like to have this option in every include, something like .Include(o=>o.Addresses, LoadingBehaviour.Eager) - maybe this will exist in the future.

Lazy loading isn't what's preventing the instantiation of your properties, lacking a proper constructor in them is.
EF Core aside, it's very strange that those types, e.g. LeadOrganization need to be passed in an instance of themselves in their constructor. It's kind of a chicken and egg problem – how do you create the first one?
public class LeadOrganization
{
public IEnumerable<LeadOrganizationAddress> Addresses { get; set; }
public LeadOrganization(ExampleContext.Data.LeadOrganization dbLeadOrganization)
{
Addresses = dbLeadOrganization.LeadOrganizationAddresses.Select(loa => new LeadOrganizationAddress(loa));
}
}
In any case, EF Core only supports simple constructors with parameters based on convention (basically a 1-1 mapping of parameters to properties), so it doesn't know how to instantiate and hydrate those nested classes, eager or lazy.
I suggest making those constructors parameterless, or if you want EF to hydrate the objects via properties, at least add a parameterless constructor to your classes.

EF tables with Many to many relationship, Will this consume memory?

I have 2 tables that saved family members, when I use include to retrieve the family members, the generated T-SQL is what I'm expected, but when I see the result from VS, like image below, it's look like never ending.
My questions:
It's this normal?
Should I avoid include when the relationship becomes complex?
If it is normal, will this very memory consumption?
POCO
public class Cust_ProfileTbl
{
[Key]
public long bintAccountNo { get; set; }
public string nvarCardName { get; set; }
public string varEmail { get; set; }
public virtual ICollection<Cust_ProfileFamilyTbl> profileFamilyParents { get; set; }
public virtual ICollection<Cust_ProfileFamilyTbl> profileFamilyChildren { get; set; }
}
public class Cust_ProfileFamilyTbl
{
[Key]
public int intProfileFamily { get; set; }
public long bintAccountNo { get; set; }
public long bintAccountNoMember { get; set; }
public virtual Cust_ProfileTbl custProfileParent { get; set; }
public virtual Cust_ProfileTbl custProfileChild { get; set; }
}
LINQ
var rs = from family in context.member.Include("profileFamilyParents.custProfileChild")
select family;
rs = rs.Where(x => x.bintAccountNo.Equals(1));
var result = rs.ToList();
In onModelCreating
modelBuilder.Entity<Cust_ProfileFamilyTbl>()
.HasRequired(m => m.custProfileParent)
.WithMany(t => t.profileFamilyParents)
.HasForeignKey(m => m.bintAccountNo)
.WillCascadeOnDelete(false);
modelBuilder.Entity<Cust_ProfileFamilyTbl>()
.HasRequired(m => m.custProfileChild)
.WithMany(t => t.profileFamilyChildren)
.HasForeignKey(m => m.bintAccountNoMember)
.WillCascadeOnDelete(false);

When people use an ORM like EF in their application, many times the application design gets driven by this ORM and the entities defined in its model. When the app is a simple "CRUD" application, that's not a problem, but an advantage, because you spare a lot of time.
However when things start to get more complicated, an "ORM guided design" becomes a problem. This looks to be the case.
There are at least two problems, recovered from the comments:
the data retrieved from the DB is more than needed
in this case, because of some particular relationships between entities, there is a circular reference, which creates an endless loop and a stack overflow when trying to show the model in the view
When this kind of situation shows up, the most advisable is to break the tight tie between the ORM and the rest of the app, which can be dine by defining a new class, and projecting the data into it. Let's give a generic ProfileDto name.
public class ProfileDto { ... }
DTO is a generic name for this kind of classes: Data Transfer Objects - but, when they have specific purposes, they can get other names like view models, when they're going to be used as the model sent to an MVC view
And then, what you need to do is to project the result of the query into the DTO:
var model = theQuery.Select(i => new ProfileDto { a = i.a, b = i.b...}).ToList();
With a good design of the Dto you'll only recover the needed data from the DB, and you'll avoid the loop problem (by not including the navigation property that creates the loop).
NOTE: many times people uses mappers, like AutoMapper or ValueInjecter to make the mapping, or part of the mapping, automatic
Code standardization is a very good idea until it becomes a source of problems. The main purpose of writing code is implementing the business logic. If code standardization, technology, or whatever, makes it harder to implement business logic, instead of contributing to the solution, they become a problem, so you need to avoid them.

Mapping you created is Normal but use of Include depends upon its usage
Use of Include depends on situation of use for example if you want to cache it in memory then you may use include, Where as if you are using only showing properties of Cust_ProfileTbl
class in some grid and on click you want show details of Cust_ProfileFamilyTbl then you might don't want to use include. But be careful if you are using Automapper or something because when It will try to map related properties it will query database.
It will consume memeory when you execute ToList() as doing so you are Loading query result into List collection. Where as If you again want to query the result then you can use ToQueryable() or just want to iterate the you can don't load them to List.

Telling EF 6 to Ignore a Private Property

I'm using Entity Framework 6.0.2 to map some simple hand-coded models to an existing database structure. The primary model at the moment is:
public class Occurrence
{
public int ID { get; set; }
public Guid LegacyID { get; set; }
public string Note { get; set; }
public virtual ICollection<OccurrenceHistory> History { get; set; }
}
(The OccurrenceHistory model isn't really relevant to this, but that part is working fine whereby EF loads up the child records for this model.)
The mapping is simple, and I try to be as explicit as I can be (since as the application grows there will be some less-intuitive mapping):
public class OccurrenceMap : EntityTypeConfiguration<Occurrence>
{
public OccurrenceMap()
{
ToTable("Occurrence");
HasKey(o => o.ID);
Property(o => o.ID).IsRequired().HasColumnName("ID");
Property(o => o.LegacyID).IsRequired().HasColumnName("LegacyID");
Property(o => o.Note).IsUnicode().IsOptional().HasColumnName("Note");
}
}
But if I add a private property to the model, EF tries to map it to the database. Something like this:
private OccurrenceHistory CurrentHistory { get; set; }
(Internal to the model I would have some logic for maintaining that field, for other private operations.) When EF generates a SELECT statement it ends up looking for a column called CurrentHistory_ID which of course doesn't exist.
I can make the property public and set the mapping to ignore it:
Ignore(o => o.CurrentHistory);
But I don't want the property to be public. The model is going to internally track some information which the application code shouldn't see.
Is there a way to tell EF to just ignore any and all private members? Even if it's on a per-map basis? I'd particularly like to do this without having to add EF data annotations to the models themselves, since that would not only be a bit of a leaky abstraction (persistence-ignorant models would then have persistence information on them) but it would also mean that the domain core assembly which holds the models would carry a reference to EntityFramework.dll everywhere it goes, which isn't ideal.

A colleague pointed me to a blog post that led to a very practical approach.
So what I have is a private property:
private OccurrenceHistory CurrentHistory { get; set; }
The core of the problem is that I can't use that in my mapping:
Ignore(o => o.CurrentHistory);
Because, clearly, the property is private and can't be accessed in this context. What the blog post suggests is exposing a public static expression which references the private property:
private OccurrenceHistory CurrentHistory { get; set; }
public static readonly Expression<Func<Occurrence, OccurrenceHistory>> CurrentHistoryExpression = o => o.CurrentHistory;
I can then reference that in the mapping:
Ignore(Occurrence.CurrentHistoryExpression);
As with anything, it's a mix of pros and cons. But in this case I think the pros far outweigh the cons.
Pros:
The domain core assembly doesn't need to carry a reference to EntityFramework.dll.
The persistence mapping is entirely encapsulated within the DAL assembly.
Cons:
Models need to expose a little information about their inner workings.
The con breaks encapsulation, but only slightly. Consuming code still can't access that property or its value on instances, it can only see that the property exists statically. Which, really, isn't a big deal, since developers can see it anyway. I feel that the spirit of encapsulation is still preserved on any given instance of the model.

Serialization of Entity Framework objects with One to Many Relationship

I am attempting to use EF with Code First and the Web API. I don't have any problems until I get into serializing Many-to-Many relationships. When I attempt to execute the following web api method below I get the following error message:
public class TagsController : ApiController
{
private BlogDataContext db = new BlogDataContext();
// GET api/Tags
public IEnumerable<Tag> GetTags()
{
return db.Tags.AsEnumerable();
}
}
I get the following error:
'System.Data.Entity.DynamicProxies.Tag_FF17EDDE6893000F7672649A39962DB0CA591C699DDB73E8C2A56203ED7C7B6D'
with data contract name
'Tag_FF17EDDE6893000F7672649A39962DB0CA591C699DDB73E8C2A56203ED7C7B6D:http://schemas.datacontract.org/2004/07/System.Data.Entity.DynamicProxies'
is not expected. Consider using a DataContractResolver or add any
types not known statically to the list of known types - for example,
by using the KnownTypeAttribute attribute or by adding them to the
list of known types passed to DataContractSerializer.
I have read some SO articles (article 1, article 2) that the fix is to add the following attribute:
[DataContract (IsReference=true)]
but this has had no effect. Also using [IgnoreDataMember] does not have an effect. The only option that does seem to work is to set Configuration.ProxyCreationEnabled to false. Is this my only option? Am I missing something?
Sample POCO objects:
Tag
[DataContract(IsReference = true)]
public class Tag
{
public Tag()
{
this.Blogs = new HashSet<Blog>();
}
[Key]
[DataMember]
public int Id { get; set; }
[DataMember]
public string Name { get; set; }
[IgnoreDataMember]
public virtual ICollection<Blog> Blogs { get; set; }
}
Blog
[DataContract(IsReference = true)]
public class Blog
{
public Blog()
{
this.Tags = new HashSet<Tag>();
}
[Key]
[DataMember]
public int Id { get; set; }
[DataMember]
public string Name { get; set; }
[IgnoreDataMember]
public virtual ICollection<Tag> Tags { get; set; }
}

When you see an object like:
System.Data.Entity.DynamicProxies.Tag_FF17EDDE6893000F7672649A39962DB0CA591C699DDB73E8C2A56203ED7C7B6D
It is a runtime EF Generated version of a proxy to what would normally be considered a POCO object.
Entity Framework has created this object because it tracks when the objects has changed so when you call .SaveChanges() it can optimize what to do. The downfall of this is that you aren't actually using the specific object you defined, thus Data Contracts and Frameworks (Json.net) cannot use them as they would your original POCO object.
To Prevent EF from returning this object you have two choices (ATM):
First, Try turning off Proxy object creation on your DbContext.
DbContext.Configuration.ProxyCreationEnabled = false;
This will completely disable the create of Proxy objects for every query to the specific DbContext. (This does not affect the cached object in the ObjectContext).
Secondly, use EntityFramework 5.0+ with AsNoTracking()
(ProxyCreationEnabled is still available in EF 5.0 as well)
You should also be able to
DbContext.Persons.AsNoTracking().FirstOrDefault();
or
DbContext.Persons.
.Include(i => i.Parents)
.AsNoTracking()
.FirstOrDefault();
Instead of globally disabling proxy creation for the DbContext, this only turns it off per query. (This DOES affect the cached object in the ObjectContext, it is not cached)

I wanted to leave proxy creation inteact, and found that using the ProxyDataContractResolver seemed to resolve the issue for me. See msdn for a reference on how to use it in wcf, which isn't exactly WebAPI, but should get you going along the right path.

Entity Framework - Multiple Project support

I am looking into migrate a large project to Entity Framework 4.0 but am not sure if it can handle my inheritance scenario.
I have several projects that inherit from an object in the “main” project. Here is a sample base class:
namespace People
{
public class Person
{
public int age { get; set; }
public String firstName { get; set; }
public String lastName { get; set; }
}
}
and one of the sub-classes:
namespace People.LawEnforcement
{
public class PoliceOfficer : People.Person
{
public string badgeNumber { get; set; }
public string precinct { get; set; }
}
}
And this is what the project layout looks like:
People - People.Education - People.LawEnforcement http://img51.imageshack.us/img51/7293/efdemo.png
Some customers of the application will use classes from the People.LawEnforcement and other users will use People.Education and some will use both. I only ship the assembles that the users will need. So the Assembles act somewhat like plug-ins in that they add features to the core app.
Is there anyway in Entity Framework to support this scenario?
Based on this SO question I'm think something like this might work:
ctx.MetadataWorkspace.LoadFromAssembly(typeof(PoliceOfficer).Assembly);
But even if that works then it seams as if my EDMX file will need to know about all the projects. I would rather have each project contain the metadata for the classes in that project but I'm not sure if that is possible.
If this isn't possible with entity framework is there another solution (NHibernate, Active Record, etc.) that would work?

Yes this is possible, using the LoadFromAssembly(..) method you've already found.
... but it will only work if you have an specialized model (i.e. EDMX) for each distinct type of client application.
This is because EF (and most other ORMs) require a class for each entity in the model, so if some clients don't know about some classes, you will need a model without the corresponding entities -- i.e. a customized EDMX for each scenario.
To make it easier to create a new model for each client application, if I was you I'd use Code-Only following the best practices laid out on my blog, to make it easy to grab only the fragments of the model you need actually need.
Hope this helps
Alex

Alex is correct (+1), but I'd strongly urge you to reconsider your model. In the real world, a police officer is not a subtype of a person. Rather, it's an attribute of that person's employment. I think programmers frequently tend to over-emphasize inheritance at the expense of composition in object oriented design, but it's especially problematic in O/R mapping. Remember that an object instance can only ever have one type. When that object is stored in the database, the instance can only have that type for as long as it exists, across multiple application sessions. What if a person had two jobs, as a police officer and a teacher? Perhaps that scenario is unlikely, but the general problem is more common than you might expect.
More relevant to your question, I think you can solve your actual problem at hand by making your mapped entity model more generic, and your application-specific data projections on the entities rather than entities themselves. Consider entities like:
public class JobType
{
public Guid Id { get; set; }
// ...
}
public class Job
{
public JobType JobType { get; set; }
public string EmployeeNumber { get; set; }
}
public class Person
{
public EntityCollection<Job> Jobs { get; set; }
}
Now your law enforcement app can do:
var po = from p in Context.People
let poJob = p.Jobs.Where(j => j.JobType == JobType.PoliceOfficerId).FirstOrDefault()
where poJob != null
select new PoliceOfficer
{
Id = p.Id,
BadgeNumber = poJob.EmployeeNumber
};
Where PoliceOfficer is just a POCO, not a mapped entity of any kind.
And with that you've achieved your goal of having a common data model, but having the "job type specific" elements in separate projects.