I am working on a project where we are using Entity Framework 6.1.3. Right now we are experiencing pretty big performance issues when adding a child object to a list of the parent entity (see code sample below).
We are using lazy-loading, so what I noticed is that everything works fine until we call _parent.Children.Add(child); since it seems to load all children from the database just to be able to add a new one. Since some of our parent objects have about 50,000 children, this is delaying this simple insert call by 7-8 seconds and sometimes even causing timeouts.
To me it doesn't really make sense for Entity Framework to load all children just in order to add one, so is there a way I can avoid this or is this an Entity Framework design-flaw and should we find a workaround?
I'd obviously like to find a solution for this and would prefer not having to implement pure ADO queries for this one problem.
Thanks!
public class Parent
{
public Guid Id { get; set; }
public virtual ICollection<Child> Children { get; set; }
}
public class Child
{
public Guid Id { get; set; }
}
public class ParentAggregate
{
private readonly Parent _state;
public ParentAggregate(Parent state)
{
_state = state;
}
public void AddChild(Guid id)
{
var child = new Child { Id = id };
_state.Children.Add(child);
}
}
To me it doesn't really make sense for Entity Framework to load all children just in order to add one
The lazy loading occurs the first time you access a navigation property through its getter. And the sample code
_parent.Children.Add(child);
consists of two operations:
(1) retrieve the Children property (through the property getter!):
var children = _parent.Children;
(2) perform some operation on it (call Add method in this case):
children.Add(child);
The lazy loading occurs because of the operation (1). As you can see, EF has nothing to do with that because it has no control over it. And there is no way to know what you are going to do with that property value - enumerate it, take a count or use Add, Remove etc. methods.
Here are some solutions.
First, why using lazy loading at all? It has so many side effects and inefficiencies, and all they can easily be solved by EF provided out of the box eager loading via Include methods. That's why by default EF Core ("the future of EF") does not use lazy loading by default and requires a special package and procedure for enabling it.
Second, if you insist using lazy loading, then you have the following two options:
(A) Disable lazy loading during the data modifications (requires access to/control of the DbContext instance):
dbContext.Configuration.LazyLoadingEnabled = false;
_parent.Children.Add(child);
dbContext.Configuration.LazyLoadingEnabled = true;
This also requires the collection property to be initialized in order to avoid NRE.
(B) Use explicit backing field and provide some direct access to it (to avoid triggering the lazy load by property accessor). For instance:
public class Parent
{
public Guid Id { get; set; }
private ICollection<Child> children;
public virtual ICollection<Child> Children { get => children; set => children = value; }
public void Add(Child child)
{
// use the backing field directly
if (children == null) children = new HashSet<Child>();
children.Add(child);
}
}
Related
I`m gettng the following error in EF Core: No backing field could be found for property 'ParentId' of entity type 'Child' and the property does not have a getter.
This is my configuration for the Child entity:
// create shadow property for Parent
builder.Property<int>("ParentId").IsRequired();
// make shadow property and foreign key the PK as well
// i know this may not make the most sense here
// in this scenario but it's what I want to do.
builder.HasKey("ParentId");
// configure FK
builder.HasOne(o => o.Parent)
.WithOne()
.HasForeignKey<Child>("ParentId");
And my entities:
public class Parent
{
public int Id { get; set; }
}
public class Child
{
public Parent Parent { get; private set; }
public void SetParent(Parent p) => Parent = p;
}
The error occurs when I call dbContext.Children.Contains(myChild):
var child = new Child();
child.Parent = new Parent();
dbContext.Children.Add(child);
dbContext.SaveChanges();
// works fine
Assert.True(dbContext.Children.Any());
// throws InvalidOperationException
Assert.True(dbContext.Children.Contains(myChild)):
If I add the shadow property as a real property to the model as such:
public class Child
{
public int ParentId { get; set;}
public Parent Parent { get; private set; }
public void SetParent(Parent p) => Parent = p;
}
then everything works. But I'd like to keep it a shadow property if possible.
Update:
Just checked the EF Core 3.1.7 released 3 days ago, and the exception is gone, which means it has been reported/identified and fixed. Hence you can simply upgrade to that version.
Original:
The configuration of the shadow property is fine. Unfortunately you are hitting one of the EF Core 3.x bugs.
In order to translate Contains method to SQL, EF Core has to convert it to PK equality comparison, something like (in pseudo code)
dbContext.Children.Any(c => PK(c) == PK(myChild))
which can be seen from the exception stack trace, and apparently is failing to do so when the PK is shadow property.
I've checked the latest at this time EF 5 preview and it appears to be fixed (the issue is gone). So until it gets released, the workaround is to replace implicit entity equalty comparisons (e.g. Contains(entity), Equals(entity), == entity etc.) with the corresponding explicit PK comparisons.
In this case, instead of
dbContext.Children.Contains(child)
you can use either
dbContext.Children.Any(c => c.Parent.Id == child.Parent.Id)
(worse SQL translation due to another v3.x defect)
or
dbContext.Children.Any(c => EF.Property<int>(c, "ParentId") == child.Parent.Id)
(proper SQL translation as in v5.0, but uses "magic" strings and requires explicitly specifying the shadow property name and type)
public class Child
{
public int ParentId{get;set;}
public Parent Parent { get; private set; }
public void SetParent(Parent p) => Parent = p;
}
you should define your foreign key variable and remember that private set doesn't create relationship in db
I have 2 models, one of which has a child collection of the other:
[Table("ParentTable")]
public class Parent
{
[Key, Column("Parent")]
public string Id { get; set; }
[Column("ParentName")]
public string Name { get; set; }
public virtual ICollection<Widget> Widgets { get; set; }
}
[Table("WidgetTable")]
public class Widget
{
public string Year { get; set; }
[Column("Parent")]
public string ParentId { get; set; }
public string Comments { get; set; }
[Key, Column("ID_Widget")]
public int Id { get; set; }
[ForeignKey("ParentId"), JsonIgnore]
public virtual Parent Parent { get; set; }
}
This code works for > 99% of widgets:
var parent = _dbContext.Parents.FirstOrDefault(p => p.Id == parentId);
Usually, parent.Widgets is a collection with more than one item. In a couple of instances, however, parent.Widgets is null (not a collection with no items).
I have used Query Analyzer to trace both the query for the parent and the query for widgets belonging to that parent. Both return exactly the rows I expect; however, the model for one or two parent IDs results in a null value for the Widgets collection. What could cause a lazy-loaded collection to be null in some instances but not others?
This situation commonly comes up when a dbContext lifetime is left open across an Add, saveChanges, and then retrieval.
For example:
var context = new MyDbContext(); // holding Parents.
var testParent = new Parent{Id = "Parent1", Name = "Parent 1"};
context.Parents.Add(testParent);
At this point if you were to do:
var result = context.Parents.FirstOrDefault(x=> x.ParentId == "Parent1");
you wouldn't get a parent. Selection comes from committed state.. So...
context.SaveChanges();
var result = context.Parents.FirstOrDefault(x=> x.ParentId == "Parent1");
This will return you a reference to the parent you had inserted since the context knows about this entity and has a reference to the object you created. It doesn't go to data state. Since your definition for Widgets was just defined with a get/set auto-property the Widgets collection in this case will be #null.
if you do this:
context.Dispose();
context = new MyDbContext();
var result = context.Parents.FirstOrDefault(x=> x.ParentId == "Parent1");
In this case the parent is not known by the new context so it goes to data state. EF will return you a proxy list for lazy loading the Widgets, which there are none so you get back an empty list, not #null.
When dealing with collection classes in EF it's best to avoid auto-properties or initialize them in your constructor to avoid this behaviour; you'll typically want to assign Widgets after creating a Parent. Initializing a default member is better because you don't want to encourage ever using a setter on the collection property.
For example:
private readonly List<Widget> _widgets = new List<Widget>();
public virtual ICollection<Widget> Widgets
{
get { return _widgets; }
protected set { throw new InvalidOperationException("Do not set the Widget collection. Use Clear() and Add()"); }
}
Avoid performing a Set operation on a collection property as this will screw up in entity reference scenarios. For instance, if you wanted to sort your Widget collection by year and did something like:
parent.Widgets = parent.Widgets.OrderBy(x=> x.Year).ToList();
Seems innocent enough, but when the Widgets reference was an EF proxy, you've just blown it away. EF now cannot perform change tracking on the collection.
Initialize your collection and you should avoid surprises with #null collection references. Also I would look at the lifetime of your dbContext. It's good to keep one initialized over the lifetime of a request or particular operation, but avoid keeping them alive longer than necessary. Context change tracking and such consume resources and you can find seemingly intermittent odd behaviour like this when they cross operations.
I'm trying to create a function that generically loads the related child entities with a filter.
All my entities are derived from my own Base Class "BusinessObject"
public abstract class BusinessObject : BaseObject, IBaseObject, ILocalObject
{
[Browsable(false)]
[Key]
public int ID { get; set; }
[Browsable(false)]
public int? HqID { get; set; }
private bool _deleted;
[Browsable(false)]
public bool Deleted
{
get { return _deleted; }
set { CheckPropertyChanged(ref _deleted, value); }
}
}
I have created the following function that when supplied an entity will load all the related child objects.
When defining my entities, all child collections are flagged by my own attribute "EntityChildCollectionAttribute" so I can easily find the collections I want to load.
public virtual void OnLoadEntityChildren(object entity)
{
var propNames = entity.GetPropertyNames();
foreach (var propName in propNames.Where(propName => entity.PropertyHasCustomAttribute(propName, typeof(EntityChildCollectionAttribute))))
{
MyData.Entry(entity).Collection(propName).Load();
}
}
This works lovely!
My Problem comes when I want to filter the child collection.
In this case I want to only load child entities where Deleted == false.
I cannot work out how to do this!
I have had many attempts and replacing MyData.Entry(entity).Collection(propName).Load(); with
MyData.Entry(entity).Collection(propName).Query().Cast<BusinessObject>().Where(x=>x.Deleted.Equals(false)).Load();
compiles but then I get the error;
"Unable to cast the type 'FmOrderProcessing.Entities.OpDocumentDetail' to type 'FwBaseEntityFramework.BusinessObject'. LINQ to Entities only supports casting EDM primitive or enumeration types."
Any Help/Pointers/Answers will be gratefully received
Thanks in advance
Lance
I was implementing a "Soft Delete" pattern which means the records in the database are flagged as deleted rather than removed (for audit and replication purposes).
All entities are derived from a base definition with a bool Deleted property.
I found the answer here:
https://www.nuget.org/packages/EntityFramework.DynamicFilters
This package allows the definition of global filters in the data context.
I fixed my issue with one line of code in the OnModelCreating override.
modelBuilder.Filter("Deleted", (IBaseObject d) =>d.Deleted, false);
The filter function is applied globally to any entity presenting (in my case) the IBaseObject interface.
I hope this helps any body else with a similar issue
How can I set a collection to modified in the same way that I would do
_context.Entry(thing).Property(x => x.MyProperty).isModified = true;
like:
_context.Entry(thing).Collection(x => x.MyCollection).isModified = true;
EDIT: The purpose of this, is that my collection is a list of objects stored in a lookup table. I will only have a list of stubs with their id's in this collection and I would like to update the relationships without messing with the audit values and whatever else is contains within the lookup objects.
For instance, a contact will have multiple contact types, which for whatever reason are complex objects in this scenario. I want to be able to add and remove types using only the FKs and let EF handle the relationship fixups.
public class Contact
{
public int Id {get;set;}
public list<ContactTypes> ContactTypes {get;set;}
//audit values/other properties
}
public class ContactType
{
public int Id {get;set;}
public string Value {get;set;}
}
context.Entry represents a single entity, never a collection. So you have to loop through the collection and mark each entity as modified.
If you have a list of ForeignKey objects, you probably know how frustrating it is to force EF's Relationship Fixup on them. Here's slick way to do that.
public void SetContactTypesToUnchanged(Contact contact)
{
contact.ContactTypes.Each(type => _context.Entry(type).State = EntityState.Unchanged);
_context.SaveChanges();
}
In nearly all examples of POCO classes created for Entity Framework 4.1 collections are defined using the ICollection interface:
public class TravelTicket
{
public virtual int Id { get; set; }
public string Destination { get; set; }
public virtual ICollection<Person> Members { get; set; }
}
However, this causes an issue in my code where I need to access a member of the collection by index, e.g.:
Person Paul = TravelTicket.Members[3];
Cannot apply indexing with [] to an expression of type 'System.Collections.Generic.ICollection
So how do I get round this problem and should I always use ICollection for my POCO collections?
It is because once you mark your navigation property virtual the proxy of your entity is created and it uses HashSet for navigation properties - hash set doesn't allow indexing. Accessing related entities by index doesn't seem like good approach because retrieving entity from database doesn't ensure that related entities will always have the same index in the collection.
Just use ToList():
Person Paul = TravelTicket.Members.ToList()[3];
EF isn't going to query data until you actually try to access it - and a collection doesn't try until you iterate it, while ToList must instantiate each instance.
Even better, be more specific:
Person Paul = TravelTicket.Members.Where(m=>m.Id == 3); // or some such similar filter
Then you only instance ONE Member - the one you want.
Note you may need Members.AsQueryable().Where instead - I never remember...
ICollection implements IEnumerable, you should be able to get the item by index using Enumerable.ElementAt<TSource> method