We Keep Coding

Problem with EF Core updating nested entities when using automapper

I am maintaining an application which uses EF Core to persist data to a SQL database.
I am trying to implement a new feature which requires me to retrieve an object from the database (Lets pretend its an order) manipulate it and some of the order lines which are attached to it and save it back into the database. Which wouldn't be a problem but I have inherited some of this code so need to try to stick to the existing way of doing things.
The basic process for data access is :
UI -> API -> Service -> Repository -> DataContext
The methods in the repo follow this pattern (Though I have simplified it for the purposes of this question)
public Order GetOrder(int id)
{
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
The service is where business logic and mapping to DTOs are applied, this is what the GetOrder method would look like :
public OrderDTO GetOrder(int id)
{
var ord = _repo.GetOrder(id);
return _mapper.Map<OrderDto>(ord);
}
So to retrieve and manipulate an order my code would look something like this
public void ManipulateAnOrder()
{
// Get the order DTO from the service
var order = _service.GetOrder(3);
// Manipulate the order
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
_service.SaveOrder(order);
}
And the method in the service which allows this to be saved back to the DB would look something like this:
public void SaveOrder(OrderDTO order)
{
// Get the original item from the database
var original = _repo.GetOrder(order.Id);
// Merge the original and the new DTO together
_mapper.Map(order, original);
_repo.Save(original);
}
Finally the repositories save method looks like this
public void Save(Order order){
_context.Update(order)
_context.SaveChanges();
}
The problem that I am encountering is using this method of mapping the Entities from the context into DTOs and back again causes the nested objects (in this instance the OrderLines) to be changed (or recreated) by AutoMapper in such a way that EF no longer recognises them as being the entities that it has just given to us.
This results in errors when updating along the lines of
InvalidOperationException the instance of ProductLine cannot be tracked because another instance with the same key value for {'Id'} is already being tracked.
Now to me, its not that there is ANOTHER instance of the object being tracked, its the same one, but I understand that the mapping process has broken that link and EF can no longer determine that they are the same object.
So, I have been looking for ways to rectify this, There are two ways that have jumped out at me as being promising,
the answer mentioned here EF & Automapper. Update nested collections
Automapper.Collection
Automapper.collection seems to be the better route, but I cant find a good working example of it in use, and the implementation that I have done doesn't seem to work.
So, I'm looking for advice from anyone who has either used automapper collections before successfully or anyone that has any suggestions as to how best to approach this.
Edit, I have knocked up a quick console app as an example, Note that when I say quick I mean... Horrible there is no DI or anything like that, I have done away with the repositories and services to keep it simple.
I have also left in a commented out mapper profile which does work, but isn't ideal.. You will see what I mean when you look at it.
Repo is here https://github.com/DavidDBD/AutomapperExample

Ok, after examining every scenario and counting on the fact that i did what you're trying to do in my previous project and it worked out of the box.
Updating your EntityFramework Core nuget packages to the latest stable version (3.1.8) solved the issue without modifying your code.

AutoMapper in fact "has broken that link" and the mapped entities you are trying to save are a set of new objects, not previously tracked by your DbContext. If the mapped entities were the same objects, you wouldn't have get this error.
In fact, it has nothing to do with AutoMapper and the mapping process, but how the DbContext is being used and how the entity states are being managed.
In your ManipulateAnOrder method after getting the mapped entities -
var order = _service.GetOrder(3);
your DbContext instance is still alive and at the repository layer it is tracking the entities you just retrieved, while you are modifying the mapped entities -
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
Then, when you are trying to save the modified entities -
_service.SaveOrder(order);
this mapped entities reach the repository layer and DbContext tries to add them to its tracking list, but finds that it already has entities of same type with same Ids in the list (the previously fetched ones). EF can track only one instance of a specific type with a specific key. Hence, the complaining message.
One way to solve this, is when fetching the Order, tell EF not to track it, like at your repository layer -
public Order GetOrder(int id, bool tracking = true) // optional parameter
{
if(!tracking)
{
return _context.Orders.Include(o=>o.OrderLines).AsNoTracking().FirstOrDefault(x=>x.Id == id);
}
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
(or you can add a separate method for handling NoTracking calls) and then at your Service layer -
var order = _repo.GetOrder(id, false); // for this operation tracking is false

EF Core: What's the most optimized way to "refetch" an entity in order to get access to FK objects during a creation of a new object?

I want to create a new entity and then aggregate properties on FK objects then save them to the newly created object. Think of populating a keywords field on the object to drive a google like search.
Currently, I've found that the only way to do this is to Add the new entity, call context.SaveChanges(), fetch the newly created entity with all of the include statements, and then use the fetched entity to create the keyword field.
This seems inefficient considering the number of DB hits that need to happen.
Other ORMs have the capability of refetching on save.
What is the most optimized way to handle this in EF Core?

What about this syntax?
context.Entry(blog)
.Collection(b => b.Posts)
.Load();
context.Entry(blog)
.Reference(b => b.Owner)
.Load();
You can load the references onto an existing Entity that way, if I'm understanding the process you're describing correctly.
See the docs for reference
https://learn.microsoft.com/en-us/ef/core/querying/related-data#explicit-loading

Track changes to collections using Entity Framework change tracker

For auditing/history purposes, I am using the Entity Framework change tracker to determine, before writing changes, what has changed and serialize the changes. I can get the changed entities by calling this.ChangeTracker.Entries() in my DbContext derivative and looking at the values for anything marked EntityState.Added, EntityState.Deleted, or EntityState.Modified. This all works great.
My problem is that this method does not work to track changes to collections of EF objects (for instance, an ICollection<Person> property on a PersonGroup object).
I'm sure the EF context must track this somehow -- how else would the database update work, after all? But is it available to me?

What you're looking for is relationship change tracking. You can find it in ObjectStateManager of the underlying ObjectContext, here is how you get all added relationships:
//you need to call DetectChanges
((IObjectContextAdapter)context).ObjectContext.DetectChanges();
var addedRelations = ((IObjectContextAdapter)context).ObjectContext
.ObjectStateManager.GetObjectStateEntries(EntityState.Added)
.Where(e=>e.IsRelationship).ToList();

It turns out you can get at the relationships with this code (assuming it's running inside your DbContext derivative):
((IObjectContextAdapter) this).ObjectContext.ObjectStateManager
.GetObjectStateEntries(EntityState.Added)
.Where(e => e.IsRelationship)
.Select(r => new {EntityKeyInfo = r.CurrentValues[0],
CollectionMemberKeyInfo = r.CurrentValues[1], r.State});
Obviously you can tweak this based on what you need and it's up to do you something useful with it. The first two CurrentValues entries represent EntityKey objects which will allow you to get the IDs of the entities in question.
If you want to deal with deleted entities this won't work and you need to use reflection. Instead of CurrentValues[0] and CurrentValues[1] you can look at the internal properties Key0 and Key1, which are defined in an internal class you can't access at compile time. This will work: r.GetType().GetProperty("Key0", BindingFlags.Instance | BindingFlags.NonPublic).Invoke(r, new object[0]). Note that this is probably not an intended use and could blow up whenever.

Remove item from collection entity framework

I have a model called 'target' retreived by entityframework with a collection addresses.
After removing all items that are not in excesting in another collection i'm saving my entity framework context.
However, when checking my database the records are still there. While my linq code made sure to remove the items from the collection.
Here is my linq code:
using (IUnitOfWork uow = _uow.CreateUnitOfWork())
{
var target = _repository.GetByBron(uow, bron.BronId);
target.Adressen.RemoveAll(x => source.Adressen.All(y => !y.Equals(x)));
//Which calls Context.SaveChanges(); inside the unit of work class
uow.Save(_logger);
}
Update: The problem is not removing my record from the collection. Its when i'm calling the save on the context. The relation record in my database is still there... nothing has been deleted... aldo it was removed from the collection.
Solved
I'm directly removing it from the context now. (with a seperated repository object)

This is very dependent upon the configuration of the relationships. See Entity Framework .Remove() vs. .DeleteObject()
Since your relationship sounds to be a many to many, you will likely need to call DeleteObject for the addresses themselves, as EF won't automatically delete the orphaned records.

Why can't you attach new objects to tracked objects using navigation references?

In short, why does this fail (myChildObject is not added to the database). Note that it works with ObjectContext:
using (var db = new dbEntities())
{
var myParentObject = db.parentObjects.First(); // this definitely exists in the db
// this **should** attach myChildObject to the context,
// and therefore add it to the db on .SaveChanges() - it doesn't!
var myChildObject = new childObject(){
parentObject = myParentObject
};
db.SaveChanges();
}
This MSDN Blog Post says
You can add a new entity to the context by hooking it up to another entity that is already being tracked. This could be by adding the new entity to the collection navigation property of another entity or by setting a reference navigation property of another entity to point to the new entity.
Surely the above code should work because myChildObject references myParentObject which is a tracked object. EF should be smart enough to figure out that it needs adding into the childObjects collection. It worked fine when I was using ObjectContext and now I'm finding that I need to rewrite all of my code to get it to work with dbContext.
To get it to work I have to rewrite it like this:
using (var db = new dbEntities())
{
var myParentObject = db.parentObjects.First(); // this definitely exists in the db
var myChildObject = new childObject();
myParentObject.childObjects.Add(myChildObject);
db.SaveChanges();
}

If you were using POCO entities with ObjectContext it worked indeed. But not because EF's change tracking worked differently than with DbContext but because the POCO entities generated by the EF 4 T4 templates contained "relationship fixup" methods.
Basically the property setter for the line parentObject = myParentObject wasn't only an object assignment but the setter included a call to a method that in the end exactly did what you are doing manually now, namely: myParentObject.childObjects.Add(myChildObject). At this point the rule "You can add a new entity to the context by hooking it up to another entity that is already being tracked" applies and myChildObject gets added to the context and inserted into the database.
For the T4 templates that generate POCO entities for DbContext those fixup methods have been removed because they were causing trouble in other scenarios. Especially when lazy loading is involved your reference assignment and the automatic call of myParentObject.childObjects... in the property setter would trigger lazy loading on the collection and load all childObjects first that are already stored for myParentObject before the new child is added to the collection. If those are thousands this is a huge unnecessary overhead, performance gets disastrous and the reason for a suddenly bad performance (just because you assigned a single reference property) isn't easy to detect if you are not aware of the fixup methods that run behind the scenes.
Here and here and here and here are examples about the confusion that relationship fixup methods were causing.
You could modify the T4 templates and add relationship fixup methods again - or if you are using Code-First just write them by hand in your entity classes - to get the old behaviour back. But this might be more complex than and at least as much work as changing your existing code the way you've outlined in your last code snippet - which I would certainly prefer over having those bothersome fixup methods back.

#theyetiman you're doing a little interpretation mistake of the blog text.
see:
[...]
or by setting a reference navigation property of another entity to point to the new entity.
In this part the blog said you can set a reference navigation property of a tracked object with a new entity.
like this:
[tracked entity].NavigationProperty = [new entity];
But you tring to do:
[new entity].Navigation Property = [tracked entity];
This not works. If your childObject was tracked and parentObject not you would be able to add parentObject setting it in childObject property, but the opposite is not true.