We Keep Coding

Problem with EF Core updating nested entities when using automapper

I am maintaining an application which uses EF Core to persist data to a SQL database.
I am trying to implement a new feature which requires me to retrieve an object from the database (Lets pretend its an order) manipulate it and some of the order lines which are attached to it and save it back into the database. Which wouldn't be a problem but I have inherited some of this code so need to try to stick to the existing way of doing things.
The basic process for data access is :
UI -> API -> Service -> Repository -> DataContext
The methods in the repo follow this pattern (Though I have simplified it for the purposes of this question)
public Order GetOrder(int id)
{
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
The service is where business logic and mapping to DTOs are applied, this is what the GetOrder method would look like :
public OrderDTO GetOrder(int id)
{
var ord = _repo.GetOrder(id);
return _mapper.Map<OrderDto>(ord);
}
So to retrieve and manipulate an order my code would look something like this
public void ManipulateAnOrder()
{
// Get the order DTO from the service
var order = _service.GetOrder(3);
// Manipulate the order
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
_service.SaveOrder(order);
}
And the method in the service which allows this to be saved back to the DB would look something like this:
public void SaveOrder(OrderDTO order)
{
// Get the original item from the database
var original = _repo.GetOrder(order.Id);
// Merge the original and the new DTO together
_mapper.Map(order, original);
_repo.Save(original);
}
Finally the repositories save method looks like this
public void Save(Order order){
_context.Update(order)
_context.SaveChanges();
}
The problem that I am encountering is using this method of mapping the Entities from the context into DTOs and back again causes the nested objects (in this instance the OrderLines) to be changed (or recreated) by AutoMapper in such a way that EF no longer recognises them as being the entities that it has just given to us.
This results in errors when updating along the lines of
InvalidOperationException the instance of ProductLine cannot be tracked because another instance with the same key value for {'Id'} is already being tracked.
Now to me, its not that there is ANOTHER instance of the object being tracked, its the same one, but I understand that the mapping process has broken that link and EF can no longer determine that they are the same object.
So, I have been looking for ways to rectify this, There are two ways that have jumped out at me as being promising,
the answer mentioned here EF & Automapper. Update nested collections
Automapper.Collection
Automapper.collection seems to be the better route, but I cant find a good working example of it in use, and the implementation that I have done doesn't seem to work.
So, I'm looking for advice from anyone who has either used automapper collections before successfully or anyone that has any suggestions as to how best to approach this.
Edit, I have knocked up a quick console app as an example, Note that when I say quick I mean... Horrible there is no DI or anything like that, I have done away with the repositories and services to keep it simple.
I have also left in a commented out mapper profile which does work, but isn't ideal.. You will see what I mean when you look at it.
Repo is here https://github.com/DavidDBD/AutomapperExample

Ok, after examining every scenario and counting on the fact that i did what you're trying to do in my previous project and it worked out of the box.
Updating your EntityFramework Core nuget packages to the latest stable version (3.1.8) solved the issue without modifying your code.

AutoMapper in fact "has broken that link" and the mapped entities you are trying to save are a set of new objects, not previously tracked by your DbContext. If the mapped entities were the same objects, you wouldn't have get this error.
In fact, it has nothing to do with AutoMapper and the mapping process, but how the DbContext is being used and how the entity states are being managed.
In your ManipulateAnOrder method after getting the mapped entities -
var order = _service.GetOrder(3);
your DbContext instance is still alive and at the repository layer it is tracking the entities you just retrieved, while you are modifying the mapped entities -
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
Then, when you are trying to save the modified entities -
_service.SaveOrder(order);
this mapped entities reach the repository layer and DbContext tries to add them to its tracking list, but finds that it already has entities of same type with same Ids in the list (the previously fetched ones). EF can track only one instance of a specific type with a specific key. Hence, the complaining message.
One way to solve this, is when fetching the Order, tell EF not to track it, like at your repository layer -
public Order GetOrder(int id, bool tracking = true) // optional parameter
{
if(!tracking)
{
return _context.Orders.Include(o=>o.OrderLines).AsNoTracking().FirstOrDefault(x=>x.Id == id);
}
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
(or you can add a separate method for handling NoTracking calls) and then at your Service layer -
var order = _repo.GetOrder(id, false); // for this operation tracking is false

DDD repositories with EF explicitly loading

I'm starting to get my head into Domain Driven Design and I'm having some issues with the repositories and the fact that EF Core explicitly loading will automatically fill my navigational properties.
I have a repository that I use to load my aggregate root and its children. However, some of the aggregate children need to be loaded later on (I need to load those entities based on a date range).
Example:
Load schedule owners
Calculate a date range
Load schedule owner's schedules
I'm trying to keep my data access layer isolated from the core layer and this is where I have some questions.
Imagine this method on my repository:
public List<Schedule> GetSchedules(Guid scheduleOwnePk, DateRange dateRange)
{
var schedules = dbContext.Schedules.Where(x => x.PkScheduleOwner == scheduleOwnerPk && x.StartDate >= dateRange.Start && x.EndDate <= dateRange.End).ToList();
return schedules;
}
I can call this method from the core layer in two ways:
//Take advantage of EF core ability to fill the navigational property automatically
scheduleOwnerRepository.GetSchedules(scheduleOwner.Pk, dateRange)
or
var schedules = scheduleOwnerRepository.GetSchedules(scheduleOwner.Pk, dateRange);
//At this moment EF core already loaded the navigational property, so I need to clear it to avoid duplicated results
scheduleOwner.Schedules.Clear();
//Schedules is implemented as an IEnumerable to protect it from being changed outside the aggregator root
scheduleOwner.AddSchedules(schedules);
The problem with the first approach is that it leaks EF core to the core layer, meaning that the property ScheduleOwner.Schedules will no longer be filled if I move away from EF core.
The second approach abstracts EF core but requires some extra steps to get ScheduleOwner.Schedules filled. Since EF core will automatically load the navigational property after the repository method is called, I'm forced to clear it before adding the results, otherwise I'll be inserting duplicated results.
How do you guys deal with this kind of situation? Do you take advantage of EF core features or do you follow the more natural approach of calling a repository method and use its results to fill some property?
Thanks for the help.

There are a couple of things to consider here.
Try to avoid using your domain model for querying. Rather use a read model through a query layer.
An aggregate is a complete unit as it were so when loaded you load everything. When you run into a scenario where you do not need all of the related data it may indicate that the data is not part of the aggregate but it may, in fact, only be related in a weaker sense.
An example is Order to Customer. Although an Order may very well require a Customer the Order is an aggregate in its own right. The Customer may have a list of OrderIds but that may become large rather quickly. One would typically not require a complete list of orders to determine whether an aggregate is valid or complete. However, you may very well need a list of ActiveOrder value objects of sorts if that is required for, say, keep a maximum order amount although there are various ways to deal with that case also.
Back to your scenario. An EF entity is not your domain model and when I have had to make use of EF in the past I would load the entity and then map to my domain entity in the repository. The repository would only deal with domain aggregates and you should avoid query methods on the repository. As a minimum a repository would typically have at least a Get(id) and a Save(aggregate) method.
I would recommend querying using a separate layer that returns as simple a result as possible. For something like a Count I may return an int whereas something like IScheduleQuery.Search(specification) I may return IEnumerable<DataRow> or, if it contains more complex data or I have a need for a read model I may return IEnumerable<Query.Schedule>.

Determine if the context of an entity proxy has been disposed

In a EF 6 project, I am writing validation functions for entities. some are static while others are instance methods of the entities themselves.
Ignoring whether this is bad practice or not, I'd like to check whether the entities were created using a context and if so, whether they are still attached.
Please note that these functions do NOT have access to the context object, just the entity classes.
As an example, a method validates Department entity and cascades validation to all associated Department.Employee instances.
If the hierarchy was created manually, validation will succeed.
If the hierarchy was created using a context which is still alive, validation will succeed albeit slower.
If the hierarchy was created using a context which has been disposed, validation will fail with an ObjectDisposedException (provided proxy-creation was enabled and .Include(***) was not used).
So the question, is it possible to detect the above scenarios without access to a DbContext instance? If not, how can we best validate entire hierarchies irrespective of how they were created.
var result = true;
var departments = ???; // Constructed manually or through a DbContext instance.
foreach (var department in departments)
{
result &= department.Validate();
foreach (var employee in department.Employees)
{
result &= employee.Validate();
}
}
EDIT: Please note that this is for a desktop application that cannot have long-running DbContext instances. they are almost always disposed immediately after retrieving data. Re-querying the database does not seem a viable option for validation since it is triggered by trivial user input and would slow down the entire user experience.

From your question
Please note that these functions do NOT have access to the context object, just the entity classes.
two solutions come to mind, none really palatable:
Build your own tracker and make it available to these methods somehow.
Add something to your entities, for example a WasLoaded property that gets set when you query your context. That WasLoaded could be set by either
Writing an EF interceptor that sets it.
Adding an artificial bit column with all values set to 1. Then map that to the property; the property will be false if you constructed it outside of the context, true if loaded from the context.
The tracker seems to be the cleanest because it doesn't pollute your model. The interceptor is a decent alternative if you're not concerned about your model.
And while it doesn't answer your question directly, you could avoid the use of proxies, in which case your validation works the same way regardless because you have your model in memory. There's the usual trade-offs to consider though.
I'm not sure how you'd detect the last scenario. I suppose you could have your tracker track more than the entities... have it also track the context's state.

Linq-to-Sql, Use of entity classes and parent/child objects

For my project I use SqlCE 3.5, Linq2Sql and generate entity classes (dbml file) with SqlMetal.
If I use extensions of the entity classes as my business object classes, how shall I deal with child objects?
Can I use the existing EntitySet(Of ChildClass) that is already in the dbml file? Or should I create a new collection property in my partial class extension, like:
public partial class ParentClass
{
public List<ChildClass> children { get; set; }
}
I got this vague assumption that EntitySet(Of ChildClass) has some type of direct connection to the complete database table. And that I perhaps should use this as purely a data access object, and for business logic, I should keep this other collection object that can hold a subset of the table only.
But I'm not sure if I have totally misunderstood the concept? I really like to know how this should be used properly..?
EDIT1:
One thing that possibly is a performance issue for me is when binding my object collections to a datagridview. A few properties are updated quite frequently from a stock-exchange data feed, and I wonder if using the EntitySets as the business layer collection makes this slow. Though I do not call submitchanges on those updates. Still I'm not able to scroll the datagridview, due to all the updates. May this be due to using the EntitySets, or just the datagridview itself updating so frequently that scrolling will seem impossible?

If you are using the entities generated by the Designer, which are Proxy and not real POCO objects you can also use their ObjectSet to get the child colletions. They are of type IQueryable so until you touch one of the item or you use a ToList() or .Where() LinQ extension, EF won't call the database.
Anyway, the BEST PRACTICE is to expose them using an IList and use a Repository to fill the List. The ObjectSet is not only a List, it is also a sort of repository for the entity.

Add/Update a list of entities using the entity framework

I'm returning a List (own class) from Silverlight to a service, in this list there are old entities with updated values and completely new entities, how do I save this back to the database using the entity framework?
Using cEnts.CardItems.AddObject gives me duplicates obviously.

You want to use Attach() instead of AddObject().
Attach will take your disconnect object and let the container know to consider it for updates. The new objects, without a PrimaryKey, will be added.

If you are using the same entity context for selecting and update/insert you have to call AddTo...() method to insert the new entities and ApplyPropertyChanges to the changed ones.
If you are using different contexts the problem is more complicated because you have to detach entities from one context and attach them to another. Once detached entities lose their changed state and you have to explicitly specify which properties have been changed (For more info check this: http://www.abadjimarinov.net/blog/2009/12/13/AttachAlreadyChangedObjectToADataContextInEntityFramework.xhtml ).