We Keep Coding

Problem with EF Core updating nested entities when using automapper

I am maintaining an application which uses EF Core to persist data to a SQL database.
I am trying to implement a new feature which requires me to retrieve an object from the database (Lets pretend its an order) manipulate it and some of the order lines which are attached to it and save it back into the database. Which wouldn't be a problem but I have inherited some of this code so need to try to stick to the existing way of doing things.
The basic process for data access is :
UI -> API -> Service -> Repository -> DataContext
The methods in the repo follow this pattern (Though I have simplified it for the purposes of this question)
public Order GetOrder(int id)
{
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
The service is where business logic and mapping to DTOs are applied, this is what the GetOrder method would look like :
public OrderDTO GetOrder(int id)
{
var ord = _repo.GetOrder(id);
return _mapper.Map<OrderDto>(ord);
}
So to retrieve and manipulate an order my code would look something like this
public void ManipulateAnOrder()
{
// Get the order DTO from the service
var order = _service.GetOrder(3);
// Manipulate the order
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
_service.SaveOrder(order);
}
And the method in the service which allows this to be saved back to the DB would look something like this:
public void SaveOrder(OrderDTO order)
{
// Get the original item from the database
var original = _repo.GetOrder(order.Id);
// Merge the original and the new DTO together
_mapper.Map(order, original);
_repo.Save(original);
}
Finally the repositories save method looks like this
public void Save(Order order){
_context.Update(order)
_context.SaveChanges();
}
The problem that I am encountering is using this method of mapping the Entities from the context into DTOs and back again causes the nested objects (in this instance the OrderLines) to be changed (or recreated) by AutoMapper in such a way that EF no longer recognises them as being the entities that it has just given to us.
This results in errors when updating along the lines of
InvalidOperationException the instance of ProductLine cannot be tracked because another instance with the same key value for {'Id'} is already being tracked.
Now to me, its not that there is ANOTHER instance of the object being tracked, its the same one, but I understand that the mapping process has broken that link and EF can no longer determine that they are the same object.
So, I have been looking for ways to rectify this, There are two ways that have jumped out at me as being promising,
the answer mentioned here EF & Automapper. Update nested collections
Automapper.Collection
Automapper.collection seems to be the better route, but I cant find a good working example of it in use, and the implementation that I have done doesn't seem to work.
So, I'm looking for advice from anyone who has either used automapper collections before successfully or anyone that has any suggestions as to how best to approach this.
Edit, I have knocked up a quick console app as an example, Note that when I say quick I mean... Horrible there is no DI or anything like that, I have done away with the repositories and services to keep it simple.
I have also left in a commented out mapper profile which does work, but isn't ideal.. You will see what I mean when you look at it.
Repo is here https://github.com/DavidDBD/AutomapperExample

Ok, after examining every scenario and counting on the fact that i did what you're trying to do in my previous project and it worked out of the box.
Updating your EntityFramework Core nuget packages to the latest stable version (3.1.8) solved the issue without modifying your code.

AutoMapper in fact "has broken that link" and the mapped entities you are trying to save are a set of new objects, not previously tracked by your DbContext. If the mapped entities were the same objects, you wouldn't have get this error.
In fact, it has nothing to do with AutoMapper and the mapping process, but how the DbContext is being used and how the entity states are being managed.
In your ManipulateAnOrder method after getting the mapped entities -
var order = _service.GetOrder(3);
your DbContext instance is still alive and at the repository layer it is tracking the entities you just retrieved, while you are modifying the mapped entities -
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
Then, when you are trying to save the modified entities -
_service.SaveOrder(order);
this mapped entities reach the repository layer and DbContext tries to add them to its tracking list, but finds that it already has entities of same type with same Ids in the list (the previously fetched ones). EF can track only one instance of a specific type with a specific key. Hence, the complaining message.
One way to solve this, is when fetching the Order, tell EF not to track it, like at your repository layer -
public Order GetOrder(int id, bool tracking = true) // optional parameter
{
if(!tracking)
{
return _context.Orders.Include(o=>o.OrderLines).AsNoTracking().FirstOrDefault(x=>x.Id == id);
}
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
(or you can add a separate method for handling NoTracking calls) and then at your Service layer -
var order = _repo.GetOrder(id, false); // for this operation tracking is false

EF resolve navigation property after manual attach

I'm using Entity Framework. I've attached a POCO object representing an entity in the DB to my dbcontext using:
var entity = new MyEntity() { ID = 1, AnotherItemID = 10 };
context.Set<T>().Attach(entity);
So far so good. I can access the set and work with the entity I've added. It's added in the Unchanged state. However, it is only a POCO and not a Proxy. Therefore, when I try to access a navigation property, e.g. myEntity.AnotherItem, I just get a null back.
Does anyone know if there is a way to have EF resolve navigation properties for POCO classes attached in this way? Or of a way to cast the POCO to a proxy class?
Thanks
Update
There are two ways to solve this (of course there may be others too!). One is the Explicit Loading option in the answer below. The other way, which allows lazy loading to work, is to use the DBSet Create method rather than the POCO new keyword when creating entities to be attached. More info about that here:
EF4.3 Code-First, MVC, Lazy Loading After Attaching in POST Action

You can use Explicity Loading:
//When you want to load a reference navigation property
context.Entry(entity).Reference(p => p.AnotherItem).Load();
//When you want to load a collection navigation property
context.Entry(post).Collection(p => p.Items).Load();

Entity Framework DBSet Include without proxies

I am trying to create a method that accesses my Entity Framework (database first) context using reflection for a REST web service. I have gotten as far as getting the table, and converting it to a list for returns, but I am running into trouble when I try to use Include to get some of the related tables.
I have a couple tables that I am testing with, they are Project and Person. Project has a reference to Person for the person who manages the project and a reference back from Person to Project for all the projects a person manages. In order to get the original return to work I added
Configuration.ProxyCreationEnabled = false;
Configuration.LazyLoadingEnabled = false;
to my context's class so that the referential loop is removed and the JSON serialization works correctly.
The problem I am facing is that I am now trying to explicitly get all the projects, and the person record related to the project, without including the list of the person's projects. But when I try to include people, I get the JSON serialization error because it is pulling back the circular reference. I currently have the following code:
Entities context = new Entities();
// Normally these will be a parameters to the calling method
string tableName = "Projects";
string includeTableName = "Person";
System.Reflection.PropertyInfo propertyInfo = context.GetType().GetProperty(tableName);
Type type = propertyInfo.PropertyType;
dynamic list = propertyInfo.GetValue(context);
var include = typeof(QueryableExtensions).GetMethod("Include", new[] { typeof(IQueryable), typeof(string) });
var toList = typeof(Enumerable).GetMethod("ToList").MakeGenericMethod(type.GenericTypeArguments[0]);
list = include.Invoke(null, new object[] { list, includeTableName });
return toList.Invoke(null, new object[] { list });
The code executes properly but then I make the call I get the following error:
"Message":"An error has occurred.","ExceptionMessage":"Self referencing loop detected with type 'DDBAPI.EntityFramework.Project'. Path '[8].Person.Projects'."
Is there anyway with Include to stop it from loading the circular reference? I saw similar questions that referenced making DTOs to limit what gets pulled into the return but since normally I will not know what table I will be calling against, I will not know which DTO I would need as I am trying to avoid any logic based around the table names being passed in.

Removing the proxy creation doesn't solve the circular reference problem. It has nothing to do with it.
The proxies are simply created to handle the change tracking, but they have the same properties of the original entities.
Your problem is that there is a navigation property from table Person to Project and viceversa. That's the circular reference, and you could only break it if you removed the navigation property in one of the tables, i.e. the Person property in Project entity, or the Projects property in Person entity.
Most probably you don't want to do this. So, what you need to do is to instruct the serializer so that it can handle the circular references. I suppose you're using JSON.NET, which is the current JSON serializer by default. I do also suppose you're using Web API. If that's the case, you can find the JSON.NET serializer settings like this:
JsonSerializerSettings serializerSettings = GlobalConfiguration
.Configuration.Formatters.JsonFormatter.SerializerSettings;
Then you need to choose one of these two options:
ReferenceLoopHandling = ReferenceLoopHandling.Ignore. Docs here.
PreserveReferencesHandling = PreserveReferencesHandling.Objects Docs here.
There is still another solution: instead of removing one of the navigation properties responsible for the circular reference, you can instruct JSON.NET to not serialize it by decorating it with [JsonIgnore] attribute. Docs here.

EF Code First - Get POCO from Proxy

Question:
Is it possible when dealing with dynamic proxy to pull out the underlying POCO when we need to serialize them?
Rationale:
I have a requirement to serialize (XML) my POCO entities using EF Code First but quickly found that DbContext creates dynamic proxy for my POCO and that makes it hard to serialize.
I have tried the following:
Disable proxy creation in the DbContext and work only with pure POCO. This allows me to serialize the instances any way I like. The only catch is that navigational properties are not being tracked, therefore, I must attach all related entities manually when I want to save otherwise new entities will always be created (see code example).
Implement ISerializable interface on the POCO to manually handle serialization. This is a lot of work and is not a sustainable solution.
Code example.
// Attach and update tags
foreach (var tag in entity.Tags)
{
Context.Entry(tag).State = Context.Tags.Any(t => t.ID == tag.ID)
? EntityState.Modified
: EntityState.Added;
}
// Attach and update state.
Context.Entry(entity).State = Context.Resources.Any(x => x.ID == entity.ID)
? EntityState.Modified
: EntityState.Added;
As you can imagine, the complexity can get out of hand when my entity have more relationships.

Is it possible when dealing with dynamic proxy to pull out the
underlying POCO when we need to serialize them?
No because there is no underlying POCO - the proxy is not a wrapper around entity instance. It is entity instance directly.
You can use DataContractSerializer and ProxyDataContractResolver to serialize proxied POCOs but serializing proxied entities sounds like you are trying to serialize entities with lazy loading enabled - that can serialize much more than you expect because every property will be lazy loaded recursively until there is no single non-loaded navigation property in the whole object graph.
You must also handle circular references when using DataContractSerializer by marking your entities with [DataContract(IsReference = true)] and every serializable property with [DataMember].
The only catch is that navigational properties are not being tracked
Entities without proxy are tracked as well. The tracking is dependent on entity being attached not entity being proxied.
I must attach all related entities manually when I want to save
You must always attach deserialized entities if you want to persist them.

Why Does Entity Framework Code First Change my Type Names?

I generated a bunch of classes using EF 4.1 Power Toys to reverse engineer my database. My classes and maps look good and work well, but when I check the name of a type that is returned, it appears that EF has added a GUID to the type name. So a method with this for the body:
var context = new DbContext();
var myVehicle = context.Vehicles.First();
return myVehicle.GetType().Name;
...would return something like:
"Vehicle_F31E8AC6EB21A3220F761E7C2FFEB3B433CEFC7CF567E9A0CF53E8F4B598C2B9"
Why is this and is there any way to turn it off?

It is called dynamic proxy. When you query the type from entity framework for the first time it will dynamically create class derived from your entity type and return it instead. The name you see is the name of that derived class.
The reason why entity framework does this is to support some advanced features like lazy loading or dynamic change tracking. It can be turned off by calling:
context.Configuration.ProxyCreationEnabled = false;