We Keep Coding

Problem with EF Core updating nested entities when using automapper

I am maintaining an application which uses EF Core to persist data to a SQL database.
I am trying to implement a new feature which requires me to retrieve an object from the database (Lets pretend its an order) manipulate it and some of the order lines which are attached to it and save it back into the database. Which wouldn't be a problem but I have inherited some of this code so need to try to stick to the existing way of doing things.
The basic process for data access is :
UI -> API -> Service -> Repository -> DataContext
The methods in the repo follow this pattern (Though I have simplified it for the purposes of this question)
public Order GetOrder(int id)
{
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
The service is where business logic and mapping to DTOs are applied, this is what the GetOrder method would look like :
public OrderDTO GetOrder(int id)
{
var ord = _repo.GetOrder(id);
return _mapper.Map<OrderDto>(ord);
}
So to retrieve and manipulate an order my code would look something like this
public void ManipulateAnOrder()
{
// Get the order DTO from the service
var order = _service.GetOrder(3);
// Manipulate the order
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
_service.SaveOrder(order);
}
And the method in the service which allows this to be saved back to the DB would look something like this:
public void SaveOrder(OrderDTO order)
{
// Get the original item from the database
var original = _repo.GetOrder(order.Id);
// Merge the original and the new DTO together
_mapper.Map(order, original);
_repo.Save(original);
}
Finally the repositories save method looks like this
public void Save(Order order){
_context.Update(order)
_context.SaveChanges();
}
The problem that I am encountering is using this method of mapping the Entities from the context into DTOs and back again causes the nested objects (in this instance the OrderLines) to be changed (or recreated) by AutoMapper in such a way that EF no longer recognises them as being the entities that it has just given to us.
This results in errors when updating along the lines of
InvalidOperationException the instance of ProductLine cannot be tracked because another instance with the same key value for {'Id'} is already being tracked.
Now to me, its not that there is ANOTHER instance of the object being tracked, its the same one, but I understand that the mapping process has broken that link and EF can no longer determine that they are the same object.
So, I have been looking for ways to rectify this, There are two ways that have jumped out at me as being promising,
the answer mentioned here EF & Automapper. Update nested collections
Automapper.Collection
Automapper.collection seems to be the better route, but I cant find a good working example of it in use, and the implementation that I have done doesn't seem to work.
So, I'm looking for advice from anyone who has either used automapper collections before successfully or anyone that has any suggestions as to how best to approach this.
Edit, I have knocked up a quick console app as an example, Note that when I say quick I mean... Horrible there is no DI or anything like that, I have done away with the repositories and services to keep it simple.
I have also left in a commented out mapper profile which does work, but isn't ideal.. You will see what I mean when you look at it.
Repo is here https://github.com/DavidDBD/AutomapperExample

Ok, after examining every scenario and counting on the fact that i did what you're trying to do in my previous project and it worked out of the box.
Updating your EntityFramework Core nuget packages to the latest stable version (3.1.8) solved the issue without modifying your code.

AutoMapper in fact "has broken that link" and the mapped entities you are trying to save are a set of new objects, not previously tracked by your DbContext. If the mapped entities were the same objects, you wouldn't have get this error.
In fact, it has nothing to do with AutoMapper and the mapping process, but how the DbContext is being used and how the entity states are being managed.
In your ManipulateAnOrder method after getting the mapped entities -
var order = _service.GetOrder(3);
your DbContext instance is still alive and at the repository layer it is tracking the entities you just retrieved, while you are modifying the mapped entities -
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
Then, when you are trying to save the modified entities -
_service.SaveOrder(order);
this mapped entities reach the repository layer and DbContext tries to add them to its tracking list, but finds that it already has entities of same type with same Ids in the list (the previously fetched ones). EF can track only one instance of a specific type with a specific key. Hence, the complaining message.
One way to solve this, is when fetching the Order, tell EF not to track it, like at your repository layer -
public Order GetOrder(int id, bool tracking = true) // optional parameter
{
if(!tracking)
{
return _context.Orders.Include(o=>o.OrderLines).AsNoTracking().FirstOrDefault(x=>x.Id == id);
}
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
(or you can add a separate method for handling NoTracking calls) and then at your Service layer -
var order = _repo.GetOrder(id, false); // for this operation tracking is false

Linq to entities lazy loading

I have the following class generated by entity framework:
public partial class Branch
{
public short Id { get; set; }
public short CompanyId { get; set; }
public string Code { get; set; }
public string Title { get; set; }
public virtual Company Ts_Companies { get; set; }
}
I have the following method which takes all of the branches out of the database:
public Branch[] LoadBranches(int companyId, int page, int limit, string search, string sort, string sortOrder)
{
using (var dbContext = new TimeShedulerEntities())
{
var _branches = (from ct in dbContext.Branches
where ct.Title.Contains(search) || ct.Code.Contains(search)
select ct).OrderBy(c => c.Title).Skip((page - 1) * limit).Take(limit);
return _branches.ToArray();
}
}
In my model designer I see that the Lazy Loading is set to true, but when I iterate over the branches, the property Ts_Companies is null. Also I get the following exception:
An exception of type 'System.ObjectDisposedException' occurred in
EntityFramework.dll but was not handled in user code
Additional information: The ObjectContext instance has been disposed
and can no longer be used for operations that require a connection.
Am I forgetting something?

You created and disposed of the context during your function since it was inside the using statement. Each entity happens to know from which context it was created so that lazy loading is possible.
When you accessed the Ts_Companies property, the entity realized that it had not yet loaded that property since it is probably a navigation property and attempted to ask its ObjectContext (TimeShedulerEntities) to load that property. However, the context had been disposed and so that it what caused that exception.
You need to modify your query as follows to 'pre-load' the Ts_Companies:
var _branches = (from ct in dbContext.Branches.Include("Ts_Companies")
where ct.Title.Contains(search) || ct.Code.Contains(search)
select ct).OrderBy(c => c.Title).Skip((page - 1) * limit).Take(limit);
It will take possibly quite a bit longer to load depending on the size of the Ts_Companies object and how many you end up bringing back at once, but the entity will stop asking its object context to load the Ts_Companies since you would have already loaded them.
A side note: I have found that creation and disposal of object context on a per-method basis causes problems when the entities are passed outside the function. If you want to create and destroy the object context in every function, you probably want to have the function return something that is not an entity. In other words, have an object that can be constructed from an entity and has the properties you need, but don't have it reference the entity. In java these are often called Data Transfer Objects (DTOs). You lose the read-write ability of entity framework, but you don't have unexpected ObjectDisposedExceptions flying all over the place.
The problem comes when you ask an entity to be associated with another (for example, adding on entity to a ICollection property of another entity) when they come from different objectcontexts. This will cause headaches for you since you would have to manually attach the objects to the same context before performing that operation. Additionally, you lose the ability to save changes to those entities without manually attaching them to a different context.
My opinion on how I would do it:
I've found it easier to either have an object containing all of these database access functions control the lifetime of the context (i.e. have your containing object be IDisposable and during disposal, destroy the context) or simply not return entities and have the datastore be read-old, write-new essentially without any modification ability.
For example, I have my object (I will call it my data access object) with a bunch of methods for getting database objects. These methods return entities. The data access object also has a SaveChanges method which simply calls the context's SaveChanges method. The data access object contains the context in a protected property and keeps it around until the data access object itself is disposed. Nobody but the data access object is allowed to touch its context. At that point, the context is disposed by manually calling 'Dispose'. The data access object could then used inside a using statement if that is your use case.
In any case, it is probably best to avoid passing entities attached to a context outside the scope in which their context exists since entity framework keeps references to that context all over the place in the individual entities

But you didn't load your Ts_Companies, use Eager Loading instead:
var _branches = dbContext.Branches
.Where(b => b.Title.Contains(search) || b.Code.Contains(search))
.Include("Ts_Companies")
.OrderBy(c => c.Title)
.Skip((page - 1) * limit)
.Take(limit);
And I came across the same issue before System.ObjectDisposedException, in my MVC project and I didn't use using blocks,instead I define my context on class level.If I need to return and use an array (in my View) I use that context.If I need to just update some information then I have used using blocks.I hope this helps.

ASP.NET C# Entity Framework - How to update Foreign Key properly?

I'M quite new to this EF but I think I'M making progress. Anyway, it appears I do NOT know how to Update an object that is RELATED by a Foreign Key.
My DbRelation is:
And I'm trying to UPDATE the one members LANGUAGEID and here is the Context I invoke:
public class ManagerBase
{
private static NoxonEntities _entities = null;
public NoxonEntities Entities
{
get
{
if (_entities == null)
_entities = new NoxonEntities();
return _entities;
}
}
}
There are many things I have tried. Here is one:
1)
MemberManager currentMemberManager = new MemberManager();
var Mem = currentMemberManager.MyEntities.Member.SingleOrDefault(c => c.Id == 2);
var Lang = currentLanguageManager.Entities.Language.SingleOrDefault(c => c.Id == 1);
Mem.Language = Lang;
//Or
Mem.LanguageId = Lang.Id;
currentMemberManager.Save(Mem);
In appreach 1, I get an error like
The changes to the database were committed successfully, but an error occurred while updating the object context. The ObjectContext might be in an inconsistent state. Inner exception message: A referential integrity constraint violation occurred: The property values that define the referential constraints are not consistent between principal and dependent objects in the relationship.
2)
//Managers uses ManagerBase class as a Base class
MemberManager currentMemberManager = new MemberManager();
currentMemberManager.Save(Globals.CurrentMember.Id, Globals.CurrentLanguage.Id);
//These Global objects coming from a Http Session
//You may also say not to keep whole member object in session which I'll not after I figure this out
Here is my SAVE method and where I have the actual problem:
public void Save(Member entity)
{
var Data = base.Entities.Member.First(c => c.Id == entity.Id);
if (Data != null)
{
Data = entity;
base.Entities.SaveChanges();
}
}
}
In appreach 1, I get an error in this code in EF model edmx file
[EdmScalarPropertyAttribute(EntityKeyProperty=false, IsNullable=false)]
[DataMemberAttribute()]
public global::System.Int32 LanguageId
{
get
{
return _LanguageId;
}
set
{
OnLanguageIdChanging(value);
ReportPropertyChanging("LanguageId");
_LanguageId = StructuralObject.SetValidValue(value);
ReportPropertyChanged("LanguageId");
OnLanguageIdChanged();
}
}
And the error is
Object reference not set to an instance of an object.
Clearly I'm in the wrong way with EF.
Could you please help me and show how to properly Update a relation Id (ForeignKeyId) ?
Thank you so much.

Yes, you are using EF in many wrong ways. First, as Arran points out, you should never make your data context static. I know that seems easier, but it's a huge problem because data contexts are designed to be created and destroyed frequently.
If you don't, then the object graph continues to grow until the app pool is eventually exhausted, plus you can run into all kinds of problems with concurrent access. Static objects are shared between all threads, so imagine if two users are using your app at the same time, they will both be using the same data context and would stomp all over each other.
Second, you're using a singleton, which is one of the most reviled patterns there are, for a lot of reasons. They have their uses, but they're far more rare than most people use them.
Third, based on the error messages, it sounds like your data model may not be in sync with your EF model, and thus it's getting confused and throwing exceptions. Have you made changes to your database structure without updating your ef model? Bear in mind that regenerating doesn't always update everything, and sometimes you have to either update it manually, or delete your objects and re-add them.
Fourth, your real problem (trust me, the ones I've laid out are also real problems, that WILL bite you in the rear at some point) lies in this code:
var Data = base.Entities.Member.First(c => c.Id == entity.Id);
if (Data != null)
{
Data = entity;
base.Entities.SaveChanges();
}
The first thing you have to realize is that EF returns tracked objects, these objects have references in EF and track the changes that occur. What you are doing here is getting an object, and then throwing it away and replacing it with a non-tracked object that was passed in.
You have to update the object returned from the First method, and not replace it with a new one.

How do I update Entity Framework objects without changing ALL instances of that object?

I have this code which uses Entity Framework 4.1 to access database entities:
public string Test()
{
Navigation nav = db.Navigations.FirstOrDefault();
List<Navigation> lNav = db.Navigations.ToList();
foreach (var item in lNav)
{
item.Label += " [Edited]";
}
return nav.Label;
}
When I run this in asp.net mvc it returns this:
News [Edited]
I expected it to return:
News
Because I thought my foreach would only modify the contents of lNav. Instead, it seems to modify all instances of the entity objects.
How can I modify lNav without also modifying nav?

Try AsNoTracking():
Navigation nav = db.Navigations.AsNoTracking().FirstOrDefault();
List<Navigation> lNav = db.Navigations.AsNoTracking().ToList();
This way loaded entities are not attached to the context and two different instances should be created for the first entity. By using AsNoTracking you disable the identity mapping between key values of an entity and object references which is responsible for the behaviour you observed. (There can always only be one entity reference with a given key in the context and EF does not create a new object if you load an entity with the same key. Instead it returns the object which already exists in the context.)
Be aware that you cannot use this if you intent to update entities with the help of EFs change tracking mechanism. AsNoTracking() is designed for readonly scenarios.

Attaching and detaching entities from context correctly in EF4.1

I am trying to implement caching mechanism for entities. And to use the entities correctly and seamlessly with the caching i need to detach the entity from the current context before i put it in a cache and attach it back the the new context when i get it from the cache. (My context lifetime is per http request)
The requirements are that -
All the navigational properties that are associated with it (which i have already populated) should not be removed when the entity is detached.
I can update the cached items if i want ( so correctly attaching them to the new context is important).
This is my attempt at creating an EntityCache class - (ServerCache here is my wrapper class that pushes the object to ASP.NET cache)
public static class EntityCache
{
private static DbContext context
{
get
{
return (DbContext)HttpContext.Current.Items[ObjectKeys.ContextRequestItemKey];
}
}
private static void Detach(object entity)
{
var trackedEntity = entity as IEntityWithChangeTracker;
trackedEntity.SetChangeTracker(null);
((IObjectContextAdapter)context).ObjectContext.Detach(entity);
}
private static void Attach(object entity)
{
((IObjectContextAdapter)context).ObjectContext.Attach((IEntityWithKey)entity);
}
public static void Remove(string key)
{
ServerCache.Remove(key);
}
public static object Get(string key)
{
object output = ServerCache.Get(key);
if (output != null)
Attach(output);
return output;
}
public static void ShortCache(String key, object data)
{
if (data != null)
{
Detach(data);
ServerCache.ShortCache(key, data);
}
}
public static void LongCache(String key, object data)
{
if (data != null)
{
Detach(data);
ServerCache.LongCache(key, data);
}
}
}
When i put an entity in the cache it is of type DynamicProxy and NOT the real class.
Attaching doesnt work at all - i get an exception that i cannot case object that is of type Dynamic_{blahblah} to IEntityWithKey.
I just saw these examples of attach and detach online and tried them, I am open to any new implementation of the Attach/Detach methods here.
Thank you.
Follow up question -
context.Entry(entity).State = EntityState.Detached;
Works, but makes all the navigational properties that are loaded NULL, how do we make it keep the navigational properties and NOT replace(or lose) them with NULL when we detach from context.

IEntityWithKey is interface for other types of entities. It is for "big" entities. For example EntityObject implement this interface. These entities are not considered as POCO and are not supported by DbContext API.
If you want to use IEntityWithKey your classes must implement it - it is not something that would happen automatically.
Correct attaching with DbContext API should be:
dbContext.Set(typeof(entity)).Attach(entity);
and this should hopefully also work:
dbContext.Entry(entity).State = EntityState.Unchanged;
Correct detaching with DbContext API should be:
dbContext.Entry(entity).State = EntityState.Detached;
Also it is better to you generic methods instead of object.

To your follow-up question:
...how do we make it keep the navigational properties and NOT
replace(or lose) them with NULL when we detach from context...
I believe it's not possible to detach an object graph from the context while maintaining it's navigation properties. From MSDN:
In an independent association, the relationship information is not
maintained for a detached object.
Although this statement is about independent associations it does not mean that navigation properties are maintained in foreign key assocation (relationships which expose a foreign key property in the model). Yes, "Relationship information" is maintained because the foreign key properties (which are scalar properties) will be alive and contain the correct foreign key value after detaching. But the corresponding navigation properties will still be null for reference properties or, for navigation collections the reference will be removed from the collection.
I think the only way to detach a complete object graph from a context is either disposing the context altogether or create a copy of the graph before starting to detach the original graph. Creating a copy would require writing Clone methods which copy all the properties and navigate through the graph or using a "trick" like this which serializes the graph into a binary stream and then deserializes it back to new objects. For this the entities would need to be serializable. Also reference cycles (which we often have when using bidirectional navigation properties between entities) can cause trouble. (Also attention if your objects are proxies which contain references to EF's internal objects and which you probably don't want to copy, serialize and deserialize.)
In this aspect Detach is not the counterpart of Attach because it behaves quite differently: Attach attaches the whole object graph and maintains navigation properties. Detach detaches only single entities without the related objects and destroys navigation properties. From the same page linked above:
Detach only affects the specific object passed to the method. If the
object being detached has related objects in the object context, those
objects are not detached.
I could imagine that this is the main reason why the DbContext API doesn't have an explicite Detach method (in contrast to ObjectContext) - detaching is considered as an advanced feature which does not behave as one might expect.
MSDN mentions as the only reason to detach an object from the context "to conserve resources" (again the article above):
In Entity Framework applications, you can detach objects from the
object context. You might detach objects to conserve resources, as
executing repeated queries in the same object context increases the
memory requirements of the object context.
I think this method is just not prepared and designed for working with the objects after they've been detached from the context. It's main purpose is to release them for immediate garbage collection.