We Keep Coding

Entity Framework Instance tracking error with mapping sub-objects - is there an elegant solution?

Some 2 years+ ago I asked this question which was kindly solved by Steve Py.
I am having a similar but different problem now when mapping with sub-objects. I have had this issue a few times and worked around it, but facing doing so again, I can't help thinking there must be a more elegant solution. I am coding a memebership system in Blazor Wasm and wanting update membership details via a web-api. All very normal.
I have a library function to update the membership:
public async Task<MembershipLTDTO> UpdateMembershipAsync(APDbContext context, MembershipLTDTO sentmembership)
{
Membership? foundmembership = context.Memberships.Where(x =>x.Id == sentmembership.Id)
.Include(x => x.MembershipTypes)
.FirstOrDefault();
if (foundmembership == null)
{
return new MembershipLTDTO { Status = new InfoBool(false, "Error: Membership not found", InfoBool.ReasonCode.Not_Found) };
}
try
{
_mapper.Map(sentmembership, foundmembership, typeof(MembershipLTDTO), typeof(Membership));
//context.Entry(foundmembership).State = EntityState.Modified; <-This was a 'try-out'
context.Memberships.Update(foundmembership);
await context.SaveChangesAsync();
sentmembership.Status = new InfoBool(true, "Membership successfully updated");
return sentmembership;
}
catch (Exception ex)
{
return new MembershipLTDTO { Status = new InfoBool(false, $"{ex.Message}", InfoBool.ReasonCode.Not_Found) };
}
}
The Membership object is an EF DB object and references a many to many list of MembershipTypes:
public class Membership
{
[Key]
public int Id { get; set; }
...more stuff...
public List<MembershipType>? MembershipTypes { get; set; } // The users membership can be several types. e.g. Employee + Director + etc..
}
The MembershipLTDTO is a lightweight DTO with a few heavy objects removed.
Executing the code, I get an EF exception:
The instance of entity type 'MembershipType' cannot be tracked because another instance with the same key value for {'Id'} is already being tracked. When attaching existing entities, ensure that only one entity instance with a given key value is attached.
I think (from the previous question I asked some time ago) that I understand what is happening, and previously, I have worked around this by having a seperate function that would in this case update the membership types. Then, stripping it out of the 'found' and 'sent' objects to allow Mapper to do the rest.
In my mapping profile I have the mappings defines as follows for these object types:
CreateMap<Membership, MembershipLTDTO>();
CreateMap<MembershipLTDTO, Membership>();
CreateMap<MembershipTypeDTO, MembershipType>();
CreateMap<MembershipType, MembershipTypeDTO>();
As I was about to go and do that very thing again, I was wondering if I am missing a trick with my use of Mapper, or Entity Framework that would allow it to happen more seamlessly?

A couple of things come to mind. The first thing is that the call to context.Memberships.Update(foundmembership); isn't required here so long as you haven't disabled tracking in the DbContext. Calling SaveChanges will build an UPDATE SQL statement for whatever values change (if any) where Update will attempt to overwrite the entitiy(ies).
The issue you are likely encountering is common when dealing with references, and I would recommend a different approach because of this. To outline this, lets look at Membership Types. These would typically be a known list that we want to associate to new and existing memberships. We're not going to ever expect to create a new membership type as part of an operation where we create or update a membership, just add or remove associations to existing memberships.
The problem with using Automapper for this is when we want to associate another membership type in our passed in DTO. Say we have existing data that had a membership associated with Membership Type #1, and we want to add MemberShip Type #2. We load the original entity types to copy values across, eager loading membership types so we get the membership and Type #1, so far so good. However, when we call Mapper.Map() it sees a MemberShip Type #2 in the DTO, so it will add a new entity with ID #2 into the collection of our loaded Membership's Types collection. From here, one of three things can happen:
1) The DbContext was already tracking an instance with ID #2 and
will complain when Update tries to associate another entity reference
with ID #2.
2) The DbContext isn't tracking an instance, and attempts to add #2
as a new entity.
2.1) The database is set up for an Identity column, and the new
membership type gets inserted with the next available ID. (I.e. #16)
2.2) The database is not set up for an Identity column and the
`SaveChanges` raises a duplicate constraint error.
The issue here is that Automapper doesn't have knowledge that any new Membership Type should be retrieved from the DbContext.
Using Automapper's Map method can be used to update child collections, though it should only be used to update references that are actual children of the top-level entity. For instance if you have a Customer and a collection of Contacts where updating the customer you want to update, add, or remove contact detail records because those child records are owned by, and explicitly associated to their customer. Automapper can add to or remove from the collection, and update existing items. For references like many-to-many/many-to-one we cannot rely on that since we will want to associate existing entities, not add/remove them.
In this case, the recommendation would be to tell Automapper to ignore the Membership Types collection, then handle these afterwards.
_mapper.Map(sentmembership, foundmembership, typeof(MembershipLTDTO), typeof(Membership));
var memberShipTypeIds = sentmembership.MembershipTypes.Select(x => x.MembershipTypeId).ToList();
var existingMembershipTypeIds = foundmembership.MembershipTypes.Select(x => x.MembershipTypeId).ToList();
var idsToAdd = membershipTypeIds.Except(existingMembershipTypeIds).ToList();
var idsToRemove = existingMembershipTypeIds.Except(membershipTypeIds).ToList();
if(idsToRemove.Any())
{
var membershipTypesToRemove = foundmembership.MembershipTypes.Where(x => idsToRemove.Contains(x.MembershipTypeId)).ToList();
foreach (var membershipType in membershipTypesToRemove)
foundmembership.MembershipTypes.Remove(membershipType;
}
if(idsToAdd.Any())
{
var membershipTypesToAdd = context.MembershipTypes.Where(x => idsToRemove.Contains(x.MembershipTypeId)).ToList();
foundmembership.MembershipTypes.AddRange(membershipTypesToAdd); // if declared as List, otherwise foreach and add them.
}
context.SaveChanges();
For items being removed, we find those entities in the loaded data state and remove them from the collection. For new items being added, we go to the context, fetch them all, and add them to the loaded data state's collection.

Notwithstanding marking Steve Py's solution as the answer, because it is a solution that works, though not as 'elegant' as I would have liked.
I was pointed in another direction however by the comment from
Lucian Bargaoanu, which, though a little cryptic, after some digging I found could be made to work.
To do this I had to add 'AutoMapper.Collection' and 'AutoMapper.Collection.EntityFrameworkCore' to my solution. There was a bit of jiggery pokery around setting it up as the example [here][2], didn't match up with my set up. I used this in my program.cs:
// Auto Mapper Configurations
var mappingConfig = new MapperConfiguration(mc =>
{
mc.AddProfile(new MappingProfile());
mc.AddCollectionMappers();
});
I also had to modify my mapping profile for the object - DTO mapping to this:
//Membership Types
CreateMap<MembershipTypeDTO, MembershipType>().EqualityComparison((mtdto, mt) => mtdto.Id == mt.Id);
Which is used to tell AutoMapper which fields to use for an equality.
I took out the context.Memberships.Update as recommended by Steve Py and it works.
Posted on behalf of the question asker

Overwrites collection item's properties back to old data on SaveChanges()

I'm currently developing an MVVM app using a Model-Designer based code-first design. So far I've been able to get all the basic CRUD operations working on single entities, however I can't seem to change the properties of collection objects at all using SaveChanges() - I've used an SQL profiler to see that it's attempting to UPDATE with the old value, and a step right after SaveChanges() my changes get reverted to their old values!
Some other info:
my dbContext is loaded using DI from PRISM/Unity and kept as a Unit-of-Work for a "details" page the user will edit and then save.
My WPF UI is bound correctly and can modify the changes on an object-level.
I can successfully use Add() to insert entities.
I've verified the entity state of the entity in the child collection is Modified both by setting it and simplify debugging.
I've attempted to manually Attach() and AddOrUpdate() on any or all items.
I've turned off all Lazy Loading and instead manually included all collections.
I've manually set the Entry() properties of IsModified and CurrentValue to their desired settings.
I've tried binding my VM properties to their data by either
dbContext.Classes.Local.ToBindingList() or new ObservableCollection<Class>(Entity.Property).
Is there anything that I could be missing here? Here's one attempt I've tried:
// Assigning an Index object that contains relationships
Index = await _model.PersonIndexes.Include(i => i.PersonAddresses).FirstAsync(i => i.Id == IndexId);
// Grabbing a filtered set of Addresses based on their data
var query = Index.PersonAddresses.Where(e => e.Condition == true );
Addresses = new ObservableCollection<PersonAddress>(await query.ToListAsync());
// Ensure state is tracked (I've tried with and without all combinations of these)
foreach (PersonAddress addr in Addresses)
{
//_model.PersonAddresses.AddOrUpdate(a => a.Id, addr);
if (addr.PendingRemoval)
{
_model.PersonAddresses.Attach(addr);
_model.Entry(addr).Property(a => a.PendingRemoval).CurrentValue = true;
_model.Entry(addr).Property(a => a.PendingRemoval).IsModified = true;
}
}
// Saving (after this line the properties of the entities in the related collection get reverted to their old values - i.e. if I change a phone number in the UI, the save below will replace the new values with the previous number.
await _model.SaveChangesAsync();

So it turns out this was an unfortunate error of configuration and a bad coincidence:
1) Check your models and server schema to ensure they are in sync (especially if using generated code from EF). In my case they were not, which lead to...
2) SaveChanges() was overwriting my properties in question because I had not noticed they were incorrectly set to have their StoredGeneratorPattern set to Computed in my model code. This caused the changed values to be ignored and overwritten.
3) The test case surrounding this had only implemented the same properties that were marked incorrectly, making it appear that all changes were being reverted - causing the confusion on where the problem code actually was. If another column had been modified and watched along with the others, this might have been caught sooner.

Entity Framework updating many to many

I am trying to find the suitable form of updating a many to many relationship but i am find some issues on it.
The application is an asp.net mvc with simple injector(set up per context)
I have an entity People which has an IEnumerable and also i have a entity Team which has an IEnumerable.
The People entity has some other fields like Description, Email, etc and in its View, there are some check boxes so the user can choose the Teams.
I had tried to search on the net for the best approach for updating a many to many relationship and all that i found was deleting everything in the third table that is created and then add the Teams again.
Under is what i am trying to do, but i am getting pk's already exists. I know it is happening because firstly i load the People entity with Find method(to remove the list of Teams inside a foreach) and after i try to Attach(when the error happens) the modified object to set it's State to Modified.
public override void Modify(People obj)
{
var ppl = SearchById(obj.Id);
if (ppl.Teams.Count > 0)
{
foreach (var team in ppl.Teams.ToList())
{
ppl.Teams.Remove(team);
}
}
var entry = lpcContext.Entry(obj);
if (lpcContext.Entry(obj).State == EntityState.Detached)
dbSet.Attach(obj);
entry.State = EntityState.Modified;
}
To air it out some things, i am using the Unit Of Work pattern, so i SaveChanges later.
Are there any other approach or i have to remove the Teams one by one, SaveChanges and after that, update the object and SaveChanges again?

Unfortunately, working with detached entities isnt that straight forward in EF (yet). Attach() in EF will work for connected entities only. That means if you load an object from DB, pass it on to a view (or page is asp.net). When you read the object back from that view/page, EF will not be tracking that object anymore. If you now try to use Attach(), you will get an error that the key already exists in the DBContext. To workaround this, you need to find the entry and make changes to the entity using SetValues(). Something like this:
public virtual void Update(T entity)
{
DbEntityEntry dbEntityEntry = DbContext.Entry(entity);
if (dbEntityEntry.State == EntityState.Detached)
{
var pkey = _dbset.Create().GetType().GetProperty("Id").GetValue(entity);//assuming Id is the key column
var set = DbContext.Set<T>();
T attachedEntity = set.Find(pkey);
if (attachedEntity != null)
{
var attachedEntry = DbContext.Entry(attachedEntity);
attachedEntry.CurrentValues.SetValues(entity);
}
}
}
Please note that this will ignore any nested objects. Hence, you should make DB trip, and compare the object returned from DB to find out if you should invoke Add, Update or Delete on each child object. This is the best workaround I could find when working with disconnected objects in EF. I guess nHibernate doesnt have this bug. Last I read about this, Microsoft was going to work on this after EF 6.x. So, we'll have to wait for this, I guess. Please go through the below article to understand the issue (and possible solutions) in length:
http://blog.maskalik.com/entity-framework/2013/12/23/entity-framework-updating-database-from-detached-objects/
To talk about your specfic scenario, you should make a DB hit and find out if any new teams were selected or some existing team was dropped and call add or delete as appropriate by comparing the Team collection of People object returned by DB vs People object returned from view/page. To update the People object itself, you can use the Update() as given above.

DbSet: missing added item

In a DbSet entity collection of Entity Framework (6.1.3), when I add a new item, it is not returned from the collection afterwards. This is strange and unexpected. Here's some gathered sample code:
dbContext.Entities.ToArray();
// contains 3 entries
dbContext.Entities.Add(new Entity());
dbContext.Entities.ToArray();
// still contains 3 entries
How can this be? When I query dbContext.Entities in the immediate window in Visual Studio, it says something like "Local: Count = 4". Why does it hide the new item from me?
Update: If this collection doesn't do the obvious thing – returning what was added before – what do I need to do instead? It must return all records from the database when called first, and it must also include all changes (add and remove) when called later. SaveChanges is only called when the user has finished editing things. The collection is needed before that! SaveChanges might also be called somewhere in between when the user is finished editing, but the code might return and the view be displayed again at a later time.

A DbSet has a property Local. This is an ObservableCollection that contains all elements. The DbSet object itself represents just the query to the database.
From the documentation of the Local property:
Gets an ObservableCollection that represents a local view of all
Added, Unchanged, and Modified entities in this set. This local view
will stay in sync as entities are added or removed from the context.
Likewise, entities added to or removed from the local view will
automatically be added to or removed from the context.
So if you want to access the elements, always use the Local property to do it.

After adding new entity, you have to Save the change using dbContext object,Use
dbContext.SaveChanges(); or dbContext.EntityState.Added

You have to save the changes. Try
dbContext.State = EntityState.Added;
dbContext.SaveChanges();

you can use following code to get all the items,
foreach (var track in dbContext.ChangeTracker.Entries())
{
if (track.State == EntityState.Deleted)
Entity s = (Entity)track.OriginalValues.ToObject();
else
Entity s = (Entity)track.CurrentValues.ToObject();
}

Entity Framework SaveChanges two behaviours depending on how I add to DbContext

I have overridden my db.SaveChanges() so I can call my FluentValidation validators before it actually attempts to save it.
I have a validator for each entity marked with IValidatableEntity and if the entity matches it will call it and pass the objectStateEntry in.
public virtual IEnumerable<string> SaveChanges(User user)
{
List<string> validationErrors = new List<string>();
if (this.Configuration.ValidateOnSaveEnabled)
{
foreach (var entry in ((IObjectContextAdapter)this).ObjectContext.ObjectStateManager
.GetObjectStateEntries(System.Data.Entity.EntityState.Added | System.Data.Entity.EntityState.Deleted | System.Data.Entity.EntityState.Modified | System.Data.Entity.EntityState.Unchanged)
.Where(entry => (entry.Entity is IValidatableEntity)))
{
validationErrors.AddRange(((IValidatableEntity)entry.Entity).Validate(entry));
}
}
if (!validationErrors.Any())
{ .....
The problem I have is that I get two different behaviours depending on how I add the object to the dbContext. I Guess because it only marks the aggregate root as being modified and only gives it an entry?
// Example A - Calls the Organisation Validator Only
organisation.Client.Add(client);
// Example B - Calls the Client Validator - which is correct
db.Client.Add(client);
Is there anyway to get EF automatically detect child properties have changed (Add / Modified) and call them? It kind of breaks my validation model if it doesn't, I was banking on updating the aggregate root and having EF call the necessary child validations as they should have unique entries.
Do I have to chain validators inside my Fluent Validations to catch these? I Didn't want a case of where my fluent Validator will have to check potentially hundreds of child entities. (some contain db lookups etc).
Thanks

Try to call DetectChanges at the beginning of your overridden SaveChanges method (it must be before you call GetObjectStateEntries):
this.ChangeTracker.DetectChanges();
The difference between the two lines to add a Client is that organisation.Client.Add(client) does not call any EF code directly (it's just adding an item to a collection in a POCO) while db.Client.Add(client) does and the DbSet<T>.Add method will call change detection automatically to update the entity states.
In the first case if you don't call any EF method before SaveChanges the base.SaveChanges will detect the changes as the very last place to ensure that all entity states are correct and all changes are saved. But base.SaveChanges is too late for the code in your overridden SaveChanges because it is after your evaluation of GetObjectStateEntries. At that point the entity state of the added client could still be Detached (i.e. not existing in the state manager) instead of Added. In order to fix this you have to call DetectChanges manually early enough to retrieve the final entity states in GetObjectStateEntries.

I assume organisation is a simple POCO, so the following code:
organisation.Client.Add(client);
just adds another POCO in an ICollection of POCOs. EF has no way to detect you are adding an entity to the context.
In the other hand, the following code:
db.Client.Add(client);
adds a POCO directly in an implementation of ICollection (DbCollectionEntry) that is related to Entity Framework and is in charge of what's called change tracking (among other things). This is possible thanks to dynamic proxy types generated at runtime (see https://stackoverflow.com/a/14321968/870604).
So you'll have to detect changes manually (see #Slauma answer). Another option would be to use a proxy object instead of your organisation POCO. This would be possible by calling:
var newOrganisation = dbContext.Set<Organisation>().Create();
The above code of course works for a new organisation instance.