We Keep Coding

Entity Framework Instance tracking error with mapping sub-objects - is there an elegant solution?

Some 2 years+ ago I asked this question which was kindly solved by Steve Py.
I am having a similar but different problem now when mapping with sub-objects. I have had this issue a few times and worked around it, but facing doing so again, I can't help thinking there must be a more elegant solution. I am coding a memebership system in Blazor Wasm and wanting update membership details via a web-api. All very normal.
I have a library function to update the membership:
public async Task<MembershipLTDTO> UpdateMembershipAsync(APDbContext context, MembershipLTDTO sentmembership)
{
Membership? foundmembership = context.Memberships.Where(x =>x.Id == sentmembership.Id)
.Include(x => x.MembershipTypes)
.FirstOrDefault();
if (foundmembership == null)
{
return new MembershipLTDTO { Status = new InfoBool(false, "Error: Membership not found", InfoBool.ReasonCode.Not_Found) };
}
try
{
_mapper.Map(sentmembership, foundmembership, typeof(MembershipLTDTO), typeof(Membership));
//context.Entry(foundmembership).State = EntityState.Modified; <-This was a 'try-out'
context.Memberships.Update(foundmembership);
await context.SaveChangesAsync();
sentmembership.Status = new InfoBool(true, "Membership successfully updated");
return sentmembership;
}
catch (Exception ex)
{
return new MembershipLTDTO { Status = new InfoBool(false, $"{ex.Message}", InfoBool.ReasonCode.Not_Found) };
}
}
The Membership object is an EF DB object and references a many to many list of MembershipTypes:
public class Membership
{
[Key]
public int Id { get; set; }
...more stuff...
public List<MembershipType>? MembershipTypes { get; set; } // The users membership can be several types. e.g. Employee + Director + etc..
}
The MembershipLTDTO is a lightweight DTO with a few heavy objects removed.
Executing the code, I get an EF exception:
The instance of entity type 'MembershipType' cannot be tracked because another instance with the same key value for {'Id'} is already being tracked. When attaching existing entities, ensure that only one entity instance with a given key value is attached.
I think (from the previous question I asked some time ago) that I understand what is happening, and previously, I have worked around this by having a seperate function that would in this case update the membership types. Then, stripping it out of the 'found' and 'sent' objects to allow Mapper to do the rest.
In my mapping profile I have the mappings defines as follows for these object types:
CreateMap<Membership, MembershipLTDTO>();
CreateMap<MembershipLTDTO, Membership>();
CreateMap<MembershipTypeDTO, MembershipType>();
CreateMap<MembershipType, MembershipTypeDTO>();
As I was about to go and do that very thing again, I was wondering if I am missing a trick with my use of Mapper, or Entity Framework that would allow it to happen more seamlessly?

A couple of things come to mind. The first thing is that the call to context.Memberships.Update(foundmembership); isn't required here so long as you haven't disabled tracking in the DbContext. Calling SaveChanges will build an UPDATE SQL statement for whatever values change (if any) where Update will attempt to overwrite the entitiy(ies).
The issue you are likely encountering is common when dealing with references, and I would recommend a different approach because of this. To outline this, lets look at Membership Types. These would typically be a known list that we want to associate to new and existing memberships. We're not going to ever expect to create a new membership type as part of an operation where we create or update a membership, just add or remove associations to existing memberships.
The problem with using Automapper for this is when we want to associate another membership type in our passed in DTO. Say we have existing data that had a membership associated with Membership Type #1, and we want to add MemberShip Type #2. We load the original entity types to copy values across, eager loading membership types so we get the membership and Type #1, so far so good. However, when we call Mapper.Map() it sees a MemberShip Type #2 in the DTO, so it will add a new entity with ID #2 into the collection of our loaded Membership's Types collection. From here, one of three things can happen:
1) The DbContext was already tracking an instance with ID #2 and
will complain when Update tries to associate another entity reference
with ID #2.
2) The DbContext isn't tracking an instance, and attempts to add #2
as a new entity.
2.1) The database is set up for an Identity column, and the new
membership type gets inserted with the next available ID. (I.e. #16)
2.2) The database is not set up for an Identity column and the
`SaveChanges` raises a duplicate constraint error.
The issue here is that Automapper doesn't have knowledge that any new Membership Type should be retrieved from the DbContext.
Using Automapper's Map method can be used to update child collections, though it should only be used to update references that are actual children of the top-level entity. For instance if you have a Customer and a collection of Contacts where updating the customer you want to update, add, or remove contact detail records because those child records are owned by, and explicitly associated to their customer. Automapper can add to or remove from the collection, and update existing items. For references like many-to-many/many-to-one we cannot rely on that since we will want to associate existing entities, not add/remove them.
In this case, the recommendation would be to tell Automapper to ignore the Membership Types collection, then handle these afterwards.
_mapper.Map(sentmembership, foundmembership, typeof(MembershipLTDTO), typeof(Membership));
var memberShipTypeIds = sentmembership.MembershipTypes.Select(x => x.MembershipTypeId).ToList();
var existingMembershipTypeIds = foundmembership.MembershipTypes.Select(x => x.MembershipTypeId).ToList();
var idsToAdd = membershipTypeIds.Except(existingMembershipTypeIds).ToList();
var idsToRemove = existingMembershipTypeIds.Except(membershipTypeIds).ToList();
if(idsToRemove.Any())
{
var membershipTypesToRemove = foundmembership.MembershipTypes.Where(x => idsToRemove.Contains(x.MembershipTypeId)).ToList();
foreach (var membershipType in membershipTypesToRemove)
foundmembership.MembershipTypes.Remove(membershipType;
}
if(idsToAdd.Any())
{
var membershipTypesToAdd = context.MembershipTypes.Where(x => idsToRemove.Contains(x.MembershipTypeId)).ToList();
foundmembership.MembershipTypes.AddRange(membershipTypesToAdd); // if declared as List, otherwise foreach and add them.
}
context.SaveChanges();
For items being removed, we find those entities in the loaded data state and remove them from the collection. For new items being added, we go to the context, fetch them all, and add them to the loaded data state's collection.

Notwithstanding marking Steve Py's solution as the answer, because it is a solution that works, though not as 'elegant' as I would have liked.
I was pointed in another direction however by the comment from
Lucian Bargaoanu, which, though a little cryptic, after some digging I found could be made to work.
To do this I had to add 'AutoMapper.Collection' and 'AutoMapper.Collection.EntityFrameworkCore' to my solution. There was a bit of jiggery pokery around setting it up as the example [here][2], didn't match up with my set up. I used this in my program.cs:
// Auto Mapper Configurations
var mappingConfig = new MapperConfiguration(mc =>
{
mc.AddProfile(new MappingProfile());
mc.AddCollectionMappers();
});
I also had to modify my mapping profile for the object - DTO mapping to this:
//Membership Types
CreateMap<MembershipTypeDTO, MembershipType>().EqualityComparison((mtdto, mt) => mtdto.Id == mt.Id);
Which is used to tell AutoMapper which fields to use for an equality.
I took out the context.Memberships.Update as recommended by Steve Py and it works.
Posted on behalf of the question asker

EntityFramework core - Update a collection of data without selecting the entities

How would you Upsert without select? the upsert would be a collection of entities received by a method which contains DTOs that may not be available in the database so you can NOT use attach range for example.
One way theoretically is to load the ExistingData partially with a select like dbContext.People.Where(x => x exists in requested collection).Select(x => new Person { Id = x.Id, State = x.State }).ToList() which just loads a part of the entity and not the heavy parts. But here if you update one of these returned entityItems from this collection it will not update because of the new Person its not tracking it and you also cannot say dbContext.Entry<Person>(person).State = Modified because it will throw an error and will tell you that ef core is already "Tracking" it.
So what to do.
One way would be to detach all of them from the ChangeTracker and then do the state change and it will do the update but not just on one field even if you say dbContext.Entry<Person>(person).Property(x => x.State).Modified = true. It will overwrite every fields that you haven't read from the database to their default value and it will make a mess in the database.
The other way would be to read the ChangeTracker entries and update them but it will also overwrite and it will consider like everything is chanaged.
So techinically I don't know how ef core can create the following SQL,
update People set state = 'Approved' where state != 'Approved'
without updating anything else. or loading the person first completely.
The reason for not loading your data is that you may want to update like 14000 records and those records are really heavy to load because they contain byte[] and have images stored on them for example.
BTW the lack of friendly documentation on EFCore is a disaster compare to Laravel. Recently it has cost us the loss of a huge amount of data.
btw, the examples like the code below will NOT work for us because they are updating one field which they know that it exists in database. But we are trying to upsert a collection which some of those DTOs may not be available in the database.
try
{
using (var db = new dbContext())
{
// Create new stub with correct id and attach to context.
var entity = new myEntity { PageID = pageid };
db.Pages.Attach(entity);
// Now the entity is being tracked by EF, update required properties.
entity.Title = "new title";
entity.Url = "new-url";
// EF knows only to update the propeties specified above.
db.SaveChanges();
}
}
catch (DataException)
{
// process exception
}
Edit: The used ef core version is #3.1.9

Fantastic, I found the solution (You need to also take care about your unit tests).
Entityframework is actually working fine it can be just a lack of experience which I'm documenting here in case anyone else got into the same issue.
Consider that we have an entity for Person which has a profile picture saved as Blob on it which causes that if you do something like the following for let's say 20k people the query goes slow even when you've tried to have enough correct index on your table.
You want to do this query to update these entities based on a request.
var entityIdsToUpdate = request.PeopleDtos.Select(p => p.Id);
var people = dbContext.People.Where(x => entityIdsToUpdate.Contains(x.Id)).ToList();
This is fine and it works perfectly, you will get the People collection and then you can update them based on the given data.
In these kind of updates you normally will not need to update images even if you do, then you need to increase the `TimeOut1 property on your client but for our case we did not need to update the images.
So the above code will change to this.
var entityIdsToUpdate = request.PeopleDtos.Select(p => p.Id);
var people = dbContext.People
.Select(p => new Person {
Id = p.Id,
Firstname = p.Firstname,
Lastname = p.Lastname,
//But no images to load
})
.Where(p => entityIdsToUpdate.Contains(p.Id)).ToList();
But then with this approach, EntityFramework will lose the track of your entities.
So you need to attach it like this and I will tell you how NOT to attach it.
This is the correct way for a collection
dbContext.People.AttachRange(people); //These are the people you've already queried
Now DO NOT do this, you may want to do this because you get an error from the first one from EntityFramework which says the entity is already being tracked, trust it because it already is. I will explain after the code.
//Do not do this
foreach(var entry in dbContext.ChangeTracker.Entries())
{
entry.State = EntityState.Detached;
}
//and then on updating a record you may write the following to attach it back
dbContext.Entry(Person).State = EntityState.Modified;
The above code will cause EntityFramework not to follow the changes on the entities anymore and by the last line you will tell it literally everything edited or not edited is changed and will cause you to LOSE your unedited properties like the "image".
Note: Now what can u do by mistake that even messes up the correct approach.
Well since you are not loading your whole entity, you may assume that it is still fine to assign values to the unloaded ones even if the value is not different than the one in the database. This causes entity framework to assume that something is changed and if you are setting a ModifiedOn on your records it will change it for no good reason.
And now about testing:
While you test, you may get something out from database and create a dto from that and pass the dto with the same dbContext to your SystemUnderTest the attach method will throw an error here which says this entity is already bein tracked because of that call in your test method. The best way would be create a new dbContext for each process and dispose them after you are done with them.
BTW in testing it may happen that with the same dbContext you update an entity and after the test you want to fetch if from the database. Please take note that this one which is returning to you is the "Cached" one by EntityFramework and if you have fetched it in the first place not completely like just with Select(x => ) then you will get some fields as null or default value.
In this case you should do DbContext.Entry(YOUR_ENTRY).Reload().
It is a really complete answer it may not directly be related to the question but all of the things mentioned above if you don't notice them may cause a disaster.

C# EF 6 CurrentValues.SetValues cannot change Object's Key Information

I have seen other questions about this same error, but I am unable to correct the error with those suggestions in my code; I think that this is a different problem and not a duplicate.
I have an app that makes a series of rules, of which the user can set properties in the GUI. There is a table of Rules in a connected database, with the primary key on the Rule.Id. When the user saves changes to a rule, the existing rule gets "IsActive=0" to hide it, then a new database record is made with the properties from the GUI written to the database. It looks to the user as though they have edited the rule, but the database actually sees a new rule reflecting the new properties (this allows for a history to be kept), connected to the old rule by another reference field.
In the C# code for the app, the View Model for each rule contains an EF Rule object property. When the user clicks "save" I use the parameters set in the view to build the ruleViewModel.Rule for each ruleViewModel they want to save, with porperties matching the GUI. The MainViewModel contains the DbContext object called dbo, so I use the ruleViewModel.Rule to write to the mainViewModel.dbo.Entry which I save to the Entity Framework. Here are the three basic steps performed for each saveable Rule View Model:
// get the rule from the GUI and use it to make sure we are updating the right rule in EF (which is connected to the mainViewModel)
var dboItem = ruleViewModel.MainViewModel.dbo.Rules.Single(r => r.Id == ruleViewModel.Rule.Id);
// set the values in the EF item to be those we got from the GUI
ruleViewModel.MainViewModel.dbo.Entry(dboItem).CurrentValues.SetValues(ruleViewModel.Rule);
// Save the differences
ruleViewModel.MainViewModel.dbo.SaveChanges();
If the user only saves a single rule, it all works fine, but if they subsequently try to save another, or if they save more than one at once, they get the following error, which is return by the ..SetValues(..) line:
Message = "The property 'Id' is part of the object's key information and cannot be modified. "
I see from other questions on this subject that there is a feature of EF that stops you from writing the same object twice to the database with a different Id, so this error often happens within a loop. I have tried using some of the suggestions, like adding
viewModel.MainViewModel.dbo.Rules.Add(dboItem);
and
viewModel.MainViewModel.dbo.Entry(dboItem).Property(x => x.Id).IsModified = false;
before the SaveChanges() command, but that has not helped with the problem (not to mention changing the function of the code). I see that some other suggestions say that the Entry should be created within the loop, but in this case, the entries are all existing rules in the database - it seems to me (perhaps erroneously) that I cannot create them inside the save loop, since they are the objects over which the loop is built - for each entity I find, I want to save changes.
I'm really confused about what to do and tying myself increasingly in knots trying to fix the error. It's been several days now and my sanity and self-esteem is beginning to wane! Any pointers to get me working in the right direction to stop the error appearing and allow me to set the database values would be really welcome as I feel like I have hit a complete dead end! The first time around the loop, everything works perfectly.

Aside from the questionable location of the DbContext and view models containing entities, this looks like it would work as expected. I'm assuming from the MVVM tag that this is a Windows application rather than a web app. The only issue is that this assumes that the Rule entity in your ruleViewModel is detached from the DbContext. If the DbContext is still tracking that entity reference then getting the entity from the DbContext again would pass you back the same reference.
It would probably be worth testing this once in a debug session. If you add the following:
var dboItem = ruleViewModel.MainViewModel.dbo.Rules.Single(r => r.Id == ruleViewModel.Rule.Id);
bool isReferenceSame = Object.ReferenceEquals(dboItem, ruleViewModel.Rule);
Do you get an isReferenceSame value of True or False? If True, the DbContext in your main view model is still tracking the Rule entity and the whole get dboItem and SetValues isn't necessary. If False, then the ruleViewModel is detached.
If the entities are attached and being tracked then edits to the view model entities would be persisted when you call a SaveChanges on the DbContext. (No load & SetValues needed) This should apply to single or multiple entity edits.
If the entities are detached then normally the approach for updating an entity across DbContext instances would look more like:
var context = mainViewModel.dbo;
foreach( var ruleViewModel in updatedRuleViewModels)
{
// This should associate the Entity in the ruleViewModel with the DbContext and set it's tracking state to Modified.
context.Entry(ruleViewModel.Rule).State = EntityState.Modified;
}
context.SaveChanges();
There are a couple of potential issues with this approach that you should consider avoiding if possible. A DbContext should be kept relatively short lived, so seeing a reference to a DbContext within a ViewModel is a bit of a red flag. Overall I don't recommend putting entity references inside view models or passing them around outside of the scope of the DbContext they were created in. EF certainly supports it, but it requires a bit more care and attention to assess whether entities are tracked or not, and in situations like web applications, opens the domain to invalid tampering. (Trusting the entity coming in where any change is attached or copied across overwriting the data state)

How can I "transport" an entity object from a WCF service to a client application with Entity Framework?

I am completely new to Entitfy Framework. I am using a WCF service to pass data to an ASP.NET Web Forms application. I have classes that were generated according to the database schema I have.
First, I tried to get a simple result, executing an operation contract, named 'GetPublciations', which returns all the records, stored in my Publciation table, as an array of Publication objects. Everything compiled fine, but I got a run-time error. I fixed that by following the instructions in this question and its answer.
Basically, I added a line, like the one below in my method:
yourContextObject.Configuration.ProxyCreationEnabled = false;
Now, I have a new problem. My Publication table has a foreign key CreationUserId, that references another table, named User. But when I try to get the value of some of the properties of the User object, it turns out that it actually has a value of NULL, which is imposible because in my database this foreign key column is NOT NULL. I tested this in a Console Application, without setting the ProxyCreationEnabled to false and then all of my "child" objects were created appropriately (meaning I had a User object inside my Publication object, which was not NULL).
I guess I can overcome this problem simply by creating a view in my database, for example, DetailedPublciation, then creating an appropriate auto-generated class with Entity Framework and then passing an object of this DetailedPublciation through my WCF service. Meaning, the newly created class will have only properties of primitive types like string, int, long, etc. and I won't have to reference a 'child' object.
So, my question is how I can fix that and whether creating a view like the mentioned above is the correct way to fix this problem.
Edit 1:
I tried with this code inside my GetPublications method:
SomeEntities someEntities = new SomeEntities();
someEntities.Configuration.ProxyCreationEnabled = false;
return someEntities.Publications.Include("User").ToList();
I get a big exception 'tree', but the inner exception states:
An existing connection was forcibly closed by the remote host
Edit 2:
I tried the following code in both - a Console Application and a WCF Service.
SomeEntities someEntities = new SomeEntities();
someEntities.Configuration.ProxyCreationEnabled = false;
return someEntities.Publications.Include(p => p.User).ToList();
In the Console Application everything works fine but in the service, I get a StackOverflow Exception.

ProxyCreationEnabled essentially turns off lazy loading which means you need to manually specify any extra objects you want retrieved from your database. Using Include for example, if your query looks like this:
var publications = from p in context.Publications
where p.Column = someValue
select p;
Then you should change it to this:
var publications = (from p in context.Publications
where p.Column = someValue
select p)
.Include(p => p.CreationUser);
The Include function will force Entity Framework to (eagerly) load the CreationUser object.

Why Is A Navigation Property Updating Another Entity When Executing An Update In My DataContext? (C# Entity Framework)

I'm running into a situation using the Entity Framework (EF) that has me totally stumped. I'm doing a simple update and the error I'm getting is
Violation of PRIMARY KEY constraint 'PK__tblProducts_Mark__03E07F87'. Cannot insert duplicate key in object 'healthc.tblProducts_MarketSegmentGroups'.
The statement has been terminated.
Let me give you some background on the problem.
I am using Web Forms and have a button click event fire to save some data in several text box controls on my page.
I have a table in my database called tblMetaProducts, which is a table used to store product information from various vendors we work with. The entity for this table is called Products.
I have another table called tblTechAssessment, which holds data for technical questions about the vendor's product, (e.g. what operating system can the software run on, version number etc.). The entity for this table is called TechnicalAssessment. A product can have many technical assessments and they are related by the product id.
I finally have a lookup table in the database called tblProducts_MarketSegmentGroups, which holds a product id and another id (which we don't care about for this problem). The entity for this table is called ProductMarketSegmentGroup. a product can have many product market segment groups and they are releated by the product id.
Here is the code I'm executing to perform the EF save
private void UpdateTechnicalAssessments(int productID)
{
var technicalAssessments = VendorDirectoryController.GetTechnicalAssessments(productID);
var technicalAssessmentTypes = Enum.GetValues(typeof(TechnicalAssessmentType)).Cast<TechnicalAssessmentType>();
foreach (var technicalAssessmentType in technicalAssessmentTypes)
{
var typeName = technicalAssessmentType.ToString();
var id = "SaveToProduction" + typeName + "TextBox";
var results = ProductInformationPanel.FindDescendantsByType<TextBox>().Single(x => x.ID == id).Text;
technicalAssessments.Single(x => x.QuestionID == (int)technicalAssessmentType).Results = results;
}
VendorDirectoryController.SaveChanges();
}
The SaveChanges() method drills down to my domain layer and calls the dataContext.SaveChanges() method.
So my questions are:
1) What can I do to get this to save my TechnicalAssessment entities?
2) Why does my save affect the ProductMarketSegmentGroup entity?

You might be hitting a bug in EF. I also stubmled on something similar (even though I use stored procedures).
The solution was to apply hotfix mentioned in: hotfix: Principal entity in an SQL application generates unnecessary updates - do it still effect EF 4.3.1?

The solution I ended up using was to have our database admin create a proc for the update. I was never able to figure out why the navigation property was causing such a fuss.