I am using entity framework to alter data on my remote tables.
One working call I do is this:
public async Task<List<PictureItems>?> GetUserThumbnailByuserId(string userid)
{
await InitializeAsync();
try
{
return await _table.GetAsyncItems().Where(x => x.BelongsToUserId == userid).ToListAsync();
}
catch (Exception e)
{
return null;
}
}
Now, I am trying to delete an item where Ids match.
But all I can do to delete the correct item is:
await _table.DeleteItemAsync(item);
But this would only work if I inserted an identical item into the (). So I could get the right item, if the item is not null then insert into the delete and the item is deleted.
But I dont wanna get the item all the times when I have the ID already locally available, this would just create unneccissary traffic.
So what I would need is something like:
_table.DeleteItemAsync().Where(x => x.BelongsToUserId == userid).ToListAsync();
Anyone got any help for me how to approach this?
Best,
J
If you have knowledge about the primary ID of the items to be deleted beforehand, you can simply instantiate an object only containing that ID and pass it to EF Core:
table.Remove(new Entity() { Id = <yourId> });
This removes the need to query data first - EF Core will delete the corresponding existing item with the respective ID property if it is already available on the DB.
Of course, whether or not this works in your case depends on whether BelongsToUserId is a key EF can work with, or whether it is only a regular property.
You have several options:
As already mentioned in other answer, if you know the primary keys of the entities that you want to delete, you can construct empty entities with IDs without querying and call DbContext.RemoveRange(entities);
If you're up for trying or using EF Core v.7.0, then you can use The new ExecuteDelete method
If you know what you're doing and are confident in the quality of your inputs: use ExecuteSqlRaw and just remove the objects you want manually this way: DbContext.Database.ExecuteSqlRaw("DELETE FROM PictureItems WHERE ...");
Use a (commercial) extension library that supports bulk-delete on EF Core. I generally avoid recommending specific tools, but "ef core bulk delete" search in Google gives some relevant results. Search for an open-source analogue lead me to this small library that should fit your needs - taken from this answer.
Related
Some 2 years+ ago I asked this question which was kindly solved by Steve Py.
I am having a similar but different problem now when mapping with sub-objects. I have had this issue a few times and worked around it, but facing doing so again, I can't help thinking there must be a more elegant solution. I am coding a memebership system in Blazor Wasm and wanting update membership details via a web-api. All very normal.
I have a library function to update the membership:
public async Task<MembershipLTDTO> UpdateMembershipAsync(APDbContext context, MembershipLTDTO sentmembership)
{
Membership? foundmembership = context.Memberships.Where(x =>x.Id == sentmembership.Id)
.Include(x => x.MembershipTypes)
.FirstOrDefault();
if (foundmembership == null)
{
return new MembershipLTDTO { Status = new InfoBool(false, "Error: Membership not found", InfoBool.ReasonCode.Not_Found) };
}
try
{
_mapper.Map(sentmembership, foundmembership, typeof(MembershipLTDTO), typeof(Membership));
//context.Entry(foundmembership).State = EntityState.Modified; <-This was a 'try-out'
context.Memberships.Update(foundmembership);
await context.SaveChangesAsync();
sentmembership.Status = new InfoBool(true, "Membership successfully updated");
return sentmembership;
}
catch (Exception ex)
{
return new MembershipLTDTO { Status = new InfoBool(false, $"{ex.Message}", InfoBool.ReasonCode.Not_Found) };
}
}
The Membership object is an EF DB object and references a many to many list of MembershipTypes:
public class Membership
{
[Key]
public int Id { get; set; }
...more stuff...
public List<MembershipType>? MembershipTypes { get; set; } // The users membership can be several types. e.g. Employee + Director + etc..
}
The MembershipLTDTO is a lightweight DTO with a few heavy objects removed.
Executing the code, I get an EF exception:
The instance of entity type 'MembershipType' cannot be tracked because another instance with the same key value for {'Id'} is already being tracked. When attaching existing entities, ensure that only one entity instance with a given key value is attached.
I think (from the previous question I asked some time ago) that I understand what is happening, and previously, I have worked around this by having a seperate function that would in this case update the membership types. Then, stripping it out of the 'found' and 'sent' objects to allow Mapper to do the rest.
In my mapping profile I have the mappings defines as follows for these object types:
CreateMap<Membership, MembershipLTDTO>();
CreateMap<MembershipLTDTO, Membership>();
CreateMap<MembershipTypeDTO, MembershipType>();
CreateMap<MembershipType, MembershipTypeDTO>();
As I was about to go and do that very thing again, I was wondering if I am missing a trick with my use of Mapper, or Entity Framework that would allow it to happen more seamlessly?
A couple of things come to mind. The first thing is that the call to context.Memberships.Update(foundmembership); isn't required here so long as you haven't disabled tracking in the DbContext. Calling SaveChanges will build an UPDATE SQL statement for whatever values change (if any) where Update will attempt to overwrite the entitiy(ies).
The issue you are likely encountering is common when dealing with references, and I would recommend a different approach because of this. To outline this, lets look at Membership Types. These would typically be a known list that we want to associate to new and existing memberships. We're not going to ever expect to create a new membership type as part of an operation where we create or update a membership, just add or remove associations to existing memberships.
The problem with using Automapper for this is when we want to associate another membership type in our passed in DTO. Say we have existing data that had a membership associated with Membership Type #1, and we want to add MemberShip Type #2. We load the original entity types to copy values across, eager loading membership types so we get the membership and Type #1, so far so good. However, when we call Mapper.Map() it sees a MemberShip Type #2 in the DTO, so it will add a new entity with ID #2 into the collection of our loaded Membership's Types collection. From here, one of three things can happen:
1) The DbContext was already tracking an instance with ID #2 and
will complain when Update tries to associate another entity reference
with ID #2.
2) The DbContext isn't tracking an instance, and attempts to add #2
as a new entity.
2.1) The database is set up for an Identity column, and the new
membership type gets inserted with the next available ID. (I.e. #16)
2.2) The database is not set up for an Identity column and the
`SaveChanges` raises a duplicate constraint error.
The issue here is that Automapper doesn't have knowledge that any new Membership Type should be retrieved from the DbContext.
Using Automapper's Map method can be used to update child collections, though it should only be used to update references that are actual children of the top-level entity. For instance if you have a Customer and a collection of Contacts where updating the customer you want to update, add, or remove contact detail records because those child records are owned by, and explicitly associated to their customer. Automapper can add to or remove from the collection, and update existing items. For references like many-to-many/many-to-one we cannot rely on that since we will want to associate existing entities, not add/remove them.
In this case, the recommendation would be to tell Automapper to ignore the Membership Types collection, then handle these afterwards.
_mapper.Map(sentmembership, foundmembership, typeof(MembershipLTDTO), typeof(Membership));
var memberShipTypeIds = sentmembership.MembershipTypes.Select(x => x.MembershipTypeId).ToList();
var existingMembershipTypeIds = foundmembership.MembershipTypes.Select(x => x.MembershipTypeId).ToList();
var idsToAdd = membershipTypeIds.Except(existingMembershipTypeIds).ToList();
var idsToRemove = existingMembershipTypeIds.Except(membershipTypeIds).ToList();
if(idsToRemove.Any())
{
var membershipTypesToRemove = foundmembership.MembershipTypes.Where(x => idsToRemove.Contains(x.MembershipTypeId)).ToList();
foreach (var membershipType in membershipTypesToRemove)
foundmembership.MembershipTypes.Remove(membershipType;
}
if(idsToAdd.Any())
{
var membershipTypesToAdd = context.MembershipTypes.Where(x => idsToRemove.Contains(x.MembershipTypeId)).ToList();
foundmembership.MembershipTypes.AddRange(membershipTypesToAdd); // if declared as List, otherwise foreach and add them.
}
context.SaveChanges();
For items being removed, we find those entities in the loaded data state and remove them from the collection. For new items being added, we go to the context, fetch them all, and add them to the loaded data state's collection.
Notwithstanding marking Steve Py's solution as the answer, because it is a solution that works, though not as 'elegant' as I would have liked.
I was pointed in another direction however by the comment from
Lucian Bargaoanu, which, though a little cryptic, after some digging I found could be made to work.
To do this I had to add 'AutoMapper.Collection' and 'AutoMapper.Collection.EntityFrameworkCore' to my solution. There was a bit of jiggery pokery around setting it up as the example [here][2], didn't match up with my set up. I used this in my program.cs:
// Auto Mapper Configurations
var mappingConfig = new MapperConfiguration(mc =>
{
mc.AddProfile(new MappingProfile());
mc.AddCollectionMappers();
});
I also had to modify my mapping profile for the object - DTO mapping to this:
//Membership Types
CreateMap<MembershipTypeDTO, MembershipType>().EqualityComparison((mtdto, mt) => mtdto.Id == mt.Id);
Which is used to tell AutoMapper which fields to use for an equality.
I took out the context.Memberships.Update as recommended by Steve Py and it works.
Posted on behalf of the question asker
How would you Upsert without select? the upsert would be a collection of entities received by a method which contains DTOs that may not be available in the database so you can NOT use attach range for example.
One way theoretically is to load the ExistingData partially with a select like dbContext.People.Where(x => x exists in requested collection).Select(x => new Person { Id = x.Id, State = x.State }).ToList() which just loads a part of the entity and not the heavy parts. But here if you update one of these returned entityItems from this collection it will not update because of the new Person its not tracking it and you also cannot say dbContext.Entry<Person>(person).State = Modified because it will throw an error and will tell you that ef core is already "Tracking" it.
So what to do.
One way would be to detach all of them from the ChangeTracker and then do the state change and it will do the update but not just on one field even if you say dbContext.Entry<Person>(person).Property(x => x.State).Modified = true. It will overwrite every fields that you haven't read from the database to their default value and it will make a mess in the database.
The other way would be to read the ChangeTracker entries and update them but it will also overwrite and it will consider like everything is chanaged.
So techinically I don't know how ef core can create the following SQL,
update People set state = 'Approved' where state != 'Approved'
without updating anything else. or loading the person first completely.
The reason for not loading your data is that you may want to update like 14000 records and those records are really heavy to load because they contain byte[] and have images stored on them for example.
BTW the lack of friendly documentation on EFCore is a disaster compare to Laravel. Recently it has cost us the loss of a huge amount of data.
btw, the examples like the code below will NOT work for us because they are updating one field which they know that it exists in database. But we are trying to upsert a collection which some of those DTOs may not be available in the database.
try
{
using (var db = new dbContext())
{
// Create new stub with correct id and attach to context.
var entity = new myEntity { PageID = pageid };
db.Pages.Attach(entity);
// Now the entity is being tracked by EF, update required properties.
entity.Title = "new title";
entity.Url = "new-url";
// EF knows only to update the propeties specified above.
db.SaveChanges();
}
}
catch (DataException)
{
// process exception
}
Edit: The used ef core version is #3.1.9
Fantastic, I found the solution (You need to also take care about your unit tests).
Entityframework is actually working fine it can be just a lack of experience which I'm documenting here in case anyone else got into the same issue.
Consider that we have an entity for Person which has a profile picture saved as Blob on it which causes that if you do something like the following for let's say 20k people the query goes slow even when you've tried to have enough correct index on your table.
You want to do this query to update these entities based on a request.
var entityIdsToUpdate = request.PeopleDtos.Select(p => p.Id);
var people = dbContext.People.Where(x => entityIdsToUpdate.Contains(x.Id)).ToList();
This is fine and it works perfectly, you will get the People collection and then you can update them based on the given data.
In these kind of updates you normally will not need to update images even if you do, then you need to increase the `TimeOut1 property on your client but for our case we did not need to update the images.
So the above code will change to this.
var entityIdsToUpdate = request.PeopleDtos.Select(p => p.Id);
var people = dbContext.People
.Select(p => new Person {
Id = p.Id,
Firstname = p.Firstname,
Lastname = p.Lastname,
//But no images to load
})
.Where(p => entityIdsToUpdate.Contains(p.Id)).ToList();
But then with this approach, EntityFramework will lose the track of your entities.
So you need to attach it like this and I will tell you how NOT to attach it.
This is the correct way for a collection
dbContext.People.AttachRange(people); //These are the people you've already queried
Now DO NOT do this, you may want to do this because you get an error from the first one from EntityFramework which says the entity is already being tracked, trust it because it already is. I will explain after the code.
//Do not do this
foreach(var entry in dbContext.ChangeTracker.Entries())
{
entry.State = EntityState.Detached;
}
//and then on updating a record you may write the following to attach it back
dbContext.Entry(Person).State = EntityState.Modified;
The above code will cause EntityFramework not to follow the changes on the entities anymore and by the last line you will tell it literally everything edited or not edited is changed and will cause you to LOSE your unedited properties like the "image".
Note: Now what can u do by mistake that even messes up the correct approach.
Well since you are not loading your whole entity, you may assume that it is still fine to assign values to the unloaded ones even if the value is not different than the one in the database. This causes entity framework to assume that something is changed and if you are setting a ModifiedOn on your records it will change it for no good reason.
And now about testing:
While you test, you may get something out from database and create a dto from that and pass the dto with the same dbContext to your SystemUnderTest the attach method will throw an error here which says this entity is already bein tracked because of that call in your test method. The best way would be create a new dbContext for each process and dispose them after you are done with them.
BTW in testing it may happen that with the same dbContext you update an entity and after the test you want to fetch if from the database. Please take note that this one which is returning to you is the "Cached" one by EntityFramework and if you have fetched it in the first place not completely like just with Select(x => ) then you will get some fields as null or default value.
In this case you should do DbContext.Entry(YOUR_ENTRY).Reload().
It is a really complete answer it may not directly be related to the question but all of the things mentioned above if you don't notice them may cause a disaster.
Some previous code I had written used the Find() method to retrieve single entities by their primary key:
return myContext.Products.Find(id)
This worked great because I had this code tucked into a generic class, and each entity had a different field name as its primary key.
But I had to replace the code because I noticed that it was returning cached data, and I need it to return data from the database each call. Microsoft's documentation confirmed this is the behavior of Find().
So I changed my code to use SingleOrDefault or FirstOrDefault. I haven't found anything in documentation that states these methods return cached data.
Now I am executing these steps:
Save an entity via EF.
Execute an UPDATE statement in SSMS to update the recently saved
record's Description field.
Retrieve the entity into a new entity variable using SingleOrDefault
or FirstOrDefault.
The entities being returned still have the old value in the Description field.
I have run a SQL trace, and verified that the data is being queried during step 3. This baffles me - if EF is making a round trip to the database, why is it returning cached data?
I've searched online, and most answers apply to the Find() method. Furthermore, they suggest some solutions that are merely workarounds (dispose the DbContext and instantiate a new one) or solutions that won't work for me (use the AsNoTracking() method).
How can I retrieve my entities from the database and bypass the EF cache?
The behaviour you're seeing is described in Microsoft's How Queries Work article under point 3:
For each item in the result set
a. If this is a tracking query, EF checks if the data represents an entity already in the change tracker for the context instance
If so, the existing entity is returned
It's described a little better in this blog post:
It turns out that Entity Framework uses the Identity Map pattern. This means that once an entity with a given key is loaded in the context’s cache, it is never loaded again for as long as that context exists. So when we hit the database a second time to get the customers, it retrieved the updated 851 record from the database, but because customer 851 was already loaded in the context, it ignored the newer record from the database (more details).
All of this is saying that if you make a query, it checks the primary key first to see if it already has it in the cache. If so, it uses what's in the cache.
How do you avoid it? The first is to make sure you're not keeping your DbContext object alive too long. DbContext objects are only designed to be used for one unit of work. Bad things happen if you keep it around too long, like excessive memory consumption.
Do you need to retrieve data to display to the user? Create a DbContext to get the data and discard that DbContext.
Do you need to update a record? Create a new DbContext, update the record and discard that DbContext.
This is why, when you use EF Core with dependency injection in ASP.NET Core, it is created with a scoped lifetime, so any DbContext object only lives for the life of one HTTP request.
In the rare case you really do need to get fresh data for a record you already have an object for, you can use EntityEntry.Reload()/EntityEntry.ReloadAsync like this:
myContext.Entry(myProduct).Reload();
That doesn't help you if you only know the ID though.
If you really really need to reload an entity that you only have the ID for, you could do something weird like this:
private Product GetProductById(int id) {
//check if it's in the cache already
var cachedEntity = myContext.ChangeTracker.Entries<Product>()
.FirstOrDefault(p => p.Entity.Id == id);
if (cachedEntity == null) {
//not in cache - get it from the database
return myContext.Products.Find(id);
} else {
//we already have it - reload it
cachedEntity.Reload();
return cachedEntity.Entity;
}
}
But again, this should only be used in limited cases, when you've already addressed any cases of long-living DbContext object because unwanted caching isn't the only consequence.
Ok, I have the same problem and finally found the answer,
You doing everything right, that's just how EF works.
You can use .AsNoTracking() for your purposes:
return myContext.Products.AsNoTracking().Find(id)
make sure you addedusing Microsoft.EntityFrameworkCore; at the top.
It works like a magic
I am trying to find the suitable form of updating a many to many relationship but i am find some issues on it.
The application is an asp.net mvc with simple injector(set up per context)
I have an entity People which has an IEnumerable and also i have a entity Team which has an IEnumerable.
The People entity has some other fields like Description, Email, etc and in its View, there are some check boxes so the user can choose the Teams.
I had tried to search on the net for the best approach for updating a many to many relationship and all that i found was deleting everything in the third table that is created and then add the Teams again.
Under is what i am trying to do, but i am getting pk's already exists. I know it is happening because firstly i load the People entity with Find method(to remove the list of Teams inside a foreach) and after i try to Attach(when the error happens) the modified object to set it's State to Modified.
public override void Modify(People obj)
{
var ppl = SearchById(obj.Id);
if (ppl.Teams.Count > 0)
{
foreach (var team in ppl.Teams.ToList())
{
ppl.Teams.Remove(team);
}
}
var entry = lpcContext.Entry(obj);
if (lpcContext.Entry(obj).State == EntityState.Detached)
dbSet.Attach(obj);
entry.State = EntityState.Modified;
}
To air it out some things, i am using the Unit Of Work pattern, so i SaveChanges later.
Are there any other approach or i have to remove the Teams one by one, SaveChanges and after that, update the object and SaveChanges again?
Unfortunately, working with detached entities isnt that straight forward in EF (yet). Attach() in EF will work for connected entities only. That means if you load an object from DB, pass it on to a view (or page is asp.net). When you read the object back from that view/page, EF will not be tracking that object anymore. If you now try to use Attach(), you will get an error that the key already exists in the DBContext. To workaround this, you need to find the entry and make changes to the entity using SetValues(). Something like this:
public virtual void Update(T entity)
{
DbEntityEntry dbEntityEntry = DbContext.Entry(entity);
if (dbEntityEntry.State == EntityState.Detached)
{
var pkey = _dbset.Create().GetType().GetProperty("Id").GetValue(entity);//assuming Id is the key column
var set = DbContext.Set<T>();
T attachedEntity = set.Find(pkey);
if (attachedEntity != null)
{
var attachedEntry = DbContext.Entry(attachedEntity);
attachedEntry.CurrentValues.SetValues(entity);
}
}
}
Please note that this will ignore any nested objects. Hence, you should make DB trip, and compare the object returned from DB to find out if you should invoke Add, Update or Delete on each child object. This is the best workaround I could find when working with disconnected objects in EF. I guess nHibernate doesnt have this bug. Last I read about this, Microsoft was going to work on this after EF 6.x. So, we'll have to wait for this, I guess. Please go through the below article to understand the issue (and possible solutions) in length:
http://blog.maskalik.com/entity-framework/2013/12/23/entity-framework-updating-database-from-detached-objects/
To talk about your specfic scenario, you should make a DB hit and find out if any new teams were selected or some existing team was dropped and call add or delete as appropriate by comparing the Team collection of People object returned by DB vs People object returned from view/page. To update the People object itself, you can use the Update() as given above.
I have an update function in my repository which updates TerminalCertification entity. But this entity has a many to many relation to another class ( GomrokJustification ).
my update function update entity correctly but does not anything on related entity.
my update function is below:
public void UpdateTerminalCertification(TerminalCertification terminalCertification)
{
var lastCertification =
db.terminalCertifications.Include("TimeInfo").Include("GomrokJustifications").Where(item=>item.TerminalCertificationID==terminalCertification.TerminalCertificationID).ToList();
if (lastCertification.Count==0)
throw new TerminalCertificationNotFoundException(terminalCertification);
terminalCertification.TimeInfo = lastCertification[0].TimeInfo;
((IObjectContextAdapter)db).ObjectContext.Detach(lastCertification[0]);
((IObjectContextAdapter)db).ObjectContext.AttachTo("terminalCertifications", terminalCertification);
foreach (var gomrokJustification in terminalCertification.GomrokJustifications)
{
((IObjectContextAdapter)db).ObjectContext.AttachTo("gomrokJustifications", gomrokJustification);
((IObjectContextAdapter)db).ObjectContext.ObjectStateManager.ChangeObjectState(gomrokJustification, EntityState.Modified);
}
((IObjectContextAdapter) db).ObjectContext.ObjectStateManager.ChangeObjectState(terminalCertification,EntityState.Modified);
}
and my TerminalCetrification has a list of GomrokJustifications which was filled before by some entities. I want to those last entity being replaced by new ones. but this was not happen.
does anyone have any idea?
Instead of doing this:
var lastCertification = db.terminalCertifications
.Include("TimeInfo")
.Include("GomrokJustifications")
.Where(item=>item.TerminalCertificationID==terminalCertification.TerminalCertificationID)
.ToList();
if (lastCertification.Count==0)
throw new TerminalCertificationNotFoundException(terminalCertification);
you could just do this:
var lastCertification = db.terminalCertifications
.Include("TimeInfo")
.Include("GomrokJustifications")
.Where(item=>item.TerminalCertificationID==terminalCertification.TerminalCertificationID)
.FirstOrDefault();
if (lastCertification == null)
throw new TerminalCertificationNotFoundException(terminalCertification);
First throws an exception if there are no elements in the collection, so if you don't care about the terminalcertificationnotfoundexception you could even remove that custom exception. Your logic even seems to assume that there will be only one element in the returned list so you could even use Single(). That expresses more what you want to achieve compared to calling tolist and then retrieving the first item.
After looking carefully at your code I actually don't get the point you are trying to achieve here. You have an existing terminalcertification entity to start with, you then retrieve it again in that first query, why? You then take the timeinfo from conceptually the same entity (cause you did a get by id) to the one you get as input parameter. Why not continue working on the one that was retrieved from the database? You then detach the entity you received from the database, why? And continue working with the input terminalcertification. I think you need to look a bit more carefully on the entity framework documentation about entity state etc. Take a look at ApplyCurrentValues and detaching and attaching objects here: http://msdn.microsoft.com/en-us/library/bb896271.aspx
We'll need some more info to help you along.