Some previous code I had written used the Find() method to retrieve single entities by their primary key:
return myContext.Products.Find(id)
This worked great because I had this code tucked into a generic class, and each entity had a different field name as its primary key.
But I had to replace the code because I noticed that it was returning cached data, and I need it to return data from the database each call. Microsoft's documentation confirmed this is the behavior of Find().
So I changed my code to use SingleOrDefault or FirstOrDefault. I haven't found anything in documentation that states these methods return cached data.
Now I am executing these steps:
Save an entity via EF.
Execute an UPDATE statement in SSMS to update the recently saved
record's Description field.
Retrieve the entity into a new entity variable using SingleOrDefault
or FirstOrDefault.
The entities being returned still have the old value in the Description field.
I have run a SQL trace, and verified that the data is being queried during step 3. This baffles me - if EF is making a round trip to the database, why is it returning cached data?
I've searched online, and most answers apply to the Find() method. Furthermore, they suggest some solutions that are merely workarounds (dispose the DbContext and instantiate a new one) or solutions that won't work for me (use the AsNoTracking() method).
How can I retrieve my entities from the database and bypass the EF cache?
The behaviour you're seeing is described in Microsoft's How Queries Work article under point 3:
For each item in the result set
a. If this is a tracking query, EF checks if the data represents an entity already in the change tracker for the context instance
If so, the existing entity is returned
It's described a little better in this blog post:
It turns out that Entity Framework uses the Identity Map pattern. This means that once an entity with a given key is loaded in the context’s cache, it is never loaded again for as long as that context exists. So when we hit the database a second time to get the customers, it retrieved the updated 851 record from the database, but because customer 851 was already loaded in the context, it ignored the newer record from the database (more details).
All of this is saying that if you make a query, it checks the primary key first to see if it already has it in the cache. If so, it uses what's in the cache.
How do you avoid it? The first is to make sure you're not keeping your DbContext object alive too long. DbContext objects are only designed to be used for one unit of work. Bad things happen if you keep it around too long, like excessive memory consumption.
Do you need to retrieve data to display to the user? Create a DbContext to get the data and discard that DbContext.
Do you need to update a record? Create a new DbContext, update the record and discard that DbContext.
This is why, when you use EF Core with dependency injection in ASP.NET Core, it is created with a scoped lifetime, so any DbContext object only lives for the life of one HTTP request.
In the rare case you really do need to get fresh data for a record you already have an object for, you can use EntityEntry.Reload()/EntityEntry.ReloadAsync like this:
myContext.Entry(myProduct).Reload();
That doesn't help you if you only know the ID though.
If you really really need to reload an entity that you only have the ID for, you could do something weird like this:
private Product GetProductById(int id) {
//check if it's in the cache already
var cachedEntity = myContext.ChangeTracker.Entries<Product>()
.FirstOrDefault(p => p.Entity.Id == id);
if (cachedEntity == null) {
//not in cache - get it from the database
return myContext.Products.Find(id);
} else {
//we already have it - reload it
cachedEntity.Reload();
return cachedEntity.Entity;
}
}
But again, this should only be used in limited cases, when you've already addressed any cases of long-living DbContext object because unwanted caching isn't the only consequence.
Ok, I have the same problem and finally found the answer,
You doing everything right, that's just how EF works.
You can use .AsNoTracking() for your purposes:
return myContext.Products.AsNoTracking().Find(id)
make sure you addedusing Microsoft.EntityFrameworkCore; at the top.
It works like a magic
Related
EF Core 6 and .NET 6.
Suppose all my entities have a LastUpdateAt property, which is a DateTime that gets updated every time an entity is added or modified.
I get an entity from the context and show it to the user (web page, WPF window, whatever). At some point, the user clicks a Save button.
Before I save, I want to check if the entity has been updated by someone else since I got my copy. However, I'm struggling to see how to do this.
If I query the context, it just gives me back the entity I already have (including any changes my user has made).
If I refresh the entity, it overwrites the one in my context, losing my user's changes.
How do I check if the database version has a newer time stamp than the one in my context?
Thanks
Moving the discussion here since I need to paste longer text. In this article it's said, during SaveChanges(), if the DATABASE version was modified in the mean time it will throw DbUpdateConcurrencyException. In that exception you have all 3 values and YOU can decide on how to resolve the conflict:
Resolving a concurrency conflict involves merging the pending changes from the current DbContext with the values in the database. What values get merged will vary based on the application and may be directed by user input.
There are three sets of values available to help resolve a concurrency conflict:
Current values are the values that the application was attempting to write to the database.
Original values are the values that were originally retrieved from the database, before any edits were made.
Database values are the values currently stored in the database.
If you are loading an entity, keeping a DbContext instance open, updating that entity, then saving to the same DbContext instance then by default you are relying on EF to manage concurrency. This follows a "last in wins". You can let EF manage the concurrency by adding a [ConcurrencyCheck] on the LastUpdateAt property or using a Row Version via [Timestamp]. This will cause EF to fail updating if the underlying data has been updated. From there you have to decide how you want to handle it.
If you want to perform the concurrency check yourself then there are a couple of options.
Structure your code to shorten the lifespan of the DbContext using either detached entities or projected View Models. This will generally have flow-on benefits to your code performance as the original longer-lived DbContext can easily find ways to cause bloat, or accumulate "poisoned" entities if alive too long. Automapper is a great tool to assist here where you can use ProjectTo to get the view models, then Map(source, destination) to copy the values across afterward. In this way you load the data including the last modified at value, make your changes, then when saving, you load the data, validate the modified at etc. then copy the values across and save.
Scope a DbContext instance to check the data before saving.
.
private DateTime getFooLastUpdateAt(int fooId)
{
using(var context = new AppDbContext())
{
var lastUpdateAt = context.Foos
.Where(x => x.FooId == fooId)
.Select(x => x.LastUpdateAt)
.Single();
return lastUpdateAt;
}
}
This could use an injected DbContext factory or such to create the DbContext instance..
I have this problem where I have 2 entities connected by foreign key.
AEntity: id, idOfEntityB (foreign key, constraint), fields...
BEntity: id, fields...
I save both of them to the database with SaveChanges(), later when I try to get AEntity's idOfEntityB, I succeed but when I try to get BEntity according to the id I got from AEntity, I get nothing:
context.AEntities.Add(new AEntity {
BEntity = new BEntity { ... }
});
context.SaveChanges();
.
.
.
var id1 = context.AEntities.Select(x => x.idOfEntityB);
var bEntities = context.BEntities.Where(x => id1.Contains(x.id));
bEntities has nothing in it. but the fact I was able to have values in id1 is even more confusing since they have foreign key relations (with constraint) and furthermore, id could not be created if it was not saved to the DB.
Later, when I look in the DB I see both entities as should be.
It happens sometimes and I cant reproduce the problem, I cant give more then this as an example since there's a lot of code, I believe it has something to do with caching, and therefore would like to ask if something like that is possible or not and how.
is there a way entities are saved to the DB while the context (a different one used from the context that saved) does not hold all of them in completion?
This is likely the issue you are encountering if you are relying on seeing changes between state changes between different DbContext instances. When a DbContext has loaded entities, then another DbContext instance makes changes to those records or the records change behind the scenes in the database, that original DbContext will not refresh the entities from the database.
EF does support the ability to reload entities from the database, but when dealing with child collections it gets a bit more complicated to perform a full refresh. You effectively need to tell the DbContext to forget all of the child collections, stop tracking the parent, clear the parent's child collection, then re-attach and reload the child collection. I recently covered this in the answer for this question: Replacing a entity collection in Entity Framework Core causes DbContext to fetch the new values when not saved to db. How to reload the collection?
Ultimately a DbContext lifespan should be kept as short as possible.
I am maintaining an application which uses EF Core to persist data to a SQL database.
I am trying to implement a new feature which requires me to retrieve an object from the database (Lets pretend its an order) manipulate it and some of the order lines which are attached to it and save it back into the database. Which wouldn't be a problem but I have inherited some of this code so need to try to stick to the existing way of doing things.
The basic process for data access is :
UI -> API -> Service -> Repository -> DataContext
The methods in the repo follow this pattern (Though I have simplified it for the purposes of this question)
public Order GetOrder(int id)
{
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
The service is where business logic and mapping to DTOs are applied, this is what the GetOrder method would look like :
public OrderDTO GetOrder(int id)
{
var ord = _repo.GetOrder(id);
return _mapper.Map<OrderDto>(ord);
}
So to retrieve and manipulate an order my code would look something like this
public void ManipulateAnOrder()
{
// Get the order DTO from the service
var order = _service.GetOrder(3);
// Manipulate the order
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
_service.SaveOrder(order);
}
And the method in the service which allows this to be saved back to the DB would look something like this:
public void SaveOrder(OrderDTO order)
{
// Get the original item from the database
var original = _repo.GetOrder(order.Id);
// Merge the original and the new DTO together
_mapper.Map(order, original);
_repo.Save(original);
}
Finally the repositories save method looks like this
public void Save(Order order){
_context.Update(order)
_context.SaveChanges();
}
The problem that I am encountering is using this method of mapping the Entities from the context into DTOs and back again causes the nested objects (in this instance the OrderLines) to be changed (or recreated) by AutoMapper in such a way that EF no longer recognises them as being the entities that it has just given to us.
This results in errors when updating along the lines of
InvalidOperationException the instance of ProductLine cannot be tracked because another instance with the same key value for {'Id'} is already being tracked.
Now to me, its not that there is ANOTHER instance of the object being tracked, its the same one, but I understand that the mapping process has broken that link and EF can no longer determine that they are the same object.
So, I have been looking for ways to rectify this, There are two ways that have jumped out at me as being promising,
the answer mentioned here EF & Automapper. Update nested collections
Automapper.Collection
Automapper.collection seems to be the better route, but I cant find a good working example of it in use, and the implementation that I have done doesn't seem to work.
So, I'm looking for advice from anyone who has either used automapper collections before successfully or anyone that has any suggestions as to how best to approach this.
Edit, I have knocked up a quick console app as an example, Note that when I say quick I mean... Horrible there is no DI or anything like that, I have done away with the repositories and services to keep it simple.
I have also left in a commented out mapper profile which does work, but isn't ideal.. You will see what I mean when you look at it.
Repo is here https://github.com/DavidDBD/AutomapperExample
Ok, after examining every scenario and counting on the fact that i did what you're trying to do in my previous project and it worked out of the box.
Updating your EntityFramework Core nuget packages to the latest stable version (3.1.8) solved the issue without modifying your code.
AutoMapper in fact "has broken that link" and the mapped entities you are trying to save are a set of new objects, not previously tracked by your DbContext. If the mapped entities were the same objects, you wouldn't have get this error.
In fact, it has nothing to do with AutoMapper and the mapping process, but how the DbContext is being used and how the entity states are being managed.
In your ManipulateAnOrder method after getting the mapped entities -
var order = _service.GetOrder(3);
your DbContext instance is still alive and at the repository layer it is tracking the entities you just retrieved, while you are modifying the mapped entities -
order.UpdatedBy = "Daneel Olivaw";
order.OrderLines.ForEach(ol=>ol.UpdatedBy = "Daneel Olivaw");
Then, when you are trying to save the modified entities -
_service.SaveOrder(order);
this mapped entities reach the repository layer and DbContext tries to add them to its tracking list, but finds that it already has entities of same type with same Ids in the list (the previously fetched ones). EF can track only one instance of a specific type with a specific key. Hence, the complaining message.
One way to solve this, is when fetching the Order, tell EF not to track it, like at your repository layer -
public Order GetOrder(int id, bool tracking = true) // optional parameter
{
if(!tracking)
{
return _context.Orders.Include(o=>o.OrderLines).AsNoTracking().FirstOrDefault(x=>x.Id == id);
}
return _context.Orders.Include(o=>o.OrderLines).FirstOrDefault(x=>x.Id == id);
}
(or you can add a separate method for handling NoTracking calls) and then at your Service layer -
var order = _repo.GetOrder(id, false); // for this operation tracking is false
I have seen other questions about this same error, but I am unable to correct the error with those suggestions in my code; I think that this is a different problem and not a duplicate.
I have an app that makes a series of rules, of which the user can set properties in the GUI. There is a table of Rules in a connected database, with the primary key on the Rule.Id. When the user saves changes to a rule, the existing rule gets "IsActive=0" to hide it, then a new database record is made with the properties from the GUI written to the database. It looks to the user as though they have edited the rule, but the database actually sees a new rule reflecting the new properties (this allows for a history to be kept), connected to the old rule by another reference field.
In the C# code for the app, the View Model for each rule contains an EF Rule object property. When the user clicks "save" I use the parameters set in the view to build the ruleViewModel.Rule for each ruleViewModel they want to save, with porperties matching the GUI. The MainViewModel contains the DbContext object called dbo, so I use the ruleViewModel.Rule to write to the mainViewModel.dbo.Entry which I save to the Entity Framework. Here are the three basic steps performed for each saveable Rule View Model:
// get the rule from the GUI and use it to make sure we are updating the right rule in EF (which is connected to the mainViewModel)
var dboItem = ruleViewModel.MainViewModel.dbo.Rules.Single(r => r.Id == ruleViewModel.Rule.Id);
// set the values in the EF item to be those we got from the GUI
ruleViewModel.MainViewModel.dbo.Entry(dboItem).CurrentValues.SetValues(ruleViewModel.Rule);
// Save the differences
ruleViewModel.MainViewModel.dbo.SaveChanges();
If the user only saves a single rule, it all works fine, but if they subsequently try to save another, or if they save more than one at once, they get the following error, which is return by the ..SetValues(..) line:
Message = "The property 'Id' is part of the object's key information and cannot be modified. "
I see from other questions on this subject that there is a feature of EF that stops you from writing the same object twice to the database with a different Id, so this error often happens within a loop. I have tried using some of the suggestions, like adding
viewModel.MainViewModel.dbo.Rules.Add(dboItem);
and
viewModel.MainViewModel.dbo.Entry(dboItem).Property(x => x.Id).IsModified = false;
before the SaveChanges() command, but that has not helped with the problem (not to mention changing the function of the code). I see that some other suggestions say that the Entry should be created within the loop, but in this case, the entries are all existing rules in the database - it seems to me (perhaps erroneously) that I cannot create them inside the save loop, since they are the objects over which the loop is built - for each entity I find, I want to save changes.
I'm really confused about what to do and tying myself increasingly in knots trying to fix the error. It's been several days now and my sanity and self-esteem is beginning to wane! Any pointers to get me working in the right direction to stop the error appearing and allow me to set the database values would be really welcome as I feel like I have hit a complete dead end! The first time around the loop, everything works perfectly.
Aside from the questionable location of the DbContext and view models containing entities, this looks like it would work as expected. I'm assuming from the MVVM tag that this is a Windows application rather than a web app. The only issue is that this assumes that the Rule entity in your ruleViewModel is detached from the DbContext. If the DbContext is still tracking that entity reference then getting the entity from the DbContext again would pass you back the same reference.
It would probably be worth testing this once in a debug session. If you add the following:
var dboItem = ruleViewModel.MainViewModel.dbo.Rules.Single(r => r.Id == ruleViewModel.Rule.Id);
bool isReferenceSame = Object.ReferenceEquals(dboItem, ruleViewModel.Rule);
Do you get an isReferenceSame value of True or False? If True, the DbContext in your main view model is still tracking the Rule entity and the whole get dboItem and SetValues isn't necessary. If False, then the ruleViewModel is detached.
If the entities are attached and being tracked then edits to the view model entities would be persisted when you call a SaveChanges on the DbContext. (No load & SetValues needed) This should apply to single or multiple entity edits.
If the entities are detached then normally the approach for updating an entity across DbContext instances would look more like:
var context = mainViewModel.dbo;
foreach( var ruleViewModel in updatedRuleViewModels)
{
// This should associate the Entity in the ruleViewModel with the DbContext and set it's tracking state to Modified.
context.Entry(ruleViewModel.Rule).State = EntityState.Modified;
}
context.SaveChanges();
There are a couple of potential issues with this approach that you should consider avoiding if possible. A DbContext should be kept relatively short lived, so seeing a reference to a DbContext within a ViewModel is a bit of a red flag. Overall I don't recommend putting entity references inside view models or passing them around outside of the scope of the DbContext they were created in. EF certainly supports it, but it requires a bit more care and attention to assess whether entities are tracked or not, and in situations like web applications, opens the domain to invalid tampering. (Trusting the entity coming in where any change is attached or copied across overwriting the data state)
I recently upgraded our solution from EF4.1 to EF6. Our previous create method added the detached object to the context, saved changes, then requeried the object based on the new id. We requery the object because we do not use lazy loading, and use includes/selects to get navigational properties - which are added by taking the IDbSet and returning an IQueryable. In short, I cannot just return the existing in-memory copy - because it would not be fully loaded up. I have tried detaching the in-memory copy. I cannot flush the dbcontext (because I am working with other objects in the same context).
Anyway, the big issue, my big issue is that in the EF6 branch when I requery that object it returns a null record - not finding the one I just created. The record is persisted to the database - I can see it. If I create a new dbcontext and query for that record I can find it.
dbContext.Set<TEntity>().Add(model);
dbContext.SaveChanges();
var newCopy = dbContext.Set<TEntity>().SingleOrDefault(p => p.Id == model.Id);
In this instance, newCopy is always null. But if I get a new dbContext - I can get the object fine.
Anyway, I need to solve this issue. Any ideas?
I actually discovered what the issue was/is. In EF4, it really doesn't matter if the mappings are HasRequired or HasOptional. However, if you use a HasRequired on a nullable integer field (legacy db) - then it actually is performing an inner join and will not select the record - resulting in a return of null. Since I am not using lazy loading - but rather eager loading - if I have an include on the above described mapped field - it returns a null.
Thanks everyone for their help - it was appreciated!
Does the following work:
dbContext.Set<TEntity>().Add(model);
dbContext.SaveChanges();
var newCopy = dbContext.Set<TEntity>().Find(model.Id);
I won't claim to know the ins an outs of it but I am using that in the same situation and it works.
You could also check that the model properties are not virtual as EF6 takes this as lazy loading (You may have already done this).