We Keep Coding

SingleOrDefault and FirstOrDefault returning cached data

Some previous code I had written used the Find() method to retrieve single entities by their primary key:
return myContext.Products.Find(id)
This worked great because I had this code tucked into a generic class, and each entity had a different field name as its primary key.
But I had to replace the code because I noticed that it was returning cached data, and I need it to return data from the database each call. Microsoft's documentation confirmed this is the behavior of Find().
So I changed my code to use SingleOrDefault or FirstOrDefault. I haven't found anything in documentation that states these methods return cached data.
Now I am executing these steps:
Save an entity via EF.
Execute an UPDATE statement in SSMS to update the recently saved
record's Description field.
Retrieve the entity into a new entity variable using SingleOrDefault
or FirstOrDefault.
The entities being returned still have the old value in the Description field.
I have run a SQL trace, and verified that the data is being queried during step 3. This baffles me - if EF is making a round trip to the database, why is it returning cached data?
I've searched online, and most answers apply to the Find() method. Furthermore, they suggest some solutions that are merely workarounds (dispose the DbContext and instantiate a new one) or solutions that won't work for me (use the AsNoTracking() method).
How can I retrieve my entities from the database and bypass the EF cache?

The behaviour you're seeing is described in Microsoft's How Queries Work article under point 3:
For each item in the result set
a. If this is a tracking query, EF checks if the data represents an entity already in the change tracker for the context instance
If so, the existing entity is returned
It's described a little better in this blog post:
It turns out that Entity Framework uses the Identity Map pattern. This means that once an entity with a given key is loaded in the context’s cache, it is never loaded again for as long as that context exists. So when we hit the database a second time to get the customers, it retrieved the updated 851 record from the database, but because customer 851 was already loaded in the context, it ignored the newer record from the database (more details).
All of this is saying that if you make a query, it checks the primary key first to see if it already has it in the cache. If so, it uses what's in the cache.
How do you avoid it? The first is to make sure you're not keeping your DbContext object alive too long. DbContext objects are only designed to be used for one unit of work. Bad things happen if you keep it around too long, like excessive memory consumption.
Do you need to retrieve data to display to the user? Create a DbContext to get the data and discard that DbContext.
Do you need to update a record? Create a new DbContext, update the record and discard that DbContext.
This is why, when you use EF Core with dependency injection in ASP.NET Core, it is created with a scoped lifetime, so any DbContext object only lives for the life of one HTTP request.
In the rare case you really do need to get fresh data for a record you already have an object for, you can use EntityEntry.Reload()/EntityEntry.ReloadAsync like this:
myContext.Entry(myProduct).Reload();
That doesn't help you if you only know the ID though.
If you really really need to reload an entity that you only have the ID for, you could do something weird like this:
private Product GetProductById(int id) {
//check if it's in the cache already
var cachedEntity = myContext.ChangeTracker.Entries<Product>()
.FirstOrDefault(p => p.Entity.Id == id);
if (cachedEntity == null) {
//not in cache - get it from the database
return myContext.Products.Find(id);
} else {
//we already have it - reload it
cachedEntity.Reload();
return cachedEntity.Entity;
}
}
But again, this should only be used in limited cases, when you've already addressed any cases of long-living DbContext object because unwanted caching isn't the only consequence.

Ok, I have the same problem and finally found the answer,
You doing everything right, that's just how EF works.
You can use .AsNoTracking() for your purposes:
return myContext.Products.AsNoTracking().Find(id)
make sure you addedusing Microsoft.EntityFrameworkCore; at the top.
It works like a magic

Entity Framework 6 - .Create new object, save changes, re-query object based on new id, null return

I recently upgraded our solution from EF4.1 to EF6. Our previous create method added the detached object to the context, saved changes, then requeried the object based on the new id. We requery the object because we do not use lazy loading, and use includes/selects to get navigational properties - which are added by taking the IDbSet and returning an IQueryable. In short, I cannot just return the existing in-memory copy - because it would not be fully loaded up. I have tried detaching the in-memory copy. I cannot flush the dbcontext (because I am working with other objects in the same context).
Anyway, the big issue, my big issue is that in the EF6 branch when I requery that object it returns a null record - not finding the one I just created. The record is persisted to the database - I can see it. If I create a new dbcontext and query for that record I can find it.
dbContext.Set<TEntity>().Add(model);
dbContext.SaveChanges();
var newCopy = dbContext.Set<TEntity>().SingleOrDefault(p => p.Id == model.Id);
In this instance, newCopy is always null. But if I get a new dbContext - I can get the object fine.
Anyway, I need to solve this issue. Any ideas?

I actually discovered what the issue was/is. In EF4, it really doesn't matter if the mappings are HasRequired or HasOptional. However, if you use a HasRequired on a nullable integer field (legacy db) - then it actually is performing an inner join and will not select the record - resulting in a return of null. Since I am not using lazy loading - but rather eager loading - if I have an include on the above described mapped field - it returns a null.
Thanks everyone for their help - it was appreciated!

Does the following work:
dbContext.Set<TEntity>().Add(model);
dbContext.SaveChanges();
var newCopy = dbContext.Set<TEntity>().Find(model.Id);
I won't claim to know the ins an outs of it but I am using that in the same situation and it works.
You could also check that the model properties are not virtual as EF6 takes this as lazy loading (You may have already done this).

Reflect all changes in the ObjectContext without persisting to the database

Because ObjectContext.SaveChanges() occurs within a transaction I decided it would be best for my app to first perform all queries / updates on the ObjectContext before calling SaveChanges() once to write the data to the database.
However doing this means that if I create an object and subsquently query the ObjectContext it denies it exists (presumbily because the database has yet to be updated). I assumed I could get around this with some of the SaveOptions but it doesn't look like I can.
I basically want the ObjectContext to act like a proxy that I can modify as I wish to commit all in once go. Is there a way I can achieve this?

If you don't save changes your new objects don't exist in the database and no query executed against database (= no Linq to entities query) will return them. To get objects added to context but not saved yet you must query ObjectStateManager:
var data = context.ObjectStateManager
.GetObjectStateEntries(EntityState.Added)
.Where(e => !e.IsRelationship)
.Select(e => e.Entity)
.OfType<YourEntityType>();

I've had to do something along these lines in a similar situation:
Given a list of keys and values, where some of the keys are new, and wanting to update/insert keys and values in a single SaveChanges, I first pull down all the existing keys, then maintain a separate collection of keysIncludingNewOnes, which starts off containing just the existing keys, but also gets any newly-created keys added to it.
Then when I'm looking for a key object to associate with a value, I look in keysIncludingNewOnes, rather than querying the context, since (as you have found) the context doesn't know about the new keys until a SaveChanges.
It should be relatively easy for you to remember about your 'pending addition' objects; if it's not you may have a too-long-lived context, which in general creates its own problems.

EF4 update a value for all rows in a table without doing a select

I need to reset a boolean field in a specific table before I run an update.
The table could have 1 million or so records and I'd prefer not to have to have to do a select before update as its taking too much time.
Basically what I need in code is to produce the following in TSQL
update tablename
set flag = false
where flag = true
I have some thing close to what I need here http://www.aneyfamily.com/terryandann/post/2008/04/Batch-Updates-and-Deletes-with-LINQ-to-SQL.aspx
but have yet to implement it but was wondering if there is a more standard way.
To keep within the restrictions we have for this project, we cant use SPROCs or directly write TSQL in an ExecuteStoreCommand parameter on the context which I believe you can do.
I'm aware that what I need to do may not be directly supported in EF4 and we may need to look at a SPROC for the job [in the total absence of any other way] but I just need to explore fully all possibilities first.
In an EF ideal world the call above to update the flag would be possible or alternatively it would be possible to get the entity with the id and the boolean flag only minus the associated entities and loop through the entity and set the flag and do a single SaveChanges call, but that may not be the way it works.
Any ideas,
Thanks in advance.
Liam

I would go to stakeholder who introduced restirctions about not using SQL or SProc directly and present him these facts:
Updates in ORM (like entity framework) work this way: you load object you perform modification you save object. That is the only valid way.
Obviously in you case it would mean load 1M entities and execute 1M updates separately (EF has no command batching - each command runs in its own roundtrip to DB) - usually absolutely useless solution.
The example you provided looks very interesting but it is for Linq-To-Sql. Not for Entity framework. Unless you implement it you can't be sure that it will work for EF, because infrastructure in EF is much more complex. So you can spent several man days by doing this without any result - this should be approved by stakeholder.
Solution with SProc or direct SQL will take you few minutes and it will simply work.
In both solution you will have to deal with another problem. If you already have materialized entities and you will run such command (via mentioned extension or via SQL) these changes will not be mirrored in already loaded entities - you will have to iterate them and set the flag.
Both scenarios break unit of work because some data changes are executed before unit of work is completed.
It is all about using the right tool for the right requirement.
Btw. loading of realted tables can be avoided. It is just about the query you run. Do not use Include and do not access navigation properties (in case of lazy loading) and you will not load relation.
It is possible to select only Id (via projection), create dummy entity (set only id and and flag to true) and execute only updates of flag but it will still execute up to 1M updates.
using(var myContext = new MyContext(connectionString))
{
var query = from o in myContext.MyEntities
where o.Flag == false
select o.Id;
foreach (var id in query)
{
var entity = new MyEntity
{
Id = id,
Flag = true
};
myContext.Attach(entity);
myContext.ObjectStateManager.GetObjectStateEntry(entity).SetModifiedProperty("Flag");
}
myContext.SaveChanges();
}
Moreover it will only work in empty object context (or at least no entity from updated table can be attached to context). So in some scenarios running this before other updates will require two ObjectContext instances = manually sharing DbConnection or two database connections and in case of transactions = distributed transaction and another performance hit.

Make a new EF model, and only add the one Table you need to make the update on. This way, all of the joins don't occur. This will greatly speed up your processing.

ObjectContext.ExecuteStoreCommand ( _
commandText As String, _
ParamArray parameters As Object() _
) As Integer
http://msdn.microsoft.com/en-us/library/system.data.objects.objectcontext.executestorecommand.aspx
Edit
Sorry, did not read the post all the way.

Error Using ADO.NET Entity Framework

I want to convert a list to EntityCollection.
List<T> x = methodcall();
EntityCOllection<T> y = new EntityCollection<T>();
foreach(T t in x)
y.Add(t);
I get this error.
The object could not be added to the
EntityCollection or EntityReference.
An object that is attached to an
ObjectContext cannot be added to an
EntityCollection or EntityReference
that is not associated with a source
object.
Anyone know about this error?

It sounds like x is the result of an ObjectContext query. Each ObjectContext tracks the entities it reads from the database to enable update scenarios. It tracks the entities to know when (or if) they are modified, and which properties are modified.
The terminology is that the entities are attached to the ObjectContext. In your case, the entities in x are still attached to the ObjectContext that materialized them, so you can't add them to another EntityCollection at the same time.
You may be able to do that if you first Detach them, but if you do that, the first ObjectContext stops tracking them. If you never want to update those items again, it's not a problem, but if you later need to update them, you will have to Attach them again.

Basically all entity objects are controlled by an object context which serves as change tracker. The idea here is that the entities themselves are dumb to their environment, but the object context knows what's going on.
This is an inversion of the DataSet model where the tables track their own changes.
So objects are added to an object context and its entity collections directly. Here you've created an EntityCollection that's not associated with an object context and therefore can't have other objects added to them. They must first be attached to the object context.
Really what you probably want is to return IQueryable instead of IList. That would allow you to execute queries against the results of methodcall().