I am trying to handle a situation in a bulk insert process where there may be entities with the same primary key, which of course is going to make SaveChanges throw an exception.
Here's what I have:
try
{
_context.SaveChanges();
_context.Dispose();
_context = null;
_context = SelectContext<T>();
_commitCount = 0;
}
catch (System.Data.UpdateException updateEx)
{
//Remove from _context all the entries that errored in the SaveChange() process...
if (updateEx.StateEntries != null)
{
foreach (ObjectStateEntry stateEntry in updateEx.StateEntries)
{
if ((System.Data.EntityState)stateEntry.Entity.GetType().GetProperty("EntityState").GetValue(stateEntry.Entity, null) != System.Data.EntityState.Detached)
{
_context.Detach(stateEntry.Entity);
}
}
}
//Save context changes again this time without erroneous entries...
try
{
_context.SaveChanges();
_context.Dispose();
_context = null;
_context = SelectContext<T>();
_commitCount = 0;
}
catch (Exception ex)
{
//Welp, at the point, I'm clueless...
}
If I look in the ObjectStateManager, the entity is indeed removed (the count goes down by the number of time the foreach loop is iterated.)
But it still throws an exception on the second attempt, whinning about a dupe PK.
I tought detaching an entity was the same is if it was never in the context in the first place. Do I need to do something else ?
Thanks.
Deatch doesn't fully undo the changes. Per the msdn doc "Only the entity is removed; if there are any related objects that are being tracked by the same ObjectStateManager, those will not be detached automatically."
Only the entity you pass to detach is removed. Other changes will cause a failure. Will there be contention for you db? If this application is the only thing making changes to the db you can undo changes with context.Refresh(RefreshMode.StoreWins, object); otherwise, it's a bit more complicated.
You have to do something like this;
var entry = context.ObjectStateManager.GetObjectStateEntry(((IEntityWithKey)object).EntityKey);
for (int i = 0; i < entry.OriginalValues.FieldCount; i++)
{
entry.CurrentValues.SetValue(i, entry.OriginalValues[i]);
}
entry.AcceptChanges();
The above code uses the ObjectSateManager to return all modified objects to their original states, after that context.SaveChanges(); shoudl succeed.
You may also find this useful http://dotnetadventurer.blogspot.com/2010/09/discarding-changes-to-objectcontext-in.html I haven't tried his code and I usually initialize my context differently so I'm not sure if it will work with your example but it's very simple so if it does, it's the route I would go.
This is what I ended up doing in my case:
_context.ObjectStateManager.ChangeObjectState(stateEntry.Entity, System.Data.EntityState.Unchanged);
_context.ApplyOriginalValues<T>(entityType, (T)stateEntry.Entity);
_context.Detach(stateEntry.Entity);
After that, I'm able to save the context changes.
Related
I need to insert some objects (about 4 million) in the database using C# and EF (using .NET 3.5). My method that adds the objects is in a for:
private DBModelContainer AddToContext(DBModelContainer db, tblMyTable item, int count)
{
db.AddTottblMyTable (item);
if ((count % 10000== 0) || (count == this.toGenerate))
{
try
{
db.SaveChanges();
}
catch (Exception e)
{
Console.WriteLine(e.StackTrace);
}
}
return db;
}
How to detach the added objects (of type tblMyTable) from the context object? I don't need them for a later use and when more than 300000 objects are added, the execution time between db saving ( db.SaveChanges()) increases considerably.
Regards
Entity Framework may not be the best tool for this type of operation. You may be better off with plain ADO.Net, some stored procedures... But if you had to use it, here are a number of suggestions:
Keep the active Context Graph small by using a new context for each
Unit of Work
Turn off AutoDetechChangesEnabled - context.Configuration.AutoDetectChangesEnabled = false;
Batching, in your loop, Call SaveChanges periodically
EDIT
using(var db = new DBModelContainer())
{
db.tblMyTable.MergeOption = MergeOption.NoTracking;
// Narrow the scope of your db context
db.AddTottblMyTable (item);
db.SaveChanges();
}
Keeping a long running db context is not advisable, so consider refactoring your Add method to not keep attempting to reuse the same context.
See Rick Strahl's post on bulk inserts for more details
AFAK EF does not support directly the BulkInsert so it will be tedious to do such thing manually.
try to consider EntityFramework.BulkInsert
using (var ctx = GetContext())
{
using (var transactionScope = new TransactionScope())
{
// some stuff in dbcontext
ctx.BulkInsert(entities);
ctx.SaveChanges();
transactionScope.Complete();
}
}
You may try Unit Of Work and dont save context (SaveChanges) on every record insert but save it at end
I have the following update function
public void UpdateBatchDefinition(BatchDefinition batchToUpdate)
{
if (batchToUpdate == null)
{
throw new ArgumentNullException("batchToUpdate");
}
BatchDefinition foundDefinition =
this.context.BatchDefinitions.SingleOrDefault(definition => definition.Id == batchToUpdate.Id);
if (foundDefinition != null)
{
if (!string.IsNullOrWhiteSpace(batchToUpdate.Name))
{
foundDefinition.Name = batchToUpdate.Name;
}
if (!string.IsNullOrWhiteSpace(batchToUpdate.Description))
{
foundDefinition.Description = batchToUpdate.Description;
}
if (!string.IsNullOrWhiteSpace(batchToUpdate.LoadType))
{
foundDefinition.LoadType = batchToUpdate.LoadType;
}
if (batchToUpdate.JobId != Guid.Empty)
{
foundDefinition.JobId = batchToUpdate.JobId;
}
foundDefinition.Tables = batchToUpdate.Tables;
this.context.SaveChanges();
}
}
the issue I am having Is when I am trying to update the Tables list. Tables is a List of Table and Table is a Entity of another table
Tables could be added to, removed from or left alone. I need to update that with what ever is being passed in
when I run this right now I get an 'EntityValidationErrors' error, though it wont tell me what the validation issue actually is.
on Inserting I got the same error but was able to fix it using the following
var underlyingContext = this.context as DbContext;
if (underlyingContext != null)
{
foreach (var table in batchDefinition.Tables)
{
// Need to mark the table entity as unchanged or
// else EF will treat it as a new table
underlyingContext.Entry(table).State = EntityState.Unchanged;
}
}
so I tried using that in this update function
var underlyingContext = this.context as DbContext;
if (underlyingContext != null)
{
foreach (var table in foundDefinition.Tables)
{
// Need to mark the table entity as unchanged or
//else EF will treat it as a new table
underlyingContext.Entry(table).State = EntityState.Unchanged;
}
}
foundDefinition.Tables = batchToUpdate.Tables;
and I get the following error instead:
AcceptChanges cannot continue because the object's key values conflict
with another object in the ObjectStateManager. Make sure that the key
values are unique before calling AcceptChanges.
Any thoughts one what I am missing here?
Change end of your update method like this:
foreach (var t in foundDefinition.Tables.ToList())
Context.Tables.Remove(t);
foundDefinition.Tables = batchToUpdate.Tables;
this.context.SaveChanges();
And about your last error, it is said that there are some duplicates in your context. So, EF can't save the context changes into the db (because there are duplicates in the context!)
In fact, I don't know the last error is from add or delete - you didn't mention clearly. So, I don't know the last two code samples are from your add method, or your update method...
However for update, the trick I mentioned here, must solve your problem for update...
I am new to Entity Framework so please bear with me.
I have a program that I want to select multiple records from a table and store it in a queue:
private Queue<RecordsToProcess> getRecordsToProcess()
{
Queue<RecordsToProcess> results = new Queue<RecordsToProcess>();
using (MyEntity context = new MyEntity())
{
var query = from v in context.RecordsToProcess
where v.Processed == false
select v;
foreach (RecordsToProcess record in query)
{
results.Enqueue(record);
}
}
}
Then I spin up multiple worker threads. Each worker thread takes one of the items in queue, processes it, and then saves it to the database.
private void processWorkerThread(object stateInfo)
{
while (workQueue.Count > 0)
{
RecordToProcess record = new RecordToProcess;
lock(workQueue)
{
if (workQueue.Count > 0)
RecordToProcess = workQueue.Dequeue();
else
break;
}
//Do the record processing here
//How do I save that record here???
}
}
My understanding is that to save changes back to the database you just call context.SaveChanges() but I can't do that in this situation can I?
Any help is appreciated.
Thanks!
Since you are disposing your MyEntity context in the first method (by wrapping it in a using statement), the entities that are enqueued will be in a "detached" state. That means, among other things, that changes done to the entity will not be tracked and you will not be able to lazy load navigation properties.
It is perfectly fine to dequeue these entities, "attaching" them to a different context, update them, and then call SaveChanges to persist the changes.
You can read about Attaching and Detaching Objects and Add/Attach and Entity States
It might be safer if you save off the primary key in the queue instead and retrieve the entities again. This way you are more likely avoid any data concurrency issues.
static Object LockEx=new Object();
public void SaveMyData(IEnumerable<MyData> list)
{
lock (LockEx)
{
using (PersistencyContext db = new PersistencyContext())
{
foreach (var el in list)
{
try
{
db.MyData.Add(el);
db.SaveChanges();
}
catch (DbUpdateException)
{
db.Entry(el).State = EntityState.Modified;
db.SaveChanges();
}
}
}
}
}
This methods is called from multiple threads. Right now I use a static lock to avoid 2 threads to save data at the same time. Though this is wrong because I only want to save data. The catch is used to create an update query in case the insert (Add) fails because the entry already exists.
What happens if I remove the lock. How will the SaveChanges work? How should my code look like? Thanks
I would remove the lock because the database already handles concurrency anyway by design, then I will also verify if the record exists before trying to add it, then I would do the add or update depending on this result. Just to avoid exceptions because they are performance killers.
Building on Davide's answer, you could also call SaveChanges once after you added all the new entities. That should be faster.
edit: in case anyone is wondering, the actionhandler invokes code that creates and disposes the same kind of datacontext, in case that might have anything to do with this behaviour. the code doesn't touch the MatchUpdateQueue table, but i figure i should mention it just in case.
double edit: everyone who answered was correct! i gave the answer to the respondent who suffered most of my questioning. fixing the problem allowed another problem (hidden within the handler) to pop up, which happened to throw exactly the same exception. whoops!
I'm having some issues with deleting items in LINQ. The DeleteOnSubmit call in the code below causes a LINQ Exception with the message "Cannot add an entity with a key that is already in use." I'm not sure what I'm doing wrong here, it is starting to drive me up the wall. The primary key is just an integer autoincrement column and I have no other problems until I try to remove an item from the database queue. Hopefully I'm doing something painfully retarded here that is easy to spot for anyone who isn't me!
static void Pacman()
{
Queue<MatchUpdateQueue> waiting = new Queue<MatchUpdateQueue>();
events.WriteEntry("matchqueue worker thread started");
while (!stop)
{
if (waiting.Count == 0)
{
/* grab any new items available */
aDataContext db = new aDataContext();
List<MatchUpdateQueue> freshitems = db.MatchUpdateQueues.OrderBy(item => item.id).ToList();
foreach (MatchUpdateQueue item in freshitems)
waiting.Enqueue(item);
db.Dispose();
}
else
{
/* grab & dispatch waiting item */
MatchUpdateQueue item = waiting.Peek();
try
{
int result = ActionHandler.Handle(item);
if (result == -1)
events.WriteEntry("unknown command consumed : " + item.actiontype.ToString(), EventLogEntryType.Error);
/* remove item from queue */
waiting.Dequeue();
/* remove item from database */
aDataContext db = new aDataContext();
db.MatchUpdateQueues.DeleteOnSubmit(db.MatchUpdateQueues.Single(i => i == item));
db.SubmitChanges();
db.Dispose();
}
catch (Exception ex)
{
events.WriteEntry("exception while handling item : " + ex.Message, EventLogEntryType.Error);
stop = true;
}
}
/* to avoid hammering database when there's nothing to do */
if (waiting.Count == 0)
Thread.Sleep(TimeSpan.FromSeconds(10));
}
events.WriteEntry("matchqueue worker thread halted");
}
You could do something to the effect of
db.MatchUpdateQueues.DeleteOnSubmit(db.MatchUpdateQueues.Single(theItem => theItem == item));
Just a note as other answers hinted towards Attach.. you will not be able to use attach on a context other then the original context the item was received on unless the entity has been serialized.
Try wrapping the entire inside of the while loop in a using statement for a single data context:
Queue<MatchUpdateQueue> waiting = new Queue<MatchUpdateQueue>();
events.WriteEntry("matchqueue worker thread started");
while (!stop)
{
using (var db = new aDataContext())
{
if (waiting.Count == 0)
{
/* grab any new items available */
List<MatchUpdateQueue> freshitems = db.MatchUpdateQueues
.OrderBy(item => item.id)
.ToList();
foreach (MatchUpdateQueue item in freshitems)
waiting.Enqueue(item);
}
...
}
}
Use:
aDataContext db = new aDataContext();
item = new MatchUpdateQueue { id=item.id }; // <- updated
db.MatchUpdateQueues.Attach(item);
db.MatchUpdateQueues.DeleteOnSubmit(item);
db.SubmitChanges();
Since you are using a new datacontext it doesn't know that the object is already in the db.
Remove the first db.Dispose() dispose. It can be the problem code because the entities keep a reference to their data context, so you don't want to dispose it while you are still be working with the instances. This won't affect connections, as they are open/closed only when doing operations that need them.
Also don't dispose the second data context like that, since an exception won't call that dispose code anyway. Use the using keyword, which will make sure to call dispose whether or not an exception occurs. Also grab the item from the db to delete it (to avoid having to serialize/deserialize/attach).
using (aDataContext db = new aDataContext())
{
var dbItem = db.MatchUpdateQueues.Single(i => i.Id == item.Id);
db.MatchUpdateQueues.DeleteOnSubmit(dbItem);
db.SubmitChanges();
}
try this if your TEntity's (here Area) Primary Key is of type Identity column;
Just it, without any change in your SP or Model:
public void InitForm()
{
'bnsEntity is a BindingSource and cachedAreas is a List<Area> created from dataContext.Areas.ToList()
bnsEntity.DataSource = cachedAreas;
'A nominal ID
newID = cachedAreas.LastOrDefault().areaID + 1;
'grdEntity is a GridView
grdEntity.DataSource = bnsEntity;
}
private void tsbNew_Click(object sender, EventArgs e)
{
var newArea = new Area();
newArea.areaID = newID++;
dataContext.GetTable<Area>().InsertOnSubmit(newArea);
bnsEntity.Add(newArea);
grdEntity.MoveToNewRecord();
}