static Object LockEx=new Object();
public void SaveMyData(IEnumerable<MyData> list)
{
lock (LockEx)
{
using (PersistencyContext db = new PersistencyContext())
{
foreach (var el in list)
{
try
{
db.MyData.Add(el);
db.SaveChanges();
}
catch (DbUpdateException)
{
db.Entry(el).State = EntityState.Modified;
db.SaveChanges();
}
}
}
}
}
This methods is called from multiple threads. Right now I use a static lock to avoid 2 threads to save data at the same time. Though this is wrong because I only want to save data. The catch is used to create an update query in case the insert (Add) fails because the entry already exists.
What happens if I remove the lock. How will the SaveChanges work? How should my code look like? Thanks
I would remove the lock because the database already handles concurrency anyway by design, then I will also verify if the record exists before trying to add it, then I would do the add or update depending on this result. Just to avoid exceptions because they are performance killers.
Building on Davide's answer, you could also call SaveChanges once after you added all the new entities. That should be faster.
Related
I need to insert some objects (about 4 million) in the database using C# and EF (using .NET 3.5). My method that adds the objects is in a for:
private DBModelContainer AddToContext(DBModelContainer db, tblMyTable item, int count)
{
db.AddTottblMyTable (item);
if ((count % 10000== 0) || (count == this.toGenerate))
{
try
{
db.SaveChanges();
}
catch (Exception e)
{
Console.WriteLine(e.StackTrace);
}
}
return db;
}
How to detach the added objects (of type tblMyTable) from the context object? I don't need them for a later use and when more than 300000 objects are added, the execution time between db saving ( db.SaveChanges()) increases considerably.
Regards
Entity Framework may not be the best tool for this type of operation. You may be better off with plain ADO.Net, some stored procedures... But if you had to use it, here are a number of suggestions:
Keep the active Context Graph small by using a new context for each
Unit of Work
Turn off AutoDetechChangesEnabled - context.Configuration.AutoDetectChangesEnabled = false;
Batching, in your loop, Call SaveChanges periodically
EDIT
using(var db = new DBModelContainer())
{
db.tblMyTable.MergeOption = MergeOption.NoTracking;
// Narrow the scope of your db context
db.AddTottblMyTable (item);
db.SaveChanges();
}
Keeping a long running db context is not advisable, so consider refactoring your Add method to not keep attempting to reuse the same context.
See Rick Strahl's post on bulk inserts for more details
AFAK EF does not support directly the BulkInsert so it will be tedious to do such thing manually.
try to consider EntityFramework.BulkInsert
using (var ctx = GetContext())
{
using (var transactionScope = new TransactionScope())
{
// some stuff in dbcontext
ctx.BulkInsert(entities);
ctx.SaveChanges();
transactionScope.Complete();
}
}
You may try Unit Of Work and dont save context (SaveChanges) on every record insert but save it at end
I am currently trying to save an EntityCollection that is populated with both new and Dirty Entity objects in different scenarios.
I have set up a transaction to roll back in the Event of failure while saving.
However, it always seems to fail and throws an Error...in both cases, saving a new or an existing EntityCollection.
I also have a method that picks and adds individual Entities i.e LanguagetranslationEntity to an Entitycollection that is defined as property in the class.
public EntityCollection<LanguageTranslationEntity> LanguagetranslationCollection { get; set; }
public void AddLanguageTranslationToCollection(LanguageTranslationEntity prompt,bool isnew)
{
//Add the prompt to the collection
LanguagetranslationCollection.Add(prompt);
Isnewcollection = isnew;
}
However, an exception is always thrown regardless of whether i try to save new or old entities like as shown below.
An exception was caught during the execution of an action query: Violation of PRIMARY KEY constraint 'PK_LanguageTranslations'. Cannot insert duplicate key in object 'dbo.LanguageTranslations'. The duplicate key value is (translation_10374, 1).
public void SaveLanguageTranslationCollection(DataAccessAdapter adapter)
{
using (DataAccessAdapter newadapter = adapter)
{
adapter.SaveEntityCollection(LanguagetranslationCollection);
}
}
Should i save each Entity on its Own?? and also, how should i use the SaveEntityCollection()?
I intend to use it for saving a number of LanguageTranslationEntities by populating them into an EntityCollection and saving them all at once,using a Transaction for purposes of Rollback in the Event an Exception is thrown.
Kindly help
The exception suggests that one of the entities inside LanguagetranslationCollection is marked as 'new' but the primary key is already used in your DB.
So, you don't have to save them individually, but it actually could help to identify what is the duplicate entity. Once you identify it, you can investigate further Why is it using an already used PK.
I finally figured it out :-)
Within every transaction, one must always remember that they shouldnt have any methods reinitializing the DataaccessAdapter i.e
using(var adapter = new DataAccessAdapter())
{
//Do saving here
SaveLanguageTranslationCollection(adapter);
};
this is what causes the OurOfSyncException to be thrown,as the state data is cleared and initialized a new for the transaction that had been created with the initial dataAccessAdapter.
here is an Example.
public void Save(PromptEntity prompt)
{
using (var adapter = new DataAccessAdapter())
{
//start transaction
adapter.StartTransaction(IsolationLevel.ReadCommitted, "SavePrompt");
try
{
//saving occurs here.
adapter.SaveEntity(prompt);
SaveLanguageTranslationCollection(adapter);
adapter.Commit();
}
catch (Exception)
{
adapter.Rollback();
throw;
}
}
}
you must pass the same adapter running the transaction to the methods saving. i.e
private void savetranslationprompt(LanguageTranslationEntity translationentity,
DataAccessAdapter adapter)
{
adapter.SaveEntity(translationentity);
}
I am new to Entity Framework so please bear with me.
I have a program that I want to select multiple records from a table and store it in a queue:
private Queue<RecordsToProcess> getRecordsToProcess()
{
Queue<RecordsToProcess> results = new Queue<RecordsToProcess>();
using (MyEntity context = new MyEntity())
{
var query = from v in context.RecordsToProcess
where v.Processed == false
select v;
foreach (RecordsToProcess record in query)
{
results.Enqueue(record);
}
}
}
Then I spin up multiple worker threads. Each worker thread takes one of the items in queue, processes it, and then saves it to the database.
private void processWorkerThread(object stateInfo)
{
while (workQueue.Count > 0)
{
RecordToProcess record = new RecordToProcess;
lock(workQueue)
{
if (workQueue.Count > 0)
RecordToProcess = workQueue.Dequeue();
else
break;
}
//Do the record processing here
//How do I save that record here???
}
}
My understanding is that to save changes back to the database you just call context.SaveChanges() but I can't do that in this situation can I?
Any help is appreciated.
Thanks!
Since you are disposing your MyEntity context in the first method (by wrapping it in a using statement), the entities that are enqueued will be in a "detached" state. That means, among other things, that changes done to the entity will not be tracked and you will not be able to lazy load navigation properties.
It is perfectly fine to dequeue these entities, "attaching" them to a different context, update them, and then call SaveChanges to persist the changes.
You can read about Attaching and Detaching Objects and Add/Attach and Entity States
It might be safer if you save off the primary key in the queue instead and retrieve the entities again. This way you are more likely avoid any data concurrency issues.
I am trying to handle a situation in a bulk insert process where there may be entities with the same primary key, which of course is going to make SaveChanges throw an exception.
Here's what I have:
try
{
_context.SaveChanges();
_context.Dispose();
_context = null;
_context = SelectContext<T>();
_commitCount = 0;
}
catch (System.Data.UpdateException updateEx)
{
//Remove from _context all the entries that errored in the SaveChange() process...
if (updateEx.StateEntries != null)
{
foreach (ObjectStateEntry stateEntry in updateEx.StateEntries)
{
if ((System.Data.EntityState)stateEntry.Entity.GetType().GetProperty("EntityState").GetValue(stateEntry.Entity, null) != System.Data.EntityState.Detached)
{
_context.Detach(stateEntry.Entity);
}
}
}
//Save context changes again this time without erroneous entries...
try
{
_context.SaveChanges();
_context.Dispose();
_context = null;
_context = SelectContext<T>();
_commitCount = 0;
}
catch (Exception ex)
{
//Welp, at the point, I'm clueless...
}
If I look in the ObjectStateManager, the entity is indeed removed (the count goes down by the number of time the foreach loop is iterated.)
But it still throws an exception on the second attempt, whinning about a dupe PK.
I tought detaching an entity was the same is if it was never in the context in the first place. Do I need to do something else ?
Thanks.
Deatch doesn't fully undo the changes. Per the msdn doc "Only the entity is removed; if there are any related objects that are being tracked by the same ObjectStateManager, those will not be detached automatically."
Only the entity you pass to detach is removed. Other changes will cause a failure. Will there be contention for you db? If this application is the only thing making changes to the db you can undo changes with context.Refresh(RefreshMode.StoreWins, object); otherwise, it's a bit more complicated.
You have to do something like this;
var entry = context.ObjectStateManager.GetObjectStateEntry(((IEntityWithKey)object).EntityKey);
for (int i = 0; i < entry.OriginalValues.FieldCount; i++)
{
entry.CurrentValues.SetValue(i, entry.OriginalValues[i]);
}
entry.AcceptChanges();
The above code uses the ObjectSateManager to return all modified objects to their original states, after that context.SaveChanges(); shoudl succeed.
You may also find this useful http://dotnetadventurer.blogspot.com/2010/09/discarding-changes-to-objectcontext-in.html I haven't tried his code and I usually initialize my context differently so I'm not sure if it will work with your example but it's very simple so if it does, it's the route I would go.
This is what I ended up doing in my case:
_context.ObjectStateManager.ChangeObjectState(stateEntry.Entity, System.Data.EntityState.Unchanged);
_context.ApplyOriginalValues<T>(entityType, (T)stateEntry.Entity);
_context.Detach(stateEntry.Entity);
After that, I'm able to save the context changes.
I am building an application which requires me to make use of DataContext's inside threads. My application keeps throwing InvalidOperationException's similar to:
There is already an open DataReader associated with this Command which must be closed first
ExecuteReader requires an open and available Connection. The connection's current state is connecting
These exceptions are intermittant.
Here is a snippet of my code:
var repo = new Repository();
var entities = repo.GetAllEntities();
foreach (var entity in entities)
{
ThreadPool.QueueUserWorkItem(
delegate
{
try
{
ProcessEntity(entity);
}
catch (Exception)
{
throw;
}
});
}
I think it may have something to with passing an entity to a thread from the main thread as the error seems to throw as soon as I try and access a property of entity.
Anyone have any idea's why this is happening and how I can resolve it?
Update
Here is what I decided to go with:
var events = new Dictionary<int, AutoResetEvent>();
var repo = new Repository();
var entities = repo.GetAllEntities();
foreach (var entity in entities)
{
events.Add(entity.ID, new AutoResetEvent(false));
ThreadPool.QueueUserWorkItem(
delegate
{
var repo = new Repository();
try
{
ProcessHierarchy(repo.GetEntity(entity.ID), ReportRange.Weekly);
}
catch (Exception)
{
throw;
}
finally
{
events[entity.ID].Set();
}
});
}
WaitHandle.WaitAll(events.Values.ToArray());
Improvements/Suggestions welcome, but this seems to have done the trick.
The exception is thrown since some properties of entity executes new query while a previous reader has not been closed yet. You cannot execute more than one query on the data context at the same time.
As a workaround you can "visit" the properties you access in ProcessEntity() and make the SQL run prior to the thread.
For example:
var repo = new Repository();
var entities = repo.GetAllEntities();
foreach (var entity in entities)
{
var localEntity = entity; // Verify the callback uses the correct instance
var entityCustomers = localEntity.Customers;
var entityOrders = localEntity.Orders;
var invoices = entityOrders.Invoices;
ThreadPool.QueueUserWorkItem(
delegate
{
try
{
ProcessEntity(localEntity);
}
catch (Exception)
{
throw;
}
});
}
This workaround will execute the SQL only on the main thread and the processing will be done in other threads. You loose here some of the efficiency since all the queries are done in the single thread. This solution is good if you have a lot of logic in ProcessEntity() and the queries are not very heavy.
Try creating the Repository inside the new thread instead of passing it in.
Be aware that a SqlConnection instance is NOT thread safe. Whether you still have an open reader or not. In general, the access of a SqlConnection instance from multiple threads will cause you unpredictable intermittent problems.
See: http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlconnection.aspx
The solution for me was LightSpeed Persistence framework, which is free until 8 entities. Per thread create the unitwork.
http://www.mindscapehq.com/products/LightSpeed/default.aspx