I am developing student marking system using ASP.NET MVC and Entity Framework. There is a heavy method I am using for calculate marking. At a given time there are about 50 users enter marks to system and that heavy method call by all the users. It gives me deadlock most of the time. I am using a TransactionScope.
This is my code:
try
{
using (context = new SIMSDBAPPEntities())
{
using (TransactionScope scope = new TransactionScope())
{
// My heavy calculation
}
scope.Complete();
context.SaveChanges();
}
catch (Exception ex)
{
throw ex;
}
}
My heavy method is running inside a TransactionScope. I want to know whether my code has any problems? If yes, what can I do to avoid deadlock situation?
If you are using just one context for saving data. You dont need to use TransactionScope. when you call SaveChanges Ef automatically enlist all changes and apply them on single transaction. For distributed transactions must use TransactionScope.
[link]Using Transactions or SaveChanges(false) and AcceptAllChanges()?
https://coderwall.com/p/jnniww/why-you-shouldn-t-use-entity-framework-with-transactions
Consider using an async method to do the heavy calculating.
using (var transaction = Database.BeginTransaction())
{
try
{
var heavyCalulation = await heavyCalulationMethodAsync();
await context.SaveChangesAsync();
}
catch
{
transaction.Rollback();
}
}
The only way to avoid a deadlock is to access and update records in the same table order every time for every operation that does an update.
This can be very tricky to accomplish in ORMs like EF and nHibernate because the concept of table operations is hidden behind the implementation. You're at the mercy of whatever strategy the framework decides. To work around it, you'd have to be careful to update objects individually and save your changes on a per-object basis. You'd also have to make sure you always save these objects in the same order for every operation that does an update.
What I've done in the past to minimize this issue is to enable the database for isolated snapshots, and use that method for your transaction scope.
https://msdn.microsoft.com/en-us/library/tcbchxcb(v=vs.110).aspx
There are some things that can go wrong here as well (such as writing back to a stale record). But I've found it happens less often than deadlocks.
I found this article helpful too (and gives examples of snapshot isolation)..
http://blogs.msdn.com/b/diego/archive/2012/04/01/tips-to-avoid-deadlocks-in-entity-framework-applications.aspx
Related
I'm implementing a DAL abstraction layer on top of the C# Mongo DB driver using the Repository + Unit of Work pattern.
My current design is that every unit of work instance opens (and closes) a new Mongo DB session.
The problem is that Mongo DB only allows a 1:1 ratio between a session and a transaction, so multiple units of work under the same .NET transaction will not be possible.
The current implementation is:
public class MongoUnitOfWork
{
private IClientSessionHandle _sessionHandle;
public MongoUnitOfWork(MongoClient mongoClient)
{
_sessionHandle = mongoClient.StartSession();
}
public void Dispose()
{
if (_sessionHandle != null)
{
// Must commit transaction, since the session is closing
if (Transaction.Current != null)
_sessionHandle.CommitTransaction();
_sessionHandle.Dispose();
}
}
}
And because of this, the following code won't work. The first batch of data will be committed ahead of time:
using (var transactionScope = new TransactionScope())
{
using (var unitOfWork = CreateUnitOfWork())
{
//... insert items
unitOfWork.SaveChanges();
} // Mongo DB unit of work implementation will commit the changes when disposed
// Do other things
using (var unitOfWork = CreateUnitOfWork())
{
//... insert some more items
unitOfWork.SaveChanges();
}
transactionScope.Complete();
}
Obviously the immediate answer would be to bring all of the changes into one unit of work, but it's not always possible, and also this leaks the Mongo DB limitation.
I thought about session pooling, so that multiple units of work will use the same session, and commit/rollback when the transient transaction completes/aborts.
What other solutions are possible?
Clarification:
The question here is specifically regarding Unit-Of-Work implementation over MongoDB using the MongoDB 4.0 (or later) built-in transaction support.
I never used MongoDB; do not know anything about it. I am only answering in terms of TransactionScope; so not sure if this will help you.
Please refer the Magic Of TransactionScope. IMO, there are three factors you should look for:
Connection to the database should be opened inside the TransactionScope.
So remember, the connection must be opened inside the TransactionScope block for it to enlist in the ambient transaction automatically. If the connection was opened before that, it will not participate in the transaction.
Not sure but it looks that you can manually enlist the connection opened outside the scope using connection.EnlistTransaction(Transaction.Current).
Looking at your comment and the edit, this is not an issue.
All operations should run on same thread.
The ambient transaction provided by the TransactionScope is a thread-static (TLS) variable. It can be accessed with static Transaction.Current property. Here is the TransactionScope code at referencesource.microsoft.com. ThreadStatic ContextData, contains the CurrentTransaction.
and
Remember that Transaction.Current is a thread static variable. If your code is executing in a multi-threaded environments, you may need to take some precautions. The connections that need to participate in ambient transactions must be opened on the same thread that creates the TransactionScope that is managing that ambient transaction.
So, all the operations should run on same thread.
Play with TransactionScopeOption (pass it to constructor of TransactionScope) values as per your need.
Upon instantiating a TransactionScope by the new statement, the transaction manager determines which transaction to participate in. Once determined, the scope always participates in that transaction. The decision is based on two factors: whether an ambient transaction is present and the value of the TransactionScopeOption parameter in the constructor.
I am not sure what your code expected to do. You may play with this enum values.
As you mentioned in the comment, you are using async/await.
Lastly, if you are using async/await inside the TransactionScope block, you should know that it does not work well with TransactionScope and you might want to look into new TransactionScope constructor in .NET Framework 4.5.1 that accepts a TransactionScopeAsyncFlowOption. TransactionScopeAsyncFlowOption.Enabled option, which is not the default, allows TransactionScope to play well with asynchronous continuations.
For MongoDB, see if this helps you.
The MongoDB driver isn't aware of ambient TransactionScopes. You would need to enlist with them manually, or use JohnKnoop.MongoRepository which does this for you: https://github.com/johnknoop/MongoRepository#transactions
In a web application, we provide paginated search panels for various database tables in our application. We currently allow users to select individual rows, and via a UI, execute some operation in each selected instance.
For example, a panel of document records offers an ability to delete documents. A user may check 15 checkboxes representing 15 document identifiers, and choose Options > Delete. This works just fine.
I wish to offer the users an option to execute some operation for all rows matching the query used to display the data in the panel.
We may have 5,000 documents matching some search criteria, and wish to allow a user to delete all 5,000. (I understand this example is a bit contrived; let's ignore the 'wisdom' to allowing users to delete documents in bulk!)
Execution of a method for thousands of rows is a long-running operation, so I will queue the operation instead. Consider this an equivalent of Gmail's ability to apply a filter to all email conversations matching some search criteria.
I need to execute a query that will return an unknown number of rows, and for each row, insert a row into a queue (in the code below, the queue is represented by ImportFileQueue).
I coded it as follows:
using (var reader = await source.InvokeDataReaderAsync(operation, parameters))
{
Parallel.ForEach<IDictionary<string, object>>(reader.Enumerate(), async properties =>
{
try
{
var instance = new ImportFileQueueObject(User)
{
// application tier calculation here; cannot do in SQL
};
await instance.SaveAsync();
}
catch (System.Exception ex)
{
// omitted for brevity
}
});
}
When running this in a unit test that wraps the call with a Transaction, I receive a System.Data.SqlClient.SqlException: Transaction context in use by another session. error.
This is easily resolved by either:
Change the database call from async to sync, or
Removing the Parallel.Foreach, and iterating through the reader in a serial manner.
I opted for this former:
using (var reader = await source.InvokeDataReaderAsync(operation, parameters))
{
Parallel.ForEach<IDictionary<string, object>>(reader.Enumerate(), properties =>
{
try
{
var instance = new ImportFileQueueObject(User)
{
// Omitted for brevity
};
instance.Save();
}
catch (System.Exception ex)
{
// omitted for brevity
}
});
}
My thought process is, in typical use cases:
the outer reader will often have thousands of rows
the instance.Save() call is "lightweight"; inserting a single row into the db
Two questions:
Is there a reasonable way to use async/await inside the Parallel.Foreach, where the inner code is using SqlConnection (avoiding the TransactionContext error)
If not, given my expected typical use case, is my choice to leverage TPL and forfeit async/await for the single-row saves reasonable
The answer suggested in What is the reason of “Transaction context in use by another session” says:
Avoid multi-threaded data operations if it's possible (no matter
loading or saving). E.g. save SELECT/UPDATE/ etc... requests in a
single queue and serve them with a single-thread worker;
but I'm trying to minimize total execution time, and figured the Parallel.Foreach was more likely to reduce execution time.
It's almost always a bad idea to open a transaction and then wait for I/O while holding it open. You'll get much better performance (and fewer deadlocks) by buffering the data first. If there's more total data than you can easily buffer in memory, buffer it into chunks of a thousand or so rows at a time. Put each of those in a separate transaction if possible.
Whenever you open a transaction, any locks taken remain open until it is committed (and locks get taken whether you want to or not when you're inserting data). Those locks cause other updates or reads without WITH(NOLOCK) to sit and wait until the transaction is committed. In a high-performance system, if you're doing I/O while those locks are held, it's pretty-much guaranteed to cause problems as other callers start an operation and then sit and wait while this operation does I/O outside the transaction.
Consider this scenario:
The repository:
public void Save(object entity) {
lock (static object) {
context.Add(entity);
context.SaveChanges();
}
}
The application:
//somewhere
new Thread(() => Repository.Save(entity)).Start()
//...
This "action" occurs several times. Randomly the db context raises an AccessViolationException during saving operation (unhandled exception despite the try-catch o.0).
I have already read that this kind of exception can be caused by a concurrent access on the context object.
The question is: why does this exception occur in my case? Note that the lock should ensure the thread-safe access. How can I resolve this problem?
I have an idea... I think that the context.SaveChanges is not "blocking". When a thread commits the saving another thread enters in the Save method causes a concurrency access...
What you are doing with your DbContext sounds wrong and will inevitably lead to problems as your application grows. The EntityFramework DbContext is designed as a Unit of Work. Create it, use it, dispose it; and definitely do not start doing things with it on different threads. A transaction cannot span multiple threads simultaneously - it just can't (or even if there was some funky way of wangling it, it shouldn't), every thread gets it's own transaction.
You need to rework things so that whenever you work with your DbContext you do it as:
using (var context = new MyContext())
{
// 1. get entities
// 2. work with entities
// 3. save changes
}
// ALL IN A SINGLE THREAD
The EntityFramework itself will not prevent you from working in some other way, but as you are beginning to discover at some point things will break in a hard to diagnose way.
Can I do any other things (not related to DataBase) in the TransactionScope ?
Will it be a bad practice ?
e.g
Starting a workflow inside the scope.
I don't want to save in DB unless starting the workflow fails.
If it's a bad practice, what would be a good practice?
Thanks in advance.
using (TransactionScope scope = new TransactionScope())
{
using (ComponentCM.Audit auditComponent = new ComponentCM.Audit())
{
using (AccessCM.Biz dataAccess = new AccessCM.Biz())
{
auditComponent.SaveInDB();
dataAccess.SaveinDB()
StartWorkflow();
}
}
scope.Complete();
}
You want transactions to take as little time as possible, as they can cause locking on the database.
In this regard, it is better to do other work outside of the transaction scope.
I agree with Oded.
I'd also add that the scope of the TransactionScope should define the state changes that are defined in your unit of work. In other words, state changes inside the scope should all succeed or fail together and are usually part of the same logical operation.
In order to keep the scope of the as small as possible, one common pattern is optimistic concurrency. When using optimistic concurrency, you typically only open a transaction scope when performing a write operation on the state store. In this case, you are excepting that usually the data you are working on will not have changed (by another operation) between the time that you query and commit.
I'm writing an integration test where I will be inserting a number of objects into a database and then checking to make sure whether my method retrieves those objects.
My connection to the database is through NHibernate...and my usual method of creating such a test would be to do the following:
NHibernateSession.BeginTransaction();
//use nhibernate to insert objects into database
//retrieve objects via my method
//verify actual objects returned are the same as those inserted
NHibernateSession.RollbackTransaction();
However, I've recently found out about TransactionScope which apparently can be used for this very purpose...
Some example code I've found is as follows:
public static int AddDepartmentWithEmployees(Department dept)
{
int res = 0;
DepartmentAdapter deptAdapter = new DepartmentAdapter();
EmployeeAdapter empAdapter = new EmployeeAdapter();
using (TransactionScope txScope = new TransactionScope())
{
res += deptAdapter.Insert(dept.DepartmentName);
//Custom method made to return Department ID
//after inserting the department "Identity Column"
dept.DepartmentID = deptAdapter.GetInsertReturnValue();
foreach(Employee emp in dept.Employees)
{
emp.EmployeeDeptID = dept.DepartmentID;
res += empAdapter.Insert(emp.EmployeeName, emp.EmployeeDeptID);
}
txScope.Complete();
}
return res;
}
I believe that if I don't include the line txScope.Complete() that the data inserted will be rolled back. But unfortunately I don't understand how that is possible... how does the txScope object keep a track of the deptAdapter and empAdapter objects and their transactions on the database.
I feel like I'm missing a bit of information here...am I really able to replace my BeginTransaction() and RollbackTransaction() calls by surrounding my code using TransactionScope?
If not, how then does TransactionScope work to roll back transactions?
Essentially TransactionScope doesn't track your Adapter's, what it does is it tracks database connections. When you open a DB connection the connections will looks if there is an ambient transaction (Transaction Scope) and if so enlist with it. Caution if there are more the one connection to the same SQL server this will escalate to a Distribtued Transaction.
What happens since you're using a using block you are ensuring dispose will be called even if an exception occurs. So if dispose is called before txScope.Complete() the TransactionScope will tell the connections to rollback their transactions (or the DTC).
The TransactionScope class works with the Transaction class, which is thread-specific.
When the TransactionScope is created, it checks to see if there is a Transaction for the thread; if one exists then it uses that, otherwise, it creates a new one and pushes it onto the stack.
If it uses an existing one, then it just increments a counter for releases (since you have to call Dispose on it). On the last release, if the Transaction was not comitted, it rolls back all the work.
As for why classes seem to magically know about transactions, that is left as an implementation detail for those classes that wish to work with this model.
When you create your deptAdapter and emptAdapter instances, they check to see if there is a current transaction on the thread (the static Current property on the Transaction class). If there is, then it registers itself with the Transaction, to take part in the commit/rollback sequence (which Transaction controls, and might propogate to varying transaction coordinators, such as kernel, distributed, etc.).