What does it mean for an SqlConnection to be "enlisted" in a transaction? Does it simply mean that commands I execute on the connection will participate in the transaction?
If so, under what circumstances is an SqlConnection automatically enlisted in an ambient TransactionScope Transaction?
See questions in code comments. My guess to each question's answer follows each question in parenthesis.
Scenario 1: Opening connections INSIDE a transaction scope
using (TransactionScope scope = new TransactionScope())
using (SqlConnection conn = ConnectToDB())
{
// Q1: Is connection automatically enlisted in transaction? (Yes?)
//
// Q2: If I open (and run commands on) a second connection now,
// with an identical connection string,
// what, if any, is the relationship of this second connection to the first?
//
// Q3: Will this second connection's automatic enlistment
// in the current transaction scope cause the transaction to be
// escalated to a distributed transaction? (Yes?)
}
Scenario 2: Using connections INSIDE a transaction scope that were opened OUTSIDE of it
//Assume no ambient transaction active now
SqlConnection new_or_existing_connection = ConnectToDB(); //or passed in as method parameter
using (TransactionScope scope = new TransactionScope())
{
// Connection was opened before transaction scope was created
// Q4: If I start executing commands on the connection now,
// will it automatically become enlisted in the current transaction scope? (No?)
//
// Q5: If not enlisted, will commands I execute on the connection now
// participate in the ambient transaction? (No?)
//
// Q6: If commands on this connection are
// not participating in the current transaction, will they be committed
// even if rollback the current transaction scope? (Yes?)
//
// If my thoughts are correct, all of the above is disturbing,
// because it would look like I'm executing commands
// in a transaction scope, when in fact I'm not at all,
// until I do the following...
//
// Now enlisting existing connection in current transaction
conn.EnlistTransaction( Transaction.Current );
//
// Q7: Does the above method explicitly enlist the pre-existing connection
// in the current ambient transaction, so that commands I
// execute on the connection now participate in the
// ambient transaction? (Yes?)
//
// Q8: If the existing connection was already enlisted in a transaction
// when I called the above method, what would happen? Might an error be thrown? (Probably?)
//
// Q9: If the existing connection was already enlisted in a transaction
// and I did NOT call the above method to enlist it, would any commands
// I execute on it participate in it's existing transaction rather than
// the current transaction scope. (Yes?)
}
I've done some tests since asking this question and found most if not all answers on my own, since no one else replied. Please let me know if I've missed anything.
Q1: Is connection automatically enlisted in transaction?
Yes, unless enlist=false is specified in the connection string. The connection pool finds a usable connection. A usable connection is one that's not enlisted in a transaction or one that's enlisted in the same transaction.
Q2: If I open (and run commands on) a second connection now, with an identical connection string, what, if any, is the relationship of this second connection to the first?
The second connection is an independent connection, which participates in the same transaction. I'm not sure about the interaction of commands on these two connections, since they're running against the same database, but I think errors can occur if commands are issued on both at the same time: errors like "Transaction context in use by another session"
Q3: Will this second connection's automatic enlistment in the current transaction scope cause the transaction to be escalated to a distributed transaction?
Yes, it gets escalated to a distributed transaction, so enlisting more than one connection, even with the same connection string, causes it to become a distributed transaction, which can be confirmed by checking for a non-null GUID at Transaction.Current.TransactionInformation.DistributedIdentifier.
*Update: I read somewhere that this is fixed in SQL Server 2008, so that MSDTC is not used when the same connection string is used for both connections (as long as both connections are not open at the same time). That allows you to open a connection and close it multiple times within a transaction, which could make better use of the connection pool by opening connections as late as possible and closing them as soon as possible.
Q4: If I start executing commands on the connection now, will it automatically become enlisted in the current transaction scope?
No. A connection opened when no transaction scope was active, will not be automatically enlisted in a newly created transaction scope.
Q5: If not enlisted, will commands I execute on the connection now participate in the ambient transaction?
No. Unless you open a connection in the transaction scope, or enlist an existing connection in the scope, there basically is NO TRANSACTION. Your connection must be automatically or manually enlisted in the transaction scope in order for your commands to participate in the transaction.
Q6: If commands on this connection are not participating in the current transaction, will they be committed even if rollback the current transaction scope?
Yes, commands on a connection not participating in a transaction are committed as issued, even though the code happens to have executed in a transaction scope block that got rolled back. If the connection is not enlisted in the current transaction scope, it's not participating in the transaction, so committing or rolling back the transaction will have no effect on commands issued on a connection not enlisted in the transaction scope... as this guy found out. That's a very hard one to spot unless you understand the automatic enlistment process: it occurs only when a connection is opened inside an active transaction scope.
Q7: Does the above method explicitly enlist the pre-existing connection in the current ambient transaction, so that commands I execute on the connection now participate in the ambient transaction?
Yes. An existing connection can be explicitly enlisted in the current transaction scope by calling EnlistTransaction(Transaction.Current). You can also enlist a connection on a separate thread in the transaction by using a DependentTransaction, but like before, I'm not sure how two connections involved in the same transaction against the same database may interact... and errors may occur, and of course the second enlisted connection causes the transaction to escalate to a distributed transaction.
Q8: If the existing connection was already enlisted in a transaction when I called the above method, what would happen? Might an error be thrown?
An error may be thrown. If TransactionScopeOption.Required was used, and the connection was already enlisted in a transaction scope transaction, then there is no error; in fact, there's no new transaction created for the scope, and the transaction count (##trancount) does not increase. If, however, you use TransactionScopeOption.RequiresNew, then you get a helpful error message upon attempting to enlist the connection in the new transaction scope transaction: "Connection currently has transaction enlisted. Finish current transaction and retry." And yes, if you complete the transaction the connection is enlisted in, you can safely enlist the connection in a new transaction.
*Update: If you previously called BeginTransaction on the connection, a slightly different error is thrown when you try to enlist in a new transaction scope transaction: "Cannot enlist in the transaction because a local transaction is in progress on the connection. Finish local transaction and retry." On the other hand, you can safely call BeginTransaction on the SqlConnection while its enlisted in a transaction scope transaction, and that will actually increase ##trancount by one, unlike using the Required option of a nested transaction scope, which does not cause it to increase. Interestingly, if you then go on to create another nested transaction scope with the Required option, you will not get an error, because nothing changes as a result of already having an active transaction scope transaction (remember ##trancount is not increased when a transaction scope transaction is already active and the Required option is used).
Q9: If the existing connection was already enlisted in a transaction and I did NOT call the above method to enlist it, would any commands I execute on it participate in its existing transaction rather than the current transaction scope?
Yes. Commands participate in whatever transaction the connection is enlisted in, regardless of what the active transaction scope is in the C# code.
Nice work Triynko, your answers all look quite accurate and complete to me. Some other things I would like to point out:
(1) Manual enlistment
In your code above, you (correctly) show manual enlistment like this:
using (SqlConnection conn = new SqlConnection(connStr))
{
conn.Open();
using (TransactionScope ts = new TransactionScope())
{
conn.EnlistTransaction(Transaction.Current);
}
}
However, it is also possible to do it like this, using Enlist=false in the connection string.
string connStr = "...; Enlist = false";
using (TransactionScope ts = new TransactionScope())
{
using (SqlConnection conn1 = new SqlConnection(connStr))
{
conn1.Open();
conn1.EnlistTransaction(Transaction.Current);
}
using (SqlConnection conn2 = new SqlConnection(connStr))
{
conn2.Open();
conn2.EnlistTransaction(Transaction.Current);
}
}
There is another thing to note here. When conn2 is opened, the connection pool code doesn't know that you want to later enlist it in the same transaction as conn1, which means that conn2 is given a different internal connection than conn1. Then when conn2 is enlisted, there are now 2 connections enlisted so the transaction must be promoted to MSDTC. This promotion can only be avoided by using automatic enlistment.
(2) Before .Net 4.0, I highly recommend setting "Transaction Binding=Explicit Unbind" in the connection string. This issue is fixed in .Net 4.0, making Explicit Unbind totally unnecessary.
(3) Rolling your own CommittableTransaction and setting Transaction.Current to that is essentially the same thing as what TransactionScope does. This is rarely actually useful, just FYI.
(4) Transaction.Current is thread-static. This means that Transaction.Current is only set on the thread that created the TransactionScope. So multiple threads executing the same TransactionScope (possibly using Task) is not possible.
One other bizarre situation we've seen is that if you construct an EntityConnectionStringBuilder it will muck with TransactionScope.Current and (we think) enlist in the transaction. We've observed this in the debugger, where TransactionScope.Current's current.TransactionInformation.internalTransaction shows enlistmentCount == 1 before constructing, and enlistmentCount == 2 afterward.
To avoid this, construct it inside
using (new TransactionScope(TransactionScopeOption.Suppress))
and possibly outside the scope of your operation (we were constructing it every time we needed a connection).
Related
We've got a web-based, multi-tiered eHealth system platform that uses a class/object hierarchy. As an example, a "patient clinic visit" object may consist of multiple diagnosis objects, multiple medication objects, multiple observation objects, laboratory test objects and so on. Database transactions are handled using the .NET System.Transactions.TransactionScope.
When persisting data into the database each nested object in the hierarchy:
instantiates a TransactionScope (with the default transaction
required option)
instantiates an NpgsqlConnection connection
does its own SQL and
sets transactionsScope.Complete(), if all went well
To simplify, objects in the object hierarchy are doing like the code sample below:
void RootMethod()
{
using(TransactionScope scope = new TransactionScope())
{
/* Perform transactional work here */
SomeMethod();
AnotherMethod();
scope.Complete();
}
}
void SomeMethod()
{
using(TransactionScope scope = new TransactionScope())
{
using(NpgsqlConnection connection = new NpgsqlConnection(connectionString))
{
connection.Open()
/* Perform transactional work here */
scope.Complete();
}
}
}
void AnotherMethod()
{
using(TransactionScope scope = new TransactionScope())
{
using(NpgsqlConnection connection = new NpgsqlConnection(connectionString))
{
connection.Open()
/* Perform transactional work here */
scope.Complete();
}
}
}
Program code that is encapsulated in "using(TransactionScope scope = new TransactionScope()) {...}" code blocks gets enlisted in the same transaction fine on Oracle (with an db Oracle driver) but with Npgsql on Postgres separate transactions would appear to get generated instead of one transaction.
As a result, transactions on Postgres fail because of foreign key constraints - data cannot be persisted in child tables as the separate transactions that get created don't see data inserted in parent tables (in separate transaction) before the data is commited by parent objects.
We've got Enlist=true in the Npgsql connectString and the server parameter max_prepared_transactions set to a bigger number than 0 value on our Postgres server.
The Npgsql driver versions we've tested are 4.0.7 and 4.1.2. The Postgres server version in our devt environment is version 10.
Npgsql documentation https://www.npgsql.org/doc/transactions.html says System.Transactions.TransactionScope is supported (and has been since v3.3), as do other Npgsql related answers we've search for on StackOverflow.
At first glance the Npgsql unit tests would appear to use one database connection in transactional unit tests.
QUESTIONS:
are multiple TransactionScopes and multiple participating database connections as per TransactionScope implementation guidelines (e.g. https://learn.microsoft.com/en-us/dotnet/framework/data/transactions/implementing-an-implicit-transaction-using-transaction-scope) supported on Npgsql?
Anything obvious we are missing here?
As stated above, with Npgsql 4.x each opened database connection in "using(TransactionScope scope = new TransactionScope()) {...}" code block would appear to generate a new transaction instead of enlisting in one and the same transaction.
One needs to close an NpgsqlConnection opened in an outer TransactionScope before opening a second NpgsqlConnection in an inner TransactionScope (to allow Npgsql to internally reuse the same physical connection, without escalation to a distributed transaction to occur).
I am using the SQL Server 2012 and There are multiple SQL Connections to same database inside the TransactionScope Using block. But if first Update SQL Operation does not result desired output then I am skipping next call to SQL Operation and also call to TransactionScope.Complete not called at the end of the Using block.
Most of the information you need is fairly well laid out here Completing a transaction scope
When your application completes all the work it wants to perform in a
transaction, you should call the Complete method only once to inform
the transaction manager that it is acceptable to commit the
transaction. It is very good practice to put the call to Complete as
the last statement in the using block.
Failing to call this method aborts the transaction, because the
transaction manager interprets this as a system failure, or equivalent
to an exception thrown within the scope of the transaction. However,
calling this method does not guarantee that the transaction wil be
committed. It is merely a way of informing the transaction manager of
your status. After calling the Complete method, you can no longer
access the ambient transaction by using the Current property, and
attempting to do so will result in an exception being thrown.
Failing to call the transaction complete will cause in the abortion of the transaction, that is the reason why it is a a very good parcitce to call transaction.complete();
What will happen is that the transaction manager will interpret it as system failure and will throw exceptions (happened to me once, not nice), however and it is a big however, it does not promise the transaction will be committed so you will have to call commit as well.
for example:
using (TransactionScope scope = new TransactionScope())
{
using (SqlConnection connection1 = new SqlConnection(connectString1))
{
// Opening the connection automatically enlists it in the
// TransactionScope as a lightweight transaction.
connection1.Open();
// Create the SqlCommand object and execute the first command.
SqlCommand command1 = new SqlCommand(commandText1, connection1);
returnValue = command1.ExecuteNonQuery();
writer.WriteLine("Rows to be affected by command1: {0}", returnValue);
// If you get here, this means that command1 succeeded. By nesting
// the using block for connection2 inside that of connection1, you
// conserve server and network resources as connection2 is opened
// only when there is a chance that the transaction can commit.
using (SqlConnection connection2 = new SqlConnection(connectString2))
{
// The transaction is escalated to a full distributed
// transaction when connection2 is opened.
connection2.Open();
// Execute the second command in the second database.
returnValue = 0;
SqlCommand command2 = new SqlCommand(commandText2, connection2);
returnValue = command2.ExecuteNonQuery();
writer.WriteLine("Rows to be affected by command2: {0}", returnValue);
}
}
// The Complete method commits the transaction. If an exception has been thrown,
// Complete is not called and the transaction is rolled back.
scope.Complete();
}
the commit will happen at the end of the using block, that is if the TransactionScope object created the work initially. Else, the commit will occur whenever the commit line was called.
After opening a connection, Do I have to explicitly close it? or does .net close it automatically? What about if an exception is thrown?
using (DataContext pContext = new DataContext())
{
pContext.Connection.Open()
using (var trx = pContext.Connection.BeginTransaction())
{
//make changes to objects
//Make calls to ExecuteStoreCommand("Update table SET pee=1 AND poop=2 WHERE ETC");
pContext.SaveChanges();
trx.Commit();
}
}
Does pContext.Connection.Close()` get called in all instances by the framework? I know that in a normal using statement it does, but what about when I manually open it?
I am using the Connection.BeginTransaction because In my tests, (using SaveChanges() alone without transactions) if this code executes and fails for some reason, the commands sent during ExecuteStoreCommand() are saved even though the changes to my objects aren't.
I have to call Connection.Open() before calling pContext.Connection.BeginTransaction() otherwise I get the error:
Inner Exception:
The connection is not open.
Any help would be appreciated.
There is no need to call DataContext.Connection.Open(). In fact, it's bad practice to do so.
The context opens a connection only when it needs to. By calling Connection.Open, you open the connection for far longer that is needed, increasing the chance that locks will accumulate and lead to blocking. It is also a good way to exhaust the connection pool in high traffic systems.
SaveChanges opens a connection and a transaction itself, so there is no need to manually open the connection or start the transaction. From the documentation:
SaveChanges operates within a transaction. SaveChanges will roll back that transaction and throw an exception if any of the dirty ObjectStateEntry objects cannot be persisted
Explicitly using connections and transactions makes sense only if you want to mix raw SQL commands inside the EF session. By itself this is not a good practice, but if you do, you don't need to close the connection because the DataContext will close the connection when it is disposed. There is a sample in the documentation, at How to: Manually Open the Connection from the Object Context
You can also handle a transaction using a TransactionScope, as described in How to: Manage Transactions in the Entity Framework. In this case, you don't need to open the connection as all changes are automatically enlisted in the current transaction. This is described in http://msdn.microsoft.com/en-us/library/bb738523(v=vs.100).aspx and an example would be :
using (DataContext pContext = new DataContext())
{
using (TransactionScope transaction = new TransactionScope())
{
//execute commands
pContext.SaveChanges();
transaction.Complete()
}
}
In any case, manually handling connections and transactions is not trivial. You should read Managing Connections and Transactions for some of the things you need to be aware
This means
using (DataContext pContext = new DataContext())
that pContext is disposed after leaving using
it's equivalent to:
DataContext pContext = new DataContext()
try
{
//do stuff here
}
finally
{
if(pContext!=null)
((IDisposable)pContext).Dispose();
}
so answer is -> you don't need to call close after your code because
Connections are released back into the pool when you call Close or Dispose on the Connection..."
from here
If I have pooling in the connection string set to true the following code works fine. If turned of it fails with: "MSDC is unavailable".
Is it just pure luck that both DataContext's picks the same connection from the pool when pooling is on or is there some kind of coordination done by the TransactionScope?
using(var scope = new TransactionScope())
{
using(var db = new DataContext(connectionString))
{
//Do stuff
}
using(var db = new DataContext(connectionString))
{
//Do stuff
}
scope.Complete();
}
In my real code I can't currently pass a specific connection to the DataContext but have use the connection string. Also I would like to avoid using the Distributed Transaction Coordinator if possible.
You are getting lucky here because both of your connection strings match, so you are getting the same pooled connection when pooling is enabled. However, the results will be undefined if there are multiple concurrent accesses to connections with this connection string, because you may not get the same pooled connection in the second DataContext instance.
If you have to create two DataContext objects within the same transaction, then you will need to use the DTC. If you can't use the DTC, then you'll need to find a way to use only one DataContext object within the transaction.
While using ADO.NET (maybe i am wrong, i dont know what its called) i notice that i can only begin a transaction with a connection and a command seems to have command.Transaction which gets me the transaction data but doesnt start a transaction itself? Actually while looking i see this in System.Data.SQLite
// Summary:
// The transaction associated with this command. SQLite only supports one transaction
// per connection, so this property forwards to the command's underlying connection.
[Browsable(false)]
[DesignerSerializationVisibility(DesignerSerializationVisibility.Hidden)]
public SQLiteTransaction Transaction { get; set; }
So SQLite only supports one transaction period? i tried opening another connection but then my transaction threw an exception about the DB being locked. So i cannot have more then one connection concurrent as well?
One transaction per connection, yes, but it can have more than one connection (each with its own active transaction).
Update: interesting. I didn't know about shared-cache mode. If your connection is using that mode, only one transaction is available for all the connections using the same shared-cache. See SQLite shared-cache mode.
I'm not sure about multiple connections, it probably has to do with the fact that a connection locks the file since SQLite is a file-based DB and not a server-based DB (on a server-based DB the server keeps all of the files locked and deals with concurrent connections).
You can only have one transaction OPEN at a time. This should be make intuitive sense, since everything that happens after you begin a transaction is in that transaction, until either a rollback or commit. Then you can start a new one. SQLite requires all command to be in a transaction, so if you don't manually open a new one, it'll do so for you.
If you're worried about nested transactions, you can fake them with savepoint. Documentation
Within 1 transaction you can only read/write to 1 connection until the transaction is done. Therefore you have to pass the connection object if you do a business transaction that spans several sql statements like this:
public class TimeTableService
{
ITimeTableDataProvider _provider = new TimeTableDataProvider();
public void CreateLessonPlanner(WizardData wizardData)
{
using (var con = _provider.GetConnection())
using (var trans = new TransactionScope())
{
con.Open();
var weekListA = new List<Week>();
var weekListB = new List<Week>();
LessonPlannerCreator.CreateLessonPlanner(weekListA, weekListB, wizardData);
_provider.DeleteLessonPlanner(wizardData.StartDate, con);
_provider.CreateLessonPlanner(weekListA, con);
_provider.CreateLessonPlanner(weekListB, con);
_provider.DeleteTimeTable(TimeTable.WeekType.A, con);
_provider.StoreTimeTable(wizardData.LessonsWeekA.ToList<TimeTable>(), TimeTable.WeekType.A, con);
_provider.DeleteTimeTable(TimeTable.WeekType.B, con);
_provider.StoreTimeTable(wizardData.LessonsWeekB.ToList<TimeTable>(), TimeTable.WeekType.B, con);
trans.Complete();
}
}
}
The connection and transactoin resources are automatically released/closed by the using-statement.
In every dataprovider method you then do
using(var cmd = new SQLiteCommand("MyStatement",con)
{
// Create params + ExecuteNonQuery
}
The TransactionScope class is new in .NET 3.5 and is doing automatically a rollback if an exception occurs. Easy handling...