We Keep Coding

TransactionScope doesn't rollback transactions [duplicate]

What does it mean for an SqlConnection to be "enlisted" in a transaction? Does it simply mean that commands I execute on the connection will participate in the transaction?
If so, under what circumstances is an SqlConnection automatically enlisted in an ambient TransactionScope Transaction?
See questions in code comments. My guess to each question's answer follows each question in parenthesis.
Scenario 1: Opening connections INSIDE a transaction scope
using (TransactionScope scope = new TransactionScope())
using (SqlConnection conn = ConnectToDB())
{
// Q1: Is connection automatically enlisted in transaction? (Yes?)
//
// Q2: If I open (and run commands on) a second connection now,
// with an identical connection string,
// what, if any, is the relationship of this second connection to the first?
//
// Q3: Will this second connection's automatic enlistment
// in the current transaction scope cause the transaction to be
// escalated to a distributed transaction? (Yes?)
}
Scenario 2: Using connections INSIDE a transaction scope that were opened OUTSIDE of it
//Assume no ambient transaction active now
SqlConnection new_or_existing_connection = ConnectToDB(); //or passed in as method parameter
using (TransactionScope scope = new TransactionScope())
{
// Connection was opened before transaction scope was created
// Q4: If I start executing commands on the connection now,
// will it automatically become enlisted in the current transaction scope? (No?)
//
// Q5: If not enlisted, will commands I execute on the connection now
// participate in the ambient transaction? (No?)
//
// Q6: If commands on this connection are
// not participating in the current transaction, will they be committed
// even if rollback the current transaction scope? (Yes?)
//
// If my thoughts are correct, all of the above is disturbing,
// because it would look like I'm executing commands
// in a transaction scope, when in fact I'm not at all,
// until I do the following...
//
// Now enlisting existing connection in current transaction
conn.EnlistTransaction( Transaction.Current );
//
// Q7: Does the above method explicitly enlist the pre-existing connection
// in the current ambient transaction, so that commands I
// execute on the connection now participate in the
// ambient transaction? (Yes?)
//
// Q8: If the existing connection was already enlisted in a transaction
// when I called the above method, what would happen? Might an error be thrown? (Probably?)
//
// Q9: If the existing connection was already enlisted in a transaction
// and I did NOT call the above method to enlist it, would any commands
// I execute on it participate in it's existing transaction rather than
// the current transaction scope. (Yes?)
}

I've done some tests since asking this question and found most if not all answers on my own, since no one else replied. Please let me know if I've missed anything.
Q1: Is connection automatically enlisted in transaction?
Yes, unless enlist=false is specified in the connection string. The connection pool finds a usable connection. A usable connection is one that's not enlisted in a transaction or one that's enlisted in the same transaction.
Q2: If I open (and run commands on) a second connection now, with an identical connection string, what, if any, is the relationship of this second connection to the first?
The second connection is an independent connection, which participates in the same transaction. I'm not sure about the interaction of commands on these two connections, since they're running against the same database, but I think errors can occur if commands are issued on both at the same time: errors like "Transaction context in use by another session"
Q3: Will this second connection's automatic enlistment in the current transaction scope cause the transaction to be escalated to a distributed transaction?
Yes, it gets escalated to a distributed transaction, so enlisting more than one connection, even with the same connection string, causes it to become a distributed transaction, which can be confirmed by checking for a non-null GUID at Transaction.Current.TransactionInformation.DistributedIdentifier.
*Update: I read somewhere that this is fixed in SQL Server 2008, so that MSDTC is not used when the same connection string is used for both connections (as long as both connections are not open at the same time). That allows you to open a connection and close it multiple times within a transaction, which could make better use of the connection pool by opening connections as late as possible and closing them as soon as possible.
Q4: If I start executing commands on the connection now, will it automatically become enlisted in the current transaction scope?
No. A connection opened when no transaction scope was active, will not be automatically enlisted in a newly created transaction scope.
Q5: If not enlisted, will commands I execute on the connection now participate in the ambient transaction?
No. Unless you open a connection in the transaction scope, or enlist an existing connection in the scope, there basically is NO TRANSACTION. Your connection must be automatically or manually enlisted in the transaction scope in order for your commands to participate in the transaction.
Q6: If commands on this connection are not participating in the current transaction, will they be committed even if rollback the current transaction scope?
Yes, commands on a connection not participating in a transaction are committed as issued, even though the code happens to have executed in a transaction scope block that got rolled back. If the connection is not enlisted in the current transaction scope, it's not participating in the transaction, so committing or rolling back the transaction will have no effect on commands issued on a connection not enlisted in the transaction scope... as this guy found out. That's a very hard one to spot unless you understand the automatic enlistment process: it occurs only when a connection is opened inside an active transaction scope.
Q7: Does the above method explicitly enlist the pre-existing connection in the current ambient transaction, so that commands I execute on the connection now participate in the ambient transaction?
Yes. An existing connection can be explicitly enlisted in the current transaction scope by calling EnlistTransaction(Transaction.Current). You can also enlist a connection on a separate thread in the transaction by using a DependentTransaction, but like before, I'm not sure how two connections involved in the same transaction against the same database may interact... and errors may occur, and of course the second enlisted connection causes the transaction to escalate to a distributed transaction.
Q8: If the existing connection was already enlisted in a transaction when I called the above method, what would happen? Might an error be thrown?
An error may be thrown. If TransactionScopeOption.Required was used, and the connection was already enlisted in a transaction scope transaction, then there is no error; in fact, there's no new transaction created for the scope, and the transaction count (##trancount) does not increase. If, however, you use TransactionScopeOption.RequiresNew, then you get a helpful error message upon attempting to enlist the connection in the new transaction scope transaction: "Connection currently has transaction enlisted. Finish current transaction and retry." And yes, if you complete the transaction the connection is enlisted in, you can safely enlist the connection in a new transaction.
*Update: If you previously called BeginTransaction on the connection, a slightly different error is thrown when you try to enlist in a new transaction scope transaction: "Cannot enlist in the transaction because a local transaction is in progress on the connection. Finish local transaction and retry." On the other hand, you can safely call BeginTransaction on the SqlConnection while its enlisted in a transaction scope transaction, and that will actually increase ##trancount by one, unlike using the Required option of a nested transaction scope, which does not cause it to increase. Interestingly, if you then go on to create another nested transaction scope with the Required option, you will not get an error, because nothing changes as a result of already having an active transaction scope transaction (remember ##trancount is not increased when a transaction scope transaction is already active and the Required option is used).
Q9: If the existing connection was already enlisted in a transaction and I did NOT call the above method to enlist it, would any commands I execute on it participate in its existing transaction rather than the current transaction scope?
Yes. Commands participate in whatever transaction the connection is enlisted in, regardless of what the active transaction scope is in the C# code.

Nice work Triynko, your answers all look quite accurate and complete to me. Some other things I would like to point out:
(1) Manual enlistment
In your code above, you (correctly) show manual enlistment like this:
using (SqlConnection conn = new SqlConnection(connStr))
{
conn.Open();
using (TransactionScope ts = new TransactionScope())
{
conn.EnlistTransaction(Transaction.Current);
}
}
However, it is also possible to do it like this, using Enlist=false in the connection string.
string connStr = "...; Enlist = false";
using (TransactionScope ts = new TransactionScope())
{
using (SqlConnection conn1 = new SqlConnection(connStr))
{
conn1.Open();
conn1.EnlistTransaction(Transaction.Current);
}
using (SqlConnection conn2 = new SqlConnection(connStr))
{
conn2.Open();
conn2.EnlistTransaction(Transaction.Current);
}
}
There is another thing to note here. When conn2 is opened, the connection pool code doesn't know that you want to later enlist it in the same transaction as conn1, which means that conn2 is given a different internal connection than conn1. Then when conn2 is enlisted, there are now 2 connections enlisted so the transaction must be promoted to MSDTC. This promotion can only be avoided by using automatic enlistment.
(2) Before .Net 4.0, I highly recommend setting "Transaction Binding=Explicit Unbind" in the connection string. This issue is fixed in .Net 4.0, making Explicit Unbind totally unnecessary.
(3) Rolling your own CommittableTransaction and setting Transaction.Current to that is essentially the same thing as what TransactionScope does. This is rarely actually useful, just FYI.
(4) Transaction.Current is thread-static. This means that Transaction.Current is only set on the thread that created the TransactionScope. So multiple threads executing the same TransactionScope (possibly using Task) is not possible.

One other bizarre situation we've seen is that if you construct an EntityConnectionStringBuilder it will muck with TransactionScope.Current and (we think) enlist in the transaction. We've observed this in the debugger, where TransactionScope.Current's current.TransactionInformation.internalTransaction shows enlistmentCount == 1 before constructing, and enlistmentCount == 2 afterward.
To avoid this, construct it inside
using (new TransactionScope(TransactionScopeOption.Suppress))
and possibly outside the scope of your operation (we were constructing it every time we needed a connection).

EF DBContext dispose not closing the connection

I am using the EF 6.1.0
I have below custom DBContex object as DBEntites
public partial class DbEntities : DbContext
{
public DbEntities()
: base("name=DbEntities")
{
////Configuration.LazyLoadingEnabled = true;
////Configuration.ProxyCreationEnabled = false;
}
//// I have ALL Entites added as below
////public virtual IDbSet<CCode> CCodes { get; set; }
}
I have the below operations on context object
using (var context = new DbEntities())
{
var entitySet = context.Set<T>();
var res = entitySet.Where<T>(predicate).ToList();
if (context.Database.Connection.State == ConnectionState.Open)
{
context.Database.Connection.Close();
}
return res;
}
But after disposing the context object still i can see a active DB Connection. On connection state condition i can see that the connection is already closed(the connection had never true).
I am using the below query to see the connection on SQL.
select db_name(dbid) , count(*) 'connections count'
from master..sysprocesses
where spid > 50 and spid != ##spid
group by db_name(dbid)
order by count(*) desc
At the below statement a sql connection count increased. But it was never down even after disposing . (I mean after using block excuted it supposed to close the connection).
var res = entitySet.Where<T>(predicate).ToList();
Any help would be greatly appreciated.

As was figured out in comments, the reason is indeed connection pooling performed by .NET. .NET maintains a pool of connections for every connection string you use in your application, for perfomance reasons (because opening and closing connections often might be costly in terms of perfomance). That pool has certain minimum and maximum size (controlled by MinPoolSize and MaxPoolSize connection string parameters). When you open a connection (via SqlConnection.Open) - it might be taken out of the pool and not really opened afresh. When you close connection (which is also done by disposing EF context) - connection might be put into the pool instead, and not really closed. When connection is idle for certain time (about 5 minutes) - it might be removed from the pool.
If you (for some reason) want to avoid that, you can either set MaxPoolSize to 0 for your connection string, or clear pool explicitly by SqlConnection.ClearPool or SqlConnection.ClearAllPools.

Just to register my experience here. I had same problem with EF Core 6 + Postgres and the solution was just to disable "Pooling" in the connection string.
Pooling=false;

EF using and BeginTransaction, when to call Connection.Close()?

After opening a connection, Do I have to explicitly close it? or does .net close it automatically? What about if an exception is thrown?
using (DataContext pContext = new DataContext())
{
pContext.Connection.Open()
using (var trx = pContext.Connection.BeginTransaction())
{
//make changes to objects
//Make calls to ExecuteStoreCommand("Update table SET pee=1 AND poop=2 WHERE ETC");
pContext.SaveChanges();
trx.Commit();
}
}
Does pContext.Connection.Close()` get called in all instances by the framework? I know that in a normal using statement it does, but what about when I manually open it?
I am using the Connection.BeginTransaction because In my tests, (using SaveChanges() alone without transactions) if this code executes and fails for some reason, the commands sent during ExecuteStoreCommand() are saved even though the changes to my objects aren't.
I have to call Connection.Open() before calling pContext.Connection.BeginTransaction() otherwise I get the error:
Inner Exception:
The connection is not open.
Any help would be appreciated.

There is no need to call DataContext.Connection.Open(). In fact, it's bad practice to do so.
The context opens a connection only when it needs to. By calling Connection.Open, you open the connection for far longer that is needed, increasing the chance that locks will accumulate and lead to blocking. It is also a good way to exhaust the connection pool in high traffic systems.
SaveChanges opens a connection and a transaction itself, so there is no need to manually open the connection or start the transaction. From the documentation:
SaveChanges operates within a transaction. SaveChanges will roll back that transaction and throw an exception if any of the dirty ObjectStateEntry objects cannot be persisted
Explicitly using connections and transactions makes sense only if you want to mix raw SQL commands inside the EF session. By itself this is not a good practice, but if you do, you don't need to close the connection because the DataContext will close the connection when it is disposed. There is a sample in the documentation, at How to: Manually Open the Connection from the Object Context
You can also handle a transaction using a TransactionScope, as described in How to: Manage Transactions in the Entity Framework. In this case, you don't need to open the connection as all changes are automatically enlisted in the current transaction. This is described in http://msdn.microsoft.com/en-us/library/bb738523(v=vs.100).aspx and an example would be :
using (DataContext pContext = new DataContext())
{
using (TransactionScope transaction = new TransactionScope())
{
//execute commands
pContext.SaveChanges();
transaction.Complete()
}
}
In any case, manually handling connections and transactions is not trivial. You should read Managing Connections and Transactions for some of the things you need to be aware

This means
using (DataContext pContext = new DataContext())
that pContext is disposed after leaving using
it's equivalent to:
DataContext pContext = new DataContext()
try
{
//do stuff here
}
finally
{
if(pContext!=null)
((IDisposable)pContext).Dispose();
}
so answer is -> you don't need to call close after your code because
Connections are released back into the pool when you call Close or Dispose on the Connection..."
from here

sqlite only supports 1 transaction?

While using ADO.NET (maybe i am wrong, i dont know what its called) i notice that i can only begin a transaction with a connection and a command seems to have command.Transaction which gets me the transaction data but doesnt start a transaction itself? Actually while looking i see this in System.Data.SQLite
// Summary:
// The transaction associated with this command. SQLite only supports one transaction
// per connection, so this property forwards to the command's underlying connection.
[Browsable(false)]
[DesignerSerializationVisibility(DesignerSerializationVisibility.Hidden)]
public SQLiteTransaction Transaction { get; set; }
So SQLite only supports one transaction period? i tried opening another connection but then my transaction threw an exception about the DB being locked. So i cannot have more then one connection concurrent as well?

One transaction per connection, yes, but it can have more than one connection (each with its own active transaction).
Update: interesting. I didn't know about shared-cache mode. If your connection is using that mode, only one transaction is available for all the connections using the same shared-cache. See SQLite shared-cache mode.

I'm not sure about multiple connections, it probably has to do with the fact that a connection locks the file since SQLite is a file-based DB and not a server-based DB (on a server-based DB the server keeps all of the files locked and deals with concurrent connections).
You can only have one transaction OPEN at a time. This should be make intuitive sense, since everything that happens after you begin a transaction is in that transaction, until either a rollback or commit. Then you can start a new one. SQLite requires all command to be in a transaction, so if you don't manually open a new one, it'll do so for you.
If you're worried about nested transactions, you can fake them with savepoint. Documentation

Within 1 transaction you can only read/write to 1 connection until the transaction is done. Therefore you have to pass the connection object if you do a business transaction that spans several sql statements like this:
public class TimeTableService
{
ITimeTableDataProvider _provider = new TimeTableDataProvider();
public void CreateLessonPlanner(WizardData wizardData)
{
using (var con = _provider.GetConnection())
using (var trans = new TransactionScope())
{
con.Open();
var weekListA = new List<Week>();
var weekListB = new List<Week>();
LessonPlannerCreator.CreateLessonPlanner(weekListA, weekListB, wizardData);
_provider.DeleteLessonPlanner(wizardData.StartDate, con);
_provider.CreateLessonPlanner(weekListA, con);
_provider.CreateLessonPlanner(weekListB, con);
_provider.DeleteTimeTable(TimeTable.WeekType.A, con);
_provider.StoreTimeTable(wizardData.LessonsWeekA.ToList<TimeTable>(), TimeTable.WeekType.A, con);
_provider.DeleteTimeTable(TimeTable.WeekType.B, con);
_provider.StoreTimeTable(wizardData.LessonsWeekB.ToList<TimeTable>(), TimeTable.WeekType.B, con);
trans.Complete();
}
}
}
The connection and transactoin resources are automatically released/closed by the using-statement.
In every dataprovider method you then do
using(var cmd = new SQLiteCommand("MyStatement",con)
{
// Create params + ExecuteNonQuery
}
The TransactionScope class is new in .NET 3.5 and is doing automatically a rollback if an exception occurs. Easy handling...

.net SqlConnection not being closed even when within a using { }

Please help!
Background info
I have a WPF application which accesses a SQL Server 2005 database. The database is running locally on the machine the application is running on.
Everywhere I use the Linq DataContext I use a using { } statement, and pass in a result of a function which returns a SqlConnection object which has been opened and had an SqlCommand executed using it before returning to the DataContext constructor.. I.e.
// In the application code
using (DataContext db = new DataContext(GetConnection()))
{
... Code
}
where getConnection looks like this (I've stripped out the 'fluff' from the function to make it more readable, but there is no additional functionality that is missing).
// Function which gets an opened connection which is given back to the DataContext constructor
public static System.Data.SqlClient.SqlConnection GetConnection()
{
System.Data.SqlClient.SqlConnection Conn = new System.Data.SqlClient.SqlConnection(/* The connection string */);
if ( Conn != null )
{
try
{
Conn.Open();
}
catch (System.Data.SqlClient.SqlException SDSCSEx)
{
/* Error Handling */
}
using (System.Data.SqlClient.SqlCommand SetCmd = new System.Data.SqlClient.SqlCommand())
{
SetCmd.Connection = Conn;
SetCmd.CommandType = System.Data.CommandType.Text;
string CurrentUserID = System.String.Empty;
SetCmd.CommandText = "DECLARE #B VARBINARY(36); SET #B = CAST('" + CurrentUserID + "' AS VARBINARY(36)); SET CONTEXT_INFO #B";
try
{
SetCmd.ExecuteNonQuery();
}
catch (System.Exception)
{
/* Error Handling */
}
}
return Conn;
}
I do not think that the application being a WPF one has any bearing on the issue I am having.
The issue I am having
Despite the SqlConnection being disposed along with the DataContext in Sql Server Management studio I can still see loads of open connections with :
status : 'Sleeping'
command : 'AWAITING COMMAND'
last SQL Transact Command Batch : DECLARE #B VARBINARY(36); SET #B = CAST('GUID' AS VARBINARY(36)); SET CONTEXT_INFO #B
Eventually the connection pool gets used up and the application can't continue.
So I can only conclude that somehow running the SQLCommand to set the Context_Info is meaning that the connection doesn't get disposed of when the DataContext gets disposed.
Can anyone spot anything obvious that would be stopping the connections from being closed and disposed of when the DataContext they are used by are disposed?

From MSDN (DataContext Constructor (IDbConnection)):
If you provide an open connection, the
DataContext will not close it.
Therefore, do not instantiate a
DataContext with an open connection
unless you have a good reason to do
this.
So basically, it looks like your connections are waiting for GC to finalize them before they will be released. If you have lots of code that does this, one approach might be to overide Dispose() in the data-context's partial class, and close the connection - just be sure to document that the data-context assumes ownership of the connection!
protected override void Dispose(bool disposing)
{
if(disposing && this.Connection != null && this.Connection.State == ConnectionState.Open)
{
this.Connection.Close();
this.Connection.Dispose();
}
base.Dispose(disposing);
}
Personally, I would happily give it (regular data-context, w/o the hack above) an open connection as long as I was "using" the connection (allowing me to perform multiple operations) - i.e.
using(var conn = GetConnection())
{
// snip: some stuff involving conn
using(var ctx = new FooContext(conn))
{
// snip: some stuff involving ctx
}
// snip: some more stuff involving conn
}

The SqlProvider used by the LINQ DataContext only closes the SQL connection (through SqlConnectionManager.DisposeConnection) if it was the one to open it. If you give an already-open SqlConnection object to the DataContext constructor, it will not close it for you. Thus, you should write:
using (SqlConnection conn = GetConnection())
using (DataContext db = new DataContext(conn))
{
... Code
}

I experienced the same issue using the Entity Framework. My ObjectContext was wrapped around a using block.
A connection was established when I called SaveChanges(), but after the using statement was out of scope, I noticed that SQL Management Studio still had a "AWAITING COMMAND" for the .NET SQL Client.
It looks like this has to do with the behavior of the ADO.NET provider which has connection pooling turned on by default.
From "Using Connection Pooling with SQL Server" on MSDN (emphasis mine):
Connection pooling reduces the number of times that new connections need to be opened. The pooler maintains ownership of the physical connection. It manages connections by keeping alive a set of active connections for each given connection configuration. Whenever a user calls Open on a connection, the pooler looks to see if there is an available connection in the pool. If a pooled connection is available, it returns it to the caller instead of opening a new connection. When the application calls Close on the connection, the pooler returns it to the pooled set of active connections instead of actually closing it. Once the connection is returned to the pool, it is ready to be reused on the next Open call.
Also ClearAllPools and ClearPool seems useful to explicitly close all pooled connections if needed.

I think the connection, while no longer referenced, is waiting for the GC to dispose of it fully.
Solution:
Create your own DataContext class which derives from the auto-generated one. (rename the base one so you don't have to change any other code).
In your derived DataContext - add a Dispose() function. In that - dispose the inner connection.

Well thanks for the help chaps, it has been solved now..
Essentially I took elements of most of the answers above and implemented the DataContext constructor as above (I already had overloaded the constructors so it wasn't a big change).
// Variable for storing the connection passed to the constructor
private System.Data.SqlClient.SqlConnection _Connection;
public DataContext(System.Data.SqlClient.SqlConnection Connection) : base(Connection)
{
// Only set the reference if the connection is Valid and Open during construction
if (Connection != null)
{
if (Connection.State == System.Data.ConnectionState.Open)
{
_Connection = Connection;
}
}
}
protected override void Dispose(bool disposing)
{
// Only try closing the connection if it was opened during construction
if (_Connection!= null)
{
_Connection.Close();
_Connection.Dispose();
}
base.Dispose(disposing);
}
The reason for doing this rather than some of the suggestions above is that accessing this.Connection in the dispose method throws a ObjectDisposedException.
And the above works as well as I was hoping!

The Dispose should close the connections, as MSDN points out:
If the SqlConnection goes out of
scope, it won't be closed. Therefore,
you must explicitly close the
connection by calling Close or
Dispose. Close and Dispose are
functionally equivalent. If the
connection pooling value Pooling is
set to true or yes, the underlying
connection is returned back to the
connection pool. On the other hand, if
Pooling is set to false or no, the
underlying connection to the server is
closed.
My guess would be that your problem has something to do with GetContext().