We Keep Coding

Using a common SQL Server connection's between all thread and using at same time

I need every thread to connect to one database and execute some query and maybe some thread in one time can execute the query.
I create a static class for connecting to SQL Server database with ADO.NET:
public static class ADOFire
{
static System.Data.SqlClient.SqlConnection Cnn = new System.Data.SqlClient.SqlConnection();
public static string CnnString { get; set; }
public static void CreateConnection()
{
if (Cnn.State == ConnectionState.Open)
return;
Cnn.ConnectionString = CnnString = ConfigurationManager.ConnectionStrings["CnnString"].ToString();
if (Cnn.State != System.Data.ConnectionState.Open)
Cnn.Open();
}
public static System.Data.DataTable GetTable(System.Data.SqlClient.SqlCommand Com, System.Data.SqlClient.SqlDataAdapter Ada, string ComText)
{
CreateConnection();
Com.Connection = Cnn;
Ada.SelectCommand = Com;
try
{
System.Data.DataTable T = new System.Data.DataTable();
Com.CommandText = ComText;
Ada.Fill(T);
return T;
}
catch { return null; }
}
}
And in here in each thread I call static function like this:
System.Data.SqlClient.SqlCommand Com = new System.Data.SqlClient.SqlCommand();
System.Data.SqlClient.SqlDataAdapter Ada = new System.Data.SqlClient.SqlDataAdapter();
Datatable dt = ADOFire.GetTable(Com, Ada, "Some Query 'select * from x'");
Based on this link, doesn't make much difference between open a new connection or use from existing connection
Whenever a user calls Open on a connection, the pooler looks for an available connection in the pool. If a pooled connection is available, it returns it to the caller instead of opening a new connection. When the application calls Close on the connection, the pooler returns it to the pooled set of active connections instead of closing it. Once the connection is returned to the pool, it is ready to be reused on the next Open call
My questions are:
Can a connection serve different threads at the same time? (one connection for all)
Isn't the problem of data clutter due to the static function?

No, ADO.NET does not two threads accessing the same.conection at the same time
Yes, but it isn't the statix method that is a problem - it is the static connection field that is a huge problem
What you should do is have the static method create (new) and return the connection to the caller, and remove the field completely. Typical usage:
using (var conn = CreateConnection()) {
//... Use it!
}
I also have grave concerns about:
- why you're passing in a command and command text and adapter
- the lack of parameters; suggests a huge security problem (SQL injection)
- the use of data table (which is almost never the most appropriate tool)

How do I hook into TransactionScope on completing "event"?

I want to know if there's a way to hook into a currently running transaction and have stuff be done when that transaction completes.
Currently I'm in the process of implementing an EventPublisher that uses MassTransit/RabbitMQ to publish messages, but I want to only have those messages be published when a TransactionScope is getting completed.
I would check inside the EventPublisher.PublishEvent() method if there's currently a transaction running, if no, then fire off the messages, if yes, then collect the messages and wait for the transaction to complete to send them off.
var ep = container.GetInstance<IEventPublisher>();
using(var scope = new TransactionScope())
{
... do some stuff
SaveEntity(entity);
ep.PublishEvent(new EntitySaved(entity.Id));
... do some more stuff ...
UpdateEntity(differentEntity);
ep.PublishEvent(new EntityUpdated(differentEntity.Id));
... do even more stuff ...
ep.PublishEvent(new UnrelatedMessage(someData));
scope.Complete(); // <- only want the actual sending off to RabbitMQ to happen here.
}
I found this TransactionCompleted Event here https://learn.microsoft.com/en-us/dotnet/api/system.transactions.transaction.transactioncompleted
But it seems to only be fired after the scope.Complete() bits are over and done.
I could use this to check if the transaction status is completed and then like actually fire off those messages I collected during the transaction. But my problem is that the connection to RabbitMQ could be down. And then I wouldn't be able to send those messages, but all the work above has been done already, but I never was able to send off those messages.
What I actually want is to somehow hook into the bit where the transaction is currently completing and during that process I fire off my messages and if that fails I can throw an exception and have that still running transaction go out the window.
Maybe there's a way to do that with MassTransit, but the documentation isn't really forthcoming there.
Here's some example code showing the problem.
internal class Program
{
private static void Main(string[] args)
{
const string connectionString = "Data Source=.;Initial Catalog=MyDatabase;Integrated Security=True";
var sql = #"insert into [SomeTable] (
[Id]
,[Name]
,[Index]
,[RelationId]
) values (#param1, #param2, #param3, #param4) ";
using (var scope = new TransactionScope())
{
Transaction.Current.TransactionCompleted += CurrentOnTransactionCompleted;
using (var con = new SqlConnection(connectionString))
{
con.Open();
using (var cmd = new SqlCommand(sql, con))
{
cmd.Parameters.Add("#param1", SqlDbType.UniqueIdentifier).Value = Guid.NewGuid();
cmd.Parameters.Add("#param2", SqlDbType.NVarChar, 128).Value = "Blah";
cmd.Parameters.Add("#param3", SqlDbType.SmallInt).Value = 1;
cmd.Parameters.Add("#param4", SqlDbType.UniqueIdentifier).Value = Guid.Parse("a401866d-3bdd-48a4-a78b-d40864c8471b");
cmd.CommandType = CommandType.Text;
cmd.ExecuteNonQuery();
}
}
scope.Complete();
}
}
private static void CurrentOnTransactionCompleted(object sender, TransactionEventArgs e)
{
// I want to do stuff here but if this stuff fails I need the whole transaction to roll back.
... do some stuff that can fail ...
e.Transaction.Rollback(new Exception("Bad transaction!"));
// or
throw new Exception("Bad transaction!");
}
}

I was looking in the wrong place. After looking through microsoft/referencesource I found what I was looking for in TransactionContext.cs.
What you need is to hook up an IEnlistmentNotification which is described here: https://learn.microsoft.com/en-us/dotnet/api/system.transactions.transaction.enlistvolatile
I need to put my code inside the Prepare method and if it fails call ForceRollback.

SQL Server connection timeouts and using statements with command and connection objects

Please read entire question before responding. And I apologize, I never seem to write short questions...
I am supporting a C# internal web app that hits SQL Server 2008 R2 running on a Windows Small Business Server 2011 SP1 box.
We have been getting a lot of SQL timeouts lately, here is an example exception:
System.Web.HttpUnhandledException: Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
I have checked a few things, one of them being how the code handles connections and closing of connections. I have read in other threads that using a Using statement with your connection is adequate as it "...wraps the connection create in a try .. finally and places the connection disposal call inside the finally". The connection is closed even in the event of an exception.
So, I agree with and have used that method for years. Others have recommended explicitly closing connections even when using a Using statement with your connection. I think that would be redundant...
My question, however, is regarding the command object. Someone else wrote a large library of db methods for this app and they have (in all of the db methods) declared the SqlCommand object BEFORE the SqlConnection object using statement. They have also assigned the connection object to the command object before the connection using statement.
Is it better practice to declare and use the command object inside the connection using statement, and could doing it the other way cause sql connection timeouts (barring other causes of sql connection timeouts)? Take this code for example:
public Musician GetMusician(int recordId)
{
Musician objMusician = null;
SqlConnection con = new SqlConnection(_connectionString);
SqlCommand cmd = new SqlCommand();
cmd.Connection = con;
cmd.CommandText = "selectMusician";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#id", recordId);
using (con)
{
con.Open();
SqlDataReader reader = cmd.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
objMusician = new Musician((int)reader["id"]);
objMusician.Name = (string)reader["name"];
}
}
if objMusician != null)
{
objMusician.Albums = Albums.GetAlbums((int)objMusician.ID);
objMusician.Tours = Tours.GetTours((int)objMusician.ID);
objMusician.Interviews = Interviews.GetInterviews((int)objMusician.ID);
}
return objMusician;
}
Also know that the calling pages have try catches in them and it is the page that logs the error to our logging db. We let the exception bubble up to the calling method on the page, and it gets handled there.

You should explicitly close the connection when you're finished with it. You're never closing any connections so after you hit the connection pool limit you're going to get errors until you manually recycle the pool or it cycles on its own. Move the property assignment block inside the using block and do a con.Close(); cmd.Dispose(); before returning your objMusician:
using (con)
{
con.Open();
SqlDataReader reader = cmd.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
objMusician = new Musician((int)reader["id"]);
objMusician.Name = (string)reader["name"];
}
if objMusician != null)
{
objMusician.Albums = Albums.GetAlbums((int)objMusician.ID);
objMusician.Tours = Tours.GetTours((int)objMusician.ID);
objMusician.Interviews = Interviews.GetInterviews((int)objMusician.ID);
}
con.Close();
cmd.Dispose();
return objMusician;
}

Don't know if it will help your timeout problem, but I've always structured my code like the following and not had that problem:
using(var cmd = new SqlCommand())
{
using(var con = new SqlConnection(ConnectionString))
{
con.Open();
cmd.Connection = con;
cmd.CommandText = "selectMusician";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#id", recordId);
...
}
}
Was just reading on MSDN, it said "Call Dispose when you are finished using the Component. The Dispose method leaves the Component in an unusable state. After calling Dispose, you must release all references to the Component so the garbage collector can reclaim the memory that the Component was occupying." This means in order for the GC to immediately collect the connection, you must dispose the connection before disposing the command, otherwise the connection hangs around until the GC gets around to calling the Finalize on it.

Refactor your method as follows. You are likely running into a situation where a data reader has a reference to a connection, and it has not yet been disposed of.
public Musician GetMusician(int recordId)
{
Musician objMusician = null;
using(SqlConnection con = new SqlConnection(_connectionString))
{
con.Open();
using (SqlCommand cmd = new SqlCommand())
{
cmd.Connection = con;
cmd.CommandText = "selectMusician";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#id", recordId);
using (SqlDataReader reader = cmd.ExecuteReader(CommandBehavior.CloseConnection))
{
if (reader.HasRows)
{
reader.Read();
objMusician = new Musician((int) reader["id"]);
objMusician.Name = (string) reader["name"];
}
if objMusician != null)
{
objMusician.Albums = Albums.GetAlbums((int)objMusician.ID);
objMusician.Tours = Tours.GetTours((int)objMusician.ID);
objMusician.Interviews = Interviews.GetInterviews((int)objMusician.ID);
}
}
}
return objMusician;
}
}

SqlConnection with a using clause - calling close on the connection

When I have a SQLConnection within a using clause as illustrated below do I need to explicitly close the connection?
protected SqlConnection Connection
{
get
{
if (this.connection == null)
{
this.connection = new SqlConnection(this.ConnectionString);
}
if (this.connection.State != ConnectionState.Open)
{
this.connection.Open();
}
return this.connection;
}
}
using (SqlConnection connection = this.Connection)
{
using (SqlCommand cmd = connection.CreateCommand())
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "....";
using (SqlDataReader reader = cmd.ExecuteReader())
{
while (reader.Read())
{
etc
}
}
}
}

No you don't. The Dispose() method of a connection will call Close if the connection is already open.
You should also change your code as John Gathogo suggests to create a new connection object each time it needs one. Your code will fail as it is because the second time you try to use the connection it will already be disposed.
ADO.NET uses connection pooling to keep a pool of open connections which it provides to whoever calls Open. This means that creating and opening new connection doesn't cost anything as long as there are available connections in the pool. Keeping a connection open for longer than necessary will degrade performance.

The using block will always call Dispose, which will close the connection.
But, you are keeping the connection object and intend to reuse it, which is not possible once it has been disposed. You shouldn't keep the connection object, you should just throw it away and create a new one when needed. The actual connections to the database are pooled, so when you create a new connection object it will reuse one of the connections from the pool. When you dispose the connection object, the actual connection is returned to the pool.

You could easily change you code to below and achieve what you want:
using (SqlConnection connection = new SqlConnection(this.ConnectionString))
{
connection.Open();
using (SqlCommand cmd = connection.CreateCommand())
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "....";
using (SqlDataReader reader = cmd.ExecuteReader())
{
while (reader.Read())
{
//etc
}
}
}
}
The runtime will take care of closing the connection and disposing the resources for you

TransactionScope helper that exhausts connection pool without fail - help?

A while back I asked a question about TransactionScope escalating to MSDTC when I wasn't expecting it to. (Previous question)
What it boiled down to was, in SQL2005, in order to use a TransactionScope, you can only instance and open a single SqlConnection within the life of the TransactionScope. With SQL2008, you can instance multiple SqlConnections, but only a single one can be open at any given time. SQL2000 will always escalate to DTC...we don't support SQL2000 in our application, a WinForms app, BTW.
Our solution to single-connection-only problem was to create a TransactionScope helper class, called LocalTransactionScope (aka 'LTS'). It wraps a TransactionScope and, most importantly, creates and maintains a single SqlConnection instance for our application. The good news is, it works - we can use LTS across disparate pieces of code and they all join the ambient transaction. Very nice. The trouble is, every root LTS instance created will create and effectively kill a connection from the connection pool. By 'Effectively Kill' I mean it will instance a SqlConnetion, which will open a new connection (for whatever reason, it never reuses a connection from the pool,) and when that root LTS is disposed, it closes and disposes the SqlConnection which is supposed to release the connection back to the pool so that it can be reused, however, it clearly never is reused. The pool bloats until it's maxed out, and then the application fails when a max-pool-size+1 connection is established.
Below I've attached a stripped down version of the LTS code and a sample console application class that will demonstrate the connection pool exhaustion. In order to watch your connection pool bloat, use SQL Server Managment Studio's 'Activity Monitor' or this query:
SELECT DB_NAME(dbid) as 'DB Name',
COUNT(dbid) as 'Connections'
FROM sys.sysprocesses WITH (nolock)
WHERE dbid > 0
GROUP BY dbid
I'm attaching LTS here, and a sample console application that you can use to demonstrate for yourself that it will consume connections from the pool and never re-use nor release them. You will need to add a reference to System.Transactions.dll for LTS to compile.
Things to note: It's the root-level LTS that opens and closes the SqlConnection, which always opens a new connection in the pool. Having nested LTS instances makes no difference because only the root LTS instance establishes a SqlConnection. As you can see, the connection string is always the same, so it should be reusing the connections.
Is there some arcane condition we're not meeting that causes the connections not to be re-used? Is there any solution to this other than turning pooling off entirely?
public sealed class LocalTransactionScope : IDisposable
{
private static SqlConnection _Connection;
private TransactionScope _TransactionScope;
private bool _IsNested;
public LocalTransactionScope(string connectionString)
{
// stripped out a few cases that need to throw an exception
_TransactionScope = new TransactionScope();
// we'll use this later in Dispose(...) to determine whether this LTS instance should close the connection.
_IsNested = (_Connection != null);
if (_Connection == null)
{
_Connection = new SqlConnection(connectionString);
// This Has Code-Stink. You want to open your connections as late as possible and hold them open for as little
// time as possible. However, in order to use TransactionScope with SQL2005 you can only have a single
// connection, and it can only be opened once within the scope of the entire TransactionScope. If you have
// more than one SqlConnection, or you open a SqlConnection, close it, and re-open it, it more than once,
// the TransactionScope will escalate to the MSDTC. SQL2008 allows you to have multiple connections within a
// single TransactionScope, however you can only have a single one open at any given time.
// Lastly, let's not forget about SQL2000. Using TransactionScope with SQL2000 will immediately and always escalate to DTC.
// We've dropped support of SQL2000, so that's not a concern we have.
_Connection.Open();
}
}
/// <summary>'Completes' the <see cref="TransactionScope"/> this <see cref="LocalTransactionScope"/> encapsulates.</summary>
public void Complete() { _TransactionScope.Complete(); }
/// <summary>Creates a new <see cref="SqlCommand"/> from the current <see cref="SqlConnection"/> this <see cref="LocalTransactionScope"/> is managing.</summary>
public SqlCommand CreateCommand() { return _Connection.CreateCommand(); }
void IDisposable.Dispose() { this.Dispose(); }
public void Dispose()
{
Dispose(true); GC.SuppressFinalize(this);
}
private void Dispose(bool disposing)
{
if (disposing)
{
_TransactionScope.Dispose();
_TransactionScope = null;
if (!_IsNested)
{
// last one out closes the door, this would be the root LTS, the first one to be instanced.
LocalTransactionScope._Connection.Close();
LocalTransactionScope._Connection.Dispose();
LocalTransactionScope._Connection = null;
}
}
}
}
This is a Program.cs that will exhibit the connection pool exhaustion:
class Program
{
static void Main(string[] args)
{
// fill in your connection string, but don't monkey with any pooling settings, like
// "Pooling=false;" or the "Max Pool Size" stuff. Doesn't matter if you use
// Doesn't matter if you use Windows or SQL auth, just make sure you set a Data Soure and an Initial Catalog
string connectionString = "your connection string here";
List<string> randomTables = new List<string>();
using (var nonLTSConnection = new SqlConnection(connectionString))
using (var command = nonLTSConnection.CreateCommand())
{
command.CommandType = CommandType.Text;
command.CommandText = #"SELECT [TABLE_NAME], NEWID() AS [ID]
FROM [INFORMATION_SCHEMA].TABLES]
WHERE [TABLE_SCHEMA] = 'dbo' and [TABLE_TYPE] = 'BASE TABLE'
ORDER BY [ID]";
nonLTSConnection.Open();
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
string table = (string)reader["TABLE_NAME"];
randomTables.Add(table);
if (randomTables.Count > 200) { break; } // got more than enough to test.
}
}
nonLTSConnection.Close();
}
// we're going to assume your database had some tables.
for (int j = 0; j < 200; j++)
{
// At j = 100 you'll see it pause, and you'll shortly get an InvalidOperationException with the text of:
// "Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool.
// This may have occurred because all pooled connections were in use and max pool size was reached."
string tableName = randomTables[j % randomTables.Count];
Console.Write("Creating root-level LTS " + j.ToString() + " selecting from " + tableName);
using (var scope = new LocalTransactionScope(connectionString))
using (var command = scope.CreateCommand())
{
command.CommandType = CommandType.Text;
command.CommandText = "SELECT TOP 20 * FROM [" + tableName + "]";
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
Console.Write(".");
}
Console.Write(Environment.NewLine);
}
}
Thread.Sleep(50);
scope.Complete();
}
Console.ReadKey();
}
}

The expected TransactionScope/SqlConnection pattern is, according to MSDN:
using(TransactionScope scope = ...) {
using (SqlConnection conn = ...) {
conn.Open();
SqlCommand.Execute(...);
SqlCommand.Execute(...);
}
scope.Complete();
}
So in the MSDN example the conenction is disposed inside the scope, before the scope is complete. Your code though is different, it disposes the connection after the scope is complete. I'm not an expert in matters of TransactionScope and its interaction with the SqlConnection (I know some things, but your question goes pretty deep) and I can't find any specifications what is the correct pattern. But I'd suggest you revisit your code and dispose the singleton connection before the outermost scope is complete, similarly to the MSDN sample.
Also, I hope you do realize your code will fall apart the moment a second thread comes to play into your application.

Is this code legal?
using(TransactionScope scope = ..)
{
using (SqlConnection conn = ..)
using (SqlCommand command = ..)
{
conn.Open();
SqlCommand.Execute(..);
}
using (SqlConnection conn = ..) // the same connection string
using (SqlCommand command = ..)
{
conn.Open();
SqlCommand.Execute(..);
}
scope.Complete();
}