We Keep Coding

Get a rid of SqlConnection, which remains in Activity Monitor after being closed/disposed

I'm creating database, using ADO.NET. Basically, I'm executing SQL commands in next way:
private bool ExecuteSqlCommand(string command)
{
var success = true;
using (var connection = GetSqlConnection())
{
if (connection == null)
return false;
using (var myCommand = new SqlCommand("query", connection))
{
try
{
connection.Open();
myCommand.CommandText = command;
myCommand.ExecuteNonQuery();
}
catch (Exception ex)
{
success = false;
Log.LogMessage(string.Format("Unable to execute SQL command: {0}", ex.Message));
}
}
}
return success;
}
GetSqlConnection just gets some proper SqlConnection with connection string like
"Server={0}\\{1};User Id={2};Password={3};Application Name={4};"
It works well, and executes command properly, with one exception - according to Sql Manager Studio activity monitor, it remains as active connection even after method was executed.
According to question Why does my SqlConnection remain in the SSMS Processes list after closing it?, this is correct behavior, since connection might be reused. But the serius issue is, that later, READ_COMMITTED_SNAPSHOT command will be called for this database, while using different SqlConnection. Which leads to exception, since READ_COMMITTED_SNAPSHOT requires, that connection, which is used to call this command, should be the only connection to database.
I can't reuse this connection for further operations with database, since I use different connection string for them, with database specified as InitialCatalog (obviously, I can't use it, while database doesn't exist).
So, can I somehow remove this initial connection?

Execute SqlConnection.ClearPool This will mark all connections to be discarded instead of recycled.

Sql Connection Pool timeout

[Disclaimer] : I think I have read every stackoverflow post about this already
I have been breaking my head over this for quite some time now. I am getting the following exception in my asp.net web.api.
Exception thrown: 'System.InvalidOperationException' in mscorlib.dll
Additional information: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
Most people suggested that I should look for leaked connections in my application. Here is my code. Now I am sure that I am not leaking any connections
public async Task<IEnumerable<string>> Get()
{
var ds = new DataSet();
var constring = "Data Source=xxx;Initial Catalog=xxx;User Id=xxx;Password=xxx;Max Pool Size=100";
var asyncConnectionString = new SqlConnectionStringBuilder(constring)
{
AsynchronousProcessing = true
}.ToString();
using (var con = new SqlConnection(asyncConnectionString))
using (var cmd = new SqlCommand("[dbo].[xxx]", con))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#x1", 1);
cmd.Parameters.AddWithValue("#x2", "something");
await con.OpenAsync();
using (var rdr =await cmd.ExecuteReaderAsync())
{
if (rdr.HasRows)
{
ds.Load(rdr, LoadOption.OverwriteChanges, "MyTable");
}
rdr.Close();
con.Close();
ds.Dispose();
}
}
//I know this looks wrong, just an empty api method to show the code
return new string[] { "value1", "value2" };
}
The exception does not occur when I am using my local Sql Server. Only happens when I connect to our 'test server'. Are there anything else I can look at when trying resolve this issue. Like Sql server settings / network settings etc.
The stored procedure I call does not lock up the db I have checked for that as well. If that was the case it should have failed on my local Sql instance as well.
I am using jmeter to generate load, 1500 - threads(users). Surely I should be able to handle way more than that.
Thanks in advance

You have not specified any Connection Time out property, so it's 15 seconds default. using Max Pool Size=100 is not a good idea until you don't have proper hardware resources.
You started 1500 threads, so it seems that the some the threads keep waiting for 15 seconds to get their chance for connection opening. And as time goes out, you get the connection time out error.
So I think increasing the 'Connection Timeout' property in connection string may resolve your issue.

Cannot configure timeout from connection string

I have created this stored procedure in SQL Server 2008 R2:
CREATE PROCEDURE dbo.test AS
BEGIN
WAITFOR DEALY '00:00:40'
END
I have also written a C# console app to test the connect timeout property.
static void Main(string[] args)
{
OpenSqlConnection();
Console.ReadLine();
}
private static void OpenSqlConnection()
{
string connectionString = GetConnectionString();
using (SqlConnection connection = new SqlConnection(connectionString))
{
using (var command = new SqlCommand("dbo.test", connection) { CommandType = CommandType.StoredProcedure })
{
connection.Open();
Console.WriteLine("ConnectionTimeout: {0}", connection.ConnectionTimeout);
command.ExecuteNonQuery();
Console.WriteLine("Finished");
connection.Close();
}
}
}
static private string GetConnectionString()
{
return
"Data Source=test.com;Initial Catalog=test_db;Integrated Security=True;MultipleActiveResultSets=True;Connect Timeout=10; Application Name=Test";
}
There was a connection timeout exception returned as excepted (timeout in connection string < 40s). However, when I changed the Connect Timeout property to 120, I still got the same timeout exception. After some trial and error, it seems that the timeout value is always 30s regardless of my connection string value.
Is the connection timeout controlled by somewhere else? (if yes, where?)
Why the connect timeout value in connection string has no effect on my console app?
Update
Thanks all. I have just noticed that the command timeout is different from connection timeout(timeout in connection string).
Instead of changing my app.config, I would have to specify the CommandTimeout property as Sudipta Maiti suggested and recompile my code.
There is a similar issue in:
http://forums.asp.net/t/1197160.aspx?Can+you+change+command+timeout+via+the+connection+string+

Use CommandTimeout:
using (var command = new SqlCommand("dbo.test", connection) {
CommandType = CommandType.StoredProcedure })
{
connection.Open();
Console.WriteLine("ConnectionTimeout: {0}",
connection.ConnectionTimeout);
command.CommandTimeout = 120;
command.ExecuteNonQuery();
Console.WriteLine("Finished");
connection.Close();
}

I strongly suspect the given Timeout, it may not be enough to complete your operations. So make it Connection Timeout=30;
Connect Timeout -or- Connection Timeout by default the value will be 15 seconds. Based on your operation performance you might need to extend the seconds.
When a connection is returned to the pool, its creation time is compared with the current time, and the connection is destroyed if that time span (in seconds) exceeds the value specified by Connection Lifetime. This is useful in clustered configurations to force load balancing between a running server and a server just brought online.
A value of zero (0) causes pooled connections to have the maximum connection timeout.
There is a Sample Source almost same as yours.

SQL Server Error while pulling hundreds of records

I am getting this error when I try to get all the username password from production copied local database, I guess it is because of not closing the connection properly, but I am not sure how . I am using the Microsoft Enterprise Library, ant thought or comment about it?
Timeout expired. The timeout period elapsed prior to
obtaining a connection from the pool. This may have occurred
because all pooled connections were in use and max pool size was reached.
this is the mothod that is getting the username and password and producing the error.
private Model.UsernameandPass GetUsernamePass(string AccountNumber)
{
Model.UsernameandPass model = null;
string myConnection = System.Configuration.ConfigurationManager.ConnectionStrings[connectionName].ToString();
SqlDatabase db = new SqlDatabase(myConnection);
using (DbCommand command = db.GetStoredProcCommand("Get_TheUsernamePassWordFromProduction"))
{
db.AddInParameter(command, "AccountNumber", DbType.String, AccountNumber);
var result = db.ExecuteReader(command);
try
{
while (result.Read())
{
model = new Model.UsernameandPass();
model.Username = result.GetString(1);
model.Password = result.GetString(2);
}
}
catch (Exception ex)
{
}
}
db = null;
return model;
}
I am getting the error in this line after program runs for a while.
var result = db.ExecuteReader(command);

You're getting that error because a connection cannot be established, not only because they aren't being closed properly. Check permissions for the user you're trying to authenticate against the database with. Also be sure to call .open()/.close() when/if you are programatically opening/closing connections.
Check this link. You may want to increase your pool size, or check for long-running queries.

TransactionScope helper that exhausts connection pool without fail - help?

A while back I asked a question about TransactionScope escalating to MSDTC when I wasn't expecting it to. (Previous question)
What it boiled down to was, in SQL2005, in order to use a TransactionScope, you can only instance and open a single SqlConnection within the life of the TransactionScope. With SQL2008, you can instance multiple SqlConnections, but only a single one can be open at any given time. SQL2000 will always escalate to DTC...we don't support SQL2000 in our application, a WinForms app, BTW.
Our solution to single-connection-only problem was to create a TransactionScope helper class, called LocalTransactionScope (aka 'LTS'). It wraps a TransactionScope and, most importantly, creates and maintains a single SqlConnection instance for our application. The good news is, it works - we can use LTS across disparate pieces of code and they all join the ambient transaction. Very nice. The trouble is, every root LTS instance created will create and effectively kill a connection from the connection pool. By 'Effectively Kill' I mean it will instance a SqlConnetion, which will open a new connection (for whatever reason, it never reuses a connection from the pool,) and when that root LTS is disposed, it closes and disposes the SqlConnection which is supposed to release the connection back to the pool so that it can be reused, however, it clearly never is reused. The pool bloats until it's maxed out, and then the application fails when a max-pool-size+1 connection is established.
Below I've attached a stripped down version of the LTS code and a sample console application class that will demonstrate the connection pool exhaustion. In order to watch your connection pool bloat, use SQL Server Managment Studio's 'Activity Monitor' or this query:
SELECT DB_NAME(dbid) as 'DB Name',
COUNT(dbid) as 'Connections'
FROM sys.sysprocesses WITH (nolock)
WHERE dbid > 0
GROUP BY dbid
I'm attaching LTS here, and a sample console application that you can use to demonstrate for yourself that it will consume connections from the pool and never re-use nor release them. You will need to add a reference to System.Transactions.dll for LTS to compile.
Things to note: It's the root-level LTS that opens and closes the SqlConnection, which always opens a new connection in the pool. Having nested LTS instances makes no difference because only the root LTS instance establishes a SqlConnection. As you can see, the connection string is always the same, so it should be reusing the connections.
Is there some arcane condition we're not meeting that causes the connections not to be re-used? Is there any solution to this other than turning pooling off entirely?
public sealed class LocalTransactionScope : IDisposable
{
private static SqlConnection _Connection;
private TransactionScope _TransactionScope;
private bool _IsNested;
public LocalTransactionScope(string connectionString)
{
// stripped out a few cases that need to throw an exception
_TransactionScope = new TransactionScope();
// we'll use this later in Dispose(...) to determine whether this LTS instance should close the connection.
_IsNested = (_Connection != null);
if (_Connection == null)
{
_Connection = new SqlConnection(connectionString);
// This Has Code-Stink. You want to open your connections as late as possible and hold them open for as little
// time as possible. However, in order to use TransactionScope with SQL2005 you can only have a single
// connection, and it can only be opened once within the scope of the entire TransactionScope. If you have
// more than one SqlConnection, or you open a SqlConnection, close it, and re-open it, it more than once,
// the TransactionScope will escalate to the MSDTC. SQL2008 allows you to have multiple connections within a
// single TransactionScope, however you can only have a single one open at any given time.
// Lastly, let's not forget about SQL2000. Using TransactionScope with SQL2000 will immediately and always escalate to DTC.
// We've dropped support of SQL2000, so that's not a concern we have.
_Connection.Open();
}
}
/// <summary>'Completes' the <see cref="TransactionScope"/> this <see cref="LocalTransactionScope"/> encapsulates.</summary>
public void Complete() { _TransactionScope.Complete(); }
/// <summary>Creates a new <see cref="SqlCommand"/> from the current <see cref="SqlConnection"/> this <see cref="LocalTransactionScope"/> is managing.</summary>
public SqlCommand CreateCommand() { return _Connection.CreateCommand(); }
void IDisposable.Dispose() { this.Dispose(); }
public void Dispose()
{
Dispose(true); GC.SuppressFinalize(this);
}
private void Dispose(bool disposing)
{
if (disposing)
{
_TransactionScope.Dispose();
_TransactionScope = null;
if (!_IsNested)
{
// last one out closes the door, this would be the root LTS, the first one to be instanced.
LocalTransactionScope._Connection.Close();
LocalTransactionScope._Connection.Dispose();
LocalTransactionScope._Connection = null;
}
}
}
}
This is a Program.cs that will exhibit the connection pool exhaustion:
class Program
{
static void Main(string[] args)
{
// fill in your connection string, but don't monkey with any pooling settings, like
// "Pooling=false;" or the "Max Pool Size" stuff. Doesn't matter if you use
// Doesn't matter if you use Windows or SQL auth, just make sure you set a Data Soure and an Initial Catalog
string connectionString = "your connection string here";
List<string> randomTables = new List<string>();
using (var nonLTSConnection = new SqlConnection(connectionString))
using (var command = nonLTSConnection.CreateCommand())
{
command.CommandType = CommandType.Text;
command.CommandText = #"SELECT [TABLE_NAME], NEWID() AS [ID]
FROM [INFORMATION_SCHEMA].TABLES]
WHERE [TABLE_SCHEMA] = 'dbo' and [TABLE_TYPE] = 'BASE TABLE'
ORDER BY [ID]";
nonLTSConnection.Open();
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
string table = (string)reader["TABLE_NAME"];
randomTables.Add(table);
if (randomTables.Count > 200) { break; } // got more than enough to test.
}
}
nonLTSConnection.Close();
}
// we're going to assume your database had some tables.
for (int j = 0; j < 200; j++)
{
// At j = 100 you'll see it pause, and you'll shortly get an InvalidOperationException with the text of:
// "Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool.
// This may have occurred because all pooled connections were in use and max pool size was reached."
string tableName = randomTables[j % randomTables.Count];
Console.Write("Creating root-level LTS " + j.ToString() + " selecting from " + tableName);
using (var scope = new LocalTransactionScope(connectionString))
using (var command = scope.CreateCommand())
{
command.CommandType = CommandType.Text;
command.CommandText = "SELECT TOP 20 * FROM [" + tableName + "]";
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
Console.Write(".");
}
Console.Write(Environment.NewLine);
}
}
Thread.Sleep(50);
scope.Complete();
}
Console.ReadKey();
}
}

The expected TransactionScope/SqlConnection pattern is, according to MSDN:
using(TransactionScope scope = ...) {
using (SqlConnection conn = ...) {
conn.Open();
SqlCommand.Execute(...);
SqlCommand.Execute(...);
}
scope.Complete();
}
So in the MSDN example the conenction is disposed inside the scope, before the scope is complete. Your code though is different, it disposes the connection after the scope is complete. I'm not an expert in matters of TransactionScope and its interaction with the SqlConnection (I know some things, but your question goes pretty deep) and I can't find any specifications what is the correct pattern. But I'd suggest you revisit your code and dispose the singleton connection before the outermost scope is complete, similarly to the MSDN sample.
Also, I hope you do realize your code will fall apart the moment a second thread comes to play into your application.

Is this code legal?
using(TransactionScope scope = ..)
{
using (SqlConnection conn = ..)
using (SqlCommand command = ..)
{
conn.Open();
SqlCommand.Execute(..);
}
using (SqlConnection conn = ..) // the same connection string
using (SqlCommand command = ..)
{
conn.Open();
SqlCommand.Execute(..);
}
scope.Complete();
}