We Keep Coding

SqlConnection error -- Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool [duplicate]

I'm doing some experimentation with ADO.Net and EF in order to better understand how it handles the SQL Server connections.
I found something very interesting with ADO.Net. I am creating multiple tasks that calls a simple insert SQL script with the proper wait to dispose SqlConnection and SqlCommand. Nothing special here, but when the 10k tasks finishes processing, all the SQL connections are still hanging (I confirmed by running sp_who). The only way those connections get cleared up is when the application instance is closed.
How is that even possible? I tried many things to force it to close, = null the the data access instance + forced GC but nothing...
I'm trying to make sense from this behavior but I am failing. Any clues?
static void Main(string[] args)
{
Console.WriteLine(DateTime.Now.ToString("HH:mm:ss"));
for (int i = 0; i < 10000; i++)
{
Task.Run(() =>
{
var dbLegacy = new DataAccessLegacy();
dbLegacy.TableBInsert();
dbLegacy = null;
});
}
Console.ReadKey();
}
public void TableBInsert()
{
using (SqlConnection connection = new SqlConnection(#"Password=qpqp;Persist Security Info=True;User ID=sqlUser2;Initial Catalog=DatabaseA;Data Source=VM2HOSTNAME\VM2INSTANCEA"))
{
using (SqlCommand command = new SqlCommand("DatabaseBInsert", connection))
{
command.CommandType = CommandType.StoredProcedure;
command.Parameters.Add("ColAInt", SqlDbType.Int);
command.Parameters[0].Value = (new Random()).Next(0, 5000);
command.Parameters.Add("ColBTinyInt", SqlDbType.TinyInt);
command.Parameters[1].Value = (new Random()).Next(1, 255);
command.Parameters.Add("ColCVarchar", SqlDbType.VarChar);
command.Parameters[2].Value = Convert.ToChar((new Random()).Next(1, 255)).ToString();
command.Parameters.Add("ColDVarcharMax", SqlDbType.VarChar);
command.Parameters[3].Value = Convert.ToChar((new Random()).Next(1, 255)).ToString();
command.Parameters.Add("ColEDecimal", SqlDbType.Decimal);
command.Parameters[4].Value = (new Random()).Next(0, 5000) + 0.5;
command.Parameters.Add("ColFSmallInt", SqlDbType.SmallInt);
command.Parameters[5].Value = (new Random()).Next(0, 5000);
command.Parameters.Add("ColGDateTime", SqlDbType.DateTime);
command.Parameters[6].Value = DateTime.Now;
command.Parameters.Add("ColHChar", SqlDbType.Char);
command.Parameters[7].Value = Convert.ToChar((new Random()).Next(1, 255)).ToString();
command.Parameters.Add("ColINVarchar", SqlDbType.NVarChar);
command.Parameters[8].Value = Convert.ToChar((new Random()).Next(1, 255)).ToString();
command.Parameters.Add("ColJNChar", SqlDbType.NChar);
command.Parameters[9].Value = Convert.ToChar((new Random()).Next(1, 255)).ToString();
connection.Open();
command.ExecuteScalar();
connection.Close();
command.Dispose();
}
connection.Dispose();
}
}

By default, ADO.Net uses connection pooling. From the docs (emphasis mine):
Connection pooling reduces the number of times that new connections must be opened. The pooler maintains ownership of the physical connection. It manages connections by keeping alive a set of active connections for each given connection configuration. Whenever a user calls Open on a connection, the pooler looks for an available connection in the pool. If a pooled connection is available, it returns it to the caller instead of opening a new connection. When the application calls Close on the connection, the pooler returns it to the pooled set of active connections instead of closing it. Once the connection is returned to the pool, it is ready to be reused on the next Open call.

Postgres connection state stays active when an exception occurs

The state of the connection stays active in pool, when there is an exception while executing a query or stored procedure through c#.
The Npgsql version that I am using is 4.1.7.
Here is the code that I am trying to execute.
NpgsqlCommand cmd = null;
NpgsqlDataAdapter sda = null;
NpgsqlConnection conn = null;
try {
string sql = "a_test";
conn = new NpgsqlConnection("Server=localhost;Port=5432;Username=admin;Password=test;Database=majordb;SearchPath=dbs;CommandTimeout=300;MaxPoolSize=500;Connection Idle Lifetime=180;Connection Pruning Interval=5;");
cmd = new NpgsqlCommand(sql);
cmd.CommandType = CommandType.StoredProcedure;
sda = new NpgsqlDataAdapter(cmd);
cmd.Connection = conn;
sda.Fill(dataTable);
}
catch (Exception e) {
//log
}
finally {
if(null != sda)
{
try
{
sda.Dispose();
}
catch (Exception)
{
}
}
try
{
cmd.Connection.Close();
cmd.Connection.Dispose();
}
catch (Exception)
{
}
try
{
cmd.Dispose();
}
catch (Exception)
{
}
}
If the above code executes properly without any exception, the connection state in pool goes to idle, which is correct. But if an exception occurs while executing, like below:
"Npgsql.NpgsqlException (0x80004005): Exception while reading from stream --->
System.IO.IOException: Unable to read data from the transport connection: A connection attempt
failed because the connected party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond."
The connection state in pool shows as active for about 5 mins or so, even though the close/dispose methods are called in finally block. This means the close/dispose did not properly executed by Npgsql. If the program keeps the connection state in pool active for every connection ran within 5 mins, then there can be an issue with MaxPoolSize error.
I wanted to see the connection state to idle, even when there is an exception. How do I do this.
Please note: I am not looking for a solution to the exception that I listed above. I am looking for a solution where the connection state is changed to idle from active when there is an exception while executing the above code.
To know if the connection state is active or not I used the following query:
SELECT
pid,
usename,
application_name,
datname,
client_addr,
rank() over (partition by client_addr order by backend_start ASC) as rank,
state,
state_change,
current_timestamp,
query,
query_start,
backend_start,
FROM
pg_stat_activity
WHERE
pid <> pg_backend_pid( )
AND
application_name !~ '(?:psql)|(?:pgAdmin.+)'
AND
datname = current_database()
AND
usename = current_user
Any help is really appreciated.

how to handle ASP.NET dbtransaction timeout

I have been experiencing db transaction timemouts in an ASP.NET 2.0 & SQL Server 2008 application for several days now.
The issue is something using SQL Server profiler tracing to "a function calling web service".
We have transaction include several stored procedure (sp). But stored procedure #3 has some bytes to save in DB, it consumes time. When the time goes over 25 seconds, it throws a timeout exception message:
Message : System.Web.Services.Protocols.SoapException: Server was unable to process request. ---> System.Data.SqlClient.SqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
My code:
using (System.Data.Common.DbConnection connection = o_DB.CreateConnection())
{
connection.Open();
System.Data.Common.DbTransaction o_Transaction = connection.BeginTransaction();
try
{
- exec sp1
- exec sp2
foreach{
- exec sp3
}
}
We tried some solutions in the web but it did not work. I hope someone can give a hand to me. Many Thanks.
We do not have timeout in the connection string in web.config, it does not suffer on exception after 15s (default)
We set the transaction timeout in web.config:
<system.transactions>
<machineSettings maxTimeout="00:00:30" />
</system.transactions>

Something like that
using (var ts = CreateTransactionScope(TimeSpan.FromSeconds(mySecondsVar)))
{
using (System.Data.Common.DbConnection connection = o_DB.CreateConnection())
{
using (IDbTransaction tran = connection.BeginTransaction()) {
try
{
// your code
}
catch {
tran.Rollback();
}
}
}
ts.Complete();
}
If is "ok", the dispose do the commit automatically.

This you can directly handle by code also.
Please see the below code
public DataSet getData(string command)
{
DataSet ds = new DataSet();
string connectionString = ConfigurationManager.ConnectionStrings["TESTDB"].ConnectionString;
using (var conn = new SqlConnection(connectionString))
{
using (var cmd = new SqlCommand(command, conn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandTimeout = 0;
SqlDataAdapter adapt = new SqlDataAdapter(cmd);
conn.Open();
adapt.Fill(ds);
conn.Close();
}
}
return ds;
}
The blow line will change the execution time to infinity or until query execution complete.
cmd.CommandTimeout = 0;

SQL Server connection timeouts and using statements with command and connection objects

Please read entire question before responding. And I apologize, I never seem to write short questions...
I am supporting a C# internal web app that hits SQL Server 2008 R2 running on a Windows Small Business Server 2011 SP1 box.
We have been getting a lot of SQL timeouts lately, here is an example exception:
System.Web.HttpUnhandledException: Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
I have checked a few things, one of them being how the code handles connections and closing of connections. I have read in other threads that using a Using statement with your connection is adequate as it "...wraps the connection create in a try .. finally and places the connection disposal call inside the finally". The connection is closed even in the event of an exception.
So, I agree with and have used that method for years. Others have recommended explicitly closing connections even when using a Using statement with your connection. I think that would be redundant...
My question, however, is regarding the command object. Someone else wrote a large library of db methods for this app and they have (in all of the db methods) declared the SqlCommand object BEFORE the SqlConnection object using statement. They have also assigned the connection object to the command object before the connection using statement.
Is it better practice to declare and use the command object inside the connection using statement, and could doing it the other way cause sql connection timeouts (barring other causes of sql connection timeouts)? Take this code for example:
public Musician GetMusician(int recordId)
{
Musician objMusician = null;
SqlConnection con = new SqlConnection(_connectionString);
SqlCommand cmd = new SqlCommand();
cmd.Connection = con;
cmd.CommandText = "selectMusician";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#id", recordId);
using (con)
{
con.Open();
SqlDataReader reader = cmd.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
objMusician = new Musician((int)reader["id"]);
objMusician.Name = (string)reader["name"];
}
}
if objMusician != null)
{
objMusician.Albums = Albums.GetAlbums((int)objMusician.ID);
objMusician.Tours = Tours.GetTours((int)objMusician.ID);
objMusician.Interviews = Interviews.GetInterviews((int)objMusician.ID);
}
return objMusician;
}
Also know that the calling pages have try catches in them and it is the page that logs the error to our logging db. We let the exception bubble up to the calling method on the page, and it gets handled there.

You should explicitly close the connection when you're finished with it. You're never closing any connections so after you hit the connection pool limit you're going to get errors until you manually recycle the pool or it cycles on its own. Move the property assignment block inside the using block and do a con.Close(); cmd.Dispose(); before returning your objMusician:
using (con)
{
con.Open();
SqlDataReader reader = cmd.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
objMusician = new Musician((int)reader["id"]);
objMusician.Name = (string)reader["name"];
}
if objMusician != null)
{
objMusician.Albums = Albums.GetAlbums((int)objMusician.ID);
objMusician.Tours = Tours.GetTours((int)objMusician.ID);
objMusician.Interviews = Interviews.GetInterviews((int)objMusician.ID);
}
con.Close();
cmd.Dispose();
return objMusician;
}

Don't know if it will help your timeout problem, but I've always structured my code like the following and not had that problem:
using(var cmd = new SqlCommand())
{
using(var con = new SqlConnection(ConnectionString))
{
con.Open();
cmd.Connection = con;
cmd.CommandText = "selectMusician";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#id", recordId);
...
}
}
Was just reading on MSDN, it said "Call Dispose when you are finished using the Component. The Dispose method leaves the Component in an unusable state. After calling Dispose, you must release all references to the Component so the garbage collector can reclaim the memory that the Component was occupying." This means in order for the GC to immediately collect the connection, you must dispose the connection before disposing the command, otherwise the connection hangs around until the GC gets around to calling the Finalize on it.

Refactor your method as follows. You are likely running into a situation where a data reader has a reference to a connection, and it has not yet been disposed of.
public Musician GetMusician(int recordId)
{
Musician objMusician = null;
using(SqlConnection con = new SqlConnection(_connectionString))
{
con.Open();
using (SqlCommand cmd = new SqlCommand())
{
cmd.Connection = con;
cmd.CommandText = "selectMusician";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#id", recordId);
using (SqlDataReader reader = cmd.ExecuteReader(CommandBehavior.CloseConnection))
{
if (reader.HasRows)
{
reader.Read();
objMusician = new Musician((int) reader["id"]);
objMusician.Name = (string) reader["name"];
}
if objMusician != null)
{
objMusician.Albums = Albums.GetAlbums((int)objMusician.ID);
objMusician.Tours = Tours.GetTours((int)objMusician.ID);
objMusician.Interviews = Interviews.GetInterviews((int)objMusician.ID);
}
}
}
return objMusician;
}
}

Cannot configure timeout from connection string

I have created this stored procedure in SQL Server 2008 R2:
CREATE PROCEDURE dbo.test AS
BEGIN
WAITFOR DEALY '00:00:40'
END
I have also written a C# console app to test the connect timeout property.
static void Main(string[] args)
{
OpenSqlConnection();
Console.ReadLine();
}
private static void OpenSqlConnection()
{
string connectionString = GetConnectionString();
using (SqlConnection connection = new SqlConnection(connectionString))
{
using (var command = new SqlCommand("dbo.test", connection) { CommandType = CommandType.StoredProcedure })
{
connection.Open();
Console.WriteLine("ConnectionTimeout: {0}", connection.ConnectionTimeout);
command.ExecuteNonQuery();
Console.WriteLine("Finished");
connection.Close();
}
}
}
static private string GetConnectionString()
{
return
"Data Source=test.com;Initial Catalog=test_db;Integrated Security=True;MultipleActiveResultSets=True;Connect Timeout=10; Application Name=Test";
}
There was a connection timeout exception returned as excepted (timeout in connection string < 40s). However, when I changed the Connect Timeout property to 120, I still got the same timeout exception. After some trial and error, it seems that the timeout value is always 30s regardless of my connection string value.
Is the connection timeout controlled by somewhere else? (if yes, where?)
Why the connect timeout value in connection string has no effect on my console app?
Update
Thanks all. I have just noticed that the command timeout is different from connection timeout(timeout in connection string).
Instead of changing my app.config, I would have to specify the CommandTimeout property as Sudipta Maiti suggested and recompile my code.
There is a similar issue in:
http://forums.asp.net/t/1197160.aspx?Can+you+change+command+timeout+via+the+connection+string+

Use CommandTimeout:
using (var command = new SqlCommand("dbo.test", connection) {
CommandType = CommandType.StoredProcedure })
{
connection.Open();
Console.WriteLine("ConnectionTimeout: {0}",
connection.ConnectionTimeout);
command.CommandTimeout = 120;
command.ExecuteNonQuery();
Console.WriteLine("Finished");
connection.Close();
}

I strongly suspect the given Timeout, it may not be enough to complete your operations. So make it Connection Timeout=30;
Connect Timeout -or- Connection Timeout by default the value will be 15 seconds. Based on your operation performance you might need to extend the seconds.
When a connection is returned to the pool, its creation time is compared with the current time, and the connection is destroyed if that time span (in seconds) exceeds the value specified by Connection Lifetime. This is useful in clustered configurations to force load balancing between a running server and a server just brought online.
A value of zero (0) causes pooled connections to have the maximum connection timeout.
There is a Sample Source almost same as yours.