Please read entire question before responding. And I apologize, I never seem to write short questions...
I am supporting a C# internal web app that hits SQL Server 2008 R2 running on a Windows Small Business Server 2011 SP1 box.
We have been getting a lot of SQL timeouts lately, here is an example exception:
System.Web.HttpUnhandledException: Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
at System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection)
at System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.SqlClient.SqlConnection.Open()
I have checked a few things, one of them being how the code handles connections and closing of connections. I have read in other threads that using a Using statement with your connection is adequate as it "...wraps the connection create in a try .. finally and places the connection disposal call inside the finally". The connection is closed even in the event of an exception.
So, I agree with and have used that method for years. Others have recommended explicitly closing connections even when using a Using statement with your connection. I think that would be redundant...
My question, however, is regarding the command object. Someone else wrote a large library of db methods for this app and they have (in all of the db methods) declared the SqlCommand object BEFORE the SqlConnection object using statement. They have also assigned the connection object to the command object before the connection using statement.
Is it better practice to declare and use the command object inside the connection using statement, and could doing it the other way cause sql connection timeouts (barring other causes of sql connection timeouts)? Take this code for example:
public Musician GetMusician(int recordId)
{
Musician objMusician = null;
SqlConnection con = new SqlConnection(_connectionString);
SqlCommand cmd = new SqlCommand();
cmd.Connection = con;
cmd.CommandText = "selectMusician";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#id", recordId);
using (con)
{
con.Open();
SqlDataReader reader = cmd.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
objMusician = new Musician((int)reader["id"]);
objMusician.Name = (string)reader["name"];
}
}
if objMusician != null)
{
objMusician.Albums = Albums.GetAlbums((int)objMusician.ID);
objMusician.Tours = Tours.GetTours((int)objMusician.ID);
objMusician.Interviews = Interviews.GetInterviews((int)objMusician.ID);
}
return objMusician;
}
Also know that the calling pages have try catches in them and it is the page that logs the error to our logging db. We let the exception bubble up to the calling method on the page, and it gets handled there.
You should explicitly close the connection when you're finished with it. You're never closing any connections so after you hit the connection pool limit you're going to get errors until you manually recycle the pool or it cycles on its own. Move the property assignment block inside the using block and do a con.Close(); cmd.Dispose(); before returning your objMusician:
using (con)
{
con.Open();
SqlDataReader reader = cmd.ExecuteReader();
if (reader.HasRows)
{
reader.Read();
objMusician = new Musician((int)reader["id"]);
objMusician.Name = (string)reader["name"];
}
if objMusician != null)
{
objMusician.Albums = Albums.GetAlbums((int)objMusician.ID);
objMusician.Tours = Tours.GetTours((int)objMusician.ID);
objMusician.Interviews = Interviews.GetInterviews((int)objMusician.ID);
}
con.Close();
cmd.Dispose();
return objMusician;
}
Don't know if it will help your timeout problem, but I've always structured my code like the following and not had that problem:
using(var cmd = new SqlCommand())
{
using(var con = new SqlConnection(ConnectionString))
{
con.Open();
cmd.Connection = con;
cmd.CommandText = "selectMusician";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#id", recordId);
...
}
}
Was just reading on MSDN, it said "Call Dispose when you are finished using the Component. The Dispose method leaves the Component in an unusable state. After calling Dispose, you must release all references to the Component so the garbage collector can reclaim the memory that the Component was occupying." This means in order for the GC to immediately collect the connection, you must dispose the connection before disposing the command, otherwise the connection hangs around until the GC gets around to calling the Finalize on it.
Refactor your method as follows. You are likely running into a situation where a data reader has a reference to a connection, and it has not yet been disposed of.
public Musician GetMusician(int recordId)
{
Musician objMusician = null;
using(SqlConnection con = new SqlConnection(_connectionString))
{
con.Open();
using (SqlCommand cmd = new SqlCommand())
{
cmd.Connection = con;
cmd.CommandText = "selectMusician";
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#id", recordId);
using (SqlDataReader reader = cmd.ExecuteReader(CommandBehavior.CloseConnection))
{
if (reader.HasRows)
{
reader.Read();
objMusician = new Musician((int) reader["id"]);
objMusician.Name = (string) reader["name"];
}
if objMusician != null)
{
objMusician.Albums = Albums.GetAlbums((int)objMusician.ID);
objMusician.Tours = Tours.GetTours((int)objMusician.ID);
objMusician.Interviews = Interviews.GetInterviews((int)objMusician.ID);
}
}
}
return objMusician;
}
}
Related
I was getting quite often the following error:
Npgsql.NpgsqlException: 'The connection pool has been exhausted, either raise MaxPoolSize (currently 100) or Timeout (currently 15 seconds)'
Then I looked for possible causes and solutions in here and found out that I should be applying the using statement. So I reviewed all my code and did that.
However, I keep getting that error while testing a button that gets information from my database, does some calculation and writes the results in a few textboxes. It usually crashes at the fifth-ish time I click it. A piece of the code follows:
private void CalcTemp(Cable cable)
{
string sqlString = "Server=172.19.2.40; Port=5432; User Id=postgres; Password=password; Database=PROLIG;";
using (NpgsqlConnection sqlCon = new NpgsqlConnection(sqlString))
{
string cmdString = #"SELECT tempamb, elevmaxonan, elevmaxonaf, elevmaxonaf2, topoil1_2, topoil1_4, especial1factor, especial1topoil,
especial2factor, especial2topoil, especial3factor, especial3topoil, especial4factor, especial4topoil,
especial5factor, especial5topoil, especial6factor, especial6topoil FROM correntes WHERE prolig_ofs_id = #id;";
NpgsqlCommand sqlCmd = new NpgsqlCommand(cmdString, sqlCon);
sqlCmd.Parameters.AddWithValue("id", StartOF.MyOF.id);
NpgsqlDataAdapter sqlDa = new NpgsqlDataAdapter(sqlCmd);
DataTable dt = new DataTable();
sqlDa.Fill(dt);
//does calculation
}
Any thoughts on why this is happening and how to fix it?
Thanks a lot.
Just add using to your command creation:
using (NpgsqlCommand sqlCmd = new NpgsqlCommand(cmdString, sqlCon))
{
Disposing all objects which implement IDisposable is a good practice.
Since your command is not disposed of in time, your connection is not closed and returned to the pool.
It is a lot of stuff on dispose which must be executed directly (or by using).
protected override void Dispose(bool disposing)
{
if (State == CommandState.Disposed)
return;
if (disposing)
{
// Note: we only actually perform cleanup here if called from Dispose() (disposing=true), and not
// if called from a finalizer (disposing=false). This is because we cannot perform any SQL
// operations from the finalizer (connection may be in use by someone else).
// We can implement a queue-based solution that will perform cleanup during the next possible
// window, but this isn't trivial (should not occur in transactions because of possible exceptions,
// etc.).
if (_prepared == PrepareStatus.Prepared)
_connector.ExecuteBlind("DEALLOCATE " + _planName);
}
Transaction = null;
Connection = null;
State = CommandState.Disposed;
base.Dispose(disposing);
}
So you need the following code:
private void CalcTemp(Cable cable)
{
string sqlString = "Server=172.19.2.40; Port=5432; User Id=postgres; Password=password; Database=PROLIG;";
using (NpgsqlConnection sqlCon = new NpgsqlConnection(sqlString))
{
string cmdString = #"SELECT * FROM correntes WHERE prolig_ofs_id = #id;";
using (NpgsqlCommand sqlCmd = new NpgsqlCommand(cmdString, sqlCon))
{
sqlCmd.Parameters.AddWithValue("id", StartOF.MyOF.id);
NpgsqlDataAdapter sqlDa = new NpgsqlDataAdapter(sqlCmd);
DataTable dt = new DataTable();
sqlDa.Fill(dt);
//does calculation
} //end using command (calls dispose on command, even if exception happens)
} //end using connection (calls dispose on connection object, even if exception happens)
}
Next advice - do not use data tables in case of large amount of data. Use DataReader instead.
I have a .NET Core program that uses the MySqlConnection class. My Database is a ClearDB Database that is stored in Azure.
When I launch the program it is working like it should. But when I wait for like 10 minuts doing nothing, it wont connect to the database anymore(Timeout?). Restarting the program and it works again.
When looking at the connections on the ClearDB webpage it isn't closing when I close it in my program. After 10 minuts or so it closes automaticly, as I see in ClearDB webpage. But with the program still running it wont connect to the database anymore. Restarting program is only solution.
Code for now looks something like this:
private static async Task<uint> getDeviceId(string macAddress)
{
using (var connection = new MySqlConnection(ConnectionString))
{
uint returnvalue = 0;
var cmd = connection.CreateCommand() as MySqlCommand;
cmd.CommandText = #"SELECT id FROM devices WHERE mac = '" + macAddress + "'";
connection.Open();
Console.WriteLine(connection.State);
DbDataReader reader = await cmd.ExecuteReaderAsync();
using (reader)
{
while (await reader.ReadAsync())
{
returnvalue = await reader.GetFieldValueAsync<uint>(0);
}
}
reader.Dispose();
cmd.Dispose();
return returnvalue;
}
}
I have tried the following:
Using statement
Close/dispose connection,reader and command
Pooling=false in connectionstring
But none of them works. Somebody got an idea?
Assuming MySql provider is like the MSSQL provider, it does not actually close the connection in the database, it just releases it back to the pool.
You do not want to disable pooling, you will kill efficiency.
This is by design, and what you want.
The using statement from the code snippet should close your connections. However, I'm not sure how that interacts with async, or how ClearDB differs from normal MySql. Given the issues in the question and that lack of clarity, you might try this, just to see if it helps:
private static async Task<uint> getDeviceId(string macAddress)
{
uint returnvalue = 0;
MySqlConnection connection;
try
{
connection = new MySqlConnection(ConnectionString);
var cmd = connection.CreateCommand() as MySqlCommand;
//Don't EVER(!) use string concatenation like that in a query!
cmd.CommandText = #"SELECT id FROM devices WHERE mac = #macAddress";
cmd.Parameters.Add("#macAddress", MySqlDbType.VarChar, 18).Value = macAddress;
connection.Open();
Console.WriteLine(connection.State);
DbDataReader reader = await cmd.ExecuteReaderAsync();
using (reader)
{
while (await reader.ReadAsync())
{
returnvalue = await reader.GetFieldValueAsync<uint>(0);
}
}
reader.Dispose();
cmd.Dispose();
}
finally
{
connection.Close();
connection.Dispose();
}
return returnvalue;
}
A using block basically just re-writes your code as try/finally anyway, so doing this step by-hand can sometimes make debugging easier (you can log where it hits the .Close() call).
If this does resolve the problem, I wouldn't stop there, but rather start from there and see just how close to "normal" code you can get. I'm also concerned here that you have disabled connection pooling, and that this method is static.
I have a c# program working with a SQL Server database running on different client PCs.
Now, I am getting this exception which [if not catched] closes my application:
Transaction (Process ID ...) was deadlocked on lock | communication
buffer resources with another process and has been chosen as the
deadlock victim. Rerun the transaction.
and the stack trace shows that the exception happens here:
at Gui.DB.sqlServerWrapper.MarkAsNonFreshSample(String barcode, Int32 devID)
Where the mentioned method is this:
public void MarkAsNonFreshSample(string barcode, int devID)
{
mux.WaitOne();
var sql = "DELETE FROM results WHERE barcode=#barcode AND devID=#devID";
var command = new SqlCommand(sql, conn);
command.Parameters.AddWithValue("barcode", barcode);
command.Parameters.AddWithValue("devID", devID);
command.ExecuteNonQuery();
mux.ReleaseMutex();
}
I am using the "mux" mutex to exclusive access to the sql connection "conn" because my program has multiple threads.
I am not using a complex query or a long transaction (and I guess this short query is one transaction which should not cause a deadlock. am I wrong?).
Where is the problem and how I should fix it?
Sharing a connection like that is not a good practice
public void MarkAsNonFreshSample(string barcode, int devID)
{
using (SqlConnection con = new SqlConnection(conString))
{
con.Open();
using (SqlCommand command = con.CreateCommand())
{
command.CommandText = "DELETE FROM results WHERE barcode=#barcode AND devID=#devID";
command.Parameters.AddWithValue("barcode", barcode);
command.Parameters.AddWithValue("devID", devID);
command.ExecuteNonQuery();
}
}
}
Normaly I would use either "Using" or just connection.open() and connection.close(). But when I'm calling stored procedure, this is not needed. How come? (Yes the codesnippit below works without using or open).
try {
SqlCommand cmd = new SqlCommand("***", connectionSiteDb);
DataTable dt = new DataTable();
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add(new SqlParameter("#ProcessSegmentID", ProcessSSegmentID));
cmd.Parameters.Add(new SqlParameter("#PO_RecipeID", PO_RecipeID));
cmd.Parameters.Add(new SqlParameter("#ProductSegmentVersion", ProductSegmentVersion));
SqlDataAdapter da = new SqlDataAdapter();
da.SelectCommand = cmd;
da.Fill(dt);
return dt;
}
catch (Exception e) {
Console.WriteLine(e);
return null;
}
Using using statement or not doesn't depends on your CommandType is Text or StoredProcedure.
SqlDataAdapter.Fill opens connection itself.
From Populating a DataSet from a DataAdapter
The Fill method implicitly opens the Connection that the DataAdapter
is using if it finds that the connection is not already open. If Fill
opened the connection, it will also close the connection when Fill is
finished. This can simplify your code when dealing with a single
operation such as a Fill or an Update.
Also DbDataAdapter.Fill(DataTable)
The
connection object associated with the SELECT statement must be valid,
but it does not need to be open. If the connection is closed before
Fill is called, it is opened to retrieve data, then closed. If the
connection is open before Fill is called, it remains open.
Since SqlDataAdapter doesn't implement IDisposable, you don't need to use using statement with it.
If you want to look under the hood, you can check QuietClose method and QuietOpen method implementations;
static private void QuietClose(IDbConnection connection, ConnectionState originalState)
{
// close the connection if:
// * it was closed on first use and adapter has opened it, AND
// * provider's implementation did not ask to keep this connection open
if ((null != connection) && (ConnectionState.Closed == originalState)) {
// we don't have to check the current connection state because
// it is supposed to be safe to call Close multiple times
connection.Close();
}
}
// QuietOpen needs to appear in the try {} finally { QuietClose } block
// otherwise a possibility exists that an exception may be thrown, i.e. ThreadAbortException
// where we would Open the connection and not close it
static private void QuietOpen(IDbConnection connection, out ConnectionState originalState)
{
originalState = connection.State;
if (ConnectionState.Closed == originalState) {
connection.Open();
}
}
When I have a SQLConnection within a using clause as illustrated below do I need to explicitly close the connection?
protected SqlConnection Connection
{
get
{
if (this.connection == null)
{
this.connection = new SqlConnection(this.ConnectionString);
}
if (this.connection.State != ConnectionState.Open)
{
this.connection.Open();
}
return this.connection;
}
}
using (SqlConnection connection = this.Connection)
{
using (SqlCommand cmd = connection.CreateCommand())
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "....";
using (SqlDataReader reader = cmd.ExecuteReader())
{
while (reader.Read())
{
etc
}
}
}
}
No you don't. The Dispose() method of a connection will call Close if the connection is already open.
You should also change your code as John Gathogo suggests to create a new connection object each time it needs one. Your code will fail as it is because the second time you try to use the connection it will already be disposed.
ADO.NET uses connection pooling to keep a pool of open connections which it provides to whoever calls Open. This means that creating and opening new connection doesn't cost anything as long as there are available connections in the pool. Keeping a connection open for longer than necessary will degrade performance.
The using block will always call Dispose, which will close the connection.
But, you are keeping the connection object and intend to reuse it, which is not possible once it has been disposed. You shouldn't keep the connection object, you should just throw it away and create a new one when needed. The actual connections to the database are pooled, so when you create a new connection object it will reuse one of the connections from the pool. When you dispose the connection object, the actual connection is returned to the pool.
You could easily change you code to below and achieve what you want:
using (SqlConnection connection = new SqlConnection(this.ConnectionString))
{
connection.Open();
using (SqlCommand cmd = connection.CreateCommand())
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "....";
using (SqlDataReader reader = cmd.ExecuteReader())
{
while (reader.Read())
{
//etc
}
}
}
}
The runtime will take care of closing the connection and disposing the resources for you