We Keep Coding

Postgres connection state stays active when an exception occurs

The state of the connection stays active in pool, when there is an exception while executing a query or stored procedure through c#.
The Npgsql version that I am using is 4.1.7.
Here is the code that I am trying to execute.
NpgsqlCommand cmd = null;
NpgsqlDataAdapter sda = null;
NpgsqlConnection conn = null;
try {
string sql = "a_test";
conn = new NpgsqlConnection("Server=localhost;Port=5432;Username=admin;Password=test;Database=majordb;SearchPath=dbs;CommandTimeout=300;MaxPoolSize=500;Connection Idle Lifetime=180;Connection Pruning Interval=5;");
cmd = new NpgsqlCommand(sql);
cmd.CommandType = CommandType.StoredProcedure;
sda = new NpgsqlDataAdapter(cmd);
cmd.Connection = conn;
sda.Fill(dataTable);
}
catch (Exception e) {
//log
}
finally {
if(null != sda)
{
try
{
sda.Dispose();
}
catch (Exception)
{
}
}
try
{
cmd.Connection.Close();
cmd.Connection.Dispose();
}
catch (Exception)
{
}
try
{
cmd.Dispose();
}
catch (Exception)
{
}
}
If the above code executes properly without any exception, the connection state in pool goes to idle, which is correct. But if an exception occurs while executing, like below:
"Npgsql.NpgsqlException (0x80004005): Exception while reading from stream --->
System.IO.IOException: Unable to read data from the transport connection: A connection attempt
failed because the connected party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond."
The connection state in pool shows as active for about 5 mins or so, even though the close/dispose methods are called in finally block. This means the close/dispose did not properly executed by Npgsql. If the program keeps the connection state in pool active for every connection ran within 5 mins, then there can be an issue with MaxPoolSize error.
I wanted to see the connection state to idle, even when there is an exception. How do I do this.
Please note: I am not looking for a solution to the exception that I listed above. I am looking for a solution where the connection state is changed to idle from active when there is an exception while executing the above code.
To know if the connection state is active or not I used the following query:
SELECT
pid,
usename,
application_name,
datname,
client_addr,
rank() over (partition by client_addr order by backend_start ASC) as rank,
state,
state_change,
current_timestamp,
query,
query_start,
backend_start,
FROM
pg_stat_activity
WHERE
pid <> pg_backend_pid( )
AND
application_name !~ '(?:psql)|(?:pgAdmin.+)'
AND
datname = current_database()
AND
usename = current_user
Any help is really appreciated.

Setting CommandTimeout to solve lock wait timeout exceeded try restarting transaction

I have an error like this
lock wait timeout exceeded try restarting transaction
Maybe I didn't understand it. But I've a solution if I set CommandTimeout=1000 or something higher. I didn't try it in production yet. But I'd like to hear any opinion on this.
// 40 lines of command.Parameters here
command.Parameters.AddWithValue("sample1", sam1);
command.Parameters.AddWithValue("sample2", sam2);
command.Parameters.AddWithValue("sample3", sam2);
try
{
command.ExecuteNonQuery();
}
catch (MySqlException mex)
{

I was receiving "lock wait timeout exceeded try restarting transaction" intermittently. Then I started wrapping everything in transactions and I stopped receiving those errors. This should prevent table locks from remaining after the query is executed.
(Assuming "conn" is a MySqlConnection, "iLevel" is the isolation level you want to use, and "query" contains your query as a string)
int rowCount = 0; // In case you want the number of rows affected
try
{
if (conn.State != ConnectionState.Open)
conn.Open();
MySqlCommand command = new MySqlCommand(query, conn);
using(var transaction = conn.BeginTransaction(iLevel))
{
command.Transaction = transaction;
command.CommandTimeout = int.MaxValue;
// Set parameters etc...
try
{
rowCount = command.ExecuteNonQuery();
transaction.Commit();
}
catch(Exception ex)
{
transaction.Rollback();
// Handle query exception...
}
}
}
catch(Exception ex)
{
// Handle general exception...
}

You could try (just for testing purposes) set transaction isolation level =
"READ COMMITTED" and if this fails try set to "READ UNCOMMITTED"
MySql reference
check for dead lock:
"SHOW ENGINE INNODB STATUS" from the MySQL Command line client (not a
query browser) will give you info on deadlocks.
Deadlocks can also be caused by uncommitted transactions (usually
program bugs) and the person who is running the uncommitted
transaction will not see the problem as they will be working fine
(through their data will not be committed). Quote from here

How does SQLTransaction.Commit() work?

A few day ago, I have studied SqlTransaction and I know the purpose of SqlTransaction.Commit() - it should "commit the database transaction." - MSDN.
But HOW DOES IT WORK?
For example: I wrote a piece of code like this:
using (SqlTransaction tran = connection.BeginTransaction())
{
try
{
using (SqlCommand cmd = connection.CreateCommand())
{
cmd.CommandText = msg.command;
cmd.Transaction = tran;
cmd.ExecuteNonQuery();
}
}
catch (Exception)
{
// if all of above have any exception, that's mean my transaction is
// failure and my database has no change.
return false;
}
tran.Commit();
// if all of above have no problems, that's mean my transaction is successful
return true;
connection.Dispose();
}
In this case, SQL Server is on another computer.
I guess: commit method has two periods, Period 1: when I implement tran.Commit(), compiler will signal SQL Server and talk to SQL Server that: "I'm ok, please help me commit (change) data", and then SQL Server will implement compiler's request. Period 2: when SQL Server implement compiler's request completely, implement result will be return to our compiler. When our compiler receive implement result, our compiler will continue compile the next command line ("return true").
But if in second period, the connection is broken and implement result isn't transferred back to our compiler. In this case, our transaction is success or not? Is data persisted in SQL Server or not?
Additional question: my prediction about two period of SQLTransaction.Commit() is true or not?
Thanks!

try
{
using (var conn = new SqlConnection(/* connection string or whatever */))
{
conn.Open();
using (var trans = conn.BeginTransaction())
{
try
{
using (var cmd = conn.CreateCommand())
{
cmd.Transaction = trans;
/* setup command type, text */
/* execute command */
}
trans.Commit();
}
catch (Exception ex)
{
trans.Rollback();
/* log exception and the fact that rollback succeeded */
}
}
}
}
catch (Exception ex)
{
/* log or whatever */
}
and read this also
https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqltransaction.commit(v=vs.110).aspx

Close connection when using a Transaction

I have the following method which uses a transaction.
private string getDocumentDetailsByNumber(string DocNumber)
{
SqlTransaction transaction = DALDBConnection.SqlConnection.BeginTransaction();
try
{
DataSet DocNum = new DataSet();
string sDocNumber = "";
string[] taleNamesDoc = new string[1];
taleNamesDoc[0] = "docnumber";
SqlParameter[] paramDoc = new SqlParameter[1];
paramDoc[0] = new SqlParameter("#DocumentNumber", DocNumber.ToString().Trim());
SqlHelper.FillDataset(transaction, CommandType.StoredProcedure, "spGetDocumentDetailsByNumber", DocNum, taleNamesDoc, paramDoc);
string docTitle = DocNum.Tables["docnumber"].Rows[0][0].ToString();
transaction.Commit();
return docTitle;
}
catch (Exception ex)
{
transaction.Rollback();
throw ex;
}
}
after running the method several times, user ended up getting the error message below.
the timeout period elapsed prior to obtaining a connection from the
pool
Error occurred because I haven't closed the connection and the connection pool has over flown.
I tried to close the connection before committing the transaction.
transaction.Connection.Close();
transaction.Commit();
Then got the following error.
This SqlTransaction has completed; it is no longer usable
How can I close the connection to avoid the error?

You cannot exhaust your pool by using a single connection. You need to close all connections you are using. Preferably after the transaction has ended one way or another. using blocks are your friend for almost all database related objects.
By the way:
throw ex;
This damages your exception by replacing the original stacktrace. Use:
throw;
to rethrow the exception you caught unchanged.

As mentioned, you should dispose of the connection properly. I've modified your code to demonstrate. Please note you will need to substitute the connection string with yours.
private string getDocumentDetailsByNumber(string DocNumber)
{
using (var connection = new SqlConnection("My Connection String"))
{
SqlTransaction transaction = connection.BeginTransaction();
DataSet DocNum = new DataSet();
string sDocNumber = "";
string[] taleNamesDoc = new string[1];
taleNamesDoc[0] = "docnumber";
SqlParameter[] paramDoc = new SqlParameter[1];
paramDoc[0] = new SqlParameter("#DocumentNumber", DocNumber.ToString().Trim());
SqlHelper.FillDataset(transaction, CommandType.StoredProcedure, "spGetDocumentDetailsByNumber", DocNum, taleNamesDoc, paramDoc);
string docTitle = DocNum.Tables["docnumber"].Rows[0][0].ToString();
transaction.Commit();
return docTitle;
} // Connection is disposed and cleaned up.
}
Opening new connections are cheap and should not be frowned upon. Each one you call the database you should open a new one like this. By maintaining a connection and not disposing of it, you are taking resources away from the database as well. It does not have an infinite amount of connections in its pool that can be used at once.
edit
Removed try/catch as mentioned in comments. If an exception is thrown while in the using block, a rollback will occur and the exception passed up the stack.

Have you considered CALLNIG CLOSE ON IT? Would be obvious to close a connection or?
Anything that implements IDIsposable should be disposed, btw., not just closed. And SqlConnection implements IDisposable. THis has nothing to do with SqlTransaction - you violae a fundamental rule of the .NET world by not disposing a disposable routine.

SQLTransaction has completed error

I got following error once in my application.
This SQLTransaction has completed; it is no longer usable
Stack Trace is attached below – It says about Zombie Check and Rollback.
What is the mistake in the code?
Note: This error came only once.
UPDATE
From MSDN - SqlTransaction.Rollback Method
A Rollback generates an InvalidOperationException if the connection is terminated or if the transaction has already been rolled back on the server.
From Zombie check on Transaction - Error
One of the most frequent reasons I have seen this error showing up in various applications is, sharing SqlConnection across our application.
CODE
public int SaveUserLogOnInfo(int empID)
{
int? sessionID = null;
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
SqlTransaction transaction = null;
try
{
transaction = connection.BeginTransaction();
sessionID = GetSessionIDForAssociate(connection, empID, transaction);
//Other Code
//Commit
transaction.Commit();
}
catch
{
//Rollback
if (transaction != null)
{
transaction.Rollback();
transaction.Dispose();
transaction = null;
}
//Throw exception
throw;
}
finally
{
if (transaction != null)
{
transaction.Dispose();
}
}
}
return Convert.ToInt32(sessionID,CultureInfo.InvariantCulture);
}
Stack Trace
REFERENCE:
What is zombie transaction?
Zombie check on Transaction - Error
SqlTransaction has completed
http://forums.asp.net/t/1579684.aspx/1
"This SqlTransaction has completed; it is no longer usable."... configuration error?
dotnet.sys-con.com - SqlClient Connection Pooling Exposed
Thread abort leaves zombie transactions and broken SqlConnection

You should leave some of the work to compiler, to wrap that in a try/catch/finally for you.
Also, you should expect that Rollback can occasionally throw an exception, if a problem occurs in Commit stage, or if a connection to server breaks. For that reason you should wrap it in a try/catch.
try
{
transaction.Rollback();
}
catch (Exception ex2)
{
// This catch block will handle any errors that may have occurred
// on the server that would cause the rollback to fail, such as
// a closed connection.
Console.WriteLine("Rollback Exception Type: {0}", ex2.GetType());
Console.WriteLine(" Message: {0}", ex2.Message);
}
This is copied exactly from MSDN documentation page for Rollback method.
I see that you're worried that you have a zombie transaction. In case you pasted, it doesn't sound like you have a problem. You're transaction has been completed, and you should no longer have anything to do with it. Remove references to it if you hold them, and forget about it.
From MSDN - SqlTransaction.Rollback Method
A Rollback generates an InvalidOperationException if the connection is terminated or if the transaction has already been rolled back on the server.
Rethrow a new exception to tell user that data may not have been saved, and ask her to refresh and review

Note: This error came only once.
then it is very hard to say much; it could be simply that the // Other Code etc simply took to long, and the entire thing got killed. Maybe your connection died, or an admin deliberately killed it because you were blocking.
What is the mistake in the code?
over-complicating it; it can be much simpler:
using (var connection = new SqlConnection(connectionString))
{
connection.Open();
using(var transaction = connection.BeginTransaction())
{
try
{
sessionID = GetSessionIDForAssociate(connection, empID, transaction);
//Other Code
transaction.Commit();
}
catch
{
transaction.Rollback();
throw;
}
}
}
much less code to get wrong.

I use code below can reproduce this error, I use 1000 tasks to execute Sql, after about 300 tasks Successfully Completed, lots of exception about timeout error start to occur on ExecuteNonQuery(),
then next error This SqlTransaction has completed will occur on transaction.RollBack(); and its call stack also contains ZombieCheck().
(If single program with 1000 tasks pressure not enough, you can execute multiple compiled exe file at the same time, or even use multi computers execute to one DataBase.)
So I guess one of the reason cause this error can be something wrong in Connection, then cause the transaction error happens as well.
Task[] tasks = new Task[1000];
for (int i = 0; i < 1000; i++)
{
int j = i;
tasks[i] = new Task(() =>
ExecuteSqlTransaction("YourConnectionString", j)
);
}
foreach (Task task in tasks)
{
task.Start();
}
/////////////
public void ExecuteSqlTransaction(string connectionString, int exeSqlCou)
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
SqlCommand command = connection.CreateCommand();
SqlTransaction transaction;
// Start a local transaction.
transaction = connection.BeginTransaction();
// Must assign both transaction object and connection
// to Command object for a pending local transaction
command.Connection = connection;
command.Transaction = transaction;
try
{
command.CommandText =
"select * from Employee";
command.ExecuteNonQuery();
// Attempt to commit the transaction.
transaction.Commit();
Console.WriteLine("Execute Sql to database."
+ exeSqlCou);
}
catch (Exception ex)
{
Console.WriteLine("Commit Exception Type: {0}", ex.GetType());
Console.WriteLine(" Message: {0}", ex.Message);
// Attempt to roll back the transaction.
try
{
transaction.Rollback();
}
catch (Exception ex2)
{
// This catch block will handle any errors that may have occurred
// on the server that would cause the rollback to fail, such as
// a closed connection.
Console.WriteLine("Rollback Exception Type: {0}", ex2.GetType());
Console.WriteLine(" Message: {0}", ex2.Message);
}
}
}
}
Besides I find if I commit twice sequentailly will invoke this exception as well.
transaction.Commit();
transaction.Commit();
Or if Connection Closed before commit also invoke this error.
connection.Close();
transaction.Commit();
Update:
I find it strange that I create another new table and insert 500 thousand data to it,
then use 100000 tasks with select * from newtable sql, running 5 programs at the same time, this time the Timeout Error occur, but when transaction.Rollback() it didn't invoke the SQLTransaction has completed error.
but if the Timeout Error occur, jump into the catch block, and in the catch block do transaction.Commit() again, the SQLTransaction has completed error will happen.

I have experienced this error once and i was stuck and unable to know what is going wrong. Actually i was deleting a record and in the Stored procedure i was not deleting its child and specially the delete statement in Stored Procedure was inside the Transaction boundary. I removed that transaction code from stored procedure and got rid of getting this Error of “This SqlTransaction has completed; it is no longer usable.”

This message is simply because that you wrote code that throws an exception after the transaction has been already committed successfully.Please try to check the code you wrote after the Commit method or you can handle it by using Try..Catch and finally Blocks :) .