We Keep Coding

Postgres connection state stays active when an exception occurs

The state of the connection stays active in pool, when there is an exception while executing a query or stored procedure through c#.
The Npgsql version that I am using is 4.1.7.
Here is the code that I am trying to execute.
NpgsqlCommand cmd = null;
NpgsqlDataAdapter sda = null;
NpgsqlConnection conn = null;
try {
string sql = "a_test";
conn = new NpgsqlConnection("Server=localhost;Port=5432;Username=admin;Password=test;Database=majordb;SearchPath=dbs;CommandTimeout=300;MaxPoolSize=500;Connection Idle Lifetime=180;Connection Pruning Interval=5;");
cmd = new NpgsqlCommand(sql);
cmd.CommandType = CommandType.StoredProcedure;
sda = new NpgsqlDataAdapter(cmd);
cmd.Connection = conn;
sda.Fill(dataTable);
}
catch (Exception e) {
//log
}
finally {
if(null != sda)
{
try
{
sda.Dispose();
}
catch (Exception)
{
}
}
try
{
cmd.Connection.Close();
cmd.Connection.Dispose();
}
catch (Exception)
{
}
try
{
cmd.Dispose();
}
catch (Exception)
{
}
}
If the above code executes properly without any exception, the connection state in pool goes to idle, which is correct. But if an exception occurs while executing, like below:
"Npgsql.NpgsqlException (0x80004005): Exception while reading from stream --->
System.IO.IOException: Unable to read data from the transport connection: A connection attempt
failed because the connected party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond."
The connection state in pool shows as active for about 5 mins or so, even though the close/dispose methods are called in finally block. This means the close/dispose did not properly executed by Npgsql. If the program keeps the connection state in pool active for every connection ran within 5 mins, then there can be an issue with MaxPoolSize error.
I wanted to see the connection state to idle, even when there is an exception. How do I do this.
Please note: I am not looking for a solution to the exception that I listed above. I am looking for a solution where the connection state is changed to idle from active when there is an exception while executing the above code.
To know if the connection state is active or not I used the following query:
SELECT
pid,
usename,
application_name,
datname,
client_addr,
rank() over (partition by client_addr order by backend_start ASC) as rank,
state,
state_change,
current_timestamp,
query,
query_start,
backend_start,
FROM
pg_stat_activity
WHERE
pid <> pg_backend_pid( )
AND
application_name !~ '(?:psql)|(?:pgAdmin.+)'
AND
datname = current_database()
AND
usename = current_user
Any help is really appreciated.

Oracle DataAccess Lock Code Stops

When I use "Select ... for update" query code stops without exception or any freeze. It just won't go to the next instruction. It work fine for a regular select query. Any idea why ?
string selectQuery = "select * from table_name where id = 1 FOR UPDATE"; // Code stops on data fill
string selectQueryNoLock = "select * from table_name where id = 1"; // Code execute normally
string connectionString = "fake";
OracleConnection oracleConnection = new OracleConnection(connectionString);
Console.WriteLine("Open Connection");
oracleConnection.Open();
OracleTransaction oracleTransaction = oracleConnection.BeginTransaction();
OracleCommand oracleCommand = new OracleCommand(selectQuery, oracleConnection);
oracleCommand.Transaction = oracleTransaction;
OracleDataAdapter dataAdapter = new OracleDataAdapter(oracleCommand);
DataSet dataSet = new DataSet();
dataAdapter.Fill(dataSet);
// This line is never reached!
Console.WriteLine("Press any key...");

A standard select ... for update will wait indefinitely if another transaction is currently holding a lock on that row. In normal apps, this is not really a problem, as transactions should be short-lived.
However, if an indefinite wait is not the behavior that you want, and you would rather be told if you are unable to acquire a lock after a certain amount of time, you can specify a timeout like this:
select ... for update wait 10
... which will return an error if the timeout elapses and you are unable to acquire the lock.
Alternatively, if you don't want to wait at all, you can also do:
select ... for update nowait
... which will immediately return an error if you can't acquire the lock right away.

If my C# times out with a stored procedure call, does the procedure continue running?

I have just a general type of question. If I have a C# application that calls a SQL Server stored procedure, and the C# application times out, does the procedure call on the server continue running to it's completion?

No. Below is a reproduction. When the timeout occurs the running process will be killed, halting it immediately. If you do not have a transaction specified, work that has been done in the stored procedure prior to the timeout will be persisted. Similarly, if the connection to the server is severed by some outside force, SQL Server will kill the running process.
using (var conn = new SqlConnection(#"Data Source=.;Initial Catalog=Test;Integrated Security=True"))
{
conn.Open();
using (var setupTable = new SqlCommand(#"
IF NOT EXISTS (
SELECT *
FROM
sys.schemas s
INNER JOIN sys.tables t ON
t.[schema_id] = s.[schema_id]
WHERE
s.name = 'dbo' AND
T.name = 'TimeoutTest')
BEGIN
CREATE TABLE dbo.TimeoutTest
(
ID int IDENTITY(1,1) PRIMARY KEY,
CreateDate datetime DEFAULT(getdate())
);
END
-- remove any rows from previous runs
TRUNCATE TABLE dbo.TimeoutTest;", conn))
{
setupTable.ExecuteNonQuery();
}
using (var checkProcExists = new SqlCommand(#"
SELECT COUNT(*)
FROM
sys.schemas s
INNER JOIN sys.procedures p ON
p.[schema_id] = s.[schema_id]
WHERE
s.name = 'dbo' AND
p.name = 'AddTimeoutTestRows';", conn))
{
bool procExists = ((int)checkProcExists.ExecuteScalar()) == 1;
if (!procExists)
{
using (var setupProc = new SqlCommand(#"
CREATE PROC dbo.AddTimeoutTestRows
AS
BEGIN
DECLARE #stop_time datetime;
SET #stop_time = DATEADD(minute, 1, getdate());
WHILE getdate() < #stop_time
BEGIN
INSERT INTO dbo.TimeoutTest DEFAULT VALUES;
-- wait 10 seconds between inserts
WAITFOR DELAY '0:00:10';
END
END", conn))
{
setupProc.ExecuteNonQuery();
}
}
}
bool commandTimedOut = false;
try
{
using (var longExecution = new SqlCommand("EXEC dbo.AddTimeoutTestRows;", conn))
{
// The time in seconds to wait for the command to execute.
// Explicitly setting the timeout to 30 seconds for clarity.
longExecution.CommandTimeout = 30;
longExecution.ExecuteNonQuery();
}
}
catch (SqlException ex)
{
if (ex.Message.Contains("Timeout"))
{
commandTimedOut = true;
}
else
{
throw;
}
}
Console.WriteLine(commandTimedOut.ToString());
// Wait for an extra 30 seconds to let any execution on the server add more rows.
Thread.Sleep(30000);
using (var checkTableCount = new SqlCommand(#"
SELECT COUNT(*)
FROM
dbo.TimeoutTest t;", conn))
{
// Only expecting 3, but should be 6 if server continued on without us.
int rowCount = (int)checkTableCount.ExecuteScalar();
Console.WriteLine(rowCount.ToString("#,##0"));
}
}
Console.ReadLine();
produces the following output
True
3
even though running the stored procedure from Management Studio will add 6 rows in the one minute time frame.

Short answer is Yes .... here is some info to back up my claim
There are actually several places where a application can 'time out' but one is a Command Execution time out ...
command execution time out - This property is the cumulative time-out for all network reads during command execution or processing of the results. A time-out can still occur after the first row is returned, and does not include user processing time, only network read time.
A Command is not going to rollback on its own if it times out. This will need a transaction around the code if it times out.
if a timeout can occur when rows are being returned than means a timeout can occur anytime C# is not going to tell SQL Server to stop running the command. Things can be done about that such as wrapping the command in a transaction
Source: https://blogs.msdn.microsoft.com/mattn/2008/08/29/sqlclient-timeouts-revealed/ ... and experience

If you are using the SQlCommand Class, once the app times out , the query execution will be rolled back.

My thoughts about that are that it's all about the connection opened by the call to the procedure.
If your code is executed within a using block or if it is garbage collected then i think that the execution of the SP will be rolled back.

Conn = new SqlConnection(ConnStr);
Conn.Open();
myCommand = new SqlCommand();
myCommand.CommandTimeout = 180000;
myCommand.Connection = Conn;
myCommand.CommandType = System.Data.CommandType.StoredProcedure;

Setting CommandTimeout to solve lock wait timeout exceeded try restarting transaction

I have an error like this
lock wait timeout exceeded try restarting transaction
Maybe I didn't understand it. But I've a solution if I set CommandTimeout=1000 or something higher. I didn't try it in production yet. But I'd like to hear any opinion on this.
// 40 lines of command.Parameters here
command.Parameters.AddWithValue("sample1", sam1);
command.Parameters.AddWithValue("sample2", sam2);
command.Parameters.AddWithValue("sample3", sam2);
try
{
command.ExecuteNonQuery();
}
catch (MySqlException mex)
{

I was receiving "lock wait timeout exceeded try restarting transaction" intermittently. Then I started wrapping everything in transactions and I stopped receiving those errors. This should prevent table locks from remaining after the query is executed.
(Assuming "conn" is a MySqlConnection, "iLevel" is the isolation level you want to use, and "query" contains your query as a string)
int rowCount = 0; // In case you want the number of rows affected
try
{
if (conn.State != ConnectionState.Open)
conn.Open();
MySqlCommand command = new MySqlCommand(query, conn);
using(var transaction = conn.BeginTransaction(iLevel))
{
command.Transaction = transaction;
command.CommandTimeout = int.MaxValue;
// Set parameters etc...
try
{
rowCount = command.ExecuteNonQuery();
transaction.Commit();
}
catch(Exception ex)
{
transaction.Rollback();
// Handle query exception...
}
}
}
catch(Exception ex)
{
// Handle general exception...
}

You could try (just for testing purposes) set transaction isolation level =
"READ COMMITTED" and if this fails try set to "READ UNCOMMITTED"
MySql reference
check for dead lock:
"SHOW ENGINE INNODB STATUS" from the MySQL Command line client (not a
query browser) will give you info on deadlocks.
Deadlocks can also be caused by uncommitted transactions (usually
program bugs) and the person who is running the uncommitted
transaction will not see the problem as they will be working fine
(through their data will not be committed). Quote from here

"Unspecified error" in sql that goes away after few seconds

I built a parser that takes data stored in an xml file and sends it into a Microsoft Access database using linq-to-sql. I have the sql insert commands and they work... until they don't.
It's odd, I have each SQL command run (I keep them in a List and execute each command one at a time) and the first 40 or so run fine until they start hitting "unspecified error"s. The thing is, if I swallow the exception and instead have the exception catcher keep retrying, after a few seconds, they start working again. This means it's not an error of the SQL query itself (or at least how it's written).
This pattern repeats (there are thousands of inserts) many times. If I do normal exception handling, the program will just skip a few records while the error happens and keep inserting when whatever causes it temporarily goes away. if I let it run it's course, it inserts some records, skips some, inserts, skips, repeat and eventually inserts less than 2/3 of the records.
Is there any reason why my computer would only run 40 or so Inserts and then refuse to run more for a random but short interval?
I'm at a loss on what could be causing this.
The application is natively run; it does not use any server/web communication and all I found when looking for "unspecified error" pointed me to occurrences in ADO.NET applications.
Here's the code the error happens in:
public static string insertQuery(string sql)
{
string connetionString = #"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\Documents and Settings\Owner\Desktop\Arbeit\TrademarkParserproject1\TrademarkParserproject\bin\x86\Debug\Database.accdb";
OleDbConnection connection;
OleDbDataAdapter oledbAdapter = new OleDbDataAdapter();
connection = new OleDbConnection(connetionString);
string success = "false";
try
{
connection.Open();
oledbAdapter.InsertCommand = new OleDbCommand(sql, connection);
oledbAdapter.InsertCommand.ExecuteNonQuery();
}
catch (Exception ex)
{
success = ex.ToString();
return success;
}
success = "true";
return success;
}
Note, I have the application running in X86 mode to avoid errors with the ACE.OLEDB.12.0 adapter.

One thing that stands out, is you never close/dispose your SqlConnection. OleDbDataAdapter is also disposable and should be disposed. A 'using' statement is a convenient construct here:
public static string insertQuery(string sql)
{
string connetionString = #"Provider=Microsoft.ACE.OLEDB.12.0;Data Source=C:\Documents and Settings\Owner\Desktop\Arbeit\TrademarkParserproject1\TrademarkParserproject\bin\x86\Debug\Database.accdb";
using(var oledbAdapter = new OleDbDataAdapter())
using(var connection = new OleDbConnection(connetionString))
{
string success = "false";
try
{
connection.Open();
oledbAdapter.InsertCommand = new OleDbCommand(sql, connection);
oledbAdapter.InsertCommand.ExecuteNonQuery();
}
catch (Exception ex)
{
success = ex.ToString();
return success;
}
success = "true";
return success;
}
}