We Keep Coding

Postgres connection state stays active when an exception occurs

The state of the connection stays active in pool, when there is an exception while executing a query or stored procedure through c#.
The Npgsql version that I am using is 4.1.7.
Here is the code that I am trying to execute.
NpgsqlCommand cmd = null;
NpgsqlDataAdapter sda = null;
NpgsqlConnection conn = null;
try {
string sql = "a_test";
conn = new NpgsqlConnection("Server=localhost;Port=5432;Username=admin;Password=test;Database=majordb;SearchPath=dbs;CommandTimeout=300;MaxPoolSize=500;Connection Idle Lifetime=180;Connection Pruning Interval=5;");
cmd = new NpgsqlCommand(sql);
cmd.CommandType = CommandType.StoredProcedure;
sda = new NpgsqlDataAdapter(cmd);
cmd.Connection = conn;
sda.Fill(dataTable);
}
catch (Exception e) {
//log
}
finally {
if(null != sda)
{
try
{
sda.Dispose();
}
catch (Exception)
{
}
}
try
{
cmd.Connection.Close();
cmd.Connection.Dispose();
}
catch (Exception)
{
}
try
{
cmd.Dispose();
}
catch (Exception)
{
}
}
If the above code executes properly without any exception, the connection state in pool goes to idle, which is correct. But if an exception occurs while executing, like below:
"Npgsql.NpgsqlException (0x80004005): Exception while reading from stream --->
System.IO.IOException: Unable to read data from the transport connection: A connection attempt
failed because the connected party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond."
The connection state in pool shows as active for about 5 mins or so, even though the close/dispose methods are called in finally block. This means the close/dispose did not properly executed by Npgsql. If the program keeps the connection state in pool active for every connection ran within 5 mins, then there can be an issue with MaxPoolSize error.
I wanted to see the connection state to idle, even when there is an exception. How do I do this.
Please note: I am not looking for a solution to the exception that I listed above. I am looking for a solution where the connection state is changed to idle from active when there is an exception while executing the above code.
To know if the connection state is active or not I used the following query:
SELECT
pid,
usename,
application_name,
datname,
client_addr,
rank() over (partition by client_addr order by backend_start ASC) as rank,
state,
state_change,
current_timestamp,
query,
query_start,
backend_start,
FROM
pg_stat_activity
WHERE
pid <> pg_backend_pid( )
AND
application_name !~ '(?:psql)|(?:pgAdmin.+)'
AND
datname = current_database()
AND
usename = current_user
Any help is really appreciated.

C# SQLConnection.Open() hangs, with no exception

I am using C# in Visual Studio 2019, with Xamarin.Forms, and SQl in SSMS 2018 and have the below code (where [] is used to replace unneccessary information)
try
{
using(SqlConnection connection = new SqlConnection())
{
connection.ConnectionString = "Server=[ServerName]; Database=[DatabaseName]; User Id= [UserID]; Password=[Password]";
connection.Open();
SqlCommand command = new SqlCommand("SELECT * from [TableName]", connection);
[Does stuff here]
}
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine(ex)
}
When I run this, it hangs indefinitely at connection.Open(). Debug mode continues to run and appears to move on from Connection.Open(), but never reaches the next line.
I have attempted this with different versions of the ConnectionString, using different databases and with Trusted_Connection=true instead of specifiying the username and password but they have made no difference. Adding Connection Timeout = 5 to the connectionString has no effect.
I believe it is probably an issue with my settings in SQL but as I am a novice with this I have no idea where to start and the similar forums posts I have checked have been given answers along the lines of Connection Timeout (Connection.open for hangs indefinitely, no exception is thrown) or never got answered.
Any advice would be greatly appreciated.

Can you log into SSMS with the credentials that are in the connection string?
Otherwise I've had luck making sure the connection isn't already open or broken first:
if (connection.State == ConnectionState.Closed || connection.State == ConnectionState.Broken)
{
connection.Open();
}

Can you try code changing line as per below -
connection.ConnectionString = "Data Source=[ServerName]; Initial Catalog=[DatabaseName]; Integrated Security=SSPI;"

A workaround to this problem is to pass in a cancellation token instance as shown below,
public async Task<IEnumerable<Toy>> ShowToybox(CancellationToken t)
{
// notice I turned this into an async operation.
// Reason is to let this operation to find its way out if something happens
var connString = "Server=xyz; Connection Timeout=250";
//Timeout might not happen, if that is not the case see below..
using(SqlConnection connection = new SqlConnection(connString))
{
if ( t.IsCancellationRequested) {
t.ThrowIfCancellationRequested();
}
// await query here. To fetch records from the toybox table
return matches;
}
Main issue is that you cannot trust connection.State
To get this working, code that consumes this method should expect something might go wrong.
class Program
{
static void Main()
{
var box = new ToxBox(); //it has method shown above
var s = new CancellationTokenSource();
s.CancelAfter(400); //it prevents method from hanging
var task = Task.Run(() = box.ShowToybox(s.Token));
try
{
task.Wait(s.Token);
var myToys = task.Result();
Console.WriteLine($"I have {myToys.Count()} toys");
}
catch (Exception e)
{
Console.WriteLine("Something happened!");
}
finally
{
s.Dispose(); //this is important
}
}
}
See https://learn.microsoft.com/en-us/dotnet/standard/threading/cancellation-in-managed-threads

SqlCommand timeout in C# MMO application

I have a C# project that is working with TCP socket in an asynchronous way.
Every request comes from client and ask question from SQL Server stored procedure, opens and closes a SQL connection after ending of question.
I've used this code:
using (var con = new SqlConnection(setting.ConnectionString))
{
try
{
//some codes (edited)
SqlCommand command = new SqlCommand(con);
command.CommandText = "procedurename1";
command.CommandType = CommandType.StoredProcedure;
command.Parameters.Add(new SqlParameter("#name", sb.ToString()));
SqlDataAdapter adapter = new SqlDataAdapter(command);
try
{
adapter.Fill(dataSet);
}
catch (Exception ex)
{
con.Close();
con.Dispose();
throw ex;
}
finally {
con.Close();
con.Dispose();
}
}
catch(Exception ex)
{}
finally
{
con.close();
con.dispose();
}
}
I've used
netstat -a -n | find /c "1433"
to count SQL connections open and close.
Problem is SQL connections count increases and it rarely decreases and count down.
Main problem, is when my program works under lots of requests about 30 minutes, I get
SqlCommand timeout error (default 30 seconds passed)
and after restarting my C# program, the SqlCommand timeout will be gone.
Is this a problem of my program or SQL Server side?
Remember it always calls a stored procedure in SQL Server, not executing query
directly.
main method:
public void main()
{
Task.Factory.StartNew(() =>
{
allDone.Reset();
mySocket.AcceptAsync(e);
allDone.WaitOne();
});
}
public void e_Completed(object sender, SocketAsyncEventArgs e)
{
var socket = (Socket)sender;
ThreadPool.QueueUserWorkItem(HandleTcpRequest, e.AcceptSocket);
e.AcceptSocket = null;
socket.AcceptAsync(e);
}
public void HandleTcpRequest(object state)
{
//do some code and connection to SQL server
DLL.Request httprequest = new DLL.Request(dataSet.Tables[0], fileDt);
DLL.IHttpContext _context = new DLL.HttpContext(httprequest);
_context.GetResults();
}

Main problem, is when my program works under lots of requests about 30 minutes,
To isolate the root problem of the time-out, I suggest testing the sql query of the stored procedure independent of TCP socket calls for 30 minutes
and log the time-out exception details for inspection
Run the following query within 30 minutes to simulate your working environment:
public void RunQuery()
{
using (var con = new SqlConnection(setting.ConnectionString))
{
try
{
//some codes
}
catch(SqlException ex)
{
//test for timeout
if (ex.Number == -2) {
Console.WriteLine ("Timeout occurred");
// log ex details for more inspection
}
}
}
}
Read How to handle the CommandTimeout properly?
As you use async calls, I suggest you to try to use Asynchronous Database Calls With Task-based Asynchronous Programming Model (TAP)

I'm going to take a long-shot based on the way the limited Sql-related code we can see is written since we can't see "//some codes".
I'm going to guess that some of the disposable things like SqlCommand, DataReader, SqlDataAdapter, TransactionScope, etc are not in 'using' blocks, so are holding resources open on the database.
It may also be worth raising the possibility that this kind of problem could be in the code shown in the question or any other program accessing that database, including your own applications and SSMS (e.g. if a developer has an uncommitted transaction running in a window).
P.S. I would suggest deleting everything in the using block except the "//some codes" part.
UPDATE after more code was added
Here is your code after correction. this will ensure that the resources are disposed, which will prevent the leaking resources that are probably causing your problem.
using (var con = new SqlConnection(setting.ConnectionString))
{
//some codes (edited)
using (SqlCommand command = new SqlCommand(con))
{
command.CommandText = "procedurename1";
command.CommandType = CommandType.StoredProcedure;
command.Parameters.Add(new SqlParameter("#name", sb.ToString()));
using (var adapter = new SqlDataAdapter(command))
{
adapter.Fill(dataSet);
}
}
}
P.S. don't ever write "throw ex;" from inside a catch ever again. It causes the stack trace to be lost - just use "throw;".

.NET Core wont close my MySqlConnection

I have a .NET Core program that uses the MySqlConnection class. My Database is a ClearDB Database that is stored in Azure.
When I launch the program it is working like it should. But when I wait for like 10 minuts doing nothing, it wont connect to the database anymore(Timeout?). Restarting the program and it works again.
When looking at the connections on the ClearDB webpage it isn't closing when I close it in my program. After 10 minuts or so it closes automaticly, as I see in ClearDB webpage. But with the program still running it wont connect to the database anymore. Restarting program is only solution.
Code for now looks something like this:
private static async Task<uint> getDeviceId(string macAddress)
{
using (var connection = new MySqlConnection(ConnectionString))
{
uint returnvalue = 0;
var cmd = connection.CreateCommand() as MySqlCommand;
cmd.CommandText = #"SELECT id FROM devices WHERE mac = '" + macAddress + "'";
connection.Open();
Console.WriteLine(connection.State);
DbDataReader reader = await cmd.ExecuteReaderAsync();
using (reader)
{
while (await reader.ReadAsync())
{
returnvalue = await reader.GetFieldValueAsync<uint>(0);
}
}
reader.Dispose();
cmd.Dispose();
return returnvalue;
}
}
I have tried the following:
Using statement
Close/dispose connection,reader and command
Pooling=false in connectionstring
But none of them works. Somebody got an idea?

Assuming MySql provider is like the MSSQL provider, it does not actually close the connection in the database, it just releases it back to the pool.
You do not want to disable pooling, you will kill efficiency.
This is by design, and what you want.

The using statement from the code snippet should close your connections. However, I'm not sure how that interacts with async, or how ClearDB differs from normal MySql. Given the issues in the question and that lack of clarity, you might try this, just to see if it helps:
private static async Task<uint> getDeviceId(string macAddress)
{
uint returnvalue = 0;
MySqlConnection connection;
try
{
connection = new MySqlConnection(ConnectionString);
var cmd = connection.CreateCommand() as MySqlCommand;
//Don't EVER(!) use string concatenation like that in a query!
cmd.CommandText = #"SELECT id FROM devices WHERE mac = #macAddress";
cmd.Parameters.Add("#macAddress", MySqlDbType.VarChar, 18).Value = macAddress;
connection.Open();
Console.WriteLine(connection.State);
DbDataReader reader = await cmd.ExecuteReaderAsync();
using (reader)
{
while (await reader.ReadAsync())
{
returnvalue = await reader.GetFieldValueAsync<uint>(0);
}
}
reader.Dispose();
cmd.Dispose();
}
finally
{
connection.Close();
connection.Dispose();
}
return returnvalue;
}
A using block basically just re-writes your code as try/finally anyway, so doing this step by-hand can sometimes make debugging easier (you can log where it hits the .Close() call).
If this does resolve the problem, I wouldn't stop there, but rather start from there and see just how close to "normal" code you can get. I'm also concerned here that you have disabled connection pooling, and that this method is static.

C# SqlCommand Returning Input Parameters when Inserting multiple Records

I have an .exe project with 4 threads. Each thread makes a call to a WCF service hosted in a Windows Service and inserts a record (loop from 1 to 5,000 records). The test project will try to insert 20,000 records into the WCF service. The service behavior in the WCF service is per session.
I use a stored procedure to insert the records into SQL Server 2008R2 Express. The problem I'm having is with the SqlCommand. When only one thread is running, no error happens, but when two or more threads are running, the code throws an error, but not sure about the error type.
If you look at the code below, the error is raised when reading the result from the .ExecuteReader (it's a cast exception error). It does not return the errors that I have defined in the stored procedure (I'm guessing it never gets to the database), it returns an XML with all the parameters of the txn record, but it does not return only the current transaction, it also return record from transactions running on a different thread. If I execute the stored procedure directly in SQL Server Management Studio, it works fine, so I discarded any isolation level issue at the database side.
As you can see the method is not static, the SqlCommand is created and disposed on each call, so I'm really concerned about this. Any ideas?
private InsertInvoiceDataTable SaveTransaction(Transaction Trans, ClientInfo InfoCliente)
{
InsertInvoiceDataTable returnData = new InsertInvoiceDataTable();
try
{
using (SqlConnection con = new SqlConnection(ConnStr1)
{
using (SqlCommand cmd = new SqlCommand("InsertInvoice", con))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.Add("#TIPO", SqlDbType.Int).Value = Trans.InvoiceType;
cmd.Parameters.Add("#CAJERO", SqlDbType.VarChar).Value = Trans.Cashier;
cmd.Parameters.Add("#TERMID", SqlDbType.Int).Value = Trans.Term;
cmd.Parameters.Add("#DOB", SqlDbType.DateTime).Value = Trans.DOB;
cmd.Parameters.Add("#CLIID", SqlDbType.VarChar).Value = InfoCliente.ClientId;
cmd.Parameters.Add("#VENTANETA", SqlDbType.Decimal).Value = Convert.ToDecimal(Trans.SubTotal);
cmd.Parameters.Add("#IMPUESTO", SqlDbType.Decimal).Value = Convert.ToDecimal(Trans.TaxTotal);
cmd.Parameters.Add("#VENTATOTAL", SqlDbType.Decimal).Value = Convert.ToDecimal(Trans.Total);
con.Open();
using (SqlDataReader results = cmd.ExecuteReader())
{
while (results.Read())
{
InsertInvoiceRow row = returnData.NewInsertInvoiceRow();
try
{
row.TIPO_log = results["Type_log"].ToString();
row.VALOR_LOG = results["Value_log"].ToString();
}
catch (Exception ex)
{
returnData.AddInsertInvoiceRow("ERROR", ex.Message);
break;
}
returnData.AddInsertInvoiceRow(row);
}
}
con.Close();
cmd.Dispose();
}
}
}
catch (Exception ex)
{
Log.Error(ex);
returnData.AddInsertInvoiceRow("ERROR", ex.Message);
}
return returnData;
}

You are performing a DML operation, in your case INSERT (from your posted code new SqlCommand("InsertInvoice", con)) then why ExecuteReader() it rather should be cmd.ExecuteNonQuery()