We Keep Coding

Postgresql and .Net - Connection Pooling with Multiple Endpoints

Basic Intro:
Connection string: Host=IP;Port=somePort;Username=someUser;Password=somePass;Database=someDb;Maximum Pool Size=100
My web application has several dozen endpoints available via WS and HTTP. Every one of these endpoints opens a new NPGSQL connection (all using the same connection string as above), processes data, then closes via the using statement.
Issue:
When the application restarts for an update, there is typically 2-3,000 users all reconnecting. This typically leads to errors regarding the connection pool being full and new connections being rejected due to too many clients already. However, once it can finally come online it typically only uses between 5-10 connections at any given time.
Question:
Is the logic below the proper way to use connection pooling? With every endpoint creating a new NPGSQL connection using the same connection string specifying a connection pool of 100?
It seems that the connection pool often shoots right up to 100, but ~80/100 of those connections are shown as idle in a DB viewer with new connection requests being denied due to pool overflow.
Better option?
I could also try and force a more "graceful" startup by slowly allowing new users to re-connect, but I'm not sure if the logic for creating a new connection with every endpoint is correct.
// DB Connection String - Used for all NPGSQL connections
const string connectionStr "Host=IP;Port=somePort;Username=someUser;Password=somePass;Database=someDb;Maximum Pool Size=100";
// Endpoint 1 available via Websocket
public async Task someRequest(someClass someArg)
{
/* Create a new SQL connection for this user's request using the same global connections string */
using var conn = new NpgsqlConnection(connectionStr);
conn.Open();
/* Call functions and pass this SQL connection for any queries to process this user request */
somefunction(conn, someArg);
anotherFunction(conn, someArg);
/* Request processing is done */
/* conn is closed automatically by the "using" statement above */
}
// Endpoint 2 available via Websocket
public async Task someOtherRequest(someClass someArg)
{
/* Create a new SQL connection for this user's request using the same global connections string */
using var conn = new NpgsqlConnection(connectionStr);
conn.Open();
/* Call functions and pass this SQL connection for any queries to process this user request */
somefunction(conn, someArg);
anotherFunction(conn, someArg);
/* Request processing is done */
/* conn is closed automatically by the "using" statement above */
}
// endpoint3();
// endpoint4();
// endpoint5();
// endpoint6();
// etc.
EDIT:
I've made the change suggested, by closing connections and sending them back to the pool during processing. However, the issue still persists on startup.
Application startup - 100 connections claimed for pooling. Almost all of them are idle. Application receives connection pool exhaustion errors, little to no transactions are even processed.
Transactions suddenly start churning, not sure why? Is this after some sort of timeout perhaps? I know there was some sort of 300 second timeout default in documentation somewhere... this might match up here.
Transactions lock up again, pool exhaustion errors resume.
Transactions spike and resume, user requests start coming through again.
Application levels out as normal.
EDIT 2:
This startup issue seems to consistently be taking 5 minutes from startup to clear a deadlock of idle transactions and start running all of the queries.
I know 5 minutes is the default value for idle_in_transaction_session_timeout. However, I tried running SET SESSION idle_in_transaction_session_timeout = '30s'; and 10s during the startup this time and it didn't seem to impact it at all.
I'm not sure why those 100 pooled connections would be stuck in idle like that on startup, taking 5 minutes to clear and allow queries to run if that's the case...

I had forgotten to update this post with some of the latest information. There was a few other internal optimizations I had made in the code.
One of the major ones, was simply changing conn.Open(); to await conn.OpenAsync(); and conn.Close(); to conn.CloseAsync();.
Everything else I had was properly async, but there was still IO blocking for all of the new connections in NPGSQL, causing worse performance with large bursts.
A very obvious change, but I didn't even think to look for an async method for the connections opening and closing.

A connection is released to the pool once you close it in your code. From what you wrote, you are keeping it open for the entire time of a request, so basically 1 user = 1 connection and pooling is just used as a waiting room (timeout setting, 15 seconds by default). Open/Close the connection each time you need to access the DB, so the connection is returned to the pool and can be used by another user when time is spent in .net code.
Example, in pseudo code:
Enter function
Do some computations in .net, like input validation
Open connection (grab it from the pool)
Fetch info#1
Close connection (return it to the pool)
Do some computations in .net, like ordering the result, computing an age from a date etc
Open connection (grab it from the pool)
Fetch info #2
Close connection (return it to the pool)
Do some computations in .net
Return

Oracle Error: Pooled connection request timed out

Im using Oracle12c with the application written in C# and using Oracle.ManagedDataAccess.dll to handle the DB Connection.
A product we have has started to occasionally throw this exception after running fine for years:
Oracle.ManagedDataAccess.Client.OracleException (0xFFFFFC0C): Pooled connection request timed out
at OracleInternal.ConnectionPool.PoolManager`3.Get(ConnectionString csWithDiffOrNewPwd, Boolean bGetForApp, String affinityInstanceName, Boolean bForceMatch)
at OracleInternal.ConnectionPool.OraclePoolManager.Get(ConnectionString csWithNewPassword, Boolean bGetForApp, String affinityInstanceName, Boolean bForceMatch)
at OracleInternal.ConnectionPool.OracleConnectionDispenser`3.Get(ConnectionString cs, PM conPM, ConnectionString pmCS, SecureString securedPassword, SecureString securedProxyPassword)
at Oracle.ManagedDataAccess.Client.OracleConnection.Open()
I know the cause of this error. Looking at the code neither the OracleConnection or the OracleCommand objects are being disposed. So these connections are building up until it eventually throws this exception.
The fix is straight forward. Wrap these in Using statements. I dont need help with that.
However my interests me is why this problem has started now. This software was running for years without issue. They did some database maintenance, updated other software on the same server, then this problem started. I dont know what Db maintenance they did.
The connection string in the application does not specify any pool attributes.
Is there a oracle db setting which would cause a lower amount of simultaneous connections in the database that could have caused this to start occurring?
UPATE
I wrote a little test app to check the limit.
It just loops around and opens a connection, performs a basic query and doesnt dispose the connection.
On my test system it starts throwing this exception after 640ish loops. It varies give or take 10 loops each time i run it.
What is setting this limit?

I just had the same problem.
The reason, why you get that exception is, that the Oracle Pool Manager doesn't have a free connection anymore (per default you can have up to 100 connections). Often the reason for that problem are not closed connections (so 'using' was the right ways).
Even though you already found a solid solutions for that problem, I want to add those:
Try/catch and using is always a good idea
Adjust the Min Pool Size (or Max Pool Size, but default is 100 should be enough): So there's always a connection ready to use and the connection doesn't have to get established (see https://docs.oracle.com/en/database/oracle/oracle-database/21/jjucp/optimizing-ucp-behavior.html#GUID-FFCAB66D-45B3-4D7B-991B-40F1480630FD)
Update to Oracle.ManagedDataAcess >= 21.8.0 (
Bug 34322469 - CONNECTION POOL THROWS "CONNECTION REQUEST TIMED OUT" EXCEPTION DUE TO LOOPING WITHIN POOLMANAGER.GET() is fixed there)

sql timeout expired

Logging:System.Data.SqlClient.SqlException: Timeout expired. The
timeout period elapsed prior to completion of the operation or the
server is not responding.
i am a beginner when i saw in the application log files the above is the most frequent error i saw and also it is getting repeated everyday. on the database when i saw time taken for executing the particular procedure which the above function is calling is less than 5 secs.
But in the application we gave connection timeout=200s and by default command timeout=30 secs our manager says we don't have to increase the command timeout by anymore further as it is true. But still the exception is keep coming.
can anyone suggest me any solution so i can get rid of the above problem thanks

The setting in the web config, if it's the timeout in the connection string setting, is the connection timeout. It only applies to the time it takes to make a connection. From your problem description, it doesn't sound like a connection timeout is what's happening.
Command timeouts are specified in other ways. If you are using DataContext, for example, the timeout is set using the CommandTimeout property.
http://msdn.microsoft.com/en-us/library/system.data.linq.datacontext.commandtimeout.aspx
If you can give a code snippet of how you are hitting the database so we can see what classes you are using, more specific recommendations can be made.

How to get better control over connection pool?

I've been getting this error recently, after several reloads of the same page:
System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached
So I am thinking there must be some queries or calls in the app I used incorrectly that causes them not to release the connection. Is there any tools out there that allows me to somehow peek into the pool to see who is hanging on to what?

There's a timeout property on your connection object that you can change. This will change the time it waits to get a connection, there's also a command timeout which controls how long it waits until the command times out once it is running (but the first one sounds like what you need) see here (anything that inherits from DBConnection should have this if you arn't using sql server).
Have a look here too, might help :)

MySql.Data.MySqlClient.MySqlException: Timeout expired

In recent times, a particular page in my web app throws the
Exception Details: MySql.Data.MySqlClient.MySqlException: Timeout
expired. The timeout period elapsed prior to completion of the
operation or the server is not responding.
Though I use Ibtais as persistence layer, this error occurs. I have restarted the MySql service instance but stil i get the same error. It didn't happen earlier but happens frequently in recent times.
All the web applications deployed on the server uses Ibatis and the DB server remains on the same machine where IIS is installed. There are about 8000 records in which around 300 to 500 would be filtered on page load
Any insights for the cause of the problem?

I encountered the same problem with yours, and I found this MySQLConnection--Specifying default command timeout. Just add "default command timeout=xxx" into your connectString, this key's value is in seconds.
I tried and it worked for me.

You could set command timeout to 0, its not a good idea though. Some requests could go on indefinitely.
There is an underlying problem that is causing the queries to time out in the first place. Are you inserting, updating, or in any way working with large binary values that would lock the table? That is the most common reason I see for an error like this on such a small amount of data.

Make your connection string look like this:
DotNet Framework:
<add key="MYSQL_CONNECTION_STRING_RDS" value="Uid=userid;Password=pass;
Server=localhost;Port=3306;
Database=dbname;default command timeout=0;SslMode=none"
/>
DotNet Core
"MYSQL_CONNECTION_STRING_RDS": {
"ConnectionString": "Server=localhost;Database=dbname;user=userid;password=pass;default command timeout=0;",
"ServerVersion": "8.0.22"
}