We Keep Coding

C# Connection timeout to oracle database when application pool recycles

We have a web page which uses ODP.Net to connect to oracle database from ASP MVC page. During the heavy user load if application pool recycles for any reason, than we get few connection timeout errors from internal oracle class.
Inner Exception: StackTrace: at OracleInternal.ConnectionPool.PoolManager`3.Get(ConnectionString csWithDiffOrNewPwd, Boolean bGetForApp, String affinityInstanceName, Boolean bForceMatch)
How can we handle this scenario ? what could be the reason for this error ?

If you are using a new connection string that differs per user or with some other variable parameter you can exceed your max connections. Pooling only works if you use the exact same string. Under the covers your "close" is ignored and the connection manager keeps the connection open. In SQL Server you can see a connection reset in the trace console every time an existing connection is reused.
Oracle probably has something similar.
Tracing your connections and possibly connection resets will most likely shed some light on what is creating so many connections.

Website loses connection to database sporadically

I've been struggling with this error for just over a month now and can't get it figured out.
Background/Architecture:
MVC 4 application
NHibernate
FluentNHibernate
SQL Server 2008 R2
Azure website
Sporadically throughout the day, my views are throwing errors because when they try to read/display properties on my objects, it says:
NHibernate.Exceptions.GenericADOException: could not initialize a
collection: [WrestleStat.BL.School.Schedule#54][SQL: SELECT
...(removed to keep this short)....] --->
System.Data.SqlClient.SqlException: Timeout expired. The timeout
period elapsed prior to completion of the operation or the server is
not responding. This failure occurred while attempting to connect to
the routing destination. The duration spent while attempting to
connect to the original server was - [Pre-Login] initialization=3;
handshake=23; [Login] initialization=0; authentication=0; [Post-Login]
complete=1; ---> System.ComponentModel.Win32Exception: The wait
operation timed out
--- End of inner exception stack trace --- at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException
exception, Boolean breakConnection, Action`1 wrapCloseInAction) at
System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject
stateObj, Boolean callerHasConnectionLock, Boolean asyncClose) at
System.Data.SqlClient.TdsParserStateObject.ReadSniError(TdsParserStateObject
stateObj, UInt32 error) at
System.Data.SqlClient.TdsParserStateObject.ReadSniSyncOverAsync() at
System.Data.SqlClient.TdsParserStateObject.TryReadNetworkPacket() at
System.Data.SqlClient.TdsParserStateObject.TryPrepareBuffer() at
System.Data.SqlClient.TdsParserStateObject.TryReadByteArray(Byte[]
buff, Int32 offset, Int32 len, Int32& totalRead) at
System.Data.SqlClient.TdsParserStateObjec
2 weeks ago, I was running my website on AppHarbor, and I thought it was because I was on their "basic" (free) plan. The problem seemed to go away if I'd up my worker threads from 1 to 2. Because of that, I decided to switch over to Azure (I was going to anyway, but not this early). Since the switchover to Azure, the problem has not gone away.
I get approximately a dozen of these errors per day on the site. It appears as though that the errors always occur when the razor views are trying to access properties of an object that it's trying to read/display, as opposed to throwing the error inside the controller trying to retrieve the data. Maybe it has something to do with the lazy loading??? But why would that matter, why would the connection be lost in that extremely short timeframe from retrieving the data, then presenting it on the page?
I'm at a loss on how to debug this issue, because it obviously never happens in my development environment. I'm going to switch over to Dapper eventually, but I'm not able to do/start that for at least another 3 months.
Any ideas here?
Edit: here's the website http://www.wrestlestats.com

These are transient errors normal in a cloud environment. You need to implement a retry logic/policy.
Check this answer: How do I add Retry Logic in NHibernate to handle Transient Failures in SQL Azure?

All of a sudden getting lots of Wait Operation Time out issues on SQL Azure

Two days ago, with no code changes or changes to the DB, I am not getting a lot (every 5 minutes or so) of errors with The wait operation timed out error with two different underlining full errors on about the pre-login and the other about the post:
System.Data.Entity.Core.EntityException: The underlying provider failed on Open. ---> System.Data.SqlClient.SqlException: Connection Timeout Expired. The timeout period elapsed while attempting to consume the pre-login handshake acknowledgement. This could be because the pre-login handshake failed or the server was unable to respond back in time. The duration spent while attempting to connect to this server was - [Pre-Login] initialization=21; handshake=14988; ---> System.ComponentModel.Win32Exception: The wait operation timed out
System.Data.Entity.Core.EntityException: The underlying provider failed on Open. ---> System.Data.SqlClient.SqlException: Connection Timeout Expired. The timeout period elapsed during the post-login phase. The connection could have timed out while waiting for server to complete the login process and respond; Or it could have timed out while attempting to create multiple active connections. This failure occurred while attempting to connect to the routing destination. The duration spent while attempting to connect to the original server was - [Pre-Login] initialization=5; handshake=3098; [Login] initialization=0; authentication=0; [Post-Login] complete=7; The duration spent while attempting to connect to this server was - [Pre-Login] initialization=20; handshake=5; [Login] initialization=0; authentication=0; [Post-Login] complete=11003; ---> System.ComponentModel.Win32Exception: The wait operation timed out
I am using Entity Framework and my web site is hosted on an Azure Web App. I have done some digging and most SO questions I find about this are NOT related to Entity Framework but ADO.Net the few posts I found lead me updated from a Basic to Standard (S0) service for the DB and creating a GlobalDBConfig with
public class GlobalDBConfig : DbConfiguration
{
public GlobalDBConfig()
{
SetExecutionStrategy("System.Data.SqlClient", () => new SqlAzureExecutionStrategy(2, TimeSpan.FromSeconds(30)));
}
}
How can I figure out what else is going wrong and fix it? This is a very simple DB with simple queries and very little traffic to the site (less then 1000 visits a DAY)

We resolved this issue, along with other types of random timeouts on SQL Azure by switching to "contained users". Using server-level logins on SQL Azure can cause issues:
This is not very efficient as in SQL DB master and user can sit on
two different SQL servers potentially in two different machines. Also
when a server has multiple user databases then master will be the
bottleneck in the login process, and under load this may result in
high response time for logins. If Microsoft is updating the software
on the machine / server then master will be unavailable for a few
seconds and all the logins to the user database can fail too at this
time (http://www.sqlindepth.com/contained-users-in-sql-azure-db-v12/)
As in your case, I had my doubts because my database was not under heavy load, but switching to contained users made a tremendous difference anyway.
The SQL to create these users is as follows (run this on the database itself, not on the master database as you would for creating server-level logins):
Create user ContainedUser with password = 'Password'
ALTER AUTHORIZATION ON SCHEMA::[db_owner] TO [ContainedUser]
ALTER ROLE [db_owner] ADD MEMBER [ContainedUser]

Here are a few options to try: I strongly recommend going with (1) and (3) if possible
User database firewall rules and contained user authentication
Increase connection timeout to a large value (60-120 seconds?)
If possible update your client drivers to latest version (7.4 and above)

We had similar problems and please note that there is no such thing as AUTO scaling on standalone databases on Azure and since you are using Entity Framework, here are some suggestions below
If you are calling Web-API to fetch and transact with your database on Azure SQL, make sure on Azure portal you set the "ALLWAYS ON" option for the Web-API.
Then your client app should probably retry if it fails to connect in the first attempt.
If the database queries are resulting in timeouts due to volume of the data and the indexes not able to catch up with that, you will need to increase the time out of the command executions a bit and most importantly you will need to update the stats on the database and recompile all the objects in the database.

How to get rid of SqlException Timeout expired in WCF Service?

I'm working on asp.net web application, this application interacts with sql server Database through WCF Service. I get the following Exception sometimes but not every time.
System.Data.SqlClient.SqlException (0x80131904): Timeout expired.
The timeout period elapsed prior to completion of the operation or the server is not responding.
This failure occured while attempting to connect to the Principle server.
---> System.ComponentModel.Win32Exception (0x80004005):
The wait operation timed out
I know there are multiple reasons for this. There's a deadlock somewhere or The query is too complex.
At some place I have read that setting CommandTimeout of SqlCommand object can solve this issue.
So before applying this logic I want to know the strong reason of this issue and If there is any other workaround to be done to get rid of this issue please let me know. If I'm missing anything please let me know.

SQLConnection Leakage Issue (Timeout Expired due to Pooling - ASP.NET / SQL Server)

I am developing a web application which is based on ASP.NET 4.0, jQuery, AJAX, Javascript using SQL Server 2008 database. Our application is an ERP application which has almost around 400 to 500 users using the application. We have made it live since a month ago.
Now, I'm facing a timeout issue and application crashed literally which is occurred after some hours of application usage.
Server Error in '/' Application.
--------------------------------------------------------------------------------
Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
Source Error:
Line 36: param[1].Value = Password;
Line 37:
Line 38: return SqlHelper.ExecuteDataset(CDSGlobal.ConnectionString, CommandType.StoredProcedure, "ValidateLogin_sp", param);
Line 39: }
Line 40:
Source File: e:\abcd\App_Code\User.cs Line: 38
Stack Trace:
[InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.]
System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection) +6352273
System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory) +6352606
System.Data.SqlClient.SqlConnection.Open() +300
DBConn.SqlHelper.ExecuteDataset(String connectionString, CommandType commandType, String commandText, SqlParameter[] commandParameters) +270
User.ValidateLogin(String Employee_Code, String Password) in e:\abcd\App_Code \User.cs:38
User_UserLogin.btnLogin_Click(Object sender, EventArgs e) in e:\abcd \User\UserLogin.ascx.cs:79
System.Web.UI.WebControls.Button.RaisePostBackEvent(String eventArgument) +154
System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +3707
--------------------------------------------------------------------------------
Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.0.30319.272
For temporary resolution, we are restarting the server and restarting the MSSQLSERVER service.
After searching for above, I came to know it is due to connection leakage. Some opened connections are never get closed.
I am using ready made Microsoft Data Access Application Block for .NET i.e. SQLHelper Class which in my application for data access using Dataset and all.
Note: I am NOT using SqlDataReader anywhere.
I have examined the class and its closing the connections and I have NEVER used manual connection code instead used SQLHelper everywhere.
I also have the following nested transactions structure in stored procedures as follows:
BEGIN TRY
BEGIN TRANSACTION
-- Every below stored procedure has the transation structure as this SP i.e. Parent SP. (BEGIN TRY, BEGIN TRANSACTION etc)
exec sp1
exec sp2
exec sp3
---
---
COMMIT TRANSACTION
END TRY
BEGIN CATCH
IF ##TRANCOUNT>0
ROLLBACK TRANSACTION
PRINT 'ERROR OCCURED In SP'
END CATCH
Now, The problem is why connection leakage is occurring.
Is above transaction structure is responsible i.e it is blocking the transactions and requests.
If request is in process and user has closed the browser, does the application closes the connection?
Any help will be greatly appreciated.

Solved !!
It was not due to Nested Transactions but it was due to Bad Indexes Strategy used. Used normal Indexes and Indexes with include to solve the performance issues.
Also ran SQL Profiler to examine the most read and writes on columns and accordingly applied indexes and review maintenance and execution plans. !!

You can also use this,
SqlConnection.ClearAllPools();
The above code then closes all of the pools upon execution. I had the same problem and cleared the pools each time and then it should work successfully due to the pools being cleared.