We Keep Coding

AWSSDK .net error while copying. I think its a time syncing issue

So I've written a sync function using the low level sdk calls and I'm getting errors sometimes. not always, it seems to be after its been syncing for aroun 20 or 30 mins. I'm uploading about 30gb of data so it takes a while. If i do it in a linear fashion its okay, but I'm running it in a big Parallel.For loop. Anyway, here is the error:
`
Amazon.S3.AmazonS3Exception: The difference between the request time and the current time is too large. ---> Amazon.Runtime.Internal.HttpErrorResponseException: The remote server returned an error: (403) Forbidden. ---> System.Net.WebException: The remote server returned an error: (403) Forbidden.
at System.Net.HttpWebRequest.EndGetResponse (System.IAsyncResult asyncResult) [0x00058] in <ef151b6abb5d474cb2c1cb8906a8b5a4>:0
at System.Threading.Tasks.TaskFactory`1[TResult].FromAsyncCoreLogic (System.IAsyncResult iar, System.Func`2[T,TResult] endFunction, System.Action`1[T] endAction, System.Threading.Tasks.Task`1[TResult] promise, System.Boolean requiresSynchronization) [0x0000f] in <eae584ce26bc40229c1b1aa476bfa589>:0
`
I thought at first that perhaps some of the files were timing out or something but from reading into the error it looks like its a syncing issue. I found this documentation: https://docs.aws.amazon.com/AWSEC2/latest/WindowsGuide/windows-set-time.html#windows-configuring-ntp which gives advice on how to configure NTP on widows using w32tm. I have no idea what any of this means but it seems to be a way to keep the time in sync which I would assume fixes my error. I've followed the configuration step w32tm /config /manualpeerlist:169.254.169.123 /syncfromflags:manual /update and started Windows Time in the Services window.
I'm still getting the error when uploading, and when I run w32tm /query /sourceI get this response:
"Local CMOS Clock".
Help I'm very confuse

What can cause "Cannot import the transaction" errors when receiving from MSMQ?

The other day I received these errors trying to receive messages from transactional, public queues.
System.Messaging.MessageQueueException (0x80004005): Cannot import the transaction.
at System.Messaging.MessageQueue.ReceiveCurrent(TimeSpan timeout, Int32 action, CursorHandle cursor, MessagePropertyFilter filter, MessageQueueTransaction internalTransaction, MessageQueueTransactionType transactionType)
at System.Messaging.MessageQueue.Receive(TimeSpan timeout, MessageQueueTransactionType transactionType)
This is an application that has been running for years, 24/7, and this is the first time the error occurred. The app is configured to shut itself down after a couple of repeated errors like this, but when it was restarted some 15 minutes later, everything worked fine again.
Fwiw, the app is receiving from 3 different queues, on a per-queue dedicated thread. The error occurred on all 3 threads, and between every error (retrying without pausing in-between), there was an interval of between 1 and 2 seconds, and this error state went on for about 6 seconds, before the app shut itself down.
I do not expect a distributed transaction to occur in this app, but I do not explicitly set TransactionScopeOption to RequiresNew either.
I cannot seem to find a detailed explanation of that error and would like to know how to get to the bottom of this temporary glitch.
Does this error only occur for transactions involving DTC? Or could it occur for internal transactions as well?

Azure SQL Timeout

I have a Azure SQL db that powers my webapp.
There is a Web API 2.0 sitting in a virtual directory.
My web app does some lightweight db stuff, the API is responsible for entering data in and the webapp part is for users to log in and view their tracked email data.
Pretty simple stuff.
Now the problem seems to be the db. I keep getting these pesky Sql Timeout errors.
The weird thing is, it's the same error all throughout. I should mention that the webapi is built using NHibernate. The web portal is mainly Entity Framework using Code First.
I've contacted some SQL "experts" from Microsoft, and the only helpful information that they've provided me is that they've pointed out every time I get these intermittent timeout erros, the "writelog" has spiked up immediately.
Now this presents a problem because I do not know how to diagnose this issue.
I'm pasting the "generic" stack trace so I can illicit some helpful info from anyone on here whose a SQL Azure expert.
Stack Trace:
Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding. This failure occurred while attempting to connect to the routing destination. The duration spent while attempting to connect to the original server was - [Pre-Login] initialization=2; handshake=9; [Login] initialization=0; authentication=0; [Post-Login] complete=1; ---> System.ComponentModel.Win32Exception (0x80004005): The wait operation timed out at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action 1 wrapCloseInAction) at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)...
`

Entity Framework supports configurable retry strategies depending on how servers are deployed. As you're deploying to Azure, you should configure it to use the SQL Azure strategy by including this code:
public class MyConfiguration : DbConfiguration
{
public MyConfiguration()
{
SetExecutionStrategy("System.Data.SqlClient", () => new SqlAzureExecutionStrategy());
}
}
As documented on MSDN: https://msdn.microsoft.com/en-us/data/dn456835.aspx?f=255&MSPPError=-2147217396.
Timeout errors are generally considered a transient error in the SqlAzureExecutionStrategy. You can see the source code here: https://entityframework.codeplex.com/SourceControl/latest#src/EntityFramework.SqlServer/SqlAzureRetriableExceptionDetector.cs
If the timeout errors you are seeing are not handled by default by the SqlAzureExecutionStrategy, you can create your own by subclassing SqlAzureExecutionStrategy and overriding ShouldRetryOn. This blog post has a nice walkthrough of doing just that.

SQLConnection Leakage Issue (Timeout Expired due to Pooling - ASP.NET / SQL Server)

I am developing a web application which is based on ASP.NET 4.0, jQuery, AJAX, Javascript using SQL Server 2008 database. Our application is an ERP application which has almost around 400 to 500 users using the application. We have made it live since a month ago.
Now, I'm facing a timeout issue and application crashed literally which is occurred after some hours of application usage.
Server Error in '/' Application.
--------------------------------------------------------------------------------
Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.
Exception Details: System.InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.
Source Error:
Line 36: param[1].Value = Password;
Line 37:
Line 38: return SqlHelper.ExecuteDataset(CDSGlobal.ConnectionString, CommandType.StoredProcedure, "ValidateLogin_sp", param);
Line 39: }
Line 40:
Source File: e:\abcd\App_Code\User.cs Line: 38
Stack Trace:
[InvalidOperationException: Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool. This may have occurred because all pooled connections were in use and max pool size was reached.]
System.Data.ProviderBase.DbConnectionFactory.GetConnection(DbConnection owningConnection) +6352273
System.Data.ProviderBase.DbConnectionClosed.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory) +6352606
System.Data.SqlClient.SqlConnection.Open() +300
DBConn.SqlHelper.ExecuteDataset(String connectionString, CommandType commandType, String commandText, SqlParameter[] commandParameters) +270
User.ValidateLogin(String Employee_Code, String Password) in e:\abcd\App_Code \User.cs:38
User_UserLogin.btnLogin_Click(Object sender, EventArgs e) in e:\abcd \User\UserLogin.ascx.cs:79
System.Web.UI.WebControls.Button.RaisePostBackEvent(String eventArgument) +154
System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint) +3707
--------------------------------------------------------------------------------
Version Information: Microsoft .NET Framework Version:4.0.30319; ASP.NET Version:4.0.30319.272
For temporary resolution, we are restarting the server and restarting the MSSQLSERVER service.
After searching for above, I came to know it is due to connection leakage. Some opened connections are never get closed.
I am using ready made Microsoft Data Access Application Block for .NET i.e. SQLHelper Class which in my application for data access using Dataset and all.
Note: I am NOT using SqlDataReader anywhere.
I have examined the class and its closing the connections and I have NEVER used manual connection code instead used SQLHelper everywhere.
I also have the following nested transactions structure in stored procedures as follows:
BEGIN TRY
BEGIN TRANSACTION
-- Every below stored procedure has the transation structure as this SP i.e. Parent SP. (BEGIN TRY, BEGIN TRANSACTION etc)
exec sp1
exec sp2
exec sp3
---
---
COMMIT TRANSACTION
END TRY
BEGIN CATCH
IF ##TRANCOUNT>0
ROLLBACK TRANSACTION
PRINT 'ERROR OCCURED In SP'
END CATCH
Now, The problem is why connection leakage is occurring.
Is above transaction structure is responsible i.e it is blocking the transactions and requests.
If request is in process and user has closed the browser, does the application closes the connection?
Any help will be greatly appreciated.

Solved !!
It was not due to Nested Transactions but it was due to Bad Indexes Strategy used. Used normal Indexes and Indexes with include to solve the performance issues.
Also ran SQL Profiler to examine the most read and writes on columns and accordingly applied indexes and review maintenance and execution plans. !!

You can also use this,
SqlConnection.ClearAllPools();
The above code then closes all of the pools upon execution. I had the same problem and cleared the pools each time and then it should work successfully due to the pools being cleared.

How are SQL Server timeout exceptions handled in Ajax post backs?

I have an Ajax postback within which I make a SQL Server query. The structure's a little too complex to post here. If I have customErrors=Off", then the error is returned to the client in the HTTP response, from whence I pull it out and display the appropriate things to the user:
114|error|500|Timeout expired. The timeout period elapsed prior to
completion of the operation or the server is not responding.|
I have other "application exceptions" which I also catch at the client in the same way. All this is free: asp.net puts the above stuff in the http response without my assistance.
I then switch customErrors=On, because in other circumstances I don't want to show a full stack dump to users. At this point all my exceptions as above are stripped of the message, so they become simply:
0|error|500||
Which is entirely reasonable. So I then add some exception handling within my Ajax postback code to trap the Application errors... and then I explicitly write the exception data I need back to the client:
39|error|202|You do not have access to this record. |
This all works well.
The problem then is with one particular type of exception - SQL Server database timeouts. These appear to be generic "unhandled exceptions", with the database timeout exception inside them. The catch is that this exception doesn't hit my handler at all - I can't find it and report what I want!
Looking at the exception traces, the timeout trace is of course longer than my "you have no access" trace, because it binds several things before it hits my (forced for test purposes) timeout. Looking at just the last bits of the two traces, I have (application eception):
System.Web.HttpUnhandledException (0x80004005): Exception of type System.Web.HttpUnhandledException' was thrown. ---> System.ApplicationException: You do not have access to this record.
at pages_***.HideEventLists(Boolean becauseNotAccessible) in *
at pages_**.AjaxTabLoad(Object sender, AjaxTabEventArgs e) in *
at userControls_tabs.panel_Load(Object sender, EventArgs e) in *
at System.Web.UI.Control.OnLoad(EventArgs e)
Versus (SQL Server timeout):
System.Web.HttpUnhandledException (0x80004005): Exception of type 'System.Web.HttpUnhandledException' was thrown. ---> System.Data.SqlClient.SqlException (0x80131904): Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
So I'm wondering where precisely I should be trapping that SQL Server timeout exception? I don't really want to wrap every database access with it's own handler! Is there something I can do to get this exception passed the same way as my application exception?