I'm using SqlBulkCopy to copy a batch of records from MySQL to SQL Server.
After exactly 30 seconds, I get this
System.Data.SqlClient.SqlException (0x80131904): Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
The statement has been terminated.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlCommand.RunExecuteNonQueryTds(String methodName, Boolean async)
at System.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(DbAsyncResult result, String methodName, Boolean sendToPipe)
at System.Data.SqlClient.SqlCommand.ExecuteNonQuery()
There's one 'Error' object inside the exception, with the following details:
Message: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
LineNumber: 0
Source: .Net SqlClient Data Provider
Procedure:
Index #1
Message: The statement has been terminated.
LineNumber: 1
Source: .Net SqlClient Data Provider
Procedure:
Here's the code
using (MySqlConnection sourceConnection = new MySqlConnection(AccManConnectionString)) {
sourceConnection.Open();
MySqlCommand commandSourceData = new MySqlCommand(string.Format(sql, VersionNum.ToString()), sourceConnection);
for (int i = 0; i < ParamNames.Length; i++)
{
commandSourceData.Parameters.AddWithValue(ParamNames[i], SetIDList[i]);
}
MySqlDataReader reader = commandSourceData.ExecuteReader();
using (SqlConnection destinationConnection = new SqlConnection(TimetableConnectionString))
{
try
{
destinationConnection.Open();
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(destinationConnection))
{
bulkCopy.DestinationTableName = "NetworkID";
// Configure the batch sizes and timeouts (cofig code omitted)
bulkCopy.BatchSize = batchSize;
bulkCopy.BulkCopyTimeout = timeout;
try
{
bulkCopy.WriteToServer(reader);
SqlCommand update = new SqlCommand(string.Format("UPDATE p SET p.Username = n.Username FROM NetworkID n INNER JOIN Person p ON n.PersonID = p.PersonID and n.VersionID = {0} where p.VersionID = {0}", VersionNum), destinationConnection);
update.ExecuteNonQuery();
}
catch (SqlException ex)
{
log.Error("Exception caught", ex);
}
finally
{
reader.Close();
}
}
}
catch (Exception e)
{
log.Error("Exception caught", e);
}
}
}
I know there are plenty of timeout/batch size parameters I can (and have) experiment with. But my question is, from a coding point of view, is there any way of determining which database server is the one giving me problems?
Thanks
The timeout you are experiencing is likely to be influenced by the settting SqlBulkCopy.BulkCopyTimeOut which has a default of 30 seconds.
As for determining where the problem lies, your best bet is to catch the SqlException to see if it contains any more details, but in your instance I believe it will be your code (the client) timing out.
The documentation on SqlException has a good example of how to enumerate the errors contained in the exception.
Update 1
I can see you are using MySqlCommand, I'm guessing this is Devart, if so you haven't set a timeout on this command, for this one you'll need to use the syntax CommandTimeout.
MySqlCommand commandSourceData = new MySqlCommand(string.Format(sql, VersionNum.ToString()), sourceConnection);
commandSourceData.CommandTimeout = timeout;
You should also put one on your SqlCommand.
SqlCommand update = new SqlCommand(string.Format("UPDATE p SET p.Username = n.Username FROM NetworkID n INNER JOIN Person p ON n.PersonID = p.PersonID and n.VersionID = {0} where p.VersionID = {0}", VersionNum), destinationConnection);
update.CommandTimeout = timeout;
Update 2
Just reading the documentation on SqlBulkCopy and noticed the following:
If multiple active result sets (MARS) is disabled, WriteToServer makes
the connection busy. If MARS is enabled, you can interleave calls to
WriteToServer with other commands in the same connection.
I'm not sure if you are using MARS, but, your code above is calling into SQL to do an update after the WriteToServer method, but before the BulkCopy is closed (via Using). Can you try explicitly closing the SQLBulkCopy by calling close before the Update or move the Update outside of the Using statement for the BulkCopy.
Related
The state of the connection stays active in pool, when there is an exception while executing a query or stored procedure through c#.
The Npgsql version that I am using is 4.1.7.
Here is the code that I am trying to execute.
NpgsqlCommand cmd = null;
NpgsqlDataAdapter sda = null;
NpgsqlConnection conn = null;
try {
string sql = "a_test";
conn = new NpgsqlConnection("Server=localhost;Port=5432;Username=admin;Password=test;Database=majordb;SearchPath=dbs;CommandTimeout=300;MaxPoolSize=500;Connection Idle Lifetime=180;Connection Pruning Interval=5;");
cmd = new NpgsqlCommand(sql);
cmd.CommandType = CommandType.StoredProcedure;
sda = new NpgsqlDataAdapter(cmd);
cmd.Connection = conn;
sda.Fill(dataTable);
}
catch (Exception e) {
//log
}
finally {
if(null != sda)
{
try
{
sda.Dispose();
}
catch (Exception)
{
}
}
try
{
cmd.Connection.Close();
cmd.Connection.Dispose();
}
catch (Exception)
{
}
try
{
cmd.Dispose();
}
catch (Exception)
{
}
}
If the above code executes properly without any exception, the connection state in pool goes to idle, which is correct. But if an exception occurs while executing, like below:
"Npgsql.NpgsqlException (0x80004005): Exception while reading from stream --->
System.IO.IOException: Unable to read data from the transport connection: A connection attempt
failed because the connected party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond."
The connection state in pool shows as active for about 5 mins or so, even though the close/dispose methods are called in finally block. This means the close/dispose did not properly executed by Npgsql. If the program keeps the connection state in pool active for every connection ran within 5 mins, then there can be an issue with MaxPoolSize error.
I wanted to see the connection state to idle, even when there is an exception. How do I do this.
Please note: I am not looking for a solution to the exception that I listed above. I am looking for a solution where the connection state is changed to idle from active when there is an exception while executing the above code.
To know if the connection state is active or not I used the following query:
SELECT
pid,
usename,
application_name,
datname,
client_addr,
rank() over (partition by client_addr order by backend_start ASC) as rank,
state,
state_change,
current_timestamp,
query,
query_start,
backend_start,
FROM
pg_stat_activity
WHERE
pid <> pg_backend_pid( )
AND
application_name !~ '(?:psql)|(?:pgAdmin.+)'
AND
datname = current_database()
AND
usename = current_user
Any help is really appreciated.
I have been experiencing db transaction timemouts in an ASP.NET 2.0 & SQL Server 2008 application for several days now.
The issue is something using SQL Server profiler tracing to "a function calling web service".
We have transaction include several stored procedure (sp). But stored procedure #3 has some bytes to save in DB, it consumes time. When the time goes over 25 seconds, it throws a timeout exception message:
Message : System.Web.Services.Protocols.SoapException: Server was unable to process request. ---> System.Data.SqlClient.SqlException: Timeout expired. The timeout period elapsed prior to completion of the operation or the server is not responding.
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
My code:
using (System.Data.Common.DbConnection connection = o_DB.CreateConnection())
{
connection.Open();
System.Data.Common.DbTransaction o_Transaction = connection.BeginTransaction();
try
{
- exec sp1
- exec sp2
foreach{
- exec sp3
}
}
We tried some solutions in the web but it did not work. I hope someone can give a hand to me. Many Thanks.
We do not have timeout in the connection string in web.config, it does not suffer on exception after 15s (default)
We set the transaction timeout in web.config:
<system.transactions>
<machineSettings maxTimeout="00:00:30" />
</system.transactions>
Something like that
using (var ts = CreateTransactionScope(TimeSpan.FromSeconds(mySecondsVar)))
{
using (System.Data.Common.DbConnection connection = o_DB.CreateConnection())
{
using (IDbTransaction tran = connection.BeginTransaction()) {
try
{
// your code
}
catch {
tran.Rollback();
}
}
}
ts.Complete();
}
If is "ok", the dispose do the commit automatically.
This you can directly handle by code also.
Please see the below code
public DataSet getData(string command)
{
DataSet ds = new DataSet();
string connectionString = ConfigurationManager.ConnectionStrings["TESTDB"].ConnectionString;
using (var conn = new SqlConnection(connectionString))
{
using (var cmd = new SqlCommand(command, conn))
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandTimeout = 0;
SqlDataAdapter adapt = new SqlDataAdapter(cmd);
conn.Open();
adapt.Fill(ds);
conn.Close();
}
}
return ds;
}
The blow line will change the execution time to infinity or until query execution complete.
cmd.CommandTimeout = 0;
I have just a general type of question. If I have a C# application that calls a SQL Server stored procedure, and the C# application times out, does the procedure call on the server continue running to it's completion?
No. Below is a reproduction. When the timeout occurs the running process will be killed, halting it immediately. If you do not have a transaction specified, work that has been done in the stored procedure prior to the timeout will be persisted. Similarly, if the connection to the server is severed by some outside force, SQL Server will kill the running process.
using (var conn = new SqlConnection(#"Data Source=.;Initial Catalog=Test;Integrated Security=True"))
{
conn.Open();
using (var setupTable = new SqlCommand(#"
IF NOT EXISTS (
SELECT *
FROM
sys.schemas s
INNER JOIN sys.tables t ON
t.[schema_id] = s.[schema_id]
WHERE
s.name = 'dbo' AND
T.name = 'TimeoutTest')
BEGIN
CREATE TABLE dbo.TimeoutTest
(
ID int IDENTITY(1,1) PRIMARY KEY,
CreateDate datetime DEFAULT(getdate())
);
END
-- remove any rows from previous runs
TRUNCATE TABLE dbo.TimeoutTest;", conn))
{
setupTable.ExecuteNonQuery();
}
using (var checkProcExists = new SqlCommand(#"
SELECT COUNT(*)
FROM
sys.schemas s
INNER JOIN sys.procedures p ON
p.[schema_id] = s.[schema_id]
WHERE
s.name = 'dbo' AND
p.name = 'AddTimeoutTestRows';", conn))
{
bool procExists = ((int)checkProcExists.ExecuteScalar()) == 1;
if (!procExists)
{
using (var setupProc = new SqlCommand(#"
CREATE PROC dbo.AddTimeoutTestRows
AS
BEGIN
DECLARE #stop_time datetime;
SET #stop_time = DATEADD(minute, 1, getdate());
WHILE getdate() < #stop_time
BEGIN
INSERT INTO dbo.TimeoutTest DEFAULT VALUES;
-- wait 10 seconds between inserts
WAITFOR DELAY '0:00:10';
END
END", conn))
{
setupProc.ExecuteNonQuery();
}
}
}
bool commandTimedOut = false;
try
{
using (var longExecution = new SqlCommand("EXEC dbo.AddTimeoutTestRows;", conn))
{
// The time in seconds to wait for the command to execute.
// Explicitly setting the timeout to 30 seconds for clarity.
longExecution.CommandTimeout = 30;
longExecution.ExecuteNonQuery();
}
}
catch (SqlException ex)
{
if (ex.Message.Contains("Timeout"))
{
commandTimedOut = true;
}
else
{
throw;
}
}
Console.WriteLine(commandTimedOut.ToString());
// Wait for an extra 30 seconds to let any execution on the server add more rows.
Thread.Sleep(30000);
using (var checkTableCount = new SqlCommand(#"
SELECT COUNT(*)
FROM
dbo.TimeoutTest t;", conn))
{
// Only expecting 3, but should be 6 if server continued on without us.
int rowCount = (int)checkTableCount.ExecuteScalar();
Console.WriteLine(rowCount.ToString("#,##0"));
}
}
Console.ReadLine();
produces the following output
True
3
even though running the stored procedure from Management Studio will add 6 rows in the one minute time frame.
Short answer is Yes .... here is some info to back up my claim
There are actually several places where a application can 'time out' but one is a Command Execution time out ...
command execution time out - This property is the cumulative time-out for all network reads during command execution or processing of the results. A time-out can still occur after the first row is returned, and does not include user processing time, only network read time.
A Command is not going to rollback on its own if it times out. This will need a transaction around the code if it times out.
if a timeout can occur when rows are being returned than means a timeout can occur anytime C# is not going to tell SQL Server to stop running the command. Things can be done about that such as wrapping the command in a transaction
Source: https://blogs.msdn.microsoft.com/mattn/2008/08/29/sqlclient-timeouts-revealed/ ... and experience
If you are using the SQlCommand Class, once the app times out , the query execution will be rolled back.
My thoughts about that are that it's all about the connection opened by the call to the procedure.
If your code is executed within a using block or if it is garbage collected then i think that the execution of the SP will be rolled back.
Conn = new SqlConnection(ConnStr);
Conn.Open();
myCommand = new SqlCommand();
myCommand.CommandTimeout = 180000;
myCommand.Connection = Conn;
myCommand.CommandType = System.Data.CommandType.StoredProcedure;
I am getting a timeout error thrown. How can I increase the amount of time before the transaction times out?
Database dbSvc = DatabaseFactory.CreateDatabase();
//use one connection
using (DbConnection conn = dbSvc.CreateConnection())
{
conn.Open();
DbTransaction transaction = conn.BeginTransaction();
try
{
....
}
catch (Exception ex)
{
transaction.Rollback();
ret.IsSuccess = false;
ret.ExceptionInfo = ex;
}
finally
{
ret.InvoiceInfo = invoiceOut;
}
Thanks!
If you are using ADO.Net add timeout property to your connection string. Increase accordingly until it no longer times out, but generally if you are adding a timeout property you should be looking at your db and analyzing weaknesses such as lack of indexes this can be done via analyzing your queries via the sql profilier.
I have a connection class that handles my Informix database queries. It has two functions; one to perform simple queries, and one to return a datatable. Intermittently (especially when I let the session sit for a bit, such as ten minutes) I'll get an abandoned mutex error on the conn.open command when requesting new info.
Here is the code in question:
public DataTable CallDtQuery(string query)
{
DataTable dt = new DataTable();
using (IBM.Data.Informix.IfxConnection conn = new
IBM.Data.Informix.IfxConnection(sqlConnection))
{
try
{
IBM.Data.Informix.IfxDataAdapter adapter = new IfxDataAdapter();
adapter.SelectCommand = new IBM.Data.Informix.IfxCommand(query, conn);
conn.Open(); //Error location.
adapter.Fill(dt);
conn.Close();
}
catch (IBM.Data.Informix.IfxException ex)
{
LogError(ex, query);
SendErrorEmail(ex, query);
DisplayError();
}
}
return dt;
}
Additionally, here is the simple query function, which is the only other function in the application that connects to the database:
public string CallSimpleQuery(string query, string command)
{
string result = "";
using (IBM.Data.Informix.IfxConnection conn = new
IBM.Data.Informix.IfxConnection(sqlConnection))
{
try
{
IBM.Data.Informix.IfxDataAdapter adapter = new IfxDataAdapter();
conn.Open();
switch (command)
{
case "UPDATE":
adapter.UpdateCommand = new IBM.Data.Informix.IfxCommand(query, conn);
result = adapter.UpdateCommand.ExecuteNonQuery().ToString();
break;
case "DELETE":
adapter.DeleteCommand = new IBM.Data.Informix.IfxCommand(query, conn);
result = adapter.DeleteCommand.ExecuteNonQuery().ToString();
break;
case "SELECT":
adapter.SelectCommand = new IBM.Data.Informix.IfxCommand(query, conn);
result = adapter.SelectCommand.ExecuteScalar().ToString();
break;
case "INSERT":
adapter.InsertCommand = new IBM.Data.Informix.IfxCommand(query, conn);
result = adapter.InsertCommand.ExecuteNonQuery().ToString();
break;
}
conn.Close();
}
catch (IBM.Data.Informix.IfxException ex)
{
LogError(ex, query);
SendErrorEmail(ex, query);
DisplayError();
}
}
return result;
}
Here is the error generated:
Error Message = The wait completed due to an abandoned mutex.
Message Source:
mscorlib
=============================
Message Target:
Boolean WaitOne(Int64, Boolean)
=============================
Stack Trace:
at System.Threading.WaitHandle.WaitOne(Int64 timeout, Boolean exitContext)
at System.Threading.WaitHandle.WaitOne(Int32 millisecondsTimeout, Boolean exitContext)
at System.Threading.WaitHandle.WaitOne()
at IBM.Data.Informix.IfxConnPoolManager.GetPool(IfxConnSettings key)
at IBM.Data.Informix.IfxConnPoolManager.Open(IfxConnection connection)
at IBM.Data.Informix.IfxConnection.Open()
at XXX.Connections.CallDtQuery(String query) in d:\Inetpub\wwwroot\intranet\CWSheet-test2\App_Code\Connections.cs:line 75
at XXX.details.Page_Load(Object sender, EventArgs e) in d:\Inetpub\wwwroot\intranet\CWSheet-test2\Details.aspx.cs:line 29
at System.Web.Util.CalliHelper.EventArgFunctionCaller(IntPtr fp, Object o, Object t, EventArgs e)
at System.Web.Util.CalliEventHandlerDelegateProxy.Callback(Object sender, EventArgs e)
at System.Web.UI.Control.OnLoad(EventArgs e)
at System.Web.UI.Control.LoadRecursive()
at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
When the error occurs, instead of going into the catch block and running the Log, Send, and Display error functions, it instead drops into the Application_Error block on the global.asax. Since everything is wrapped inside the try/catch block, I'm not sure what could cause this. Additionally, for whatever reason the application hangs on the CallDtQuery on occasion. I'll be flipping through pages of records in a formview and it will suddenly hang on the CallDtQuery request. Sometimes it might go through after a minute or two, and sometimes it will hang indefinitely until the application times out after 30 minutes.
I've been reading a bit about mutex, but have not ever used it before. Any mutex being used is being generated automatically by the ASP.NET application. With that in mind, I'm not really sure how to troubleshoot or resolve this issue. Any suggestions?
So it turns out the issue was with the IBM.Data.Informix.dll version I was using (2.90.) I found documentation explaining the issue here: http://www.iiug.org/forums/development-tools/index.cgi/read/109
Once I updated to a newer version (3.50), the Abandoned Mutex errors went away. The intermittent hang issue also disappeared as well.
Well, assuming you're not doing any weird threading stuff there are several things to consider
The informix C# classes have some issues with concurrency management (i recall this APAR now)
I can't say why, but creating the command before opening the connection feels odd.
I'd dispose the ifxcommand to, and in particular calling Dispose on the IfxConnection automatically calls the Close method, maybe in the double close some handles are getting mixed up, take a look at this