Wihin my C# code I am using the CommandTimeout function to ensure that any query that executes longer than 30s is terminated both from the server and database. However when listing the currently running queries on the database the query that was set to cancel after 30s runs well beyond 30s
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
SqlCommand sqlCommand = new SqlCommand(query, connection);
//Set Timeout to 30s
sqlCommand.CommandTimeout = 30;
SqlDataAdapter da = new SqlDataAdapter(sqlCommand);
da.Fill(response);
connection.Close();
da.Dispose();
}
Why is the query still running in the DB? Is my only option right now is to send another query from the server to kill the query (KILL [session_id]) after 30s?
EDIT: 300Mb of data is being returned for this query.
There are a number of posts on StackOverflow indicating that SqlCommand.CommandTimeout won't affect the behavior of SqlDataAdapter.Fill. Instead, you supposedly have to set the SqlDataAdapter's SelectCommand.CommandTimeout property.
However, there are other posts which seem to indicate that even this doesn't work. This one in particular makes me think that the query will only be canceled if the timeout occurs before the query starts yielding results. Once results start coming in, it appears to ignore all timeouts.
My recommendation would be to reconsider using SqlDataAdapter. Depending on your use case, maybe a library like Dapper would work better for you?
You may also want to consider reporting this as a defect to the .NET team. I've had mixed success in the past reporting such errors; it depends on whether the team wants to prioritize fixing the issue.
Update
It looks like this may be the intended, documented behavior, as Marc Gravell points out here.
lol: from the documentation
(https://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlcommand.commandtimeout(v=vs.110).aspx)
For example, with a 30 second time out, if Read requires two network
packets, then it has 30 seconds to read both network packets. If you
call Read again, it will have another 30 seconds to read any data that
it requires.
So: this timeout resets itself every Read. So: the only way it'll trip
is if any single Read operation takes longer than 2s. As long as the
SQL Server manages to get at least one row onto the pipe in that time:
it won't timeout via either API.
I have some code that reads results into a List from a SqlDataReader, parsing them into domain objects as it goes using reader.GetXXX(int ordinal) methods inside a while reader.Read() loop.
This code generally works fine over very large datasets and across a wide range of queries and tables, but one query hangs midway through reading this list of results and eventually times out with a "Wait Operation Timed Out" error.
I've repeated this a lot of times, and it always hangs on the same record (roughly 336k records into a 337k record set).
If I pause execution while it's hanging i can see that it is midway through parsing a record, and it is hanging on a reader.GetXXX call.
I have tried the following:
executing the proc manually in ssms (works fine)
Chunking the calls to the database such that it reads 250k records in a chunk and then requeries to get the rest (it still hung on the same record, but now the record was in the second batch)
Checking the ID for each record before parsing, and skipping the one that it hangs on (the record after that is parsed but it hangs on the next record).
Updating stats (parsing gets about 3000 records further before hanging after this)
I'm tempted to blame the database, but given the query runs without a hitch in SSMS, I'm not convinced. Any help much appreciated!
other stuff that may help:
the proc takes a table valued parameter and joins onto that along with another database table to get its results
this was originally spotted on a VM running a long way from the client machine, but i have subsequently reproduced it by restoring and connecting to a machine that is under my desk.
edit: as requested, the query is:
CREATE PROCEDURE [trading].[Trade_SelectLegalEntityPositionChunked]
#PartyPositions trading.PartyPositionType READONLY,
#Offset INT,
#ChunkSize INT
AS
BEGIN
SET NOCOUNT ON;
DECLARE #OffsetLocal INT = #Offset;
DECLARE #ChunkSizeLocal INT = #ChunkSize;
SELECT /* stuff */
FROM [trading].[Trade]
JOIN [refdata].[Book] ON [refdata].[Book].[BookId] = [trading].[Trade].[BookId]
JOIN #PartyPositions pos
ON pos.[PartyId] = [refdata].[Book].[LegalEntityId]
AND [trading].[Trade].[TradeType] = pos.[TradeType]
AND [trading].[Trade].[InstrumentId] = pos.[InstrumentId]
ORDER BY [trading].[Trade].[TradeId]
OFFSET #OffsetLocal ROWS FETCH NEXT #ChunkSizeLocal ROWS ONLY;;
END
edit: i've also checked:
other threads - there are none that should be getting in the way: the only threads that are running are the one running the query and the usual supporting threads such as the main thread and message loop (I'm testing from a CUI)
edit (bored yet?)
if i reduce the scope of the query a bit i can see it blocking while parsing a record for about 4 mins. It then carries on reading stuff until it hangs again.
I still cannot see anything much going on in the client - i now have the GcNotification class from CLR via C# running and there isn't any GC going on.This points to the database, as #Crowcoder says but the fact it runs fine in SSMS and it pausing midway through reading a record means I am loathe to blame it. This is probably down to my lack of knowledge about databases though!
The question involves SQL Server and C#.
I am trying to execute multiple delete statements in one query. The format has the following structure:
DELETE from dbo.table where column = 'value'
repeated more than 100000 times
I build the command through a StringBuilder and call it that way in my C# code:
cmd.Connection = con;
int rows = cmd.ExecuteNonQuery();
However it takes a lot of time to execute and ends with this error:
The timeout period elapsed prior to completion of the operation or the server. Executing through Management Studio takes also a lot of time, and let me think that the query isn't performant enough. However, in this case the query is executed successfully.
Obviously, using fewer DELETE statements, the query ends properly due it's a very simple query. What is the best way to execute multiple statements and avoid this error?
Can't you do only a single DELETE execution with the "IN" instead of "=" in your query?
Do something like:
DELETE from dbo.table where column in ('Param1','Param2','(...)')
Also, you could check this article:
http://social.technet.microsoft.com/wiki/contents/articles/20651.sql-server-delete-a-huge-amount-of-data-from-a-table.aspx
Cheers!
I am using ADO.Net + C# + VSTS 2008 + ADO.Net to connect to SQL Server 2008 Enterprise. I am using almost the same pattern/sample mentioned here -- using ADO.Net DataReader to retrieve data one entry (row) by one entry (row).
http://msdn.microsoft.com/en-us/library/haa3afyz.aspx
My question is, if I set the SqlCommand timeout in this sample,
1. I think the timeout applies to how much time we could use as maximum value to retrieve one specifc row, not the total timeout for the whole data entry-by-entry loop?
BTW: loop I mean,
while (reader.Read())
{
Console.WriteLine("{0}\t{1}", reader.GetInt32(0),
reader.GetString(1));
}
2.
and this timeout only matters how much time it takes to retrieve data entry from database, and this timeout has nothing to do with how much time we deal with each entry (e.g. if we set timeout to 20 seconds, and if it takes 1 second to retrieve one data entry from database, and it takes 30 seconds for my application logics to manipulate the data entry, timeout will never happen).
Correct understanding?
The command timeout that you can set applies to how long you give ADO.NET to do its job.
If you call cmdQuery.ExecuteNonQuery() which returns nothing but performs a SQL statement it's the time needed to perform that statement.
If you call cmdQuery.ExecuteReader() which returns a data reader, it's the time needed for ADO.NET to ste up / construct that data reader so that you can then use it.
If you call cmdQuery.ExecuteScalar() which returns a single scalar value, it's the time needed to execute the query and grab that single result.
If you use the dataAdapter.Fill() to fill a data table or data set, it's the time needed for ADO.NET to retrieve the data and then fill the data table or data set.
So overall : the timeout applies to the portion of the job that ADO.NET can do - execute the statement, fill a data set, return a scalar value.
Of course it does NOT apply to the time it takes YOU to iterate through the results (in case of a data reader). That wouldn't make sense at all...
Marc
Yes you are right. The CommandTimeout means the Time the Database needs to execute the command (any command)
Update: Looks like the query does not throw any timeout. The connection is timing out.
This is a sample code for executing a query. Sometimes, while executing time consuming queries, it throws a timeout exception.
I cannot use any of these techniques:
1) Increase timeout.
2) Run it asynchronously with a callback. This needs to run in a synchronous manner.
please suggest any other techinques to keep the connection alive while executing a time consuming query?
private static void CreateCommand(string queryString,
string connectionString)
{
using (SqlConnection connection = new SqlConnection(
connectionString))
{
SqlCommand command = new SqlCommand(queryString, connection);
command.Connection.Open();
command.ExecuteNonQuery();
}
}
Since you are using ExecuteNonQuery which does not return any rows, you can try this polling based approach. It executes the query in an asyc manner (without callback)
but the application will wait (inside a while loop) until the query is complete. From MSDN. This should solve the timeout problem. Please try it out.
But, I agree with others that you should think more about optimizing the query to perform under 30 seconds.
IAsyncResult result = command.BeginExecuteNonQuery();
int count = 0;
while (!result.IsCompleted)
{
Console.WriteLine("Waiting ({0})", count++);
System.Threading.Thread.Sleep(1000);
}
Console.WriteLine("Command complete. Affected {0} rows.",
command.EndExecuteNonQuery(result));
You should first check your query to see if it's optimized and it isn't somehow running on missing indexes. 30 seconds is allot for most queries, even on large databases if they are properly tuned. If you have solid proof using the query plan that the query can't be executed any faster than that, then you should increase the timeout, there's no other way to keep the connection, that's the purpose of the timeout to terminate the connection if the query doesn't complete in that time frame.
I have to agree with Terrapin.
You have a few options on how to get your time down. First, if your company employs DBAs, I'd recommend asking them for suggestions.
If that's not an option, or if you want to try some other things first here are your three major options:
Break up the query into components that run under the timeout. This is probably the easiest.
Change the query to optimize the access path through the database (generally: hitting an index as closely as you can)
Change or add indices to affect your query's access path.
If you are constrained from using the default process of changing the timeout value you will most likely have to do a lot more work. The following options come to mind
Validate with your DBA's and another code review that you have truly optimized the query as best as possible
Work on the underlying DB structure to see if there is any gain you can get on the DB side, creating/modifying an idex(es).
Divide it into multiple parts, even if this means running procedures with multiple return parameters that simply call another param. (This option is not elegant, and honestly if your code REALLY is going to take this much time I would be going to management and re-discussing the 30 second timeout)
We recently had a similar issue on a SQL Server 2000 database.
During your query, run this query on your master database on the db server and see if there are any locks you should troubleshoot:
select
spid,
db_name(sp.dbid) as DBname,
blocked as BlockedBy,
waittime as WaitInMs,
lastwaittype,
waitresource,
cpu,
physical_io,
memusage,
loginame,
login_time,
last_batch,
hostname,
sql_handle
from sysprocesses sp
where (waittype > 0 and spid > 49) or spid in (select blocked from sysprocesses where blocked > 0)
SQL Server Management Studio 2008 also contains a very cool activity monitor which lets you see the health of your database during your query.
In our case, it was a networkio lock which kept the database busy. It was some legacy VB code which didn't disconnect its result set quick enough.
If you are prohibited from using the features of the data access API to allow a query to last more than 30 seconds, then we need to see the SQL.
The performance gains to be made by optimizing the use of ADO.NET are slight in comparison to the gains of optimizing the SQL.
And you already are using the most efficient method of executing SQL. Other techniques would be mind numbingly slower (although, if you did a quick retrieval of your rows and some really slow client side processing using DataSets, you might be able to get the initial retrieval down to less than 30 seconds, but I doubt it.)
If we knew if you were doing inserts, then maybe you should be using bulk insert. But we don't know the content of your sql.
This is an UGLY hack, but might help solve your problem temporarily until you can fix the real problem
private static void CreateCommand(string queryString,string connectionString)
{
int maxRetries = 3;
int retries = 0;
while(true)
{
try
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
SqlCommand command = new SqlCommand(queryString, connection);
command.Connection.Open();
command.ExecuteNonQuery();
}
break;
}
catch (SqlException se)
{
if (se.Message.IndexOf("Timeout", StringComparison.InvariantCultureIgnoreCase) == -1)
throw; //not a timeout
if (retries >= maxRetries)
throw new Exception( String.Format("Timedout {0} Times", retries),se);
//or break to throw no error
retries++;
}
}
}
command.CommandTimeout *= 2;
That will double the default time-out, which is 30 seconds.
Or, put the value for CommandTimeout in a configuration file, so you can adjust it as needed without recompiling.
You should break your query up into multiple chunks that each execute within the timeout period.
If you absolutely cannot increase the timeout, your only option is to reduce the time of the query to execute within the default 30 second timeout.
I tend to dislike increasing the connection/command timeout since in my mind that would be a matter of taking care of the symptom, not the problem
have you thought about breaking the query down into several smaller chunks?
Also, have you ran your query against the Database Engine Tuning Advisor in:
Management Studio > Tools > Database Engine Tuning Advisor
Lastly, could we get a look at the query itself?
cheers
Have you tried wrapping your sql inside a stored procedure, they seem to have better memory management. Have seen timeouts like this before in plan sql statement with internal queries using classic ADO. i.e. select * from (select ....) t inner join somthingTable. Where the internal query was returning a very large number of results.
Other tips
1. Performing reads with the with(nolock) execution hint, it's dirty and I don't recommend it but it will tend to be faster.
2. Also look at the execution plan of the sql your trying to run and reduce the row scanning, the order in which you join tables.
3. look at adding some indexes to your tables for faster reads.
4. I've also found that deleting rows is very expensive, you could try and limit the number of rows per call.
5. Swap #table variables with #temporary tables has also worked for me in the past.
6. You may also have saved bad execution plan (heard, never seen).
Hope this helps
Update: Looks like the query does not
throw any timeout. The connection is
timing out.
I.o.w., even if you don't execute a query, the connection times out? because there are two time-outs: connection and query. Everybody seems to focus on the query, but if you get connection timeouts, it's a network problem and has nothing to do with the query: the connection first has to be established before a query can be ran, obviously.
It might be worth trying paging the results back.
just set sqlcommand's CommandTimeout property to 0, this will cause the command to wait until the query finishes...
eg:
SqlCommand cmd = new SqlCommand(spName,conn);
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandTimeout = 0;