In my service I have a background thread that does a best effort saving of a stream of object of certain entity type. Code roughly is following:
while (AllowRun)
{
try
{
using (DbContext context = GetNewDbContext())
{
while (AllowRun && context.GetConnection().State == ConnectionState.Open)
{
TEntity entity = null;
try
{
while (pendingLogs.Count > 0)
{
lock (pendingLogs)
{
entity = null;
if (pendingLogs.Count > 0)
{
entity = pendingLogs[0];
pendingLogs.RemoveAt(0);
}
}
if (entity != null)
{
context.Entities.Add(entity);
}
}
context.SaveChanges();
}
catch (Exception e)
{
// (1)
// Log exception and continue execution
}
}
}
}
catch (Exception e)
{
// Log context initialization failure and continue execution
}
}
(this is mostly the actual code, I omitted few non-relevant parts that attempt to keep popped objects in memory until we are able to save stuff to DB again when exception is caught at (1) block)
So, essentially, there is an endless loop, trying to read items from some list and save them to Db. If we detect that connection to DB failed for some reason, it just attempts to reopen it and continue. The issue is that sometimes (I failed to figure out how to reproduce it so far), the code above when context.SaveChanges() is called starts to produce following exception (caught in (1) block):
System.Data.EntityException: An error occurred while starting a transaction on the provider connection. See the inner exception for details. --->
System.InvalidOperationException: The requested operation cannot be completed because the connection has been broken.
The error is logged, but when the execution returns to the context.GetConnection().State == ConnectionState.Open check, it evaluates to true. So we are in a state when context reports that its DB connection is open, but we can't run queries against that context. Restarting the service removes the issue (as well as messing with AllowRun variable in debugger to force recreation of context). So the question is since I can't trust context's connection state, how do I verify that I can run queries against DB?
Also, is there a clean way to figure out that connection is not in a "healthy" state? I mean, the EntityException by itself is not an indication that I should reset the connection, only if its InnerException is InvalidOperationException with some specific Message, then yes, it is time to reset it. But, now I guess there would be other situations when ConnectionState indicates that everything is fine, but I can't query DB. Can I catch those proactively, not waiting until it starts to bite me?
What is the log frequency?
if this loop take longer than connection timeout, connection closed when savechanges executing.
while (pendingLogs.Count > 0)
{
lock (pendingLogs)
{
entity = null;
if (pendingLogs.Count > 0)
{
entity = pendingLogs[0];
pendingLogs.RemoveAt(0);
}
}
if (entity != null)
{
context.Entities.Add(entity);
}
}
context.SaveChanges();
From my experience working on similar services, garbage collection won't occur until the end of the using block.
If there were a lot of Pending logs to write, this could use a lot of memory, but I also guess it might starve the dbConnection pool.
You can analyse memory usage using RedGate ANTS or a similar tool, and check dbConnections that are open using the following script from this StackOverflow question: how to see active SQL Server connections?
SELECT
DB_NAME(dbid) as DBName,
COUNT(dbid) as NumberOfConnections,
loginame as LoginName
FROM
sys.sysprocesses
WHERE
dbid > 0
GROUP BY
dbid, loginame
;
I think it's good practice to free up the context as often as you can in order to give GC a change of cleaning up, so you could rewrite the loop as:
while (AllowRun)
{
try
{
while (pendingLogs.Count > 0)
{
using (DbContext context = GetNewDbContext())
{
while (AllowRun && context.GetConnection().State == ConnectionState.Open)
{
TEntity entity = null;
try
{
lock (pendingLogs)
{
entity = null;
if (pendingLogs.Count > 0)
{
entity = pendingLogs[0];
pendingLogs.RemoveAt(0);
}
}
if (entity != null)
{
context.Entities.Add(entity);
context.SaveChanges();
}
}
catch (Exception e)
{
// (1)
// Log exception and continue execution
}
}
}
}
}
catch (Exception e)
{
// Log context initialization failure and continue execution
}
}
I recommend go through below url:
Timeout Expired is usually thrown when a sql query takes too long to run.
Sounds like a SQL job is running, backup? That might be locking tables or restarting the service.
ADONET async execution - connection broken error
Related
I need to save the information about an exception to the database.
i.e
SomeMethod()
{
try
{
using(transaction = context.beginTransaction())
{
try
{
// here is the database error
await transaction.CommitAsync();
}
catch(Exception e)
{
await transaction.Rollback();
throw;
}
}
}
catch(Exception exception)
{
// Here I get the same error that was generated inside the transaction
context.Set<LogEntity>().Add(new LogEntity(....));
await context.SaveChangesAsync();
}
}
Do I understand correctly that a context is a connection to a database and that in one context I can perform several transactions?
As I understand it, the context in this stage is in a dirty state?
How can I write to the database after a transaction error has occurred?
As I understand it, the context in this stage is in a dirty state?
Correct. The "database error" must have occurred in a SaveChanges(Async) call, so if you repeat that call on the same context, the exception will occur again. The remedy is simple: use a new context in the catch block.
Side note: catching the exception in the using block isn't necessary. If the transaction isn't committed before it's disposed, it will be rolled back.
public static void SetStatus( Status statusObject,int retryCount)
{
if (statusObject != null)
{
using (SqliteConnection dbConn = new SqliteConnection(dbURL))
{
IDbTransaction dbTransaction = null;
try
{
dbConn.Open();
dbTransaction = dbConn.BeginTransaction();
new SqliteCommand(some_query, dbConn).ExecuteNonQuery();
}
catch (Exception e)
{
dbTransaction.Rollback();
dbConn.Close();
if (retryCount > 0)
{
SetStatus(statusObject, --retryCount);
return;
}
else
throw e;
}
finally
{
try { dbTransaction.Commit(); }
catch (Exception e)
{
}
}
}
}
}
Whenever ExecuteNonQuery fails due due to some exception I have a retry mechanism which will run the same query again.In that case during the second time(while retrying) the following exception comes-
"Connection must be valid and open to commit transaction"
The problem is basically, that you commit the transaction in the finally block—which runs always per design—even though you may already have rolled back the transaction and closed the connection in the catch block.
The easier way would be to commit the transaction while still in the try block, because after all, you only want to commit the transaction if all the code in the try succeeded.
That being said, you are having some odd nesting with the try and using constructs there. You should try to keep things like usings as concise as possible. That way you will also not have to deal with closing connections yourself. Something like this:
public static void SetStatus(Status statusObject, int retryCount)
{
if (statusObject == null)
return;
try
{
using (var dbConn = new SqliteConnection(dbURL))
{
IDbTransaction dbTransaction = null;
try
{
dbConn.Open();
dbTransaction = dbConn.BeginTransaction();
new SqliteCommand(some_query, dbConn).ExecuteNonQuery();
dbTransaction.Commit();
}
catch
{
// transaction might be null if the connection couldn’t be opened
dbTransaction?.Rollback();
throw;
}
}
}
catch (Exception ex)
{
if (retryCount > 0)
SetStatus(statusObject, --retryCount);
else
throw ex;
}
}
One final note, one could argue that using a transaction is absolutely not necessary here: You are only executing a single query on this database connection, so either that works, or it will fail in which case there is not something to rollback anyway. So unless you are having multiple queries there, you don’t need an explicit transaction (but I’ll just assume that you just reduced your example to show just a single query).
My bad,screwed up using "finally" improperly myself.So if the query fails at first attempt it would go to catch block which will fire the call to re run the query.Second time query execution would be successful and transaction would be committed.Then the control goes to finally of first query which will again try to commit the already rolled back transaction. So,it would throw that exception.
So,if the moving the commit to main try block fixed the issue.
The following code is causing an intermittent exception:
public int UnblockJob(int jobId)
{
using (var connect = MakeConnect())
{
var tag = connect.JobTag.SingleOrDefault(jt => jt.JobId == jobId && jt.Name == Metrics.TagNameItemBlockCaller);
if (tag == null)
{
return 0;
}
connect.JobTag.Remove(tag);
return connect.SaveChanges();
}
}
How can I correct or troubleshoot it?
From the documentation for DbUpdateConcurrencyException:
Exception thrown by DbContext when it was expected that SaveChanges for an entity would result in a database update but in fact no rows in the database were affected.
This means that the record you are attempting to delete has since been removed from the database. It would appear that you have another process that is deleting records or this function is able to be called concurrently.
There are several solutions, here are a couple:
Fix the source problem Stop other processes affecting the data.
Catch the error Wrap this method in a try/catch block, after all you may only care that the record has been deleted:
try
{
//Existing code here
}
catch(DbUpdateConcurrencyException)
{
//Safely ignore this exception
}
catch(Exception e)
{
//Something else has occurred
throw;
}
I'm getting strange things since updated to EF6,no sure this is related or not, but used to be good
I'm doing a set of work, then save it to DB , then do another , save another.
after a while,i check SQL server by sp_who2 , i found many dead connections from my computer.
Job is huge then there goes to 700 connections,
I have to kill them all manually in cycle.
program like:
while (jobDone == false)
{
var returnData=doOneSetJob();
myEntity dbconn= new myEntity;
foreach( var one in retrunData)
{
dbconn.targetTable.add(one );
try
{
dbconn.savechange();
/// even i put a dispose() here , still lots of dead connections
}
catch
{
console.writeline("DB Insertion Fail.");
dbconn.dispose();
dbconn= new myEntity();
}
}
dbconn.dispose()
}
You should consider refactoring your code so that your connection is cleaned up after your job is complete. For example:
using (var context = new DbContext())
{
while (!jobDone)
{
// Execute job and get data
var returnData = doOneSetJob();
// Process job results
foreach (var one in returnData)
{
try
{
context.TargetTable.Add(one);
context.SaveChanges();
}
catch (Exception ex)
{
// Log the error
}
}
}
}
The using statement will guarantee that your context is cleaned up properly, even if an error occurs while you are looping through the results.
In this case you should use a using statement. Taken from MSDN:
The using statement ensures that Dispose is called even if an exception occurs while you are calling methods on the object. You can achieve the same result by putting the object inside a try block and then calling Dispose in a finally block; in fact, this is how the using statement is translated by the compiler.
So, your code would look better like this:
using(var dbconn = new DbContext())
{
while (!jobDone)
{
foreach(var one in retrunData)
{
try
{
targetTable row = new TargetTable();
dbconn.TargetTable.add(row);
dbconn.SaveChanges();
}
catch (Exception ex)
{
Console.WriteLine("DB Insertion Fail.");
}
}
}
}
This way, even if your code fails at some point, the Context, resources and connections will be properly disposed.
I have some problems with my database connection in my application.
My application stores some data into a mysql database and now I got the error mentioned in the title.
The problem is:
Right before I call the method that throws this exception I call another method which also connects to the database but successfully.
At first I thought that there is an error in my source but there is no error.
Here is the bad method:
public static bool checkExistence(object obj)
{
MySqlCommand checkQuery = new MySqlCommand(MySQLQueries.getCheckQueryByObject(obj), connection);
object checkResult = null;
if (connection.State == System.Data.ConnectionState.Closed)
{
OpenConnection();
try
{
checkResult = checkQuery.ExecuteScalar();
}
catch (MySqlException ex)
{
ErrorHandler(ex);
return true;
}
catch (Exception ex)
{
ErrorHandler(ex);
return true;
}
finally
{
CloseConnection();
}
}
else
{
checkResult = checkQuery.ExecuteScalar();
}
if (checkResult != null)
return true;
else
return false;
}
As you can see I first create a new command which later will check if an object exists.
The result of the query will be stored in object checkResult and at the end I simply check if checkResult == null.
Because of some problems with opening / closing the connection I decided to check if the connection is opened or not.
Now if the bad method is called the connection is closed.
Then after calling OpenConnection() the connection is opened again with the same settings as before.
As soon as the application executes checkResult = checkQuery.ExecuteScalar(); the exception is thrown.
I really don't know what actually is the problem.
To ensure that the connection is valid from my point of view I put connection into the watchlist and tracked every change made to it by the application.
Nothing was different than in any other method that successfully used the database.
A friend of mine said that there might be a kind of optimization error while building the application.
He himself doesn't work with C# but sometime with C so it was just a vague guess.
Is there maybe any error in the code that I missed or any guess why this happens?
Thanks in advance!
Greetz
Edit: I use the latest version of MySQL's C# connector