Using entity framework, I have a function that basically goes something like this:
using (var ctx = new Dal.MyEntities())
{
try
{
//...
// create a temp entity
Dal.Temp temp = new Dal.Temp();
// populate its children
// note that temp is set to cascade deletes down to it's children
temp.Children = from foo in foos
select new Dal.Children()
{
// set some properties...
Field1 = foo.field1,
Field2 = foo.field2
}
//...
// add temp row to temp table
ctx.Temp.Add(temp);
ctx.SaveChanges();
// some query that joins on the temp table...
var results = from d in ctx.SomeOtherTable
join t in temp.Children
on new { d.Field1, d.Field2 } equals new { t.Field1, d.Field2 }
select d;
if (results.Count() == 0)
{
throw new Exception("no results")
}
// Normal processing and return result
return results;
}
finally
{
if (temp != null && temp.ID != 0)
{
ctx.TempTables.Remove(temp);
ctx.SaveChanges();
}
}
}
The idea is that as part of the processing of a request I need to build a temporary table with some data that then gets used to join the main query and filter the results. Once the query has been processed, the temp table should be deleted. I put the deletion part in the finally clause so that if there is a problem with the query (an exception thrown), the temporary table will always get cleaned up.
This seems to work fine, except intermittently I have a problem were the SaveChanges in the finally block throws a deadlock exception with an error message along the lines of:
Transaction (Process ID 89) was deadlocked on lock resources with another process and
has been chosen as the deadlock victim. Rerun the transaction.
I can't reliably reproduce it, but it seems to happen most often if the previous query threw the "no results" exception. Note that, due to an error that was discovered on the front end, two identically requests were being submitted under certain circumstances, but nevertheless, the code should be able to handle that.
Does anybody have an clues as to what might be happening here? Is throwing an exception inside the using block a problem? Should I handle that differently?
Update, so the exception might be a red herring. I removed it altogether (instead returning an empty result) and I still have the problem. I've tried a bunch of variations on:
using (new TransactionScope(TransactionScopeOption.Required, new TransactionOptions { IsolationLevel = IsolationLevel.ReadUncommitted })
using (var ctx = new Dal.MyEntities())
{
}
But despite what I've read, it doesn't seem to make any difference. I still get intermittent deadlocks on the second SaveChanges to remove the temp table.
how about adding a
using (var ctx = new Dal.MyEntities())
{
try
{
//...
Dal.TempTable temp = new Dal.TempTable();
//...
ctx.TempTables.Add(temp);
// some query that joins on the temp table...
if (no
results are
returned)
{
throw new Exception("no results")
}
// Normal processing and return result
}
catch
{
ctx.TempTables.Remove(temp);
ctx.SaveChanges();
}
finally
{
if (temp != null && temp.ID != 0)
{
ctx.TempTables.Remove(temp);
ctx.SaveChanges();
}
}
}
Related
my requirement is to process multiple cost files, which has million of records. after processing and validation I have to add those records to database.
For better performance I am using "yield" in foreach loop and return one record at a time, process that record and immediately add that one record to database with file number. During this file reading process if I come across any data validation error, I throw InvalidRecordException.
My requirement is to delete all the records from table related that file. in short, even if one record is invalid I want to mark that file as invalid file and not add even a single record of that file to database.
can anyone help me here, how can i make use of TransactionScope here.
public class CostFiles
{
public IEnumerable<string> FinancialRecords
{
get
{
//logic to get list of DataRecords
foreach (var dataRecord in DataRecords)
{
//some processing... which can throw InvalidRecord exception
yield return dataRecord;
}
yield break;
}
}
}
public void ProcessFileRecords(CostFiles costFile, int ImportFileNumber)
{
Database db = new Database();
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.Required))
{
try
{
foreach (var record in costFile.FinancialRecords)
{
db.Add(record, ImportFileNumber);
}
}
catch(InvalidRecordException ex)
{
//here i want to delete all the records from the table where import file number is same as input paramter ImportFileNumber
}
}
}
The purpose of a transaction scope is to create an "all or nothing" scenario, so either the whole transaction commits, or nothing at all commits. It looks like you already have the right idea (at least in terms of the TransactionScope. The scope won't actually commit the records to the database until you call TransactionScope.Complete(). If Complete() is not called, then the records are discarded when you leave the transaction scope. You could easily do something like this:
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.Required))
{
bool errorsEncountered = false;
try
{
foreach (var record in costFile.FinancialRecords)
{
db.Add(record, ImportFileNumber);
}
}
catch(InvalidRecordException ex)
{
//here i want to delete all the records from the table where import file number is same as input paramter ImportFileNumber
errorsEncountered = true;
}
if (!errorsEncountered)
{
scope.Complete();
}
}
Or you can just let the Add throw an exception and handle it outside of the transaction scope instead, as the exception will cause Complete() not to be called, and therefore no records added. This method has the additional advantage of stopping processing of additional records when we already know it will do nothing.
try
{
using (var scope = new TransactionScope(TransactionScopeOptions.Required))
{
foreach(var record in costFile.FinancialRecords)
{
db.Add(record, ImportFileNumber);
}
// if an exception is thrown during db.Add(), then Complete is never called
scope.Complete()
}
catch(Exception ex)
{
// handle your exception here
}
}
EDIT If you don't want your transaction elevated to a distributed transaction (which may have additional security/network requirements), make sure you reuse the same SqlConnection object for every database call within your transaction scope.
using (var conn = new SqlConnection("myConnectionString"))
{
conn.Open();
using (var scope = new TransactionScope(...))
{
foreach(var foo in foos)
{
db.Add(foo, conn);
}
scope.Complete();
}
}
The following code is causing an intermittent exception:
public int UnblockJob(int jobId)
{
using (var connect = MakeConnect())
{
var tag = connect.JobTag.SingleOrDefault(jt => jt.JobId == jobId && jt.Name == Metrics.TagNameItemBlockCaller);
if (tag == null)
{
return 0;
}
connect.JobTag.Remove(tag);
return connect.SaveChanges();
}
}
How can I correct or troubleshoot it?
From the documentation for DbUpdateConcurrencyException:
Exception thrown by DbContext when it was expected that SaveChanges for an entity would result in a database update but in fact no rows in the database were affected.
This means that the record you are attempting to delete has since been removed from the database. It would appear that you have another process that is deleting records or this function is able to be called concurrently.
There are several solutions, here are a couple:
Fix the source problem Stop other processes affecting the data.
Catch the error Wrap this method in a try/catch block, after all you may only care that the record has been deleted:
try
{
//Existing code here
}
catch(DbUpdateConcurrencyException)
{
//Safely ignore this exception
}
catch(Exception e)
{
//Something else has occurred
throw;
}
I'm getting strange things since updated to EF6,no sure this is related or not, but used to be good
I'm doing a set of work, then save it to DB , then do another , save another.
after a while,i check SQL server by sp_who2 , i found many dead connections from my computer.
Job is huge then there goes to 700 connections,
I have to kill them all manually in cycle.
program like:
while (jobDone == false)
{
var returnData=doOneSetJob();
myEntity dbconn= new myEntity;
foreach( var one in retrunData)
{
dbconn.targetTable.add(one );
try
{
dbconn.savechange();
/// even i put a dispose() here , still lots of dead connections
}
catch
{
console.writeline("DB Insertion Fail.");
dbconn.dispose();
dbconn= new myEntity();
}
}
dbconn.dispose()
}
You should consider refactoring your code so that your connection is cleaned up after your job is complete. For example:
using (var context = new DbContext())
{
while (!jobDone)
{
// Execute job and get data
var returnData = doOneSetJob();
// Process job results
foreach (var one in returnData)
{
try
{
context.TargetTable.Add(one);
context.SaveChanges();
}
catch (Exception ex)
{
// Log the error
}
}
}
}
The using statement will guarantee that your context is cleaned up properly, even if an error occurs while you are looping through the results.
In this case you should use a using statement. Taken from MSDN:
The using statement ensures that Dispose is called even if an exception occurs while you are calling methods on the object. You can achieve the same result by putting the object inside a try block and then calling Dispose in a finally block; in fact, this is how the using statement is translated by the compiler.
So, your code would look better like this:
using(var dbconn = new DbContext())
{
while (!jobDone)
{
foreach(var one in retrunData)
{
try
{
targetTable row = new TargetTable();
dbconn.TargetTable.add(row);
dbconn.SaveChanges();
}
catch (Exception ex)
{
Console.WriteLine("DB Insertion Fail.");
}
}
}
}
This way, even if your code fails at some point, the Context, resources and connections will be properly disposed.
Below is a simplified code of what I currently have.
I noticed that "status" remained as 0 instead of 1 or 2 when Point 1/2 throws an exception.
I initially thought Update() would do the trick, but it seems like I have to call Commit() for the changes to be in the DB.
What could be a good way for me to do this?
(showing status of 1 and 2 in DB upon returning/exception).
Any help is much appreciated.
using(var tx = session.BeginTransaction())
{
Monitor monitor = monitorDao.Get(id);
if (someStatus)
{
monitor.status = 1; // initially monitor.status == 0 in DB
// Point 1: some codes that might return or throw exception
}
else
{
monitor.status = 2;
// Point 2: some codes that might return or throw exception
}
monitor.status = 3;
tx.Commit();
}
It seems you have to refactor you code to get the behavior you want:
Monitor monitor = monitorDao.Get(id);
if (someStatus)
{
using(var tx = session.BeginTransaction())
{
monitor.status = 1; // initially monitor.status == 0 in DB
tx.Commit();
}
// Point 1: some codes that might return or throw exception
}
else
{
using(var tx = session.BeginTransaction())
{
monitor.status = 2; // initially monitor.status == 0 in DB
tx.Commit();
}
// Point 2: some codes that might return or throw exception
}
using(var tx = session.BeginTransaction())
{
monitor.status = 3;
tx.Commit();
}
When you had an exception, the first status update was never send to your db. By giving it a separate transaction, you first update your db by committing it. Then you do you logic which can fail. If you are doing anything at Point 1/2 which like saving to the db and what has to be in the transaction with status = 3. Then you need to refactor your code so that logic is in the second transaction.
This setting is controlled by FlushMode on ISession. You should google this topic, for example this link gives some more details on FlushMode options: http://weblogs.asp.net/ricardoperes/nhibernate-pitfalls-flush-mode.
Never: changes are never flushed automatically, you must call
ISession.Flush() explicitly;
Commit: changes are sent as soon as the
current ITransaction is committed, no need to call Flush();
Auto: the
session is flushed if a query is requested for some entity type and
there are dirty local entity instances, no need to call Flush(); this
is the default;
Always: the session is flushed before any query is
executed, also no need to call Flush().
When I iterate through a foreach with the following code it successfully catches the first exception that occurs and adds the id to my error list. On all the subsequent iterations of the loop, it will continue to catch the previous exception.
How can I appropriately catch the exception and undo or clear the failed DeleteObject request so that subsequent deletes can be performed.
public ActionResult Delete(int[] ListData)
{
List<int> removed = new List<int>();
List<int> error = new List<int>();
Item deleteMe;
foreach (var id in ListData)
{
deleteMe = this.getValidObject(id);
if (deleteMe == null)
{
error.Add(id);
continue;
}
try
{
this.DB.Items.DeleteObject(deleteMe);
this.DB.SaveChanges();
removed.Add(id);
}
catch (DataException ex)
{
// revert change to this.DB.Items?
error.Add(id);
}
}
if (error.Count > 0)
{
return Json(new { Success = false, Removed = removed, Error = error });
}
return Json(new { Success = true, Removed = removed });
}
I have searched SO and google and most people will process all the delete objects first and then save changes so that it is one transaction. But I need it to process each transaction individually so a single failure does not stop the rest of the transactions.
I am using Entity Framework 4.
The exception I get for this specific example caused by foreign keys being associated to the item that is being removed. While in production I will be handling this scenario, it should be able to continue on no matter what the exception is.
I assume that the the same context, this.DB, is being used in this.getValidObject(id) to retrieve an entity. If that is the case, in the exception block call: this.DB.detach(deleteme). That should prevent the SaveChanges() to try to delete the problematic entity on the next iteration.
The code you present looks good. What is the error you see? As you've noted, maybe you need to un-tag something in this.DB.Items, though I don't think so. You could also try creating a new DataContext for each loop such that the old, failed DataContext's state on the world is irrelevant.
If I understood correctly, you cannot remove the entity(Item) because it has a foreign key association(child) to it.
You will first have to update all child(related) entities using the Parent(Item) you want to delete, by removing the relationship, updating the entity to relate too an alternative parent(Item) or deleting the child entity(entities) and then finally removing the Parent(Item) entity.