i'm stuck within a problem that is going to happen to everyone in ADO.NET mixed with Entity Framework contexts.
I've got a big procedure that handles and saves data into database by using multiple ways like ADO.NET dataadapters and direct CRUD commands against DB. All the procedure is wrapped by 2 using() blocks that creates and releases a DBConnection/DBTransaction and a try/catch block to commit or rollback the transaction. Unfortunately, in the middle of this routine, i have to recall a saving procedure implemented by using Entity Framework. This leads me to a problem:
According to the official documentation, Entity Framework 5 allows me to pass a connection with an transaction associated transaction (it should work, in debug mode, when i call SaveChanges() i don't receive any TimeOutException due to deadlocks, conversely if i pass a new connection it does), but unfortunately after SaveChanges() kicks in the connection is closed and associated transaction committed! Even if i set the flag 'contextOwnsConnection'!
As far as i know, if i migrate EF5 to EF6, things should work (am i right?), but unfortunately i can't, because the project i'm working on is very large and involves a lot of dependecies and it would take a large amount of time.
How can i make it work with EF5? Is there any trick or pattern to achieve the desired result? Am i right about the behavior of EF6? Does it worth EF6 migration?
Here you are a simple example of how does my code looks like.
For privacy reasons i can't post the original code but just imagine a situation like this with a lot of more complexity:
using(DbConnection conn = DBProvider.CreateConnection()){
//Open the created connection
conn.Open();
//Create a new transaction
using(DbTransaction tr = DBProvider.CreateTransaction()){
//Begin a new transaction
tr.Begin();
bool saveOk;
try{
//Updates customers by using dataadapter
dataAdapterCustomers.InsertCommand.Transaction = tr;
dataAdapterCustomers.UpdateCommand.Transaction = tr;
dataAdapterCustomers.DeleteCommand.Transaction = tr;
dataAdapterCustomers.Update();
//Updates stock items by using dataadapter
stockAdapterCustomers.InsertCommand.Transaction = tr;
stockAdapterCustomers.UpdateCommand.Transaction = tr;
stockAdapterCustomers.DeleteCommand.Transaction = tr;
stockAdapterCustomers.Update();
//...Many other DB accessing here...
//Updates stock quantity by using simple DBCommand
quantityUpdateCmd.Transaction = tr;
quantityUpdateCmd.ExecuteNonQuery();
//Updates stock statistics by using a simple DBCommand
updateStockStatsCmd.Transaction = tr;
updateStockStatsCmd.ExecuteNonQuery();
//...Many other DB accessing here...
//HERE:
//Creates a new activity and save it using EF.
//I use a UnitOfWork and i pass to it my connection and 'false' as contextOwnsConnection parameter
//(it 'll be used by the DBContext contained in my Unit of work)
using(ActivityUoW uow = new ActivityUoW(conn, false)){
Activity act = new Activity();
act.Name = "Saving activity";
act.Description = "Done by user";
act.Date = DateTime.Now;
uow.Activities.Add(act);
uow.SaveChanges();
}
//Based on activity result, launch a store procedure that makes other complex things.
UNFORTUNATELY THE CONNECTION HAS BEEN CLOSED AND TRANSACTION COMMITTED, SO THE FOLLOWING INSTRUCTION WILL FAIL.
launchActivityUpdateSpCmd.Transaction = tr;
launchActivityUpdateSpCmd.ExecuteNonQuery();
//...Many other DB accessing here...
//Data saved correctly
saveOk = true;
}
catch(Exception ex){
//There was an error during save
saveOk = false;
}
//Commit or rollback transaction according to save procedure result
if(saveOk)
tr.Commit();
else
tr.Rollback();
}
}
I didn't quite follow your question(s) and wasn't sure if your issue was related to how to handle transactions or if you had a question about EF5 to EF6 migrations. That being said, you have an interesting mixture of data access code.
Regarding transactions - I would look into using the TransactionScope which is part of the System.Transactions namespace.
For example:
try
{
using (var scope = new TransactionScope())
{
using (var conn = new SqlConnection("your connection string"))
{
conn.Open();
// your EF and ADO.NET code
}
scope.Complete();
}
}
catch (TransactionAbortedException ex)
{
}
catch (ApplicationException ex)
{
}
Related
I have an asp.net application that inserts to several tsql tables. The format is the following:
Car myCar = new Car();
myCar.InsertNewCar(); //Makes SP call usp_InsertCar()
Truck myTruck = new Truck();
myTruck.InsertNewTruck(); //Makes SP call usp_InsertTruck()
Customer myCustomer = new Customer();
myCustomer.InsertNewCustomers(); //Makes SP call usp_InsertCustomers()
Each of these methods have a try...catch exception. The problem is that it may break in myCustomer.InsertNewCustomers(), but the 2 previous inserts have been completed. I then have to manually delete all inserts and then try again. We already do data check, but this still happens.
I was considering using COMMIT & ROLLBACK with each of these stored procedures, so the first pass would call all 5 methods that would rollback the transaction. If everything was OK, then I'd call the same stored procedures with a COMMIT. That way I can be certain that the inserts will be made correctly.
Would that make sense?
You can manage the transactions from your C# code. Add a using (TransactionScope scope = new TransactionScope()) block to wrap your calls. I think you might need to use the same connection for this though - so you could have to pass the connection object as a parameter to your methods.
Edit: Just to add some more detail....
using (TransactionScope scope = new TransactionScope())
{
// Assuming you are using SQL Server....
using (SqlConnection conn = new SqlConnection(connectString1))
{
conn.Open();
InsertToTable1(conn);
// Snip...
InsertToTable5(conn);
// If this point is reached, everything is tickety boo
// Commit the transaction using Complete.
// If the scope.Complete line is not hit before the using block
// is exited (i.e. an Exception is thrown, the transaction is rolled
// back.
scope.Complete();
}
}
Your InsertIntoTable methods would then look something like
public void InsertIntoTable1(SqlConnection conn)
{
// Some non-database code could be here....
SqlCommand cmd = conn.CreateCommand();
// Configure command and execute
// Some non-database code could also be here....
}
Due to a few restrictions I can't use entity Framework and thus need to use SQL Connections, commands and Transactions manually.
While writing unit tests for the methods calling these data layer operations I stumbled upon a few problems.
For the unit tests I NEED to do them in a Transaction as most of the operations are changing data by their nature and thus doing them outside a Transaction is problematic as that would change the whole base data. Thus I need to put a Transaction around these (with no commit fired at the end).
Now I have 2 different variants of how These BL methods work.
A few have Transactions themselves inside of them while others have no Transactions at all. Both of these variants cause problems.
Layered Transaction: Here I get errors that the DTC cancelled the distributed Transaction due to timeouts (although the timeout is being set to 15 minutes and it is running for only 2 minutes).
Only 1 Transaction: Here I get an error about the state of the Transaction when I come to the "new SQLCommand" line in the called method.
My question here is what can I do to correct this and get unit testing with manual normal and layered Transactions working?
Unit testing method example:
using (SqlConnection connection = new SqlConnection(Properties.Settings.Default.ConnectionString))
{
connection.Open();
using (SqlTransaction transaction = connection.BeginTransaction())
{
MyBLMethod();
}
}
Example for a Transaction using method (very simplified)
using (SqlConnection connection = new SqlConnection(Properties.Settings.Default.ConnectionString))
{
connection.Open();
using (SqlTransaction transaction = connection.BeginTransaction())
{
SqlCommand command = new SqlCommand();
command.Connection = connection;
command.Transaction = transaction;
command.CommandTimeout = 900; // Wait 15 minutes before a timeout
command.CommandText = "INSERT ......";
command.ExecuteNonQuery();
// Following commands
....
Transaction.Commit();
}
}
Example for a non Transaction using method
using (SqlConnection connection = new SqlConnection(Properties.Settings.Default.ConnectionString))
{
connection.Open();
SqlCommand command = new SqlCommand();
command.Connection = connection;
command.CommandTimeout = 900; // Wait 15 minutes before a timeout
command.CommandText = "INSERT ......";
command.ExecuteNonQuery();
}
On the face of it, you have a few options, depending upon what you want to test and your ability to spend money / change your code base.
At the moment, you’re effectively writing integration tests. If the database isn’t available then your tests will fail. This means the tests can be slow, but on the plus side if they pass you’re pretty confident that you code can hit the database correctly.
If you don’t mind hitting the database, then the minimum impact to changing your code / spending money would be for you to allow the transactions to complete and verify them in the database. You can either do this by taking database snapshots and resetting the database each test run, or by having a dedicated test database and writing your tests in such a way that they can safely hit the database over and over again and then verified. So for example, you can insert a record with an incremented id, update the record, and then verify that it can be read. You may have more unwinding to do if there are errors, but if you’re not modifying the data access code or the database structure that often then this shouldn’t be too much of an issue.
If you’re able to spend some money and you want to actually turn your tests into unit tests, so that they don’t hit the database, then you should consider looking into TypeMock. It’s a very powerful mocking framework that can do some pretty scary stuff. I believe it using the profiling API to intercept calls, rather than using the approach used by frameworks like Moq. There's an example of using Typemock to mock a SQLConnection here.
If you don’t have money to spend / you’re able to change your code and don’t mind continuing to rely on the database then you need to look at some way to share your database connection between your test code and your dataaccess methods. Two approaches that spring to mind are to either inject the connection information into the class, or make it available by injecting a factory that gives access to the connection information (in which case you can inject a mock of the factory during testing that returns the connection you want).
If you go with the above approach, rather than directly injecting SqlConnection, consider injecting a wrapper class that is also responsible for the transaction. Something like:
public class MySqlWrapper : IDisposable {
public SqlConnection Connection { get; set; }
public SqlTransaction Transaction { get; set; }
int _transactionCount = 0;
public void BeginTransaction() {
_transactionCount++;
if (_transactionCount == 1) {
Transaction = Connection.BeginTransaction();
}
}
public void CommitTransaction() {
_transactionCount--;
if (_transactionCount == 0) {
Transaction.Commit();
Transaction = null;
}
if (_transactionCount < 0) {
throw new InvalidOperationException("Commit without Begin");
}
}
public void Rollback() {
_transactionCount = 0;
Transaction.Rollback();
Transaction = null;
}
public void Dispose() {
if (null != Transaction) {
Transaction.Dispose();
Transaction = null;
}
Connection.Dispose();
}
}
This will stop nested transactions from being created + committed.
If you’re more willing to restructure your code, then you might want to wrap your dataaccess code in a more mockable way. So, for example you could push your core database access functionality into another class. Depending on what you’re doing you’ll need to expand on it, however you might end up with something like this:
public interface IMyQuery {
string GetCommand();
}
public class MyInsert : IMyQuery{
public string GetCommand() {
return "INSERT ...";
}
}
class DBNonQueryRunner {
public void RunQuery(IMyQuery query) {
using (SqlConnection connection = new SqlConnection(Properties.Settings.Default.ConnectionString)) {
connection.Open();
using (SqlTransaction transaction = connection.BeginTransaction()) {
SqlCommand command = new SqlCommand();
command.Connection = connection;
command.Transaction = transaction;
command.CommandTimeout = 900; // Wait 15 minutes before a timeout
command.CommandText = query.GetCommand();
command.ExecuteNonQuery();
transaction.Commit();
}
}
}
}
This allows you to unit test more of your logic, like the command generation code, without having to actually worry about hitting the database and you can test your core dataaccess code (the Runner) against the database once, rather than for every command you want to run against the database. I would still write integration tests for all dataaccess code, but I’d only tend to run them whilst actually working on that section of code (to ensure column names etc have been specified correctly).
I've looked around but couldn't find an appropriate (or one that is satisfactory to me) on how to address an issue we are having.
I use Dapper and ODP.NET 12 Managed drivers. The problem is not encountered when TransactionScope is not used.
When performing commands under a transaction scope, I get an error "Transaction has aborted" via the TransactionAbortedException thrown.
Observed behaviors:
1) TransactionAbortedException is thrown if and only if the transaction is completed and the TransactionScope is disposed. The point at when the exception is thrown is during dispose.
2) Despite the exception, the transaction concept actually works! After Complete() is invoked, the changes are committed into the database.
Below is the code snippet.
// Conn string: "Data Source=OraDB;Persist Security Info=True;User ID=userxxx;Password=passwordxxx;" providerName="Oracle.ManagedDataAccess.Client
// Note: GetDbFactory().Create() returns a DbConnection object
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.RequiresNew, new TransactionOptions { IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted }))
using (var dbConn = GetDbFactory().Create())
{
foreach (MyDTO dto in dtoList)
{
var tableDAO= new TableDAO(dbConn);
MyEntity entity = new MyEntity()
{
Field1 = dto.Field1,
Field2 = dto.Field2
};
tableDAO.AddOrUpdate(entity);
}
// Commit changes
scope.Complete();
}
// This method is under the DAO class
public void AddOrUpdate(MyEntity entity)
{
// Verify arguments
entity.AsArgumentThrowExceptionIfNull("entity");
// build param
OracleDynamicParameters parameters = new OracleDynamicParameters();
parameters.Add("P_FIELD1", entity.Field1);
parameters.Add("P_FIELD2", entity.Field2);
// execute SP
dbConnection.Execute("PRC_MY_ENTITY_ADDORUPDATE", parameters, commandType: CommandType.StoredProcedure);
}//-- end AddOrUpdate()
==================================================================
UPDATE (09-Apr-15)
I have changed my approach and use the following pattern for now for Oracle. Our code deals with connections in both Oracle and SQL Server so I'd much prefer that the coding pattern is consistent, but until a solution is found with using Oracle+TransactionScope, we'll use the pattern below for Oracle command execution:
using (var dbConnection = dbConnFactory.Create())
{
// Open db connection
dbConnection.Open();
using (var trans = dbConnection.BeginTransaction())
{
bool isSuccess = false;
try
{
// Perform DB operations here
trans.Commit();
isSuccess = true;
}
finally
{
if(!isSuccess) trans.Rollback();
}
}
}
First, the exception is documented as possible:
A call to the Dispose method marks the end of the transaction scope. Exceptions that occur after calling this method may not affect the transaction.
This is in the documentation on the TransactionScope class (https://msdn.microsoft.com/en-us/library/system.transactions.transactionscope%28v=vs.110%29.aspx).
One other thing I noticed is that the complete on the transaction is being before the connection is closed. I'd change this to close the connection and then complete the transaction.
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.RequiresNew, new TransactionOptions { IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted }))
{
using (var dbConn = GetDbFactory().Create())
{
foreach (MyDTO dto in dtoList)
{
var tableDAO= new TableDAO(dbConn);
MyEntity entity = new MyEntity()
{
Field1 = dto.Field1,
Field2 = dto.Field2
};
tableDAO.AddOrUpdate(entity);
}
}
// Commit changes
scope.Complete();
}
I don't see anything wrong with that code. However, if that loop runs long enough, the transaction is going to timeout. You'll then get the exception in question the next time you do an operation against the database. I would try increasing the timeout - Timeout is a property on the TransactionScopeOption class.
I am resorting to using BeginTransaction() as the final approach (refer to my update in my original post). I have read more about why TransactionScope() was failing.
1) ODP.Net promotes to distributed transaction even when using a single DB connection when connecting to Oracle 10g and below (source). Lo and behold, the database I'm connecting to is indeed 10g.
2) You'll need Oracle MTS Service installed. This I didn't have setup on my dev machine.
I need to insert some objects (about 4 million) in the database using C# and EF (using .NET 3.5). My method that adds the objects is in a for:
private DBModelContainer AddToContext(DBModelContainer db, tblMyTable item, int count)
{
db.AddTottblMyTable (item);
if ((count % 10000== 0) || (count == this.toGenerate))
{
try
{
db.SaveChanges();
}
catch (Exception e)
{
Console.WriteLine(e.StackTrace);
}
}
return db;
}
How to detach the added objects (of type tblMyTable) from the context object? I don't need them for a later use and when more than 300000 objects are added, the execution time between db saving ( db.SaveChanges()) increases considerably.
Regards
Entity Framework may not be the best tool for this type of operation. You may be better off with plain ADO.Net, some stored procedures... But if you had to use it, here are a number of suggestions:
Keep the active Context Graph small by using a new context for each
Unit of Work
Turn off AutoDetechChangesEnabled - context.Configuration.AutoDetectChangesEnabled = false;
Batching, in your loop, Call SaveChanges periodically
EDIT
using(var db = new DBModelContainer())
{
db.tblMyTable.MergeOption = MergeOption.NoTracking;
// Narrow the scope of your db context
db.AddTottblMyTable (item);
db.SaveChanges();
}
Keeping a long running db context is not advisable, so consider refactoring your Add method to not keep attempting to reuse the same context.
See Rick Strahl's post on bulk inserts for more details
AFAK EF does not support directly the BulkInsert so it will be tedious to do such thing manually.
try to consider EntityFramework.BulkInsert
using (var ctx = GetContext())
{
using (var transactionScope = new TransactionScope())
{
// some stuff in dbcontext
ctx.BulkInsert(entities);
ctx.SaveChanges();
transactionScope.Complete();
}
}
You may try Unit Of Work and dont save context (SaveChanges) on every record insert but save it at end
I would like to run multiple insert statements on multiple tables. I am using dapper.net. I don't see any way to handle transactions with dapper.net.
Please share your ideas on how to use transactions with dapper.net.
Here the code snippet:
using System.Transactions;
....
using (var transactionScope = new TransactionScope())
{
DoYourDapperWork();
transactionScope.Complete();
}
Note that you need to add reference to System.Transactions assembly because it is not referenced by default.
I preferred to use a more intuitive approach by getting the transaction directly from the connection:
// This called method will get a connection, and open it if it's not yet open.
using (var connection = GetOpenConnection())
using (var transaction = connection.BeginTransaction())
{
connection.Execute(
"INSERT INTO data(Foo, Bar) values (#Foo, #Bar);", listOf5000Items, transaction);
transaction.Commit();
}
There are 3 approaches to doing transactions in Dapper.
Simple Transaction
Transaction from Transaction Scope
Using Dapper Transaction (additional nuget package and most favored approach)
You can find out more about these transaction approaches from the official tutorial website here
For reference here's a breakdown of the transaction approaches
1. Simple Transaction
In this example, you create a transaction on an existing db connection, and then pass in the transaction to the Execute method on dapper (which is an optional parameter).
Once you've done all your work, simply commit the transaction.
string sql = "INSERT INTO Customers (CustomerName) Values (#CustomerName);";
using (var connection = new SqlConnection(FiddleHelper.GetConnectionStringSqlServerW3Schools()))
{
connection.Open();
using (var transaction = connection.BeginTransaction())
{
connection.Execute(sql, new {CustomerName = "Mark"}, transaction: transaction);
connection.Execute(sql, new {CustomerName = "Sam"}, transaction: transaction);
connection.Execute(sql, new {CustomerName = "John"}, transaction: transaction);
transaction.Commit();
}
}
2. Transaction from Transaction Scope
If you'd like to create a transaction scope, you will need to do this before the db connection is created. Once you've created the transaction scope, you can simply perform all your operations and then do a single call to complete the transaction, which will then commit all the commands
using (var transaction = new TransactionScope())
{
var sql = "INSERT INTO Customers (CustomerName) Values (#CustomerName);";
using (var connection = My.ConnectionFactory())
{
connection.Open();
connection.Execute(sql, new {CustomerName = "Mark"});
connection.Execute(sql, new {CustomerName = "Sam"});
connection.Execute(sql, new {CustomerName = "John"});
}
transaction.Complete();
}
3. Using Dapper Transaction
In my opinion, this is the most favorable approach to achieve transaction in code, because it makes the code easy to read and easy to implement. There is an extended implementation of SQL Transaction called Dapper Transaction (which you can find here), which allows you to run the SQL executes off the transactions directly.
string sql = "INSERT INTO Customers (CustomerName) Values (#CustomerName);";
using (var connection = new SqlConnection(FiddleHelper.GetConnectionStringSqlServerW3Schools()))
{
connection.Open();
using (var transaction = connection.BeginTransaction())
{
transaction.Execute(sql, new {CustomerName = "Mark"});
transaction.Execute(sql, new {CustomerName = "Sam"});
transaction.Execute(sql, new {CustomerName = "John"});
transaction.Commit();
}
}
You should be able to use TransactionScope since Dapper runs just ADO.NET commands.
using (var scope = new TransactionScope())
{
// open connection
// insert
// insert
scope.Complete();
}
Considering all your tables are in single database, I disagree with TransactionScope solution suggested in some answers here. Refer this answer.
TransactionScope is generally used for distributed transactions; transaction spanning different databases may be on different system. This needs some configurations on operating system and SQL Server without which this will not work. This is not recommended if all your queries are against single instance of database.
But, with single database this may be useful when you need to include the code in transaction that is not under your control. With single database, it does not need special configurations either.
connection.BeginTransaction is ADO.NET syntax to implement transaction (in C#, VB.NET etc.) against single database. This does not work across multiple databases.
So, connection.BeginTransaction() is better way to go.
Even the better way to handle the transaction is to implement UnitOfWork as explained in this answer.
Daniel's answer worked as expected for me. For completeness, here's a snippet that demonstrates commit and rollback using a transaction scope and dapper:
using System.Transactions;
// _sqlConnection has been opened elsewhere in preceeding code
using (var transactionScope = new TransactionScope())
{
try
{
long result = _sqlConnection.ExecuteScalar<long>(sqlString, new {Param1 = 1, Param2 = "string"});
transactionScope.Complete();
}
catch (Exception exception)
{
// Logger initialized elsewhere in code
_logger.Error(exception, $"Error encountered whilst executing SQL: {sqlString}, Message: {exception.Message}")
// re-throw to let the caller know
throw;
}
} // This is where Dispose is called