For an application we are developing we need to read n rows from a table and then selectively update those rows based on domain specific criteria. During this operation all other users of the database need to be locked out to avoid bad reads.
I begin a transaction, read the rows, and while iterating on the recordset build up a string of update statements. After I'm done reading the recordset, I close the recordset and run the updates. At this point I commit the transaction, however none of the updates are being performed on the database.
private static SQLiteConnection OpenNewConnection()
{
try
{
SQLiteConnection conn = new SQLiteConnection();
conn.ConnectionString = ConnectionString;//System.Configuration.ConfigurationManager.AppSettings["ConnectionString"];
conn.Open();
return conn;
}
catch (SQLiteException e)
{
LogEvent("Exception raised when opening connection to [" + ConnectionString + "]. Exception Message " + e.Message);
throw e;
}
}
SQLiteConnection conn = OpenNewConnection();
SQLiteCommand command = new SQLiteCommand(conn);
SQLiteTransaction transaction = conn.BeginTransaction();
// Also fails transaction = conn.BeginTransaction();
transaction = conn.BeginTransaction(IsolationLevel.ReadCommitted);
command.CommandType = CommandType.Text;
command.Transaction = transaction;
command.Connection = conn;
try
{
string sql = "select * From X Where Y;";
command.CommandText = sql;
SQLiteDataReader ranges;
ranges = command.ExecuteReader();
sql = string.Empty;
ArrayList ret = new ArrayList();
while (MemberVariable > 0 && ranges.Read())
{
// Domain stuff
sql += "Update X Set Z = 'foo' Where Y;";
}
ranges.Close();
command.CommandText = sql;
command.ExecuteNonQuery();
// UPDATES NOT BEING APPLIED
transaction.Commit();
return ret;
}
catch (Exception ex)
{
transaction.Rollback();
throw;
}
finally
{
transaction.Dispose();
command.Dispose();
conn.Close();
}
return null;
If I remove the transaction everything works as expected. The "Domain stuff" is domain specfic and other than reading values from the recordset doesn't access the database. Did I forget a step?
When you put a breakpoint on your transaction.Commit() line do you see it getting hit?
Final answer:
SQLite's locking does not work like you're assuming see http://www.sqlite.org/lockingv3.html. Given that, I think you're having a transaction scoping issue which can be easily resolved by reorganizing your code as such:
string selectSql = "select * From X Where Y;";
using(var conn = OpenNewConnection()){
StringBuilder updateBuilder = new StringBuilder();
using(var cmd = new SQLiteCommand(selectSql, conn))
using(var ranges = cmd.ExecuteReader()) {
while(MemberVariable > 0 && ranges.Read()) {
updateBuilder.Append("Update X Set Z = 'foo' Where Y;");
}
}
using(var trans = conn.BeginTransaction())
using(var updateCmd = new SQLiteCommand(updateBuilder.ToString(), conn, trans) {
cmd.ExecuteNonQuery();
trans.Commit();
}
}
Additional notes regarding some comments in this post/answer about transactions in SQLite. These apply to SQLite 3.x using Journaling and may or may not apply to different configurations - WAL is slightly different but I am not familiar with it. See locking in SQLite for the definitive information.
All transactions in SQLite are SERIALIZABLE (see the read_uncommitted pragma for one small exception). A new read won't block/fail unless the write process has started (there is an EXCLUSIVE/PENDING lock held) and a write won't start until all outstanding reads are complete and it can obtain an EXCLUSIVE lock (this is not true for WAL but the transaction isolation is still the same).
That is the entire sequence above won't be atomic in code and the sequence may be read(A) -> read(B) -> write(A) -> read(B), where A and B represent different connections (imagine on different threads). At both read(B) the data is still consistent even though there was a write in-between.
To make the sequence of code itself atomic a lock or similar synchronization mechanism is required. Alternatively, the lock/synchronization can be created with SQLite itself by using a locking_mode pragma of "exclusive". However, even if the code above is not atomic the data will adhere to the SQL serializable contract (excluding a serious bug ;-)
Happy coding
See Locking in SQLite, SQLite pragmas and Atomic Commit in SQLite
Related
I have referred this to perform rollback operation in my wpf c# application. The code that I tried is as follows:
using (OdbcConnection connection = new OdbcConnection("connectionString"))
{
OdbcCommand command = new OdbcCommand();
OdbcTransaction transaction = null;
command.Connection = connection;
try
{
connection.Open();
transaction = connection.BeginTransaction();
command.Connection = connection;
command.Transaction = transaction;
command.CommandText = "INSERT INTO TableA (A, B, C) VALUES (10,10,10)";
command.ExecuteNonQuery();
command.CommandText = "NSERT INTO TableB (D,E,F) VALUES (20,20,20)";
command.ExecuteNonQuery();
transaction.Commit();
}
catch(Exception ex)
{
Console.WriteLine(ex.Message);
try
{
transaction.Rollback();
}
catch
{
}
}
Intentionally the second query has been made wrong. My intention is that when I enter the catch block on calling transaction.Rollback() the values added due to executing of the first query in TableA are not reflected since Rollback was called. However this is not the case the values are not rolledback and are present in TableA. I have searched various resources online with no luck. I cannot use SqlConnection instead of OdbcConnection my application does not support that. Is there any work around this or alternative method that can achieve what I have in mind. Please help me out.
You have basically MSDN example. I had once another problem with ODBC and the issue was with ODBC vendor drivers. I would strongly recommend check that possibility.
Ok so i have a webform and 5 FileUpload control..a user can upload any number of files from 1 to 5 but if anyone of these files does not get uploaded then I want to rollback everything...
For ex:
if user has selected 4 files and if something unexpected occurs at 4th then I want to remove or rollback all the previous 3 file uploads..
I tried this..
try
{
using (TransactionScope scope = new TransactionScope())
{
dboperation dbinsert=new dboperation();
if (file1.ContentLength > 0)
{
.......
.......
dbo.insert(bytes, lastid, file2.FileName);
}
if (file2.ContentLength > 0)
{
.......
.......
dbo.insert(bytes, lastid, file2.FileName);
}
if (file3.ContentLength > 0)
{
.......
.......
dbo.insert(bytes, lastid, file2.FileName);
}//till ...file5
scope.Complete();
}//end of transactionscope
}
catch { }
'dboperation' is a class in c# file and 'dbinsert' is a method which is executing an insert stored procedure. My guess is that I need to use Transaction Scope but I am not sure if I am correct and even if I am how am I supposed to achieve this?
You need to implement transaction. You should start the transaction before inserting first one and catch any errors that occur. in case of error you have to rollback the transaction. And if all goes well you can commit your transaction.
You should also move you connection outside the dboperation or make a method in dboperation that takes connection from outside and uses that
for this you need to use Transaction something like this. I give you example.
class WithTransaction
{
public WithTransaction()
{
string FirstQuery = "INSERT INTO Table1 VALUES('Vineeth',24)";
string SecondQuery = "INSERT INTO Table2 VALUES('HisAddress')";
int ErrorVar = 0;
using (SqlConnection con = new SqlConnection("your connection string"))
{
try
{
SqlCommand ObjCommand = new SqlCommand(FirstQuery, con);
SqlTransaction trans;
con.Open();
trans = con.BeginTransaction();
ObjCommand.Transaction = trans;
//Executing first query
//What ever operation on your database do here
ObjCommand.ExecuteNonQuery(); //Exected first query
ObjCommand.CommandText = SecondQuery;
ObjCommand.ExecuteNonQuery(); //Exected first query
//Everything gone fine. So commiting
ObjCommand.Transaction.Commit();
}
catch (Exception ex)
{
Console.WriteLine("Error but we are rollbacking");
ObjCommand.Transaction.Rollback();
}
con.Close();
}
}
}
Or you can use TransactionScope
check this Link
TransactionScope
I hope this will help you.
I have the following method for bulk insert of the data in tables.
First my code populates the data in data tables and inserts this data in corresponding tables using the SqlBulkCopy claas of the .net .
I have requirement that data should get inserted in all tables or neither of them.
For this I have used SqlTransaction class of the .net.
Scenario is, multiple threads execute the following code block at the same time.
public void Import()
{
using (SqlConnection sqlConnection = new SqlConnection(connectionString))
{
SqlTransaction sqlTrans =null;
try
{
sqlConnection.Open();
sqlTrans = sqlConnection.BeginTransaction(IsolationLevel.Serializable)
SqlCommand cmd = sqlConnection.CreateCommand();
cmd.CommandText = "select top 1 null from lockTable with(xlock)";
cmd.CommandTimeout = 3600*3;
cmd.Transaction = sqlTrans;
SqlDataReader reader = cmd.ExecuteReader();
foreach (DataTable dt in DataTables)
{
ImportIntoDatabase(sqlConnection, dt, sqlTrans);
}
reader.Close();
sqlTrans.Commit();
}
catch (Exception ex)
{
sqlTrans.Rollback();
throw ex;
}
}
}
private void ImportIntoDatabase(SqlConnection sqlConn, DataTable dt, SqlTransaction sqlTrans)
{
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(sqlConn, SqlBulkCopyOptions.Default, sqlTrans))
{
bulkCopy.BulkCopyTimeout = dt.Rows.Count * 10;
try
{
bulkCopy.DestinationTableName = dt.TableName;
bulkCopy.WriteToServer(dt);
}
catch (Exception ex)
{
throw ex;
}
}
}
To handle this concurrency, I have created one dummy table(table named 'lockTable'), in the database where the other table resides(the bulk insert tables). I am getting exclusive lock on this dummy table in the SqlTransaction having command time out as high as 3 hours.
Problem:
I am getting following exception
: Cannot access destination table 'Tbl1' (tbl1 is the table for bulk inserting)
followed by another exception, while rolling back the transaction in catch block
: Error While executing activity The server failed to resume the transaction. Desc:3a00000001.
The transaction active in this session has been committed or aborted by another session.
Can any one help me for this weird behavior of the code. I have already searched a lot on this issue on the internet, but I have not found anything helpful for me.
In Import (DataTable dt in DataTables) is not going to be thread safe.
sqlConnection already has an active reader from Import so that connection cannot be used in ImportIntoDatabase.
Echo smp - if you are locking a table then why multi threads?
If you want to build up the input while the SQL inserts are taking place then use
Asynch method such as SqlCommand.BeginExecuteReader. You get asynch without the overhead of a thread. And DataTables are relatively slow. I insert using TVP and light weight objects. A huge factor in insert performance is index fragmentation. If at all possible insert order by the order of the clustered index. The loop is simple build input, wait for asynch, run asych. Or build input may be read input from a queue. SQL insert to the same table(s) are typically not going to go faster in parallel. My experience is ordered serial inserts with no gap in time between inserts.
I got my problem solved out.
Following are the changes which I have made to my Import method
public void Import()
{
using (SqlConnection sqlConnection = new SqlConnection(connectionString))
{
sqlConnection.Open();
using (SqlTransaction sqlTrans = sqlConnection.BeginTransaction())
{
try
{
SqlCommand cmd = sqlConnection.CreateCommand();
cmd.CommandText = "select top 1 null from lockTable with(xlock)";
cmd.CommandTimeout = LOCK_TIME_OUT;
cmd.Transaction = sqlTrans;
SqlDataReader reader = cmd.ExecuteReader();
foreach (DataTable dt in DataTables)
{
ImportIntoDatabase(sqlConnection, dt, sqlTrans);
}
reader.Close();
sqlTrans.Commit();
}
catch (Exception ex)
{
sqlTrans.Rollback();
throw ex;
}
}
sqlConnection.Close();
}
}
If multiple threads have access to the "Import" Method, then shouldn't you be locking the content of this method?
I don't think you need a dummy table, you just need to lock the two methods above.
I would also mention that you should join all threads, so that you can tell when they have finished.
would it work to dispose a command assigned to a transaction before the transaction is committed? I tested the following code myself, and it seems to have worked fine, but this is a rather small example, so I'm looking for confirmation if someone positively knows.
internal static void TestTransaction()
{
try
{
Program.dbConnection.Open();
using (SqlTransaction transaction = Program.dbConnection.BeginTransaction())
{
Boolean doRollback = false;
for (int i = 0; i < 10; i++)
{
using (SqlCommand cmd = new SqlCommand("INSERT INTO [testdb].[dbo].[transactiontest] (testvalcol) VALUES (#index)", Program.dbConnection, transaction))
{
cmd.Parameters.AddWithValue("#index", i);
try
{
cmd.ExecuteNonQuery();
}
catch (SqlException)
{
doRollback = true;
break;
}
}
}
if (doRollback)
transaction.Rollback();
else
transaction.Commit();
}
}
finally
{
Program.dbConnection.Close();
}
}
The connection, transaction and command objects are just vehicles to send commands to a database. Once a command is executed the database has received it. Whatever you do with the command object afterwards, dispose it, burn it, or shoot it to the moon, this fact does not change. (It can only be rolled back).
You can create and dispose as many commands as you like within the scope of one SqlConnection (with or without SqlTransaction). And you can start and dispose as many transactions as you like within one SqlConnection. To demonstrate this, see:
using (var conn = new SqlConnection(#"server=(local)\sql2008;database=Junk;Integrated Security=SSPI"))
{
conn.Open();
// Repeat this block as often as you like...
using (var tran = conn.BeginTransaction())
{
using (var cmd = new SqlCommand("INSERT INTO Mess VALUES ('mess1')", conn, tran))
{
cmd.ExecuteNonQuery(); // Shows in Sql profiler
}
tran.Commit(); // Commits
}
using (var cmd = new SqlCommand("INSERT INTO Mess VALUES ('mess2')", conn))
{
cmd.ExecuteNonQuery(); // Executes and commits (implicit transaction).
}
}
Of course, for healthy code you need to dispose of all objects in the correct order. Disposing a command after disposing a SqlConnection may cause the connection object to stay in memory.
Yes, it's probably safe. The using() is closing the Command, not the Connection.
But you should put that Connection in another using() block or in a try/finally construct.
Confirmed, this works very well and (at least here at our company) is even considered the correct approach.
create connection
create transaction
create command(s), use the transaction, execute
dispose command(s)
commit transaction
dispose connection
I'm trying to make a quick dummy app so I can learn the ins and outs of System.Transactions. This app interacts with 2 different SQLExpress DBs. If I pull up my transaction statistics in component services, I can see a transaction start up in the outerScope when the second connection is opened. If failOuter is true the transaction aborts, but it doesn't throw any exceptions. When failInner is true a TransactionAbortedException is thrown.
From MSDN:
When your application completes all the work it wants to perform in a transaction, you should call the Complete method only once to inform the transaction manager that it is acceptable to commit the transaction. It is very good practice to put the call to Complete as the last statement in the using block.
Failing to call this method aborts the transaction, because the transaction manager interprets this as a system failure, or equivalent to an exception thrown within the scope of transaction.
A TransactionAbortedException is thrown if the scope creates the transaction, and the transaction is aborted.
Based on that I would expect my outerScope to throw a TransactionAbortedException since my Transaction Statistics are showing an aborted transaction everytime I run my app with failOuter set to true. My method returns a true since no exceptions are thrown even if the transaction aborts. Unless I abort the inner transaction, then it behaves as I would expect. Any clarification would be most appreciated.
public bool CreateNestedTransaction(bool failOuter, bool failInner)
{
try
{
using (TransactionScope outerScope = new TransactionScope())
{
/* Perform transactional work here */
using (SqlConnection myConnection = new SqlConnection("server=(local)\\SQLExpress;Integrated Security=SSPI;database=test1"))
{
SqlCommand myCommand = new SqlCommand();
myConnection.Open();
myCommand.Connection = myConnection;
myCommand.CommandText = "update test set Value = ((select Value from test where Id = (select max(Id) from test))+1) where Id = (select max(Id) from test)";
myCommand.ExecuteNonQuery();
}
using (SqlConnection myConnection = new SqlConnection("server=(local)\\SQLExpress;Integrated Security=SSPI;database=test1"))
{
SqlCommand myCommand = new SqlCommand();
myConnection.Open();
myCommand.Connection = myConnection;
myCommand.CommandText = "update test set Value = Value";
myCommand.ExecuteNonQuery();
}
using (TransactionScope innerScope = new TransactionScope())
{
using (SqlConnection myConnection = new SqlConnection("server=(local)\\SQLExpress;Integrated Security=SSPI;database=test2"))
{
SqlCommand myCommand = new SqlCommand();
myConnection.Open();
myCommand.Connection = myConnection;
myCommand.CommandText = "update test set Value = ((select Value from test where Id = (select max(Id) from test))+1) where Id = (select max(Id) from test)";
myCommand.ExecuteNonQuery();
}
if (failInner == false) { innerScope.Complete(); }
}
if (failOuter == false) { outerScope.Complete(); }
}
}
catch (TransactionAbortedException)
{
return false;
}
return true;
}
Normally you don't get an exception thrown by failing to call TransactionScope.Complete() before the TransactionScope goes out of scope and gets disposed. The transaction will just quietly roll back.
The exception in your case is happening because you are trying to call Complete on the outer TransactionScope and it can't complete properly because the inner TransactionScope already failed - hence this throws an exception.
Does that make sense?
If you want to carry out some operation if your outer transaction aborted you might try something like this:
// Inside each using TransactionScope(), hhok up the current transaction completed event
Transaction.Current.TransactionCompleted += new TransactionCompletedEventHandler(Current_TransactionCompleted);
// handle the event somewhere else
void Current_TransactionCompleted(object sender, TransactionEventArgs e)
{
// check the status of the transaction
if(e.Transaction.TransactionInformation.Status == TransactionStatus.Aborted)
// do something here
}
Although I think a cleaner pattern for general usage would be to always call Complete() inside your TransactionScope and handle any resulting exception if you want to do something specific on transaction failure.