I am reviewing some code which is inserting some data into a MSSQL Database (MSSQL2014) using a Stored Procedure.
Each row of data results in a call to that database.
This is obviously inefficient for a number of reasons; not least of which is the fact that a call has to be made over the network for each line of data.
I am surmising that if they simply wrap all these individual sqlcommand calls (ExecuteNonQuery) into a transaction then this will ultimately result in a "batch insert" of sorts. When you actually "COMMIT" the transaction, the call accross the connection is made.
Is this correct? Will this send all the sqlcommands to the server in a single call? I have not been able to find a diagram or documentation which outlines the communication between the client and server when a transaction is used.
--> Begin Transaction
-> ExecuteNonQuery
-> ExecuteNonQuery
-> ExecuteNonQuery
-> ExecuteNonQuery
-> ExecuteNonQuery
-> ExecuteNonQuery
-> ExecuteNonQuery
-> ...
--> Commit Transaction (my assumption is that each of the SqlCommands
are sent over the connection object at this point)
On a sidenote I am more inclined to recommend that the developer rewrites the routine to make use of SQLBulkCopy or TableValued Parameters. This would however necessitate the re-factoring of the database stored procedure.
Thanks in Advance
There will be no batch insert. Neither the client nor the server optimize across statements. ADO.NET has no machinery to understand the SQL that you are sending. It cannot optimize anything. The server could but does not.
There will be a performance gain by using a transaction like this because the inserts do not need to flush the log as long as the transaction is pending.
There will be no batch insert. All the transaction will do is give you the ability to commit the inserts so others would be able to see the changes. No performance gains and if anything a performance hit. Not knowing what you are wanting to do here it would be hard to give you a proper answer
Related
I execute several SQL Transactions from my ASP.NET application. One transaction executes several stored procedures one after another. Transactions are quite long running ones.
When two instances of my application i.e. two different processes are connecting to one database, I sometimes run into SQL deadlock issue. In that case SQL server automatically rolls back a transaction from any one process.
When this happens, I want to execute all commands (stored procedures) in the SqlTransaction which was rolled back.
How can I do this in C#?
Is there easy way to re-execute all the commands in a SQL transaction which were executed before the transaction was rolled back by the SQL server?
Or do I have to write my own logic to "remember" (and execute them once again whenever required) all the stored procedures I had executed?
Actually its a bad practice to do so since as a dev you need to find what was the issue for rollback and fix that even so i would say declare a flag of int and increment it after each sp executions when something goes wrong in catch section where you have wrote the rollback just call the sps according to the flag.
Again thats a wrong practice to do so
I need to update several rows of one of my tables as an atomic operation.
The update concerns incrementing some values in int columns of certain rows. I need to increment values in several rows as a single action.
What would be the best way to do this?
Answering this question for me comes down to answering the following two:
If I use LINQ to SQL, how do I achieve the atomicity of the increment
operation (do I use transaction, or is there a better way)?
Are stored procedures executed atomically (in case I invoke the procedure on the DB)?
I am working in C# with SQL Server.
In SQL Server Atomicity between different operations is achieved by using Explicit Transactions, Where the user Explicitly Starts a transaction by using the key words BEGIN TRANSACTION and once all the operations are done without any erros you can commit the transaction by using key words COMMIT TRANSACTION, in case of an error/exception you can undo the work anywhere in the ongoing transaction by using key words ROLLBACK TRANSACTION
Write Ahead Strategy
SQL server uses Write Ahead Strategy to make sure the atomicity of the transactions and durability of data, When we are making any changes/Updates to the data, SQL Server takes following steps
Loads data pages into a buffer cache.
Updates the copy in the buffer.
Creates a log record in a log cache.
Saves the log record to disk via the checkpoint process.
Saves the data to disk.
So anywhere in the process of all these steps if you decide to ROLLBACK the transaction. Your is actual data on the disk is left unchanged.
My Suggestion
BEGIN TRY
BEGIN TRANSACTION
------ Your Code Here ------
---- IF everything Goes fine (No errors/No Exceptions)
COMMIT TRANSACTION
END TRY
BEGIN CATCH
ROLLBACK TRANSACTION --< this will ROLLBACK any half done operations
-- Your Code here ---------
END CATCH
I found my answer: The increment cannot be realized through LINQ to SQL directly. However, stored procedures can be called from LINQ, and increment can be realized there.
My solution was to create a stored procedure that would execute necessary updates within a single while loop in a transaction. This way all the updates are executed as a single, atomic, operation.
The UPDATE statement is atomic by itself.
I'm maintaining a ASP/C# program that uses an MS SQL Server 2008 R2 for its database requirements.
On normal and perfect days, everything works fine as it is. But we don't live in a perfect world.
An Application (for Leave, Sick Leave, Overtime, Undertime, etc.) Approval process requires up to ten separate connections to the database. The program connects to the database, passes around some relevant parameters, and uses stored procedures to do the job. Ten times.
Now, due to the structure of the entire thing, which I can not change, a dip in the connection, or heck, if I put a debug point in VS2005 and let it hang there long enough, the Application Approval Process goes incomplete. The tables are often just joined together, so a data mismatch - a missing data here, a primary key that failed to update there - would mean an entire row would be useless.
Now, I know that there is nothing I can do to prevent this - this is a connection issue, after all.
But are there ways to minimize connection lag / failure? Or a way to inform the users that something went wrong with the process? A rollback changes feature (either via program, or SQL), so that any incomplete data in the database will be undone?
Thanks.
But are there ways to minimize connection lag / failure? Or a way to
inform the users that something went wrong with the process? A
rollback changes feature (either via program, or SQL), so that any
incomplete data in the database will be undone?
As we discussed in the comments, transactions will address many of your concerns.
A transaction comprises a unit of work performed within a database
management system (or similar system) against a database, and treated
in a coherent and reliable way independent of other transactions.
Transactions in a database environment have two main purposes:
To provide reliable units of work that allow correct recovery from failures and keep a database consistent even in cases of system
failure, when execution stops (completely or partially) and many
operations upon a database remain uncompleted, with unclear status.
To provide isolation between programs accessing a database concurrently. If this isolation is not provided, the program's outcome
are possibly erroneous.
Source
Transactions in .Net
As you might expect, the database is integral to providing transaction support for database-related operations. However, creating transactions from your business tier is quite easy and allows you to use a single transaction across multiple database calls.
Quoting from my answer here:
I see several reasons to control transactions from the business tier:
Communication across data store boundaries. Transactions don't have to be against a RDBMS; they can be against a variety of entities.
The ability to rollback/commit transactions based on business logic that may not be available to the particular stored procedure you are calling.
The ability to invoke an arbitrary set of queries within a single transaction. This also eliminates the need to worry about transaction count.
Personal preference: c# has a more elegant structure for declaring transactions: a using block. By comparison, I've always found transactions inside stored procedures to be cumbersome when jumping to rollback/commit.
Transactions are most easily declared using the TransactionScope (reference) abstraction which does the hard work for you.
using( var ts = new TransactionScope() )
{
// do some work here that may or may not succeed
// if this line is reached, the transaction will commit. If an exception is
// thrown before this line is reached, the transaction will be rolled back.
ts.Complete();
}
Since you are just starting out with transactions, I'd suggest testing out a transaction from your .Net code.
Call a stored procedure that performs an INSERT.
After the INSERT, purposely have the procedure generate an error of any kind.
You can validate your implementation by seeing that the INSERT was rolled back automatically.
Transactions in the Database
Of course, you can also declare transactions inside a stored procedure (or any sort of TSQL statement). See here for more information.
If you use the same SQLConnection, or other connection types that implement IDbConnection, you can do something similar to transactionscopes but without the need to create the security risk that is a transactionscope.
In VB:
Using scope as IDbTransaction = mySqlCommand.Connection.BeginTransaction()
If blnEverythingGoesWell Then
scope.Commit()
Else
scope.Rollback()
End If
End Using
If you don't specify commit, the default is to rollback the transaction.
I have to implement Transactions in my code. I have the following options:-
TransactionScope from C# code:- I will use this case if i have some logical code in transaction along with my Database calls.Transaction rolls back and locks are released if command timeouts are there.
Having Explicit transactions in SP:- In case of command timeout, Transactions remain opens and locks are not released.
Has any one of you faced similar issues. Please suggest. Also tell me setting XACT_ABORT_ON will help in the second case.
If your stored procedure is not an "interface procedure" (called by third-party outside your code) I recommend using TransactionScope or creating SqlTransaction in code for your procedure call. Handling transactions in code is much easier. You may read this article: http://www.code-magazine.com/Article.aspx?quickid=0305111 - as you can see if you start using transactions in stored procedures things may get unnecesarily complicated.
I always use SET XACT_ABORT ON
Server side transactions don't require MSDTC and have less overhead
TRY/CATCH makes the stored procs more reliable
See my answer here for a safe stored proc template which will deal with nested transactions too: Nested stored procedures containing TRY CATCH ROLLBACK pattern?
I'm researching SqlClient's SqlBulkCopy in ADO.Net and have the following questions.
What will happen, if there was a network error during a SqlBulkCopy operation, running under as part of a transaction over a huge number of records?
Will the transaction be left open (neither committed, nor rolled back) in the server, until we manually kill it?
What is the best approach for sending a large number of records in two DataTables (InvoiceHeader, InvoiceDetails) in a DataSet to respective SQL Server tables(InvoiceHeader, InvoiceDetails)?
Thank you.
EDIT:
A few details I wanted to add, but forgot:
This is for .Net v3.5; I'm using Enterprise Library for all database interactions.
Assuming you are using a TransactionScope, my understanding is that no, the transaction will not be left open, because SQL Server will detect the ambient transaction and auto enlist. This means that the worst case is that the transaction times out, rolling back. You can change the transaction binding to specify what to do in the event of a timeout (you probably want explicit unbind).