When I do an insert with the SQL Anywhere 16 .net provider, it creates a shared lock on the table. even with a commit afterwards. How do I prevent it? (Or what am I doing wrong?)
DbCommand command = new SACommand();
command.CommandTimeout = this.Timeout;
bool mustCloseConnection = false;
PrepareCommand(command, connection, null, commandType, commandText, commandParameters, ref mustCloseConnection);
int num2 = command.ExecuteNonQuery();
command.Parameters.Clear();
if (mustCloseConnection)
{
connection.Close();
}
private void PrepareCommand(IDbCommand command, IDbConnection connection, IDbTransaction transaction, CommandType commandType, string commandText, IDataParameter[] commandParameters, ref bool mustCloseConnection)
{
if (command == null)
{
throw new ArgumentNullException("command");
}
if ((commandText == null) || (commandText.Length == 0))
{
throw new ArgumentNullException("commandText");
}
if (connection.State != ConnectionState.Open)
{
connection.Open();
mustCloseConnection = true;
}
else
{
mustCloseConnection = false;
}
command.Connection = connection;
command.CommandText = commandText;
command.CommandTimeout = this.Timeout;
if (transaction != null)
{
if (transaction.Connection == null)
{
throw new ArgumentException("The transaction was rollbacked or commited, please provide an open transaction.", "transaction");
}
command.Transaction = transaction;
}
command.CommandType = commandType;
if (commandParameters != null)
{
AttachParameters(command, commandParameters);
}
}
Ok, lets check some documentation:
SQL Anywhere® Server - Programming > Using SQL in Applications > Controlling transactions in applications > Cursors and transactions
In general, a cursor closes when a COMMIT is performed. There are two exceptions to this behavior:
The close_on_endtrans database option is set to Off.
**A cursor is opened WITH HOLD**, which is the default with Open Client and JDBC.
If either of these two cases is true, the cursor remains open on a COMMIT.
And that One:
http://infocenter.sybase.com/help/topic/com.sybase.help.sqlanywhere.12.0.1/pdf/dbprogramming12.pdf
Cursor behavior when opening cursors
You can configure the following aspects of cursor behavior when you open the cursor:
● Isolation level You can explicitly set the isolation level of operations on a cursor to be different
from the current isolation level of the transaction. To do this, set the isolation_level option.
● Holding By default, cursors in embedded SQL close at the end of a transaction. Opening a cursor
WITH HOLD allows you to keep it open until the end of a connection, or until you explicitly close it.
ADO.NET, ODBC, JDBC, and Open Client leave cursors open at the end of transactions by default
Maybe there are some more fresh docs, but I don't thing anything has been changed..
I'm almost sure you have WITH HOLD Cursor so even after COMMIT it remains open.
Actually you already have that answer, just without explanation why you should close the connection..
I hope this helps...
You should Dispose your Connection and Command Object. Call Dispose or wrap it in a using statement like this: (code just typed for example)
using (IDBConnection con = new DbConnection())
{
using (IDBCommand com = new DbCommand())
{
// do your sql stuff here
}
con.Close();
}
I've had such errors with MSSQL because I haven't disposed my command object.
If you are 100% sure all your connections, commands, transactions, etc. are getting gracefully disposed, then you might want to check how you are instantiating your transactions.
The TransactionScope default constructor was incorrectly designed by Microsoft and sooner or later it causes locks (similar issues could apply to other transaction classes). The default constructor utilize the "Serializable isolation level" which is prone to cause locks.
The issue is even worse: Should you instantiate an TransactionScope with a better isolation level (for example, ReadCommitted), in the event that your SQL operation fails and performs a rollback, it will internally change the isolation level again to "Serializable", which will be prone to locks to any other command using the same transaction instance.
This was reported as a bug to Microsoft, and MS simply replied with the typical "Well, if this is a bug, we can't fix it because it could affect systems out there, and why do you even create TransactionScope instances using the default constructor? You shouldn't do that ever" excuse.
The only safe-way to use then the TransactionScope class, is to ALWAYS AND EACH TIME instantiate it using explicitly the desired isolation level that doesn't cause locks (the usual scenario is using ReadCommitted, but you could use others).
You can either use a static builder as proposed in here http://blogs.msdn.com/b/dbrowne/archive/2010/06/03/using-new-transactionscope-considered-harmful.aspx?Redirected=true :
public class TransactionUtils {
public static TransactionScope CreateTransactionScope()
{
var transactionOptions = new TransactionOptions();
transactionOptions.IsolationLevel = IsolationLevel.ReadCommitted;
transactionOptions.Timeout = TransactionManager.MaximumTimeout;
return new TransactionScope(TransactionScopeOption.Required, transactionOptions);
}
}
Or using a custom wrapper class:
namespace System.Transactions {
/**************************************************************************************************************
* IMPORTANT: This class must ALWAYS be used instead of the regular "TransactionScope" class. This class
* enforces having transactions that read "only committed" data.
*
* This is because the implementation of TransactionScope is faulty (wrong design) and it gives issues because
* it changes the connections available at the the connection pool. To read more, check this link:
* http://blogs.msdn.com/b/dbrowne/archive/2010/05/21/using-new-transactionscope-considered-harmful.aspx
*
* The namespace was set to "System.Transactions" in order to provide ease of use when updating legacy code
* that was using the old class
**************************************************************************************************************/
public class SafeTransactionScope : IDisposable {
private TransactionScope _transactionScope;
private bool _disposed;
#region Constructors
public SafeTransactionScope()
: this(TransactionManager.MaximumTimeout) {
}
public SafeTransactionScope(TimeSpan scopeTimeout)
: this(TransactionScopeOption.Required, scopeTimeout) {
}
public SafeTransactionScope(TransactionScopeOption scopeOption)
: this(scopeOption, TransactionManager.MaximumTimeout) {
}
public SafeTransactionScope(TransactionScopeOption scopeOption, TimeSpan scopeTimeout) {
this._disposed = false;
this._transactionScope = CreateTransactionScope(scopeOption, scopeTimeout);
}
#endregion
#region Disposing methods
public void Dispose() {
Dispose(true);
// Use SupressFinalize in case a subclass
// of this type implements a finalizer.
GC.SuppressFinalize(this);
}
private void Dispose(bool disposing) {
if(!this._disposed) {
if(disposing) {
if(this._transactionScope != null) {
this._transactionScope.Dispose();
this._transactionScope = null;
}
}
// Indicate that the instance has been disposed.
this._disposed = true;
}
}
#endregion
public void Complete() {
if(this._disposed) {
throw new ObjectDisposedException("SafeTransactionScope");
}
if(this._transactionScope == null) {
// This should never happen
throw new ObjectDisposedException("SafeTransactionScope._transactionScope");
}
this._transactionScope.Complete();
}
private static TransactionScope CreateTransactionScope(TransactionScopeOption scopeOption, TimeSpan scopeTimeout) {
var transactionOptions = new TransactionOptions() {
IsolationLevel = IsolationLevel.ReadCommitted,
Timeout = scopeTimeout
};
return new TransactionScope(scopeOption, transactionOptions);
}
}
}
Related
Suppose I have a database method that looks like this:
public void insertRow(SqlConnection c)
{
using (var cmd = new SqlCommand("insert into myTable values(#dt)",c))
{
cmd.Parameters.Add(new SqlParameter("#dt",DbType.DateTime)).Value = DateTime.Now;
cmd.ExecuteNonQuery();
}
}
Now suppose I want to test this method. So, I write a test case that attempts to wrap this method inside a transaction, so that I can rollback the change after testing the result of the insertion:
public void testInsertRow()
{
SqlConnection c = new SqlConnection("connection.string.here");
SqlTransaction trans = c.BeginTransaction();
insertRow();
// do something here to evaluate what happened, e.g. query the DB
trans.Rollback();
}
This however fails to work, because:
ExecuteNonQuery requires the command to have a transaction when the connection assigned to the command is in a pending local transaction. The Transaction property of the command has not been initialized.
Is there a way to accomplish this, without having to rewrite every single database method to accept a transaction, and then rewrite every single call to pass null into the method for the transaction?
For example, this would work:
public void insertRow(SqlConnection c, SqlTransaction t)
{
using (var cmd = new SqlCommand("insert into myTable values(#dt)",c))
{
if (t != null) cmd.Transaction = t;
cmd.Parameters.Add(new SqlParameter("#dt",DbType.DateTime)).Value = DateTime.Now;
cmd.ExecuteNonQuery();
}
c.Close();
}
But then, I have to either rewrite each and every call to the database method to include that null parameter, or write override signatures for each and every database method that automatically pass in a null, e.g.
public void insertRow(SqlConnection c) { insertRow(c, null); }
What's the best way to allow transaction-based testing of database calls?
You can use a TransactionScope to add the connections automatically in to a transaction
public void testInsertRow()
{
using(TransactionScope scope = new TransactionScope(TransactionScopeOption.RequiresNew))
{
SqlConnection c = new SqlConnection("connection.string.here");
insertRow(c);
// do something here to evaluate what happened, e.g. query the DB
//do not call scope.Complete() so we get a rollback.
}
}
Now this will cause tests to block each other if you have multiple parallel tests running. If you database is set up to support it you could do Snapshot isolation so updates from concurrent tests won't lock each other out.
public void testInsertRow()
{
using(TransactionScope scope = new TransactionScope(TransactionScopeOption.RequiresNew,
new TransactionOptions(IsolationLevel = IsolationLevel.Snapshot))
{
SqlConnection c = new SqlConnection("connection.string.here");
insertRow(c);
// do something here to evaluate what happened, e.g. query the DB
//do not call scope.Complete() so we get a rollback.
}
}
Due to a few restrictions I can't use entity Framework and thus need to use SQL Connections, commands and Transactions manually.
While writing unit tests for the methods calling these data layer operations I stumbled upon a few problems.
For the unit tests I NEED to do them in a Transaction as most of the operations are changing data by their nature and thus doing them outside a Transaction is problematic as that would change the whole base data. Thus I need to put a Transaction around these (with no commit fired at the end).
Now I have 2 different variants of how These BL methods work.
A few have Transactions themselves inside of them while others have no Transactions at all. Both of these variants cause problems.
Layered Transaction: Here I get errors that the DTC cancelled the distributed Transaction due to timeouts (although the timeout is being set to 15 minutes and it is running for only 2 minutes).
Only 1 Transaction: Here I get an error about the state of the Transaction when I come to the "new SQLCommand" line in the called method.
My question here is what can I do to correct this and get unit testing with manual normal and layered Transactions working?
Unit testing method example:
using (SqlConnection connection = new SqlConnection(Properties.Settings.Default.ConnectionString))
{
connection.Open();
using (SqlTransaction transaction = connection.BeginTransaction())
{
MyBLMethod();
}
}
Example for a Transaction using method (very simplified)
using (SqlConnection connection = new SqlConnection(Properties.Settings.Default.ConnectionString))
{
connection.Open();
using (SqlTransaction transaction = connection.BeginTransaction())
{
SqlCommand command = new SqlCommand();
command.Connection = connection;
command.Transaction = transaction;
command.CommandTimeout = 900; // Wait 15 minutes before a timeout
command.CommandText = "INSERT ......";
command.ExecuteNonQuery();
// Following commands
....
Transaction.Commit();
}
}
Example for a non Transaction using method
using (SqlConnection connection = new SqlConnection(Properties.Settings.Default.ConnectionString))
{
connection.Open();
SqlCommand command = new SqlCommand();
command.Connection = connection;
command.CommandTimeout = 900; // Wait 15 minutes before a timeout
command.CommandText = "INSERT ......";
command.ExecuteNonQuery();
}
On the face of it, you have a few options, depending upon what you want to test and your ability to spend money / change your code base.
At the moment, you’re effectively writing integration tests. If the database isn’t available then your tests will fail. This means the tests can be slow, but on the plus side if they pass you’re pretty confident that you code can hit the database correctly.
If you don’t mind hitting the database, then the minimum impact to changing your code / spending money would be for you to allow the transactions to complete and verify them in the database. You can either do this by taking database snapshots and resetting the database each test run, or by having a dedicated test database and writing your tests in such a way that they can safely hit the database over and over again and then verified. So for example, you can insert a record with an incremented id, update the record, and then verify that it can be read. You may have more unwinding to do if there are errors, but if you’re not modifying the data access code or the database structure that often then this shouldn’t be too much of an issue.
If you’re able to spend some money and you want to actually turn your tests into unit tests, so that they don’t hit the database, then you should consider looking into TypeMock. It’s a very powerful mocking framework that can do some pretty scary stuff. I believe it using the profiling API to intercept calls, rather than using the approach used by frameworks like Moq. There's an example of using Typemock to mock a SQLConnection here.
If you don’t have money to spend / you’re able to change your code and don’t mind continuing to rely on the database then you need to look at some way to share your database connection between your test code and your dataaccess methods. Two approaches that spring to mind are to either inject the connection information into the class, or make it available by injecting a factory that gives access to the connection information (in which case you can inject a mock of the factory during testing that returns the connection you want).
If you go with the above approach, rather than directly injecting SqlConnection, consider injecting a wrapper class that is also responsible for the transaction. Something like:
public class MySqlWrapper : IDisposable {
public SqlConnection Connection { get; set; }
public SqlTransaction Transaction { get; set; }
int _transactionCount = 0;
public void BeginTransaction() {
_transactionCount++;
if (_transactionCount == 1) {
Transaction = Connection.BeginTransaction();
}
}
public void CommitTransaction() {
_transactionCount--;
if (_transactionCount == 0) {
Transaction.Commit();
Transaction = null;
}
if (_transactionCount < 0) {
throw new InvalidOperationException("Commit without Begin");
}
}
public void Rollback() {
_transactionCount = 0;
Transaction.Rollback();
Transaction = null;
}
public void Dispose() {
if (null != Transaction) {
Transaction.Dispose();
Transaction = null;
}
Connection.Dispose();
}
}
This will stop nested transactions from being created + committed.
If you’re more willing to restructure your code, then you might want to wrap your dataaccess code in a more mockable way. So, for example you could push your core database access functionality into another class. Depending on what you’re doing you’ll need to expand on it, however you might end up with something like this:
public interface IMyQuery {
string GetCommand();
}
public class MyInsert : IMyQuery{
public string GetCommand() {
return "INSERT ...";
}
}
class DBNonQueryRunner {
public void RunQuery(IMyQuery query) {
using (SqlConnection connection = new SqlConnection(Properties.Settings.Default.ConnectionString)) {
connection.Open();
using (SqlTransaction transaction = connection.BeginTransaction()) {
SqlCommand command = new SqlCommand();
command.Connection = connection;
command.Transaction = transaction;
command.CommandTimeout = 900; // Wait 15 minutes before a timeout
command.CommandText = query.GetCommand();
command.ExecuteNonQuery();
transaction.Commit();
}
}
}
}
This allows you to unit test more of your logic, like the command generation code, without having to actually worry about hitting the database and you can test your core dataaccess code (the Runner) against the database once, rather than for every command you want to run against the database. I would still write integration tests for all dataaccess code, but I’d only tend to run them whilst actually working on that section of code (to ensure column names etc have been specified correctly).
I've looked around but couldn't find an appropriate (or one that is satisfactory to me) on how to address an issue we are having.
I use Dapper and ODP.NET 12 Managed drivers. The problem is not encountered when TransactionScope is not used.
When performing commands under a transaction scope, I get an error "Transaction has aborted" via the TransactionAbortedException thrown.
Observed behaviors:
1) TransactionAbortedException is thrown if and only if the transaction is completed and the TransactionScope is disposed. The point at when the exception is thrown is during dispose.
2) Despite the exception, the transaction concept actually works! After Complete() is invoked, the changes are committed into the database.
Below is the code snippet.
// Conn string: "Data Source=OraDB;Persist Security Info=True;User ID=userxxx;Password=passwordxxx;" providerName="Oracle.ManagedDataAccess.Client
// Note: GetDbFactory().Create() returns a DbConnection object
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.RequiresNew, new TransactionOptions { IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted }))
using (var dbConn = GetDbFactory().Create())
{
foreach (MyDTO dto in dtoList)
{
var tableDAO= new TableDAO(dbConn);
MyEntity entity = new MyEntity()
{
Field1 = dto.Field1,
Field2 = dto.Field2
};
tableDAO.AddOrUpdate(entity);
}
// Commit changes
scope.Complete();
}
// This method is under the DAO class
public void AddOrUpdate(MyEntity entity)
{
// Verify arguments
entity.AsArgumentThrowExceptionIfNull("entity");
// build param
OracleDynamicParameters parameters = new OracleDynamicParameters();
parameters.Add("P_FIELD1", entity.Field1);
parameters.Add("P_FIELD2", entity.Field2);
// execute SP
dbConnection.Execute("PRC_MY_ENTITY_ADDORUPDATE", parameters, commandType: CommandType.StoredProcedure);
}//-- end AddOrUpdate()
==================================================================
UPDATE (09-Apr-15)
I have changed my approach and use the following pattern for now for Oracle. Our code deals with connections in both Oracle and SQL Server so I'd much prefer that the coding pattern is consistent, but until a solution is found with using Oracle+TransactionScope, we'll use the pattern below for Oracle command execution:
using (var dbConnection = dbConnFactory.Create())
{
// Open db connection
dbConnection.Open();
using (var trans = dbConnection.BeginTransaction())
{
bool isSuccess = false;
try
{
// Perform DB operations here
trans.Commit();
isSuccess = true;
}
finally
{
if(!isSuccess) trans.Rollback();
}
}
}
First, the exception is documented as possible:
A call to the Dispose method marks the end of the transaction scope. Exceptions that occur after calling this method may not affect the transaction.
This is in the documentation on the TransactionScope class (https://msdn.microsoft.com/en-us/library/system.transactions.transactionscope%28v=vs.110%29.aspx).
One other thing I noticed is that the complete on the transaction is being before the connection is closed. I'd change this to close the connection and then complete the transaction.
using (TransactionScope scope = new TransactionScope(TransactionScopeOption.RequiresNew, new TransactionOptions { IsolationLevel = System.Transactions.IsolationLevel.ReadCommitted }))
{
using (var dbConn = GetDbFactory().Create())
{
foreach (MyDTO dto in dtoList)
{
var tableDAO= new TableDAO(dbConn);
MyEntity entity = new MyEntity()
{
Field1 = dto.Field1,
Field2 = dto.Field2
};
tableDAO.AddOrUpdate(entity);
}
}
// Commit changes
scope.Complete();
}
I don't see anything wrong with that code. However, if that loop runs long enough, the transaction is going to timeout. You'll then get the exception in question the next time you do an operation against the database. I would try increasing the timeout - Timeout is a property on the TransactionScopeOption class.
I am resorting to using BeginTransaction() as the final approach (refer to my update in my original post). I have read more about why TransactionScope() was failing.
1) ODP.Net promotes to distributed transaction even when using a single DB connection when connecting to Oracle 10g and below (source). Lo and behold, the database I'm connecting to is indeed 10g.
2) You'll need Oracle MTS Service installed. This I didn't have setup on my dev machine.
Note: I've went through millions of questions when the issue is not disposing the reader/connection properly, or when the error is because of badly handled lazy loading. I believe that this issue is a different one, and probably related to MySQL's .NET connector.
I'm using MySQL server (5.6) database extensively through its .NET connector (6.8.3). All tables are created with MyISAM engine for performance reasons. I have only one process with one thread (update: in fact, it's not true, see below) accessing the DB sequentially, so there is no need for transactions and concurrency.
Today, after many hours of processing the following piece of code:
public IEnumerable<VectorTransition> FindWithSourceVector(double[] sourceVector)
{
var sqlConnection = this.connectionPool.Take();
this.selectWithSourceVectorCommand.Connection = sqlConnection;
this.selectWithSourceVectorCommand.Parameters["#epsilon"].Value
= this.epsilonEstimator.Epsilon.Min() / 10;
for (int d = 0; d < this.dimensionality; ++d)
{
this.selectWithSourceVectorCommand.Parameters["#source_" + d.ToString()]
.Value = sourceVector[d];
}
// *** the following line (201) throws the exception presented below
using (var reader = this.selectWithSourceVectorCommand.ExecuteReader())
{
while (reader.Read())
{
yield return ReaderToVectorTransition(reader);
}
}
this.connectionPool.Putback(sqlConnection);
}
threw the following exception:
MySqlException: There is already an open DataReader associated with this Connection which must be closed first.
Here is the relevant part of the stack trace:
at MySql.Data.MySqlClient.ExceptionInterceptor.Throw(Exception exception)
at MySql.Data.MySqlClient.MySqlConnection.Throw(Exception ex)
at MySql.Data.MySqlClient.MySqlCommand.CheckState()
at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader(CommandBehavior behavior)
at MySql.Data.MySqlClient.MySqlCommand.ExecuteReader()
at implementation.VectorTransitionsMySqlTable.d__27.MoveNext() in C:\Users\bartoszp...\implementation\VectorTransitionsMySqlTable.cs:line 201
at System.Linq.Enumerable.d__3a1.MoveNext()
at System.Linq.Buffer1..ctor(IEnumerable1 source)
at System.Linq.Enumerable.ToArray[TSource](IEnumerable1 source)
at implementation.VectorTransitionService.Add(VectorTransition vectorTransition) in C:\Users\bartoszp...\implementation\VectorTransitionService.cs:line 38
at Program.Go[T](Environment`2 p, Space parentSpace, EpsilonEstimator epsilonEstimator, ThresholdEstimator thresholdEstimator, TransitionTransformer transitionTransformer, AmbiguityCalculator ac, VectorTransitionsTableFactory vttf, AxesTableFactory atf, NeighbourhoodsTableFactory ntf, AmbiguitySamplesTableFactory astf, AmbiguitySampleMatchesTableFactory asmtf, MySqlConnectionPool connectionPool, Boolean rejectDuplicates, Boolean addNew) in C:\Users\bartoszp...\Program.cs:line 323
The connectionPool.Take returns the first connection that satisfies the following predicate:
private bool IsAvailable(MySqlConnection connection)
{
var result = false;
try
{
if (connection != null
&& connection.State == System.Data.ConnectionState.Open)
{
result = connection.Ping();
}
}
catch (Exception e)
{
Console.WriteLine("Ping exception: " + e.Message);
}
return result && connection.State == System.Data.ConnectionState.Open;
}
(This is related to my previous question, when I resolved a different, but similar issue: MySQL fatal error during information_schema query (software caused connection abort))
The FindWithSourceVector method is called by the following piece of code:
var existing
= this.vectorTransitionsTable
.FindWithSourceVector(vectorTransition.SourceVector)
.Take(2)
.ToArray();
(I need to find at most two duplicate vectors) - this is the VectorTransitionService.cs:line 38 part of the stack trace.
Now the most interesting part: when the debugger stopped execution after the exception occured, I've investigated the sqlConnection object to find, that it doesn't have a reader associated with it (picture below)!
Why is this happening (apparently at "random" - this method was being called almost every minute for the last ~20h)? Can I avoid that (in ways other then guess-adding some sleeps when Ping throws an exception and praying it'll help)?
Additional information regarding the implementation of the connection pool:
Get is intended for methods that call only simple queries and are not using readers, so the returned connection can be used in a re-entrant way. It is not used directly in this example (because of the reader involved):
public MySqlConnection Get()
{
var result = this.connections.FirstOrDefault(IsAvailable);
if (result == null)
{
Reconnect();
result = this.connections.FirstOrDefault(IsAvailable);
}
return result;
}
The Reconnect method just iterates though the whole array and recreates and opens the connections.
Take uses Get but also removes the returned connection from the list of available connections so in case of some methods that during their usage of a reader call other methods that also need a connection, it will not be shared. This is also not the case here, as the FindSourceVector method is simple (doesn't call other methods that use the DB). However, the Take is used for the sake of convention - if there is a reader, use Take:
public MySqlConnection Take()
{
var result = this.Get();
var index = Array.IndexOf(this.connections, result);
this.connections[index] = null;
return result;
}
Putback just puts a connection to the first empty spot, or just forgets about it if the connection pool is full:
public void Putback(MySqlConnection mySqlConnection)
{
int index = Array.IndexOf(this.connections, null);
if (index >= 0)
{
this.connections[index] = mySqlConnection;
}
else if (mySqlConnection != null)
{
mySqlConnection.Close();
mySqlConnection.Dispose();
}
}
I suspect this is the problem, at the end of the method:
this.connectionPool.Putback(sqlConnection);
You're only taking two elements from the iterator - so you never complete the while loop unless there's actually only one value returned from the reader. Now you're using LINQ, which will automatically be calling Dispose() on the iterator, so your using statement will still be disposing of the reader - but you're not putting the connection back in the pool. If you do that in a finally block, I think you'll be okay:
var sqlConnection = this.connectionPool.Take();
try
{
// Other stuff here...
using (var reader = this.selectWithSourceVectorCommand.ExecuteReader())
{
while (reader.Read())
{
yield return ReaderToVectorTransition(reader);
}
}
}
finally
{
this.connectionPool.Putback(sqlConnection);
}
Or ideally, if your connection pool is your own implementation, make Take return something which implements IDisposable and returns the connection back to the pool when it's done.
Here's a short but complete program to demonstrate what's going on, without any actual databases involved:
using System;
using System.Collections.Generic;
using System.Linq;
class DummyReader : IDisposable
{
private readonly int limit;
private int count = -1;
public int Count { get { return count; } }
public DummyReader(int limit)
{
this.limit = limit;
}
public bool Read()
{
count++;
return count < limit;
}
public void Dispose()
{
Console.WriteLine("DummyReader.Dispose()");
}
}
class Test
{
static IEnumerable<int> FindValues(int valuesInReader)
{
Console.WriteLine("Take from the pool");
using (var reader = new DummyReader(valuesInReader))
{
while (reader.Read())
{
yield return reader.Count;
}
}
Console.WriteLine("Put back in the pool");
}
static void Main()
{
var data = FindValues(2).Take(2).ToArray();
Console.WriteLine(string.Join(",", data));
}
}
As written - modelling the situation with the reader only finding two values - the output is:
Take from the pool
DummyReader.Dispose()
0,1
Note that the reader is disposed, but we never get as far as returning anything from the pool. If you change Main to model the situation where the reader only has one value, like this:
var data = FindValues(1).Take(2).ToArray();
Then we get all the way through the while loop, so the output changes:
Take from the pool
DummyReader.Dispose()
Put back in the pool
0
I suggest you copy my program and experiment with it. Make sure you understand everything about what's going on... then you can apply it to your own code. You might want to read my article on iterator block implementation details too.
TyCobb and Jon Skeet have correctly guessed, that the problem was the pool implementation and multi-threading. I forgot that actually I did start some tiny Tasks in the Reconnect method. The first connection was created and opened synchronously but all other where opened asynchronously.
The idea was that because I only need one connection at time, there others can reconnect in different threads. However, because I didn't always put the connection back (as explained in Jon's answer) reconnecting was happening quite frequently, and because the system was quite loaded these reconnection threads weren't fast enough, which eventually led to race conditions. The fix is to reconnect in a more simple and straightforward manner:
private void Reconnect()
{
for (int i = 0; i < connections.Length; ++i)
{
if (!IsAvailable(this.connections[i]))
{
this.ReconnectAt(i);
}
}
}
private void ReconnectAt(int index)
{
try
{
this.connections[index] = new MySqlConnection(this.connectionString);
this.connections[index].Open();
}
catch (MySqlException mse)
{
Console.WriteLine("Reconnect error: " + mse.Message);
this.connections[index] = null;
}
}
Which pattern is better for SqlConnection object? Which is better in performance?
Do you offer any other pattern?
class DataAccess1 : IDisposable
{
private SqlConnection connection;
public DataAccess1(string connectionString)
{
connection = new SqlConnection(connectionString);
}
public void Execute(string query)
{
using (SqlCommand command = connection.CreateCommand())
{
command.CommandText = query;
command.CommandType = CommandType.Text;
// ...
command.Connection.Open();
command.ExecuteNonQuery();
command.Connection.Close();
}
}
public void Dispose()
{
connection.Dispose();
}
}
VS
class DataAccess2 : IDisposable
{
private string connectionString;
public DataAccess2(string connectionString)
{
this.connectionString = connectionString;
}
public void Execute(string query)
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
SqlCommand command = connection.CreateCommand();
command.CommandText = query;
command.CommandType = CommandType.Text;
// ...
command.Connection.Open();
command.ExecuteNonQuery();
command.Connection.Close();
}
}
public void Dispose()
{
}
}
There's no real way to answer this question. The short, canonical answer is that the connection should stay alive for the lifetime of your unit of work. Because we have no way of knowing how DataAccess is used (does it exist for the lifetime of your application, or do you instantiate it and dispose it whenever you do something?), it's impossible to give a concrete answer.
That being said, I would recommend the first pattern, but instantiate and dispose of your DataAccess object as needed; don't keep it around longer than necessary.
Suggest going with DataAccess2. It's a personal preference though. Some might even suggest your class be static. It'd be difficult to say that one is more performant than the other. You're on the path of IDisposable, which is great.
I'd be happy to read and maintain both styles shown above in your question.
Consider having your DAL be able to read the connection string from a .config as well, rather than exclusively allowing the value to be passed in the constructor.
public DataAccess2(string connStr)
{
this.connectionString = connStr;
}
public DataAccess2()
{
this.connectionString =
ConfigurationManager.ConnectionStrings["foo"].ConnectionString;
}
Consider wrapping your SqlCommand in a using as well.
using (var conn = new SqlConnection(connectionString))
{
using(var cmd = conn.CreateCommand())
{
}
}
I think it depends on how your DataAccess object is intended to be used, if it's used within a 'using' clause then the connection is guaranteed to be disposed of after it's done.
But in general I prefer the second pattern as the sql connection is created and disposed of within the Execute method so it's less likely to be left open when you forget to dispose of your DataAccess object.
Considering that sql connection can be a scarse resource I think every attempt should be made to ensure that they're not wasted.
The first will result in errors if you make concurrent calls.
The second will ensure you use a clean connection for each command resulting in more connections being made.
I agree with the statements above that it depends on the scenario for use, to get over the problem related to the first I have a wrapper that needs to use such a pattern so I set a field value boolean to show that a command is being executed on the connection already then "queue" the next command for execution.
There will of course be situations where you may prefer to use multiple connections ...