A typical example for C#/MySQL interaction involves code like this (I'm skipping try/catches and error checking for simplicity's sake):
conn = new MySqlConnection(cs);
conn.Open();
string stm = "SELECT * FROM Authors";
MySqlCommand cmd = new MySqlCommand(stm, conn);
rdr = cmd.ExecuteReader();
The scenario is a class library for both a standalone application and a WCF web-service. So, should I open a connection whenever I make a query? Or should I open it once when I open the program?
To expand on HackedByChinese's recommendation consider the following. You have one main coordinating method that handles creating the connection, opening it, setting the transaction, and then calling the worker methods that do the different types of work (queries).
public static void UpdateMyObject(string connection, object myobject)
{
try
{
using (SqlConnection con = new SqlConnection(connection))
{
con.Open();
using (SqlTransaction trans = con.BeginTransaction())
{
WorkingMethod1(con, myobject);
WorkingMethod2(con, myobject);
WorkingMethod3(con, myobject);
trans.Commit();
}
con.Close();
}
}
catch (Exception ex)
{
MessageBox.Show("SOMETHING BAD HAPPENED!!!!!!! {0}", ex.Message);
}
}
private static void WorkingMethod1(SqlConnection con, object myobject)
{
// Do something here against the database
}
private static void WorkingMethod2(SqlConnection con, object myobject)
{
// Do something here against the database
}
private static void WorkingMethod3(SqlConnection con, object myobject)
{
// Do something here against the database
}
You typically want to open one connection per unit of work. The MySQL ADO.NET driver can pool connections for you, however, I would recommend against having each query open a connection. What can happen is you start using several methods in order to service one business transaction, and since each is opening a connection, you can end up that one business transaction starving your available connections. This of course can lead to timeouts, poor performance, etc.
Instead, consider defining an IUnitOfWork/UnitOfWork API that creates and opens a connection and transaction, and having your data access layer request the current IUnitOfWork, which will provide the current connection and transaction.
The trick, of course, is knowing when a unit of work begins and when it ends. They are associated with a complete, meaningful action (a "business transaction"). For example, when you are servicing a request to a method on one of your WCF services. When the service instance starts, in response to a request, a unit of work should be created. Then you DAL components can ask for the current unit of work and use it. Then when the request is completed, the unit of work should be committed (which should commit the transaction and close the connection).
Related
For my data access I use TransactionScopes at the API level to wrap entire operations in a single transaction so that my SQL operations can be somewhat composable. I have a web project that hosts an API and a separate service library that is the implementation and calls to SQL. At the beginning of an Operation (an API entry-point) I open the TransactionScope. Whenever a SqlConnection is needed within the processing of the Operation, ask for the AmbientConnection instead of directly making a new connection. AmbientConnection finds or creates a new SqlConnection for the current transation. Doing this is supposed to allow for good composibility but also avoid the invocation of the MSDTC because it should keep using the same connection for the each suboperation within the transaction. When the transaction is completed (with scope.complete()), the connection is automatically closed.
The problem is that every once in a while the MSDTC is still getting invoked and I cannot figure out why. I've used this before sucessfully and I believe I never got an MSDTC invoked. Two things seem different to me this time though: 1) I'm using SQL Server 2008 R1 (10.50.4000) - not my choice - and I'm aware that the MSDTC behavior changed beginning with this version and perhaps not all the kinks were worked out until later versions. 2) The use of async-await is new and I believe I'm having to use TransactionScopeAsyncFlowOption.Enabled to accommodate this new feature in case some part of the implementation is async. Perhaps more measures are necessary.
I tried Pooling=false in the connection string in case it was MSDTC getting invoked because of two independent logical connections handled errantly under a single pooled connection. But that didn't work.
API Operation
// Exposed API composing multiple low-level operations within a single TransactionScope
// independent of any database platform specifics.
[HttpPost]
public async Task<IHttpActionResult> GetMeTheTwoThings()
{
using (var scope = new TransactionScope(TransactionScopeOption.Required, TransactionScopeAsyncFlowOption.Enabled))
{
var result = new TwoThings(
await serviceLayer.GetThingOne(),
await serviceLayer.GetThingTwo());
scope.Complete();
return Ok(result);
}
}
Service layer implementation
public async Task<ThingOne> GetThingOne()
{
using (var cmd = connManagement.AmbientConnection.CreateCommand())
{
cmd.CommandType = System.Data.CommandType.StoredProcedure;
cmd.CommandText = "dbo.GetThingOne";
return (ThingOne)(await cmd.ExecuteScalarAsync());
}
}
public async Task<ThingTwo> GetThingTwo()
{
using (var cmd = connManagement.AmbientConnection.CreateCommand())
{
cmd.CommandType = System.Data.CommandType.StoredProcedure;
cmd.CommandText = "dbo.GetThingTwo";
return (ThingTwo)(await cmd.ExecuteScalarAsync());
}
}
AmbientConnection implementation
internal class SQLConnManagement
{
readonly string connStr;
readonly ConcurrentDictionary<Transaction, SqlConnection> txConnections = new ConcurrentDictionary<Transaction, SqlConnection>();
private SqlConnection CreateConnection(Transaction tx)
{
var conn = new SqlConnection(this.connStr);
// When the transaction completes, close the connection as well
tx.TransactionCompleted += (s, e) =>
{
SqlConnection closing_conn;
if (txConnections.TryRemove(e.Transaction, out closing_conn))
{
closing_conn.Dispose(); // closing_conn == conn
}
};
conn.Open();
return conn;
}
internal SqlConnection AmbientConnection
{
get
{
var txCurrent = Transaction.Current;
if (txCurrent == null) throw new InvalidOperationException("An ambient transaction is required.");
return txConnections.GetOrAdd(txCurrent, CreateConnection);
}
}
public SQLConnManagement(string connStr)
{
this.connStr = connStr;
}
}
Not to overcomplicate the post, but this might be relevant because it seems to me that every time MSDTC has been invoked the logged stack trace shows that this next mechanism has been involved. Certain data I cache with the built in ObjetCache because it doesn't change often and so I just get it at most once per minute or whatever. This is a little fancy, but I don't see why the Lazy generator would be treated any differently from a more typical call and why this specifically would cause the MSSDTC to sometimes be invoked. I've tried LazyThreadSafetyMode.ExecutionAndPublication too just in case but that doesn't help anyway (and then the exception just keeps getting delivered as the cached result for subsequent requests before the expiration, of course, and that's not desirable).
/// <summary>
/// Cache element that gets the item by key, or if it is missing, creates, caches, and returns the item
/// </summary>
static T CacheGetWithGenerate<T>(ObjectCache cache, string key, Func<T> generator, DateTimeOffset offset) where T : class
{
var generatorWrapped = new Lazy<T>(generator, System.Threading.LazyThreadSafetyMode.PublicationOnly);
return ((Lazy<T>)cache.AddOrGetExisting(
key,
generatorWrapped,
offset))?.Value ?? generatorWrapped.Value;
}
public ThingTwo CachedThingTwo
{
get
{
return CacheGetWithGenerate(
MemoryCache.Default,
"Services.ThingTwoData",
() => GetThingTwo(), // ok, GetThingTwo isn't async this time, fudged example
DateTime.Now.Add(TimeSpan.FromMinutes(1)));
}
}
Do you know why MSDTC is being invoked?
PublicationOnly means that two connections can be created and one thrown away. I'm surprised you made this bug because you explicitly stated PublicationOnly (as opposed to the default safety mode which is safe). You explicitly allowed this bug.
For some reason I did not see that you tried ExecutionAndPublication already. Since not using it is a bug please fix the code in the question.
CreateConnection is also broken in the sense that in case of exception on open the connection object is not getting disposed. Probably harmless but you never know.
Also, audit this code for thread aborts which can happen when ASP.NET times out a request. You are doing very dangerous and brittle things here.
The pattern that I use is to use an IOC container to inject a connection that is shared for the entire request. The first client for that connection opens it. The request end event closes it. Simple, and does away with all that nasty shared, mutable, multi-threaded state.
Why are you using a cache for data that you do not want to lose? This is probably the bug. Don't do that.
What is ?.Value ?? generatorWrapped.Value about? The dictionary can never return null. Delete that code. If it could return null then forcing the lazy value would create a second connection so that's a logic bug as well.
I'm creating an winforms application.
In which one form is made transparent, This form is used to show some popup message boxes, using a timer this form queries database in each seconds.
Currently I'm using database connection inside using method (here postgres Data Base).
Method 1
namespace MyApplication
{
public partial class frmCheckStatus: Form
{
private void timerCheckStatus_Tick(object sender, EventArgs e)
{
using (NpgsqlConnection conn = new NpgsqlConnection("My Connection String"))
{
conn.Open();
//Database queries
//Show popup message
conn.Close();//Forsing to close
}
}
}
}
so in each seconds this connection object is created and disposed.
Note : I'm not using this object for any other purpose or inside any forms or methods.
Is it good to create and use a single connection object global to this class, and use inside timer tick function?, and dispose on form close event
Method 2
namespace MyApplication
{
public partial class frmCheckStatus: Form
{
Private NpgsqlConnection conn = new NpgsqlConnection("My Connection String");
private void timerCheckStatus_Tick(object sender, EventArgs e)
{
//Here use conn object for queries.
conn.Open();
//Database queries
//Show popup message
conn.Close();//Forsing to close
}
private void frmCheckStatus_FormClosing(object sender, FormClosingEventArgs e)
{
conn.Dispose();
}
}
}
Which will be better?, considering memory, resource usage, execution time etc. Please give proper reason for your choice of method.
Looking at the documentation for your connection class (Here), it would appear that this supports connection pooling. This will mean that connections to the same endpoint (same connection string) will reuse existing connections rather than incurring the overhead of creating new ones.
Im not familiar with your particular connection, but if the behaviour is anything like SQLConnection class for ADO.net, repeatedly creating a new connection to the same connection string should not be particularly expensive (computationally).
As an aside, i would wrap your connection logic in try / finally to ensure it gets closed in the event of an application exception.
I can't see any advantage to instantiating a new connection every time you run a new query. I know it's done often in code, but there is overhead associated with it, however small. If you're running multiple queries from the start of the program to the end of the program, I think you should re-use the existing connection object.
If your goal is to make the connection "disappear" from the server (which I wouldn't generally worry about if this program runs on one machine -- if it runs on dozens, that's another story -- look up PgBounce), then that should be just as easily accomplished by turning connection pooling off, and then the Close() method would take care of it.
You kind of asked for pros and cons, and while it's not necessarily harmful to instantiate the connection within the loop, I can't imagine how it could be better.
For what it's worth, you may want to consider carrying the connection as a property (preferably outside of the form class, since you may want to eventually use it elsewhere). Something like this:
private NpgsqlConnection _PgConnection;
public NpgsqlConnection PgConnection
{
get
{
if (_PgConnection == null)
{
NpgsqlConnectionStringBuilder sb = new NpgsqlConnectionStringBuilder();
sb.Host = "hostname.whatever.com";
sb.Port = 5432;
sb.UserName = "scott";
sb.Password = "tiger";
sb.Database = "postgres";
sb.Pooling = true;
_PgConnection = new NpgsqlConnection(sb.ToString());
}
if (!_PgConnection.State.Equals(ConnectionState.Open))
_PgConnection.Open();
return _PgConnection;
}
set { _PgConnection = value; }
}
Then, within your form (or wherever you execute your SQL), you can just call the property:
NpgSqlCommand cmd = new NpgSqlCommand("select 1", Database.PgConnection);
...
Database.PgConnection.Close();
And you don't need to worry if the connection is open or closed, or if it's even been created yet.
The only open question would be if you want that connection to actually disappear on the server, which would be changed by altering the Pooled property.
I am designing a database wrapper for C#.
Below are the two options I have:
Option A:
class DBWrapper:IDisposable
{
private SqlConnection sqlConn;
public DBWrapper()
{
sqlConn = new SqlConnection("my connection string");
sqlConn.Open();
}
public DataTable RunQuery(string Sql)
{
implementation......
}
public Dispose()
{
if(sqlConn != null)
sqlConn.Close();
}
}
Option B:
class DBWrapper
{
public DBWrapper()
{
}
public DataTable RunQuery(string Sql)
{
SqlConnection sqlConn = new SqlConnection("my connection string");
.....implementation......
sqlConn.Close();
}
}
For option A connection is opened when class is instantiated. So no matter how many times the caller calls RunQuery the connection is always ready. But If the application instantiates DBWrapper early in the application, the connection will be just opened and doing nothing until the application is finished. Also, it could have many DBWrapper instantiated during the execution. So, it's kinda wasting resources.
For option B it doesn't have the problem option A has, but the a new connection has to be opened and closed everytime the caller calls RunQuery. I am not sure how much it will hurt the performance.
Please share your expertise. Thank you for reading.
For performance reasons, you'll definitely not want to go with Option B (at least in the cases I experienced.)
Let me suggest Option C:
class DBWrapper:IDisposable {
private SqlConnection sqlConn;
public void EnsureConnectionIsOpen()
{
if (sqlConn == null)
{
sqlConn = new SqlConnection("my connection string");
sqlConn.Open();
}
}
public DataTable RunQuery(string Sql)
{
EnsureConnectionIsOpen();
implementation......
}
public Dispose()
{
if(sqlConn != null)
sqlConn.Close();
}
}
You might consider using the singleton pattern to make sure there is only one instance of your DBWrapper.
A few comments worth considering:
In the approach where you manage a (perhaps) long-lived connection, it is important to check whether the connection is open before running a query. I've run into issues before where NETCF closed unused connections after a while.
In the approach where you open a new connection per-query, ensure that your connection, commands, and (if used) data readers are all properly wrapped in using statements or try/finally+dispose() blocks to free up connections and locks.
Happy coding!
Garbage collector is triggered under rather complex conditions but basically it is invoked when memory exceeds some limit, it is invoked periodically as well but the period is not constant. You never can be sure when exactly garbage collector disposes and consequently (in another run) destroys the object. One thing you can be sure is the fact that garbage collector will never dispose and destroy the object that still has references. For example object that is referenced via static variables on the class neither will be disposed nor destroyed.
Option B is more transactional, which has its advantages. ADO.NET uses implicit connection pooling, so you do not have to worry about creating new instances of SqlConnection frequently.
You should consider whether you are using a connected or disconnected data model; as the second approach lends itself better to a disconnected model.
But as i've said above, connection pooling means that it makes virtually no difference in practical terms.
You could have an option C where the database is opened on request in RunQuery (if it is not open) and closed on disposed (when it was opened). That way the database is only opened when really needed and will be opened only once.
So in pseudo code:
class DBWrapper
{
public DBWrapper()
{
}
SqlConnection sqlConn = null;
public DataTable RunQuery(string Sql)
{
if(sqlConn == null) sqlConn = new SqlConnection("my connection string");
.....implementation......
}
public Dispose()
{
if(sqlConn != null)
sqlConn.Close();
}
}
Also mind that the moment Dispose is called is not always directly after the object is not needed anymore (for example a function variable after the function is used). As far as I know it will be executed when the garbage collector collects the object (which is not directly). But I am not totally sure about this. This behaviour might also differ between web and non web application.
In my server application I want to use DB (SQL Server) but I am quite unsure of the best method. There are clients whose requests comes to threadpool and so their processing is async. Every request usually needs to read or write to DB, so I was thinking about static method which would create connection, execute the query and return the result. I'm only afraid whether opening and closing connection is not too slow and whether some connection limit could not be reached? Is this good approach?
IMHO the best is to rely on the ADO.NET connection pooling mechanism and don't try to handle database connections manually. Write your data access methods like this:
public void SomeMethod()
{
using (var connection = new SqlConnection(connectionString))
using (var command = connection.CreateCommand())
{
connection.Open();
command.CommandText = "SELECT Field1 FROM Table1";
using (var reader = command.ExecuteReader())
{
while(reader.Read())
{
// do something with the results
}
}
}
}
Then you can call this method from wherever you like, make it static, call it from threads whatever. Remember that calling Dispose on the connection won't actually close it. It will return it to the connection pool so that it can be reused.
Surprised that no one mentioned connection pooling. If you think you are going to have a large number of requests, why not just setup a pool with a min pool size set to say 25 (arbitrary number here, do not shoot) and max pool size set to say 200.
This will decrease the number of connection attempts and make sure that if you are not leaking connection handles (something that you should take explicit care to not let happen), you will always have a connection waiting for you.
Reference article on connection pooling: http://msdn.microsoft.com/en-us/library/8xx3tyca.aspx
Another side note, why the need to have the connection string in the code? Set it in the web.config or app.config for the sake of maintainability. I had to "fix" code that did such things and I always swore copiously at the programmer responsible for such things.
I have had exactly the same problem like you. Had huge app that i started making multithreaded. Benefit over having one connection open and being reused is that you can ask DB multiple times for data as new connection is spawned on request (no need to wait for other threads to finish getting data), and if for example you loose connection to sql (and it can happen when network goes down for a second or two) you will have to always check if connection is open before submitting query anyway.
This is my code for getting Database rows in MS SQL but other stuff should be done exactly the same way. Keep in mind that the sqlConnectOneTime(string varSqlConnectionDetails) has a flaw of returning null when there's no connection so it needs some modifications for your needs or the query will fail if sql fails to establish connection. You just need to add proper code handling there :-) Hope it will be useful for you :-)
public const string sqlDataConnectionDetails = "Data Source=SQLSERVER\\SQLEXPRESS;Initial Cata....";
public static string sqlGetDatabaseRows(string varDefinedConnection) {
string varRows = "";
const string preparedCommand = #"
SELECT SUM(row_count) AS 'Rows'
FROM sys.dm_db_partition_stats
WHERE index_id IN (0,1)
AND OBJECTPROPERTY([object_id], 'IsMsShipped') = 0;";
using (var varConnection = Locale.sqlConnectOneTime(varDefinedConnection))
using (var sqlQuery = new SqlCommand(preparedCommand, varConnection))
using (var sqlQueryResult = sqlQuery.ExecuteReader())
while (sqlQueryResult.Read()) {
varRows = sqlQueryResult["Rows"].ToString();
}
return varRows;
}
public static SqlConnection sqlConnectOneTime(string varSqlConnectionDetails) {
SqlConnection sqlConnection = new SqlConnection(varSqlConnectionDetails);
try {
sqlConnection.Open();
} catch (Exception e) {
MessageBox.Show("Błąd połączenia z serwerem SQL." + Environment.NewLine + Environment.NewLine + "Błąd: " + Environment.NewLine + e, "Błąd połączenia");
}
if (sqlConnection.State == ConnectionState.Open) {
return sqlConnection;
}
return null;
}
Summary:
Defined one global variable with ConnectionDetails of your SQL Server
One global method to make connection (you need to handle the null in there)
Usage of using to dispose connection, sql query and everything when the method of reading/writing/updating is done.
The one thing that you haven't told us, that would be useful for giving you an answer that's appropriate for you is what level of load you're expecting your server application to be under.
For pretty much any answer to the above question though, the answer would be that you shouldn't worry about it. ADO.net/Sql Server provides connection pooling which removes some of the overhead of creating connections from each "var c = new SqlConnection(connectionString)" call.
I'm getting this error (Distributed transaction completed. Either enlist this session in a new transaction or the NULL transaction.) when trying to run a stored procedure from C# on a SQL Server 2005 database. I'm not actively/purposefully using transactions or anything, which is what makes this error weird. I can run the stored procedure from management studio and it works fine. Other stored procedures also work from C#, it just seems to be this one with issues. The error returns instantly, so it can't be a timeout issue. The code is along the lines of:
SqlCommand cmd = null;
try
{
// Make sure we are connected to the database
if (_DBManager.CheckConnection())
{
cmd = new SqlCommand();
lock (_DBManager.SqlConnection)
{
cmd.CommandText = "storedproc";
cmd.CommandType = System.Data.CommandType.StoredProcedure;
cmd.Connection = _DBManager.SqlConnection;
cmd.Parameters.AddWithValue("#param", value);
int affRows = cmd.ExecuteNonQuery();
...
}
}
else
{
...
}
}
catch (Exception ex)
{
...
}
It's really got me stumped. Thanks for any help
It sounds like there is a TransactionScope somewhere that is unhappy. The _DBManager.CheckConnection and _DBManager.SqlConnection sounds like you are keeping a SqlConnection hanging around, which I expect will contribute to this.
To be honest, in most common cases you are better off just using the inbuilt connection pooling, and using your connections locally - i.e.
using(var conn = new SqlConnection(...)) { // or a factory method
// use it here only
}
Here you get a clean SqlConnection, which will be mapped to an unmanaged connection via the pool, i.e. it doesn't create an actual connection each time (but will do a logical reset to clean it up).
This also allows much more flexible use from multiple threads. Using a static connection in a web app, for example, would be horrendous for blocking.
From the code it seems that you are utilizing an already opened connection. May be there's a transaction pending previously on the same connection.