There's a lot of non-detailed questions on this one, so here goes.
What is the best practice for connection handling in C# with SQL Server 2008? We have an assembly (which in our case is used by a WCF Service) that makes calls to an SQL Server. In general it seems like you need three objects to do this: The connection object, the command object, and the reader object.
The only reliable way we've been able to get the calls to work is to do the following:
Open the connection.
Create the Command in a using() { } block
Create the Reader to handle the response.
Dispose of the reader.
Implicitly dispose of the Command at the end of the using() block
Close the connection.
We ran into an unusual problem when running the same command multiple times iteratively, where it would complain that there was already a command or reader object attached to the connection that was still open. The only rock solid solution was to close and reopen the connection with every command we did, iterative or just sequential (different commands.)
So this is the question, since I come from a mysql_pconnect background on DB connection handling.
Is it going to significantly impact performance to be opening and closing a connection for each command?
If so for 1., what is the proper workaround, or code structure to handle serially repeating a command?
Is there any way to reuse a connection, command or reader at all?
If not for 3., does this really impact performance or memory usage significantly (As in, our users would notice.)
To answer point 1, if you look at the documentation for SqlConnection you'll see it explain about connection pooling. This means that the SQL Server provider has a collection of connections readily available and each SqlConnection created simply gets the next available connection. Therefore, to get the best performance, it is advisable to keep creating SqlConnection objects and using them for short operations and then disposing of them, thereby returning back to the connection pool.
For point 3, I believe you can re-use an SqlConnection if you do SqlCommand.ExecuteNonQuery(), but if you use an SqlDataReader you cannot re-use the connection - it is tied to the SqlDataReader and must be closed/disposed of once finished.
In addition to #PeterMonks answer:
The "expensive", unmanaged part of the SqlConnection is re-used by the provider (connection pooling) as long as you use the same connection string. So while there is a small overhead to creating a new managed wrapper each time, it isn't actually a 1:1 relationship with creating connections to the SQL server instance, so it isnt as expensive as you might think.
To serially repeat a command that returns a data reader, you must a) always execute the command on the same thread (commands are not thread safe) and b) Close() or Dispose() the DataReader instances before creating the next one. You can do that by putting the DataReaders in a using block as well.
Here is how you put the reader into a using block:
using (var dr = myCommand.ExecuteReader(...)) {
// Previous discussions have indicated that a close in here,
// while seemingly redundant, can possibly help with the error
// you are seeing.
dr.Close();
}
Another useful technique, as #DavidStratton mentions, is to enable MARS, but be aware that there is overhead associated with keeping resultsets open- you still want to close your readers as soon as you are done with them, because unclosed, undisposed readers do represent significant resource allocations on the server and the client.
Related
I am using old school ADO.net with C# so there is a lot of this kind of code. Is it better to make one function per query and open and close db each time, or run multiple queries with the same connection obect? Below is just one query for example purpose only.
using (SqlConnection connection = new SqlConnection(ConfigurationManager.ConnectionStrings["DBConnectMain"].ConnectionString))
{
// Add user to database, so they can't vote multiple times
string sql = " insert into PollRespondents (PollId, MemberId) values (#PollId, #MemberId)";
SqlCommand sqlCmd = new SqlCommand(sql, connection);
sqlCmd.Parameters.Add("#PollId", SqlDbType.Int);
sqlCmd.Parameters["#PollId"].Value = PollId;
sqlCmd.Parameters.Add("#MemberId", SqlDbType.Int);
sqlCmd.Parameters["#MemberId"].Value = Session["MemberId"];
try
{
connection.Open();
Int32 rowsAffected = (int)sqlCmd.ExecuteNonQuery();
}
catch (Exception ex)
{
//Console.WriteLine(ex.Message);
}
}
Well, you could measure; but as long as you are using the connections (so they are disposed even if you get an exception), and have pooling enabled (for SQL server it is enabled by default) it won't matter hugely; closing (or disposing) just returns the underlying connection to the pool. Both approaches work. Sorry, that doesn't help much ;p
Just don't keep an open connection while you do other lengthy non-db work. Close it and re-open it; you may actually get the same underlying connection back, but somebody else (another thread) might have made use of it while you weren't.
For most cases, opening and closing a connection per query is the way to go (as Chris Lively pointed out). However, There are some cases where you'll run into performance bottlenecks with this solution though.
For example, when dealing with very large volumes of relatively quick to execute queries that are dependent on previous results, I might suggest executing multiple queries in a single connection. You might encounter this when doing batch processing of data, or data massaging for reporting purposes.
Always be sure to use the 'using' wrapper to avoid mem leaks though, regardless of which pattern you follow.
If the methods are structured such that a single command is executed within a single method, then Yes: instantiate and dispose of the connection for each command.
If the methods are structured such that you have multiple commands executed in the same block of code, then the outer block needs to be the using clause for the connection.
ADO is very good about connection pooling so instantiating and disposing of the command object is going to be extremely fast and really won't impact performance.
As an example, we have a few pages that will execute update to 50 queries in order to compose the page. Because there is branching code to determine the queries to run, we have each of them wrapped with their own using (connection...) clauses.
We once ripped those out and grabbed one connection object and passed it to the individual methods. This had exactly zero performance improvement while complicating the hell out of the code with all the exception clauses every where to ensure the connection was properly disposed at the end. At the end of the test, we rolled back the code to how it was before. Much cleaner to know exactly what was going on and when a connection was being used.
Well, as always, it depends. If you have 5 database call to make within the same method call, you should probably use a single connection.
However, holding onto connection while nothing is happening isn't usually advised from a scalability standpoint.
ADO.NET is old school now? Wow, you just made me feel old. To me Rogue Wave ODBC using Borland C++ on Windows 3.1 is old school.
To answer, in general you want to understand how your data drivers work. Understand such concepts as connection pooling and learn to profile the transaction costs associate with connecting / disconnecting and executing queries. Then take that knowledge and apply it it your situation.
I have my business-logic implemented in simple static classes with static methods. Each of these methods opens/closes SQL connection when called:
public static void DoSomething()
{
using (SqlConnection connection = new SqlConnection("..."))
{
connection.Open();
// ...
connection.Close();
}
}
But I think passing the connection object around and avoiding opening and closing a connection saves performance. I made some tests long time ago with OleDbConnection class (not sure about SqlConnection), and it definitely helped to work like this (as far as I remember):
//pass around the connection object into the method
public static void DoSomething(SqlConnection connection)
{
bool openConn = (connection.State == ConnectionState.Open);
if (!openConn)
{
connection.Open();
}
// ....
if (openConn)
{
connection.Close();
}
}
So the question is - should I choose the method (a) or method (b) ? I read in another stackoverflow question that connection pooling saved performance for me, I don't have to bother at all...
PS. It's an ASP.NET app - connections exist only during a web-request. Not a win-app or service.
Stick to option a.
The connection pooling is your friend.
Use Method (a), every time. When you start scaling your application, the logic that deals with the state will become a real pain if you do not.
Connection pooling does what it says on the tin. Just think of what happens when the application scales, and how hard would it be to manually manage the connection open/close state. The connection pool does a fine job of automatically handling this. If you're worried about performance think about some sort of memory cache mechanism so that nothing gets blocked.
Always close connections as soon as you are done with them, so they underlying database connection can go back into the pool and be available for other callers. Connection pooling is pretty well optimised, so there's no noticeable penalty for doing so. The advice is basically the same as for transactions - keep them short and close when you're done.
It gets more complicated if you're running into MSDTC issues by using a single transaction around code that uses multiple connections, in which case you actually do have to share the connection object and only close it once the transaction is done with.
However you're doing things by hand here, so you might want to investigate tools that manage connections for you, like DataSets, Linq to SQL, Entity Framework or NHibernate.
Disclaimer: I know this is old, but I found an easy way to demonstrate this fact, so I'm putting in my two cents worth.
If you're having trouble believing that the pooling is really going to be faster, then give this a try:
Add the following somewhere:
using System.Diagnostics;
public static class TestExtensions
{
public static void TimedOpen(this SqlConnection conn)
{
Stopwatch sw = Stopwatch.StartNew();
conn.Open();
Console.WriteLine(sw.Elapsed);
}
}
Now replace all calls to Open() with TimedOpen() and run your program. Now, for each distinct connection string you have, the console (output) window will have a single long running open, and a bunch of very fast opens.
If you want to label them you can add new StackTrace(true).GetFrame(1) + to the call to WriteLine.
There are distinctions between physical and logical connections. DbConnection is a kind of logical connection and it uses underlying physical connection to Oracle. Closing/opening DbConnection doesn't affect your performance, but makes your code clean and stable - connection leaks are impossible in this case.
Also you should remember about cases when there are limitations for parallel connections on db server - taking that into account it is necessary to make your connections very short.
Connection pool frees you from connection state checking - just open, use and immediately close them.
Normally you should keep one connect for each transaction(no parallel computes)
e.g when user execute charge action, your application need find user's balance first and update it, they should use same connection.
Even if ado.net has its connection pool, dispatching connection cost is very low, but reuse connection is more better choice.
Why not keep only one connection in application
Because the connection is blocking when you execute some query or command,
so that means your application is only doing one db operation at sametime,
how poor performance it is.
One more issue is that your application will always have a connection even though your user is just open it but no operations.If there are many user open your application, db server will cost all of its connection source in soon while your users have not did anything.
I came across an article saying that using my sqlConnection like this :
using (SqlConnection sqlConnection = new SqlConnection(Config.getConnectionString()))
{
using (SqlDataAdapter dataAdapter = new SqlDataAdapter(query, sqlConnection))
{
dataAdapter.Fill(dataSet);
}
}
increases performance because it disposes the objects at the end of your method. So i have been coding with 'Using' for a while now, after chatting with some other developers they said that that creating and destroying the instance multiple times wont really increase performance.
What are the performance implications on the sqlserver and system resources if I am using 'Using' on all of my dataAccess methods. Will the sqlServer be hit harder because of the connection being connected and reconnected multiple times?
SqlConnection, by default, has connection pooling enabled. The Dispose() simply releases the connection to the pool sooner. This means other code can then re-use this connection, reducing the connections to the SQL server, and reducing the time to establish a physical connection.
So yes: it can improve overall performance.
The alternatives:
if your code exits cleanly and you always remember to Close() the connection, then probably no difference
if your code throws an exception (that you haven't handled), or you forget to Close() the connection, then you could be leaving unused connections lying around until there is enough memory pressure to trigger GC and the finalizer. This could mean you need more physical connections to the SQL server (a pain), and every time a new underlying connection is needed it has to take the performance hit of establishing the actual database connection
Overall, though - think of IDisposable as a contract; it is your job as a .NET developer to notice IDisposable resources, and actively Dispose() them when you are done, ideally with using if the usage is tightly scoped (like in this case).
It has no significant influence on performance in most cases.
All the using().... construct makes sure is that the SqlConnection is freed / disposed of after it's done its job. That's all there is - no magic performance boost....
Sure - creating and disposing objects does cost a bit of performance - it's either that, or then you unnecessarily keep objects in your memory and connections to your SQL Server open for much longer than needed.
I would vote for using the using() {...} approach 100% of the time - it's cleaner, it's safer, it's just better programming practice. The performance "hit" you might take is miniscule and not worth the trouble.
Marc
It increases performance only in the sense that, after your connection instance has been disposed, the physical connection in the pool can be re-used by another thread. If you kept it open, then another thread trying to open a connection would add a new physical connection to the pool.
ADO.NET has such feature as connection pooling, therefore if you intensively open connections, most likely connection will not be disposed, only returned to the pool.
If you are doing several database operations after each other, you should use the same connection instead of creating one connection for each. Otherwise you should close the connection as soon as possible, so that it is returned to the connection pool and can be reused.
You should always use a using block for the connection, so that you are sure that they are closed properly. If you fail to close a connection object it will stay in memory until the garbage collector removes it, hogging a database connection. That means that the next opreation can't reuse the connection from the pool but it has to establish a completely new connection, which takes a lot longer.
there is a performance improvement .
if you use using
E.g :
using (SqlConnection sqlConnection = new SqlConnection("ConnectionString"))
{
}
The compiler automatically add try and finally.
alt text http://img111.imageshack.us/img111/4514/using.jpg
I am using multiple queries to pull data from the same server in my application. The issue is that I have to open a new connection every time I have a new query.
Is it even possible to:
Open the connection
Run query
Pull results
Run another query
Pull another result
Run final query
Pull another result
Close connection.
Although you may not yet know it, you are doing it correctly.
Open the connection, do your query, close it. Preferably using a using block or try/finally.
This may sound like a lot of overhead, but the connection pool in the .NET Framework Data Provider for SQL Server will actually optimize this for you.
In fact closing the connection is recommended.
Here is a quote from the documentation:
It is recommended that you always
close the Connection when you are
finished using it in order for the
connection to be returned to the pool.
This can be done using either the
Close or Dispose methods of the
Connection object. Connections that
are not explicitly closed might not be
added or returned to the pool. For
example, a connection that has gone
out of scope but that has not been
explicitly closed will only be
returned to the connection pool if the
maximum pool size has been reached and
the connection is still valid.
Here is an example of some code that does this:
try {
conn.Open();
// Perform query here
} finally {
conn.Close();
}
For reference:
http://msdn.microsoft.com/en-us/library/8xx3tyca(VS.71).aspx
If you are using ASP.NET with the same connection string you will be using a pooled connection that may never get physically closed, so you will pretty much always use an available open connection.
It's very possible. Assuming that you are talking about Connection and a DataReader. If you have to create a different connection every time, it sound like something is going wrong.
Without seeing any code, I am guessing that you are leaving the DataReader open. This is a BIG mistake. By default DataReaders completely consume the connection and leaving it unclosed can lead leaks. Close the DataReader, then execute another. I'd recommend wrapping the DataReader in a using block.
Rob
Short answer: Yes. This should be possible with most data providers.
Long answer: It depends on what you are using for your data access. However, you probably do not need to worry about it. Many data provider frameworks have connection pooling built in, so the subsequent connection creation/opening shouldn't "really" open a connection.
Sure, if you're using a SqlConnection object you can just do something like this:
connection.Open();
cmd.ExecuteReader(); // or any other form of getting the data
cmd2.ExecuteReader();
.
.
.
.
connection.Close();
I'd also like to add, if you're using a few SqlDataAdapters for your queries, although you normally don't need to open the connection by yourself, if you DO explicitly call connection.Open() it then won't close the connection for you automatically, allowing you to execute multiple queries with only one connection.
If you are using C# to open a connection. use using statement will help you clean up the resource/connection even if there is some excepion throwing out.
using (SqlConnection connection =
new SqlConnection(connectionString)
{
connection.Open();
//issue command
}
And read this:
http://msdn.microsoft.com/en-us/library/8xx3tyca(VS.71).aspx, you can "Controlling Connection Pooling with Connection String Keywords", and the system will handle pooling for you.
From MSDN:
While the SqlDataReader is being used, the associated SqlConnection is busy serving the SqlDataReader, and no other operations can be performed on the SqlConnection other than closing it. This is the case until the Close method of the SqlDataReader is called. For example, you cannot retrieve output parameters until after you call Close.
a) Why couldn’t you use SqlConnection for anything else? After all, when ExecuteQuery() returns SqlDataReader object, the data from DB has already been retrieved and populated SqlDatareader object. Thus I don’t see how or why SqlConnection should still be serving SqlDataReader object?!
b) More importantly, for what reason would it be bad idea to retrieve output parameters before you call Close() on SqlDataReader?
c) When above quote mentions that no other operations can be performed on SqlConnection, what operations does it have in mind? Just those that would require to connect to remote sql server, or we can’t use any members of SqlConnection instance?
a) When ExecuteReader returns, the data has not all been retrieved and populated in the reader, it may still be streaming back from the database. That's the whole point of the SqlDataReader because it's more efficient to do this than to load it all up front.
b) You can't retrieve output parameters until after the reader has finished because of the way the Tabular Data Stream (TDS) protocol is structured. The output parameters are not physically sent down the stream until after the result set data.
c) It means none of the operations except Close are documented as being guaranteed to work. Whether they actually do work or not is irrelevant because that is an implementation detail rather than a contract, and programming against implementation details is a recipe for disaster.
Why do you want to re-use the connection anyway? The connections that SqlConnection uses are pooled behind the scenes, so when you dispose one it doesn't really get rid of all the resources, it just returns the connection to the pool. This means you can use the connection for the shortest time possible, ideally within a using block, and not have to worry about this type of thing. Just create a new connection whenever you need one, and don't bother trying to re-use them as it's already happening behind the scenes.
Uhm... you can in SQL Server 2005. Please read this page:
http://msdn.microsoft.com/en-us/library/ms345109.aspx
SqlDataReader doesn't return the entire table in one fetch, much like a StreamReader wouldn't return the entire file. Obviously in reality it will for small tables, but only one connection is being served. That's using blocking for network access though - just open a second connection if you need to do something else asynchronously.
The code you have posted is actually designed to be run with the jQuery library. You can find out more information about jQuery at http://jquery.com/. Mootools, while also a javascript library, is a different thing and the code you posted would not work (you can learn more about mootools at http://mootools.net/).
What exactly are you attempting to achieve by including this code on your page / site?