I am using old school ADO.net with C# so there is a lot of this kind of code. Is it better to make one function per query and open and close db each time, or run multiple queries with the same connection obect? Below is just one query for example purpose only.
using (SqlConnection connection = new SqlConnection(ConfigurationManager.ConnectionStrings["DBConnectMain"].ConnectionString))
{
// Add user to database, so they can't vote multiple times
string sql = " insert into PollRespondents (PollId, MemberId) values (#PollId, #MemberId)";
SqlCommand sqlCmd = new SqlCommand(sql, connection);
sqlCmd.Parameters.Add("#PollId", SqlDbType.Int);
sqlCmd.Parameters["#PollId"].Value = PollId;
sqlCmd.Parameters.Add("#MemberId", SqlDbType.Int);
sqlCmd.Parameters["#MemberId"].Value = Session["MemberId"];
try
{
connection.Open();
Int32 rowsAffected = (int)sqlCmd.ExecuteNonQuery();
}
catch (Exception ex)
{
//Console.WriteLine(ex.Message);
}
}
Well, you could measure; but as long as you are using the connections (so they are disposed even if you get an exception), and have pooling enabled (for SQL server it is enabled by default) it won't matter hugely; closing (or disposing) just returns the underlying connection to the pool. Both approaches work. Sorry, that doesn't help much ;p
Just don't keep an open connection while you do other lengthy non-db work. Close it and re-open it; you may actually get the same underlying connection back, but somebody else (another thread) might have made use of it while you weren't.
For most cases, opening and closing a connection per query is the way to go (as Chris Lively pointed out). However, There are some cases where you'll run into performance bottlenecks with this solution though.
For example, when dealing with very large volumes of relatively quick to execute queries that are dependent on previous results, I might suggest executing multiple queries in a single connection. You might encounter this when doing batch processing of data, or data massaging for reporting purposes.
Always be sure to use the 'using' wrapper to avoid mem leaks though, regardless of which pattern you follow.
If the methods are structured such that a single command is executed within a single method, then Yes: instantiate and dispose of the connection for each command.
If the methods are structured such that you have multiple commands executed in the same block of code, then the outer block needs to be the using clause for the connection.
ADO is very good about connection pooling so instantiating and disposing of the command object is going to be extremely fast and really won't impact performance.
As an example, we have a few pages that will execute update to 50 queries in order to compose the page. Because there is branching code to determine the queries to run, we have each of them wrapped with their own using (connection...) clauses.
We once ripped those out and grabbed one connection object and passed it to the individual methods. This had exactly zero performance improvement while complicating the hell out of the code with all the exception clauses every where to ensure the connection was properly disposed at the end. At the end of the test, we rolled back the code to how it was before. Much cleaner to know exactly what was going on and when a connection was being used.
Well, as always, it depends. If you have 5 database call to make within the same method call, you should probably use a single connection.
However, holding onto connection while nothing is happening isn't usually advised from a scalability standpoint.
ADO.NET is old school now? Wow, you just made me feel old. To me Rogue Wave ODBC using Borland C++ on Windows 3.1 is old school.
To answer, in general you want to understand how your data drivers work. Understand such concepts as connection pooling and learn to profile the transaction costs associate with connecting / disconnecting and executing queries. Then take that knowledge and apply it it your situation.
Related
Throughout the program which I am currently working on, I realized that whenever I need to access to SQL Server, I just type a queryString and execute my query using SqlCommand and SqlConnection in C#. At a certain point during the program, I even opened a connection and ran a query in a "for loop".
Is it unhealthy for the program to constantly open-close connections?
***I am not very familiar with the terminology, therefore you might be having some problems understanding what I am talking about:
Is doing this very frequently may cause any problem?:
string queryString = "Some SQL Query";
public void(){
SqlConnection con = new Connection(conString);
SqlCommand cmd = new SqlCommand(queryString,con);
cmd.Parameters.AddWithValue("#SomeParam",someValue);
con.Open();
cmd.ExecuteNonQuery();
con.Close();
}
I use this template almost every class I create,(usually to get,update,insert data from/into a datatable).
Well, is it harmful?
The short answer is yes - it is inefficient to constantly open and close connections. Opening an actual connection is a very expensive process and managing a connection for the lifetime of its need (which usually is the lifetime of the application or process using it) is fraught with errors.
That is why connection pooling was introduced a long time ago. There is a layer beneath your application that will manage the physical opening/closing of connections in a more efficient way. This also helps prevent the chances that an open connection is lost and continues to stay open (which causes lots of problems). By default pooling is enabled so you don't need to do anything to use it.
With pooling - you write code to open a connection and use it for the duration of a particular section of code and then close it. If the connection pool has an open but unused connection, it will reuse it rather than open a new one. When you close the connection, that simply returns the connection to the pool and makes it available to the next open attempt. You should also get familiar with the c# using statement.
I inherited a ton of C# dotnet code that apparently in some place is not closing the connection or something to sql server and I get
The timeout period elapsed prior to obtaining a connection from the pool: This may have occurred because all pooled connections were in use and max pool size was reached
This happens quite a bit and usually at the same time but I don't know of anything that is scheduled. Also searching manually every little place in code is quite not possible. Any ideas how to fix this without manually going over every single function in the code?
The problem with not wanting to go through the code is that it's very likely that it's not a single instance or area and may depend on where you're getting the most traffic in the production environment. You could have a fairly innocuous looking query that could be improved slightly with a .AsNoTracking() or a better join/include rather than a single big query that is poorly optimized. There are also instances where the entire design is flawed (I came into a project once where sql connections were attached to individual classes, so there was no real way to wrap the connection in a using statement. We had to restructure all of it.
If you're using custom transactions or sql connection requests, make sure you're wrapping with a using and a try catch finally with a dispose. Relying on the garbage collector is not always great since there's not guarantee the garbage collector whill actually dispose immediately at the end of a using statement.
using (var conn = new SqlConnection())
{
try
{
// Transactions, SQL Commands, etc
}
catch (SqlException ex)
{
// Trans rollback, error handling
}
finally
{
// Tell the garbage collector to clean up this resource now.
conn.Dispose();
}
}
If you're strictly using a database context, some statements can be optimized marginally by using .AsNoTracking() for instances where you are only retrieving a result-set but have no intention of modifying the results from that query. Something like the following:
var tier = _dbContext.Tier.AsNoTracking()
.Include(t => t.NameSchema)
.Include(t => t.TierPackageGroups)
.Where(t => t.TierNumber == tierNumber).FirstOrDefault();
Make sure you're using joins or .Include wherever possible as opposed to lazy loading.
It's not the answer that you want to hear, and I've been in your shoes, but most likely the best solution is to go through the code. You could always start by cleaning up a few areas, gauging impact, a few more areas, gauging impact, etc. This sort of creep investigation can help narrow your search as you see improvements in some areas or don't see some in others.
If you're convinced it has to be a single query that you want to find, then there are tools for evaluating sql performance, but I usually pull in a DBA to handle that so I'm not familiar with those tools. If you're using Azure there is a panel where you can see the queries and have them organized by performance impact. It's not going to point you to the code, but it could give you a hint.
Since the question is specifically for a solution that doesn't require searching for code, the one other answer that will work is to just throw more hardware at the problem, but this a poor stop-gap that can be costly and ultimately just be delaying the problem or only solving for the problem 99% of the time (until you hit a peak traffic period, for instance). One client we had a few years back has probably spent more than $50k in additional hosting costs when they could've had us fix the issue for under $5k at the time of the initial request.
There's a lot of non-detailed questions on this one, so here goes.
What is the best practice for connection handling in C# with SQL Server 2008? We have an assembly (which in our case is used by a WCF Service) that makes calls to an SQL Server. In general it seems like you need three objects to do this: The connection object, the command object, and the reader object.
The only reliable way we've been able to get the calls to work is to do the following:
Open the connection.
Create the Command in a using() { } block
Create the Reader to handle the response.
Dispose of the reader.
Implicitly dispose of the Command at the end of the using() block
Close the connection.
We ran into an unusual problem when running the same command multiple times iteratively, where it would complain that there was already a command or reader object attached to the connection that was still open. The only rock solid solution was to close and reopen the connection with every command we did, iterative or just sequential (different commands.)
So this is the question, since I come from a mysql_pconnect background on DB connection handling.
Is it going to significantly impact performance to be opening and closing a connection for each command?
If so for 1., what is the proper workaround, or code structure to handle serially repeating a command?
Is there any way to reuse a connection, command or reader at all?
If not for 3., does this really impact performance or memory usage significantly (As in, our users would notice.)
To answer point 1, if you look at the documentation for SqlConnection you'll see it explain about connection pooling. This means that the SQL Server provider has a collection of connections readily available and each SqlConnection created simply gets the next available connection. Therefore, to get the best performance, it is advisable to keep creating SqlConnection objects and using them for short operations and then disposing of them, thereby returning back to the connection pool.
For point 3, I believe you can re-use an SqlConnection if you do SqlCommand.ExecuteNonQuery(), but if you use an SqlDataReader you cannot re-use the connection - it is tied to the SqlDataReader and must be closed/disposed of once finished.
In addition to #PeterMonks answer:
The "expensive", unmanaged part of the SqlConnection is re-used by the provider (connection pooling) as long as you use the same connection string. So while there is a small overhead to creating a new managed wrapper each time, it isn't actually a 1:1 relationship with creating connections to the SQL server instance, so it isnt as expensive as you might think.
To serially repeat a command that returns a data reader, you must a) always execute the command on the same thread (commands are not thread safe) and b) Close() or Dispose() the DataReader instances before creating the next one. You can do that by putting the DataReaders in a using block as well.
Here is how you put the reader into a using block:
using (var dr = myCommand.ExecuteReader(...)) {
// Previous discussions have indicated that a close in here,
// while seemingly redundant, can possibly help with the error
// you are seeing.
dr.Close();
}
Another useful technique, as #DavidStratton mentions, is to enable MARS, but be aware that there is overhead associated with keeping resultsets open- you still want to close your readers as soon as you are done with them, because unclosed, undisposed readers do represent significant resource allocations on the server and the client.
I have my business-logic implemented in simple static classes with static methods. Each of these methods opens/closes SQL connection when called:
public static void DoSomething()
{
using (SqlConnection connection = new SqlConnection("..."))
{
connection.Open();
// ...
connection.Close();
}
}
But I think passing the connection object around and avoiding opening and closing a connection saves performance. I made some tests long time ago with OleDbConnection class (not sure about SqlConnection), and it definitely helped to work like this (as far as I remember):
//pass around the connection object into the method
public static void DoSomething(SqlConnection connection)
{
bool openConn = (connection.State == ConnectionState.Open);
if (!openConn)
{
connection.Open();
}
// ....
if (openConn)
{
connection.Close();
}
}
So the question is - should I choose the method (a) or method (b) ? I read in another stackoverflow question that connection pooling saved performance for me, I don't have to bother at all...
PS. It's an ASP.NET app - connections exist only during a web-request. Not a win-app or service.
Stick to option a.
The connection pooling is your friend.
Use Method (a), every time. When you start scaling your application, the logic that deals with the state will become a real pain if you do not.
Connection pooling does what it says on the tin. Just think of what happens when the application scales, and how hard would it be to manually manage the connection open/close state. The connection pool does a fine job of automatically handling this. If you're worried about performance think about some sort of memory cache mechanism so that nothing gets blocked.
Always close connections as soon as you are done with them, so they underlying database connection can go back into the pool and be available for other callers. Connection pooling is pretty well optimised, so there's no noticeable penalty for doing so. The advice is basically the same as for transactions - keep them short and close when you're done.
It gets more complicated if you're running into MSDTC issues by using a single transaction around code that uses multiple connections, in which case you actually do have to share the connection object and only close it once the transaction is done with.
However you're doing things by hand here, so you might want to investigate tools that manage connections for you, like DataSets, Linq to SQL, Entity Framework or NHibernate.
Disclaimer: I know this is old, but I found an easy way to demonstrate this fact, so I'm putting in my two cents worth.
If you're having trouble believing that the pooling is really going to be faster, then give this a try:
Add the following somewhere:
using System.Diagnostics;
public static class TestExtensions
{
public static void TimedOpen(this SqlConnection conn)
{
Stopwatch sw = Stopwatch.StartNew();
conn.Open();
Console.WriteLine(sw.Elapsed);
}
}
Now replace all calls to Open() with TimedOpen() and run your program. Now, for each distinct connection string you have, the console (output) window will have a single long running open, and a bunch of very fast opens.
If you want to label them you can add new StackTrace(true).GetFrame(1) + to the call to WriteLine.
There are distinctions between physical and logical connections. DbConnection is a kind of logical connection and it uses underlying physical connection to Oracle. Closing/opening DbConnection doesn't affect your performance, but makes your code clean and stable - connection leaks are impossible in this case.
Also you should remember about cases when there are limitations for parallel connections on db server - taking that into account it is necessary to make your connections very short.
Connection pool frees you from connection state checking - just open, use and immediately close them.
Normally you should keep one connect for each transaction(no parallel computes)
e.g when user execute charge action, your application need find user's balance first and update it, they should use same connection.
Even if ado.net has its connection pool, dispatching connection cost is very low, but reuse connection is more better choice.
Why not keep only one connection in application
Because the connection is blocking when you execute some query or command,
so that means your application is only doing one db operation at sametime,
how poor performance it is.
One more issue is that your application will always have a connection even though your user is just open it but no operations.If there are many user open your application, db server will cost all of its connection source in soon while your users have not did anything.
In my application, I am querying database with same sql every one second. I need to know what measure should I take. Will it ever cause any problem like " The
timeout period elapsed prior to obtaining a connection from the pool." or any other like that?
Currently, i am creating a new connection every second and the disposing it after it is used. Should I reuse connection in this case.
Whats the best approach?
IMHO best practice is to pump-and-dump connections as quickly as possible - use them to get access to what you need and close them right away. Your enemy performance-wise isn't the overhead it takes to create a connection (although there is some involved there) - it's locking a table in your database.
If you're looking to optimize your application, you should try to implement some sort of caching mechanism that saves you from having to make a round-trip to the database for each lookup. That would be to your benefit performance-wise.
Another thing you can do is use read-only connections where you can - they require less overhead than traditional ones and will improve your performance also.
You should definitely open and close the connection each time. Indeed, if your using block has much code after the last use of the connection, call Close() to get it back in the pool as soon as possible. That way the chance of another use not needing to open a completely new connection is reduced (see What does "opening a connection" actually mean? for a bit more on when a real connection is opened and when one is taken from the pool).
Is this "once a second" an average across different threads, or all on the one thread? If it's all on the one thread it doesn't matter, indeed it might even be slightly faster to keep the connection object open, because either way there won't be contention for it.
I would certainly consider caching results, though this has downsides in memory use, along with potentially complicated issues about when the cached results need to be refreshed - really this could be anywhere from trivial to impossible depending on just what you are doing.
It's also clearly a query to go that extra mile when optimising.
Why do you need to do this?
You could try caching the data to reduce the load on your database. Do you need data that is 1 second old, or is 5 seconds ok.
Closing the connection after each time you use it is OK. It does not really get closed, it just goes back into the connection pool.
If the library you're using does indeed pool the connections for you then it doesn't make a difference. If not, then it would be better to use the same connection multiple times. Creating a connection is time consuming.
A few questions...
What data are you getting from the database?
Could that data be stored in application memory?
There is no problem in such approach if you dispose connections like this:
using (SqlConnection cnn = new SqlConnection(ConnectionString))
{
SqlCommand cmd = new SqlCommand(commandText, cnn)
{
CommandType = CommandType.Text
};
DataSet ds = new DataSet();
SqlDataAdapter da = new SqlDataAdapter(cmd);
da.Fill(ds);
return ds;
}
The only problem may happen - is decreasing your db performance if your hardware isn't good enought.