I want to optimize the sql connection in my wep application its create in .net mvc 4, i read that ado.net automatically manager the connection pooling but i'm some lost respect how exactly implement that, is correct if i create a global object with the connection in the Application_Start class then pass the connection object through all data object in my application ? something like this
protected void Application_Start()
{
...
SqlConnection conn = new SqlConnection("Connection String...");
DAOPeople daoPeople = new DAOPeople(conn);
...
}
in that way i avoided create a new SqlConnection for each dao, is correct?
No, don't do that. You'll end up with a bottleneck at your connection object, as that single connection is shared across all sessions and requests to your app.
For connection pooling, you do the exact opposite: don't try to share or re-use a single connection object; do just create a new SqlConnection every time you need it, open it on the spot, and make sure it's disposed as soon as you're done via a using block. Even though your code looks like you're opening and closing a lot of connections, the connection pooling feature is built in and ensures you keep drawing from a small number of existing connections in the same pool.
That said, if you're on a really large site, you can do a little better. One thing large sites will do to help scale is avoid unnecessary memory allocations, and there is some memory that goes with creating an SqlConnection object. Instead they might, for example, have one main SqlConnection per HTTP request, with the possibility of either enabling MARS or having an additional secondary connection object in the request so they can run some things asynchronously. But this is only something the top 0.1% need to care about, and if you're at this level you're measuring to find out where the proper balance is for your particular site and load.
Related
Throughout the program which I am currently working on, I realized that whenever I need to access to SQL Server, I just type a queryString and execute my query using SqlCommand and SqlConnection in C#. At a certain point during the program, I even opened a connection and ran a query in a "for loop".
Is it unhealthy for the program to constantly open-close connections?
***I am not very familiar with the terminology, therefore you might be having some problems understanding what I am talking about:
Is doing this very frequently may cause any problem?:
string queryString = "Some SQL Query";
public void(){
SqlConnection con = new Connection(conString);
SqlCommand cmd = new SqlCommand(queryString,con);
cmd.Parameters.AddWithValue("#SomeParam",someValue);
con.Open();
cmd.ExecuteNonQuery();
con.Close();
}
I use this template almost every class I create,(usually to get,update,insert data from/into a datatable).
Well, is it harmful?
The short answer is yes - it is inefficient to constantly open and close connections. Opening an actual connection is a very expensive process and managing a connection for the lifetime of its need (which usually is the lifetime of the application or process using it) is fraught with errors.
That is why connection pooling was introduced a long time ago. There is a layer beneath your application that will manage the physical opening/closing of connections in a more efficient way. This also helps prevent the chances that an open connection is lost and continues to stay open (which causes lots of problems). By default pooling is enabled so you don't need to do anything to use it.
With pooling - you write code to open a connection and use it for the duration of a particular section of code and then close it. If the connection pool has an open but unused connection, it will reuse it rather than open a new one. When you close the connection, that simply returns the connection to the pool and makes it available to the next open attempt. You should also get familiar with the c# using statement.
I have observed an interesting behavior of the performance of connection pooling in an client application we created. When ever the user clicks on an object, more object specific data is loaded from the database. This takes somewhere between 10 and 30 queries per click depending on the object.
This was done by using connection pooling and each query was dispatched on a new connection from the pool and the connection was closed after the query ran.
I have analyzed the queries in the profiler for performance optimization and saw that there where a lot of audit login/logout entries. Additionally the performance was not optimal eventhough the queries themselfs where running well (only index seek/scan operators).
Just for trying it out I disabled the pooling and modified the code to keep one connection per client application and reusing it. This made the entire application a lot more responsive, and all the audit login/logout entries disappeared from the profiler.
How is this possible? Shouldn't the connections stay open or if they actually stay open at least not be this slow? Is it possible that we are using the SqlConnection class wrong resulting in disabled pooling?
I have read the other posts regarding pooling but have not found anything about a perceivable speed difference between pooling connections and reusing the same connection.
SqlConnection con = new SqlConnection(_connectionString);
The connection is handed off to a wrapping class Session which provides transactional functionality.
class Session{
Session(connection);
Abort();
Commit();
}
The connection is closed in Abort() and Commit(). One of these is always called.
If I understand you correctly - the connection is being "new" per session. if you want all instances to share the connection you should make it static.
put it in global.asax:
public static SqlConnection con;
protected void Application_Start(object sender, EventArgs e)
{
con = new SqlConnection(_connectionString);
}
in that way you will be sharing the same connection between your sessions.
There's a lot of non-detailed questions on this one, so here goes.
What is the best practice for connection handling in C# with SQL Server 2008? We have an assembly (which in our case is used by a WCF Service) that makes calls to an SQL Server. In general it seems like you need three objects to do this: The connection object, the command object, and the reader object.
The only reliable way we've been able to get the calls to work is to do the following:
Open the connection.
Create the Command in a using() { } block
Create the Reader to handle the response.
Dispose of the reader.
Implicitly dispose of the Command at the end of the using() block
Close the connection.
We ran into an unusual problem when running the same command multiple times iteratively, where it would complain that there was already a command or reader object attached to the connection that was still open. The only rock solid solution was to close and reopen the connection with every command we did, iterative or just sequential (different commands.)
So this is the question, since I come from a mysql_pconnect background on DB connection handling.
Is it going to significantly impact performance to be opening and closing a connection for each command?
If so for 1., what is the proper workaround, or code structure to handle serially repeating a command?
Is there any way to reuse a connection, command or reader at all?
If not for 3., does this really impact performance or memory usage significantly (As in, our users would notice.)
To answer point 1, if you look at the documentation for SqlConnection you'll see it explain about connection pooling. This means that the SQL Server provider has a collection of connections readily available and each SqlConnection created simply gets the next available connection. Therefore, to get the best performance, it is advisable to keep creating SqlConnection objects and using them for short operations and then disposing of them, thereby returning back to the connection pool.
For point 3, I believe you can re-use an SqlConnection if you do SqlCommand.ExecuteNonQuery(), but if you use an SqlDataReader you cannot re-use the connection - it is tied to the SqlDataReader and must be closed/disposed of once finished.
In addition to #PeterMonks answer:
The "expensive", unmanaged part of the SqlConnection is re-used by the provider (connection pooling) as long as you use the same connection string. So while there is a small overhead to creating a new managed wrapper each time, it isn't actually a 1:1 relationship with creating connections to the SQL server instance, so it isnt as expensive as you might think.
To serially repeat a command that returns a data reader, you must a) always execute the command on the same thread (commands are not thread safe) and b) Close() or Dispose() the DataReader instances before creating the next one. You can do that by putting the DataReaders in a using block as well.
Here is how you put the reader into a using block:
using (var dr = myCommand.ExecuteReader(...)) {
// Previous discussions have indicated that a close in here,
// while seemingly redundant, can possibly help with the error
// you are seeing.
dr.Close();
}
Another useful technique, as #DavidStratton mentions, is to enable MARS, but be aware that there is overhead associated with keeping resultsets open- you still want to close your readers as soon as you are done with them, because unclosed, undisposed readers do represent significant resource allocations on the server and the client.
I have my business-logic implemented in simple static classes with static methods. Each of these methods opens/closes SQL connection when called:
public static void DoSomething()
{
using (SqlConnection connection = new SqlConnection("..."))
{
connection.Open();
// ...
connection.Close();
}
}
But I think passing the connection object around and avoiding opening and closing a connection saves performance. I made some tests long time ago with OleDbConnection class (not sure about SqlConnection), and it definitely helped to work like this (as far as I remember):
//pass around the connection object into the method
public static void DoSomething(SqlConnection connection)
{
bool openConn = (connection.State == ConnectionState.Open);
if (!openConn)
{
connection.Open();
}
// ....
if (openConn)
{
connection.Close();
}
}
So the question is - should I choose the method (a) or method (b) ? I read in another stackoverflow question that connection pooling saved performance for me, I don't have to bother at all...
PS. It's an ASP.NET app - connections exist only during a web-request. Not a win-app or service.
Stick to option a.
The connection pooling is your friend.
Use Method (a), every time. When you start scaling your application, the logic that deals with the state will become a real pain if you do not.
Connection pooling does what it says on the tin. Just think of what happens when the application scales, and how hard would it be to manually manage the connection open/close state. The connection pool does a fine job of automatically handling this. If you're worried about performance think about some sort of memory cache mechanism so that nothing gets blocked.
Always close connections as soon as you are done with them, so they underlying database connection can go back into the pool and be available for other callers. Connection pooling is pretty well optimised, so there's no noticeable penalty for doing so. The advice is basically the same as for transactions - keep them short and close when you're done.
It gets more complicated if you're running into MSDTC issues by using a single transaction around code that uses multiple connections, in which case you actually do have to share the connection object and only close it once the transaction is done with.
However you're doing things by hand here, so you might want to investigate tools that manage connections for you, like DataSets, Linq to SQL, Entity Framework or NHibernate.
Disclaimer: I know this is old, but I found an easy way to demonstrate this fact, so I'm putting in my two cents worth.
If you're having trouble believing that the pooling is really going to be faster, then give this a try:
Add the following somewhere:
using System.Diagnostics;
public static class TestExtensions
{
public static void TimedOpen(this SqlConnection conn)
{
Stopwatch sw = Stopwatch.StartNew();
conn.Open();
Console.WriteLine(sw.Elapsed);
}
}
Now replace all calls to Open() with TimedOpen() and run your program. Now, for each distinct connection string you have, the console (output) window will have a single long running open, and a bunch of very fast opens.
If you want to label them you can add new StackTrace(true).GetFrame(1) + to the call to WriteLine.
There are distinctions between physical and logical connections. DbConnection is a kind of logical connection and it uses underlying physical connection to Oracle. Closing/opening DbConnection doesn't affect your performance, but makes your code clean and stable - connection leaks are impossible in this case.
Also you should remember about cases when there are limitations for parallel connections on db server - taking that into account it is necessary to make your connections very short.
Connection pool frees you from connection state checking - just open, use and immediately close them.
Normally you should keep one connect for each transaction(no parallel computes)
e.g when user execute charge action, your application need find user's balance first and update it, they should use same connection.
Even if ado.net has its connection pool, dispatching connection cost is very low, but reuse connection is more better choice.
Why not keep only one connection in application
Because the connection is blocking when you execute some query or command,
so that means your application is only doing one db operation at sametime,
how poor performance it is.
One more issue is that your application will always have a connection even though your user is just open it but no operations.If there are many user open your application, db server will cost all of its connection source in soon while your users have not did anything.
I read that .NET uses connection pooling.
For example, if I instantiate a bunch of SqlConnection objects with the same connection string, then internally .NET will know to use the same connection.
Is this correct?
Also, in a big web-based application, any tips on the best way to harness this "power" ?
Setting up the TCP connection between your Web application and SQL Server can be an expensive operation. Connection pooling allows connections to the database to be reused for subsequent data requests. Rather than setting up a new TCP connection on each request, a new connection is set up only when one is not available in the connection pool. When the connection is closed, it is returned to the pool where it remains connected to the database, as opposed to completely tearing down that TCP connection.
Always close your connections when you're finished with them. No matter what anyone says about garbage collection within the Microsoft .NET Framework, always call Close or Dispose explicitly on your connection when you are finished with it. Do not trust the common language runtime (CLR) to clean up and close your connection for you. The CLR will eventually destroy the class and force the connection closed, but you have no guarantee when the garbage collection on the object will actually happen.
To use connection pooling optimally, there are a couple of rules to live by. First, open the connection, do the work, and then close the connection. It's okay to open and close the connection multiple times on each request if you have to, rather than keeping the connection open and passing it around through different methods. Second, use the same connection string (and the same thread identity if you're using integrated authentication). If you don't use the same connection string, for example customizing the connection string based on the logged-in user, you won't get the same optimization value provided by connection pooling. And if you use integrated authentication while impersonating a large set of users, your pooling will also be much less effective.
The .NET CLR data performance counters can be very useful when attempting to track down any performance issues that are related to connection pooling.
http://msdn.microsoft.com/en-us/magazine/cc163854.aspx
If you use the following syntax, when ever the using block is left the dispose method will be called, even if an exception occurs.
using(SqlConnection connection = new SqlConnection())
{
// Work with connection object here.
}
//connection object gets disposed here.
not sure if this is entirely related, but I just took over a project and noticed the original programming team failed to do something very important.
when you have a SQLConnection, let's call it conn and you do this:
conn.Open();
and then you perform some SQL statement, be it a select, insert or update. it is entirely possible that it will fail. So of course, you should do this:
try { conn.Open() }
catch (SqlException ex)
{
//do your logging/exception handling
}
however, people forget to add the Finally block.
finally {
if (conn.State == System.Data.ConnectionState.Open)
conn.Close();
}
you want to make sure if you have an exception that the connection does not stay open, so make sure you close it.