I'm writing a server application that's communication with a local sql server.
Each client will need to read or write data to the database.
Would it be better to have a thread safe class which will enqueue and executes the sql commands on a single sql connection? Or should I open a new connection for each command? Does it matter much for performance?
If you have a batch of statements that have to be executed after each other, you should use the same SqlConnection.
As soon as you do not longer need the SqlConnection, and you do not know when you will need a connection again, you should close the connection.
So, if you have to execute 2 insert statements and one update statement after each other, for instance, you should use the same SqlConnection.
The most important advantage here, is that you can put those statement in a transaction if necessary. Transactions cannot be shared accross connections.
When you're finished working with the DB, you can close the connection. By default, connection pooling is used, and the connection will be returned to the pool, so that it can be reused the next time you need a connection to the DB.
Connection lifetime should be short, but you should not use a separate connection for each DbCommand.
If you are using any flavor of ADO.NET, connection pooling will automatically be used (at least with SQL Server) unless you explicitly disable it, so there's no reason to do anything special about that.
Be sure to remember to Close your connections after each used - this will simply return the connection to the connection pool.
Usually you should create a new connection for each command and take advantage of the in-built connection pooling.
Related
I have an application that connect to a SQL Server database with high frequency. Inside this service, there are many scheduled tasks that run every second, and each time I'm executing some query.
I don't understand which solution is better in this condition.
Opening a single SqlConnection and keeping it open while application is running and execute all query with that connection
Each time I want to execute query, opening a new connection and after query execution, close the connection (does this solution suitable for so many scheduled task that runs every 1 second?)
I tried second solution, but is there any better choice?
How do ORMs like EF manage connections?
As you see i have many service. I cant change interval and the interval is important for me. but the code makes so many calls and im following a better way manage connection over database. Also I'm making connection with Using Statement.
Is there any better solution?
you should use SQL Connection Pool feature for that.
It automatically manages in the background if a connection needs to be open or can be reused.
Documentation: https://learn.microsoft.com/en-us/dotnet/framework/data/adonet/sql-server-connection-pooling?source=recommendations
Example copied from that page
using (SqlConnection connection = new SqlConnection(
"Integrated Security=SSPI;Initial Catalog=Northwind"))
{
connection.Open();
// Pool A is created.
}
using (SqlConnection connection = new SqlConnection(
"Integrated Security=SSPI;Initial Catalog=pubs"))
{
connection.Open();
// Pool B is created because the connection strings differ.
}
using (SqlConnection connection = new SqlConnection(
"Integrated Security=SSPI;Initial Catalog=Northwind"))
{
connection.Open();
// The connection string matches pool A.
}
By using the "using" statement, application checks if a connection in this pool can be reused before opening a new connection. So the overhead of opening and closing the connections disappears.
But after your last edit you seem to have other problems in your current architecture. Like the other poster recommends you can try to use the "with (nolock)" parameter in your sql statements. It creates dirty reads, but maybe that's ok for your application.
Alternatively if all your services use the same select statement maybe a stored procedure or a caching mechanism could help.
I assume that you are already opening/closing your SQL connections in either a "using" statement or explicitly in your code ( try/catch/finally ). If so you are already making use of connection pooling as it is enabled in ADO.Net by default ("By default, connection pooling is enabled in ADO.NET").
Therefore I don't think that your problem is so much a connection/resource problem as it is a database concurrency issue. I assume it to be either 1 of 2 issues :
Your code is making so many calls to the SQL server that it is exhausting all the available connections and nobody else can get one
Your code is locking tables in SQL that is causing other code/applications to timeout
If it is case 1, try and redesign your code to be "less chatty" to the database. Instead of making several inserts/updates per second, perhaps buffer the changes and make a single insert/update every 3-5 seconds in batch mode ( obvs if possible ). Or maybe your SQL statements are taking longer than 1 second to execute and you are calling them every second causing in a backlog scenario?
If it is case 2, try and redesign the SQL tables in such a way that the "reading" applications are not influenced by the "writing" application. Normally this involves a service that periodically writes aggregated data to a read-only table for viewing or at very least adding a "WITH(NOLOCK)" hint to the select clauses to allow dirty reads ( i.e. it wont lock the table to read, but may result in slightly out of date dataset i.e. eventual consistency )
Good luck
I'm new in WinForms .Net, I'm using Dapper to deal with MySql DBs.
Should I start a new SqlConnexion for each query or only one SqlConnexion is enough for the whole application? (I start it in the beginning and dispose it by closing the app)
Which is the best way? And why?
As Damien wrote in his comment to the question, it's not really clear if you are using MySql (and therefor, MySqlConnection) or Sql Server (as implied by the misspelled SqlConnection in the question).
However, that detail is not relevant to the answer, since SqlConnection and MySqlConnection both implement connection pooling - so best practice is to close and dispose them as soon as possible.
The basic concept of connection pooling is that the ADO.Net provider creates a connection to the database when you first use it, but when you close the connection in your application, ADO.Net keeps the underlying connection alive, so that the next time you open the connection to the database, you don't need to go through all the overhead of creating the actual connection - ADO.Net simple re-use the existing one.
Microsoft's recommend closing and disposing SqlConnection after each use, since that's the only way the underlying connection can go back to the pool - from SQL Server Connection Pooling (ADO.NET) page:
Caution
We strongly recommend that you always close the connection when you are finished using it so that the connection will be returned to the pool. You can do this using either the Close or Dispose methods of the Connection object, or by opening all connections inside a using statement in C#, or a Using statement in Visual Basic. Connections that are not explicitly closed might not be added or returned to the pool. For more information, see using Statement or How to: Dispose of a System Resource for Visual Basic.
MySql documentation also recommends against using an application wide connection. In fact, they recommend not creating an instance of MySqlConnection at all, but instead let MySqlCommand manage the connection for you, by using overloads that takes the connection string as arguments:
To work as designed, it is best to let the connection pooling system manage all connections. Do not create a globally accessible instance of MySqlConnection and then manually open and close it. This interferes with the way the pooling works and can lead to unpredictable results or even exceptions.
One approach that simplifies things is to avoid manually creating a MySqlConnection object. Instead use the overloaded methods that take a connection string as an argument. Using this approach, Connector/NET will automatically create, open, close and destroy connections, using the connection pooling system for best performance.
There is no need for you to start a new connection for each query because the connection is first built and maintained by the DBMS and your methods. all you have to do is to start it at first with a simple try catch syntax and use it in your program.
I've a C# client application that need to checks a table on a Postgres db every 15 minutes. The problem is that I need to install this client into more or less 200 client so, for that I need to close the DB connection after the query.
I use .Close() method but, if I check on pg_stat_activity table on Postgres DB, I can see the connection still open in IDLE status. How can I fix that issue? Is it possible to close definitely the connection?
thanks,
Andrea
Like most ADO.NET providers, Npgsql uses connection pooling by default. When you Close() the NpgsqlConnection object, an internal object representing the actual underlying connection that Npgsql uses goes into a pool to be re-used, saving the overhead of creating another unnecessarily. (See What does "opening a connection" actually mean? for more).
This suits most applications well, as it's common to want to use a connection several times in the space of a second.
It doesn't suit you at all, but if you include the option Pooling=false in your connection string, it will override this default, and Close() will indeed close the actual connection.
Hay,
My system needs to execute several major SQL`s (on Oracle DB) using the same connection (asynchronous).
What`s the best practice for this issue?
1. open single connection and execute every SQL statement on different thread (does it thread safe?)
2. create new connection and “open + close” it for every SQL statement
Thanks,
Hec
We've been calling Oracle SQL statements on multiple threads, and this is probably best, if your DB can handle the load and won't be the bottleneck anyway. HOWEVER, I think you need to create the connection on the thread that will be issuing the SQL command. You can (and probably should) also use connection pooling so your connections will be reused, rather than being re-established (and Oracle seems to be fine with re-using these from one thread to another).
I am using multiple queries to pull data from the same server in my application. The issue is that I have to open a new connection every time I have a new query.
Is it even possible to:
Open the connection
Run query
Pull results
Run another query
Pull another result
Run final query
Pull another result
Close connection.
Although you may not yet know it, you are doing it correctly.
Open the connection, do your query, close it. Preferably using a using block or try/finally.
This may sound like a lot of overhead, but the connection pool in the .NET Framework Data Provider for SQL Server will actually optimize this for you.
In fact closing the connection is recommended.
Here is a quote from the documentation:
It is recommended that you always
close the Connection when you are
finished using it in order for the
connection to be returned to the pool.
This can be done using either the
Close or Dispose methods of the
Connection object. Connections that
are not explicitly closed might not be
added or returned to the pool. For
example, a connection that has gone
out of scope but that has not been
explicitly closed will only be
returned to the connection pool if the
maximum pool size has been reached and
the connection is still valid.
Here is an example of some code that does this:
try {
conn.Open();
// Perform query here
} finally {
conn.Close();
}
For reference:
http://msdn.microsoft.com/en-us/library/8xx3tyca(VS.71).aspx
If you are using ASP.NET with the same connection string you will be using a pooled connection that may never get physically closed, so you will pretty much always use an available open connection.
It's very possible. Assuming that you are talking about Connection and a DataReader. If you have to create a different connection every time, it sound like something is going wrong.
Without seeing any code, I am guessing that you are leaving the DataReader open. This is a BIG mistake. By default DataReaders completely consume the connection and leaving it unclosed can lead leaks. Close the DataReader, then execute another. I'd recommend wrapping the DataReader in a using block.
Rob
Short answer: Yes. This should be possible with most data providers.
Long answer: It depends on what you are using for your data access. However, you probably do not need to worry about it. Many data provider frameworks have connection pooling built in, so the subsequent connection creation/opening shouldn't "really" open a connection.
Sure, if you're using a SqlConnection object you can just do something like this:
connection.Open();
cmd.ExecuteReader(); // or any other form of getting the data
cmd2.ExecuteReader();
.
.
.
.
connection.Close();
I'd also like to add, if you're using a few SqlDataAdapters for your queries, although you normally don't need to open the connection by yourself, if you DO explicitly call connection.Open() it then won't close the connection for you automatically, allowing you to execute multiple queries with only one connection.
If you are using C# to open a connection. use using statement will help you clean up the resource/connection even if there is some excepion throwing out.
using (SqlConnection connection =
new SqlConnection(connectionString)
{
connection.Open();
//issue command
}
And read this:
http://msdn.microsoft.com/en-us/library/8xx3tyca(VS.71).aspx, you can "Controlling Connection Pooling with Connection String Keywords", and the system will handle pooling for you.