In my server application I want to use DB (SQL Server) but I am quite unsure of the best method. There are clients whose requests comes to threadpool and so their processing is async. Every request usually needs to read or write to DB, so I was thinking about static method which would create connection, execute the query and return the result. I'm only afraid whether opening and closing connection is not too slow and whether some connection limit could not be reached? Is this good approach?
IMHO the best is to rely on the ADO.NET connection pooling mechanism and don't try to handle database connections manually. Write your data access methods like this:
public void SomeMethod()
{
using (var connection = new SqlConnection(connectionString))
using (var command = connection.CreateCommand())
{
connection.Open();
command.CommandText = "SELECT Field1 FROM Table1";
using (var reader = command.ExecuteReader())
{
while(reader.Read())
{
// do something with the results
}
}
}
}
Then you can call this method from wherever you like, make it static, call it from threads whatever. Remember that calling Dispose on the connection won't actually close it. It will return it to the connection pool so that it can be reused.
Surprised that no one mentioned connection pooling. If you think you are going to have a large number of requests, why not just setup a pool with a min pool size set to say 25 (arbitrary number here, do not shoot) and max pool size set to say 200.
This will decrease the number of connection attempts and make sure that if you are not leaking connection handles (something that you should take explicit care to not let happen), you will always have a connection waiting for you.
Reference article on connection pooling: http://msdn.microsoft.com/en-us/library/8xx3tyca.aspx
Another side note, why the need to have the connection string in the code? Set it in the web.config or app.config for the sake of maintainability. I had to "fix" code that did such things and I always swore copiously at the programmer responsible for such things.
I have had exactly the same problem like you. Had huge app that i started making multithreaded. Benefit over having one connection open and being reused is that you can ask DB multiple times for data as new connection is spawned on request (no need to wait for other threads to finish getting data), and if for example you loose connection to sql (and it can happen when network goes down for a second or two) you will have to always check if connection is open before submitting query anyway.
This is my code for getting Database rows in MS SQL but other stuff should be done exactly the same way. Keep in mind that the sqlConnectOneTime(string varSqlConnectionDetails) has a flaw of returning null when there's no connection so it needs some modifications for your needs or the query will fail if sql fails to establish connection. You just need to add proper code handling there :-) Hope it will be useful for you :-)
public const string sqlDataConnectionDetails = "Data Source=SQLSERVER\\SQLEXPRESS;Initial Cata....";
public static string sqlGetDatabaseRows(string varDefinedConnection) {
string varRows = "";
const string preparedCommand = #"
SELECT SUM(row_count) AS 'Rows'
FROM sys.dm_db_partition_stats
WHERE index_id IN (0,1)
AND OBJECTPROPERTY([object_id], 'IsMsShipped') = 0;";
using (var varConnection = Locale.sqlConnectOneTime(varDefinedConnection))
using (var sqlQuery = new SqlCommand(preparedCommand, varConnection))
using (var sqlQueryResult = sqlQuery.ExecuteReader())
while (sqlQueryResult.Read()) {
varRows = sqlQueryResult["Rows"].ToString();
}
return varRows;
}
public static SqlConnection sqlConnectOneTime(string varSqlConnectionDetails) {
SqlConnection sqlConnection = new SqlConnection(varSqlConnectionDetails);
try {
sqlConnection.Open();
} catch (Exception e) {
MessageBox.Show("Błąd połączenia z serwerem SQL." + Environment.NewLine + Environment.NewLine + "Błąd: " + Environment.NewLine + e, "Błąd połączenia");
}
if (sqlConnection.State == ConnectionState.Open) {
return sqlConnection;
}
return null;
}
Summary:
Defined one global variable with ConnectionDetails of your SQL Server
One global method to make connection (you need to handle the null in there)
Usage of using to dispose connection, sql query and everything when the method of reading/writing/updating is done.
The one thing that you haven't told us, that would be useful for giving you an answer that's appropriate for you is what level of load you're expecting your server application to be under.
For pretty much any answer to the above question though, the answer would be that you shouldn't worry about it. ADO.net/Sql Server provides connection pooling which removes some of the overhead of creating connections from each "var c = new SqlConnection(connectionString)" call.
Related
I would like to know what could be best approach to open a SqlConnection with Sql Server 2008R2 Express Edition Database. This Version of Sql has Limitations of RAM Usage and CPU Usage so we must adopt something best to open a SqlConnection.
Right Now i am Checking Connection on Start and End of each and every Method. Here is an example of that.
private void CheckValidId(string Id)
{
CheckConnectionStatus();
try
{
sqlConnection.Open();
sqlCommand = new SqlCommand("select * from ValidId where id=#id", sqlConnection);
sqlCommand.Parameters.AddWithValue("#id", Id);
sqlDataReader = sqlCommand.ExecuteReader();
While (sqlDataReader.Read())
{
string Test = sqlDataReader["Id"].toString();
MessageBox.Show("Value of Id : " , Test);
}
}
catch (Exception exp)
{
MessageBox.Show(exp.Message.ToString(), "Exception in CheckValidId");
}
finally
{
CheckConnectionStatus();
}
}
Here is CheckConnectionStatus Method
private void CheckConnectionStatus()
{
if (sqlConnection.State == ConnectionState.Open)
{
sqlConnection.Close();
}
}
What is best approach to perform this operation.
Thanks
Just use using as it disposes of the connection once done.
using(SqlConnection conn = new SqlConnection("Connection string")){
//do sql stuff
conn.Open();
//etc etc
conn.Close();
}
You'll want to make use of the disposable pattern to ensure everything is closed and disposed properly:
var query = "select * from ValidId where id=#id";
using (var conn = new System.Data.SqlClient.SqlConnection(usingConnectionString))
using (var command = new System.Data.SqlClient.SqlCommand(query, conn))
{
command.Parameters.Add("#id", SqlDbType.Int).Value = Id;
conn.Open;
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
string Test = reader["Id"].ToString();
}
}
command.Parameters.Clear();
}
You don't need to check the connection state; it will close when it's being disposed.
One thing to note: it's best practice to explicitly specify your parameter data types. I've assumed SqlDbType.Int in your case, but you can change it to whatever it really is.
Another thing to note: you don't want to do too much inside the reader while loop. You want to build your collection or whatever and get out of there. The shorter your connection is open, the better. That's because you could potentially be holding a read lock on some of the rows in the database that might affect other users and their applications.
Your pattern for open and close is correct. However you must note that this doesn't open and close the connection to the SQL Server so doesn't really address your concerns over memory usage and CPU - in fact it wont make any difference.
What Open and Close does is lease and return a connection to the ADO Connection Pool on the client PC. This means that Closing an ADO connection is not guaranteed (and in most cases will not) close and release the connection to the SQL Server. This is becasue establishing and authenticating a connection is relatively expensive and slow, so the ADO connection pool keeps your connections in a pool, still open, just in case you want to re-establish a connection.
What makes the difference to SQL Server is the number of concurrent queries it needs to execute - and the dataset size of the queries, and the total size of the data in the database.
Concurrent queries squeeze CPU, and the datasets returned squeeze the RAM available. Obviously the bigger your database the less can be cached in RAM and so the less likely you are to get a cache hit when querying.
In practice my experience with SQL Express editions is that you wont notice any difference between it and the full edition of SQL Server unless you are doing some very specific things;
1) Writing a BI style tool which allows the user to construct user-defined or user-scoped queries.
2) Writing terrible SQL - "big SQL" may mask your bad query syntax, but Express wont be able to because it has less available RAM to play with.
If you write efficient, constrained SQL, you probably wont actually ever hit any of SQL Express's limitations.
I have a program that access database and excecute different methods that have a database call.
I have used one conenction for everything but it caused a timeout while executing a long task:
I basically had to go through the more than 6000 records and execute a stored procedure. I thing that caused a timeout since I used only one database connection for everything.
Then I changed the code, so I open and closing the connection for every method I call with "using" approach.
How should I handle the method that will be called a lot. Shouls I open/close connection everytime I access that method?
Or there is a different approach to it?
I do something like this:
foreach(record in MyCollection)//6000
{
using(connection = new SqlConnection(conString))
{
singledata = GetSingleData(record);
}
}
Here is a GetSingleData()
private byte[] GetSingleData(MyObject Data)
{
byte[] singleData = null;
using(SqlCommans......)
{
try
{
.......
//executing stored proc to get just a single row
reader = command.ExecuteReader();
while(reader.Read())
{
singleData = (byte[])reader["ColumnName"];
}
}
catch(SqlException ex)
{
if(!reader.isClosed)
reader.Close();
}
}
return singleData;
}
Is it efficient or I can set up some kind of counter and for each 500 records I can check if connection is closed and if it is then reopen it.
Thank's
Try using a persistent connection. Here's a post that might help if you want to try to tune your system (for MySQL):
http://www.mysqlperformanceblog.com/2011/04/19/mysql-connection-timeouts/
Hope that helps.
There is no such a thing as the only good way to do something. It all depends. In cases where agility is a must and you need to create ad-hoc solutions, opening and closing a connection in each method call might not be good theoretically, but accepted practically.
I urge you to read about these terms and concepts:
Connection pooling
Bulk operations (bulk update, bulk insert)
They might help you in getting more performance.
What is the best practice for opening and re-using a database connection in an ASP.NET web service (C# 4.0) that has about 5 web methods that use the same database? The answer ideally will be the same for MS Access and SQL server.
Global private variable:
a. Advantage: Easy to maintain if change database providers
b. Disadvantage: Cannot use the ‘using’ clause and connection may stay open too long – how long will connection stay open approximately?
Pass the db connection as a method parameter:
a. Advantage: can close connection when not needed use ‘using’ clause
b. Disadvantage: annoying to maintain and pass param everywhere
Open and close the connection with using clause on demand when needed in any method:
a. Question: Is this the slowest because you have to re-open the same connection multiple times? OR because of behind-the-scenes connection pooling this is not slower? Thus this would probably be the best practice…?
b. Question: Is connection pooling only relevant for SQL server, thus would not work for MS Access?
At first I was thinking of using a global private variable (OleDbConnection) as I use the connection to the main database a fair bit in the web methods and supporting methods and will be migrating from MS Access to SQL server in the near future and will be easy to make the change.
But after reading up on some articles like Obtaining SQL connection most efficiently when using ASP.NET and web services
it seems like I can just create a new connection on demand anywhere with the ‘using’ clause and not suffer a performance hit?
ASP.Net pools database connections so you tend not to have to worry about these low-level details. Take a look at Tip 3 of this MSDN article.
Best practice is to not re-use your Connection objects at all.
public void GetEmployees() As List<Employee> {
var employees = new List<Employee>();
using (var connection = new SqlConnection(Configuration.ConnectionString)) {
using (var command = connection.CreateCommand()) {
command.CommandText = "SELECT * FROM dbo.Employee";
connection.Open();
using (var reader = command.ExecuteReader()) {
while (reader.Read()) {
employees.Add(Employee.CreateRecordFromOpenReader(reader));
}
}
}
}
return employees;
}
Then if you need transactions from whatever uses this, setup DTC and do this.
using (var scope = new TransactionScope()) {
var employees = GetEmployees();
employees.Map((e) => e.Status = Status.Active);
scope.Complete();
}
I'm making a web application in C# with ASP.Net, and I'm using an SQL database for it. In the guides I saw (about SqlConnection and SqlCommand) they open and close the SQL connection each time they want to send a query. But I'm not sure this is the best way to handle the connection, because I also learned some PHP, and as far as I know in PHP you open a connection only once, at start. So what is the best way to handle the connection?
You should generally have one connection, and transaction, for the length of the web request.
If you have two connections, you could potentially have inconsistent data. e.g. in the first query you check the bank balance, then you close it. You then add $5 to the balance and save that away in the next connection. What if someone else added $5 between the two connections? It would go missing.
The easiest thing to do is to open the connection global.asax BeginRequest and save that away into the HttpContext. Whenever you need a connection, pull it from there. Make sure that you .Close the connection in your EndRequest
Also, read up here on connection pooling: http://msdn.microsoft.com/en-us/library/8xx3tyca.aspx
It is not 'expensive' to repeatedly open and close connections, provided the username is the same each time.
Ado.Net has connection pooling already managed for you, so you don't need to worry about reusing connections.
You could use the Using statement to close and dispose connection:
DataTable dt = new DataTable();
using (SqlConnection conn = new SqlConnection("my_connection_string"))
{
using (SqlDataAdapter adapter = new SqlDataAdapter("SELECT * from Table", conn))
{
conn.open();
adapter.Fill(dt);
}
}
Normal practice in .NET is to open a SqlConnection and use a SqlCommand, then dispose of both of them.
using(SqlConnection connection = new SqlConnection("myConnectionString"))
{
using(SqlCommand command = new SqlCommand("dbo.SomeProc"))
{
// Execute the command, when the code leaves the using block, each is disposed
}
}
As a connection to database is an expensive resource it's better to open sql connection as late as possible and close early. So you should open the connection just before executing a command and close it once you have executed it.
On the other hand if you are going to execute many commands in a short interval of time it's better to open the connection once and close after all the commands are executed.
For starters they do not as there is a connection pool that is being used, but SqlCommand has an overload to take a connection object, so you could keep the connection around and pass it.
Generally:
using (var ts = new TransactionScope())
{
using (var connection = new SqlConnection(...))
{
using (var command = new SqlCommand(connection, ...)
{
...
}
using (var command = new SqlCommand(connection, ...)
{
...
}
}
ts.Complete();
}
Of course I would recommend using LINQ-to-SQL or EF at this point.
Try Data Access Application Block in the Enterprise Library if you do not want to worry about opening and closing the connections.
I'm getting this error (Distributed transaction completed. Either enlist this session in a new transaction or the NULL transaction.) when trying to run a stored procedure from C# on a SQL Server 2005 database. I'm not actively/purposefully using transactions or anything, which is what makes this error weird. I can run the stored procedure from management studio and it works fine. Other stored procedures also work from C#, it just seems to be this one with issues. The error returns instantly, so it can't be a timeout issue. The code is along the lines of:
SqlCommand cmd = null;
try
{
// Make sure we are connected to the database
if (_DBManager.CheckConnection())
{
cmd = new SqlCommand();
lock (_DBManager.SqlConnection)
{
cmd.CommandText = "storedproc";
cmd.CommandType = System.Data.CommandType.StoredProcedure;
cmd.Connection = _DBManager.SqlConnection;
cmd.Parameters.AddWithValue("#param", value);
int affRows = cmd.ExecuteNonQuery();
...
}
}
else
{
...
}
}
catch (Exception ex)
{
...
}
It's really got me stumped. Thanks for any help
It sounds like there is a TransactionScope somewhere that is unhappy. The _DBManager.CheckConnection and _DBManager.SqlConnection sounds like you are keeping a SqlConnection hanging around, which I expect will contribute to this.
To be honest, in most common cases you are better off just using the inbuilt connection pooling, and using your connections locally - i.e.
using(var conn = new SqlConnection(...)) { // or a factory method
// use it here only
}
Here you get a clean SqlConnection, which will be mapped to an unmanaged connection via the pool, i.e. it doesn't create an actual connection each time (but will do a logical reset to clean it up).
This also allows much more flexible use from multiple threads. Using a static connection in a web app, for example, would be horrendous for blocking.
From the code it seems that you are utilizing an already opened connection. May be there's a transaction pending previously on the same connection.