We Keep Coding

Queries from background worker conflict?

Is there a problem if I execute queries from multiple threads using the same ConnectionString? What happens if two or more threads try to send data at the same time?
string globalConnectionString = #"some_stringHere!";
//create new backgroundWorker if new logFile is created (txt file).
// ....
private void backgroundWorker_DoWork(object sender, DoWorkEventArgs e)
{
// get some data from created logFile
string serialNumber = getSerialNumber(logFile);
string testResult = getTestResult(logFile);
// if server is online, send data
if(serverIsOnline)
{
using(SqlConnection connection = new SqlConnecton(globalConnectionString))
{
SqlCommand someCommand = new SqlCommand("some insert/update command here!", connection);
connection.Open();
Command.ExecuteNonQuery();
connection.Close();
}
}
}

Concurrent connections are OK, if used correctly
There's no problem with using multiple connections concurrently, assuming it's done for the right reason. Databases can handle thousands of concurrent client connections.
Executing the same slow query in parallel to make it finish faster will probably make it even slower as each connection may block the others. Many databases parallelize query processing already, producing far better results than crude client-side parallelism.
If you want to make a slow query go faster, you'd get better results by investigating why it's slow and fixing the perf issues. For example, if you want to insert 10K rows, it's faster to use eg SqlBulkCopy or BULK INSERT to load the rows than executing 10K INSERTs that will end up blocking each other for access to the same table and even data pages
You can use the same connection to execute asynchronous queries (eg with ExecuteNonQueryAsync(), ExecuteReaderAsync() etc, provided they execute one after the other. You can't execute multiple concurrent queries on the same connection, at least not without going through some hoops.
The real problem
The real problem is using a BackgroundWorker in the first place. That class is obsolete since 2012 when async/await were introduced. With BGW it's extremely hard to combine multiple asynchronous operations. Progress reporting is available through the Progress<T> class and cooperative cancellation through CancellationTokenSource. Check Async in 4.5: Enabling Progress and Cancellation in Async APIs for a detailed explanation.
You can replace the BGW calls in your code with only await command.ExecuteNonQueryAsync(). You could create an asynchronous method to perform insert the data into the database :
private async Task InsertTestData(string serialNumber,string testResult)
{
// if server is online, send data
if(serverIsOnline)
{
using(SqlConnection connection = new SqlConnecton(globalConnectionString))
{
var someCommand = new SqlCommand("some insert/update command here!", connection);
someCommand.Parameters.Add("#serial",SqlDbType.NVarChar,30).Value=serialNumber;
...
connection.Open();
Command.ExecuteNonQueryAsync();
}
}
}
If retrieving the serial number and test data is time consuming, you can use Task.Run to run each of them in the background :
string serialNumber = await Task.Run(()=>getSerialNumber(logFile));
string testResult = await Task.Run(()=>getTestResult(logFile));
await InsertTestData(serialNumber,testResult);
You could also use a library like Dapper to simplify the database :
private async Task InsertTestData(string serialNumber,string testResult)
{
// if server is online, send data
if(serverIsOnline)
{
using(SqlConnection connection = new SqlConnecton(globalConnectionString))
{
await connection.ExecuteAsync("INSERT .... VALUES(#serial,#test)",
new {serial=serialNumber,test=testResults});
}
}
}
Dapper will generate a parameterized query and match the parameters in the query with properties in the anonymous object by name.

Reading the connection string isn't an issue here. You would have a problem if you would share the SqlConnection object through multiple threads. But that's not the case in your code.

I believe this is a question about Isolation from ACID properties. Please have a look at them.
Based on the SQL standard a single SQL query operates on a steady (consistent) state of the table(s) which the query works on. So this definition dictates that, it can NOT see any changes while it's being executed. However, as far as I know not all DBMS software follow this rule perfectly. For example there are products and / or Isolation levels that allow dirty reads.
Here is very detailed explanation from another user.

How do I rollback a SQL CLR stored procedure when a user cancels the query?

I'm trying to create a SQL CLR stored procedure that will create a table, pass the table name onto a service which will bulk insert some data into it, display the results of the table, then clean up the table.
What I've tried so far:
Use SqlTransaction. Cancelling the transaction works, but it puts my query window into a state where I couldn't continue working on it.
The transaction active in this session has been committed or aborted by another session
Use TransactionScope. Same issue as 1.
Manually clean up the table in a finally clause by issuing a DROP TABLE SqlCommand. This doesn't seem to get run, though my SqlContext.Pipe.Send() prior to issuing the command does. It doesn't seem like it's related to any time constraints since if I issue a Thread.Sleep(2000) before printing another line, it still prints the second line whereas the command.ExecuteNonQuery() would stop before printing the second line.
Placing the manual cleanup code into a CER or SafeHandle. This doesn't work as having a CER requires some guarantees, including not allocating additional memory or calling methods that are not decorated with a ReliabilityContract.
Am I missing something obvious here? How do I handle the user cancelling their query?
Edit: There were multiple iterations of the code for each scenario, but the general actions taken are along the lines of the following:
[SqlProcedure]
public static void GetData(SqlString code)
{
Guid guid = Guid.NewGuid();
using (var connection = new SqlConnection("context connection=true"))
{
connection.Open();
try
{
SqlContext.Pipe?.Send("Constrain");
SqlCommand command1 = new SqlCommand($"CREATE TABLE qb.{code}_{guid:N} (Id INT)", connection);
command1.ExecuteNonQuery();
SqlContext.Pipe?.Send($"Create: qb.{code}_{guid:N}");
//emulate service call
Thread.Sleep(TimeSpan.FromSeconds(10));
SqlContext.Pipe?.Send($"Done: qb.{code}_{guid:N}");
}
finally
{
SqlContext.Pipe?.Send("1");
//drop table here instead of sleep
Thread.Sleep(2000);
SqlContext.Pipe?.Send("2");
}
}
}

Unfortunately SQLCLR does not handle query cancellation very well. However, given the error message, that seems to imply that the cancellation does its own ROLLBACK. Have you tried not using a Transaction within the SQLCLR code but instead handling it from outside? Such as:
BEGIN TRAN;
EXEC SQLCLR_Stored_Procedure;
IF (##TRANCOUNT > 0) ROLLBACK TRAN;
The workflow noted above would need to be enforced. This can be done rather easily by creating a wrapper T-SQL Stored Procedure that executes those 3 steps and only give EXECUTE permission to the wrapper Stored Procedure. If permissions are then needed for the SQLCLR Stored Procedure, that can be accomplished rather easily using module signing:
Create an Asymmetric Key in the same DB as the SQLCLR Stored Procedure
Create a User from that Asymmetric Key
GRANT that Key-based User EXECUTE permission on the SQLCLR Stored Procedure
Sign the wrapper T-SQL Stored Procedure, using ADD SIGNATURE, with that Asymmetric Key
Those 4 steps allow the T-SQL wrapper proc to execute the SQLCLR proc, while the actual application Login can only execute the T-SQL wrapper proc :-). And, in, the event that the cancellation aborts the execution prior to executing ROLLBACK, the Transaction should be automatically rolled-back when the connection closes.
Also, do you have XACT_ABORT set to ON or OFF? UPDATE: O.P. states that it is set to OFF, and setting to ON did not seem to behave any differently.
Have you tried checking the connection state in the finally block? I am pretty sure that the SqlConnection is Closed upon the cancellation. You could try the following approaches, both in the finally block:
Test for the connection state and if Closed, re-open the SqlConnection and then execute the non-query command.
UPDATE: O.P. states that the connection is still open. Ok, how about closing it and re-opening it?
UPDATE 2: O.P. tested and found that the connection could not be re-opened.
Since the context is still available, as proven by your print commands working, use something like SqlContext.Pipe.ExecuteAndSend(new SqlCommand("DROP TABLE..;"));
UPDATE: O.P. states that this did not work.
OR, since you create a guaranteed unique table name in the code, you can try creating the table as a global temporary table (i.e. prefixed with two pound-signs: ##TableName) which will a) be available to the bulk import process, and b) clean itself up when the connection fully closes. In this approach, you technically wouldn't need to perform any manual clean up.
Of course, when Connection Pooling is enabled, the automatic cleanup happens only after the connection is re-opened and the first command is executed. In order to force an immediate cleanup, you would have to connect to SQL Server with Connection Pooling disabled. Is it possible to use a different Connection String just when this Stored Procedure is to be executed that includes Pooling=false;? Given how this Stored Procedure is being used, it does not seem like you would suffer any noticeable performance degradation from disabling Connection Pooling on just this one specific call. To better understand how Connection Pooling – enabled or disabled – affects the automatic cleanup of temporary objects, please see the blog post I just published that details this very behavior:
Sessions, Temporary Objects, and the Afterlife
This approach is probably the best overall since you probably cannot guarantee either that ROLLBACK would be executed (first approach mentioned) or that a finally clause would be executed (assuming you ever got that to work). In the end, uncommitted Transactions will be rolled-back, but if someone executes this via SSMS and it aborts without the ROLLBACK, then they are still in an open Transaction and might not be aware of it. Also, think about the connection being forcibly closed, or the Session being killed, or the server being shutdown / restarted. In those cases tempdb is your friend, whether by using a global temporary table, or at the very least creating the permanent Table in tempdb so that it is automatically removed the next time that the SQL Server starts (due to tempdb being created new, as a copy of model, upon each start of the SQL Server service).

Stepping back there are probably much better ways to pass the data out of your CLR procedure.
1) you can simply use the SqlContext Pipe to return a resultset without creating a table.
2) you can create a temp table (#) in the calling code and access it from inside the CLR procedure. You might want to introduce a TSQL wrapper procedure to make this convenient.
Anyway using BEGIN TRAN/COMMIT | ROLLBACK worked for me:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
using System.Threading;
static class SqlConnectionExtensions
{
public static DataTable ExecuteDataTable(this SqlConnection con, string sql, params SqlParameter[] parameters)
{
var cmd = new SqlCommand(sql, con);
foreach (var p in parameters)
{
cmd.Parameters.Add(p);
}
using (var dr = cmd.ExecuteReader())
{
var dt = new DataTable();
dt.Load(dr);
return dt;
}
}
public static int ExecuteNonQuery(this SqlConnection con, string sql, params SqlParameter[] parameters)
{
var cmd = new SqlCommand(sql, con);
foreach (var p in parameters)
{
cmd.Parameters.Add(p);
}
return cmd.ExecuteNonQuery();
}
}
public partial class StoredProcedures
{
[Microsoft.SqlServer.Server.SqlProcedure]
public static void GetData(SqlString code)
{
Guid guid = Guid.NewGuid();
using (var connection = new SqlConnection("context connection=true"))
{
connection.Open();
try
{
connection.ExecuteNonQuery("begin transaction;");
SqlContext.Pipe?.Send("Constrain");
connection.ExecuteNonQuery($"CREATE TABLE qb.{code}_{guid:N} (Id INT)");
SqlContext.Pipe?.Send($"Create: qb.{code}_{guid:N}");
//emulate service call
Thread.Sleep(TimeSpan.FromSeconds(10));
SqlContext.Pipe?.Send($"Done: qb.{code}_{guid:N}");
connection.ExecuteNonQuery("commit transaction");
}
catch (Exception ex)
{
connection.ExecuteNonQuery("rollback;");
throw;
}
}
}
}

While it is not something advised for use, this would be an ideal match for sp_bindsession. With sp_bindsession, you would call sp_getbindtoken from the context session, pass the token to the server and call sp_bindsession from the service connection.
Afterwards, the two connections behave "as one", with temporaries and transactions being transparently propagated.

What is Best Approach for Opening/Closing SqlConnection in C#

I would like to know what could be best approach to open a SqlConnection with Sql Server 2008R2 Express Edition Database. This Version of Sql has Limitations of RAM Usage and CPU Usage so we must adopt something best to open a SqlConnection.
Right Now i am Checking Connection on Start and End of each and every Method. Here is an example of that.
private void CheckValidId(string Id)
{
CheckConnectionStatus();
try
{
sqlConnection.Open();
sqlCommand = new SqlCommand("select * from ValidId where id=#id", sqlConnection);
sqlCommand.Parameters.AddWithValue("#id", Id);
sqlDataReader = sqlCommand.ExecuteReader();
While (sqlDataReader.Read())
{
string Test = sqlDataReader["Id"].toString();
MessageBox.Show("Value of Id : " , Test);
}
}
catch (Exception exp)
{
MessageBox.Show(exp.Message.ToString(), "Exception in CheckValidId");
}
finally
{
CheckConnectionStatus();
}
}
Here is CheckConnectionStatus Method
private void CheckConnectionStatus()
{
if (sqlConnection.State == ConnectionState.Open)
{
sqlConnection.Close();
}
}
What is best approach to perform this operation.
Thanks

Just use using as it disposes of the connection once done.
using(SqlConnection conn = new SqlConnection("Connection string")){
//do sql stuff
conn.Open();
//etc etc
conn.Close();
}

You'll want to make use of the disposable pattern to ensure everything is closed and disposed properly:
var query = "select * from ValidId where id=#id";
using (var conn = new System.Data.SqlClient.SqlConnection(usingConnectionString))
using (var command = new System.Data.SqlClient.SqlCommand(query, conn))
{
command.Parameters.Add("#id", SqlDbType.Int).Value = Id;
conn.Open;
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
string Test = reader["Id"].ToString();
}
}
command.Parameters.Clear();
}
You don't need to check the connection state; it will close when it's being disposed.
One thing to note: it's best practice to explicitly specify your parameter data types. I've assumed SqlDbType.Int in your case, but you can change it to whatever it really is.
Another thing to note: you don't want to do too much inside the reader while loop. You want to build your collection or whatever and get out of there. The shorter your connection is open, the better. That's because you could potentially be holding a read lock on some of the rows in the database that might affect other users and their applications.

Your pattern for open and close is correct. However you must note that this doesn't open and close the connection to the SQL Server so doesn't really address your concerns over memory usage and CPU - in fact it wont make any difference.
What Open and Close does is lease and return a connection to the ADO Connection Pool on the client PC. This means that Closing an ADO connection is not guaranteed (and in most cases will not) close and release the connection to the SQL Server. This is becasue establishing and authenticating a connection is relatively expensive and slow, so the ADO connection pool keeps your connections in a pool, still open, just in case you want to re-establish a connection.
What makes the difference to SQL Server is the number of concurrent queries it needs to execute - and the dataset size of the queries, and the total size of the data in the database.
Concurrent queries squeeze CPU, and the datasets returned squeeze the RAM available. Obviously the bigger your database the less can be cached in RAM and so the less likely you are to get a cache hit when querying.
In practice my experience with SQL Express editions is that you wont notice any difference between it and the full edition of SQL Server unless you are doing some very specific things;
1) Writing a BI style tool which allows the user to construct user-defined or user-scoped queries.
2) Writing terrible SQL - "big SQL" may mask your bad query syntax, but Express wont be able to because it has less available RAM to play with.
If you write efficient, constrained SQL, you probably wont actually ever hit any of SQL Express's limitations.

"file in use by another process" error while trying to delete a database file

I have a PC.sdf file that I work with. I close the connection and I need to delete it.
I open the connection like this:
bool OpenConn()
{
try
{
Conn = new SqlCeConnection(String.Format(#"Data Source={0}\{1}", PathI, "PC.SDF"));
Conn.Open();
return true;
}
catch
{
//MessageBox.Show(err.Message, "Connetion error");
return false;
}
}
I close it like this:
Conn.Close();
Conn.Dispose();
I try to delete it like this:
if (File.Exists(#"\myPath\PC.sdf"))
File.Delete(#"\myPath\PC.sdf");
But I get this error: file in use by another process. What could be the cause of this error, and how can I fix it?

You could try and force garbage collection by running
GC.Collect();
Do this after you have Closed and Disposed of the DB object.
This will of course only work if that is the only reference to that database file.
Edit: Answer to comment about that you shouldn't use GC.Collect to "fix" other issues.
I don't think this is because of another issue. The garbage collector runs at an indeterminate time chosen by the runtime. This means that you can't rely on your object being disposed of between the followings lines.
Conn.Dispose();
//Nothing says GC will run exactly now
File.Delete(#"C:\Some-file-used-by-Conn.db");
Your options are to force garbage collection or to delay the deletion in some way.

I would try it like this below. This puts your connection inside a using block which will call dispose the connection for you once it goes out of scope. This should allow you to delete the file immediately after as all of the file locks associated with it should be released.
bool OpenConn()
{
try
{
using(Conn = new SqlCeConnection(String.Format(#"Data Source={0}\{1}", PathI, "PC.SDF")))
{
Conn.Open();
return true;
}
}
catch
{
//MessageBox.Show(err.Message, "Connetion error");
return false;
}
}
if (File.Exists(#"\myPath\PC.sdf"))
File.Delete(#"\myPath\PC.sdf");

The two responses above are both good considerations for ensuring that it is not YOUR application which is the one using the database file.
I'm not exactly sure what an .SDF file is, but a quick google search suggests that it may be a MS SQL database file. If so, depending on your system and your operating conditions, you may want to consider using the ALTER DATABASE SET SINGLE_USER command:
ALTER DATABASE [YourDbName]
SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
http://blog.sqlauthority.com/2010/02/11/sql-server-alter-database-dbname-set-single_user-with-rollback-immediate/

Access to SQL DB in multithread server app

In my server application I want to use DB (SQL Server) but I am quite unsure of the best method. There are clients whose requests comes to threadpool and so their processing is async. Every request usually needs to read or write to DB, so I was thinking about static method which would create connection, execute the query and return the result. I'm only afraid whether opening and closing connection is not too slow and whether some connection limit could not be reached? Is this good approach?

IMHO the best is to rely on the ADO.NET connection pooling mechanism and don't try to handle database connections manually. Write your data access methods like this:
public void SomeMethod()
{
using (var connection = new SqlConnection(connectionString))
using (var command = connection.CreateCommand())
{
connection.Open();
command.CommandText = "SELECT Field1 FROM Table1";
using (var reader = command.ExecuteReader())
{
while(reader.Read())
{
// do something with the results
}
}
}
}
Then you can call this method from wherever you like, make it static, call it from threads whatever. Remember that calling Dispose on the connection won't actually close it. It will return it to the connection pool so that it can be reused.

Surprised that no one mentioned connection pooling. If you think you are going to have a large number of requests, why not just setup a pool with a min pool size set to say 25 (arbitrary number here, do not shoot) and max pool size set to say 200.
This will decrease the number of connection attempts and make sure that if you are not leaking connection handles (something that you should take explicit care to not let happen), you will always have a connection waiting for you.
Reference article on connection pooling: http://msdn.microsoft.com/en-us/library/8xx3tyca.aspx
Another side note, why the need to have the connection string in the code? Set it in the web.config or app.config for the sake of maintainability. I had to "fix" code that did such things and I always swore copiously at the programmer responsible for such things.

I have had exactly the same problem like you. Had huge app that i started making multithreaded. Benefit over having one connection open and being reused is that you can ask DB multiple times for data as new connection is spawned on request (no need to wait for other threads to finish getting data), and if for example you loose connection to sql (and it can happen when network goes down for a second or two) you will have to always check if connection is open before submitting query anyway.
This is my code for getting Database rows in MS SQL but other stuff should be done exactly the same way. Keep in mind that the sqlConnectOneTime(string varSqlConnectionDetails) has a flaw of returning null when there's no connection so it needs some modifications for your needs or the query will fail if sql fails to establish connection. You just need to add proper code handling there :-) Hope it will be useful for you :-)
public const string sqlDataConnectionDetails = "Data Source=SQLSERVER\\SQLEXPRESS;Initial Cata....";
public static string sqlGetDatabaseRows(string varDefinedConnection) {
string varRows = "";
const string preparedCommand = #"
SELECT SUM(row_count) AS 'Rows'
FROM sys.dm_db_partition_stats
WHERE index_id IN (0,1)
AND OBJECTPROPERTY([object_id], 'IsMsShipped') = 0;";
using (var varConnection = Locale.sqlConnectOneTime(varDefinedConnection))
using (var sqlQuery = new SqlCommand(preparedCommand, varConnection))
using (var sqlQueryResult = sqlQuery.ExecuteReader())
while (sqlQueryResult.Read()) {
varRows = sqlQueryResult["Rows"].ToString();
}
return varRows;
}
public static SqlConnection sqlConnectOneTime(string varSqlConnectionDetails) {
SqlConnection sqlConnection = new SqlConnection(varSqlConnectionDetails);
try {
sqlConnection.Open();
} catch (Exception e) {
MessageBox.Show("Błąd połączenia z serwerem SQL." + Environment.NewLine + Environment.NewLine + "Błąd: " + Environment.NewLine + e, "Błąd połączenia");
}
if (sqlConnection.State == ConnectionState.Open) {
return sqlConnection;
}
return null;
}
Summary:
Defined one global variable with ConnectionDetails of your SQL Server
One global method to make connection (you need to handle the null in there)
Usage of using to dispose connection, sql query and everything when the method of reading/writing/updating is done.

The one thing that you haven't told us, that would be useful for giving you an answer that's appropriate for you is what level of load you're expecting your server application to be under.
For pretty much any answer to the above question though, the answer would be that you shouldn't worry about it. ADO.net/Sql Server provides connection pooling which removes some of the overhead of creating connections from each "var c = new SqlConnection(connectionString)" call.