I am trying to delete a SQLite database file from my C# application. The same application performs multiple RW queries to the DB by creating and disposing a new connection each time.
When trying to delete the DB, (on a moment when I can assure the program is not querying the DB, thus no active connections) I was facing the error:
IOException: The process cannot gain access to <filename> because is being used by another process.
I have investigated for several hours: my code, SO questions, using procmon and resmon to be sure that my process was the only one holding an active handle to the file.
After all this, I determined that the DB file is not being closed correctly whenever I create a DB connection. Explanation follows:
I have the following function to perform a query and load the results into a DataTable:
public DataTable PerformQuery(string query) {
try {
DataTable table = new DataTable();
using(SQLiteConnection connection = new SQLiteConnection(connString)) {
SQLiteCommand cmd = connection.CreateCommand();
cmd.CommandText = query;
connection.Open();
if (!query.StartsWith("SELECT")) {
cmd.ExecuteNonQuery();
} else {
SQLiteDataReader reader = cmd.ExecuteReader();
FillTable(reader, table);
}
// Despite using the using scope, close and dispose the connection manually
connection.Close();
connection.Dispose();
}
// Kill all pools and call GC
SQLiteConnection.ClearAllPools();
GC.Collect();
GC.WaitForPendingFinalizers();
return table;
} catch (Exception ex) {
// Handle error... (not relevant)
return null;
}
}
Well, using SysInternals handle.exe in an infinite loop (running every second) and debugging the program step-by-step using Visual Studio's Debugger I found out that the file handle to my DB file is not being closed despite:
Calling Close
Calling Dispose
Getting out of the using scope
Killing all pools
Executing GC and waiting for it to finish
I really need to delete my DB file from inside the code but I cannot as the file handle is never closed.
How could I solve this?
EDIT 1:
Using .NET Winforms Application with System.Data.SQLite
Already tried adding Pooling=false to the connection string.
Related
I'm trying to create a SQL CLR stored procedure that will create a table, pass the table name onto a service which will bulk insert some data into it, display the results of the table, then clean up the table.
What I've tried so far:
Use SqlTransaction. Cancelling the transaction works, but it puts my query window into a state where I couldn't continue working on it.
The transaction active in this session has been committed or aborted by another session
Use TransactionScope. Same issue as 1.
Manually clean up the table in a finally clause by issuing a DROP TABLE SqlCommand. This doesn't seem to get run, though my SqlContext.Pipe.Send() prior to issuing the command does. It doesn't seem like it's related to any time constraints since if I issue a Thread.Sleep(2000) before printing another line, it still prints the second line whereas the command.ExecuteNonQuery() would stop before printing the second line.
Placing the manual cleanup code into a CER or SafeHandle. This doesn't work as having a CER requires some guarantees, including not allocating additional memory or calling methods that are not decorated with a ReliabilityContract.
Am I missing something obvious here? How do I handle the user cancelling their query?
Edit: There were multiple iterations of the code for each scenario, but the general actions taken are along the lines of the following:
[SqlProcedure]
public static void GetData(SqlString code)
{
Guid guid = Guid.NewGuid();
using (var connection = new SqlConnection("context connection=true"))
{
connection.Open();
try
{
SqlContext.Pipe?.Send("Constrain");
SqlCommand command1 = new SqlCommand($"CREATE TABLE qb.{code}_{guid:N} (Id INT)", connection);
command1.ExecuteNonQuery();
SqlContext.Pipe?.Send($"Create: qb.{code}_{guid:N}");
//emulate service call
Thread.Sleep(TimeSpan.FromSeconds(10));
SqlContext.Pipe?.Send($"Done: qb.{code}_{guid:N}");
}
finally
{
SqlContext.Pipe?.Send("1");
//drop table here instead of sleep
Thread.Sleep(2000);
SqlContext.Pipe?.Send("2");
}
}
}
Unfortunately SQLCLR does not handle query cancellation very well. However, given the error message, that seems to imply that the cancellation does its own ROLLBACK. Have you tried not using a Transaction within the SQLCLR code but instead handling it from outside? Such as:
BEGIN TRAN;
EXEC SQLCLR_Stored_Procedure;
IF (##TRANCOUNT > 0) ROLLBACK TRAN;
The workflow noted above would need to be enforced. This can be done rather easily by creating a wrapper T-SQL Stored Procedure that executes those 3 steps and only give EXECUTE permission to the wrapper Stored Procedure. If permissions are then needed for the SQLCLR Stored Procedure, that can be accomplished rather easily using module signing:
Create an Asymmetric Key in the same DB as the SQLCLR Stored Procedure
Create a User from that Asymmetric Key
GRANT that Key-based User EXECUTE permission on the SQLCLR Stored Procedure
Sign the wrapper T-SQL Stored Procedure, using ADD SIGNATURE, with that Asymmetric Key
Those 4 steps allow the T-SQL wrapper proc to execute the SQLCLR proc, while the actual application Login can only execute the T-SQL wrapper proc :-). And, in, the event that the cancellation aborts the execution prior to executing ROLLBACK, the Transaction should be automatically rolled-back when the connection closes.
Also, do you have XACT_ABORT set to ON or OFF? UPDATE: O.P. states that it is set to OFF, and setting to ON did not seem to behave any differently.
Have you tried checking the connection state in the finally block? I am pretty sure that the SqlConnection is Closed upon the cancellation. You could try the following approaches, both in the finally block:
Test for the connection state and if Closed, re-open the SqlConnection and then execute the non-query command.
UPDATE: O.P. states that the connection is still open. Ok, how about closing it and re-opening it?
UPDATE 2: O.P. tested and found that the connection could not be re-opened.
Since the context is still available, as proven by your print commands working, use something like SqlContext.Pipe.ExecuteAndSend(new SqlCommand("DROP TABLE..;"));
UPDATE: O.P. states that this did not work.
OR, since you create a guaranteed unique table name in the code, you can try creating the table as a global temporary table (i.e. prefixed with two pound-signs: ##TableName) which will a) be available to the bulk import process, and b) clean itself up when the connection fully closes. In this approach, you technically wouldn't need to perform any manual clean up.
Of course, when Connection Pooling is enabled, the automatic cleanup happens only after the connection is re-opened and the first command is executed. In order to force an immediate cleanup, you would have to connect to SQL Server with Connection Pooling disabled. Is it possible to use a different Connection String just when this Stored Procedure is to be executed that includes Pooling=false;? Given how this Stored Procedure is being used, it does not seem like you would suffer any noticeable performance degradation from disabling Connection Pooling on just this one specific call. To better understand how Connection Pooling – enabled or disabled – affects the automatic cleanup of temporary objects, please see the blog post I just published that details this very behavior:
Sessions, Temporary Objects, and the Afterlife
This approach is probably the best overall since you probably cannot guarantee either that ROLLBACK would be executed (first approach mentioned) or that a finally clause would be executed (assuming you ever got that to work). In the end, uncommitted Transactions will be rolled-back, but if someone executes this via SSMS and it aborts without the ROLLBACK, then they are still in an open Transaction and might not be aware of it. Also, think about the connection being forcibly closed, or the Session being killed, or the server being shutdown / restarted. In those cases tempdb is your friend, whether by using a global temporary table, or at the very least creating the permanent Table in tempdb so that it is automatically removed the next time that the SQL Server starts (due to tempdb being created new, as a copy of model, upon each start of the SQL Server service).
Stepping back there are probably much better ways to pass the data out of your CLR procedure.
1) you can simply use the SqlContext Pipe to return a resultset without creating a table.
2) you can create a temp table (#) in the calling code and access it from inside the CLR procedure. You might want to introduce a TSQL wrapper procedure to make this convenient.
Anyway using BEGIN TRAN/COMMIT | ROLLBACK worked for me:
using System;
using System.Data;
using System.Data.SqlClient;
using System.Data.SqlTypes;
using Microsoft.SqlServer.Server;
using System.Threading;
static class SqlConnectionExtensions
{
public static DataTable ExecuteDataTable(this SqlConnection con, string sql, params SqlParameter[] parameters)
{
var cmd = new SqlCommand(sql, con);
foreach (var p in parameters)
{
cmd.Parameters.Add(p);
}
using (var dr = cmd.ExecuteReader())
{
var dt = new DataTable();
dt.Load(dr);
return dt;
}
}
public static int ExecuteNonQuery(this SqlConnection con, string sql, params SqlParameter[] parameters)
{
var cmd = new SqlCommand(sql, con);
foreach (var p in parameters)
{
cmd.Parameters.Add(p);
}
return cmd.ExecuteNonQuery();
}
}
public partial class StoredProcedures
{
[Microsoft.SqlServer.Server.SqlProcedure]
public static void GetData(SqlString code)
{
Guid guid = Guid.NewGuid();
using (var connection = new SqlConnection("context connection=true"))
{
connection.Open();
try
{
connection.ExecuteNonQuery("begin transaction;");
SqlContext.Pipe?.Send("Constrain");
connection.ExecuteNonQuery($"CREATE TABLE qb.{code}_{guid:N} (Id INT)");
SqlContext.Pipe?.Send($"Create: qb.{code}_{guid:N}");
//emulate service call
Thread.Sleep(TimeSpan.FromSeconds(10));
SqlContext.Pipe?.Send($"Done: qb.{code}_{guid:N}");
connection.ExecuteNonQuery("commit transaction");
}
catch (Exception ex)
{
connection.ExecuteNonQuery("rollback;");
throw;
}
}
}
}
While it is not something advised for use, this would be an ideal match for sp_bindsession. With sp_bindsession, you would call sp_getbindtoken from the context session, pass the token to the server and call sp_bindsession from the service connection.
Afterwards, the two connections behave "as one", with temporaries and transactions being transparently propagated.
I am doing windows forms application with connection to SQL Server (disconnected mode). Every action I handle in my application needs the connection. So I don't know where to close the connection.
I am asking what if i don't close the connection. I know that there is no runtime error that appears. But is there any other error that could appear?
There is a risk that if you are using the same connection for each method, from a private field or GetConnection() method etc.; you could end up trying to read/write when the connection is still open.
This could then throw errors and you could end up in a whole heap of it, if using a live SQL database.
If you can, create a new instance of a connection in a using statement each time you want to access the DB.
Like this:
var connectionString = "YourConnectionStringToTheDB";
var queryString ="YourSQLQueryHere"; // FROM * IN tbl SELECT * etc...
using (SqlConnection connection = new SqlConnection(
connectionString))
{
using (SqlCommand command = new SqlCommand(queryString, connection))
{
command.Connection.Open();
command.ExecuteNonQuery();
}
}
This way it is closed and disposed of when the code runs outside of the using statement's scope.
Notice that the SqlCommand is also in a using statement, as it too can be disposed of.
This is cleaner code and marks the objects for disposal by the Garbage Collector. (Google is your friend) - BIG topic; a lot of stuff to read.
Here is a similar question which gives you some more detail: The C# using statement, SQL, and SqlConnection
EDIT
Better still, you could wrap your SQL code in a try { } catch { } block to handle any errors at runtime.
I have used my database access code as below:
//gets the connection and create transaction
SqlTransaction transaction = SelectConnection();
try
{
dsRecordDataset = new DataSet();
SqlParameter[] param = new SqlParameter[1];
string[] taleNames = new string[1];
taleNames[0] = "RecordSet";
param[0] = new SqlParameter("#Mode", Mode);
SqlHelper.FillDataset(transaction, CommandType.StoredProcedure,
"spProject", dsRecordDataset, taleNames, param);
transaction.Commit();
return dsRecordDataset;
}
catch (Exception ex)
{
transaction.Rollback();
throw ex;
}
Problem which I have found is I have never closed the connection so that the connection pool can overflow and give an error to the user sometimes. I need to clarify the following
When does the connection pool reset according to default settings in asp.net?
Does committing transaction has anything to do with the connection?
I tried to use following code after try catch block
finally
{
if (transaction.Connection.State == ConnectionState.Open)
{
transaction.Connection.Close();
}
}
but then the connection is null and gives a error.
Is there any way to close the connection in above code?
when does the connection pool reset according to default settings in asp.net?
If MinPoolSize is either not specified in the connection string or is specified as zero, the connections in the pool will be closed after a period of inactivity. However, if the specified MinPoolSize is greater than zero, the connection pool is not destroyed until the AppDomain is unloaded and the process ends.
does committing transaction has anything to do with the connection?
No, committing a connection does not (should not) close the connection
is there any way to close the connection in above code?
You can do this using either the Close or Dispose methods of the Connection object, or by opening all connections inside a using statement in C# so that they will be returned to the connection pool. Connections that are not explicitly closed might not be added or returned to the pool.
You got the answers, but for the question 3) you might find the using statement intresting in this case. However it means a code change, but it is a good way to keep the resource critical objects safe. (like your SqlConnection)
using (SqlConnection connection = new SqlConnection(connectionString))
{
SqlCommand command = connection.CreateCommand();
command.CommandText = "mysp_GetValue";
command.CommandType = CommandType.StoredProcedure;
connection.Open();
object ret = command.ExecuteScalar();
}
It is just a very simple example, but this statement automaticly calls the Dispose method of an object which implements the IDisposable interface. (Like the SqlConnection)
If you are intrested in this, here you can find more details and explanations:
Your Friend the C# Using Statement
I have a PC.sdf file that I work with. I close the connection and I need to delete it.
I open the connection like this:
bool OpenConn()
{
try
{
Conn = new SqlCeConnection(String.Format(#"Data Source={0}\{1}", PathI, "PC.SDF"));
Conn.Open();
return true;
}
catch
{
//MessageBox.Show(err.Message, "Connetion error");
return false;
}
}
I close it like this:
Conn.Close();
Conn.Dispose();
I try to delete it like this:
if (File.Exists(#"\myPath\PC.sdf"))
File.Delete(#"\myPath\PC.sdf");
But I get this error: file in use by another process. What could be the cause of this error, and how can I fix it?
You could try and force garbage collection by running
GC.Collect();
Do this after you have Closed and Disposed of the DB object.
This will of course only work if that is the only reference to that database file.
Edit: Answer to comment about that you shouldn't use GC.Collect to "fix" other issues.
I don't think this is because of another issue. The garbage collector runs at an indeterminate time chosen by the runtime. This means that you can't rely on your object being disposed of between the followings lines.
Conn.Dispose();
//Nothing says GC will run exactly now
File.Delete(#"C:\Some-file-used-by-Conn.db");
Your options are to force garbage collection or to delay the deletion in some way.
I would try it like this below. This puts your connection inside a using block which will call dispose the connection for you once it goes out of scope. This should allow you to delete the file immediately after as all of the file locks associated with it should be released.
bool OpenConn()
{
try
{
using(Conn = new SqlCeConnection(String.Format(#"Data Source={0}\{1}", PathI, "PC.SDF")))
{
Conn.Open();
return true;
}
}
catch
{
//MessageBox.Show(err.Message, "Connetion error");
return false;
}
}
if (File.Exists(#"\myPath\PC.sdf"))
File.Delete(#"\myPath\PC.sdf");
The two responses above are both good considerations for ensuring that it is not YOUR application which is the one using the database file.
I'm not exactly sure what an .SDF file is, but a quick google search suggests that it may be a MS SQL database file. If so, depending on your system and your operating conditions, you may want to consider using the ALTER DATABASE SET SINGLE_USER command:
ALTER DATABASE [YourDbName]
SET SINGLE_USER WITH ROLLBACK IMMEDIATE;
http://blog.sqlauthority.com/2010/02/11/sql-server-alter-database-dbname-set-single_user-with-rollback-immediate/
I'm getting this error (Distributed transaction completed. Either enlist this session in a new transaction or the NULL transaction.) when trying to run a stored procedure from C# on a SQL Server 2005 database. I'm not actively/purposefully using transactions or anything, which is what makes this error weird. I can run the stored procedure from management studio and it works fine. Other stored procedures also work from C#, it just seems to be this one with issues. The error returns instantly, so it can't be a timeout issue. The code is along the lines of:
SqlCommand cmd = null;
try
{
// Make sure we are connected to the database
if (_DBManager.CheckConnection())
{
cmd = new SqlCommand();
lock (_DBManager.SqlConnection)
{
cmd.CommandText = "storedproc";
cmd.CommandType = System.Data.CommandType.StoredProcedure;
cmd.Connection = _DBManager.SqlConnection;
cmd.Parameters.AddWithValue("#param", value);
int affRows = cmd.ExecuteNonQuery();
...
}
}
else
{
...
}
}
catch (Exception ex)
{
...
}
It's really got me stumped. Thanks for any help
It sounds like there is a TransactionScope somewhere that is unhappy. The _DBManager.CheckConnection and _DBManager.SqlConnection sounds like you are keeping a SqlConnection hanging around, which I expect will contribute to this.
To be honest, in most common cases you are better off just using the inbuilt connection pooling, and using your connections locally - i.e.
using(var conn = new SqlConnection(...)) { // or a factory method
// use it here only
}
Here you get a clean SqlConnection, which will be mapped to an unmanaged connection via the pool, i.e. it doesn't create an actual connection each time (but will do a logical reset to clean it up).
This also allows much more flexible use from multiple threads. Using a static connection in a web app, for example, would be horrendous for blocking.
From the code it seems that you are utilizing an already opened connection. May be there's a transaction pending previously on the same connection.