I want to make use of the connection pooling so i could use the database without passing the MysqlConnection object to each class. I have a code like this:
Main.cs:
namespace batman
{
class Program
{
static void Main(string[] args)
{
using (MysqlConnection conn = new MysqlConnection("Server=localhost;User=root;Database=test;Password=root;Min Pool Size=3;Max Pool Size=5;Pooling=True"))
{
MonitorClass monitor = new MonitorClass();
monitor.Run();
}
//...
}
}
}
MonitorClass.cs:
namespace batman
{
public class MonitorClass
{
public void Run()
{
using (MySqlConnection conn = new MySqlConnection())
using (MySqlCommand cmd = conn.CreateCommand())
{
try
{
conn.Open();
cmd.CommandText = "SELECT id, package_type FROM package_registry WHERE finish_time <= #ftime";
cmd.Parameters.AddWithValue("#ftime", 0);
cmd.Prepare();
MySqlDataReader reader = cmd.ExecuteReader();
while (reader.Read())
{
int packageId = reader.GetInt32(0);
string packageType = reader.GetString(1);
Unirest.post("http://localhost/gears/ops/packagefinish")
.field("package", packageId)
.asStringAsync();
Console.WriteLine("[PackageMonitor] Package {0} ({1}) expired", packageId, packageType);
}
}
catch (MySqlException ex)
{
}
}
}
}
}
MonitorClass should take the connection from the connection pool.
But once I run my program it throws System.InvalidOperationException with
Additional information: Unable to connect to any of the specified MySQL hosts.
at:
cmd.Prepare();
Now, i think I'm doing something wrong, but I couldn't figure out what exactly.
Oracle docs says this
The Connector/Net supports connection pooling for better performance and scalability with database-intensive applications. This is enabled by default. You can turn it off or adjust its performance characteristics using the connection string options Pooling, Connection Reset, Connection Lifetime, Cache Server Properties, Max Pool Size and Min Pool Size. See Section 5.2, “Creating a Connector/Net Connection String” for further information.
Connection pooling works by keeping the native connection to the server live when the client disposes of a MySqlConnection. Subsequently, if a new MySqlConnection object is opened, it will be created from the connection pool, rather than creating a new native connection. This improves performance.
Well, there seem to be two different problems:
Connection is opened twice - first time in Main method and second time (independently) in Monitor class. This can be fixed by opening connection in monitor class only and passing connection string inside of it.
You never opened connection by invoking .Open() method.
Let's refactor your code keeping this in mind :
Main :
static void Main(string[] args)
{
var connStr = "Server=localhost;User=root;Database=test;Password=root;Min Pool Size=3;Max Pool Size=5;Pooling=True";
MonitorClass monitor = new MonitorClass(connStr);
monitor.Run();
//...
}
Monitor :
public class MonitorClass
{
private readonly string _connStr;
public MonitorClass(string connectionString)
{
this._connStr = connectionString;
}
public void Run()
{
using (MySqlConnection conn = new MySqlConnection(_connStr))
using (MySqlCommand cmd = conn.CreateCommand())
{
conn.Open();
...
}
}
}
Related
I need every thread to connect to one database and execute some query and maybe some thread in one time can execute the query.
I create a static class for connecting to SQL Server database with ADO.NET:
public static class ADOFire
{
static System.Data.SqlClient.SqlConnection Cnn = new System.Data.SqlClient.SqlConnection();
public static string CnnString { get; set; }
public static void CreateConnection()
{
if (Cnn.State == ConnectionState.Open)
return;
Cnn.ConnectionString = CnnString = ConfigurationManager.ConnectionStrings["CnnString"].ToString();
if (Cnn.State != System.Data.ConnectionState.Open)
Cnn.Open();
}
public static System.Data.DataTable GetTable(System.Data.SqlClient.SqlCommand Com, System.Data.SqlClient.SqlDataAdapter Ada, string ComText)
{
CreateConnection();
Com.Connection = Cnn;
Ada.SelectCommand = Com;
try
{
System.Data.DataTable T = new System.Data.DataTable();
Com.CommandText = ComText;
Ada.Fill(T);
return T;
}
catch { return null; }
}
}
And in here in each thread I call static function like this:
System.Data.SqlClient.SqlCommand Com = new System.Data.SqlClient.SqlCommand();
System.Data.SqlClient.SqlDataAdapter Ada = new System.Data.SqlClient.SqlDataAdapter();
Datatable dt = ADOFire.GetTable(Com, Ada, "Some Query 'select * from x'");
Based on this link, doesn't make much difference between open a new connection or use from existing connection
Whenever a user calls Open on a connection, the pooler looks for an available connection in the pool. If a pooled connection is available, it returns it to the caller instead of opening a new connection. When the application calls Close on the connection, the pooler returns it to the pooled set of active connections instead of closing it. Once the connection is returned to the pool, it is ready to be reused on the next Open call
My questions are:
Can a connection serve different threads at the same time? (one connection for all)
Isn't the problem of data clutter due to the static function?
No, ADO.NET does not two threads accessing the same.conection at the same time
Yes, but it isn't the statix method that is a problem - it is the static connection field that is a huge problem
What you should do is have the static method create (new) and return the connection to the caller, and remove the field completely. Typical usage:
using (var conn = CreateConnection()) {
//... Use it!
}
I also have grave concerns about:
- why you're passing in a command and command text and adapter
- the lack of parameters; suggests a huge security problem (SQL injection)
- the use of data table (which is almost never the most appropriate tool)
I have a problem. I'm writing window form application in c#.net and connect to SQL Server on Local Network.
if network disconnected my program is not responding. I mean it tries to select data from sql server but network is disconnected. How to catch this error? Thanks in advance.
Check the documentation for timing out MSDN
Here's their example code
using System;
using System.Data.SqlClient;
///
public class A {
///
public static void Main() {
string connectionString = "";
// Wait for 5 second delay in the command
string queryString = "waitfor delay '00:00:05'";
using (SqlConnection connection = new SqlConnection(connectionString)) {
connection.Open();
SqlCommand command = new SqlCommand(queryString, connection);
// Setting command timeout to 1 second
command.CommandTimeout = 1;
try {
command.ExecuteNonQuery();
}
catch (SqlException e) {
Console.WriteLine("Got expected SqlException due to command timeout ");
Console.WriteLine(e);
}
}
}
}
Notice the line
command.CommandTimeout = 1;
Definitely wrap your sql code under the 'using' too, as it will deallocate resources automatically for you.
You basically need to check for the Open Connection First!
SqlConnection con = new SqlConnection(Connection_string); //specify your connection string such as Server database etc ...
if (con.State == System.Data.ConnectionState.Open)
//statements
And you can make an event to Notify that the connection has been closed !
When I have a SQLConnection within a using clause as illustrated below do I need to explicitly close the connection?
protected SqlConnection Connection
{
get
{
if (this.connection == null)
{
this.connection = new SqlConnection(this.ConnectionString);
}
if (this.connection.State != ConnectionState.Open)
{
this.connection.Open();
}
return this.connection;
}
}
using (SqlConnection connection = this.Connection)
{
using (SqlCommand cmd = connection.CreateCommand())
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "....";
using (SqlDataReader reader = cmd.ExecuteReader())
{
while (reader.Read())
{
etc
}
}
}
}
No you don't. The Dispose() method of a connection will call Close if the connection is already open.
You should also change your code as John Gathogo suggests to create a new connection object each time it needs one. Your code will fail as it is because the second time you try to use the connection it will already be disposed.
ADO.NET uses connection pooling to keep a pool of open connections which it provides to whoever calls Open. This means that creating and opening new connection doesn't cost anything as long as there are available connections in the pool. Keeping a connection open for longer than necessary will degrade performance.
The using block will always call Dispose, which will close the connection.
But, you are keeping the connection object and intend to reuse it, which is not possible once it has been disposed. You shouldn't keep the connection object, you should just throw it away and create a new one when needed. The actual connections to the database are pooled, so when you create a new connection object it will reuse one of the connections from the pool. When you dispose the connection object, the actual connection is returned to the pool.
You could easily change you code to below and achieve what you want:
using (SqlConnection connection = new SqlConnection(this.ConnectionString))
{
connection.Open();
using (SqlCommand cmd = connection.CreateCommand())
{
cmd.CommandType = CommandType.StoredProcedure;
cmd.CommandText = "....";
using (SqlDataReader reader = cmd.ExecuteReader())
{
while (reader.Read())
{
//etc
}
}
}
}
The runtime will take care of closing the connection and disposing the resources for you
So what i'm looking to do is open a MysqlConnection and then never close it for the entire application (until the end).
I have this:
static void Main(string[] args)
{
OpenCon();
}
public static MySqlConnection OpenCon()
{
MySqlConnection masterOpenCON = new MySqlConnection(SQLStringClass.masterConString);
masterOpenCON.Open();
return masterOpenCON;
}
However, i believe this will continuously open the connect, how do i open the connection once and reference the connection throughout the app. This is how I am calling it right now.
try
{
MySqlCommand mysqlprocessCmdInsertItem = new MySqlCommand(SQLStringClass.mySQLCOMMAND, OpenCon());
mysqlprocessCmdInsertItem.ExecuteNonQuery();
}
Don't bother with this. The MySQL ADO.NET Connector uses a pool of connections meaning that when you call .Open on a connection you are not actually opening it, you are drawing it from the existing connection pool and when you call .Close you are not closing it, you are returning it to the connection pool so that it can be reused.
IIRC connection pooling is enabled by default but you could control it with the following parameters on your connection string:
Pooling = true
Max Pool Size = 100
Min Pool Size = 0
So when you want to send a SQL query all you need to do is the following and let the ADO.NET framework worry about connections:
using (var conn = new MySqlConnection(SQLStringClass.masterConString))
using (var cmd = conn.CreateCommand())
{
conn.Open();
cmd.CommandText = "SELECT Foo FROM Bar";
using (var reader = cmd.ExecuteReader())
{
while (reader.Read())
{
...
}
}
}
I would recommend you avoiding any static members and manual connection handling.
You need to store the connection in a static field or property.
For example:
public static MySqlConnection Connection { get; private set; }
Connection = OpenCon();
You need to also specify the connection object as static.
private static MySqlConnection masterOpenCON;
static void Main(string[] args)
{
OpenCon();
}
public static MySqlConnection OpenCon()
{
masterOpenCON = new MySqlConnection(SQLStringClass.masterConString);
masterOpenCON.Open();
return masterOpenCON;
}
A while back I asked a question about TransactionScope escalating to MSDTC when I wasn't expecting it to. (Previous question)
What it boiled down to was, in SQL2005, in order to use a TransactionScope, you can only instance and open a single SqlConnection within the life of the TransactionScope. With SQL2008, you can instance multiple SqlConnections, but only a single one can be open at any given time. SQL2000 will always escalate to DTC...we don't support SQL2000 in our application, a WinForms app, BTW.
Our solution to single-connection-only problem was to create a TransactionScope helper class, called LocalTransactionScope (aka 'LTS'). It wraps a TransactionScope and, most importantly, creates and maintains a single SqlConnection instance for our application. The good news is, it works - we can use LTS across disparate pieces of code and they all join the ambient transaction. Very nice. The trouble is, every root LTS instance created will create and effectively kill a connection from the connection pool. By 'Effectively Kill' I mean it will instance a SqlConnetion, which will open a new connection (for whatever reason, it never reuses a connection from the pool,) and when that root LTS is disposed, it closes and disposes the SqlConnection which is supposed to release the connection back to the pool so that it can be reused, however, it clearly never is reused. The pool bloats until it's maxed out, and then the application fails when a max-pool-size+1 connection is established.
Below I've attached a stripped down version of the LTS code and a sample console application class that will demonstrate the connection pool exhaustion. In order to watch your connection pool bloat, use SQL Server Managment Studio's 'Activity Monitor' or this query:
SELECT DB_NAME(dbid) as 'DB Name',
COUNT(dbid) as 'Connections'
FROM sys.sysprocesses WITH (nolock)
WHERE dbid > 0
GROUP BY dbid
I'm attaching LTS here, and a sample console application that you can use to demonstrate for yourself that it will consume connections from the pool and never re-use nor release them. You will need to add a reference to System.Transactions.dll for LTS to compile.
Things to note: It's the root-level LTS that opens and closes the SqlConnection, which always opens a new connection in the pool. Having nested LTS instances makes no difference because only the root LTS instance establishes a SqlConnection. As you can see, the connection string is always the same, so it should be reusing the connections.
Is there some arcane condition we're not meeting that causes the connections not to be re-used? Is there any solution to this other than turning pooling off entirely?
public sealed class LocalTransactionScope : IDisposable
{
private static SqlConnection _Connection;
private TransactionScope _TransactionScope;
private bool _IsNested;
public LocalTransactionScope(string connectionString)
{
// stripped out a few cases that need to throw an exception
_TransactionScope = new TransactionScope();
// we'll use this later in Dispose(...) to determine whether this LTS instance should close the connection.
_IsNested = (_Connection != null);
if (_Connection == null)
{
_Connection = new SqlConnection(connectionString);
// This Has Code-Stink. You want to open your connections as late as possible and hold them open for as little
// time as possible. However, in order to use TransactionScope with SQL2005 you can only have a single
// connection, and it can only be opened once within the scope of the entire TransactionScope. If you have
// more than one SqlConnection, or you open a SqlConnection, close it, and re-open it, it more than once,
// the TransactionScope will escalate to the MSDTC. SQL2008 allows you to have multiple connections within a
// single TransactionScope, however you can only have a single one open at any given time.
// Lastly, let's not forget about SQL2000. Using TransactionScope with SQL2000 will immediately and always escalate to DTC.
// We've dropped support of SQL2000, so that's not a concern we have.
_Connection.Open();
}
}
/// <summary>'Completes' the <see cref="TransactionScope"/> this <see cref="LocalTransactionScope"/> encapsulates.</summary>
public void Complete() { _TransactionScope.Complete(); }
/// <summary>Creates a new <see cref="SqlCommand"/> from the current <see cref="SqlConnection"/> this <see cref="LocalTransactionScope"/> is managing.</summary>
public SqlCommand CreateCommand() { return _Connection.CreateCommand(); }
void IDisposable.Dispose() { this.Dispose(); }
public void Dispose()
{
Dispose(true); GC.SuppressFinalize(this);
}
private void Dispose(bool disposing)
{
if (disposing)
{
_TransactionScope.Dispose();
_TransactionScope = null;
if (!_IsNested)
{
// last one out closes the door, this would be the root LTS, the first one to be instanced.
LocalTransactionScope._Connection.Close();
LocalTransactionScope._Connection.Dispose();
LocalTransactionScope._Connection = null;
}
}
}
}
This is a Program.cs that will exhibit the connection pool exhaustion:
class Program
{
static void Main(string[] args)
{
// fill in your connection string, but don't monkey with any pooling settings, like
// "Pooling=false;" or the "Max Pool Size" stuff. Doesn't matter if you use
// Doesn't matter if you use Windows or SQL auth, just make sure you set a Data Soure and an Initial Catalog
string connectionString = "your connection string here";
List<string> randomTables = new List<string>();
using (var nonLTSConnection = new SqlConnection(connectionString))
using (var command = nonLTSConnection.CreateCommand())
{
command.CommandType = CommandType.Text;
command.CommandText = #"SELECT [TABLE_NAME], NEWID() AS [ID]
FROM [INFORMATION_SCHEMA].TABLES]
WHERE [TABLE_SCHEMA] = 'dbo' and [TABLE_TYPE] = 'BASE TABLE'
ORDER BY [ID]";
nonLTSConnection.Open();
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
string table = (string)reader["TABLE_NAME"];
randomTables.Add(table);
if (randomTables.Count > 200) { break; } // got more than enough to test.
}
}
nonLTSConnection.Close();
}
// we're going to assume your database had some tables.
for (int j = 0; j < 200; j++)
{
// At j = 100 you'll see it pause, and you'll shortly get an InvalidOperationException with the text of:
// "Timeout expired. The timeout period elapsed prior to obtaining a connection from the pool.
// This may have occurred because all pooled connections were in use and max pool size was reached."
string tableName = randomTables[j % randomTables.Count];
Console.Write("Creating root-level LTS " + j.ToString() + " selecting from " + tableName);
using (var scope = new LocalTransactionScope(connectionString))
using (var command = scope.CreateCommand())
{
command.CommandType = CommandType.Text;
command.CommandText = "SELECT TOP 20 * FROM [" + tableName + "]";
using (var reader = command.ExecuteReader())
{
while (reader.Read())
{
Console.Write(".");
}
Console.Write(Environment.NewLine);
}
}
Thread.Sleep(50);
scope.Complete();
}
Console.ReadKey();
}
}
The expected TransactionScope/SqlConnection pattern is, according to MSDN:
using(TransactionScope scope = ...) {
using (SqlConnection conn = ...) {
conn.Open();
SqlCommand.Execute(...);
SqlCommand.Execute(...);
}
scope.Complete();
}
So in the MSDN example the conenction is disposed inside the scope, before the scope is complete. Your code though is different, it disposes the connection after the scope is complete. I'm not an expert in matters of TransactionScope and its interaction with the SqlConnection (I know some things, but your question goes pretty deep) and I can't find any specifications what is the correct pattern. But I'd suggest you revisit your code and dispose the singleton connection before the outermost scope is complete, similarly to the MSDN sample.
Also, I hope you do realize your code will fall apart the moment a second thread comes to play into your application.
Is this code legal?
using(TransactionScope scope = ..)
{
using (SqlConnection conn = ..)
using (SqlCommand command = ..)
{
conn.Open();
SqlCommand.Execute(..);
}
using (SqlConnection conn = ..) // the same connection string
using (SqlCommand command = ..)
{
conn.Open();
SqlCommand.Execute(..);
}
scope.Complete();
}