I am new to C# and trying to read a value out of a MySQL Database.
What i get is a single number.
I store that into an int variable and check for specific numbers to activate functions of a kuando busylight.
The whole thing is looped (tried while (true) {} and goto methods).
It works so far, but my memory usage is increasing. starting at ~6 MiB...30MiB and so on.
Somehow it saves data into the process but I did not find out how to clear the used memory.
As is said im very very new to this, maybe my code is toooooo crappy to work fine and its now problem of clearing unused data. just tell me :D
Thank you very much!
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using MySql.Data.MySqlClient;
using Plenom.Components.Busylight.Sdk;
using System.Threading;
namespace TANSS4BLL
{
class Program
{
static void Main()
{
var controller = new BusylightUcController();
int resultStatus;
while (true)
{
string sqlDataBaseSelect = "SELECT typID FROM az_manager where maID = 4328 ORDER BY datum DESC LIMIT 1";
string ConnectionString = "Server=localhost; Database=tanss; Uid=root; Pwd=password";
using (MySqlConnection connDataBase = new MySqlConnection(ConnectionString))
{
connDataBase.Open();
MySqlCommand cmd = new MySqlCommand(sqlDataBaseSelect, connDataBase);
resultStatus = (int)cmd.ExecuteScalar();
if (resultStatus == 1)
{
//Console.WriteLine("Debugtext1");
controller.Light(BusylightColor.Green);
}
else if (resultStatus == 2)
{
//Console.WriteLine("Debugtext2");
controller.Light(BusylightColor.Red);
}
else if (resultStatus == 3)
{
//Console.WriteLine("Debugtext3");
controller.Light(BusylightColor.Red);
}
else if (resultStatus == 9)
{
//Console.WriteLine("Debugtext4");
controller.Light(BusylightColor.Red);
}
connDataBase.Close();
}
//Console.WriteLine(resultStatus);
Thread.Sleep(1000);
//Console.Clear();
}
}
}
}
You are creating multiple connections to the database with your while(true) loop.
string ConnectionString = "Server=localhost; Database=tanss; Uid=root; Pwd=password";
This line can easily be move out of the while loop and you that should help with your problem.
Also, if this turns into production code, you need to set up parameters when you are working with the database. Parameters are how we avoid sql injection.
Avoiding SQL Injection
Hopefully this helps you out!
You don't have to handle it, Garbage Collector will handle it automatically. But try to avoid reconnecting to database in loop. If you need to do it this way - put MySqlCommand in loop, not MySqlConnection.
Garbage Collector automaticaly clear unused resources. You can run Garbage collector manualy, but it is quite expensive operation, so don't put it inside while(true) loop. Mor info about GC is here: http://msdn.microsoft.com/en-us/library/s5zscb2d(v=vs.85).aspx
One of the things I see that you are doing is you are not disposing of the MySqlCommand Object, that's going to continually build and take up resources. You need to properly dispose of that object whether it be through a using statement or call the dispose method on it. Also I would start you mysql Connection outside the for loop if your going to continually poll the database. I would also put some try catch logic into there as well to catch DB connection errors. Here is how I would attack this,
var controller = new BusylightUcController();
int resultStatus;
string sqlDataBaseSelect = "SELECT typID FROM az_manager where maID = 4328 ORDER BY datum DESC LIMIT 1";
string ConnectionString = "Server=localhost; Database=tanss; Uid=root; Pwd=password";
using (MySqlConnection connDataBase = new MySqlConnection(ConnectionString))
{
connDataBase.Open();
try
{
while (true)
{
using(MySqlCommand cmd = new MySqlCommand(sqlDataBaseSelect, connDataBase))
{
resultStatus = (int)cmd.ExecuteScalar();
switch(resultStatus)
{
case 1:
//Console.WriteLine("Debugtext1");
controller.Light(BusylightColor.Green);
break;
case 2:
//Console.WriteLine("Debugtext2");
controller.Light(BusylightColor.Red);
break;
case 3:
//Console.WriteLine("Debugtext3");
controller.Light(BusylightColor.Red);
break;
case 9:
//Console.WriteLine("Debugtext4");
controller.Light(BusylightColor.Red);
break;
default:
// handle when its not any of your results.
break;
}
}
//Console.WriteLine(resultStatus);
Thread.Sleep(1000);
//Console.Clear();
}
}catch(Exception ex)
{
//HandleException
}finally
{
connDataBase.Close();
}
}
Related
I am working on some code that I would like to access an Oracle database inside of a Parallel.For loop. The loop will run for several minutes, and then result in the error:
"Attempted to read or write protected memory. This is often an
indication that other memory is corrupt."
There is no inner exception. Inside my Parallel.For loop, I am creating an opening the database connection as local objects. My code looks like this:
static void CheckSinglePath(Path p)
{
string sqlBase = "select * from table where hour = #HOUR#";
Parallel.For (1, 24, i =>
{
DBManager localdbm = new DBManager();
string sql = sqlBase;
sql = sql.Replace("#HOUR#", i.ToString());
OracleDataReader reader = db.GetData(sql);
if (reader.Read())
{
//do some stuff
}
reader.Close();
});
}
class DBManager
{
OracleConnection conn;
OracleCommand cmd;
public DBManager()
{
string connStr = "blahblahblah;Connection Timeout=600;";
conn = new OracleConnection(connStr);
conn.Open();
cmd = conn.CreateCommand();
}
public OracleDataReader GetData(string sql)
{
cmd.CommandText = sql;
return cmd.ExecuteReader();//EXCEPTION HERE!
}
}
What am I doing wrong? How can I create 24 parallel Oracle connections to process the data? I'm guessing there is some sort of race condition or memory leak that is going on here which I don't fully understand because it seems to be coming from inside the OracleConnection object. Is the database connection not threadsafe? I tried changing the connection string to use a connection pool and that didn't change anything.
Memory problems is always caused by wrong resources usage. You do not properly release your connections after the loop exit.
You need to implement IDisposable interface and after that you need rewrite your code in such manner with using keyword:
// dispose the connection after command finished
using (var localdbm = new DBManager())
{
var sql = sqlBase;
sql = sql.Replace("#HOUR#", i.ToString());
using (var reader = db.GetData(sql))
{
if (reader.Read())
{
//do some stuff
}
// no need to close reader
// as it's being disposed inside using directive
}
}
For simple lookups, I need to perform some SQL statements on a DB2 machine. I'm not able to use an ORM at the moment. I have a working example through this code, however I'm wondering if it can be optimized more as this would essentially create a connection on each request. And that just seems like bad programming.
Is there a way I can optimize this Get() request to leave a connection open? Nesting using statements seems dirty, as well. How should I handle the fact that Get() really wants to return an object of User no matter what; even in error? Can I put this connection in the start of the program so that I can use it over and over again? What are some of the best practices for this?
public class UsersController : ApiController
{
String constr = WebConfigurationManager.ConnectionStrings["DB2Connection"].ConnectionString;
public User Get([FromUri] User cst)
{
if (cst == null)
{
throw new HttpResponseException(HttpStatusCode.NotFound);
}
else
{
using (OdbcConnection DB2Conn = new OdbcConnection(constr))
{
DB2Conn.Open();
using (OdbcCommand com = new OdbcCommand(
// Generic SQL Statement
"SELECT * FROM [TABLE] WHERE customerNumber = ?", DB2Conn))
{
com.Parameters.AddWithValue("#var", cst.customerNumber);
using (OdbcDataReader reader = com.ExecuteReader())
{
try
{
while (reader.Read())
{
cst.name = (string)reader["name"];
return cst;
}
}
catch
{
throw;
}
}
}
}
return cst;
}
}
}
I found a great question that doesn't really have detailed answers, I feel like similar solutions exist for both of these questions...
And that just seems like bad programming.
Why do you think that?
The underlying system should be maintaining connections in a connection pool for you. Creating a connection should be very optimized already.
From a logical perspective, what you're doing now is exactly what you want to be doing. Create the connection, use it, and dispose of it immediately. This allows other threads/processes/etc. to use it from the connection pool now that you're done with it.
This also avoids the myriad of problems which arise from manually maintaining your open connections outside of the code that uses them.
Is there a way I can optimize this Get() request to leave a connection open?
Have you measured an actual performance problem? If not, there's nothing to optimize.
And there's a very good chance that hanging on to open connections in a static context in your web application is going to have drastic performance implications.
In short... You're already doing this correctly. (Well, except for that unnecessary try/catch. You can remove that.)
Edit: If you're just looking to improve the readability of the code (which itself is a matter of personal preference), this seems readable to me:
public User Get([FromUri] User cst)
{
if (cst == null)
throw new HttpResponseException(HttpStatusCode.NotFound);
using (var DB2Conn = new OdbcConnection(constr))
using (var com = new OdbcCommand("SELECT * FROM [TABLE] WHERE customerNumber = ?", DB2Conn))
{
com.Parameters.AddWithValue("#var", cst.customerNumber);
DB2Conn.Open();
using (OdbcDataReader reader = com.ExecuteReader())
while (reader.Read())
{
cst.name = (string)reader["name"]
return cst;
}
}
return cst;
}
Note that you can further improve it by re-addressing the logic of that SQL query. Since you're fetching one value from one record then you don't need to loop over a data reader. Just fetch a single literal and return it. Note that this is free-hand and untested, but it might look something like this:
public User Get([FromUri] User cst)
{
if (cst == null)
throw new HttpResponseException(HttpStatusCode.NotFound);
using (var DB2Conn = new OdbcConnection(constr))
using (var com = new OdbcCommand("SELECT name FROM [TABLE] WHERE customerNumber = ? FETCH FIRST 1 ROWS ONLY", DB2Conn))
{
com.Parameters.AddWithValue("#var", cst.customerNumber);
DB2Conn.Open();
cst.name = (string)com.ExecuteScalar();
}
return cst;
}
#David's answer addresses your actual questions perfectly but here's some other observations that may make your code a little more pallatable to you:
remove the try/catch block - all you're doing is re-throwing the exception which is what will happen if you don't use a try/catch at all. Don't catch the exception unless you can do something about it. (I see now that #David's answer addresses that - either it was added after I read it or I missed it - my apologies for the overlap but it's worth reinforcing)
Change your query to just pull name and use ExecuteScalar instead of ExecuteReader. You are taking the name value from the first record and exiting the while loop. ExecuteScalar returns the value from the first column in the first record, so you can eliminate the while loop and the using there.
I have a SqlDataReader, but it never enters into Read().
When I debug it, it pass the loop while(readerOne.Read()). It never enter into this loop even though there is data.
public static List<Pers_Synthese> Get_ListeSynthese_all(string codeClient, DateTime DateDeb, DateTime DateFin)
{
try
{
using (var connectionWrapper = new Connexion())
{
var connectedConnection = connectionWrapper.GetConnected();
string sql_Syntax = Outils.LoadFileToString(Path.Combine(appDir, #"SQL\Get_ListeSynthese_All.sql"));
SqlCommand comm_Command = new SqlCommand(sql_Syntax, connectionWrapper.conn);
comm_Command.Parameters.AddWithValue("#codeClioent", codeClient);
comm_Command.Parameters.AddWithValue("#DateDeb", DateDeb);
comm_Command.Parameters.AddWithValue("#DateFin", DateFin);
List<Pers_Synthese> oListSynthese = new List<Pers_Synthese>();
SqlDataReader readerOne = comm_Command.ExecuteReader();
while (readerOne.Read())
{
Pers_Synthese oSyntehse = new Pers_Synthese();
oSyntehse.CodeTrf = readerOne["CODE_TARIF"].ToString();
oSyntehse.NoLV = readerOne["NOID"].ToString();
oSyntehse.PrxUnitaire = readerOne["PRIX_UNITAIRE"].ToString();
oSyntehse.ZoneId = readerOne["LE_ZONE"].ToString();
oSyntehse.LeZone = readerOne["LIB_ZONE"].ToString();
oSyntehse.LeDept = readerOne["DEPT"].ToString();
oSyntehse.LeUnite = readerOne["ENLEV_UNITE"].ToString();
oSyntehse.LePoids = Convert.ToInt32(readerOne["POID"]);
//oSyntehse.LePoidsCorr = Convert.ToInt32(readerOne["POID_CORR"]);
oSyntehse.LeColis = readerOne["NBR_COLIS"].ToString();
oSyntehse.LeCr = readerOne["NBR_CREMB"].ToString();
oSyntehse.SumMontantCR = readerOne["ENLEV_CREMB"].ToString();
oSyntehse.LeVd = readerOne["NBR_DECL"].ToString();
oSyntehse.SumMontantVD = readerOne["ENLEV_DECL"].ToString();
oSyntehse.LePrixHT = readerOne["PRIX_HT"].ToString();
oSyntehse.LePrixTTC = readerOne["PRIX_TTC"].ToString();
oSyntehse.TrDeb = readerOne["TR_DEB"].ToString();
oSyntehse.TrFin = readerOne["TR_FIN"].ToString();
oListSynthese.Add(oSyntehse);
}
readerOne.Close();
readerOne.Dispose();
return oListSynthese;
}
}
catch (Exception excThrown)
{
throw new Exception(excThrown.Message);
}
}
When I debug it with SQL Server profiler it shows the data....that meant the data is not empty, but it never enter into this loop.
while (readerOne.Read())
{
by the way my connection class:
class Connexion : IDisposable
{
public SqlConnection conn;
public SqlConnection GetConnected()
{
try
{
string strConnectionString = Properties.Settings.Default.Soft8Exp_ClientConnStr;
conn = new SqlConnection(strConnectionString);
}
catch (Exception excThrown)
{
conn = null;
throw new Exception(excThrown.InnerException.Message, excThrown);
}
// Ouverture et restitution de la connexion en cours
if (conn.State == ConnectionState.Closed) conn.Open();
return conn;
}
public Boolean IsConnected
{
get { return (conn != null) && (conn.State != ConnectionState.Closed) && (conn.State != ConnectionState.Broken); }
}
public void CloseConnection()
{
// Libération de la connexion si elle existe
if (IsConnected)
{
conn.Close();
conn = null;
}
}
public void Dispose()
{
CloseConnection();
}
}
and my SQL Statement:
exec sp_executesql N'SELECT CODE_TARIF,PRIX_UNITAIRE,TR_DEB,TR_FIN,LE_ZONE,T_TARIF_ZONE.LIBELLE as LIB_ZONE,
SUBSTRING(CP_DEST,1,2) as DEPT,T_UNITE.LIBELLE as ENLEV_UNITE,
count(NOID)as NOID,
SUM(CASE WHEN POID_CORR IS NOT NULL THEN POID_CORR ELSE POID END) as POID,sum(NBR_COLIS)as NBR_COLIS,COUNT(NULLIF(ENLEV_CREMB,0))as NBR_CREMB, sum(ENLEV_CREMB)as ENLEV_CREMB,COUNT(NULLIF(ENLEV_DECL,0))as NBR_DECL,sum(ENLEV_DECL)as ENLEV_DECL,sum(PRIX_HT)as PRIX_HT,sum(PRIX_TTC)as PRIX_TTC, sum (POID_CORR)as POID_CORR
FROM LETTRE_VOIT_FINAL
LEFT JOIN T_TARIF_ZONE ON LETTRE_VOIT_FINAL.LE_ZONE = T_TARIF_ZONE.NO_ID
LEFT JOIN T_UNITE ON LETTRE_VOIT_FINAL.ENLEV_UNITE = T_UNITE.NO_ID
where code_client = #codeClioent
and DATE_CLOTUR_REEL BETWEEN #DateDeb AND #DateFin
and STATUT_LV = 2
group by CODE_TARIF,PRIX_UNITAIRE,TR_DEB,TR_FIN,LE_ZONE,T_TARIF_ZONE.LIBELLE,SUBSTRING(CP_DEST,1,2),T_UNITE.LIBELLE
order by LE_ZONE,PRIX_UNITAIRE
',N'#codeClioent nvarchar(8),#DateDeb datetime,#DateFin datetime',#codeClioent=N'17501613',#DateDeb='2013-06-05 00:00:00',#DateFin='2013-06-05 23:59:00'
it return the data on SQL profiler:
my real query :
SELECT CODE_TARIF,PRIX_UNITAIRE,TR_DEB,TR_FIN,LE_ZONE,T_TARIF_ZONE.LIBELLE as LIB_ZONE,
SUBSTRING(CP_DEST,1,2) as DEPT,T_UNITE.LIBELLE as ENLEV_UNITE,
count(NOID)as NOID,
SUM(CASE WHEN POID_CORR IS NOT NULL THEN POID_CORR ELSE POID END) as POID,sum(NBR_COLIS)as NBR_COLIS,COUNT(NULLIF(ENLEV_CREMB,0))as NBR_CREMB, sum(ENLEV_CREMB)as ENLEV_CREMB,COUNT(NULLIF(ENLEV_DECL,0))as NBR_DECL,sum(ENLEV_DECL)as ENLEV_DECL,sum(PRIX_HT)as PRIX_HT,sum(PRIX_TTC)as PRIX_TTC, sum (POID_CORR)as POID_CORR
FROM LETTRE_VOIT_FINAL
LEFT JOIN T_TARIF_ZONE ON LETTRE_VOIT_FINAL.LE_ZONE = T_TARIF_ZONE.NO_ID
LEFT JOIN T_UNITE ON LETTRE_VOIT_FINAL.ENLEV_UNITE = T_UNITE.NO_ID
where code_client = #codeClioent
and DATE_CLOTUR_REEL BETWEEN #DateDeb AND #DateFin
and STATUT_LV = 2
group by
CODE_TARIF,PRIX_UNITAIRE,TR_DEB,TR_FIN,LE_ZONE,T_TARIF_ZONE.LIBELLE,SUBSTRING(CP_DEST,1,2),T_UNITE.LIBELLE
order by LE_ZONE,PRIX_UNITAIRE
it is strange....when the data is between :
DATE_CLOTUR_REEL BETWEEN '2013-06-05 00:00:00' and '2013-06-05 23:59:00'
but
DATE_CLOTUR_REEL BETWEEN '2013-06-01 00:00:00' and '2013-06-05 23:59:00'
it works.
This is the way it should be. You are not doing the connection.Open()
Also set up the connection string.
private static void ReadOrderData(string connectionString)
{
string queryString =
"SELECT OrderID, CustomerID FROM dbo.Orders;";
using (SqlConnection connection =
new SqlConnection(connectionString))
{
SqlCommand command =
new SqlCommand(queryString, connection);
connection.Open();
SqlDataReader reader = command.ExecuteReader();
// Call Read before accessing data.
while (reader.Read())
{
ReadSingleRow((IDataRecord)reader);
}
// Call Close when done reading.
reader.Close();
}
}
The perfect example of how to do it belongs to MSDN - Microsoft Website
NOTICE:
SqlCommand command =
new SqlCommand(queryString, connection);
connection.Open();
SqlDataReader reader = command.ExecuteReader();
Create the SqlCommand
then open the connection
You are doing it the other way, you open it and then create the command.
I also don't see where you set the query string, I just see that you add the parameters; are you missing it?
This is perhaps not the answer you're looking for, but your code sample exhibits a number of bad coding practices that are easy to fall into due to ADO.NET's poor API design. Rather than manually do all this sql-to-.net conversion, you should use a library that does this for you.
It's easier to avoid bugs when you're not using a bug-prone API.
I recommend PetaPoco - it's easier to use than your current code, and has virtually no overhead (and given your example, is probably faster). There are many other alternatives, however.
Issues with your code sample:
Improperly disposed objects: you aren't disposing SqlCommand and SqlDataReader properly. You possibly aren't disposing connections either (but that depends on Connexion internals).
Using .ToString rather than type-safe casts. You should never extract data from an SqlDataReader like that because it undermines the whole point of the type system, and it's slow to boot. (PetaPoco or something similar will help a lot here)
You're discarding stack traces on error due to the (pointless) try-catch. That just makes your code less readable and harder to debug. Don't catch unless you have have.
Keeping your query away from the code - your code is tightly coupled to the query, and this separation just makes it hard to keep them in sync. Also, loading from the filesystem each and everytime you query is slow and opens up unnecessary filesystem-related failure modes such as locking, max path lengths, and permissions. This is probably the source of your bug - your query probably doesn't do what you think it does.
With PetaPoco or something similar, your entire function would look something like this:
public static List<Pers_Synthese> Get_ListeSynthese_all(
string codeClient, DateTime DateDeb, DateTime DateFin) {
var db = new PetaPoco.Database("Soft8Exp_ClientConnStr");
//you should probably not be storing a query in a file.
//To be clear: your query should not be wrapped in exec sp_executesql,
//ADO.NET will do that for you.
string sql_Syntax = Outils.LoadFileToString(
Path.Combine(appDir, #"SQL\Get_ListeSynthese_All.sql"));
//You'll need to rename Pers_Synthese's properties to match the db,
// or vice versa, or you can annotate the properties with the column names.
return db.Fetch<Pers_Synthese>(sql_Syntax, new {
codeClioent = codeClient, //I suspect this is a typo
DateDeb,
DateFin
});
}
And in that much shorter, readable, faster form, you'll hopefully find whatever bug you have much faster.
Alternatives:
PetaPoco
Dapper (fewer features, but stackoverflow uses it!)
OrmLite (of ServiceStack fame)
Massive (older, uses dynamic which is a feature that can cause bad habits - I don't recommend this unless you really know what you're doing)
You could use heavier, more invasive ORM's like the Entity framework and NHibernate, but these require quite a bit more learning, and they're much slower, and they impose a particular workflow on you which I don't think makes them the best choice in your case.
when i debug it with sql profiler it show the data....that meant the data is not empty, but it never enter into this loop.
It's the other way round: if it never enters into this loop, then it means "the data is empty", i.e. the query returns no rows.
The bug is in your code, not SqlReader: you possibly have the wrong values in your parameters, or maybe the query you read from a file isn't what you think it is. Get the debugger out and inspect the query text and parameters.
I'm looking to figure out the best way to execute a database query using the least amount of boilerplate code. The method suggested in the SqlCommand documentation:
private static void ReadOrderData(string connectionString)
{
string queryString = "SELECT OrderID, CustomerID FROM dbo.Orders;";
using (SqlConnection connection = new SqlConnection(connectionString))
{
SqlCommand command = new SqlCommand(queryString, connection);
connection.Open();
SqlDataReader reader = command.ExecuteReader();
try
{
while (reader.Read())
{
Console.WriteLine(String.Format("{0}, {1}", reader[0], reader[1]));
}
}
finally
{
reader.Close();
}
}
}
mostly consists of code that would have to be repeated in every method that interacts with the database.
I'm already in the habit of factoring out the establishment of a connection, which would yield code more like the following. (I'm also modifying it so that it returns data, in order to make the example a bit less trivial.)
private SQLConnection CreateConnection()
{
var connection = new SqlConnection(_connectionString);
connection.Open();
return connection;
}
private List<int> ReadOrderData()
{
using(var connection = CreateConnection())
using(var command = connection.CreateCommand())
{
command.CommandText = "SELECT OrderID FROM dbo.Orders;";
using(var reader = command.ExecuteReader())
{
var results = new List<int>();
while(reader.Read()) results.Add(reader.GetInt32(0));
return results;
}
}
}
That's an improvement, but there's still enough boilerplate to nag at me. Can this be reduced further? In particular, I'd like to do something about the first two lines of the procedure. I don't feel like the method should be in charge of creating the SqlCommand. It's a tiny piece of repetition as it is in the example, but it seems to grow if transactions are being managed manually or timeouts are being altered or anything like that.
edit: Assume, at least hypothetically, there's going to have to be a bunch of different types of data being returned. And consequently the solution can't be just one one-size-fits-all method, there will have to be a few different ones depending, at minimum, on whether ExecuteNonQuery, ExecuteScalar, ExecuteReader, ExecuteReaderAsync, or any of the others are being called. I'd like to cut down on the repetition among those.
Tried Dapper?
Granted this doesn't get you a DataReader but you might just prefer it this way once you've tried it.
It's about the lightest-weight an ORM can be while still being called an ORM. No more methods to map between DataReader and strong types for me.
Used right here on all the StackExchange sites.
using (var conn = new SqlConnection(cs))
{
var dogs = connection.Query("select name, age from dogs");
foreach (dynamic dog in dogs)
{
Console.WriteLine("{0} age {1}", dog.name, dog.age);
}
}
or
using (var conn = new SqlConnection(cs))
{
var dogs = connection.Query<Dog>("select Name, Age from dogs");
foreach (Dog dog in dogs)
{
Console.WriteLine("{0} age {1}", dog.Name, dog.Age);
}
}
class Dog
{
public string Name { get; set; }
public int Age { get; set; }
}
If you want to roll data access on your own, this pattern of help methods could be one way to remove duplication:
private List<int> ReadOrderData()
{
return ExecuteList<int>("SELECT OrderID FROM dbo.Orders;",
x => x.GetInt32("orderId")).ToList();
}
private IEnumerable<T> ExecuteList(string query,
Func<IDataRecord, T> entityCreator)
{
using(var connection = CreateConnection())
using(var command = connection.CreateCommand())
{
command.CommandText = query;
connection.Open();
using(var reader = command.ExecuteReader())
{
while(reader.Read())
yield return entityCreator(reader);
}
}
}
You'll have to add support for parameters and this might not compile, but the pattern is what I'm trying to illustrate.
What I typically do is use a custom class that I wrote a while back that accepts a SQL string, and optionally a list of parameters and it returns a DataTable.
Since the thing that changes between invocations is typically just the SQL that is optimal IMHO.
If you truly do need to use a DataReader you can do something like this:
public void ExecuteWithDataReader(string sql, Action<DataReader> stuffToDo) {
using (SqlConnection connection = new SqlConnection(connectionString)) {
using (SqlCommand command = new SqlCommand(sql, connection)) {
connection.Open();
using (SqlDataReader reader = command.ExecuteReader()) {
try {
while (reader.Read()) {
stuffToDo(reader);
}
}
finally {
reader.Close();
}
}
}
}
}
private static void ReadOrderData(string connectionString) {
string sql = "SELECT OrderID, CustomerID FROM dbo.Orders;";
ExecuteWithDataReader(sql, r => Console.WriteLine(String.Format("{0}, {1}", r[0], r[1])));
}
The first two line are the most important thing you need...
but if you still wish to do it, you can turn them to a database handler class, yes it will become more of code, but in refactoring concept, every thing will move to the related topic...
try to write a singleton class, that receive a command and do action, so return result of type SqlDataReader reader...
Doing this in comments was too much.
I would suggest that the boilerplate code around
using(conn = new sqlconnection)
using(cmd = new sqlcommand) {
// blah blah blah
}
isn't something to be lightly removed and instead would encourage that you keep it exactly where it's at. Resources, especially unmanaged ones, should be opened and released at the closest point to execution as possible IMHO.
In no small part due to the ease with which other developers will fail to follow the appropriate clean up conventions.
If you do something like
private SQLConnection CreateConnection()
{
var connection = new SqlConnection(_connectionString);
connection.Open();
return connection;
}
Then you are inviting another programmer to call this method and completely fail to release the resource as soon as the query is executed. I don't know what kind of app you are building, but in a web app such a thing will lead to memory / connection / resource errors of types that are difficult to debug, unless you've been through it before.
Instead, I'd suggest you look into a lightweight ORM such as Dapper.net or similar to see how they approached it. I don't use dapper, but I hear it's pretty good. The reason I don't use it is simply that we don't allow inline sql to be executed against our databases (but that's a very different conversation).
Here's our standard:
public static DataTable StatisticsGet( Guid tenantId ) {
DataTable result = new DataTable();
result.Locale = CultureInfo.CurrentCulture;
Database db = DatabaseFactory.CreateDatabase(DatabaseType.Clients.ToString());
using (DbCommand dbCommand = db.GetStoredProcCommand("reg.StatsGet")) {
db.AddInParameter(dbCommand, "TenantId", DbType.Guid, tenantId);
result.Load(db.ExecuteReader(dbCommand));
} // using dbCommand
return result;
} // method::StatisticsGet
We make heavy use of Enterprise Library. It's short, simple and to the point and very well tested. This method just returns a datatable but you could easily have it return an object collection.. or nothing.
I have a long-running service with several threads calling the following method hundreds of times per second:
void TheMethod()
{
using (var c = new SqlConnection("..."))
{
c.Open();
var ret1 = PrepareAndExecuteStatement1(c, args1);
// some code
var ret2 = PrepareAndExecuteStatement2(c, args2);
// more code
}
}
PrepareAndExecuteStatement is something like this:
void PrepareAndExecuteStatement*(SqlConnection c, args)
{
var cmd = new SqlCommand("query", c);
cmd.Parameters.Add("#param", type);
cmd.Prepare();
cmd.Parameters["#param"] = args;
return cmd.execute().read().etc();
}
I want reuse the prepared statements, preparing once per connection and executing them until the connection breaks. I hope this will improve performance.
Can I use the built-in connection pool to achieve this? Ideally every time a new connection is made, all statements should be automatically prepared, and I need to have access to the SqlCommand objects of these statements.
Suggest taking a slightly modified approach. Close your connection immedately after use. You can certainly re-use your SqlConnection.
The work being done at //some code may take a long time. Are you interacting with other network resources, disk resources, or spending any amount of time with calculations? Could you ever, in the future, need to do so? Perhaps the intervals between executing statement are/could be so long that you'd want to reopen that connection. Regardless, the Connection should be opened late and closed early.
using (var c = new SqlConnection("..."))
{
c.Open();
PrepareAndExecuteStatement1(c, args);
c.Close();
// some code
c.Open();
PrepareAndExecuteStatement2(c, args);
c.Close();
// more code
}
Open Late, Close Early as MSDN Magazine by John Papa.
Obviously we've now got a bunch of code duplication here. Consider refactoring your Prepare...() method to perform the opening and closing operations.
Perhaps you'd consider something like this:
using (var c = new SqlConnection("..."))
{
var cmd1 = PrepareAndCreateCommand(c, args);
// some code
var cmd2 = PrepareAndCreateCommand(c, args);
c.Open();
cmd1.ExecuteNonQuery();
cmd2.ExecuteNonQuery();
c.Close();
// more code
}