We Keep Coding

C# Parallel.For and Oracle database access - memory exception

I am working on some code that I would like to access an Oracle database inside of a Parallel.For loop. The loop will run for several minutes, and then result in the error:
"Attempted to read or write protected memory. This is often an
indication that other memory is corrupt."
There is no inner exception. Inside my Parallel.For loop, I am creating an opening the database connection as local objects. My code looks like this:
static void CheckSinglePath(Path p)
{
string sqlBase = "select * from table where hour = #HOUR#";
Parallel.For (1, 24, i =>
{
DBManager localdbm = new DBManager();
string sql = sqlBase;
sql = sql.Replace("#HOUR#", i.ToString());
OracleDataReader reader = db.GetData(sql);
if (reader.Read())
{
//do some stuff
}
reader.Close();
});
}
class DBManager
{
OracleConnection conn;
OracleCommand cmd;
public DBManager()
{
string connStr = "blahblahblah;Connection Timeout=600;";
conn = new OracleConnection(connStr);
conn.Open();
cmd = conn.CreateCommand();
}
public OracleDataReader GetData(string sql)
{
cmd.CommandText = sql;
return cmd.ExecuteReader();//EXCEPTION HERE!
}
}
What am I doing wrong? How can I create 24 parallel Oracle connections to process the data? I'm guessing there is some sort of race condition or memory leak that is going on here which I don't fully understand because it seems to be coming from inside the OracleConnection object. Is the database connection not threadsafe? I tried changing the connection string to use a connection pool and that didn't change anything.

Memory problems is always caused by wrong resources usage. You do not properly release your connections after the loop exit.
You need to implement IDisposable interface and after that you need rewrite your code in such manner with using keyword:
// dispose the connection after command finished
using (var localdbm = new DBManager())
{
var sql = sqlBase;
sql = sql.Replace("#HOUR#", i.ToString());
using (var reader = db.GetData(sql))
{
if (reader.Read())
{
//do some stuff
}
// no need to close reader
// as it's being disposed inside using directive
}
}

writing more optimized code and reusing code in C#

I'm writing a desktop application using C# winforms and MSSQL server 2012. there are several classes exit in this application that need to connect to database and all uisng ADO.Net. this is on of my classes :
class Prices
{
private int id = 0;
public int Id
{
get { return id; }
set { id = value; }
}
private string materialName = string.Empty;
......
......
......
public void updateMaterialPrice()
{
string conString = ConfigurationManager.ConnectionStrings["secaloFormulaCS"].ToString();
using (SqlConnection sqlCon = new SqlConnection(conString))
using (SqlCommand sqlCmd = new SqlCommand("spUpdateMaterialPrice", sqlCon))
{
sqlCmd.CommandType = CommandType.StoredProcedure;
sqlCmd.Parameters.AddWithValue("materialName",MaterialName);
sqlCmd.Parameters.AddWithValue("unitPrice", Price);
sqlCmd.Parameters.AddWithValue("carbohydrate", Carbohydrtate);
sqlCmd.Parameters.AddWithValue("protein", Proterin);
sqlCmd.Parameters.AddWithValue("fat", Fat);
sqlCmd.Parameters.AddWithValue("humidity", Humadity);
sqlCmd.Parameters.AddWithValue("minerlas", Minerlas);
sqlCon.Open();
sqlCmd.ExecuteNonQuery();
sqlCon.Close();
sqlCon.Dispose();
}
}
public void addMaterial()
{
string ConString = ConfigurationManager.ConnectionStrings["secaloFormulaCS"].ToString();
using(SqlConnection sqlCon = new SqlConnection(ConString))
using(SqlCommand sqlCmd = new SqlCommand("spAddMaterial",sqlCon))
{
sqlCmd.CommandType = CommandType.StoredProcedure;
sqlCmd.Parameters.AddWithValue("materialName", MaterialName);
sqlCmd.Parameters.AddWithValue("unitPrice",Price);
sqlCmd.Parameters.AddWithValue("carbohydrate",Carbohydrtate);
sqlCmd.Parameters.AddWithValue("proterin", Proterin);
sqlCmd.Parameters.AddWithValue("fat",Fat);
sqlCmd.Parameters.AddWithValue("humidity", Humadity);
sqlCmd.Parameters.AddWithValue("minerals",Minerlas);
sqlCon.Open();
sqlCmd.ExecuteNonQuery();
sqlCon.Close();
sqlCon.Dispose();
}
as you can see in addMterial() and updateMaterialPrice() i use the same code to connect to database and call a stored procedure and this is repeated for several times in my other classes. how can i prevent this code repetition ? is it any way to just write the code needed for connection and query the database one time and reuse it several times according to situation needed ?

I use a Factory pattern for my Database connections, this means I never have to open a SqlConnection or pass connection strings around my program.
Here is an example for the method I use to run a query that returns multiple rows.
I would call the method from a "makeObject" method that would turn this datatable into an object.
public static class DB
{
private static readonly string connectionString = ConfigurationManager.ConnectionStrings["MyConnectionString"].ConnectionString;
private static readonly DbProviderFactory factory = DbProviderFactories.GetFactory("System.Data.SqlClient");
/// <summary>
/// Use when returning data from multiple rows
/// </summary>
/// <param name="sql">query</param>
/// <param name="parameters">declared parameters</param>
/// <returns>datatable of db rows</returns>
public static DataTable GetDataTable(string sql, SqlParameter[] parameters)
{
try
{
using (DbConnection connection = factory.CreateConnection())
{
connection.ConnectionString = connectionString;
using (DbCommand command = factory.CreateCommand())
{
command.Connection = connection;
command.CommandType = CommandType.Text;
command.CommandText = sql;
if (parameters != null)
{
foreach (var parameter in parameters)
{
if (parameter != null)
command.Parameters.Add(parameter);
}
}
using (DbDataAdapter adapter = factory.CreateDataAdapter())
{
adapter.SelectCommand = command;
DataTable dt = new DataTable();
adapter.Fill(dt);
return dt;
}
}
}
}
catch (Exception)
{
throw;
}
}
}

Well, you could make a helper method to prepare the command for you, or at least fill the parameters, eg.
void PrepareParameters(SqlCommand cmd)
{
cmd.Parameters.AddWithValue("materialName",MaterialName);
cmd.Parameters.AddWithValue("unitPrice", Price);
cmd.Parameters.AddWithValue("carbohydrate", Carbohydrtate);
cmd.Parameters.AddWithValue("protein", Proterin);
cmd.Parameters.AddWithValue("fat", Fat);
cmd.Parameters.AddWithValue("humidity", Humadity);
cmd.Parameters.AddWithValue("minerlas", Minerlas);
}
Ideally, unless you want to use a ready ORM like Entity Framework (usually a good idea), you'd create a few abstract classes to handle these kinds of things, so that you'll save on code reuse.
For example, the PrepareParameters method could be abstract, and there could be an abstract property that returns the name of the SP to update, create or delete (or better yet, you could follow a naming scheme so that you'd only need one name). Then you could write 99% of the logic in the abstract base classes, and only prepare the parameters in the actual derived classes, thus cutting code repetition a lot.

Some options are as follows:
Write a SqlHelper class which does the repetitive grunt work of executing a stored procedure. (especially ExecuteNonQuery ones, so that you don't need to worry about return types.)
e.g.
public void ExecuteQuery(string sprocName, SqlParamater[] parameters)
{
// initialize connection
// construct command with sprocName and parameters
// ExecuteNonQuery
}
Use Linq2Sql
This is an excellent quick ORM tool which simplies data access to a great deal.
Use Entity Framework
This is an increasingly used ORM tool.
All of the above approaches have their pros/cons. You need to weight them & select the right method.

C# web service and static DB Connection

If I crate a static DB Connection into a C# Web service
This instance of connection is shared on all istance of web service? So, In this way could I have problem some this?
Not allowed to change the 'ConnectionString' property while the
connection (state=Connecting)
I get the instance in this way:
public static OleDbConnection GetDatabaseConnection(string aConnectionString)
{
if (_dbConnection == null)
{
_dbConnection = new OleDbConnection(aConnectionString);
_dbConnection.Open();
}
else
{
_dbConnection.Close();
_dbConnection = new OleDbConnection(aConnectionString);
_dbConnection.Open();
}
Log.Logger.log("Aperta connessione al DB");
return _dbConnection;
}

It is always a bad practice to keep a global connection instance. You should follow the standard pattern: create, open, use, close and destroy.
Working against this will result, sooner or later, in problems.
The Connection Pooling mechanism has been studied just for this.
The Pool will help you to avoid this bad habit to keep an object so expensive like a database connection open for unforeseeable time
public static OleDbConnection GetDatabaseConnection(string aConnectionString)
{
OleDbConnection odb = new OleDbConnection(aConnectionString);
odb.Open();
Log.Logger.log("Aperta connessione al DB");
return odb;
}
then your client code use the using statement to close and destroy the connection
using(OleDbConnection cn = SqlClassHelper.GetDatabaseConnection(constring))
{
// define here your OleDbCommand, OleDbDataReader etc...
// use the objects
} // <- here the closing brace close and destroy the connection

Yes, you will have problems with your code in a multithreaded web service.
Your function GetDatabaseConnection works like a factory, everybody who calls it gets a new instance of connection. You do not need static _dbConnection class member, just make it a local variable:
public static OleDbConnection GetDatabaseConnection(string aConnectionString)
{
OleDbConnection _dbConnection = new OleDbConnection(aConnectionString);
_dbConnection.Open();
Log.Logger.log("Aperta connessione al DB");
return _dbConnection;
}

What's the most DRY-appropriate way to execute an SQL command?

I'm looking to figure out the best way to execute a database query using the least amount of boilerplate code. The method suggested in the SqlCommand documentation:
private static void ReadOrderData(string connectionString)
{
string queryString = "SELECT OrderID, CustomerID FROM dbo.Orders;";
using (SqlConnection connection = new SqlConnection(connectionString))
{
SqlCommand command = new SqlCommand(queryString, connection);
connection.Open();
SqlDataReader reader = command.ExecuteReader();
try
{
while (reader.Read())
{
Console.WriteLine(String.Format("{0}, {1}", reader[0], reader[1]));
}
}
finally
{
reader.Close();
}
}
}
mostly consists of code that would have to be repeated in every method that interacts with the database.
I'm already in the habit of factoring out the establishment of a connection, which would yield code more like the following. (I'm also modifying it so that it returns data, in order to make the example a bit less trivial.)
private SQLConnection CreateConnection()
{
var connection = new SqlConnection(_connectionString);
connection.Open();
return connection;
}
private List<int> ReadOrderData()
{
using(var connection = CreateConnection())
using(var command = connection.CreateCommand())
{
command.CommandText = "SELECT OrderID FROM dbo.Orders;";
using(var reader = command.ExecuteReader())
{
var results = new List<int>();
while(reader.Read()) results.Add(reader.GetInt32(0));
return results;
}
}
}
That's an improvement, but there's still enough boilerplate to nag at me. Can this be reduced further? In particular, I'd like to do something about the first two lines of the procedure. I don't feel like the method should be in charge of creating the SqlCommand. It's a tiny piece of repetition as it is in the example, but it seems to grow if transactions are being managed manually or timeouts are being altered or anything like that.
edit: Assume, at least hypothetically, there's going to have to be a bunch of different types of data being returned. And consequently the solution can't be just one one-size-fits-all method, there will have to be a few different ones depending, at minimum, on whether ExecuteNonQuery, ExecuteScalar, ExecuteReader, ExecuteReaderAsync, or any of the others are being called. I'd like to cut down on the repetition among those.

Tried Dapper?
Granted this doesn't get you a DataReader but you might just prefer it this way once you've tried it.
It's about the lightest-weight an ORM can be while still being called an ORM. No more methods to map between DataReader and strong types for me.
Used right here on all the StackExchange sites.
using (var conn = new SqlConnection(cs))
{
var dogs = connection.Query("select name, age from dogs");
foreach (dynamic dog in dogs)
{
Console.WriteLine("{0} age {1}", dog.name, dog.age);
}
}
or
using (var conn = new SqlConnection(cs))
{
var dogs = connection.Query<Dog>("select Name, Age from dogs");
foreach (Dog dog in dogs)
{
Console.WriteLine("{0} age {1}", dog.Name, dog.Age);
}
}
class Dog
{
public string Name { get; set; }
public int Age { get; set; }
}

If you want to roll data access on your own, this pattern of help methods could be one way to remove duplication:
private List<int> ReadOrderData()
{
return ExecuteList<int>("SELECT OrderID FROM dbo.Orders;",
x => x.GetInt32("orderId")).ToList();
}
private IEnumerable<T> ExecuteList(string query,
Func<IDataRecord, T> entityCreator)
{
using(var connection = CreateConnection())
using(var command = connection.CreateCommand())
{
command.CommandText = query;
connection.Open();
using(var reader = command.ExecuteReader())
{
while(reader.Read())
yield return entityCreator(reader);
}
}
}
You'll have to add support for parameters and this might not compile, but the pattern is what I'm trying to illustrate.

What I typically do is use a custom class that I wrote a while back that accepts a SQL string, and optionally a list of parameters and it returns a DataTable.
Since the thing that changes between invocations is typically just the SQL that is optimal IMHO.
If you truly do need to use a DataReader you can do something like this:
public void ExecuteWithDataReader(string sql, Action<DataReader> stuffToDo) {
using (SqlConnection connection = new SqlConnection(connectionString)) {
using (SqlCommand command = new SqlCommand(sql, connection)) {
connection.Open();
using (SqlDataReader reader = command.ExecuteReader()) {
try {
while (reader.Read()) {
stuffToDo(reader);
}
}
finally {
reader.Close();
}
}
}
}
}
private static void ReadOrderData(string connectionString) {
string sql = "SELECT OrderID, CustomerID FROM dbo.Orders;";
ExecuteWithDataReader(sql, r => Console.WriteLine(String.Format("{0}, {1}", r[0], r[1])));
}

The first two line are the most important thing you need...
but if you still wish to do it, you can turn them to a database handler class, yes it will become more of code, but in refactoring concept, every thing will move to the related topic...
try to write a singleton class, that receive a command and do action, so return result of type SqlDataReader reader...

Doing this in comments was too much.
I would suggest that the boilerplate code around
using(conn = new sqlconnection)
using(cmd = new sqlcommand) {
// blah blah blah
}
isn't something to be lightly removed and instead would encourage that you keep it exactly where it's at. Resources, especially unmanaged ones, should be opened and released at the closest point to execution as possible IMHO.
In no small part due to the ease with which other developers will fail to follow the appropriate clean up conventions.
If you do something like
private SQLConnection CreateConnection()
{
var connection = new SqlConnection(_connectionString);
connection.Open();
return connection;
}
Then you are inviting another programmer to call this method and completely fail to release the resource as soon as the query is executed. I don't know what kind of app you are building, but in a web app such a thing will lead to memory / connection / resource errors of types that are difficult to debug, unless you've been through it before.
Instead, I'd suggest you look into a lightweight ORM such as Dapper.net or similar to see how they approached it. I don't use dapper, but I hear it's pretty good. The reason I don't use it is simply that we don't allow inline sql to be executed against our databases (but that's a very different conversation).
Here's our standard:
public static DataTable StatisticsGet( Guid tenantId ) {
DataTable result = new DataTable();
result.Locale = CultureInfo.CurrentCulture;
Database db = DatabaseFactory.CreateDatabase(DatabaseType.Clients.ToString());
using (DbCommand dbCommand = db.GetStoredProcCommand("reg.StatsGet")) {
db.AddInParameter(dbCommand, "TenantId", DbType.Guid, tenantId);
result.Load(db.ExecuteReader(dbCommand));
} // using dbCommand
return result;
} // method::StatisticsGet
We make heavy use of Enterprise Library. It's short, simple and to the point and very well tested. This method just returns a datatable but you could easily have it return an object collection.. or nothing.

Which pattern is better for SqlConnection object?

Which pattern is better for SqlConnection object? Which is better in performance?
Do you offer any other pattern?
class DataAccess1 : IDisposable
{
private SqlConnection connection;
public DataAccess1(string connectionString)
{
connection = new SqlConnection(connectionString);
}
public void Execute(string query)
{
using (SqlCommand command = connection.CreateCommand())
{
command.CommandText = query;
command.CommandType = CommandType.Text;
// ...
command.Connection.Open();
command.ExecuteNonQuery();
command.Connection.Close();
}
}
public void Dispose()
{
connection.Dispose();
}
}
VS
class DataAccess2 : IDisposable
{
private string connectionString;
public DataAccess2(string connectionString)
{
this.connectionString = connectionString;
}
public void Execute(string query)
{
using (SqlConnection connection = new SqlConnection(connectionString))
{
SqlCommand command = connection.CreateCommand();
command.CommandText = query;
command.CommandType = CommandType.Text;
// ...
command.Connection.Open();
command.ExecuteNonQuery();
command.Connection.Close();
}
}
public void Dispose()
{
}
}

There's no real way to answer this question. The short, canonical answer is that the connection should stay alive for the lifetime of your unit of work. Because we have no way of knowing how DataAccess is used (does it exist for the lifetime of your application, or do you instantiate it and dispose it whenever you do something?), it's impossible to give a concrete answer.
That being said, I would recommend the first pattern, but instantiate and dispose of your DataAccess object as needed; don't keep it around longer than necessary.

Suggest going with DataAccess2. It's a personal preference though. Some might even suggest your class be static. It'd be difficult to say that one is more performant than the other. You're on the path of IDisposable, which is great.
I'd be happy to read and maintain both styles shown above in your question.
Consider having your DAL be able to read the connection string from a .config as well, rather than exclusively allowing the value to be passed in the constructor.
public DataAccess2(string connStr)
{
this.connectionString = connStr;
}
public DataAccess2()
{
this.connectionString =
ConfigurationManager.ConnectionStrings["foo"].ConnectionString;
}
Consider wrapping your SqlCommand in a using as well.
using (var conn = new SqlConnection(connectionString))
{
using(var cmd = conn.CreateCommand())
{
}
}

I think it depends on how your DataAccess object is intended to be used, if it's used within a 'using' clause then the connection is guaranteed to be disposed of after it's done.
But in general I prefer the second pattern as the sql connection is created and disposed of within the Execute method so it's less likely to be left open when you forget to dispose of your DataAccess object.
Considering that sql connection can be a scarse resource I think every attempt should be made to ensure that they're not wasted.

The first will result in errors if you make concurrent calls.
The second will ensure you use a clean connection for each command resulting in more connections being made.
I agree with the statements above that it depends on the scenario for use, to get over the problem related to the first I have a wrapper that needs to use such a pattern so I set a field value boolean to show that a command is being executed on the connection already then "queue" the next command for execution.
There will of course be situations where you may prefer to use multiple connections ...