We Keep Coding

ORA-00604, ORA-01000 errors in C# with Oracle; max open cursors exceeded

Hi all: I have a program that is running 4 threads that talk to an Oracle database. I have the Oracle connections local to each thread, and I'm employing the USING statement as well as manually closing the recordset and closing the connection. As I understand it, the ORA-01000 error arises when there are more open recordsets than configured cursors on the database. I do not understand why my recordsets are staying open or why I'm getting this error. Here's the code:
static void CheckPaths()
{
int pathcount = paths.Count; //paths is a typed list
Parallel.ForEach(paths, new ParallelOptions { MaxDegreeOfParallelism = 4 }, (p) =>
{
try
{
CheckSinglePathAllHours(p);
}
catch (Exception ex)
{
//there is logging here, this is where the exception hits
}
});
}
static void CheckSinglePathAllHours(Path p)
{
string sqlBase = #"Select * from table ";//this is actually a big SQL statement
using (DBManager localdbm = new DBManager())
{
string sql = sqlBase;
OracleDataReader reader = localdbm.GetData(sql);
while (reader.Read())
{
//process the path, query always returns 24 or less rows
}
reader.Close();
reader = null; //is this even necessary?
localdbm.Close(); //is this necessary in conjunction with the USING statement?
}
}
class DBManager : IDisposable
{
OracleConnection conn;
OracleCommand cmd;
public DBManager()
{
string connStr = "blah blah blah";
conn = new OracleConnection(connStr);
conn.Open();
cmd = conn.CreateCommand();
}
public OracleDataReader GetData(string sql)
{
cmd.CommandText = sql;
cmd.CommandTimeout = 900;
return cmd.ExecuteReader();
}
public void RunSQL(string sql)
{
cmd.CommandText = sql;
cmd.CommandTimeout = 900;
cmd.ExecuteNonQuery();
}
public void Close()
{
conn.Close();
}
public void Dispose()
{
try
{
conn.Close();
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
}
}
The code will usually run for about a minute or two before the exception. The exception message is two-fold: ORA-00604: error occured at recursive SQL level 1; and ORA-01000: maximum open cursors exceeded. Any ideas what I'm doing wrong?

Changed the code to call .Dispose() on OracleDataReader and OracleConnection as suggested by Paul Abbott. Also increased the number of cursors per session from 50 to 150 on the database.

Bulk Copy C# to PostgreSql

I have a data table with 1000 of records. I want to perform a bulk insert operation from C# to PGSQL. I know one way which copies a text file to pgtable.
The example syntax is as follows:
using (NpgsqlConnection conn = new NpgsqlConnection("Server=127.0.0.1;User Id=postgres;Password=postgres;Database=postgres;"))
{
onn.Open();
NpgsqlCommand command = new NpgsqlCommand("copy \"schema\".\"tablename\" (\"col1\",\"col2\") from 'C:\\datafile.txt'", conn);
command.ExecuteNonQuery();
conn.Close();
}
Is there any other function which I can use to pass the data table instead of writing data into a text file? I m using Npgsql.dll.

You should probably fully read the PostgreSQL docs for COPY.
COPY can either be used to import a file that exists in the PostgreSQL server filesystem (as your code sample shows), or it can be used to copy data from the client, which is probably what you're looking for. The latter is triggered by substituting STDIN for the filename.
If you want to import data from your client program using Npgsql, please read the Npgsql COPY docs as well. For textual data import you'll likely need to call NpgsqlConnection.BeginTextImport(), there's a sample for that in the docs.

public bool CopyFileToPostgress(String tableName, String filePath,String delimiter)
{
NpgsqlConnection conn = new NpgsqlConnection("Host=xx.xx.xx.xx;port=xxxx;Database=xxxx;Username=xxxx;Password=xxxx;Pooling=false;Timeout=300;CommandTimeout=300");
NpgsqlCommand cmd = new NpgsqlCommand();
Boolean result = true;
try
{
conn.Open();
NpgsqlTransaction transaction = conn.BeginTransaction();
if (File.Exists(filePath))
{
try
{
NpgsqlCommand command = new NpgsqlCommand($"COPY {tableName} FROM '{filePath}' (DELIMITER '{delimiter}')", conn);
command.ExecuteNonQuery();
}
catch (Exception e)
{
result = false;
transaction.Rollback();
throw e;
}
finally
{
if (result)
{
transaction.Commit();
}
transaction.Dispose();
}
}
}
catch (Exception ex)
{
result = false;
}
finally
{
cmd.Dispose();
conn.Close();
conn.Dispose();
}
return result;
}

How to copy MySqlDataReader into an array and then loop throught the array?

I am new to C# so yes this should be a faily easy question but I can't seem to find the answer to it.
I have a method that query a database.
What I am trying to do here is handle the loop though the data outside the method.
public MySqlDataReader getDataSet(string query)
{
MySqlDataReader dataset = null;
MySqlConnection conn = new MySqlConnection(conn_string);
if (startConnection(conn) == true)
{
MySqlCommand cmd = new MySqlCommand(query, conn);
dataset = cmd.ExecuteReader();
closeConnection(conn);
}
return dataset;
}
what I could do is write a while loop just before the closeConnection(conn); line and handle the data. But, I don't want to do it inside this method and I want to do it somewhere else in my code.
In one of my forms I want to read the database on the load so here is what I tried to do
public newDepartment()
{
InitializeComponent();
inputDepartmentName.Text = "Hi";
dbConnetion db = new dbConnetion();
MySqlDataReader ds = db.getDataSet("SELECT name FROM test;");
while (ds.Read())
{
//Do Something
}
}
The problem that I am having is that I get an error Invalid attempt to Read when reader is closed
Which I belive I get this issue because I close the connection and then I am trying to read it. so What I need to do is read the data from the query and put it in an array and then loop through the array and deal with the data in a different form.
How can I workaround this issue? if my idea is good then how can I copy the data into an array and how do I loop though the array?
Here is the full class
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using MySql.Data.MySqlClient;
using System.Windows.Forms;
namespace POS
{
public class dbConnetion
{
//private OdbcConnection conn;
private readonly string mServer;
private readonly string mDatabase;
private readonly string mUid;
private readonly string mPassword;
private readonly string mPort;
private readonly string conn_string;
public dbConnetion()
{
mServer = "localhost";
mDatabase = "pos";
mUid = "root";
mPassword = "";
mPort = "3306";
conn_string = String.Format("server={0};user={1};database={2};port={3};password={4};", mServer, mUid, mDatabase, mPort, mPassword);
}
//Start connection to database
private bool startConnection(MySqlConnection mConnection)
{
try
{
mConnection.Open();
return true;
}
catch (MySqlException ex)
{
MessageBox.Show(ex.Message, "Error", MessageBoxButtons.OK);
return false;
}
}
//Close connection
private bool closeConnection(MySqlConnection mConnection)
{
try
{
mConnection.Close();
return true;
}
catch (MySqlException ex)
{
MessageBox.Show(ex.Message);
return false;
}
}
public MySqlDataReader getDataSet(string query)
{
MySqlDataReader dataset = null;
MySqlConnection conn = new MySqlConnection(conn_string);
if (startConnection(conn) == true)
{
MySqlCommand cmd = new MySqlCommand(query, conn);
dataset = cmd.ExecuteReader();
closeConnection(conn);
}
return dataset;
}
public void processQuery(string strSQL, List<MySqlParameter> pars)
{
MySqlConnection conn = new MySqlConnection(conn_string);
if (startConnection(conn) == true)
{
MySqlCommand cmd = new MySqlCommand(strSQL, conn);
foreach (MySqlParameter param in pars)
{
cmd.Parameters.Add(param);
}
cmd.ExecuteNonQuery();
closeConnection(conn);
}
}
}
}

Putting the records into an array would destroy the best feature of a using a datareader: that you only need to allocate memory for one record at a time. Try doing something like this:
public IEnumerable<T> getData<T>(string query, Func<IDataRecord, T> transform)
{
using (var conn = new MySqlConnection(conn_string))
using (var cmd = new MySqlCommand(query, conn))
{
conn.Open();
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
yield return transform(rdr);
}
}
}
}
While I'm here, there's a very serious security flaw with this code and the original. A method like this that only accepts a query string, with no separate mechanism for parameters, forces you to write code that will be horribly horribly vulnerable to sql injection attacks. The processQuery() method already accounts for this, so let's extend getDataset() to avoid that security issue as well:
public IEnumerable<T> getData<T>(string query, List<MySqlParameter> pars, Func<IDataRecord, T> transform)
{
using (var conn = new MySqlConnection(conn_string))
using (var cmd = new MySqlCommand(query, conn))
{
if (pars != null)
{
foreach(MySqlParameter p in pars) cmd.Parameters.Add(p);
}
conn.Open();
using (var rdr = cmd.ExecuteReader())
{
while (rdr.Read())
{
yield return transform(rdr);
}
}
}
}
Much better. Now we don't have to write code that's just asking to get hacked anymore. Here's how your newDepartment() method will look now:
public newDepartment()
{
InitializeComponent();
inputDepartmentName.Text = "Hi";
dbConnetion db = new dbConnetion();
foreach(string name in db.getDataSet("SELECT name FROM test;", null, r => r["name"].ToString() ))
{
//Do Something
}
}
One thing about this code is that is uses a delegate to have you provide a method to create a strongly-typed object. It does this because of the way the datareaders work: if you don't create a new object at each iteration, you're working on the same object, which can have undesirable results. In this case, I don't know what kind of object you're working with, so I just used a string based on what your SELECT query was doing.
Based on a separate discussion, here's an example of calling this for a more complicated result set:
foreach(var item in db.getDataSet(" long query here ", null, r =>
new columnClass()
{
firstname = r["firstname"].ToString(),
lastname = r["lastname"].ToString(),
//...
}
) )
{
//Do something
}

Since you are new to .Net I thought I point out that there are two layers of database access in ADO.Net. There are the data reader way that you are using and all of that is online only forward reading of queries. This is the lowest level access and will give you the best performance but it is more work. For most connection types you can only execute one command or have one active data reader per connection (And you can't close the connection before you have read the query as you are doing).
The other form is the offline data adapter and requires just a little bit different code, but is generally easier to use.
public DataTable getDataSet(string query)
{
MySqlConnection conn = new MySqlConnection(conn_string);
if (startConnection(conn) == true)
{
MySqlDataAdapter adapter = new MySqlDataAdapter(query, conn);
DataTable table = new DataTable();
adapter.Fill(table);
closeConnection(conn);
return table;
}
return null;
}
This will result in you getting a DataTable with columns and rows corresponding to the result of your query (Also look into command builders if you want to post changes back to the database later on from it, but for that you will need to keep the connection open).
One nice thing with using the data adapter is that it will figure out what the correct data types should be so you don't have to worry about invalid cast exceptions while reading the data from the data reader.
As somebody pointed out though you will need to read all the data into memory which could be a problem if you are dealing with a lot of memory. Also the DataTable class is really slow when you start dealing with a lot of records. Finally DataTable and DataSet classes also generally hook well into UI components in .Net so that their contents can easily be displayed to users.

C# Returning Data Objects Issue

I am trying to return a data object from my database so that i can access (for example) a customer ID within my ASP.NET website. Upon a customer logging in the object is returned. However, i am getting the error:
'Invalid attempt to read when no data is present.'
I have completed an sql query on the database (Executing my stored procedure) which returns the correct information, so i know it is there. I can only presume that there is something wrong with the following method:
using (SqlConnection sqlConn = new SqlConnection(ConfigurationManager.ConnectionStrings["ConnectionString"].ConnectionString))
{
using (SqlCommand sqlComm = new SqlCommand("Select_Customer_By_UserName_And_Password", sqlConn))
{
sqlComm.Connection.Open();
try
{
sqlComm.CommandType = CommandType.StoredProcedure;
sqlComm.Parameters.Add("#Username", SqlDbType.NVarChar, 25).Value = pUsername;
sqlComm.Parameters.Add("#Password", SqlDbType.NVarChar, 25).Value = pPassword;
using (SqlDataReader sqlDR = sqlComm.ExecuteReader(CommandBehavior.SingleRow))
{
if (sqlDR.HasRows)
{
//Creating the new object to be returned by using the data from the database.
return new Customer
{
CustomerID = Convert.ToInt32(sqlDR["CustomerID"])
};
}
else
return null;
}
}
catch (Exception)
{
throw;
}
finally
{
sqlComm.Connection.Close();
}
}
}

You need to call sqlDR.Read(), otherwise the "record pointer" will to point to a record. HasRows only indicates there are actually rows you can read. To read each row (or just the first one), you need to call Read once or in a while loop.
For example:
if (reader.HasRows)
{
while (reader.Read())
...
}
Your code should read:
using (SqlDataReader sqlDR = sqlComm.ExecuteReader(CommandBehavior.SingleRow))
{
if (sqlDR.Read())
{
//Creating the new object to be returned by using the data from the database.
return new Customer
{
CustomerID = Convert.ToInt32(sqlDR["CustomerID"])
};
}
else
return null;
}
By the way: congrats on using using and parameterized queries!

Storing Config Values

Wondering if I am going about this the right way. I am creating a C# application that loads a few variables from my App.Config.xml file. I am loading these into a "config" class (Config.cs) along with other variables that I am loading from a MySQL database. This is what my class looks like so far:
class Config
{
public static string ServerHostname = ConfigurationManager.AppSettings["ServerHostname"];
public static string SoftwareVersion = "v0.1a";
public static int StationID = DBConnector.GetStationID();
public static string StationDescription = DBConnector.GetStationDescription();
public static string StationName = ConfigurationManager.AppSettings["StationName"];
}
I am using Config.StationName to pull the Config.StationID and Config.StationDescription from a MySQL database like this in DBConnector.cs:
public static int GetStationID()
{
try
{
MySqlConnection conn = new MySqlConnection(connStr);
conn.Open();
string sql = "select station_id from station_master where station_name = #station_name limit 1";
MySqlCommand cmd = new MySqlCommand(sql, conn);
cmd.Parameters.AddWithValue("#station_name", Config.StationName);
object result = cmd.ExecuteScalar();
conn.Close();
return Convert.ToInt32(result);
}
catch (Exception ex)
{
ErrorConnection += ex.ToString();
return 0;
}
}
public static string GetStationDescription()
{
try
{
MySqlConnection conn = new MySqlConnection(connStr);
conn.Open();
string sql = "select station_description from station_master where station_id = '" + Config.StationID +"' limit 1";
MySqlCommand cmd = new MySqlCommand(sql, conn);
// cmd.Parameters.AddWithValue("#station_name", Config.StationName.ToString());
object result = cmd.ExecuteScalar();
conn.Close();
MessageBox.Show(sql);
return (result.ToString());
}
catch (Exception ex)
{
ErrorGenericDBException += ex.ToString();
MessageBox.Show(ErrorGenericDBException);
return "Error";
}
}
The DBConnector.GetStationID class works fine. It returns the int value of my station. But when I try to display the Config.StationDescription, it throws an exception of System.NullReferenceException: Object reference not set to an instance of an object at Namespace.DBConnector.GetStationDescription().
I thought that since I was using a static class for Config.StationName, i don't have to create an instance to refer to it. I need help to understand why its throwing an exception.

To me it looks like you might be mixing up what part of the code has the problem. Since you're saying that it's GetSTationDescription that throws, I'm not sure why you then assume StationName has a problem?
I'd take a look at the line object result = cmd.ExecuteScalar();, if you put a breakpoint on the line after that, does result have a value?
Another issue is that you're currently only closing the connection if an exception is not thrown, it would be safer to have the conn.Close(); inside a finally statement (or use a using statement so you don't have to worry about closing it).