We Keep Coding

C# Windows Form using OracleDataAdapter to insert rows to a Oracle Table from a System DataTable

I'm writing a windows form that populates a DataTable and I want it to insert into an Oracle Table. I've seen some examples here that use the OracleDataAdapter to do this so I don't have to loop through all the records. The code doesn't have any errors but when I check the Table using Toad(I did refresh) I don't see it. I used the example below
Update and insert records into Oracle table using OracleDataAdapter from DataTable
Here is how my DataTable is made:
public DataTable dtMain = new DataTable();
public void FillTable(DataTable dt)
{
dtMain.Columns.Add("SERIAL", typeof(System.String));
dtMain.Columns.Add("LOCATION", typeof(System.String));
dtMain.Columns.Add("UPC", typeof(System.String));
dtMain.Columns.Add("PRODUCT", typeof(System.String));
dtMain.Columns.Add("CREATED_BY", typeof(System.String));
dtMain.Columns.Add("CREATED_DATE", typeof(System.DateTime));
dtMain.Columns.Add("SKU", typeof(System.String));
dtMain.Columns.Add("MAN_DATE", typeof(System.DateTime));
dtUpload.Columns[0].Unique = true;
dtMain.Merge(dt);
}
This is the how I'm trying to insert into the database
private void btnUpload_Click(object sender, EventArgs e)
{
DataTable dt = new DataTable();
string strSelect = "SELECT serial, upc, man_date, location, product, created_by, created_date, serial from schema.table where rownum < 2";
string strInsert = "INSERT INTO schema.table (serial, upc, man_date, location, product, created_by, created_date, serial) VALUES (:serial, :upc, :man_date, :location, :product, :created_by, :created_date, :serial)";
string conStr = ConfigurationManager.ConnectionStrings["connection"].ConnectionString;
OracleConnection connection = new OracleConnection(conStr);
connection.Open();
if (connection.State != ConnectionState.Open)
{
return;
}
try
{
OracleDataAdapter adapterS = new OracleDataAdapter();
adapterS.SelectCommand = new OracleCommand(strSelect, connection);
adapterS.Fill(dt);
dt.Rows.Remove(dt.Rows[0]);
dt.Merge(dtUpload);
}
catch (Exception ex)
{
string x = ex.Message + ex.StackTrace;
throw;
}
for (int i = 0; dt.Rows.Count > i; i++)
{
OracleDataAdapter adapter = new OracleDataAdapter();
adapter.InsertCommand = new OracleCommand(strInsert, connection);
adapter.InsertCommand.BindByName = true;
OracleParameter pSerial = new OracleParameter(":serial", OracleDbType.Varchar2);
pSerial.SourceColumn = dt.Columns[0].ColumnName;
pSerial.Value = dtUpload.Rows[i][0];
OracleParameter pLocation = new OracleParameter(":location", OracleDbType.Varchar2);
pLocation.SourceColumn = dt.Columns[1].ColumnName;
pLocation.Value = dtUpload.Rows[i][1];
OracleParameter pUPC = new OracleParameter(":upc", OracleDbType.Date);
pUPC.SourceColumn = dt.Columns[2].ColumnName;
pUPC.Value = dtUpload.Rows[i][2];
OracleParameter pProduct = new OracleParameter(":product", OracleDbType.Varchar2);
pProduct.SourceColumn = dt.Columns[3].ColumnName;
pProduct.Value = dtUpload.Rows[i][3];
OracleParameter pCreatedBy = new OracleParameter(":created_by", OracleDbType.Varchar2);
pCreatedBy.SourceColumn = dt.Columns[4].ColumnName;
pCreatedBy.Value = dtUpload.Rows[i][4];
OracleParameter pCreatedDate = new OracleParameter(":created_date", OracleDbType.Varchar2);
pCreatedDate.SourceColumn = dt.Columns[5].ColumnName;
pCreatedDate.Value = dtUpload.Rows[i][5];
OracleParameter pSKU = new OracleParameter(":SKU", OracleDbType.Date);
pSKU.SourceColumn = dt.Columns[6].ColumnName;
pSKU.Value = dtUpload.Rows[i][6];
OracleParameter pManDate = new OracleParameter(":man_date", OracleDbType.Varchar2);
pManDate.SourceColumn = dt.Columns[7].ColumnName;
pManDate.Value = dtUpload.Rows[i][7];
adapter.InsertCommand.Parameters.Add(pSerial);
adapter.InsertCommand.Parameters.Add(pLocation);
adapter.InsertCommand.Parameters.Add(pUPC);
adapter.InsertCommand.Parameters.Add(pProduct);
adapter.InsertCommand.Parameters.Add(pCreatedBy);
adapter.InsertCommand.Parameters.Add(pCreatedDate);
adapter.InsertCommand.Parameters.Add(pserial);
adapter.InsertCommand.Parameters.Add(pManDate);
}
try
{
adapter.Update(dt);
}
catch (Exception ex)
{
string x = ex.Message + ex.StackTrace;
throw;
}
connection.Close();
connection.Dispose();
}
If someone can give me some pointers that would be great, I've been Googling for 2 days and I just can't figure it out. I bet it's something simple
Update:
Thank you for the reply, it took me a bit to get back to this project. When I posted this I didn't realize I forgot to include my select statement.
For the OracleParameter value, I thought using SourceColumn would use that column for the values.
I did update the DataTable with the serial being unique. It still doesn't insert the data. If I included the Parameter.value would I have loop row by row to do this? Above I corrected/updated it with the current code.
Second Update:
Ok, I tried looping through the parameters to add the values from the DataTable, no errors but still not inserting in the database. I know my connectionstring is correct because the select query works. The code above has been updated for the changes I made. If some Oracle guru can shed some light on my problem a virtual high-five is waiting for them.

I never used OracleDataAdapter but I see these possible issues:
I don't think you can use OracleDataAdapter straight away, first you have to select an existing table from Oracle and based on this result you can do Update/Insert/Delete on it. Where is your strSelect string? I don't see it.
You created all the OracleParameter but I don't see that you assign any value to it, they are all empty. I would expect something like pOptoroLP.Value = ...; before you make any insert.
You should define a unique, resp. Primary Key column like dt.Columns["SERIAL"].Unique = true; in order to make Update() or Delete(). Maybe it is not required for Insert(), I don't know.
Did you follow the examples in Data Provider for .NET Developer's Guide

I finally found the problem, the row state in the DataTable needs to in the state of Added for it to work. If someone else has this problem here is my new code and you can compare it to what I was trying to do. Thank you Wernfried Domscheit for the Oracle part but it turned out to be my DataTable was the issue though setting the Unique column was probably an issue as well, I didn't check. The OracleCommandBuilder took care of the insert statements and parameters for me.
private void btnUpload_Click(object sender, EventArgs e)
{
DataTable dt = new DataTable();
string strSelect = "SELECT serial, upc, man_date, location, product, created_by, created_date, serial from schema.table where rownum < 2";
string strInsert = "INSERT INTO schema.table (serial, upc, man_date, location, product, created_by, created_date, serial) VALUES (:serial, :upc, :man_date, :location, :product, :created_by, :created_date, :serial)";
string conStr = ConfigurationManager.ConnectionStrings["connection"].ConnectionString;
OracleConnection connection = new OracleConnection(conStr);
connection.Open();
if (connection.State != ConnectionState.Open)
{
return;
}
OracleDataAdapter adapter = new OracleDataAdapter(strSelect, conStr);
OracleCommandBuilder builder = new OracleCommandBuilder(adapter);
adapter.Fill(dt);
dt.Columns["SERIAL"].Unique = true;
dt.Merge(dtUpload);
dt.Rows.Remove(dt.Rows[0]);
foreach (DataRow row in dt.Rows)
{
row.SetAdded();
}
try
{
adapter.Update(dt);
}
catch (Exception ex)
{
string x = ex.Message + ex.StackTrace;
throw;
}
finally
{
connection.Close();
connection.Dispose();
}
}

How can I make the MySql insert more efficient/faster? [duplicate]

I am migrating my program from Microsoft SQL Server to MySQL. Everything works well except one issue with bulk copy.
In the solution with MS SQL the code looks like this:
connection.Open();
SqlBulkCopy bulkCopy = new SqlBulkCopy(connection);
bulkCopy.DestinationTableName = "testTable";
bulkCopy.WriteToServer(rawData);
Now I try to do something similar for MySQL. Because I think there would be bad performance I don't want to write the DataTable to a CSV file and do the insert from there with the MySqlBulkLoader class.
Any help would be highly appreciated.

Because I think there would be bad performance I don't want to write the DataTable to a CSV file and do the insert from there with the MySqlBulkLoader class.
Don't rule out a possible solution based on unfounded assumptions. I just tested the insertion of 100,000 rows from a System.Data.DataTable into a MySQL table using a standard MySqlDataAdapter#Update() inside a Transaction. It consistently took about 30 seconds to run:
using (MySqlTransaction tran = conn.BeginTransaction(System.Data.IsolationLevel.Serializable))
{
using (MySqlCommand cmd = new MySqlCommand())
{
cmd.Connection = conn;
cmd.Transaction = tran;
cmd.CommandText = "SELECT * FROM testtable";
using (MySqlDataAdapter da = new MySqlDataAdapter(cmd))
{
da.UpdateBatchSize = 1000;
using (MySqlCommandBuilder cb = new MySqlCommandBuilder(da))
{
da.Update(rawData);
tran.Commit();
}
}
}
}
(I tried a couple of different values for UpdateBatchSize but they didn't seem to have a significant impact on the elapsed time.)
By contrast, the following code using MySqlBulkLoader took only 5 or 6 seconds to run ...
string tempCsvFileSpec = #"C:\Users\Gord\Desktop\dump.csv";
using (StreamWriter writer = new StreamWriter(tempCsvFileSpec))
{
Rfc4180Writer.WriteDataTable(rawData, writer, false);
}
var msbl = new MySqlBulkLoader(conn);
msbl.TableName = "testtable";
msbl.FileName = tempCsvFileSpec;
msbl.FieldTerminator = ",";
msbl.FieldQuotationCharacter = '"';
msbl.Load();
System.IO.File.Delete(tempCsvFileSpec);
... including the time to dump the 100,000 rows from the DataTable to a temporary CSV file (using code similar to this), bulk-loading from that file, and deleting the file afterwards.

Similar to SqlBulkCopy, we have MySqlBulkCopy for Mysql.
here is the example how to use it.
public async Task<bool> MySqlBulCopyAsync(DataTable dataTable)
{
try
{
bool result = true;
using (var connection = new MySqlConnector.MySqlConnection(_connString + ";AllowLoadLocalInfile=True"))
{
await connection.OpenAsync();
var bulkCopy = new MySqlBulkCopy(connection);
bulkCopy.DestinationTableName = "yourtable";
// the column mapping is required if you have a identity column in the table
bulkCopy.ColumnMappings.AddRange(GetMySqlColumnMapping(dataTable));
await bulkCopy.WriteToServerAsync(dataTable);
return result;
}
}
catch (Exception ex)
{
throw;
}
}
private List<MySqlBulkCopyColumnMapping> GetMySqlColumnMapping(DataTable dataTable)
{
List<MySqlBulkCopyColumnMapping> colMappings = new List<MySqlBulkCopyColumnMapping>();
int i = 0;
foreach (DataColumn col in dataTable.Columns)
{
colMappings.Add(new MySqlBulkCopyColumnMapping(i, col.ColumnName));
i++;
}
return colMappings;
}
You can ignore the column mapping if you don't have any identity column in your table.
If you have identity column then you have to use the column mapping otherwise it won't insert any records in the table
It will just give message like "x rows were copied but only 0 rows were inserted".
This class i available in the below library
Assembly MySqlConnector, Version=1.0.0.0

Using any of BulkOperation NuGet-package, you can easily have this done.
Here is an example using the package from https://www.nuget.org/packages/Z.BulkOperations/2.14.3/
MySqlConnection conn = DbConnection.OpenConnection();
DataTable dt = new DataTable("testtable");
MySqlDataAdapter da = new MySqlDataAdapter("SELECT * FROM testtable", conn);
MySqlCommandBuilder cb = new MySqlCommandBuilder(da);
da.Fill(dt);
instead of using
......
da.UpdateBatchSize = 1000;
......
da.Update(dt)
just following two lines
var bulk = new BulkOperation(conn);
bulk.BulkInsert(dt);
will take only 5 seconds to copy the whole DataTable into MySQL without first dumping the 100,000 rows from the DataTable to a temporary CSV file.

Draw a graph in Windows Forms Application from a DataTable in a Data Access class

I've done some research and I've managed to found information about drawing a graph where you hard code a Data Table with fixed values.
This is the link: How to create chart using data table
My problem is however;
I don't have a Data Table like that. I have DataAccess class that call the data from a database then stores it in a Data Table;
public DataTable select_top_sheep(string farmerid)
{
dt = new DataTable();
try
{
conn.Open();
SqlCommand cmd =
new SqlCommand("SELECT TOP 10
S.SheepID
,W.Weight
FROM[Farmstat_V1.0].[dbo].[Sheep] S
INNER JOIN[Farmstat_V1.0].[dbo].[Weight] W
ON S.SheepID = W.SheepID
WHERE S.FarmerID = '" + farmerid + "'
ORDER BY W.Weight DESC", conn);
SqlDataReader reader = cmd.ExecuteReader();
dt.Load(reader);
}
catch (Exception)
{
throw;
}
finally
{
conn.Close();
}
return dt;
}
Then on my form I call this method to get the data, but how can I from here display it in a graph? I can see that the data stores successfully in the DataTable when I run the program in debug mode.
I just want to use the basic Chart tool from the toolbox to display the data graphically.

I have managed to figure out the answer with the use of the link I provided in my question.
This is what I did;
protected void Page_Load(object sender, EventArgs e)
{
// Initializes a new instance of the DataAccess class
DataAccess da = new DataAccess();
// The styling of the graph
chart1.Series["Series1"].ChartType = SeriesChartType.Column;
chart1.Series["Series1"].IsValueShownAsLabel = true;
// The required lines for getting the data from the method in the DataAccess
chart1.DataSource = da.select_top_sheep(farmerID);
chart1.Series["Series1"].XValueMember = "SheepID";
chart1.Series["Series1"].YValueMembers = "Weight";
chart1.DataBind();
}

Just need to google it:
Chart sample
public void SampleCode()
{
// some code
foreach (DataRow row in myDataSet.Tables["Query"].Rows)
{
// For each Row add a new series
string seriesName = row["SalesRep"].ToString();
Chart1.Series.Add(seriesName);
Chart1.Series[seriesName].ChartType = SeriesChartType.Line;
Chart1.Series[seriesName].BorderWidth = 2;
for (int colIndex = 1; colIndex < myDataSet.Tables["Query"].Columns.Count; colIndex++)
{
// For each column (column 1 and onward) add the value as a point
string columnName = myDataSet.Tables["Query"].Columns[colIndex].ColumnName;
int YVal = (int)row[columnName];
Chart1.Series[seriesName].Points.AddXY(columnName, YVal);
}
}
DataGrid.DataSource = myDataSet;
DataGrid.DataBind();
}

Dataset to xml Null Values

I have the code below, where from 3 tables I take the data and write an xml.
I want write (when a record column has null value) the column on the xml with null value. For example if (Category_name == Null ) to write on the xml (Null) Right now the code skip the column and don’t even have this column on the xml.
string xmlFileData = "";
string[] tables = new string[] { "category", "company", "config" };
string query;
xmlFileData += "<MyXml>";
SqlConnection conn;
dbconnect obj;
obj = new dbconnect();//initailizing class object
for (int i = 0; i < tables.Length; i++)
{
string ifemptquery;
DataSet ds = new DataSet();
DataSet ds1 = new DataSet();
conn = obj.getConnection(); //calling connection function
ifemptquery = "SELECT * FROM " + tables[i] ";
SqlCommand cmd1 = new SqlCommand(ifemptquery, conn);
conn.Open();
SqlDataAdapter da1 = new SqlDataAdapter(cmd1);
DataTable dt1 = new DataTable();
da1.Fill(dt1);
conn.Close();
if (dt1.Rows.Count > 0)
{
query = "SELECT * FROM " + tables[i] ";
SqlCommand cmd = new SqlCommand(query, conn);
conn.Open();
SqlDataAdapter da = new SqlDataAdapter(cmd);
da.Fill(ds);
conn.Close();
conn.Dispose();
ds.DataSetName = tables[i];
string vartbname = tables[i];
string trimed_tbname = vartbname.Replace("_", "");
ds.Tables[0].TableName = trimed_tbname;
xmlFileData += ds.GetXml();
}
else
{
}
}
xmlFileData += "</MyXml>";
File.WriteAllText(Server.MapPath("~/xmlbackup/") + "Backup.xml", xmlFileData);

I have been searching the whole world for a solution of writing null fields to XML using DataSet.WriteXML(). The answer posted by Vlad is the one I also used in my project but I found that following works in a much more performance optimized way. I have created a function for your convenience. Change your dataset tables one after the other by calling the following function and replacing the tables.
private DataTable GetNullFilledDataTableForXML(DataTable dtSource)
{
// Create a target table with same structure as source and fields as strings
// We can change the column datatype as long as there is no data loaded
DataTable dtTarget = dtSource.Clone();
foreach (DataColumn col in dtTarget.Columns)
col.DataType = typeof(string);
// Start importing the source into target by ItemArray copying which
// is found to be reasonably fast for nulk operations. VS 2015 is reporting
// 500-525 milliseconds for loading 100,000 records x 10 columns
// after null conversion in every cell which may be usable in many
// circumstances.
// Machine config: i5 2nd Gen, 8 GB RAM, Windows 7 64bit, VS 2015 Update 1
int colCountInTarget = dtTarget.Columns.Count;
foreach (DataRow sourceRow in dtSource.Rows)
{
// Get a new row loaded with data from source row
DataRow targetRow = dtTarget.NewRow();
targetRow.ItemArray = sourceRow.ItemArray;
// Update DBNull.Values to empty string in the new (target) row
// We can safely assign empty string since the target table columns
// are all of string type
for (int ctr = 0; ctr < colCountInTarget; ctr++)
if (targetRow[ctr] == DBNull.Value)
targetRow[ctr] = String.Empty;
// Now add the null filled row to target datatable
dtTarget.Rows.Add(targetRow);
}
// Return the target datatable
return dtTarget;
}

Refer similar question here - dataSet.GetXml() doesn't return xml for null or blank columns
Apart from solutions mentioned there, you can also traverse through dataset and write XML using XmlTextWriter. This method is not recommended if you are dealing with huge data.

ReadOnlyException DataTable DataRow "Column X is read only."

I've got a short piece of code that originally created an SqlDataAdapter object over and over.
Trying to streamline my calls a little bit, I replaced the SqlDataAdapter with an SqlCommand and moved the SqlConnection outside of the loop.
Now, whenever I try to edit rows of data returned to my DataTable, I get a ReadOnlyException thrown that was not thrown before.
NOTE: I have a custom function that retrieves the employee's full name based on their ID. For simplicity here, I used "John Doe" in my example code below to demonstrate my point.
ExampleQueryOld works with the SqlDataAdapter; ExampleQueryNew fails with the ReadOnlyException whenever I try to write to an element of the DataRow:
ExampleQueryOld
This works and has no issues:
public static DataTable ExampleQueryOld(string targetItem, string[] sqlQueryStrings) {
DataTable bigTable = new DataTable();
for (int i = 0; i < sqlQueryStrings.Length; i++) {
string sqlText = sqlQueryStrings[i];
DataTable data = new DataTable(targetItem);
using (SqlDataAdapter da = new SqlDataAdapter(sqlText, Global.Data.Connection)) {
try {
da.Fill(data);
} catch (Exception err) {
Global.LogError(_CODEFILE, err);
}
}
int rowCount = data.Rows.Count;
if (0 < rowCount) {
int index = data.Columns.IndexOf(GSTR.Employee);
for (int j = 0; j < rowCount; j++) {
DataRow row = data.Rows[j];
row[index] = "John Doe"; // This Version Works
}
bigTable.Merge(data);
}
}
return bigTable;
}
ExampleQueryNew
This example throws the ReadOnlyException:
public static DataTable ExampleQueryNew(string targetItem, string[] sqlQueryStrings) {
DataTable bigTable = new DataTable();
using (SqlConnection conn = Global.Data.Connection) {
for (int i = 0; i < sqlQueryStrings.Length; i++) {
string sqlText = sqlQueryStrings[i];
using (SqlCommand cmd = new SqlCommand(sqlText, conn)) {
DataTable data = new DataTable(targetItem);
try {
if (cmd.Connection.State == ConnectionState.Closed) {
cmd.Connection.Open();
}
using (SqlDataReader reader = cmd.ExecuteReader()) {
data.Load(reader);
}
} catch (Exception err) {
Global.LogError(_CODEFILE, err);
} finally {
if ((cmd.Connection.State & ConnectionState.Open) != 0) {
cmd.Connection.Close();
}
}
int rowCount = data.Rows.Count;
if (0 < rowCount) {
int index = data.Columns.IndexOf(GSTR.Employee);
for (int j = 0; j < rowCount; j++) {
DataRow row = data.Rows[j];
try {
// ReadOnlyException thrown below: "Column 'index' is read only."
row[index] = "John Doe";
} catch (ReadOnlyException roErr) {
Console.WriteLine(roErr.Message);
}
}
bigTable.Merge(data);
}
}
}
}
return bigTable;
}
Why can I write to the DataRow element in one case, but not in the other?
Is it because the SqlConnection is still open or is the SqlDataAdapter doing something behind the scene?

using DataAdapter.Fill does not load the database schema, which includes whether a column is a primary key or not, and whether a column is read-only or not. To load the database schema, use DataAdapter.FillSchema, but then that's not your questions.
using DataReader to fill a table loads the schema. So, the index column is read-only (probably because it's the primary key) and that information is loaded into the DataTable. Thereby preventing you from modifying the data in the table.
I think #k3b got it right; by setting ReadOnly = false, you should be able to write to the data table.
foreach (System.Data.DataColumn col in tab.Columns) col.ReadOnly = false;

I kept getting the same exception while trying different approaches. What finally worked for me was to set the column's ReadOnly property to false and change the value of the Expression column instead of row[index] = "new value";

In VB, don't pass a read-only DataRow Item by reference
The likelihood that you'll run into this is low, but I was working on some VB.NET code and got the ReadOnlyException.
I ran into this issue because the code was passing the DataRow Item to a Sub ByRef. Just the act of passing-byref triggers the exception.
Sub Main()
Dim dt As New DataTable()
dt.Columns.Add(New DataColumn With {
.ReadOnly = True,
.ColumnName = "Name",
.DataType = GetType(Integer)
})
dt.Rows.Add(4)
Try
DoNothing(dt.Rows(0).Item("Name"))
Console.WriteLine("All good")
Catch ex As Exception
Console.WriteLine(ex.Message)
End Try
End Sub
Sub DoNothing(ByRef item As Object)
End Sub
Output
Column 'Name' is read only
C-sharp
You can't even write code like this in C# . DoNothing(ref dt.Rows[0].Item["Name"]) gives you a compile time error.

open the yourdataset.xsd file of your data set. click on the table or object and click on the specific column which readonly property need to be changed.
its simple solutions.