We Keep Coding

Store DataSet when ending looping in C#

I am using this code to fill DataSet
var ds = new DataSet();
List<string> list = new List<string>();
foreach (DataGridViewRow row in grdInterPOList.Rows)
{
DataGridViewCheckBoxCell chk = row.Cells[0] as DataGridViewCheckBoxCell;
if (Convert.ToBoolean(chk.Value) == true)
if (row.Cells.Count >= 2 && row.Cells[1].Value != null)
{
list.Add(row.Cells[1].Value.ToString());
}
}
foreach (var _PO_No in list)
{
ds= ShippingData_Export(_PO_No);
}
private DataSet ShippingData_Export(string X_PO_NO)
{
var ds = new DataSet();
var sqlConn = new SqlConnection(strConStr);
try
{
sqlConn.Open();
var cmd = new SqlCommand("proc_TBL_PO_M_ShippingExcel", sqlConn);
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#X_PO_NO", X_PO_NO);
var da = new SqlDataAdapter(cmd);
da.Fill(ds);
sqlConn.Close();
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
return ds;
}
If we select more than 1 _PO_No , then we only get the last value after looping.
So my question is how to store DataSet value after end loop.
For example: If we select 2 _PO_No : GV01 and GV02 then after ending loop . We will get both 2 _PO_No not only the last one (GV02)

The problem in your code is:
foreach (var _PO_No in list)
{
ds = ShippingData_Export(_PO_No);
}
You are looping through your list and calling ShippingData_Export on each item in the list but you're then overwriting ds each time you call this method. This is why only the last value is stored.
If you want to have a single DataSet contain multiple results, you need to pass your DataSet to the ShippingData_Export method and add your results to it instead of creating a new one each time.
The result would be something like this:
foreach (var _PO_No in list)
{
ShippingData_Export(_PO_No, ds);
}
private void ShippingData_Export(string X_PO_NO, DataSet ds)
{
var sqlConn = new SqlConnection(strConStr);
try
{
sqlConn.Open();
var cmd = new SqlCommand("proc_TBL_PO_M_ShippingExcel", sqlConn);
cmd.CommandType = CommandType.StoredProcedure;
cmd.Parameters.AddWithValue("#X_PO_NO", X_PO_NO);
var da = new SqlDataAdapter(cmd);
da.Fill(ds);
sqlConn.Close();
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
return ds;
}
A couple of other small issues with your code:
Add { } to the block after the line if (Convert.ToBoolean(chk.Value) == true). Get used to using { } explicitly to make your code more readable.
You are manually opening your SQL Connection but you don't close it when an exception happens. Look into the using statement to make sure you always close your connection even when things go wrong.

How do you programmatically check if a spreadsheet has headers in C#

I am creating a winform application where every day, a user will select a xlsx file with the day's shipping information to be merged with our invoicing data.
The challenge I am having is when the user does not download the xlsx file with the specification that the winform data requires. (I wish I could eliminate this step with an API connection but sadly I cannot)
My first step is checking to see if the xlsx file has headers to that my file path is valid
Example
string connString = "provider=Microsoft.ACE.OLEDB.12.0;Data Source='" + *path* + "';Extended Properties='Excel 12.0;HDR=YES;';";
Where path is returned from an OpenFileDialog box
If the file was chosen wasn't downloaded with headers the statement above throws an exception.
If change HDR=YES; to HDR=NO; then I have trouble identifying the columns I need and if the User bothered to include the correct ones.
My code then tries to load the data into a DataTable
private void loadRows()
{
for (int i = 0; i < deliveryTable.Rows.Count; i++)
{
DataRow dr = deliveryTable.Rows[i];
int deliveryId = 0;
bool result = int.TryParse(dr[0].ToString(), out deliveryId);
if (deliveryId > 1 && !Deliveries.ContainsKey(deliveryId))
{
var delivery = new Delivery(deliveryId)
{
SalesOrg = Convert.ToInt32(dr[8]),
SoldTo = Convert.ToInt32(dr[9]),
SoldName = dr[10].ToString(),
ShipTo = Convert.ToInt32(dr[11]),
ShipName = dr[12].ToString(),
};
Which all works only if the columns are in the right place.
If they are not in the right place my thought is to display a message to the user to get the right information
Does anyone have any suggestions?
(Sorry, first time posting a question and still learning to think through it)

I guess you're loading the spreadsheet into a Datatable? Hard to tell with one line of code. I would use the columns collection in the datatable and check to see if all the columns you want are there. Sample code to enumerate the columns below.
private void PrintValues(DataTable table)
{
foreach(DataRow row in table.Rows)
{
foreach(DataColumn column in table.Columns)
{
Console.WriteLine(row[column]);
}
}
}

private void GetExcelSheetForUpload(string PathName, string UploadExcelName)
{
string excelFile = "DateExcel/" + PathName;
OleDbConnection objConn = null;
System.Data.DataTable dt = null;
try
{
DataSet dss = new DataSet();
String connString = "Provider=Microsoft.ACE.OLEDB.12.0;Persist Security Info=True;Extended Properties=Excel 12.0 Xml;Data Source=" + PathName;
objConn = new OleDbConnection(connString);
objConn.Open();
dt = objConn.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
if (dt == null)
{
return;
}
String[] excelSheets = new String[dt.Rows.Count];
int i = 0;
foreach (DataRow row in dt.Rows)
{
if (i == 0)
{
excelSheets[i] = row["TABLE_NAME"].ToString();
OleDbCommand cmd = new OleDbCommand("SELECT * FROM [" + excelSheets[i] + "]", objConn);
OleDbDataAdapter oleda = new OleDbDataAdapter();
oleda.SelectCommand = cmd;
oleda.Fill(dss, "TABLE");
}
i++;
}
grdExcel.DataSource = dss.Tables[0].DefaultView;
grdExcel.DataBind();
lblTotalRec.InnerText = Convert.ToString(grdExcel.Rows.Count);
}
catch (Exception ex)
{
ViewState["Fuletypeidlist"] = "0";
grdExcel.DataSource = null;
grdExcel.DataBind();
}
finally
{
if (objConn != null)
{
objConn.Close();
objConn.Dispose();
}
if (dt != null)
{
dt.Dispose();
}
}
}
if (grdExcel.HeaderRow.Cells[0].Text.ToString() == "CODE")
{
GetExcelSheetForEmpl(PathName);
}
else
{
divStatusMsg.Style.Add("display", "");
divStatusMsg.Attributes.Add("class", "alert alert-danger alert-dismissable");
divStatusMsg.InnerText = "ERROR !!... Upload Excel Sheet in header Defined Format ";
}

Fill data controls on ASP.NET Website

I am attempting to get the data my connectionstring pulls in my C# class's to the appropriate data controls on my website. I have looked online how to go about doing this but no results. I figured it would be easy like C# and you just reference the control you want to fill in the .Net page. Any suggestions on how to get my data into my controls? I have posted my c# code to get the data.
load method:
public void load()
{
var sqlString = new StringBuilder();
sqlString.Append("SELECT CaseNum6, CaseNum9, Group, Completion ");
sqlString.Append("FROM WorkOrder ");
sqlString.Append("WHERE Group = 1 OR Group = 2 ");
sqlString.Append("AND Completion = 0 ");
SqlDataReader reader = null;
SqlConnection dbConn = DBHelper.getConnection();
SqlParameter[] parameters = new SqlParameter[] { new SqlParameter("#CaseNum6", CaseNum6 )};
try
{
reader = DBHelper.executeQuery(dbConn, sqlString.ToString(), parameters);
if (reader != null)
{
if (reader.Read())
{
CaseNum6 = (int)reader["CaseNum6"];
CaseNum9 = (int)reader["CaseNum9"];
Group = (int)reader["Group"];
Completion = (bool)reader["Completion"];
}
else
throw new Exception("No record returned");
reader.Close();
reader.Dispose();
dbConn.Close();
dbConn.Dispose();
}
}
DBHelper:
class DBHelper
{
private DBHelper() { }
public static SqlConnection getConnection()
{
return new SqlConnection(ConfigurationManager.ConnectionStrings["Connection"].ConnectionString);
}
public static SqlConnection getFRESHConnection()
{
return new SqlConnection(ConfigurationManager.ConnectionStrings["FRESHConnection"].ConnectionString);
}
public static SqlDataReader executeQuery(SqlConnection dbConn, string sqlString, SqlParameter[] parameters)
{
SqlCommand cmd = null;
SqlDataReader reader = null;
try
{
if (dbConn.State == ConnectionState.Closed)
dbConn.Open();
cmd = dbConn.CreateCommand();
cmd.CommandText = sqlString;
if (parameters != null)
{
cmd.Parameters.AddRange(parameters);
}
reader = cmd.ExecuteReader();
cmd.Dispose();
}
catch (Exception ex)
{
throw ex;
}
return reader;
}
}

Not sure what exactly is the issue. But to give an overview of how to add values to controls.
Drop a control from the toolbox into the web page(.aspx).
You will now be able to reference it in the code behind, and simply use something like
TextBox1.Text = "Hello World"; // TextBox1 is the id of the control
See more on MSDN docs
Get and Set values on Textboxes

Did not have the right file extension for background coding.

Report Viewer X Dapper

I'm feeding a ReportDataSource with a query using Dapper.
However, I have an empty report, even with an IEnumerable loaded data.
When you spend a Datatable works.
How do I pass data from a query using Dapper for ReportViewer?
this.reportViewer.LocalReport.DataSources.Clear();
DataTable dt = new DataTable();
dt = CN.Query(Sql, param);
Microsoft.Reporting.WinForms.ReportDataSource rprtDTSource = new Microsoft.Reporting.WinForms.ReportDataSource(dt.TableName, dt);
this.reportViewer.LocalReport.DataSources.Add(rprtDTSource);
this.reportViewer.RefreshReport(); –

Looks like Dapper now supports the DataTable...
From the test:
public void ExecuteReader()
{
var dt = new DataTable();
dt.Load(connection.ExecuteReader("select 3 as [three], 4 as [four]"));
dt.Columns.Count.IsEqualTo(2);
dt.Columns[0].ColumnName.IsEqualTo("three");
dt.Columns[1].ColumnName.IsEqualTo("four");
dt.Rows.Count.IsEqualTo(1);
((int)dt.Rows[0][0]).IsEqualTo(3);
((int)dt.Rows[0][1]).IsEqualTo(4);
}
Also now supported is using a DataTable as a TableValueParameter:
public void DataTableParameters()
{
try { connection.Execute("drop proc #DataTableParameters"); } catch { }
try { connection.Execute("drop table #DataTableParameters"); } catch { }
try { connection.Execute("drop type MyTVPType"); } catch { }
connection.Execute("create type MyTVPType as table (id int)");
connection.Execute("create proc #DataTableParameters #ids MyTVPType readonly as select count(1) from #ids");
var table = new DataTable { Columns = { { "id", typeof(int) } }, Rows = { { 1 }, { 2 }, { 3 } } };
int count = connection.Query<int>("#DataTableParameters", new { ids = table.AsTableValuedParameter() }, commandType: CommandType.StoredProcedure).First();
count.IsEqualTo(3);
count = connection.Query<int>("select count(1) from #ids", new { ids = table.AsTableValuedParameter("MyTVPType") }).First();
count.IsEqualTo(3);
try
{
connection.Query<int>("select count(1) from #ids", new { ids = table.AsTableValuedParameter() }).First();
throw new InvalidOperationException();
} catch (Exception ex)
{
ex.Message.Equals("The table type parameter 'ids' must have a valid type name.");
}
}

As I could not find another way to feed my ReportViewer to a query Dapper.Query then downloaded the source and added the code below.
#region CODTEC SISTEMAS
/// <summary>
/// Return a typed list of objects, reader is closed after the call
/// </summary>
public static DataTable Query(this IDbConnection cnn, string sql, object param, IDbTransaction transaction, int? commandTimeout, CommandType? commandType)
{
var identity = new Identity(sql, commandType, cnn, typeof(DapperRow), param == null ? null : param.GetType(), null);
var info = GetCacheInfo(identity);
IDbCommand cmd = null;
IDataReader reader = null;
bool wasClosed = cnn.State == ConnectionState.Closed;
try
{
cmd = SetupCommand(cnn, transaction, sql, info.ParamReader, param, commandTimeout, commandType);
if (wasClosed) cnn.Open();
reader = cmd.ExecuteReader(wasClosed ? CommandBehavior.CloseConnection : CommandBehavior.Default);
wasClosed = false; // *if* the connection was closed and we got this far, then we now have a reader
// with the CloseConnection flag, so the reader will deal with the connection; we
// still need something in the "finally" to ensure that broken SQL still results
// in the connection closing itself
DataTable dt = new DataTable();
dt.Load(reader);
// happy path; close the reader cleanly - no
// need for "Cancel" etc
reader.Dispose();
reader = null;
return dt;
}
finally
{
if (reader != null)
{
if (!reader.IsClosed) try { cmd.Cancel(); }
catch { /* don't spoil the existing exception */ }
reader.Dispose();
}
if (wasClosed) cnn.Close();
if (cmd != null) cmd.Dispose();
}
}
#endregion

To Get the DataTable from SqliteDB using Dapper.
Pass the query such as "select*from table" inside the method.
list is formed, by executing query.
Then serialize the object into Json and then deserialize the Json
into the DataTable.
public DataTable method(string query)
{
string connection = #"Data Source= C:\User\DBFolder\sampleDB.db;Version=3;New=False;Compress=True;";
using (IDbConnection dbConnection = new SQLiteConnection(connection))
{
dbConnection.Open();
var output1 = dbConnection.Query(query).ToList();
dbConnection.Close();
var json = JsonConvert.SerializeObject(output1);
DataTable dt = (DataTable)JsonConvert.DeserializeObject(json, (typeof(DataTable)));
return dt;
}
}

Not sure how Dapper works, but a datatable is binded like this:
DataTable dt = new DataTable();
DataColumn dc = dt.Columns.Add();
dc.ColumnName = "DataColumn1";
dc = dt.Columns.Add();
dc.ColumnName = "DataColumn2";
dt.Rows.Add(new object[] { "Frank", 32 });
this.reportViewer1.LocalReport.DataSources.Clear();
this.reportViewer1.LocalReport.DataSources.Add(new ReportDataSource("DataSet1_DataTable1", dt));
this.reportViewer1.RefreshReport();
and since I am using C#, I bind data sources like this:
this.bindingSource1.DataSource = getListMethod(); // this method/property returns a list of objects
this.reportViewer1.LocalReport.DataSources.Add(new ReportDataSource("Point", (this.bindingSource1)));
//Point is the datatable name in my Dataset.xsd file

Dealing with huge amount of data when inserting into sql database

in my code the user can upload an excel document wish contains it's phone contact list.Me as a developer should read that excel file turn it into a dataTable and insert it into the database .
The Problem is that some clients have a huge amount of contacts like saying 5000 and more contacts and when i am trying to insert this amount of data into the database it's crashing and giving me a timeout exception.
What would be the best way to avoid this kind of exception and is their any code that can reduce the time of the insert statement so the user don't wait too long ?
the code
public SqlConnection connection = new SqlConnection(System.Configuration.ConfigurationManager.ConnectionStrings["ConnectionString"].ConnectionString);
public void Insert(string InsertQuery)
{
SqlDataAdapter adp = new SqlDataAdapter();
adp.InsertCommand = new SqlCommand(InsertQuery, connection);
if (connection.State == System.Data.ConnectionState.Closed)
{
connection.Open();
}
adp.InsertCommand.ExecuteNonQuery();
connection.Close();
}
protected void submit_Click(object sender, EventArgs e)
{
string UploadFolder = "Savedfiles/";
if (Upload.HasFile) {
string fileName = Upload.PostedFile.FileName;
string path=Server.MapPath(UploadFolder+fileName);
Upload.SaveAs(path);
Msg.Text = "successfully uploaded";
DataTable ValuesDt = new DataTable();
ValuesDt = ConvertExcelFileToDataTable(path);
Session["valuesdt"] = ValuesDt;
Excel_grd.DataSource = ValuesDt;
Excel_grd.DataBind();
}
}
protected void SendToServer_Click(object sender, EventArgs e)
{
DataTable Values = Session["valuesdt"] as DataTable ;
if(Values.Rows.Count>0)
{
DataTable dv = Values.DefaultView.ToTable(true, "Mobile1", "Mobile2", "Tel", "Category");
double Mobile1,Mobile2,Tel;string Category="";
for (int i = 0; i < Values.Rows.Count; i++)
{
Mobile1 =Values.Rows[i]["Mobile1"].ToString()==""?0: double.Parse(Values.Rows[i]["Mobile1"].ToString());
Mobile2 = Values.Rows[i]["Mobile2"].ToString() == "" ? 0 : double.Parse(Values.Rows[i]["Mobile2"].ToString());
Tel = Values.Rows[i]["Tel"].ToString() == "" ? 0 : double.Parse(Values.Rows[i]["Tel"].ToString());
Category = Values.Rows[i]["Category"].ToString();
Insert("INSERT INTO client(Mobile1,Mobile2,Tel,Category) VALUES(" + Mobile1 + "," + Mobile2 + "," + Tel + ",'" + Category + "')");
Msg.Text = "Submitied successfully to the server ";
}
}
}

You can try SqlBulkCopy to insert Datatable to Database Table
Something like this,
using (SqlBulkCopy bulkCopy = new SqlBulkCopy(sqlConnection, SqlBulkCopyOptions.KeepIdentity))
{
bulkCopy.DestinationTableName = DestTableName;
string[] DtColumnName = YourDataTableColumns;
foreach (string dbcol in DbColumnName)//To map Column of Datatable to that of DataBase tabele
{
foreach (string dtcol in DtColumnName)
{
if (dbcol.ToLower() == dtcol.ToLower())
{
SqlBulkCopyColumnMapping mapID = new SqlBulkCopyColumnMapping(dtcol, dbcol);
bulkCopy.ColumnMappings.Add(mapID);
break;
}
}
}
bulkCopy.WriteToServer(YourDataTableName.CreateDataReader());
bulkCopy.Close();
}
For more Read http://msdn.microsoft.com/en-us/library/system.data.sqlclient.sqlbulkcopy.aspx

You are inserting 1 row at a time, which is very expensive for this amount of data
In those cases you should use bulk insert, so the round trip to DB will be only once, if you need to roll back - all is the same transaction

You can use SqlBulkCopy which is more work, or you can use the batch update feature of the SqlAdpater. Instead of creating your own insert statement, then building a sqladapter, and then manually executing it, create a dataset, fill it, create one sqldataadpater, set the number of inserts in a batch, then execute the adapter once.
I could repeat the code, but this article shows exactly how to do it: http://msdn.microsoft.com/en-us/library/kbbwt18a%28v=vs.80%29.aspx
protected void SendToServer_Click(object sender, EventArgs e)
{
DataTable Values = Session["valuesdt"] as DataTable ;
if(Values.Rows.Count>0)
{
DataTable dv = Values.DefaultView.ToTable(true, "Mobile1", "Mobile2", "Tel", "Category");
//Fix up default values
for (int i = 0; i < Values.Rows.Count; i++)
{
Values.Rows[i]["Mobile1"] =Values.Rows[i]["Mobile1"].ToString()==""?0: double.Parse(Values.Rows[i]["Mobile1"].ToString());
Values.Rows[i]["Mobile2"] = Values.Rows[i]["Mobile2"].ToString() == "" ? 0 : double.Parse(Values.Rows[i]["Mobile2"].ToString());
Values.Rows[i]["Tel"] = Values.Rows[i]["Tel"].ToString() == "" ? 0 : double.Parse(Values.Rows[i]["Tel"].ToString());
Values.Rows[i]["Category"] = Values.Rows[i]["Category"].ToString();
}
BatchUpdate(dv,1000);
}
}
public static void BatchUpdate(DataTable dataTable,Int32 batchSize)
{
// Assumes GetConnectionString() returns a valid connection string.
string connectionString = GetConnectionString();
// Connect to the database.
using (SqlConnection connection = new SqlConnection(connectionString))
{
// Create a SqlDataAdapter.
SqlDataAdapter adapter = new SqlDataAdapter();
// Set the INSERT command and parameter.
adapter.InsertCommand = new SqlCommand(
"INSERT INTO client(Mobile1,Mobile2,Tel,Category) VALUES(#Mobile1,#Mobile2,#Tel,#Category);", connection);
adapter.InsertCommand.Parameters.Add("#Mobile1",
SqlDbType.Float);
adapter.InsertCommand.Parameters.Add("#Mobile2",
SqlDbType.Float);
adapter.InsertCommand.Parameters.Add("#Tel",
SqlDbType.Float);
adapter.InsertCommand.Parameters.Add("#Category",
SqlDbType.NVarchar, 50);
adapter.InsertCommand.UpdatedRowSource = UpdateRowSource.None;
// Set the batch size.
adapter.UpdateBatchSize = batchSize;
// Execute the update.
adapter.Update(dataTable);
}
}

I know this is a super old post, but you should not need to use the bulk operations explained in the existing answers for 5000 inserts. Your performance is suffering so much because you close and reopen the connection for each row insert. Here is some code I have used in the past that keeps one connection open and executes as many commands as needed to push all the data to the DB:
public static class DataWorker
{
public static Func<IEnumerable<T>, Task> GetStoredProcedureWorker<T>(Func<SqlConnection> connectionSource, string storedProcedureName, Func<T, IEnumerable<(string paramName, object paramValue)>> parameterizer)
{
if (connectionSource is null) throw new ArgumentNullException(nameof(connectionSource));
SqlConnection openConnection()
{
var conn = connectionSource() ?? throw new ArgumentNullException(nameof(connectionSource), $"Connection from {nameof(connectionSource)} cannot be null");
var connState = conn.State;
if (connState != ConnectionState.Open)
{
conn.Open();
}
return conn;
}
async Task DoStoredProcedureWork(IEnumerable<T> workData)
{
using (var connection = openConnection())
using (var command = connection.CreateCommand())
{
command.CommandType = CommandType.StoredProcedure;
command.CommandText = storedProcedureName;
command.Prepare();
foreach (var thing in workData)
{
command.Parameters.Clear();
foreach (var (paramName, paramValue) in parameterizer(thing))
{
command.Parameters.AddWithValue(paramName, paramValue ?? DBNull.Value);
}
await command.ExecuteNonQueryAsync().ConfigureAwait(false);
}
}
}
return DoStoredProcedureWork;
}
}
This was actually from a project where I was gathering emails for a restriction list, so kind of relevant example of what a parameterizer argument might look like and how to use the above code:
IEnumerable<(string,object)> RestrictionToParameter(EmailRestriction emailRestriction)
{
yield return ("#emailAddress", emailRestriction.Email);
yield return ("#reason", emailRestriction.Reason);
yield return ("#restrictionType", emailRestriction.RestrictionType);
yield return ("#dateTime", emailRestriction.Date);
}
var worker = DataWorker.GetStoredProcedureWorker<EmailRestriction>(ConnectionFactory, #"[emaildata].[AddRestrictedEmail]", RestrictionToParameter);
await worker(emailRestrictions).ConfigureAwait(false);