I have data table with 100000 records, I want to iterate through data table for every 10,000 records I want to save the records. for the next iteration next 10000 records I want to save until for 100000 records.
DataTable dt = new DataTable();
dt = ds.tables[0]; //here i am getting 100,000 records
for (int i = 0; i < dt.rows.count; i + 10000)
{
savedatatable(dt[i]);
}
You should use the following code:
DataTable dt = new DataTable();
dt = ds.tables[0]; //here i am getting 100,000 records
//Loop through columns in rows
for (int i = 0; i < dt.rows.count && i < 100000; i += 10000)
{
foreach (DataColumn col in dt.Columns)
savedatatable(dt.Rows[col.ColumnName].ToString());
}
or
DataTable dt = new DataTable();
dt = ds.tables[0]; //here i am getting 100,000 records
//Loop through rows in columns
foreach (DataColumn col in dt.Columns)
{
for (int i = 0; i < dt.rows.count && i < 100000; i += 10000)
savedatatable(dt.Rows[col.ColumnName].ToString());
}
Here's a similar question, but I'm not sure if this is what you wanted. : Looping through a DataTable
Should be something like this:
for (int i = 0; i < dt.Rows.Count; i+=10000)
{
DataRow dr = dt.Rows[i];
// do something
}
Related
I hope you're all doing well.
I have a quick question regarding iteration. I've read several post about the speed of iteration and I couldn't figure how to make my iteration faster. Currently I'm doing something like this :
void Iteration()
{
//Creating and filling the datatable
DataTable dt = new DataTable();
dt.Columns.Add("Datetime", typeof(DateTime));
for (int i = 0; i < 150; i++)
{
DataRow row = dt.NewRow();
row["Datetime"] = DateTime.Now.AddDays(i);
dt.Rows.Add(row);
}
//Creating and filling the list
List<DateTime> _listDates = new List<DateTime>();
DateTime _startDate = DateTime.Now.AddMonths(-1);
for(int i = 0; i < 250; i++)
_listDates.Add(_startDate.AddDays(i));
//Here's the actual iteration
foreach (DateTime _date in _listDates)
{
foreach (DataRow row in dt.Rows)
{
if ((DateTime)row["Datetime"] == _date)
{
//Do something.........
}
}
}
}
I fill a List<DateTime> and a DataTable with respectively 250 and 150 rows/line. I then want to compare the two values against each other and do something when there's a match. However, in my method that means 250 * 150 = 37500 passes. Now I could break out the loop when there's a match but that seems trivial to me since the match can also be on the bottom of the list and datatable. And in my program the average lists and tables have 2500 rows. So that's millions of passes every n minutes. Needles to say that this takes a while. I'm running this calculation on a separate thread so my program stays responsive.
Is there any way to make this smarter and/or faster ? Am I on the right track ?
Cheers,
What about this? this is more efficient because both data table and datetime list are scanned only once, and HashSet.Contains time complexity is O(1).
void Iteration()
{
//Creating and filling the datatable
DataTable dt = new DataTable();
dt.Columns.Add("Datetime", typeof(DateTime));
for (int i = 0; i < 150; i++)
{
DataRow row = dt.NewRow();
row["Datetime"] = DateTime.Now.AddDays(i);
dt.Rows.Add(row);
}
//Creating and filling the list
List<DateTime> _listDates = new List<DateTime>();
DateTime _startDate = DateTime.Now.AddMonths(-1);
for (int i = 0; i < 250; i++)
_listDates.Add(_startDate.AddDays(i));
var dateSet = new HashSet<DateTime>(_listDates);
foreach (DataRow row in dt.Rows)
{
if (dateSet.Contains( (DateTime)row["Datetime"]))
{
//Do something.........
}
}
}
Hi Can any one suggest how to read data from datagrid in windowsforms application which has two columns(FileName and FilePath).
Below is the code I tried its returning all Filename and FilePath in single column(FileName).
Any suggestions would be helpful to me..
`
public System.Data.DataTable ExportToExcel()
{
System.Data.DataTable table = new System.Data.DataTable();
table.Columns.Add("FileName", typeof(string));
table.Columns.Add("FilePath", typeof(string));
for (int rows = 0; rows < dataGridView1.Rows.Count; rows++)
{
for (int col= 0; col < dataGridView1.Rows[rows].Cells.Count; col++)
{
table.Rows.Add(dataGridView1.Rows[rows].Cells[col].Value.ToString());
}
}
The issue is that the line…
table.Rows.Add(dataGridView1.Rows[rows].Cells[col].Value.ToString());
is only adding to the FIRST column. You are adding a new row for EACH column.
To add a row to a table that has two columns would be of the form…
table.Rows.Add(column1Value, column2Value);
Since you know there are TWO columns, simply loop through the rows. It is not necessary to loop through the columns.
DataTable table = new DataTable();
table.Columns.Add("FileName", typeof(string));
table.Columns.Add("FilePath", typeof(string));
for (int rows = 0; rows < dataGridView1.Rows.Count; rows++) {
if (!dataGridView1.Rows[rows].IsNewRow &&
dataGridView1.Rows[rows].Cells[0].Value != null &&
dataGridView1.Rows[rows].Cells[1].Value != null) {
table.Rows.Add(dataGridView1.Rows[rows].Cells[0].Value.ToString(), dataGridView1.Rows[rows].Cells[1].Value.ToString());
}
}
If it is unknown how many columns are in the DataGridView and you must loop through the columns… then you will need to make sure there are at least as many columns in the DataGridView as there are columns in the DataTable.
DataTable table = new DataTable();
table.Columns.Add("FileName", typeof(string));
table.Columns.Add("FilePath", typeof(string));
DataRow newRow;
if (dataGridView1.Columns.Count >= table.Columns.Count) {
for (int row = 0; row < dataGridView1.Rows.Count; row++) {
if (!dataGridView1.Rows[row].IsNewRow) {
newRow = table.NewRow();
for (int col = 0; col < table.Columns.Count; col++) {
if (dataGridView1.Rows[row].Cells[col].Value != null)
newRow[col] = dataGridView1.Rows[row].Cells[col].Value.ToString();
else
newRow[col] = "";
}
table.Rows.Add(newRow);
}
}
}
If you know what object dataGridView1.DataSource is, just visit the object. It would be much easier.
Besides, there's a simple way to export data from DataGridView to Excel:
Ctrl+A Select all cells in DataGridView.
Ctrl+C Copy them.
Ctrl+V Paste them in a sheet of Excel.
I've to do the transpose of my DataTable and I used the code I found in internet. This code is giving an extra row at the top showing the column index. How can I remove this row?This extra row is at above the header row of my DataTable, and when I use the code dt_.Rows.Remove(dt_.Rows[0]); the header row is removed but not the top row. Reference taken from:
Transpose a datatable
How can I remove the top row?
DataTable dt_ = new DataTable();
DataTable table = new DataTable();
//Get all the rows and change into columns
for (int i = 0; i <= dt.Rows.Count; i++)
{
table.Columns.Add(Convert.ToString(i));
}
DataRow dr;
//get all the columns and make it as rows
for (int j = 0; j < dt.Columns.Count; j++)
{
dr = table.NewRow();
dr[0] = dt.Columns[j].ToString();
for (int k = 1; k <= dt.Rows.Count; k++)
{
dr[k] = dt.Rows[k - 1][j];
}
table.Rows.Add(dr);
}
dt_ = table;
//dt_.Rows.Remove(dt_.Rows[0]); removes the header row
My gridview code
<asp:GridView ID="GridView1"
runat="server"
CellPadding="3"
CellSpacing="2"
AutoGenerateColumns="true"
ShowFooter="true"
FooterStyle-HorizontalAlign="Left"
RowStyle-BorderColor="Black" HeaderStyle-BackColor="#0CA3D2">
<FooterStyle BackColor="#87CEFA" />
</asp:GridView>
I've not shown the footer section sum calculation code behind because it is irrelevant here.
snapshot:
I think if you start the loop from 1 it should work
DataTable dt = new DataTable();
DataTable dt_ = new DataTable();
DataTable table = new DataTable();
//Get all the rows and change into columns
for (int i = 0; i <= dt.Rows.Count; i++)
{
table.Columns.Add(Convert.ToString(i));
}
DataRow dr;
//get all the columns and make it as rows
//HERE THE LOOK STARTS FROM ONE
for (int j = 1; j < dt.Columns.Count; j++)
{
dr = table.NewRow();
dr[0] = dt.Columns[j].ToString();
for (int k = 1; k <= dt.Rows.Count; k++)
{
dr[k] = dt.Rows[k - 1][j];
}
table.Rows.Add(dr);
}
dt_ = table;
That's because
//Get all the rows and change into columns
for (int i = 0; i <= dt.Rows.Count; i++)
{
table.Columns.Add(Convert.ToString(i));
}
is creating columns named 0, 1, and 2. the problem is with Convert.ToString(i). That, and you will also need to change how you read k.
The code will look like this (not tested):
DataTable dt_ = new DataTable();
DataTable table = new DataTable();
//Get all the rows and change into columns
for (int i = 0; i <= dt.Rows.Count; i++)
{
table.Columns.Add(dt.Rows[i][0]); // the column name is in the first cell
}
DataRow dr;
//get all the columns and make it as rows
for (int j = 0; j < dt.Columns.Count; j++)
{
dr = table.NewRow();
dr[0] = dt.Columns[j].ToString();
// changed the for condition
for (int k = 1; k < dt.Rows.Count; k++)
{
dr[k] = dt.Rows[k][j]; // changed the 'k - 1' to 'k'
}
table.Rows.Add(dr);
}
dt_ = table;
string query = q;
SqlCommand queryCommand = new SqlCommand(query, Connection);
SqlDataReader queryCommandReader = queryCommand.ExecuteReader();
DataTable dataTable = new DataTable();
dataTable.Load(queryCommandReader);
List<string> rowText = new List<string>();
for (int i = 0; i < 4; i++)
{
foreach (DataColumn columns in dataTable.Columns)
{
rowText.Add(dataTable.Rows[i][columns.ColumnName] + "");
}
}
in this example I get 4 rows from database the condition in for loop i < 4
I wanna get really numbers of rows not just 4
Use foreach for the rows as well
See http://msdn.microsoft.com/en-us/library/system.data.datatable.rows.aspx
foreach(DataRow row in dataTable.Rows)
{
foreach (DataColumn column in dataTable.Columns)
{
rowText.Add( row[column] );
}
}
In your specific example simple replace the the 4 by dataTable.Rows.Count so that you get:
for(int i = 0; i < dataTable.Rows.Count; i++)
{
// ...
}
Or switch to foreach and use the answer of Gaby.
I am trying to copy data from my datagridview to a datatable so i can export to a csv file
here is the code
public DataTable createdatatablefromdgv()
{
DataTable dsptable = new DataTable();
for (int i = 0; i < dataGridView1.Columns.Count; i++)
{
DataColumn dspcolumn = new DataColumn(dataGridView1.Columns[i].HeaderText);
dsptable.Columns.Add(dspcolumn);
}
int noOfColumns = dataGridView1.Columns.Count;
foreach (DataGridViewRow dgvr in dataGridView1.Rows)
{
DataRow dataRow = dsptable.NewRow();
for (int i = 0; i < noOfColumns; i++)
{
dataRow[i] = dgvr.Cells[i].Value.ToString();
}
}
return dsptable;
}
It seems like it copies the data from the the grid to to the table but when I return the datatable all there is the columns no rows
You are not adding dataRow to data table after assigning values to its columns.
public DataTable createdatatablefromdgv()
{
DataTable dsptable = new DataTable();
for (int i = 0; i < dataGridView1.Columns.Count; i++)
{
DataColumn dspcolumn = new DataColumn(dataGridView1.Columns[i].HeaderText);
dsptable.Columns.Add(dspcolumn);
}
int noOfColumns = dataGridView1.Columns.Count;
foreach (DataGridViewRow dgvr in dataGridView1.Rows)
{
DataRow dataRow = dsptable.NewRow();
for (int i = 0; i < noOfColumns; i++)
{
dataRow[i] = dgvr.Cells[i].Value.ToString();
}
dsptable.Rows.Add(dataRow); //Add this statement to add rows to Data Table
}
return dsptable;
}
The above answer is correct but a little explanation as to why you encountered this problem. One would think that by calling NewRow() on the DataTable you would add a new row to the DataTable and then be able to access it.
What NewRow actually does is give you an instance of a DataRow which you can then use column names as apose to column identifiers (integers).
This allows you to do something like this
DataRow dataRow = dsptable.NewRow();
foreach(DataColumn dc in dsptable.Columns)
{
dataRow[dc.ColumnName] = dgvr.Cells[dc.ColumnName].Value.ToString()
}
Alternatively you could just call Rows.Add() which takes an object array or a as you were trying to do a DataRow.
List<string> rowData = new List<string>();
for (int i = 0; i < noOfColumns; i++)
{
rowData.Add(dgvr.Cells[i].Value.ToString());
}
dsptable.Rows.Add(rowData.ToArray());
This should explain adding rows to a datatable more simply :)