I have two XML data sets, ds1 and ds2. I read these data sets with .ReadXML(name, XmlReadMode.ReadSchema). I am trying to get a DiffGram with the differences between the two by using a merged data set as shown below.
DataSet ds3 = new DataSet();
ds3.Merge(ds1);
ds3.AcceptChanges();
ds3.Merge(ds2);
DataSet ds4 = ds3.GetChanges();
ds4.WriteXml("ds4.xml", XmlWriteMode.DiffGram);
ds1 and ds2 each contains multiple elements. I created ds2 by copying the ds1 file and modifying one of the records.
However, when I look at ds4.xml after execution, it shows all of the record sets in ds1 and all of the records in ds2 (so it shows duplicate entries), and the ds2 updates are listed as ...diffgr:hasChanges="inserted">. It seems that this is only inserting, not updating existing records.
How can I get ds4 to only show the change that was made in ds2?
This behavior of inserting versus updating typically occurs due to the lack of defined primary keys. Have you set primary keys on the tables? That's how the columns are matched up during a merge. Per MSDN (emphasis mine):
When merging a new source DataSet into
the target, any source rows with a
DataRowState value of Unchanged,
Modified, or Deleted are matched to
target rows with the same primary key
values. Source rows with a
DataRowState value of Added are
matched to new target rows with the
same primary key values as the new
source rows.
Therefore, for each DataTable you should set the PrimaryKey property. I wrote a detailed example of this on the MSDN DataTable.Merge page a few years ago. You can take a look at that write up here: Merge using Primary Keys for Expected Results.
A brief example of this approach:
DataTable dt1 = new DataTable();
dt1.Columns.Add("ID", typeof(int));
dt1.Columns.Add("Customer", typeof(string));
dt1.PrimaryKey = new[] { dt1.Columns["ID"] };
dt1.Rows.Add(new object[] { 1, "Ahmad" });
DataTable dt2 = new DataTable();
dt2.Columns.Add("ID", typeof(int));
dt2.Columns.Add("Customer", typeof(string));
dt2.PrimaryKey = new[] { dt2.Columns["ID"] };
dt2.Rows.Add(new object[] { 1, "Mageed" });
// try without primary keys and it'll add a new record
dt1.Merge(dt2);
EDIT: regarding your comment, you could reject the changes on merged rows that were not really changed by passing the table through the code below. A method that accepts a DataTable would be neater. It's important that this code is used prior to calling the DataTable.AcceptChanges() method, otherwise the row states will be discarded.
With LINQ:
foreach (DataRow row in dt1.Rows)
{
if (row.RowState == DataRowState.Modified)
{
var original = dt1.Columns.Cast<DataColumn>()
.Select(c => row[c, DataRowVersion.Original]);
bool isUnchanged = row.ItemArray.SequenceEqual(original);
if (isUnchanged)
{
row.RejectChanges();
}
}
}
If LINQ isn't an option:
foreach (DataRow row in dt1.Rows)
{
if (row.RowState == DataRowState.Modified)
{
bool isUnchanged = true;
foreach (DataColumn col in dt1.Columns)
{
if (!row[col.ColumnName].Equals(row[col.ColumnName, DataRowVersion.Original]))
{
isUnchanged = false;
break;
}
}
if (isUnchanged)
{
row.RejectChanges();
}
}
}
You can call dt1.AcceptChanges() after this is done.
Related
I have a requirement where I need to copy the existing DataTable rows/values to the new DataTable,
New DataTable is of custom(different column names) structure/schema as we need this new DataTable data to be exported to Excel file later.
To copy the required column values to the new DataTable I have created string Array which contains required column names(columns from which we need to copy data to new DataTable) of existing DataTable, given below.
string[] selectedColumns = new[] { "SUPPLIER_NAME", "SUPPLIER_NO", "CONFIRMATION_NO", "RELEASE_NO", "WCO_INVOICE_NO",
"CUSTOMER_BILLED", "BALANCE_TOBILL", "SUPP_INVOICE_NO", "SUPPLIER_PAID", "BALANCE_COST" }; //TODO Add columns "WCO_INVOICE_DATE", "SUPPLIER_INVOICE_DATE" later.
For getting the required DataTable, I have created a method which will be called like below by passing existing DataTable and selected columns of existing DataTable column names as an parameters.
DataTable _dtPrjLedgerExportData = ControllerClass.dtProjectLedgerExport(dtfilter, selectedColumns);
ControllerClass Class dtProjectLedgerExport method will return the newly created DataTable i.e. DataTable with the customize Column names and the values pulled from existing DatTable,
New DataTable additionally contain two new columns (WCO_Invoice_Date, Supplier_Invoice_Date) with default blank values (set from the code).
public static DataTable dtProjectLedgerExport(DataTable dtToExport, string[] selectedColumns)
{
DataTable dt = new DataTable();
dt.Columns.Add("Supplier");
dt.Columns.Add("Supplier_No");
dt.Columns.Add("Confirmation_Number");
dt.Columns.Add("Release_Number");
dt.Columns.Add("WCO_Invoice_No");
dt.Columns.Add("WCO_Invoice_Date");
dt.Columns.Add("Customer_Billed_Amt");
dt.Columns.Add("Balance_Remaining_to_Bill");
dt.Columns.Add("Supplier_Invoice_Number");
dt.Columns.Add("Supplier_Invoice_Date");
dt.Columns.Add("Supplier_Paid_Amt");
dt.Columns.Add("Remaining_Cost_Dollar_Balance");
//temporarily set default value for non-existing rows
dt.Columns["WCO_Invoice_Date"].DefaultValue = string.Empty;
dt.Columns["Supplier_Invoice_Date"].DefaultValue = string.Empty;
//Copy rows to dt
if (dtToExport != null && dtToExport.Rows.Count > 0)
dt = new DataView(dtToExport).ToTable(false, selectedColumns);
return dt;
}
Problem:
The above code is not working as expected it is returning Data of existing DataTable with the same structure.
What changes are required in above code to get the expected result?
I'd use this approach:
public static DataTable DtProjectLedgerExport(DataTable dtToExport, string[] selectedColumns, params string[] additionalColumns)
{
DataTable dt = dtToExport.Copy(); // add columns and data
List<DataColumn> removeColumns = dt.Columns.Cast<DataColumn>()
.Where(c => !selectedColumns.Contains(c.ColumnName, StringComparer.InvariantCultureIgnoreCase))
.ToList();
removeColumns.ForEach(dt.Columns.Remove);
foreach (string colName in additionalColumns)
{
DataColumn newColumn = new DataColumn(colName);
newColumn.DefaultValue = string.Empty;
dt.Columns.Add(newColumn);
}
return dt;
}
After stating in the comments that you have different column names in each table...(Seriously that's a critical piece of information lol)! The only way is to basiaclly map the values yourself for each column like so.
public static DataTable dtProjectLedgerExport(DataTable dtToExport)
{
DataTable dt = new DataTable();
dt.Columns.Add("Supplier");
dt.Columns.Add("Supplier_No");
dt.Columns.Add("Confirmation_Number");
dt.Columns.Add("Release_Number");
dt.Columns.Add("WCO_Invoice_No");
dt.Columns.Add("WCO_Invoice_Date");
dt.Columns.Add("Customer_Billed_Amt");
dt.Columns.Add("Balance_Remaining_to_Bill");
dt.Columns.Add("Supplier_Invoice_Number");
dt.Columns.Add("Supplier_Invoice_Date");
dt.Columns.Add("Supplier_Paid_Amt");
dt.Columns.Add("Remaining_Cost_Dollar_Balance");
foreach (var expRow in dtToExport.Rows)
{
var row = dt.NewRow();
row["Supplier"] = expRow["Supplier_Name"];
//repeat for all columns you want.
dt.Rows.Add(row);
}
return dt;
}
Well, of course, because you finally override dt completely. You should enumerate the rows in dtToExport in a loop and create new rows in dt and assign the values for every field you need.
The problem: inserting rows only if primary key is bigger than existing one when merging source DataTable to actual DataTable (ActualDT.Merge(SourceDT)).
Details of my problem below:
I fill an Actual DataTable with an Int64 primary key by the API from external server after deserializing JSON to Source DataTable. Then I write rows from DataTable to my database and cleanup all rows in DataTable except the biggest primary key. Later I request new data from the API and often the response contains the same rows I already wrote to database and cleanup from my DataTable.
If I won't cleanup the DataTable rows, performance decrease and it's memory pig. So, I leave one row with the biggest primary key after cleaning.
I don't want to compare every PrimaryKey from Source DataTable before merge, comparing can take a lot of time.
What should I do to prevent merging rows that I already wrote to database and removed from Actual DataTable? Maybe I can exclude them even at deserialisation process (I use NewtonSoft JSON.net)? Or any zippy way to prevent merging rows if they primary key < primary key in Actual DataTable?
Thanks for your answers!
UPDATE: merging code
public class MyData
{
DataTable BlackPairs = new DataTable();
DataTable WhiteTable = new DataTable();
public string _Json {
set
{
DataSet TempDS = JsonConvert.DeserializeObject<DataSet>(value);
try
{
foreach (DataTable table in TempDS.Tables)
{
BlackPairs = table.Copy();
WhiteTable.Merge(BlackPairs);
}
}catch{}
}
}
public MyData()
{ //columns initialization
WhiteTable.Columns.AddRange(new DataColumn[]{columns);
WhiteTable.PrimaryKey = new DataColumn[]{tid};
}
I have created custom Merge function based on what we have talked through comments. This function is only if primary column is typeof(int) but it can be easily improved to get all types or just change it to what type you need (string, int, bool...)
public Test()
{
InitializeComponent();
DataTable smallerDatatable = new DataTable();
smallerDatatable.Columns.Add("Col1", typeof(int));
smallerDatatable.Columns.Add("Col2", typeof(string));
DataTable biggerDatatable = new DataTable();
biggerDatatable.Columns.Add("Col1", typeof(int));
biggerDatatable.Columns.Add("Col2", typeof(string));
smallerDatatable.Rows.Add(1, "Row1");
smallerDatatable.Rows.Add(2, "Row2");
smallerDatatable.Rows.Add(3, "Row3");
biggerDatatable.Rows.Add(1, "Row1");
biggerDatatable.Rows.Add(2, "Row2");
biggerDatatable.Rows.Add(3, "Row3");
biggerDatatable.Rows.Add(4, "Row4");
biggerDatatable.Rows.Add(5, "Row5");
DataTable mergedTable = MergeOnUniqueColumn(smallerDatatable, biggerDatatable, "Col1");
dataGridView1.DataSource = mergedTable;
}
private DataTable MergeOnUniqueColumn(DataTable smallTable, DataTable bigTable, string uniqueColumn)
{
DataTable m = smallTable;
for(int i = 0; i < bigTable.Rows.Count; i++)
{
if(!(smallTable.AsEnumerable().Any(row => bigTable.Rows[i][uniqueColumn].Equals(row.Field<object>(uniqueColumn)))))
{
smallTable.Rows.Add(bigTable.Rows[i].ItemArray);
}
}
return m;
}
Function above will fill every missing unique value inside smallTable from bigTable.
If you want to fill smallTable with values from bigTable only after last smallTable row then use this function.
private DataTable MergeOnUniqueColumnAfterLastID(DataTable smallTable, DataTable bigTable, string uniqueColumn)
{
DataTable m = smallTable;
int maxUnique = Convert.ToInt32(m.Compute("max([" + uniqueColumn + "])", string.Empty));
for (int i = 0; i < bigTable.Rows.Count; i++)
{
if (!(smallTable.AsEnumerable().Any(row => (int)bigTable.Rows[i][uniqueColumn] <= maxUnique)))
{
smallTable.Rows.Add(bigTable.Rows[i].ItemArray);
}
}
return m;
}
How can I copy 1 data column from 1 data table to a new datatable. When I try to do it, I get the error Column 'XXX' already belongs to another DataTable.?
dataColumn = datatable1.Columns[1];
datatable2 = new DataTable();
datatable2.Columns.Add(dataColumn);
Thanks in Advance
You cannot copy DataColumns. What you'll need to do is create a new DataColumn in the new datatable with the same data type as in the old datatable's column, and then you need to run a FOR loop to bring in all the data from the old datatable to the new datatable.
See the following code. This assumes that the datatables have exactly the same number of rows.
DataTable dt1 = new DataTable();
DataTable dt2 = new DataTable();
dt2.Columns.Add("ColumnA", dt1.Columns["ColumnA"].DataType);
for (int i = 0; i < dt1.Rows.Count; i++)
{
dt2.Rows[i]["ColumnA"] = dt1.Rows[i]["ColumnA"];
}
Also, If the data you are copying are reference types and not value types you might want to see if a .Clone() method is available for the type, or make one yourself. Just doing 'this = that' in the FOR loop will not work on reference types.
You cannot copy a DataColumn. (DataColumns are very tightly coupled with their tables)
Instead, you can add a new column with the same name and datatype.
You might be looking for DataTable.Clone(), which will create a structual copy of an entire table. (With the same schema, but no data)
Just a thought, are your DataTables both in the same DataSet?
If so, you can create a named DataRelation between the columns of two tables (think foreign key).
Then you can add a Calculated DataColumn to your table that has its Expression property set to "Child(RelationName).ColumnName" or "Parent(RelationName).ColumnName" depending on the direction of the relationship.
This will give you the same effect as copying the column, but I believe it only evaluates it lazily. So maybe it will give you what you need.
There is an example here of how this works. The example uses the Sum aggregate function, but you just need to reference the column name and it will duplicate it in your DataTable
myDataSet.Relations.Add(
"Orders2OrderLines",
myDataSet.Tables["Orders"].Columns["OrderID"],
myDataSet.Tables["OrderLines"].Columns["OrderID"]);
ordersTable.Columns.Add("OrderTotal", typeof(decimal), "Sum(Child(Orders2OrderLines).ExtendedPrice)");
HTH
The problem is caused by the c# can not reuse the object instance created and uses it on multiples DataTables. For this it is necessary to create a new object DataCollumn for each loop iteration.
foreach (DataTable table in DATASET.Tables)
{
DataColumn yourDataCollumn = new DataColumn("Name of DataCollumn", typeof(Your data type));
// your logic here
}
Hope it's help...
I used the below to merge two tables using mostly LINQ and only looping through the rows at the end. I wouldn't call it pretty but it does work. Using the join to prevent some of the assumptions listed above.
DataTable tableOne = getTableOne();
DataTable tableTwo = getTableTwo();
var oneColumns = tableOne.Columns.Cast<DataColumn>()
.Select(p => new Column(p.ColumnName, DataType))
.ToArray();
var twoColumns = tableTwo.Columns.Cast<DataColumn>()
.Select(p => new DataColumn(p.ColumnName, p.DataType))
.ToArray();
var matches = (from a in tableOne.AsEnumerable()
join b in tableTwo.AsEnumerable() on a["column_name"] equals b["column_name"]
select a.ItemArray.Concat(b.ItemArray)).ToArray();
DataTable merged = new DataTable();
merged.Columns.AddRange(oneColumns);
merged.Columns.AddRange(twoColumns);
foreach (var m in matches) { merged.Rows.Add(m.ToArray()); }
No looping required , Refer this , Hope this should solve your problem...
DataTable dt = new DataTable();
//fill the dt here
DataTable dt2 = new DataTable();
string[] strCols = {"Column Name to copy"};
dt2 = dt.DefaultView.ToTable("newTableName", false, strCols);
I have table1 and table2 in a class..
public DataTable sampletable (DataTable table1,DataTable table2)
{
// How to return the two table(table1 and table2)
}
Advance thank you
public DataTable[] sampletable (DataTable table1,DataTable table2)
{
return new DataTable[] { table1, table2 };
}
Use an array. And to retrieve a particular table:
DataTable[] dtArray = sampletable (YourFirstDt, YourSecondDt);
DataTable table1 = dtArray[0];
DataTable table2 = dtArray[1];
Assuming they have the same schema, you can use the DataTable.Merge Method
public DataTable sampletable(DataTable table1, DataTable table2)
{
table1.Merge(table2);
return table1;
}
The Merge method is used to merge two DataTable objects that have
largely similar schemas. A merge is typically used on a client
application to incorporate the latest changes from a data source into
an existing DataTable. This allows the client application to have a
refreshed DataTable with the latest data from the data source.
The merge operation takes into account only the original table, and the
table to be merged. Child tables are not affected or included. If a
table has one or more child tables, defined as part of a relationship,
each child table must be merged individually.
When merging a new source DataTable into the target, any source rows
with a DataRowState value of Unchanged, Modified, or Deleted, is
matched to target rows with the same primary key values. Source rows
with a DataRowState value of Added are matched to new target rows with
the same primary key values as the new source rows.
You can use DataSet , Create a new DataSet and Add the multiple tables to it ,
For Eg-
DataSet Ds = new DataSet();
DataTable Dt1= new DataTable();
Ds.Tables.Add(Dt1)
you can add multiple tables and to access the datatable you can use the index ( eg -
Ds.Tables[0])
Hope this answers your question!!.
public DataSet Getdatasettables()
{
DataSet ds = new DataSet();
DataTable dt1 = new DataTable();
DataTable dt2 = new DataTable();
ds.Tables.Add(dt1);
ds.Tables.Add(dt2);
return ds;
}
If I have 2 DataTables (dtOne and dtTwo) and I want to merge them and put them in another DataTable (dtAll). How can I do this in C#? I tried the Merge statement on the datatable, but this returns void. Does Merge preserve the data? For example, if I do:
dtOne.Merge(dtTwo);
Does dtOne change or does dtTwo change and if either one changes, do the changes preserve?
I know I can't do this because Merge returns void, but I want to be able to store the Merger of both dtOne and dtTwo in dtAll:
//Will Not work, How do I do this
dtAll = dtOne.Merge(dtTwo);
The Merge method takes the values from the second table and merges them in with the first table, so the first will now hold the values from both.
If you want to preserve both of the original tables, you could copy the original first, then merge:
dtAll = dtOne.Copy();
dtAll.Merge(dtTwo);
Instead of dtAll = dtOne.Copy(); in Jeromy Irvine's answer you can start with an empty DataTable and merge one-by-one iteratively:
dtAll = new DataTable();
...
dtAll.Merge(dtOne);
dtAll.Merge(dtTwo);
dtAll.Merge(dtThree);
...
and so on.
This technique is useful in a loop where you want to iteratively merge data tables:
DataTable dtAllItems = new DataTable();
foreach(var item in items)
{
DataTable dtItem = getDataTable(item); // some function that returns a data table
dtAllItems.Merge(dtItem);
}
dtAll = dtOne.Copy();
dtAll.Merge(dtTwo,true);
The parameter TRUE preserve the changes.
For more details refer to MSDN.
DataTable dtAll = new DataTable();
DataTable dt= new DataTable();
foreach (int id in lst)
{
dt.Merge(GetDataTableByID(id)); // Get Data Methode return DataTable
}
dtAll = dt;
This is what i did for merging two datatables and bind the final result to the gridview
DataTable dtTemp=new DataTable();
for (int k = 0; k < GridView2.Rows.Count; k++)
{
string roomno = GridView2.Rows[k].Cells[1].Text;
DataTable dtx = GetRoomDetails(chk, roomno, out msg);
if (dtx.Rows.Count > 0)
{
dtTemp.Merge(dtx);
dtTemp.AcceptChanges();
}
}