We Keep Coding

C# - Two DataTable comparison. Get Rows which are Updated, Deleted and Created(New)

I have 2 DataTables: 1st DataTable is created today and 2nd Datatable was created yesterday.
Each DataTable contains over 100K rows and 30 columns.
I have a unique column - "Master_ID"
I want to compare and subtract (Today's DataTable - Yesterday's DataTable) and get the rows that are updated, deleted, and newly created. I want to perform a complete row-to-row comparison.
Output -> Dictionary <string, List DataRow >
Dictionary <"New Data Rows", List DataRow >
Dictionary <"Updated Data Rows", List DataRow >
Dictionary <"Deleted Data Rows", List DataRow >
I am looking for a time-efficient approach, probably using LINQ.

Probably one of the most efficient approaches is to use a dictionary-like collection to find out if a row-identifier is new, i'd use ILookup<TKey, TValue>:
public static Dictionary<string, List<DataRow>?> GetChanges(DataTable dtNew, DataTable dtOld, string masterKeyName, StringComparer masterKeyComparer = null)
{
IEqualityComparer<string> comparer = masterKeyComparer ?? StringComparer.Ordinal;
ILookup<string, DataRow> newKeyLookup = dtNew.AsEnumerable().ToLookup(r => r.Field<string>(masterKeyName), comparer!)!;
ILookup<string, DataRow> oldKeyLookup = dtOld.AsEnumerable().ToLookup(r => r.Field<string>(masterKeyName), comparer!)!;
List<DataRow> updatedRows = new();
List<DataRow> newRows = new();
List<DataRow> deletedRows = new();
foreach (var x in newKeyLookup)
{
List<DataRow> existingRows = oldKeyLookup[x.Key].ToList();
if (existingRows.Any())
{
// potential update, check if there are changes:
var allChangedRows = x.Where(r => !existingRows.Contains(r, DataRowComparer.Default));
updatedRows.AddRange(allChangedRows);
}
if (!existingRows.Any())
{
newRows.AddRange(x);
}
}
foreach (var x in oldKeyLookup)
{
if (!newKeyLookup[x.Key].Any())
{
deletedRows.AddRange(x);
}
}
return new Dictionary<string, List<DataRow>?>
{
{"New Data Rows", newRows},
{"Updated Data Rows", updatedRows},
{"Deleted Data Rows", deletedRows},
};
}

Probably this code will help you. It takes each row in the new DataTable and looks for it in the old DataTable (to get the created and updated rows) and then takes all rows in the old table which weren't found in new table (to get deleted rows). It will work very slowly if your tables contain over 100K rows, so you'd better put it into a separate thread.
My code will treat two different rows as one updated row if their Master_ID column is the same.
DataTable oldTable, newTable; // Put there your tables
Dictionary<int, DataRow> deletedRows, newRows = new Dictionary<int, DataRow>(); // Here will be the result (key — row's Master_ID, value — the row)
var updatedRows = new Dictionary<int, Tuple<DataRow, DataRow>>(); // Here will be the result (key — row's Master_ID, value — tuple (first item — old row version, second item — new row version))
var commonIds = new List<int>();
foreach (var row in newTable.Rows) {
var id = row["Master_ID"];
if (oldTable.Contains(id)) {
commonIds.Add((int)id);
var rowInOldTable = oldTable.Rows.Find(id);
foreach (var column in oldTable.Columns) {
if (row[column] != rowInOldTable[column]) {
updatedRows.Add((int)id, Tuple.Create<DataRow, DataRow>(rowInOldTable, row));
break;
}
}
} else {
newRows.Add((int)id, row);
}
}
deletedRows = (from row in oldTable.Rows
where !commonIds.Contains((int)row["Master_ID"]))
.ToDictionary<DataRow, int>(row => (int)row["Master_ID"]);

what is missing for DocumentNode function to show text in datagridview?

What i´m trying to do is to use the documentnode method to find a specific table from the internet and put it into datagridview. My code can be seen below:
`
List<string> list = new List<string>();
DataTable dt1 = new DataTable();
var table = doc.DocumentNode.SelectNodes("xpath link")
.Descendants("tr")
.Where(tr=>tr.Elements("td").Count()>1)
.Select(td => td.InnerText.Trim())
.ToList();
foreach (var tables in table)
{ list.Add(tables.ToString());}
dataGridView1.DataSource = list
`
The result I get in the table is a list of numbers instead of text (datagridview table). As I have tried to see it the text actually appears I changed the foreach with the following code:
`
foreach (var tables in table)
{
list.Add(tables.ToString());
richTextBox1.Text += tables;
}
`
The result I get from the change is a string of the table in richTextBox1 but still a table of numbers in datagridview1 richtextbox1 text. This means I´m getting the right table from the internet and its being loaded correctly but i´m still missing something for the datagridview1 as I get a list of numbers instead of text that is being shown in richtextbox1. I followed this up by changing the DocumentNode function with removing parts in the .select part of the code and the datagridview1 stilled showed numbers (I added for example .ToString, .ToList() etc.).
What exactly have I missed in my code that makes this happen and should I have added something else to make it show the text instead of numbers?
Edit:
New code.
`
List<string> list = new List<string>();
DataTable dt1 = new DataTable();
dt1.Columns.Add("td", typeof(int));
var table = doc.DocumentNode.SelectNodes("//div[#id=\"cr_cashflow\"]/div[2]/div/table")
.Descendants("tr")
.Select(td => td.InnerText.Trim())
.ToList();
foreach (var tables in table)
{
dt1.Rows.Add(new object[] { int.Parse(tables) });
}
dataGridView1.DataSource= dt1;
`

Try something like this
List<string> list = new List<string>();
DataTable dt1 = new DataTable();
dt1.Columns.Add("td",typeof(int));
var rows = doc.DocumentNode.SelectNodes("xpath link")
.Descendants("tr")
.Where(tr=>tr.Elements("td").Count()>1)
.Select(td => td.InnerText.Trim())
.ToList();
foreach (var row in rows)
{
dt.Rows.Add(new object[] { int.Parse(row)});
}

C# add only first column text in a table to a List<string> for assertion

I have this table and I want to verify files have been uploaded successfully
I want to iterate through the first column and add file names to a list to assert against an expected list
This works but I am wondering how I can modify my method to be able to iterate through all columns and rows and I can add any column to the list. Basically make the method more useable and not use it just to verify file names but also to be able to verify other columns if needed
public List<string> ListofFilesUploaded()
{
IWebElement table = WebDriver.Driver.FindElement(By.XPath("//table[#id='files_list']//tbody"));
IList<IWebElement> rows = table.FindElements(By.TagName("tr"));
List<string> fileNames = new List<string>();
foreach (var row in rows)
{
fileNames.Add(row.Text.Split(' ').First());
}
return fileNames;
}
Does anyone have an idea how to enhance this solution or improve this?

I believe instead of returning a list you can return a dictionary of lists with each document tile as key and list of all columns As value.
public Dictionary<string, List<string>> ListofFilesUploaded()
{
IWebElement table = WebDriver.Driver.FindElement(By.XPath("//table[#id='files_list']//tbody"));
IList<IWebElement> rows = table.FindElements(By.TagName("tr"));
Dictionary<string, List<string>> fileNames = new Dictionary<string, List<string>>();
foreach (var row in rows)
{
List<string> Col_value = new List<string>();
IList<IWebElement> cols= row.FindElements(By.TagName("td"));
foreach (var col in cols)
{
Col_value.Add( col.Text);
}
fileNames.Add(row.Get_Attribute(“title”), Col_value);
}
return fileNames;
}
Now you can iterate though dictionary to get list of all files upload and corroding column value for each file. Can see below link for same
What is the best way to iterate over a dictionary?

Instead of iterating over a List of strings for just fileNames, create a List of Files with properties as Name, size, modificationDateTime, numberOfDownloads, isDeleteable etc.
List<Files> files = new List<Files>();
Then you can iterate over each row that is represented by a File from the files list.

How to convert the DataGridView content into a List<> of custom objects?

I'm working on a Windows Forms application, written with C#.
I found someone's suggestion on how to create a List<> from DataGridView control, but I need little more help on how to extract cell values.
Here is the code given; let's say two columns in the dataGridView1 are Name and Address.
How to build the List<ProjList> object?
foreach (DataGridViewRow dr in dataGridView1.Rows)
{
ProjList = new List<ProjectMasterRec>();
foreach (DataGridViewCell dc in dr.Cells)
{
// build out MyItem
// based on DataGridViewCell.OwningColumn and DataGridViewCell.Value
// how do we code this?
}
ProjList.Add(item);
}

Try It this Way
Create a list of your class type
List<ProjectMasterRec>() ProjList = new List<ProjectMasterRec>();
Make Sure that type of list belongs to type of your data in Datagridview
foreach (DataGridViewRow dr in dataGridView1.Rows)
{
//Create object of your list type pl
ProjectMasterRec pl = new ProjectMasterRec();
pl.Property1 = dr.Cells[1].Value;
pl.Property2 = dr.Cells[2].Value;
pl.Property3 = dr.Cells[3].Value;
//Add pl to your List
ProjList.Add(pl);
}

If you are able to use LINQ, you can do something like this:
var projectList = (from row in dataGridView1.Rows.OfType<DataGridViewRow>()
select new ProjectMasterRec()
{ Name = row.Cells["Name"].Value.ToString(),
Address = row.Cells["Address"].Value.ToString()
}).ToList();

The following should work, myArray should be an array of the type you need, you can find out the array size using (DataGrid1.DataSource as BindingSource).List.Count.
(DataGrid1.DataSource as BindingSource).List.CopyTo(myArray, 0);

2-Column DataTable to List<int> .NET 2.0

I have populated a DataTable from a stored procedure in an older web application written in
C# under .NET 2.0 / Visual Studio 2005.
I'm trying to populate a List with the values in the DataTable, but I keep running up against a couple issues.
My conversion process looks like this:
List<int> SpecialVendorList = new List<int>();
foreach (DataRow datarow in GetAllSpecialVendors().Rows)
{
//Loop through each row
foreach (DataColumn column in GetAllSpecialVendors().Columns)
{
SpecialVendorList.Add(column["ChildVendorId"]);
SpecialVendorList.Add(column["ParentVendorId"]);
}
}
which gives me the following error:
Can not apply indexing with [] to an expression of type 'System.Data.DataColumn'
for each of the SpecialVendorList.Add() methods.

Seems like you're trying to get column values for each row. You only the first foreach loop:
List<int> SpecialVendorList = new List<int>();
try
{
foreach (DataRow datarow in GetAllSpecialVendors().Rows)
{
//Loop through each row
SpecialVendorList.Add(Convert.ToInt32(datarow["ChildVendorId"]));
SpecialVendorList.Add(Convert.ToInt32(datarow["ParentVendorId"]));
}
}
catch(FormatException fe)
{
//handle the error
}
The string index here will get that column's value in that specific row

you need to add the actual values from the rows using the column as the index:
List<int> SpecialVendorList = new List<int>();
foreach (DataRow datarow in GetAllSpecialVendors().Rows)
{
//Loop through each row
foreach (DataColumn column in GetAllSpecialVendors().Columns)
{
int val;
if (int.TryParse(datarow[column].ToString(), out val))
SpecialVendorList.Add(val);
}
}