We Keep Coding

C# - Two DataTable comparison. Get Rows which are Updated, Deleted and Created(New)

I have 2 DataTables: 1st DataTable is created today and 2nd Datatable was created yesterday.
Each DataTable contains over 100K rows and 30 columns.
I have a unique column - "Master_ID"
I want to compare and subtract (Today's DataTable - Yesterday's DataTable) and get the rows that are updated, deleted, and newly created. I want to perform a complete row-to-row comparison.
Output -> Dictionary <string, List DataRow >
Dictionary <"New Data Rows", List DataRow >
Dictionary <"Updated Data Rows", List DataRow >
Dictionary <"Deleted Data Rows", List DataRow >
I am looking for a time-efficient approach, probably using LINQ.

Probably one of the most efficient approaches is to use a dictionary-like collection to find out if a row-identifier is new, i'd use ILookup<TKey, TValue>:
public static Dictionary<string, List<DataRow>?> GetChanges(DataTable dtNew, DataTable dtOld, string masterKeyName, StringComparer masterKeyComparer = null)
{
IEqualityComparer<string> comparer = masterKeyComparer ?? StringComparer.Ordinal;
ILookup<string, DataRow> newKeyLookup = dtNew.AsEnumerable().ToLookup(r => r.Field<string>(masterKeyName), comparer!)!;
ILookup<string, DataRow> oldKeyLookup = dtOld.AsEnumerable().ToLookup(r => r.Field<string>(masterKeyName), comparer!)!;
List<DataRow> updatedRows = new();
List<DataRow> newRows = new();
List<DataRow> deletedRows = new();
foreach (var x in newKeyLookup)
{
List<DataRow> existingRows = oldKeyLookup[x.Key].ToList();
if (existingRows.Any())
{
// potential update, check if there are changes:
var allChangedRows = x.Where(r => !existingRows.Contains(r, DataRowComparer.Default));
updatedRows.AddRange(allChangedRows);
}
if (!existingRows.Any())
{
newRows.AddRange(x);
}
}
foreach (var x in oldKeyLookup)
{
if (!newKeyLookup[x.Key].Any())
{
deletedRows.AddRange(x);
}
}
return new Dictionary<string, List<DataRow>?>
{
{"New Data Rows", newRows},
{"Updated Data Rows", updatedRows},
{"Deleted Data Rows", deletedRows},
};
}

Probably this code will help you. It takes each row in the new DataTable and looks for it in the old DataTable (to get the created and updated rows) and then takes all rows in the old table which weren't found in new table (to get deleted rows). It will work very slowly if your tables contain over 100K rows, so you'd better put it into a separate thread.
My code will treat two different rows as one updated row if their Master_ID column is the same.
DataTable oldTable, newTable; // Put there your tables
Dictionary<int, DataRow> deletedRows, newRows = new Dictionary<int, DataRow>(); // Here will be the result (key — row's Master_ID, value — the row)
var updatedRows = new Dictionary<int, Tuple<DataRow, DataRow>>(); // Here will be the result (key — row's Master_ID, value — tuple (first item — old row version, second item — new row version))
var commonIds = new List<int>();
foreach (var row in newTable.Rows) {
var id = row["Master_ID"];
if (oldTable.Contains(id)) {
commonIds.Add((int)id);
var rowInOldTable = oldTable.Rows.Find(id);
foreach (var column in oldTable.Columns) {
if (row[column] != rowInOldTable[column]) {
updatedRows.Add((int)id, Tuple.Create<DataRow, DataRow>(rowInOldTable, row));
break;
}
}
} else {
newRows.Add((int)id, row);
}
}
deletedRows = (from row in oldTable.Rows
where !commonIds.Contains((int)row["Master_ID"]))
.ToDictionary<DataRow, int>(row => (int)row["Master_ID"]);

Looping through data table to get value

I have a DataTable with multiple rows. I'm using a foreach loop to loop through each item and return the name. This is returning the same (1st) value for each row. What have I done wrong?
DataTable table = new DataTable();
table.Columns.Add("tag", typeof(string));
string name = hfSelected.Value;
string[] names = name.Split(',');
for (int i = 0; i < names.Length; i++)
table.Rows.Add(new object[] { names[i] });
DataRow row = table.Rows[0];
foreach (var item in table.Rows)
{
Value = row["tag"].ToString() // this is returning the same value for both items in the table.
}

In a comment you mentioned that you get the error:
cannot apply indexing with [] to an expression of type object
when trying to access item["tag"] in the foreach loop.
You need to explicitly declare the DataRow in the foreach.
// declare DataRow here, not var
foreach (DataRow item in table.Rows)
{
// use item here
Value = item["tag"].ToString(); // use += to concatenate string
}
The reason is that the DataRowCollection implements a non-generic IEnumerable so you index an object instead of DataRow. The solution above casts to a DataRow.
I would recommend looking at the Field<T>() and AsEnumerable() methods from System.Data.DataSetExtensions. AsEnumerable() returns an IEnumerable<DataRow>. Field() provides strongly typed access to the values (ie it casts/converts the types for you).
Then you can do:
foreach (var item in table.AsEnumerable())
{
// item is a DataRow here
var myString = item.Field<string>("tag"); // gets string
// you can also do
var myInt = item.Field<int>("Id"); // gets int
var myDate = item.Field<DateTime?>("Date"); // gets nullable DateTime?
var myValue = item.Field<decimal>("Price"); // gets decimal
}

Carl is correct, this is producing the same output, because inside the iteration, you use the same row, all the time. You should use 'item', instead of 'row' there (you don't need 'row' at all).
The exception you receive is because you declared 'item' with a dynamic type, it's
foreach (var item in table.Rows)
You can try
foreach (DataRow item in table.Rows)
this way, you'll be able to get the column info.

your iteration seems to be using the same 'row' variable instead of the 'item' variable you defined in the foreach statement.

How to search for similar string in DataTable

I want to be able to detect 2 similar strings in my datatable. How do I do that?
foreach (DataRow row2 in visualDataTable.Rows)
{
foreach (char server in serverName)
{
foreach(similar string in servername)
{
// do something..
}
}
}

The simplest approach would involve iterating two times through your rows and compare each value with one and another (similar to your above approach):
foreach (DataRow row in visualDataTable.Rows)
{
foreach (DataRow compareRow in visualDataTable.Rows))
{
if(row["<your column>"] == compareRow["<your column>"])
{
// The two rows have the same column value for <your column>
// Do something
}
}
}

How To Remove Duplicates in Datatable it contains single column only?

Here i upload one Notepad file in Datatable.it contains only phone numbers and doesnot contain any Headers.so i want to Remove the duplicates from My Datatable.it contains single column only.
When i do this i get the result but one value will be Duplicated.
9988775566
9988556644
9966332200
9988775566
like this one value will get again.i want to get the Datatable with out Duplicates.
My Snippet is
public void duplicatesinnotepad(DataTable dt, string col)
{
ArrayList unique = new ArrayList();
ArrayList duplicat = new ArrayList();
foreach (DataRow de in dt.Rows)
{
if (unique.Contains(de[col]))
duplicat.Add(de);
else
unique.Add(de[col]);
}
foreach (DataRow de in duplicat)
{
dt.Rows.Remove(de);
}
}

Try this
using Linq To Get Distinct Elements
var list = (
from row in dt.AsEnumerable()
select row.Field<string>("contacts")).Distinct();
OR
var list = dt.AsEnumerable().
GroupBy(item => item.Field<string>(dt.Columns["contacts"]));

How to access to the content of a DataRow?

I have the following problem.
I have a DataTAble object that represent a single column table, the column is named VulnerabilityReferenceId, something like this:
VulnerabilityReferenceId
167554
167555
167556
167557
167558
167559
167560
167561
So I want create a foreach that access to these row and put the value into a variable
I have done:
foreach (DataRow row in _dt.Rows)
{
Debug.WriteLine("VulnerabilityReferenceId: " );
}
But what can I do to access to the value of the current row and put it into an int variable?

This could be an approach that read the field and convert it to the required datatype.
It requires the reference to DataSetExtension assembly from NET3.5 where you could start to find the DataRowExtensions class
foreach (DataRow row in _dt.Rows)
{
int id = row.Field<int>("VulnerabilityReferenceId");
.....
}
Note: I assume that the field VulnerabilityReferenceId is of type integer

You can use column name as an indexer to get the value as object
foreach (DataRow row in _dt.Rows)
{
int vulRefId=Convert.ToInt32(row["VulnerabilityReferenceId"]);
Debug.WriteLine("VulnerabilityReferenceId: " +vulRefId );
}

Try the below:
for(int i=0;i<_dt.Rows.Count;i++)
{
Debug.WriteLine("VulnerabilityReferenceId: "+dt.Rows[i][0].ToString());
}

foreach (DataRow row in _dt.Rows)
{
String stringVal = row["VulnerabilityReferenceId"].ToString();
int myId = Convert.ToInt32(stringVal);
}