We Keep Coding

Ordering a ConcurrentDictionary. Why is this not working?

We have a C# app that populates tables on worksheets within an Excel document.
The tables must be populated in the order the rows are returned from the database.
The object DataFileColData is defined as a List and contains the result set rows. For testing purposes, I'm only using [0] of the List.
Code segment #1 below doesn't work. Row order is not preserved in that the end result has the data displayed out of order although the numbers themselves are listed in order:
if (DataFileColData[0].Count() > 0)
{
ConcurrentDictionary<int, DataRow> theRows = new ConcurrentDictionary<int, DataRow>(9, DataFileColData[0].Count());
Parallel.For(0, DataFileColData[0].Count(), i =>
{
// go through each column
int c = 0;
try
{
foreach (var Col in DataFileColData)
{
var cell = Col[i];
if (cell != null)
{
if (cell.GetType().Name == "JArray") //If Jarray then table compression was used not column compression
{
if (theRows.TryAdd(i, Dt.NewRow()))
theRows[i].ItemArray = JsonConvert.DeserializeObject<object[]>(Col[i].ToString());
}
else
{
if (theRows.TryAdd(i, Dt.NewRow()))
theRows[i][c] = cell;
}
}
c++;
}
} //try
catch (Exception e)
{
throw new Exception("Exception thrown in \"PublicMethods.cs | RenderExcelFile\" while in foreach loop over DataFileColData: " + e.ToString());
}
} //for
); //parallel
//Add the rows to the datatable in their original order
//(might have gotten skewed from the parallel.for loop)
for (int x = 0; x < theRows.Count; x++)
Dt.Rows.Add(theRows[x]);
//Set the name so it appears nicely in the Excel Name Box dropdown instead of "table1", "table2", etc etc.
Dt.TableName = ExcelTableSpec.TableTitle + " " + r.TableID;
}
code segment #2 below does work with the row order and data associated with each row preserved :
if (DataFileColData[0].Count() > 0)
{
DataRow[] theRows = new DataRow[DataFileColData[0].Count()];
Parallel.For(0, DataFileColData[0].Count(), i =>
{
DataRow Rw = Dt.NewRow();
// go through each column
int c = 0;
try
{
foreach (var Col in DataFileColData)
{
var cell = Col[i];
if (cell != null)
{
if (cell.GetType().Name == "JArray") //If Jarray then table compression was used not column compression
{
lock (theRows)
{
theRows[i] = Dt.NewRow();
theRows[i].ItemArray = JsonConvert.DeserializeObject<object[]>(Col[i].ToString());
}
}
else
{
lock (theRows)
{
theRows[i] = Dt.NewRow();
theRows[i][c] = cell;
}
}
}
c++;
}
} //try
catch (Exception e)
{
throw new Exception("Exception thrown in \"PublicMethods.cs | RenderExcelFile\" while in foreach loop over DataFileColData: " + e.ToString());
}
} //for
); //parallel
//Add the rows to the datatable in their original order
//(might have gotten skewed from the parallel.for loop)
Dt = theRows.CopyToDataTable();
//Set the name so it appears nicely in the Excel Name Box dropdown instead of "table1", "table2", etc etc.
Dt.TableName = ExcelTableSpec.TableTitle + " " + r.TableID;
}
I don't understand why. I didn't think the locking mechanism would be needed because each thread gets its own instance of "i" and a ConcurrentDictionary is supposed to be thread safe.
Would someone be able to explain to me please why the code isn't working the way I think it should?
Thank you!
UPDATED CODE as per #Enigmativity's comments below.
The MSDN documentation isn't quite clear (to me anyway), but does appear to update the DataTable even though the MSDN documentation doesn't indicate it does when executing the NewRow() method.
New working code below:
if (DataFileColData[0].Count() > 0)
{
DataRow[] theRows = new DataRow[DataFileColData[0].Count()];
Parallel.For(0, DataFileColData[0].Count(), i =>
//for (int i = 0; i < DataFileColData[0].Count(); i++)
{
lock (Dt)
{
theRows[i] = Dt.NewRow();
}
// go through each column
int c = 0;
try
{
foreach (var Col in DataFileColData)
{
var cell = Col[i];
if (cell != null)
{
if (cell.GetType().Name == "JArray") //If Jarray then table compression was used not column compression
{
theRows[i].ItemArray = JsonConvert.DeserializeObject<object[]>(Col[i].ToString());
}
else
{
theRows[i][c] = cell;
}
}
c += 1;
} //foreach
} //try
catch (Exception e)
{
throw new Exception("Exception thrown in \"PublicMethods.cs | RenderExcelFile\" while in foreach loop over DataFileColData: " + e.ToString());
}
} //for
); //parallel
//Add the rows to the datatable in their original order
//(might have gotten skewed from the parallel.for loop)
Dt = theRows.CopyToDataTable();
//Set the name so it appears nicely in the Excel Name Box dropdown instead of "table1", "table2", etc etc.
Dt.TableName = ExcelTableSpec.TableTitle + " " + r.TableID;
//cleanup
if (theRows != null)
Array.Clear(theRows, 0, theRows.Length);
theRows = null;
} //if (DataFileColData[0].Count() > 0)

Please see the documentation for (MSDN Data Tables).
The key point is:
Thread Safety
This type is safe for multithreaded read operations. You must
synchronize any write operations.
So it's not i the the ConcurrentDictionary causing your issues.
I've decompiled the NewRow method and there is a call to NewRow(int record). This code clearly shows write operations.
internal DataRow NewRow(int record)
{
if (-1 == record)
record = this.NewRecord(-1);
this.rowBuilder._record = record;
DataRow row = this.NewRowFromBuilder(this.rowBuilder);
this.recordManager[record] = row;
if (this.dataSet != null)
this.DataSet.OnDataRowCreated(row);
return row;
}

How to find rowindex of a datatable into another datatable?

I am using C#. I have two data tables and i want to find the rows of first data table into second data table.
Example.
First data table's data:
1 inam
2 sohan
Second data tables's data:
3 ranjan
1 inam
2 sohan
Now i want to know the index of first two rows of first data table into second data table.
Please help guys.
Any answer or advice
Best Regards

You can use following extension method which returns the first index of a "sub-sequence":
// I've used String.Join to get something that is comparable easily
// from the ItemArray that is the object-array of all fields
IEnumerable<string> first = table1.AsEnumerable()
.Select(r => string.Join(",",r.ItemArray)); //
IEnumerable<string> second = table2.AsEnumerable()
.Select(r => string.Join(",", r.ItemArray));
int index = second.IndexOfSequence(first, null); // 1
Here the extension:
public static int IndexOfSequence<TSource>(this IEnumerable<TSource> input, IEnumerable<TSource> sequence, IEqualityComparer<TSource> comparer)
{
if (input == null) throw new ArgumentNullException("input");
if (sequence == null) throw new ArgumentNullException("sequence");
if (!sequence.Any()) throw new ArgumentException("Sequence must not be empty", "sequence");
if (comparer == null)
{
comparer = EqualityComparer<TSource>.Default;
}
int index = -1;
int firstIndex = -1;
bool found = false;
TSource first = sequence.First();
using (IEnumerator<TSource> enumerator = input.GetEnumerator())
{
using (IEnumerator<TSource> enumerator2 = sequence.GetEnumerator())
{
enumerator2.MoveNext();
while (enumerator.MoveNext())
{
index++;
found = comparer.Equals(enumerator.Current, enumerator2.Current);
if (found && firstIndex == -1) firstIndex = index;
if (found && !enumerator2.MoveNext())
return firstIndex;
}
}
}
return -1;
}
tested with this sample data:
var table1 = new DataTable();
table1.Columns.Add("ID", typeof(int));
table1.Columns.Add("Name");
var table2 = table1.Clone();
table1.Rows.Add(1, "inam");
table1.Rows.Add(2, "Sohan");
table2.Rows.Add(3, "ranjan");
table2.Rows.Add(1, "inam");
table2.Rows.Add(2, "Sohan");

If you don't have much volume this might work....
var tableOneIndex = -1;
var tableTwoIndex = -1;
foreach (var tableOneRow in tableOne.Rows)
{
tableOneIndex++;
foreach (var tableTwoRow in tableTwo.Rows)
{
tableTwoIndex++;
if (tableOneRow["name"].ToString() == tableTwoRow["name"].ToString())
{
// Do whatever you wanted to do with the index values
}
}
}

As a simple solution, this should suffice:
// Create and populate data tables
DataTable dataTable1 = new DataTable();
dataTable1.Columns.Add("Name", typeof(string));
DataRow row1 = dataTable1.NewRow();
row1["Name"] = "Inam";
DataRow row2 = dataTable1.NewRow();
row2["Name"] = "Sohan";
dataTable1.Rows.Add(row1);
dataTable1.Rows.Add(row2);
DataTable dataTable2 = new DataTable();
dataTable2.Columns.Add("Name", typeof(string));
DataRow row3 = dataTable2.NewRow();
row3["Name"] = "Ranjan";
DataRow row4 = dataTable2.NewRow();
row4["Name"] = "Inam";
DataRow row5 = dataTable2.NewRow();
row5["Name"] = "Sohan";
dataTable2.Rows.Add(row3);
dataTable2.Rows.Add(row4);
dataTable2.Rows.Add(row5);
// Loop through rows in first table
foreach (DataRow row in dataTable1.Rows)
{
int rowIndexInSecondTable = 0;
// Loop through rows in second table
for (int i = 0; i < dataTable2.Rows.Count; i++)
{
// Check if the column values are the same
if (row["Name"] == dataTable2.Rows[i]["Name"])
{
// Set the current index and break to stop further processing
rowIndexInSecondTable = i;
break;
}
}
// The index of the row in the second table is now stored in the rowIndexInSecondTable variable, use it as needed, for example, writing to the console
Console.WriteLine("Row with name '" + row["Name"] + "' found at index " + rowIndexInSecondTable.ToString());
}

Which way is the best for removing duplicates from a dataTable with multiple columns

Which way is the best for removing duplicates from a dataTable for multiple columns?I mean below code is only for a single column.
public DataTable RemoveDuplicateRows(DataTable dTable, string colName)
{
Hashtable hTable = new Hashtable();
ArrayList duplicateList = new ArrayList();
//Add list of all the unique item value to hashtable, which stores combination of key, value pair.
//And add duplicate item value in arraylist.
foreach (DataRow drow in dTable.Rows)
{
if (hTable.Contains(drow[colName]))
duplicateList.Add(drow);
else
hTable.Add(drow[colName], string.Empty);
}
//Removing a list of duplicate items from datatable.
foreach (DataRow dRow in duplicateList)
dTable.Rows.Remove(dRow);
//Datatable which contains unique records will be return as output.
return dTable;
}
I tried using string[] colName. It throws error at dTable.Rows.Remove(dRow);
Please suggest.

The easiest and most readable is using Linq-to-DataTable:
var groups = from r in dTable.AsEnumerable()
group r by new
{
Col1 = r.Field<String>("Column1"),
Col2 = r.Field<String>("Column2"),
};
// if you only want the first row of each group:
DataTable distinctTable = groups.Select(g => g.First()).CopyToDataTable();
Notes: Enumerable.GroupBy groups the DataRows by an anonymous type with two properties(Col1 and Col2) which are initialized from a DataRow fields Column1 and Column2.
So you get groups of IEnumerable<DataRow>. Enumerable.First() returns the first DataRow of each group (you could also use different methods to select the row you want to keep, for example by ordering by a date field).
Then CopyToDataTable creates a new DataTable from the (now) distinct DataRows.
Here's a possible implementation if you're using .NET 2:
implementation of a custom IEqualityComparer<Object[]> for the dictionary:
class ObjectArrayComparer : IEqualityComparer<Object[]>
{
public bool Equals(Object[] x, Object[] y)
{
if (x == null && y == null) return true;
if (x == null || y == null) return false;
if (x.Length != y.Length) return false;
for (int i = 0; i < x.Length; i++)
{
if (x[i] == null && y[i] == null) continue;
if (x[i] == null || y[i] == null) return false;
if (!x[i].Equals(y[i])) return false;
}
return true;
}
public int GetHashCode(Object[] obj)
{
int hash = 0;
if (obj != null)
{
hash = (hash * 17) + obj.Length;
foreach (Object o in obj)
{
hash *= 17;
if (o != null) hash = hash + o.GetHashCode();
}
}
return hash;
}
}
your RemoveDuplicateRows method:
public DataTable RemoveDuplicateRows(DataTable dTable, String[] colNames)
{
var hTable = new Dictionary<object[], DataRow>(new ObjectArrayComparer());
foreach (DataRow drow in dTable.Rows)
{
Object[] objects = new Object[colNames.Length];
for (int c = 0; c < colNames.Length; c++)
objects[c] = drow[colNames[c]];
if (!hTable.ContainsKey(objects))
hTable.Add(objects, drow);
}
// create a clone with the same columns and import all distinct rows
DataTable clone = dTable.Clone();
foreach (var kv in hTable)
clone.ImportRow(kv.Value);
return clone;
}
testing:
var table = new DataTable();
table.Columns.Add("Colum1", typeof(string));
table.Columns.Add("Colum2", typeof(int));
table.Columns.Add("Colum3", typeof(string));
Random r = new Random();
for (int i = 0; i < 100; i++)
{
table.Rows.Add("Colum1_" + r.Next(1, 10), r.Next(1, 10), "Colum3_" + r.Next(1, 10));
}
int rowCount = table.Rows.Count; // 100
var unique = RemoveDuplicateRows(table, new[] { "Colum1", "Colum2" });
int uniqueRowCount = unique.Rows.Count; // around 55-65

You can use Distinct on Datatable.Select ... Link
Refer this Link

Below is your code with several amends of mine.
The main idea is to add to HashTable(Dictionary<> in my code) not value of only one column but values of specified columns and threat them these several values in atomic way(like single one).
// your code with minor amends
public DataTable RemoveDuplicateRows(DataTable dTable, string[] colNames)
{
// note that strongly typed dictionary has replaced the hash table + it uses custom comparer
var hTable = new Dictionary<DataRowInfo, string>();
var duplicateList = new ArrayList();
//Add list of all the unique item value to hashtable, which stores combination of key, value pair.
//And add duplicate item value in arraylist.
foreach (DataRow drow in dTable.Rows)
{
var dataRowInfo = new DataRowInfo(drow, colNames);
if (hTable.ContainsKey(dataRowInfo))
duplicateList.Add(drow);
else
hTable.Add(dataRowInfo, string.Empty);
}
//Removing a list of duplicate items from datatable.
foreach (DataRow dRow in duplicateList)
dTable.Rows.Remove(dRow);
//Datatable which contains unique records will be return as output.
return dTable;
}
// Helper classes
// contains values of specified columns
internal sealed class DataRowInfo
{
public object[] Values { get; private set; }
public DataRowInfo(DataRow dataRow, string[] columns)
{
Values = columns.Select(c => dataRow[c]).ToArray();
}
public override bool Equals(object obj)
{
if (ReferenceEquals(this, obj))
return true;
var other = obj as DataRowInfo;
if (other == null)
return false;
return Equals(other);
}
private bool Equals(DataRowInfo other)
{
if (this.Values.Length != other.Values.Length)
return false;
for (int i = 0; i < this.Values.Length; i++)
{
if (AreObjectsEqual(this.Values[i], other.Values[i]))
return false;
}
return true;
}
private static bool AreObjectsEqual(object left, object right)
{
if (ReferenceEquals(left, right))
return true;
if (ReferenceEquals(left, null))
return false;
if (ReferenceEquals(right, null))
return false;
if (left.GetType() != right.GetType())
return false;
return left.Equals(right);
}
public override int GetHashCode()
{
unchecked
{
int hashCode = 0;
foreach (var value in this.Values)
{
hashCode = hashCode ^ ((value != null ? value.GetHashCode() : 0) * 397);
}
return hashCode;
}
}
}
Hope this will help.
UPDATE
Simplified code a bit.

Update a DataTable in C# without using a loop?

Let suppose there are three columns in my DataTable
code
name
color
If I know the code and name, how can I update the color of that specific row whose code and name match my criteria? I want to do this without using Loops!

You can use LINQ:
DataRow dr = datatable.AsEnumerable().Where(r => ((string)r["code"]).Equals(someCode) && ((string)r["name"]).Equals(someName)).First();
dr["color"] = someColor;
Of course I'm assuming all those criteria are strings. You should change the casts to the correct types.

// Use the Select method to find all rows matching the name and code.
DataRow[] rows = myDataTable.Select("name 'nameValue' AND code = 'codeValue');
for(int i = 0; i < rows.Length; i ++)
{
rows[i]["color"] = colorValue;
}

DataTable recTable = new DataTable();
// do stuff to populate table
recTable.Select(string.Format("[code] = '{0}' and [name] = '{1}'", someCode, someName)).ToList<DataRow>().ForEach(r => r["Color"] = colorValue);

With LINQ:
var dataRows = dt.AsEnumerable().Select(c => { c["color"] = c["Code"].ToString() == "1" ? "Red" : "White"; return c; });
dt = dataRows.CopyToDataTable();

You could do:
foreach (DataRow row in datatable.Rows)
{
if(row["code"].ToString() == someCode && row["name"].ToString() == someName)
{
row["color"] = someColor;
}
}

Compare datatables

I built an application which displays the records from database in the window and checks the the database for new records every couple of seconds. The problem is that the window blinks each time I check for new records and I want to fix it. I have tried to compare the old datatable with the new one and refresh only if they are different.
Does anyone know what is the best practice for such cases? I tried to do it the following way but it doesn't work:
private bool GetBelongingMessages()
{
bool result = false;
DataTable dtTemp = OleDbWorks.GetBelongingMessages(currentCallID);
if(dtTemp != dtMessages)
{
dtMessages = dtTemp;
result = true;
}
else
{
result = false;
}
return result;
}

First off, it's important to recognize that what you're comparing in your code is the references of the datatables, not the contents of the datatables. In order to determine if both datatables have the same contents, you're going to have to loop through all of the rows and columns and see if they're equal:
//This assumes the datatables have the same schema...
public bool DatatablesAreSame(DataTable t1, DataTable t2) {
if (t1.Rows.Count != t2.Rows.Count)
return false;
foreach (DataColumn dc in t1.Columns) {
for (int i = 0; i < t1.Rows.Count; i++) {
if (t1.Rows[i][dc.ColumnName] != t2.Rows[i][dc.ColumnName]) {
return false;
}
}
}
return true;
}

I've been trying to find a way to do DataTable comparison for a while and ended up writing up my own function, here is what I got:
bool tablesAreIdentical = true;
// loop through first table
foreach (DataRow row in firstTable.Rows)
{
foundIdenticalRow = false;
// loop through tempTable to find an identical row
foreach (DataRow tempRow in tempTable.Rows)
{
allFieldsAreIdentical = true;
// compare fields, if any fields are different move on to next row in tempTable
for (int i = 0; i < row.ItemArray.Length && allFieldsAreIdentical; i++)
{
if (!row[i].Equals(tempRow[i]))
{
allFieldsAreIdentical = false;
}
}
// if an identical row is found, remove this row from tempTable
// (in case of duplicated row exist in firstTable, so tempTable needs
// to have the same number of duplicated rows to be considered equivalent)
// and move on to next row in firstTable
if (allFieldsAreIdentical)
{
tempTable.Rows.Remove(tempRow);
foundIdenticalRow = true;
break;
}
}
// if no identical row is found for current row in firstTable,
// the two tables are different
if (!foundIdenticalRow)
{
tablesAreIdentical = false;
break;
}
}
return tablesAreIdentical;
Compared to Dave Markle's solution, mine treats two table with same records but in different orders as identical. Hope this helps whoever stumbles upon this thread again.

You have to cast objects t1.Rows[i][dc.ColumnName] and t1.Rows[i][dc.ColumnName] otherwise the statement t1.Rows[i][dc.ColumnName] != t2.Rows[i][dc.ColumnName] is always true. I modified the code the following way:
for(int i = 0; i < t1.Rows.Count; i++)
{
if((string)t1.Rows[i][1] != (string)t2.Rows[i][1])
return false;
}
and it works but it's not an elegant solution.

public Boolean CompareDataTables(DataTable table1, DataTable table2)
{
bool flag = true;
DataRow[] row3 = table2.Select();
int i = 0;// row3.Length;
if (table1.Rows.Count == table2.Rows.Count)
{
foreach (DataRow row1 in table1.Rows)
{
if (!row1.ItemArray.SequenceEqual(row3[i].ItemArray))
{
flag = false;
break;
}
i++;
}
}
else
{
flag = false;
}
return flag;
}
// here this function will gave boolean as result return true if both are same else return false if both are not same