We Keep Coding

Get DataRow if column names in a string array have matching values in a string array

I'm trying to get a DataRow from a dtResult datatable if column name in [colName] list has a matching value as [grbByValue] list. my goal in the below code is to get [test1] and [test2] return datarow from dtResult and should be the same as [update] (which is hard coded). but have issue in both test1 & test2. test1 has error and don't know how fix and test2 is returning null.
rule is a DataTable that looks like this:
All the below logic is run for each row of rule.
dtResult is also a DataTable that looks like this:
EDITED CODE
string[] grpby = { "ageband","gender","code"};
List<string> grbByValue = new List<string>() { "1","85+","1","1010"};
DataTable dtResult = new DataTable();
DataColumn dc = dtResult.Columns.Add("id", typeof(int));
dc.AutoIncrement = true;
dc.AutoIncrementSeed = 1;
dc.AutoIncrementStep = 1;
dtResult.Columns.Add("DataSourceID");
dtResult.Columns["DataSourceID"].DefaultValue = "1";
dtResult.Columns.Add("RuleID");
dtResult.Columns.Add("GroupBy0");
dtResult.Columns.Add("GroupBy1");
dtResult.Columns.Add("GroupBy2");
dtResult.Columns.Add("GroupBy3");
dtResult.Columns.Add("GroupBy4");
dtResult.Columns.Add("GroupBy5");
dtResult.Columns.Add("Result", typeof(decimal));
dtResult.Columns["Result"].DefaultValue = 0.00;
var colName = (from a in dtResult.Columns.Cast<DataColumn>()
where a.ColumnName.ToString().StartsWith("GroupBy")
select a.ColumnName).OrderBy(x => x).ToList();
colName.Insert(0, "RuleID");
colName = colName.GetRange(0, grbByValue.Count);
//comment/UNCOMMENT below to test [test1]
//DataRow z = dtResult.NewRow();
//for (int i = 0; i < grbByValue.Count; i++)
//{
// z[colName[i]] = grbByValue[i];
//}
//dtResult.Rows.Add(z.ItemArray);
var distDtResult = dtResult.DefaultView.ToTable(true, colName.ToArray());
bool exist = false;
DataRow update = null;
foreach (DataRow dr in distDtResult.Rows)
{
var row = dr.ItemArray.ToList();
exist = row.SequenceEqual(grbByValue);
if (exist == true)
{
//var test1 = (from t1 in distDtResult.AsEnumerable().Where(r => r.ItemArray == dr.ItemArray)
// join t2 in (from m in dtResult.AsEnumerable()
// select new
// {
// //ideally the below column list will be derived from [colName] dynamically
// RuleID = m.Field<string>("RuleID"),
// GroupBy0 = m.Field<string>("GroupBy0"),
// GroupBy1 = m.Field<string>("GroupBy1"),
// GroupBy2 = m.Field<string>("GroupBy2")
// }) on t1.ItemArray equals t2.ItemArray
// select new
// {
// t2
// }).FirstOrDefault();
update = dtResult.AsEnumerable().Where(r =>
r.Field<int>("id") == 1 &&
r.Field<string>("DataSourceID") == "1" &&
r.Field<string>("RuleID") == "1" &&
r.Field<string>("GroupBy0") == "85+" &&
r.Field<string>("GroupBy1") == "1" &&
r.Field<string>("GroupBy2") == "1010").FirstOrDefault();
break;
}
}
if (exist == false)
{
DataRow a = dtResult.NewRow();
for (int i = 0; i < grbByValue.Count; i++)
{
a[colName[i]] = grbByValue[i];
}
dtResult.Rows.Add(a.ItemArray);
var test2 = dtResult.AsEnumerable().Where(r => r.ItemArray.Equals(a.ItemArray)).FirstOrDefault();
update = dtResult.AsEnumerable().Where(r =>
r.Field<int>("id") == 1 &&
r.Field<string>("DataSourceID") == "1" &&
r.Field<string>("RuleID") == "1" &&
r.Field<string>("GroupBy0") == "85+" &&
r.Field<string>("GroupBy1") == "1" &&
r.Field<string>("GroupBy2") == "1010").FirstOrDefault();
}

This might be a good starting point, at least to better ask questions and move towards an answer.
string[] colName = { "RuleID", "GroupBy0", "GroupBy1", "GroupBy2" };
// "All the below logic is run for each row of rule"
// this goes through each row of the rule DataTable
foreach (DataRow rule in ruleTable.Rows)
{
// This is going to be equivalent to the grpby variable you specified
var groupRules = rule.Field<string>("GroupBy").ToString().Split("|");
// Some sort of mapping may need to go here to go from "ageband" to "GroupBy0", "gender" to "GroupBy1", etc.
foreach(DataRow row in dtResult.Rows)
{
DataTable distDtResult = dtResult.DefaultView.ToTable(true, colName);
var updateTEST = from dr in distDtResult.AsEnumerable()
where dr.Field<string>("RuleID") == rule["RuleID"].ToString()
&& dr.Field<string>("GroupBy0") == row["GroupBy0"].ToString() // ageband
&& dr.Field<string>("GroupBy1") == row["GroupBy1"].ToString() // gender
&& dr.Field<string>("GroupBy2") == row["GroupBy2"].ToString() // code
// more
select dr;
}
}

Linq group by multiple columns, one of which is selectable

I have a table consisting of
MyDate(DATETIME),
Type nvarchar(255),
PropertyAId(int),
PropertyBId(int),
Data1 (float),
Data2 (float),
...
Data50 (float)
I want to return a table grouped by
MyDate,Type, and one of PropertyAId,PropertyBId (depending on user selection)
and sum all of the Data columns.I would prefer summing and selecting Data columns based on prefix or data type, and not having to repeat the same line 50 times
What I have so far is a bit ugly
DataTable dt2 = dt.Clone();
var grouped = dt.AsEnumerable().
GroupBy(r => new
{
MyDate = r.Field<DateTime>("MyDate"),
PropertyAId = selectedGroupingColumn == "PropertyAId" ? r.Field<int?>("PropertyAId") : null,
PropertyBId = selectedGroupingColumn == "PropertyBId" ? r.Field<int?>("PropertyBId") : null,
Type = r.Field<string>("Type")
});
foreach (var group in grouped)
{
DataRow row = dt2.NewRow();
foreach (var col in dt.Columns.Cast<DataColumn>())
{
if (col.ColumnName.StartsWith("Data"))
{
double sum = 0;
if (col.DataType == typeof(double))
sum = group.Sum(r => r.Field<double>(col));
row.SetField(col.ColumnName, sum);
}
else
row[col.ColumnName] = group.First()[col];
}
dt2.Rows.Add(row);
}
//dt2.Columns.Remove unselected property and return table

Your logic seems (mostly) sound, but I would suggest just using selectedGroupingColumn directly to get the value to group by, and only set the sum for "Data" columns of type double:
DataTable dt2 = dt.Clone();
var grouped = dt.AsEnumerable().
GroupBy(r => new {
MyDate = r.Field<DateTime>("MyDate"),
GroupingColumnValue = r.Field<int?>(selectedGroupingColumn),
Type = r.Field<string>("Type")
});
foreach (var group in grouped) {
DataRow row = dt2.NewRow();
foreach (var col in dt.Columns.Cast<DataColumn>())
if (col.ColumnName.StartsWith("Data") && col.DataType == typeof(double))
row.SetField(col.ColumnName, group.Sum(r => r.Field<double>(col)));
else
row[col.ColumnName] = group.First()[col];
dt2.Rows.Add(row);
}

Linq Extension Methods and Error Handling

I have the following C# code...
// We're essentially pivoting the data, using LINQ's GroupBy.
var pivotedOperands = Operands.GroupBy(o => new { outputid = (Guid)o[DETAILS_OUTPUTID], unitid = o[DETAILS_UNITID] })
.Select(g => new
{
PivotKey = g.Key,
c1 = g.Where(x => (int)x[DETAILS_OV_SEQUENCEID] == 1).Sum(x => double.Parse(x[DETAILS_VALUE].ToString())),
r1 = g.Where(x => (int)x[DETAILS_OV_SEQUENCEID] == 2).Sum(x => double.Parse(x[DETAILS_VALUE].ToString())),
a1 = g.Where(x => (int)x[DETAILS_OV_SEQUENCEID] == 3).Sum(x => double.Parse(x[DETAILS_VALUE].ToString()))
});
It takes the data in Operands (which is a List object) and uses the GroupBy() extension method to perform a pivot on the data. Essentially c1, r1 and a1 are all values in different DataRow objects with sequence IDs of 1, 2 and 3 respectively. (I can go into more detain on that if it becomes necessary, but I think it won't.)
So sometimes the value for c1 might be empty. (It's not supposed to, but bugs have happened further upstream in the process from time to time.) If c1 is not a numeric value, the double.Parse() call will raise an exception. That's fine. Here's my problem. If the Operands object contains, for example, 9 rows that will be pivoted into 3 rows and one of those nine values is not numeric, is it possible to determine which DataRow object raised the exception?
example:
If Operands contains the following values for SequenceID and Value...
OutputID UnitID SequenceID Value
A 1 1 '0'
A 1 2 '0'
A 1 3 '0'
A 2 1 ''
A 2 2 '0'
A 2 3 '0'
B 1 1 '0'
B 1 2 '0'
B 1 3 '0'
...then we will get an "Input string was not in a correct format" exception when it tries to process the empty string through the double.Parse() method for the 4th row of my data set. I want to raise a friendly exception to the users telling them which row is the problem; not just that there was a problem somewhere in this set of data. Is it possible to identify exactly what caused the exception?
If you create a new C# console application in Visual studio and dump the following code into the Main method, you will be able to reproduce my problem.
// Create a DataTable so that we can easily create new DataRows to add to our List.
DataTable dt = new DataTable();
DataColumn col = new DataColumn();
col.DataType = System.Type.GetType("System.String");
col.ColumnName = "OutputID";
dt.Columns.Add(col);
col = new DataColumn();
col.DataType = System.Type.GetType("System.Int32");
col.ColumnName = "UnitID";
dt.Columns.Add(col);
col = new DataColumn();
col.DataType = System.Type.GetType("System.Int32");
col.ColumnName = "SequenceID";
dt.Columns.Add(col);
col = new DataColumn();
col.DataType = System.Type.GetType("System.String");
col.ColumnName = "Value";
dt.Columns.Add(col);
// Create the List and add our sample data
List<DataRow> Operands = new List<DataRow>();
DataRow dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "1";
dr["SequenceID"] = 1;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "1";
dr["SequenceID"] = 2;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "1";
dr["SequenceID"] = 3;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "2";
dr["SequenceID"] = 1;
dr["Value"] = ""; // This should cause an error.
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "2";
dr["SequenceID"] = 2;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "A";
dr["UnitID"] = "2";
dr["SequenceID"] = 3;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "B";
dr["UnitID"] = "1";
dr["SequenceID"] = 1;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "B";
dr["UnitID"] = "1";
dr["SequenceID"] = 2;
dr["Value"] = "0";
Operands.Add(dr);
dr = dt.NewRow();
dr["OutputID"] = "B";
dr["UnitID"] = "1";
dr["SequenceID"] = 3;
dr["Value"] = "0";
Operands.Add(dr);
// Now pivot the data
try
{
var pivotedOperands = Operands.GroupBy(o => new { outputid = o[0], unitid = o[1] })
.Select(g => new
{
PivotKey = g.Key,
c1 = g.Where(x => (int)x[2] == 1).Sum(x => double.Parse(x[3].ToString())),
r1 = g.Where(x => (int)x[2] == 2).Sum(x => double.Parse(x[3].ToString())),
a1 = g.Where(x => (int)x[2] == 3).Sum(x => double.Parse(x[3].ToString()))
});
foreach (var o in pivotedOperands)
{
Console.WriteLine(string.Format("c1 = {0}; r1 = {1}; a1 = {2}", o.c1, o.r1, o.a1));
}
}
catch (Exception ex)
{
Console.WriteLine(ex.Message);
}
Console.WriteLine("Done.");
Console.ReadLine();

Depending on how you want the information surfaced, you can either change the type of your results to account for the possibility of failure, or you can capture contextual information about the exception and throw a new exception with more information in it.
In either case, don't be afraid to use helper methods. For example, suppose you got rid of the repetitive code in your selector by creating a method like this:
string GetSumOrErrorMessage(int idToMatch, IEnumerable<DataRow> dataRow)
{
try
{
var sum = dataRow.Where(x => (int)x[2] == idToMatch).Sum(x => double.Parse(x[3].ToString()));
return sum.ToString();
}
catch (Exception)
{
return "Error happened here"; // or something more specific
}
}
Now you can change your query like this:
var pivotedOperands = Operands.GroupBy(o => new { outputid = o[0], unitid = o[1] })
.Select(g => new
{
PivotKey = g.Key,
c1 = GetSumOrErrorMessage(1, g),
r1 = GetSumOrErrorMessage(2, g),
a1 = GetSumOrErrorMessage(3, g)
});
And your output turns into:
c1 = 0; r1 = 0; a1 = 0
c1 = Error happened here; r1 = 0; a1 = 0
c1 = 0; r1 = 0; a1 = 0
If you like this pattern, rather than just returning a string you may want to look into specialized Monadic types that can help with this. For example, you could create a class that has a generic value when an action is successful, or an error message when it's not. You can create a variety of extension methods and helpers to make this easier to deal with, similar to how my CallMeMaybe library would allow you to attempt to parse a value, but just return an empty Maybe<> if parsing fails. (e.g. Maybe.From(x[3].ToString()).ParseInt64().Select(i => i.ToString()).Else("Error happened here")).
Alternatively, if you actually want to halt when you get bad input, but still want to know where the bad input was, you can catch and throw:
double GetSum(int idToMatch, IGrouping<object, DataRow> dataRows)
{
try
{
return dataRows.Where(x => (int)x[2] == idToMatch).Sum(x => double.Parse(x[3].ToString()));
}
catch (Exception e)
{
throw new Exception($"Failure when matching {idToMatch} with group {dataRows.Key}", e);
}
}
...
var pivotedOperands = Operands.GroupBy(o => new { outputid = o[0], unitid = o[1] })
.Select(g => new
{
PivotKey = g.Key,
c1 = GetSum(1, g),
r1 = GetSum(2, g),
a1 = GetSum(3, g)
});
Output:
c1 = 0; r1 = 0; a1 = 0
Failure when matching 1 with group { outputid = A, unitid = 2 }

You try using TryParse to get around the exception. If TryParse is false then default to zero (0)
.Sum(x => {
double value = 0;
return double.TryParse(x[DETAILS_VALUE].ToString(), out value) ? value : 0;
})

Getting an error on pivot table using linq

I did pivot table from normal datatable using linq. However sometimes, i get an errors like;
There is no row at position 5(or any number like 7,8,11 etc..)
-----------------------------------------------------------
object reference not set to an instance of an object
What i didn't understand it, Why sometimes it works very well, with the same conditions like 'sample_time' etc. and why sometimes doesn't work.
Here is the my code to swap rows to columns with linq. Also when i debug, I found the part that throws an error.
var dtExist = dtTopAll.AsEnumerable().Where(l => l.Field<DateTime>("SAMPLE_TIME") >= minDate && l.Field<DateTime>("SAMPLE_TIME") <= maxDate).Any();
if (dtExist == true)
{
var dt = (from dr1 in dtTopAll.AsEnumerable()
where dr1.Field<DateTime>("SAMPLE_TIME") >= minDate && dr1.Field<DateTime>("SAMPLE_TIME") <= maxDate
group dr1 by new
{
WAIT_CLASS = dr1.Field<string>("WAIT_CLASS"),
SAMPLE_TIME = dr1.Field<DateTime>("SAMPLE_TIME")
} into g
select new
{
SAMPLE_TIME = g.Key.SAMPLE_TIME,
WAIT_CLASS = g.Key.WAIT_CLASS,
WAITS = g.Sum(z => z.Field<double>("WAITS") / 100)
}).ToDataTable();
var groups = dt.AsEnumerable().GroupBy(x => x.Field<DateTime>("SAMPLE_TIME")).ToList();
foreach (var group in groups) // From this line to end, throws an error .
{
DataRow newRow = dtPivotCustom.Rows.Add();
newRow[0] = group.Key;
foreach (string item in items)
{
newRow[item] = group.Where(x => x.Field<string>("WAIT_CLASS") == item).Select(x => x.Field<double>("WAITS")).Sum();
}
}
}

Update a DataTable in C# without using a loop?

Let suppose there are three columns in my DataTable
code
name
color
If I know the code and name, how can I update the color of that specific row whose code and name match my criteria? I want to do this without using Loops!

You can use LINQ:
DataRow dr = datatable.AsEnumerable().Where(r => ((string)r["code"]).Equals(someCode) && ((string)r["name"]).Equals(someName)).First();
dr["color"] = someColor;
Of course I'm assuming all those criteria are strings. You should change the casts to the correct types.

// Use the Select method to find all rows matching the name and code.
DataRow[] rows = myDataTable.Select("name 'nameValue' AND code = 'codeValue');
for(int i = 0; i < rows.Length; i ++)
{
rows[i]["color"] = colorValue;
}

DataTable recTable = new DataTable();
// do stuff to populate table
recTable.Select(string.Format("[code] = '{0}' and [name] = '{1}'", someCode, someName)).ToList<DataRow>().ForEach(r => r["Color"] = colorValue);

With LINQ:
var dataRows = dt.AsEnumerable().Select(c => { c["color"] = c["Code"].ToString() == "1" ? "Red" : "White"; return c; });
dt = dataRows.CopyToDataTable();

You could do:
foreach (DataRow row in datatable.Rows)
{
if(row["code"].ToString() == someCode && row["name"].ToString() == someName)
{
row["color"] = someColor;
}
}