We Keep Coding

C# Constructing a Dynamic Query From DataTable

Trying to Generate a Dynamic Linq Query, based on DataTable returned to me... The column names in the DataTable will change, but I will know which ones I want to total, and which ones I will want to be grouped.
I can get this to work with loops and writing the output to a variable, then recasting the parts back into a data table, but I'm hoping there is a more elegant way of doing this.
//C#
DataTable dt = new DataTable;
Dt.columns(DynamicData1)
Dt.columns(DynamicData1)
Dt.columns(DynamicCount)
In this case the columns are LastName, FirstName, Age. I want to total ages by LastName,FirstName columns (yes both in the group by). So one of my parameters would specify group by = LastName, FirstName and another TotalBy = Age. The next query may return different column names.
Datarow dr =..
dr[0] = {"Smith","John",10}
dr[1] = {"Smith","John",11}
dr[2] = {"Smith","Sarah",8}
Given these different potential columns names...I'm looking to generate a linq query that creates a generic group by and Total output.
Result:
LastName, FirstName, AgeTotal
Smith, John = 21
Smith, Sarah = 8

If you use a simple converter for Linq you can achieve that easily.
Here a quick data generation i did for the sample :
// create dummy table
var dt = new DataTable();
dt.Columns.Add("LastName", typeof(string));
dt.Columns.Add("FirstName", typeof(string));
dt.Columns.Add("Age", typeof(int));
// action to create easily the records
var addData = new Action<string, string, int>((ln, fn, age) =>
{
var dr = dt.NewRow();
dr["LastName"] = ln;
dr["FirstName"] = fn;
dr["Age"] = age;
dt.Rows.Add(dr);
});
// add 3 datarows records
addData("Smith", "John", 10);
addData("Smith", "John", 11);
addData("Smith", "Sarah", 8);
This is how to use my simple transformation class :
// create a linq version of the table
var lqTable = new LinqTable(dt);
// make the group by query
var groupByNames = lqTable.Rows.GroupBy(row => row["LastName"].ToString() + "-" + row["FirstName"].ToString()).ToList();
// for each group create a brand new linqRow
var linqRows = groupByNames.Select(grp =>
{
//get all items. so we can use first item for last and first name and sum the age easily at the same time
var items = grp.ToList();
// return a new linq row
return new LinqRow()
{
Fields = new List<LinqField>()
{
new LinqField("LastName",items[0]["LastName"].ToString()),
new LinqField("FirstName",items[0]["FirstName"].ToString()),
new LinqField("Age",items.Sum(item => Convert.ToInt32(item["Age"]))),
}
};
}).ToList();
// create new linq Table since it handle the datatable format ad transform it directly
var finalTable = new LinqTable() { Rows = linqRows }.AsDataTable();
And finally here are the custom class that are used
public class LinqTable
{
public LinqTable()
{
}
public LinqTable(DataTable sourceTable)
{
LoadFromTable(sourceTable);
}
public List<LinqRow> Rows = new List<LinqRow>();
public List<string> Columns
{
get
{
var columns = new List<string>();
if (Rows != null && Rows.Count > 0)
{
Rows[0].Fields.ForEach(field => columns.Add(field.Name));
}
return columns;
}
}
public void LoadFromTable(DataTable sourceTable)
{
sourceTable.Rows.Cast<DataRow>().ToList().ForEach(row => Rows.Add(new LinqRow(row)));
}
public DataTable AsDataTable()
{
var dt = new DataTable("data");
if (Rows != null && Rows.Count > 0)
{
Rows[0].Fields.ForEach(field =>
{
dt.Columns.Add(field.Name, field.DataType);
});
Rows.ForEach(row =>
{
var dr = dt.NewRow();
row.Fields.ForEach(field => dr[field.Name] = field.Value);
dt.Rows.Add(dr);
});
}
return dt;
}
}
public class LinqRow
{
public List<LinqField> Fields = new List<LinqField>();
public LinqRow()
{
}
public LinqRow(DataRow sourceRow)
{
sourceRow.Table.Columns.Cast<DataColumn>().ToList().ForEach(col => Fields.Add(new LinqField(col.ColumnName, sourceRow[col], col.DataType)));
}
public object this[int index]
{
get
{
return Fields[index].Value;
}
set
{
Fields[index].Value = value;
}
}
public object this[string name]
{
get
{
return Fields.Find(f => f.Name == name).Value;
}
set
{
var fieldIndex = Fields.FindIndex(f => f.Name == name);
if (fieldIndex >= 0)
{
Fields[fieldIndex].Value = value;
}
}
}
public DataTable AsSingleRowDataTable()
{
var dt = new DataTable("data");
if (Fields != null && Fields.Count > 0)
{
Fields.ForEach(field =>
{
dt.Columns.Add(field.Name, field.DataType);
});
var dr = dt.NewRow();
Fields.ForEach(field => dr[field.Name] = field.Value);
dt.Rows.Add(dr);
}
return dt;
}
}
public class LinqField
{
public Type DataType;
public object Value;
public string Name;
public LinqField(string name, object value, Type dataType)
{
DataType = dataType;
Value = value;
Name = name;
}
public LinqField(string name, object value)
{
DataType = value.GetType();
Value = value;
Name = name;
}
public override string ToString()
{
return Value.ToString();
}
}

I think I'd just use a dictionary:
public Dictionary<string, int> GroupTot(DataTable dt, string[] groupBy, string tot){
var d = new Dictionary<string, int>();
foreach(DataRow ro in dt.Rows){
string key = "";
foreach(string col in groupBy)
key += (string)ro[col] + '\n';
if(!d.ContainsKey(key))
d[key] = 0;
d[key]+= (int)ro[tot];
}
return d;
}
If you want the total on each row, we could get cute and create a column that is an array of one int instead of an int:
public void GroupTot(DataTable dt, string[] groupBy, string tot){
var d = new Dictionary<string, int>();
var dc = dt.Columns.Add("Total_" + tot, typeof(int[]));
foreach(DataRow ro in dt.Rows){
string key = "";
foreach(string col in groupBy)
key += (string)ro[col] + '\n'; //build a grouping key from first and last name
if(!d.ContainsKey(key)) //have we seen this name pair before?
d[key] = new int[1]; //no we haven't, ensure we have a tracker for our total, for this first+last name
d[key][0] += (int)ro[tot]; //add the total
ro[dc] = d[key]; //link the row to the total tracker
}
}
At the end of the operation every row will have an array of int in the "Total_age" column that represents the total for that First+Last name. The reason I used int[] rather than int, is because int is a value type, whereas int[] is a reference. Because as the table is being iterated each row gets assigned a reference to an int[] some of them with the same First+Last name will end up with their int[] references pointing to the same object in memory, so incrementing a later one increments all the earlier ones too (all "John Smith" rows total column holds a refernece to the same int[]. If we'd made the column an int type, then every row would point to a different counter, because every time we say ro[dc] = d[key] it would copy the current value of d[key] int into ro[dc]'s int. Any reference type would do for this trick to work, but value types wouldn't. If you wanted your column to be value type you'd have to iterate the table again, or have two dictionaries, one that maps DataRow -> total and iterate the keys, assigning the totals back into the row

Linq Loop Through Columns in Row C# [duplicate]

This question already has answers here:
Loop Through LINQ Query Columns (Not rows)
(3 answers)
Closed 8 years ago.
How do I loop through column names in Linq to SQL
var foo = (from f in db.bar select f).ToList()
What I want to do is loop through each column name, i.e.
foreach bar d in foo
{
foreach column in d
{<do something>}
}
Is that possible?

Here's a method that returns a LINQ-to-sql query as an enumerable of dictionaries:
IEnumerable<IDictionary<string, object>> GetValues<T>(DataContext context,
IQueryable<T> query)
where T : class
{
var propertyInfos = typeof(T).GetProperties().ToDictionary (pi => pi.Name);
foreach(var entity in context.GetTable<T>())
{
yield return (context.Mapping.GetTable(typeof(T)).RowType.DataMembers
.Where(dm => !dm.IsAssociation).Select (dm => dm.Name)
.Select(name => new { name,
value = propertyInfos[name].GetValue(entity, null)
})
.ToDictionary (x => x.name, y => y.value));
}
}
You get the column names and the values from the dictionaries:
var foo = (from f in db.bar select f);
foreach(var dictionary in GetValues(db, foo))
{
// do something with dictionary keys and/or values
}

If you haven't extended your mapped model with other fields, or don't mind iterating over them too, you could use:
var fields = typeof(Product).GetFields();
foreach(var item in db.Products)
{
foreach(var f in fields)
{
Console.WriteLine(f.Name + ": " + f.GetValue(item));
}
}

Here is an example of what I think you want, if this isn't good enough to get you started, please explain what you need in more detail.
var myTable = new DataTable();
myTable.Columns.Add("Column One");
myTable.Columns.Add("Column Two");
var newRow = myTable.NewRow();
newRow[0] = 11111;
newRow[1] = 22222;
myTable.Rows.Add(newRow);
var newRow2 = myTable.NewRow();
newRow2 [0] = 33333;
newRow2 [1] = 44444;
myTable.Rows.Add(newRow2);
foreach (var row in myTable.AsEnumerable())
{
foreach (DataColumn column in myTable.Columns)
{
Console.WriteLine("Column {0}, Row {1}, Value {2}", column.ColumnName, myTable.Rows.IndexOf(row), row[column]);
}
}

Re-arrange rows of Datatable in runtime

I have multiple rows in a datatable, see a sample below:
Existing Table
Name Date Value Type
ABC(I) 11/11/2013 12.36 I
DEF(I) 11/11/2013 1 I
GHI(I) -do- -do- I
JKL(P) P
MNO(P) P
PQR(D) D
STU(D) -d0- -do- D
Required Table
Name Date Value Type
JKL(P) P
MNO(P) P
PQR(D) D
STU(D) -d0- -do- D
ABC(I) 11/11/2013 12.36 I
DEF(I) 11/11/2013 1 I
GHI(I) -do- -do- I
COndition to use
Sorting should be as per the column Type. Now I need a small change in order of the rows to be shown in the gridview. That is rows of Payment will come first then all Dues and at last all Interests types will come.
What I tried:
Sorting of column but it was not what I need.
Custom Grouping suggested by Tim Schmelter here
Code was:
public DataTable GroupBy(string i_sGroupByColumn, string i_sAggregateColumn, DataTable i_dSourceTable)
{
DataView dv = new DataView(i_dSourceTable);
//getting distinct values for group column
DataTable dtGroup = dv.ToTable(true, new string[] { i_sGroupByColumn });
//adding column for the row count
dtGroup.Columns.Add("Count", typeof(int));
//looping thru distinct values for the group, counting
foreach (DataRow dr in dtGroup.Rows) {
dr["Count"] = i_dSourceTable.Compute("Count(" + i_sAggregateColumn + ")", i_sGroupByColumn + " = '" + dr[i_sGroupByColumn] + "'");
}
//returning grouped/counted result
return dtGroup;
}
I dont know where and what I am lacking/missing. Kindly help.

try linq to order your table:
var query = dtGroup.AsEnumerable()
.OrderBy(c=> c.Field<DateTime?>("Date"))
.ThenByDescending(c=> c.Field<string>("Name"));
DataView dv2 = query.AsDataView();

If I understand correctly you want first sorting on P, D, I and then on date
Dictionary<string, int> sortDictionary = new Dictionary<string, int>();
sortDictionary.Add("P", 1);
sortDictionary.Add("D", 2);
sortDictionary.Add("I", 3);
var q = from row in dtGroup.AsEnumerable()
let type = sortDictionary[row.Field<string>("Name").Substring(4, 1)]
orderby type, row.Field<string>("Name")
select row;
foreach (var r in q)
{
string x = r["Name"].ToString() + r["Date"].ToString();
}

select value from array at desired index using LINQ

i have an array which have 5 values from index 0 to 4. i want to store the values in my 5 model properties.
public IEnumerable<fields> ConvertTList(List<string[]> rows)
{
var tList = new List<fields>();
foreach (var item in rows)
{
var ListReading = new fields
{
//model properties names are:
// date, user,campaign,adgroup,changes
};
tList.Add(ListReading);
}
return (tList);
}
this is my code when foreach is executed item get 5 values. i want to store the values in model. how i can i store them using linq

Maybe you mean something like this:
public IEnumerable<fields> ConvertTList(List<string[]> rows)
{
return rows.Select(x => StringsToField(x));
}
fields StringsToField(string[] source)
{
return new fields
{
date = source[0],
user = source[1],
campaign = source[2],
adgroup = source[3],
changes = source[4],
};
}

How to select from DataTable with join to same table?

I have DataTable object, which holds some "tree data structure". Data is not stored in any database, I just use DataTable to manipulate data without SQL server.
My data looks like this (indents are only for better reading here):
DataTable dtCategories = GetCategoriesAsDataTable();
id name parentId
int string int
----------------------
1 One 0
2 OneA 1
3 OneB 1
4 Two 0
5 TwoA 4
6 TwoB 4
7 TwoAA 5
8 TwoAB 5
So far - I was thinking about selecting first level with "where parentId = 0" and putting this to separate DataTable, like this:
DataTable dtFirstLevel = dtCategories.Select("[parentId] = 0");
// and after this - create DataTable for second level
// but I don't know how can I use "IN" clause here
DataTable dtSecondLevel = dtCategories.Select(?????????);
How can I select only first 2 levels of tree?
How can I select this without SQL server (by using only data objects)?

Maybe this helps:
var rows = table.AsEnumerable();
var parents = rows.Where(r => !r.Field<int?>("parentId").HasValue);
var children = rows.Where(r => r.Field<int?>("parentId").HasValue);
var secondLevel = from parent in parents
join child in children
on parent.Field<int>("id") equals child.Field<int?>("parentId").Value
select child;
var both = parents.Concat(secondLevel).CopyToDataTable();
Note that i've used Nullable<int> instead of 0 for a parent since that is more readable and less prone of errors. Here is your sample data:
var table = new DataTable();
table.Columns.Add("id", typeof(int));
table.Columns.Add("name", typeof(string));
table.Columns.Add("parentId", typeof(int));
table.Rows.Add(1, "One", (int?)null);
table.Rows.Add(2, "OneA", 1);
table.Rows.Add(3, "OneB", 1);
table.Rows.Add(4, "Two", (int?)null);
table.Rows.Add(5, "TwoA", 4);
table.Rows.Add(6, "TwoB", 4);
table.Rows.Add(7, "TwoAA", 5);
table.Rows.Add(8, "TwoAB", 5);
Result:
1 One
4 Two
2 OneA 1
3 OneB 1
5 TwoA 4
6 TwoB 4
Since you want to stay with 0 instead of int?:
var parents = rows.Where(r => r.Field<int>("parentId") == 0);
var children = rows.Where(r => r.Field<int>("parentId") != 0);
var secondLevel = from parent in parents
join child in children
on parent.Field<int>("id") equals child.Field<int>("parentId")
select child;

I think this function might help you figure out the level of tree of each entry so you can use it in your selection:
public int level(DataTable dt, DataRow row)
{
int parentid = int.Parse(row[2].ToString());
if (parentid == 0)
return 1;
else
return 1 + level(dt, GetDataRow(dt,parentid ));
}
public DataRow GetDataRow(DataTable dt, int id)
{
foreach (DataRow r in dt.Rows)
{
if (int.Parse(r[0].ToString()) == id) return r;
}
return null;
}

You have a couple of options to your problem. As proposed by #Ali, you could use recursion like this:
public int level(DataTable dt, DataRow row)
{
int parentid = int.Parse(row[2].ToString());
if (parentid == 0)
return 1;
else
return 1 + level(dt, GetDataRow(dt,parentid ));
}
public DataRow GetDataRow(DataTable dt, int id)
{
foreach (DataRow r in dt.Rows)
{
if (int.Parse(r[0].ToString()) == id) return r;
}
return null;
}
But the problem is that you'll end up iterating though every element and then using recursion on every iteration. If you have absolutely no data relationship between your columns and their level in the tree, besides a parentId, then this is your only solution.
On the other hand, if you do have a relationship, where you have name[level of tree] like Name[A] is tree level 1 and Name[AB] is tree level two with the right node, then iteration through each like:
foreach (DataRow r in dt.Rows)
{
//Pull out the element
//Check the element's level
//Add it to the result set if level <= 2
}
I'd personally prefer to solve the problem by actually building a tree structure or using a SQL WHERE clause, but it's hard to justify the time on it. Depending on where you get this data from, you may also be able to add an additional column which tells you which level the node is in depending on where it's inserted. If it has a grandparent (i.e. two parent nodes) you don't include it in the result set.

DataTable level1 = (from t in dtCategories.AsEnumerable()
where t.Field<int>("parentId") == 0
select t).CopyToDataTable();
DataTable level2 =(from t1 in dtCategories.AsEnumerable()
join t2 in dtCategories.AsEnumerable()
on t1.Field<int>("id") equals t2.Field<int>("parentId")
where t1.Field<int>("parentId") == 0
select t2).CopyToDataTable();

Another way to do it, this will give you a new object which contains the level and the row item itself. This will work for n number of levels...
var nodes = table.AsEnumerable();
//var nodes = new List<TreeNode>();
var parentId = 0;
var countLevel = 0;
var allNods = new List<dynamic>();
while (nodes.Any(p => p.Field<int>("parentId") == parentId))// && countLevel < 2)
// countlevel< 2 only to give you the first 2 levels only...
{
var nodesWithLevel = nodes.Where(p => p.Field<int>("parentId") == parentId)
.Select(p => new { Level = parentId, Node = p });
allNods = allNods.Concat<dynamic>(nodesWithLevel).ToList();
parentId++;
countLevel++;
}
The code currently expects that the root nodes have parentId = 0. Could be changed to null, too of cause...