We Keep Coding

Why does using IEnumerable<T> inside a recursive method make it much slower than using a List<T>?

I am using a recursive method to go through a tree of items and add its children to a flat collection:
public class Thing
{
public int Id { get; set; }
public string Name { get; set; }
public int? ParentId { get; set; }
}
void Main()
{
var sampleData = new List<Thing>
{
new Thing { Id = 1, Name = "root1", ParentId = null },
new Thing { Id = 2, Name = "2", ParentId = 1 },
new Thing { Id = 3, Name = "3", ParentId = 1 },
new Thing { Id = 4, Name = "4", ParentId = 2 },
new Thing { Id = 5, Name = "5", ParentId = 2 },
new Thing { Id = 6, Name = "6", ParentId = 2 },
new Thing { Id = 7, Name = "7", ParentId = 6 },
new Thing { Id = 8, Name = "8", ParentId = 7 },
new Thing { Id = 9, Name = "9", ParentId = 8 },
new Thing { Id = 10, Name = "10", ParentId = 9 },
new Thing { Id = 11, Name = "11", ParentId = 10 },
new Thing { Id = 12, Name = "12", ParentId = 11 },
new Thing { Id = 13, Name = "13", ParentId = 12 },
new Thing { Id = 14, Name = "14", ParentId = 13 },
new Thing { Id = 15, Name = "root15", ParentId = null }
};
var subThings = new HashSet<Thing>();
var stopWatch = new Stopwatch();
stopWatch.Start();
//AddSubThings(subThings, sampleData, new List<int> { 1 });
AddSubThingsUsingList(subThings, sampleData, new List<int> { 1 });
stopWatch.Elapsed.Dump();
subThings.Dump();
}
private void AddSubThings(HashSet<Thing> resultThings, IEnumerable<Thing> sourceThings, IEnumerable<int> parentIds)
{
if (!sourceThings.Any() || !parentIds.Any())
{
return;
}
var subThings = sourceThings.Where(st => st.ParentId.HasValue && parentIds.Contains(st.ParentId.Value));
resultThings.UnionWith(subThings);
AddSubThings(resultThings, sourceThings.Except(subThings), subThings.Select(st => st.Id));
}
private void AddSubThingsUsingList(HashSet<Thing> resultThings, List<Thing> sourceThings, List<int> parentIds)
{
if (!sourceThings.Any() || !parentIds.Any())
{
return;
}
var subThings = sourceThings.Where(st => st.ParentId.HasValue && parentIds.Contains(st.ParentId.Value));
resultThings.UnionWith(subThings);
AddSubThingsUsingList(resultThings, sourceThings.Except(subThings).ToList(), subThings.Select(st => st.Id).ToList());
}
When I use the AddSubThings method it takes around 90 seconds to process. However if I use the AddSubThingsUsingList method it does not even take a second. Why is this?

The problem is because your create subThings from sourceThings like this
var subThings = sourceThings.Where(
st => st.ParentId.HasValue && parentIds.Contains(st.ParentId.Value));
Then you pass the following as sourceThings to the recursive call.
sourceThings.Except(subThings)
Which is equivalent to
sourceThings.Except(
sourceThings.Where(
st => st.ParentId.HasValue && parentIds.Contains(st.ParentId.Value)))
That query when iterated with have to iterate over the original list twice. With each recursive call the query will build up and need to iterate the original list 2^n times where n is the recursion level. And your query is being iterated by the Any and the HashSet.UnionWith calls meaning it's more like 2^(n+1).
The other one immediately iterates the query before passing them and thus avoids this doubling problem.
You could pass the following to your recursive call for sourceThings instead to make it faster as it doesn't double up the required iterating of the original list on each recursive call.
sourceThings.Where(st => !st.ParentId.HasValue || !parentIds.Contains(st.ParentId.Value))

Ok. This is a bit complex.
Operations on IEnumerable are lazy, i.e. they are not executed until you need the result. Now when you pass sourceThins and subThings to AddSubThings, you've not sent a materialized collection of things, all you've done is you've defined how these collections are calculated from the original Lists.
Now when the method calls itself recursively, it adds more filtering and selection to the data it has received.
All these layers of selection and filtering will be called when you call Any().
On the other hand, in the case of Lists, things are materialized after calls to Where, Except and Select, because you call ToList.

Can someone help me ordering this list?

I am not the best programmer, so need some help to order this list. I had a few stabs at it, but still getting some cases which are wrong.
Essentially the list is the following:
#, ID, PreceedingID
A, 1 , 0
B, 2 , 3
C, 3 , 1
D, 4 , 2
I want to order it so that the list follows the preceeding id. The first item will always have the preceeding ID of 0.
#, ID, PreceedingID
A, 1 , 0
C, 3 , 1
B, 2 , 3
D, 4 , 2
Do you think you can help?
Thanks!

How about:
var data = new[] {
new Row{ Name = "A", ID = 1, PreceedingID = 0},
new Row{ Name = "B", ID = 2, PreceedingID = 3},
new Row{ Name = "C", ID = 3, PreceedingID = 1},
new Row{ Name = "D", ID = 4, PreceedingID = 2},
};
var byLastId = data.ToDictionary(x => x.PreceedingID);
var newList = new List<Row>(data.Length);
int lastId = 0;
Row next;
while (byLastId.TryGetValue(lastId, out next))
{
byLastId.Remove(lastId); // removal avoids infinite loops
newList.Add(next);
lastId = next.ID;
}
After this, newList has the data in the desired order.
In the above, Row is:
class Row
{
public string Name { get; set; }
public int ID { get; set; }
public int PreceedingID { get; set; }
}
But obviously substitute for your own type.

You can use for example dictionary to sort it:
Dictionary<..> d = new Dictionary<..>()
foreach(var el in list){
d[el.PreceedingID] = el; //put data to dict by PreecedingID
}
List<..> result = new List<..>();
int prec = 0; //get first ID
for(int i = 0; i < list.Length; ++i){
var actEl = d[prec]; //get next element
prec = actEl.ID; //change prec id
result.Add(actEl); //put element into result list
}

Using LINQ to count value frequency

I have a table
ID|VALUE
VALUE is an integer field with possible values between 0 and 4. How can I query the count of each value?
Ideally the result should be an array with 6 elements, one for the count of each value and the last one is the total number of rows.

This simple program does just that:
class Record
{
public int Id { get; set; }
public int Value { get; set; }
}
class Program
{
static void Main(string[] args)
{
List<Record> records = new List<Record>()
{
new Record() { Id = 1, Value = 0},
new Record() { Id = 2, Value = 1 },
new Record() { Id = 3, Value = 2 },
new Record() { Id = 4, Value = 3 },
new Record() { Id = 5, Value = 4 },
new Record() { Id = 6, Value = 2 },
new Record() { Id = 7, Value = 3 },
new Record() { Id = 8, Value = 1 },
new Record() { Id = 9, Value = 0 },
new Record() { Id = 10, Value = 4 }
};
var query = from r in records
group r by r.Value into g
select new {Count = g.Count(), Value = g.Key};
foreach (var v in query)
{
Console.WriteLine("Value = {0}, Count = {1}", v.Value, v.Count);
}
}
}
Output:
Value = 0, Count = 2
Value = 1, Count = 2
Value = 2, Count = 2
Value = 3, Count = 2
Value = 4, Count = 2
Slightly modified version to return an Array with only the count of values:
int[] valuesCounted = (from r in records
group r by r.Value
into g
select g.Count()).ToArray();
Adding the rows count in the end:
valuesCounted = valuesCounted.Concat(new[] { records.Count()}).ToArray();

Here is how you would get the number of rows for each value of VALUE, in order:
var counts =
from row in db.Table
group row by row.VALUE into rowsByValue
orderby rowsByValue.Key
select rowsByValue.Count();
To get the total number of rows in the table, you can add all of the counts together. You don't want the original sequence to be iterated twice, though; that would cause the query to be executed twice. Instead, you should make an intermediate list first:
var countsList = counts.ToList();
var countsWithTotal = countsList.Concat(new[] { countsList.Sum() });

LINQ C# - Combining multiple groups

LINQ Groupby query creates a new group for each unique key. I would like to combine multiple groups into a single group based on the key value.
e.g.
var customersData = new[]
{
new { id = 1, company = "ABC" },
new { id = 2, company = "AAA" },
new { id = 3, company = "ABCD" },
new { id = 4, company = "XYZ" },
new { id = 5, company = "X.Y.Z." },
new { id = 6, company = "QQQ" },
};
var groups = from d in customersData
group d by d.company;
Let's say I want ABC, AAA, and ABCD in the same group, and XYZ, X.Y.Z. in the same group.
Is there anyway to achieve this through LINQ queries?

You will need to use the overload of GroupBy that takes an IEqualityComparer.
var groups = customersData.GroupBy(k => k.company, new KeyComparer());
where KeyComparer could look like
public class KeyComparer : IEqualityComparer
{
public bool Equals(string x, string y)
{
// put your comparison logic here
}
public int GetHashCode(string obj)
{
// same comparison logic here
}
}
You can comparer the strings any way you like in the Equals method of KeyComparer.
EDIT:
You also need to make sure that the implementation of GetHashCode obeys the same rules as the Equals method. For example if you just removed the "." and replaced with "" as in other answers you need to do it in both methods like this
public class KeyComparer : IEqualityComparer
{
public bool Equals(string x, string y)
{
return x.Replace(".", "") == y.Replace(".", "");
}
public int GetHashCode(string obj)
{
return obj.Replace(".", "").GetHashCode();
}
}

I am assuming the following:
You meant to have quotes surrounding the company "names" (as below).
Your problem is simply solved by removing the '.'s from each company name.
If these assumptions are correct, the solution is simply the following:
var customersData = new[] {
new { id = 1, company = "ABC" },
new { id = 2, company = "A.B.C." },
new { id = 3, company = "A.B.C." },
new { id = 4, company = "XYZ" },
new { id = 5, company = "X.Y.Z." },
new { id = 6, company = "QQQ" },
};
var groups = from d in customersData
group d by d.company.Replace(".", "");
If these assumptions are not correct, please clarify and I can help work closer to a solution.

var groups = from d in customersData
group d by d.company.Replace(".", "");

public void int GetId(Company c)
{
int result = //whatever you want
return result;
}
then later:
var groups = from d in customersData
group d by GetId(d.company);

I think this is what you want:
var customersData = new[]
{
new { id = 1, company = "ABC" },
new { id = 2, company = "AAA" },
new { id = 3, company = "ABCD" },
new { id = 4, company = "XYZ" },
new { id = 5, company = "X.Y.Z." },
new { id = 6, company = "QQQ" },
};
var groups = from d in customersData
group d by d.company[0];
foreach (var group in groups)
{
Console.WriteLine("Group " + group.Key);
foreach (var item in group)
{
Console.WriteLine("Item " + item.company);
}
}
Console.ReadLine();

What is the easiest way to fixed pivot transform this data in C# (linq?)?

Given this example data (in .NET classes where Po, Sku, Qty are properties):
PO, Sku, Qty
1,ABC,1
1,DEF,2
1,GHI,1
1,QWE,1
1,ASD,1
1,ZXC,5
1,ERT,1
2,QWE,1
2,ASD,11
2,ZXC,1
3,ERT,1
3,DFG,1
3,DFH,1
3,CVB,4
3,VBN,1
3,NMY,1
I need to transform it into a fixed column format, with a max of 5 SKUs per line (repeating the PO if needed for > 5):
PO, SkuA, QtyA, SkuB, QtyB, SkuC, QtyC, SkuD, QtyD, SkuE, QtyE
1, ABC, 1, DEF, 2, GHI, 1, QWE, 1, ASD, 1
1, ZXC, 5, ERT, 1, , , , , ,
2, QWE, 1, ASD, 11, ZXC, 1, , , ,
3, ERT, 1, DFG, 1, DFH, 1, CVB, 4, VBN, 1
3, NMY, 1, , , , , , , ,
Output can be CSV (which is what I'm outputting), or .NET classes - no matter there. Is there a simple way to do this in Linq by grouping by PO, then by counts of 5?
EDIT: I have no control of over the destination format. And for anyone interested, it's VendorNet and VendorBridge that require this nonsense.

Firstly, here's the query that will generate the correct hierarchy of objects. I'm using anonymous types but it's easy enough to change it to use your own proper classes.
var query = yourData
.GroupBy
(
x => x.PO
)
.SelectMany
(
x => x.Select
(
(y, i) => new { y.PO, y.Sku, y.Qty, Key = i / 5 }
)
)
.GroupBy
(
x => new { x.PO, x.Key }
);
Using LINQ to create the CSV from the query results is bit of a hack, but it gets the job done. (The "benefit" of using LINQ is that you could chain the original query and the CSV generation into a single, massive statement, should you wish.)
IEnumerable<string> csvLines = query
.Select
(
x => x.Aggregate
(
new { Count = 0, SB = new StringBuilder() },
(a, y) => new
{
Count = a.Count + 1,
SB = ((a.SB.Length == 0) ? a.SB.Append(y.PO) : a.SB)
.Append(", ").Append(y.Sku).Append(", ").Append(y.Qty)
},
a => a.SB.ToString() + string.Join(", , ", new string[6 - a.Count])
)
);
string csv = string.Join(Environment.NewLine, csvLines.ToArray());
In my opinion, creating the CSV without using LINQ makes the code much more readable:
StringBuilder sb = new StringBuilder();
foreach (var group in query)
{
int count = 0;
foreach (var item in group)
{
if (count++ == 0)
{
sb.Append(item.PO);
}
sb.Append(", ").Append(item.Sku).Append(", ").Append(item.Qty);
}
while (count++ < 5)
{
sb.Append(", , ");
}
sb.Append(Environment.NewLine);
}
string csv = sb.ToString();

Here you go. I didn't format the output the way you wanted. But this should give you an idea of how to pivot rows. Hope this helps :-)
public class MyClass
{
public int PO { get; set; }
public String SKU { get; set; }
public int Qty { get; set; }
public static IEnumerable<MyClass> GetList()
{
return new List<MyClass>()
{
new MyClass {PO = 1, SKU = "ABC", Qty = 1},
new MyClass {PO = 1, SKU = "DEF", Qty = 2},
new MyClass {PO = 1, SKU = "GHI", Qty = 1},
new MyClass {PO = 1, SKU = "QWE", Qty = 1},
new MyClass {PO = 1, SKU = "ASD", Qty = 1},
new MyClass {PO = 1, SKU = "ZXC", Qty = 5},
new MyClass {PO = 1, SKU = "ERT", Qty = 1},
new MyClass {PO = 2, SKU = "QWE", Qty = 1},
new MyClass {PO = 2, SKU = "ASD", Qty = 1},
new MyClass {PO = 2, SKU = "ZXC", Qty = 5},
};
}
}
EDIT: I've fixed the query based on Luke's comment
var lQuery =
MyClass.GetList()
.GroupBy(pArg => pArg.PO)
.Select(pArg => new
{
Test = pArg.Select((pArg1, pId) =>
new {ID = (pId / 5),
pArg1.PO, pArg1.SKU, pArg1.Qty})
.GroupBy(pArg1 => pArg1.ID)
.Select(pArg1 =>
pArg1.Aggregate(pArg.Key.ToString(),
(pSeed, pCur) =>
pSeed + pCur.SKU + ","))
});