We Keep Coding

Linq group by Chunks [duplicate]

Let's take a class called Cls:
public class Cls
{
public int SequenceNumber { get; set; }
public int Value { get; set; }
}
Now, let's populate some collection with following elements:
Sequence
Number Value
======== =====
1 9
2 9
3 15
4 15
5 15
6 30
7 9
What I need to do, is to enumerate over Sequence Numbers and check if the next element has the same value. If yes, values are aggregated and so, desired output is as following:
Sequence Sequence
Number Number
From To Value
======== ======== =====
1 2 9
3 5 15
6 6 30
7 7 9
How can I perform this operation using LINQ query?

You can use Linq's GroupBy in a modified version which groups only if the two items are adjacent, then it's easy as:
var result = classes
.GroupAdjacent(c => c.Value)
.Select(g => new {
SequenceNumFrom = g.Min(c => c.SequenceNumber),
SequenceNumTo = g.Max(c => c.SequenceNumber),
Value = g.Key
});
foreach (var x in result)
Console.WriteLine("SequenceNumFrom:{0} SequenceNumTo:{1} Value:{2}", x.SequenceNumFrom, x.SequenceNumTo, x.Value);
DEMO
Result:
SequenceNumFrom:1 SequenceNumTo:2 Value:9
SequenceNumFrom:3 SequenceNumTo:5 Value:15
SequenceNumFrom:6 SequenceNumTo:6 Value:30
SequenceNumFrom:7 SequenceNumTo:7 Value:9
This is the extension method to to group adjacent items:
public static IEnumerable<IGrouping<TKey, TSource>> GroupAdjacent<TSource, TKey>(
this IEnumerable<TSource> source,
Func<TSource, TKey> keySelector)
{
TKey last = default(TKey);
bool haveLast = false;
List<TSource> list = new List<TSource>();
foreach (TSource s in source)
{
TKey k = keySelector(s);
if (haveLast)
{
if (!k.Equals(last))
{
yield return new GroupOfAdjacent<TSource, TKey>(list, last);
list = new List<TSource>();
list.Add(s);
last = k;
}
else
{
list.Add(s);
last = k;
}
}
else
{
list.Add(s);
last = k;
haveLast = true;
}
}
if (haveLast)
yield return new GroupOfAdjacent<TSource, TKey>(list, last);
}
}
and the class used:
public class GroupOfAdjacent<TSource, TKey> : IEnumerable<TSource>, IGrouping<TKey, TSource>
{
public TKey Key { get; set; }
private List<TSource> GroupList { get; set; }
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return ((System.Collections.Generic.IEnumerable<TSource>)this).GetEnumerator();
}
System.Collections.Generic.IEnumerator<TSource> System.Collections.Generic.IEnumerable<TSource>.GetEnumerator()
{
foreach (var s in GroupList)
yield return s;
}
public GroupOfAdjacent(List<TSource> source, TKey key)
{
GroupList = source;
Key = key;
}
}

You can use this linq query
Demo
var values = (new[] { 9, 9, 15, 15, 15, 30, 9 }).Select((x, i) => new { x, i });
var query = from v in values
let firstNonValue = values.Where(v2 => v2.i >= v.i && v2.x != v.x).FirstOrDefault()
let grouping = firstNonValue == null ? int.MaxValue : firstNonValue.i
group v by grouping into v
select new
{
From = v.Min(y => y.i) + 1,
To = v.Max(y => y.i) + 1,
Value = v.Min(y => y.x)
};

MoreLinq provides this functionality out of the box
It's called GroupAdjacent and is implemented as extension method on IEnumerable:
Groups the adjacent elements of a sequence according to a specified key selector function.
enumerable.GroupAdjacent(e => e.Key)
There is even a Nuget "source" package that contains only that method, if you don't want to pull in an additional binary Nuget package.
The method returns an IEnumerable<IGrouping<TKey, TValue>>, so its output can be processed in the same way output from GroupBy would be.

You can do it like this:
var all = new [] {
new Cls(1, 9)
, new Cls(2, 9)
, new Cls(3, 15)
, new Cls(4, 15)
, new Cls(5, 15)
, new Cls(6, 30)
, new Cls(7, 9)
};
var f = all.First();
var res = all.Skip(1).Aggregate(
new List<Run> {new Run {From = f.SequenceNumber, To = f.SequenceNumber, Value = f.Value} }
, (p, v) => {
if (v.Value == p.Last().Value) {
p.Last().To = v.SequenceNumber;
} else {
p.Add(new Run {From = v.SequenceNumber, To = v.SequenceNumber, Value = v.Value});
}
return p;
});
foreach (var r in res) {
Console.WriteLine("{0} - {1} : {2}", r.From, r.To, r.Value);
}
The idea is to use Aggregate creatively: starting with a list consisting of a single Run, examine the content of the list we've got so far at each stage of aggregation (the if statement in the lambda). Depending on the last value, either continue the old run, or start a new one.
Here is a demo on ideone.

I was able to accomplish it by creating a custom extension method.
static class Extensions {
internal static IEnumerable<Tuple<int, int, int>> GroupAdj(this IEnumerable<Cls> enumerable) {
Cls start = null;
Cls end = null;
int value = Int32.MinValue;
foreach (Cls cls in enumerable) {
if (start == null) {
start = cls;
end = cls;
continue;
}
if (start.Value == cls.Value) {
end = cls;
continue;
}
yield return Tuple.Create(start.SequenceNumber, end.SequenceNumber, start.Value);
start = cls;
end = cls;
}
yield return Tuple.Create(start.SequenceNumber, end.SequenceNumber, start.Value);
}
}
Here's the implementation:
static void Main() {
List<Cls> items = new List<Cls> {
new Cls { SequenceNumber = 1, Value = 9 },
new Cls { SequenceNumber = 2, Value = 9 },
new Cls { SequenceNumber = 3, Value = 15 },
new Cls { SequenceNumber = 4, Value = 15 },
new Cls { SequenceNumber = 5, Value = 15 },
new Cls { SequenceNumber = 6, Value = 30 },
new Cls { SequenceNumber = 7, Value = 9 }
};
Console.WriteLine("From To Value");
Console.WriteLine("===== ===== =====");
foreach (var item in items.OrderBy(i => i.SequenceNumber).GroupAdj()) {
Console.WriteLine("{0,-5} {1,-5} {2,-5}", item.Item1, item.Item2, item.Item3);
}
}
And the expected output:
From To Value
===== ===== =====
1 2 9
3 5 15
6 6 30
7 7 9

Here is an implementation without any helper methods:
var grp = 0;
var results =
from i
in
input.Zip(
input.Skip(1).Concat(new [] {input.Last ()}),
(n1, n2) => Tuple.Create(
n1, (n2.Value == n1.Value) ? grp : grp++
)
)
group i by i.Item2 into gp
select new {SequenceNumFrom = gp.Min(x => x.Item1.SequenceNumber),SequenceNumTo = gp.Max(x => x.Item1.SequenceNumber), Value = gp.Min(x => x.Item1.Value)};
The idea is:
Keep track of your own grouping indicator, grp.
Join each item of the collection to the next item in the collection (via Skip(1) and Zip).
If the Values match, they are in the same group; otherwise, increment grp to signal the start of the next group.

Untested dark magic follows. The imperative version seems like it would be easier in this case.
IEnumerable<Cls> data = ...;
var query = data
.GroupBy(x => x.Value)
.Select(g => new
{
Value = g.Key,
Sequences = g
.OrderBy(x => x.SequenceNumber)
.Select((x,i) => new
{
x.SequenceNumber,
OffsetSequenceNumber = x.SequenceNumber - i
})
.GroupBy(x => x.OffsetSequenceNumber)
.Select(g => g
.Select(x => x.SequenceNumber)
.OrderBy(x => x)
.ToList())
.ToList()
})
.SelectMany(x => x.Sequences
.Select(s => new { First = s.First(), Last = s.Last(), x.Value }))
.OrderBy(x => x.First)
.ToList();

Let me propose another option, which yields lazily both sequence of groups and
elements inside groups.
Demonstration in .NET Fiddle
Implementation:
public static class EnumerableExtensions
{
public static IEnumerable<IGrouping<TKey, TSource>> GroupAdjacent<TSource, TKey>(
this IEnumerable<TSource> source,
Func<TSource, TKey> keySelector,
IEqualityComparer<TKey>? comparer = null)
{
var comparerOrDefault = comparer ?? EqualityComparer<TKey>.Default;
using var iterator = new Iterator<TSource>(source.GetEnumerator());
iterator.MoveNext();
while (iterator.HasCurrent)
{
var key = keySelector(iterator.Current);
var elements = YieldAdjacentElements(iterator, key, keySelector, comparerOrDefault);
yield return new Grouping<TKey, TSource>(key, elements);
while (iterator.HasCurrentWithKey(key, keySelector, comparerOrDefault))
{
iterator.MoveNext();
}
}
}
static IEnumerable<TSource> YieldAdjacentElements<TKey, TSource>(
Iterator<TSource> iterator,
TKey key,
Func<TSource, TKey> keySelector,
IEqualityComparer<TKey> comparer)
{
while (iterator.HasCurrentWithKey(key, keySelector, comparer))
{
yield return iterator.Current;
iterator.MoveNext();
}
}
private static bool HasCurrentWithKey<TKey, TSource>(
this Iterator<TSource> iterator,
TKey key,
Func<TSource, TKey> keySelector,
IEqualityComparer<TKey> comparer) =>
iterator.HasCurrent && comparer.Equals(keySelector(iterator.Current), key);
private sealed class Grouping<TKey, TElement> : IGrouping<TKey, TElement>
{
public Grouping(TKey key, IEnumerable<TElement> elements)
{
Key = key;
Elements = elements;
}
public TKey Key { get; }
public IEnumerable<TElement> Elements { get; }
public IEnumerator<TElement> GetEnumerator() => Elements.GetEnumerator();
IEnumerator IEnumerable.GetEnumerator() => Elements.GetEnumerator();
}
private sealed class Iterator<T> : IDisposable
{
private readonly IEnumerator<T> _enumerator;
public Iterator(IEnumerator<T> enumerator)
{
_enumerator = enumerator;
}
public bool HasCurrent { get; private set; }
public T Current => _enumerator.Current;
public void MoveNext()
{
HasCurrent = _enumerator.MoveNext();
}
public void Dispose()
{
_enumerator.Dispose();
}
}
}
Notice, that it is impossible to achieve such level of laziness with regular GroupBy operation, since it needs to look through the whole collection before yielding the first group.
Particularly, in my case migration from GroupBy to GroupAdjacent in connection with lazy handling of whole pipeline helped to resolve memory consumption issues for large sequences.
In general, GroupAdjacent can be used as lazy and more efficient alternative of GroupBy, provided that input collection satisfies condition, that keys are sorted (or at least not fragmented), and provided that all operations in pipeline are lazy.

LINQ: Group by index and value [duplicate]

This question already has answers here:
linq group by contiguous blocks
(5 answers)
Closed 4 years ago.
Lets say I have an list of strings with the following values:
["a","a","b","a","a","a","c","c"]
I want to execute a linq query that will group into 4 groups:
Group 1: ["a","a"] Group 2: ["b"] Group 3: ["a","a","a"] Group 4:
["c","c"]
Basically I want to create 2 different groups for the value "a" because they are not coming from the same "index sequence".
Anyone has a LINQ solution for this?

You just need key other than items of array
var x = new string[] { "a", "a", "a", "b", "a", "a", "c" };
int groupId = -1;
var result = x.Select((s, i) => new
{
value = s,
groupId = (i > 0 && x[i - 1] == s) ? groupId : ++groupId
}).GroupBy(u => new { groupId });
foreach (var item in result)
{
Console.WriteLine(item.Key);
foreach (var inner in item)
{
Console.WriteLine(" => " + inner.value);
}
}
Here is the result: Link

Calculate the "index sequence" first, then do your group.
private class IndexedData
{
public int Sequence;
public string Text;
}
string[] data = [ "a", "a", "b" ... ]
// Calculate "index sequence" for each data element.
List<IndexedData> indexes = new List<IndexedData>();
foreach (string s in data)
{
IndexedData last = indexes.LastOrDefault() ?? new IndexedData();
indexes.Add(new IndexedData
{
Text = s,
Sequence = (last.Text == s
? last.Sequence
: last.Sequence + 1)
});
}
// Group by "index sequence"
var grouped = indexes.GroupBy(i => i.Sequence)
.Select(g => g.Select(i => i.Text));

This is a naive foreach implementation where whole dataset ends up in memory (probably not an issue for you since you do GroupBy):
public static IEnumerable<List<string>> Split(IEnumerable<string> values)
{
var result = new List<List<string>>();
foreach (var value in values)
{
var currentGroup = result.LastOrDefault();
if (currentGroup?.FirstOrDefault()?.Equals(value) == true)
{
currentGroup.Add(value);
}
else
{
result.Add(new List<string> { value });
}
}
return result;
}
Here comes a slightly complicated implementation with foreach and yield return enumerator state machine which keeps only current group in memory - this is probably how this would be implemented on framework level:
EDIT: This is apparently also the way MoreLINQ does it.
public static IEnumerable<List<string>> Split(IEnumerable<string> values)
{
var currentValue = default(string);
var group = (List<string>)null;
foreach (var value in values)
{
if (group == null)
{
currentValue = value;
group = new List<string> { value };
}
else if (currentValue.Equals(value))
{
group.Add(value);
}
else
{
yield return group;
currentValue = value;
group = new List<string> { value };
}
}
if (group != null)
{
yield return group;
}
}
And this is a joke version using LINQ only, it is basically the same as the first one but is slightly harder to understand (especially since Aggregate is not the most frequently used LINQ method):
public static IEnumerable<List<string>> Split(IEnumerable<string> values)
{
return values.Aggregate(
new List<List<string>>(),
(lists, str) =>
{
var currentGroup = lists.LastOrDefault();
if (currentGroup?.FirstOrDefault()?.Equals(str) == true)
{
currentGroup.Add(str);
}
else
{
lists.Add(new List<string> { str });
}
return lists;
},
lists => lists);
}

Using an extension method based on the APL scan operator, that is like Aggregate but returns intermediate results paired with source values:
public static IEnumerable<KeyValuePair<TKey, T>> ScanPair<T, TKey>(this IEnumerable<T> src, TKey seedKey, Func<KeyValuePair<TKey, T>, T, TKey> combine) {
using (var srce = src.GetEnumerator()) {
if (srce.MoveNext()) {
var prevkv = new KeyValuePair<TKey, T>(seedKey, srce.Current);
while (srce.MoveNext()) {
yield return prevkv;
prevkv = new KeyValuePair<TKey, T>(combine(prevkv, srce.Current), srce.Current);
}
yield return prevkv;
}
}
}
You can create extension methods for grouping by consistent runs:
public static IEnumerable<IGrouping<int, TResult>> GroupByRuns<TElement, TKey, TResult>(this IEnumerable<TElement> src, Func<TElement, TKey> key, Func<TElement, TResult> result, IEqualityComparer<TKey> cmp = null) {
cmp = cmp ?? EqualityComparer<TKey>.Default;
return src.ScanPair(0,
(kvp, cur) => cmp.Equals(key(kvp.Value), key(cur)) ? kvp.Key : kvp.Key + 1)
.GroupBy(kvp => kvp.Key, kvp => result(kvp.Value));
}
public static IEnumerable<IGrouping<int, TElement>> GroupByRuns<TElement, TKey>(this IEnumerable<TElement> src, Func<TElement, TKey> key) => src.GroupByRuns(key, e => e);
public static IEnumerable<IGrouping<int, TElement>> GroupByRuns<TElement>(this IEnumerable<TElement> src) => src.GroupByRuns(e => e, e => e);
public static IEnumerable<IEnumerable<TResult>> Runs<TElement, TKey, TResult>(this IEnumerable<TElement> src, Func<TElement, TKey> key, Func<TElement, TResult> result, IEqualityComparer<TKey> cmp = null) =>
src.GroupByRuns(key, result).Select(g => g.Select(s => s));
public static IEnumerable<IEnumerable<TElement>> Runs<TElement, TKey>(this IEnumerable<TElement> src, Func<TElement, TKey> key) => src.Runs(key, e => e);
public static IEnumerable<IEnumerable<TElement>> Runs<TElement>(this IEnumerable<TElement> src) => src.Runs(e => e, e => e);
And using the simplest version, you can get either an IEnumerable<IGrouping>>:
var ans1 = src.GroupByRuns();
Or a version that dumps the IGrouping (and its Key) for an IEnumerable:
var ans2 = src.Runs();

Merge elements in list by property

Context
I have a list of time intervals. Time interval type is HistoMesures.
Each HistoMesure is defined by a Debut (begin) property, a Fin (end) property, and a Commentaires (a little note) property.
My list is made in such a way that :
All HistoMesure are exclusive, I mean that they can't be overlapping each other.
The list is sorted by Debut, so by the beggining of the interval.
Edit : All HistoMesure are contiguous in this configuration.
Question
I want to merge (transform two little intervals in one big interval) all adjacent HistoMesure which have the same Commentaires. Currently I achieve this that way :
//sortedHistos type is List<HistoMesure>
int i = 0;
while (i < sortedHistos.Count - 1)
{
if (sortedHistos[i].Commentaires == sortedHistos[i + 1].Commentaires)
{
sortedHistos[i].Fin = sortedHistos[i + 1].Fin;
sortedHistos.RemoveAt(i + 1);
}
else
{
++i;
}
}
But I feel that it exists a more elegant way to do this, maybe with LINQ. Do you have any suggestion ?

Your solution works fine, I would keep it.
Don't try too hard to use LINQ if it doesn't match your requirements. LINQ is great to write queries (this is the Q of LINQ), not so great to modify existing lists.

This code will produce overlapping merged intervals. I.e. if you have intervals A, B, C where A and C have same commentaries, result will be AC, B:
var result = from h in sortedHistos
group h by h.Commentaires into g
select new HistoMesure {
Debut = g.First().Debut, // thus you have sorted entries
Fin = g.Last().Fin,
Commentaires = g.Key
};
You can use Min and Max if intervals are not sorted.
UPDATE: There is no default LINQ operator which allows you to create adjacent groups. But you always can create one. Here is IEnumerable<T> extension (I skipped arguments check):
public static IEnumerable<IGrouping<TKey, TElement>> GroupAdjacent<TKey, TElement>(
this IEnumerable<TElement> source, Func<TElement, TKey> keySelector)
{
using (var iterator = source.GetEnumerator())
{
if(!iterator.MoveNext())
{
yield break;
}
else
{
var comparer = Comparer<TKey>.Default;
var group = new Grouping<TKey, TElement>(keySelector(iterator.Current));
group.Add(iterator.Current);
while(iterator.MoveNext())
{
TKey key = keySelector(iterator.Current);
if (comparer.Compare(key, group.Key) != 0)
{
yield return group;
group = new Grouping<TKey, TElement>(key);
}
group.Add(iterator.Current);
}
if (group.Any())
yield return group;
}
}
}
This extension creates groups of adjacent elements which have same key value. Unfortunately all implementations of IGrouping in .NET are internal, so you need yours:
public class Grouping<TKey, TElement> : IGrouping<TKey, TElement>
{
private List<TElement> elements = new List<TElement>();
public Grouping(TKey key)
{
Key = key;
}
public TKey Key { get; private set; }
public IEnumerator<TElement> GetEnumerator()
{
return elements.GetEnumerator();
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
public void Add(TElement element)
{
elements.Add(element);
}
}
And now your code will look like:
var result = sortedHistos.GroupAdjacent(h => h.Commentaries)
.Select(g => new HistoMesure {
Debut = g.Min(h => h.Debut),
Fin = g.Max(h => h.Fin),
Commentaries = g.Key
});

Using Linq and borrowing from this article to group by adjacent values, this should work:
Your query:
var filteredHistos = sortedHistos
.GroupAdjacent(h => h.Commentaires)
.Select(g => new HistoMesure
{
Debut = g.First().Debut,
Fin = g.Last().Fin,
Commentaires = g.Key
});
And copying from the article, the rest of the code to group by:
public class GroupOfAdjacent<TSource, TKey> : IEnumerable<TSource>, IGrouping<TKey, TSource>
{
public TKey Key { get; set; }
private List<TSource> GroupList { get; set; }
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{
return ((System.Collections.Generic.IEnumerable<TSource>)this).GetEnumerator();
}
System.Collections.Generic.IEnumerator<TSource> System.Collections.Generic.IEnumerable<TSource>.GetEnumerator()
{
foreach (var s in GroupList)
yield return s;
}
public GroupOfAdjacent(List<TSource> source, TKey key)
{
GroupList = source;
Key = key;
}
}
public static class LocalExtensions
{
public static IEnumerable<IGrouping<TKey, TSource>> GroupAdjacent<TSource, TKey>(
this IEnumerable<TSource> source,
Func<TSource, TKey> keySelector)
{
TKey last = default(TKey);
bool haveLast = false;
List<TSource> list = new List<TSource>();
foreach (TSource s in source)
{
TKey k = keySelector(s);
if (haveLast)
{
if (!k.Equals(last))
{
yield return new GroupOfAdjacent<TSource, TKey>(list, last);
list = new List<TSource>();
list.Add(s);
last = k;
}
else
{
list.Add(s);
last = k;
}
}
else
{
list.Add(s);
last = k;
haveLast = true;
}
}
if (haveLast)
yield return new GroupOfAdjacent<TSource, TKey>(list, last);
}
}

If I understood you correctly, you need something like this:
var mergedMesures = mesures
.GroupBy(_ => _.Commentaires)
.Select(_ => new HistoMesures
{
Debut = _.Min(item => item.Debut),
Fin = _.Max(item => item.Fin),
Commentaires = _.Key
});

Order parent collection by minimum values in child collection in Linq

Parent{ List<Child> Children {get;set;} }
Child { int Age {get;set;} }
I would like to order the parents by the lowest age of their children, proceeding to the second or third child in the case of a tie.
The closest I've come is this, which only orders by the youngest child:
parents.OrderBy(p => p.Children.Min(c => c.Age))
This doesn't account for second (or third, etc) youngest in the case of a tie.
Given these 3 parents with corresponding child ages, I'd like them to come out in this order.
P1 1,2,7
P2 1,3,6
P3 1,4,5

So what you're trying to do, at a conceptual level, is compare two sequences. Rather than trying to special case it for this specific sequence, we can simply write a comparer capable of comparing any two sequences.
It will go through the items in the sequence compare the items at the same position, and then if it finds a pair that aren't equal, it knows the result.
public class SequenceComparer<TSource> : IComparer<IEnumerable<TSource>>
{
private IComparer<TSource> comparer;
public SequenceComparer(IComparer<TSource> comparer = null)
{
this.comparer = comparer ?? Comparer<TSource>.Default;
}
public int Compare(IEnumerable<TSource> x, IEnumerable<TSource> y)
{
return x.Zip(y, (a, b) => comparer.Compare(a, b))
.Where(n => n != 0)
.DefaultIfEmpty(x.Count().CompareTo(y.Count()))
.First();
}
}
Now we can simply use this comparer when calling OrderBy:
var query = parents.OrderBy(parent => parent.Children
.OrderBy(child => child.Age)
.Select(child => child.Age)
, new SequenceComparer<int>());

You'll need to write something like this extension method:
var orderedParents = parents.OrderBy(p => p.Children, c => c.Age);
Generic implementation:
/// <summary>
/// Given a way to determine a collection of elements (for example
/// children of a parent) and a comparable property of those items
/// (for example age of a child) this orders a collection of elements
/// according to the sorting order of the property of the first element
/// of their respective collections. In case of a tie, fall back to
/// subsequent elements as appropriate.
/// </summary>
public static IOrderedEnumerable<T> OrderBy<T, TKey, TValue>(this IEnumerable<T> #this, Func<T, IEnumerable<TKey>> getKeys, Func<TKey, TValue> getValue)
where TValue : IComparable<TValue>
{
return #this.OrderBy(x => x, new KeyComparer<T, TKey, TValue>(getKeys, getValue));
}
private class KeyComparer<T, TKey, TValue> : IComparer<T>
where TValue : IComparable<TValue>
{
private Func<T, IEnumerable<TKey>> GetKeys;
private Func<TKey, TValue> GetValue;
public KeyComparer(Func<T, IEnumerable<TKey>> getKeys, Func<TKey, TValue> getValue)
{
this.GetKeys = getKeys;
this.GetValue = getValue;
}
public int Compare(T x, T y)
{
var xKeys = GetKeys(x).OrderBy(GetValue).Select(GetValue);
var yKeys = GetKeys(y).OrderBy(GetValue).Select(GetValue);
foreach (var pair in xKeys.Zip(yKeys, Tuple.Create))
{
if (pair.Item1.CompareTo(pair.Item2) != 0)
return pair.Item1.CompareTo(pair.Item2);
}
return xKeys.Count().CompareTo(yKeys.Count());
}
}

You could use ThenBy and take the 2nd and 3rd children. But, that is not scalable, so it depends on the needs of the impl
If you want something more robust, you could do the following. It will work for this specific case. I am going to see if I can optimize it to be more generic though :)
public static class myExt
{
public static List<Parent> OrderByWithTieBreaker(this List<Parent> parents, int depth = 0)
{
if (depth > parents[0].Children.Count())
return parents;
var returnedList = new List<Parent>();
Func<Parent, int> keySelector = x =>
{
IEnumerable<Child> enumerable = x.Children.OrderBy(y => y.Age).Skip(depth);
if (!enumerable.Any())
return 0; //If no children left, then return lowest possible age
return enumerable.Min(z => z.Age);
};
var orderedParents = parents.OrderBy(keySelector);
var groupings = orderedParents.GroupBy(keySelector);
foreach (var grouping in groupings)
{
if (grouping.Count() > 1)
{
var innerOrder = grouping.ToList().OrderByWithTieBreaker(depth + 1);
returnedList = returnedList.Union(innerOrder).ToList();
}
else
returnedList.Add(grouping.First());
}
return returnedList;
}
}
[TestFixture]
public class TestClass
{
public class Parent { public string Name { get; set; } public List<Child> Children { get; set; } }
public class Child { public int Age {get;set;} }
[Test]
public void TestName()
{
var parents = new List<Parent>
{
new Parent{Name="P3", Children = new List<Child>{new Child{Age=1}, new Child{Age=3}, new Child{Age=6}, new Child{Age=7}}},
new Parent{Name="P4", Children = new List<Child>{new Child{Age=1}, new Child{Age=3}, new Child{Age=6}, new Child{Age=7}}},
new Parent{Name="P2", Children = new List<Child>{new Child{Age=1}, new Child{Age=3}, new Child{Age=6}}},
new Parent{Name="P1", Children = new List<Child>{new Child{Age=1}, new Child{Age=2}, new Child{Age=7}}},
new Parent{Name="P5", Children = new List<Child>{new Child{Age=1}, new Child{Age=4}, new Child{Age=5}}}
};
var f = parents.OrderByWithTieBreaker();
int count = 1;
foreach (var d in f)
{
Assert.That(d.Name, Is.EqualTo("P"+count));
count++;
}
}

Recursive List Flattening

I could probably write this myself, but the specific way I'm trying to accomplish it is throwing me off. I'm trying to write a generic extension method similar to the others introduced in .NET 3.5 that will take a nested IEnumerable of IEnumerables (and so on) and flatten it into one IEnumerable. Anyone have any ideas?
Specifically, I'm having trouble with the syntax of the extension method itself so that I can work on a flattening algorithm.

Here's an extension that might help. It will traverse all nodes in your hierarchy of objects and pick out the ones that match a criteria. It assumes that each object in your hierarchy has a collection property that holds its child objects.
Here's the extension:
/// Traverses an object hierarchy and return a flattened list of elements
/// based on a predicate.
///
/// TSource: The type of object in your collection.</typeparam>
/// source: The collection of your topmost TSource objects.</param>
/// selectorFunction: A predicate for choosing the objects you want.
/// getChildrenFunction: A function that fetches the child collection from an object.
/// returns: A flattened list of objects which meet the criteria in selectorFunction.
public static IEnumerable<TSource> Map<TSource>(
this IEnumerable<TSource> source,
Func<TSource, bool> selectorFunction,
Func<TSource, IEnumerable<TSource>> getChildrenFunction)
{
// Add what we have to the stack
var flattenedList = source.Where(selectorFunction);
// Go through the input enumerable looking for children,
// and add those if we have them
foreach (TSource element in source)
{
flattenedList = flattenedList.Concat(
getChildrenFunction(element).Map(selectorFunction,
getChildrenFunction)
);
}
return flattenedList;
}
Examples (Unit Tests):
First we need an object and a nested object hierarchy.
A simple node class
class Node
{
public int NodeId { get; set; }
public int LevelId { get; set; }
public IEnumerable<Node> Children { get; set; }
public override string ToString()
{
return String.Format("Node {0}, Level {1}", this.NodeId, this.LevelId);
}
}
And a method to get a 3-level deep hierarchy of nodes
private IEnumerable<Node> GetNodes()
{
// Create a 3-level deep hierarchy of nodes
Node[] nodes = new Node[]
{
new Node
{
NodeId = 1,
LevelId = 1,
Children = new Node[]
{
new Node { NodeId = 2, LevelId = 2, Children = new Node[] {} },
new Node
{
NodeId = 3,
LevelId = 2,
Children = new Node[]
{
new Node { NodeId = 4, LevelId = 3, Children = new Node[] {} },
new Node { NodeId = 5, LevelId = 3, Children = new Node[] {} }
}
}
}
},
new Node { NodeId = 6, LevelId = 1, Children = new Node[] {} }
};
return nodes;
}
First Test: flatten the hierarchy, no filtering
[Test]
public void Flatten_Nested_Heirachy()
{
IEnumerable<Node> nodes = GetNodes();
var flattenedNodes = nodes.Map(
p => true,
(Node n) => { return n.Children; }
);
foreach (Node flatNode in flattenedNodes)
{
Console.WriteLine(flatNode.ToString());
}
// Make sure we only end up with 6 nodes
Assert.AreEqual(6, flattenedNodes.Count());
}
This will show:
Node 1, Level 1
Node 6, Level 1
Node 2, Level 2
Node 3, Level 2
Node 4, Level 3
Node 5, Level 3
Second Test: Get a list of nodes that have an even-numbered NodeId
[Test]
public void Only_Return_Nodes_With_Even_Numbered_Node_IDs()
{
IEnumerable<Node> nodes = GetNodes();
var flattenedNodes = nodes.Map(
p => (p.NodeId % 2) == 0,
(Node n) => { return n.Children; }
);
foreach (Node flatNode in flattenedNodes)
{
Console.WriteLine(flatNode.ToString());
}
// Make sure we only end up with 3 nodes
Assert.AreEqual(3, flattenedNodes.Count());
}
This will show:
Node 6, Level 1
Node 2, Level 2
Node 4, Level 3

Hmm... I'm not sure exactly what you want here, but here's a "one level" option:
public static IEnumerable<TElement> Flatten<TElement,TSequence> (this IEnumerable<TSequence> sequences)
where TSequence : IEnumerable<TElement>
{
foreach (TSequence sequence in sequences)
{
foreach(TElement element in sequence)
{
yield return element;
}
}
}
If that's not what you want, could you provide the signature of what you do want? If you don't need a generic form, and you just want to do the kind of thing that LINQ to XML constructors do, that's reasonably simple - although the recursive use of iterator blocks is relatively inefficient. Something like:
static IEnumerable Flatten(params object[] objects)
{
// Can't easily get varargs behaviour with IEnumerable
return Flatten((IEnumerable) objects);
}
static IEnumerable Flatten(IEnumerable enumerable)
{
foreach (object element in enumerable)
{
IEnumerable candidate = element as IEnumerable;
if (candidate != null)
{
foreach (object nested in candidate)
{
yield return nested;
}
}
else
{
yield return element;
}
}
}
Note that that will treat a string as a sequence of chars, however - you may want to special-case strings to be individual elements instead of flattening them, depending on your use case.
Does that help?

I thought I'd share a complete example with error handling and a single-logic apporoach.
Recursive flattening is as simple as:
LINQ version
public static class IEnumerableExtensions
{
public static IEnumerable<T> SelectManyRecursive<T>(this IEnumerable<T> source, Func<T, IEnumerable<T>> selector)
{
if (source == null) throw new ArgumentNullException("source");
if (selector == null) throw new ArgumentNullException("selector");
return !source.Any() ? source :
source.Concat(
source
.SelectMany(i => selector(i).EmptyIfNull())
.SelectManyRecursive(selector)
);
}
public static IEnumerable<T> EmptyIfNull<T>(this IEnumerable<T> source)
{
return source ?? Enumerable.Empty<T>();
}
}
Non-LINQ version
public static class IEnumerableExtensions
{
public static IEnumerable<T> SelectManyRecursive<T>(this IEnumerable<T> source, Func<T, IEnumerable<T>> selector)
{
if (source == null) throw new ArgumentNullException("source");
if (selector == null) throw new ArgumentNullException("selector");
foreach (T item in source)
{
yield return item;
var children = selector(item);
if (children == null)
continue;
foreach (T descendant in children.SelectManyRecursive(selector))
{
yield return descendant;
}
}
}
}
Design decisions
I decided to:
disallow flattening of a null IEnumerable, this can be changed by removing exception throwing and:
adding source = source.EmptyIfNull(); before return in the 1st version
adding if (source != null) before foreach in the 2nd version
allow returning of a null collection by the selector - this way I'm removing responsibility from the caller to assure the children list isn't empty, this can be changed by:
removing .EmptyIfNull() in the first version - note that SelectMany will fail if null is returned by selector
removing if (children == null) continue; in the second version - note that foreach will fail on a null IEnumerable parameter
allow filtering children with .Where clause on the caller side or within the children selector rather than passing a children filter selector parameter:
it won't impact the efficiency because in both versions it is a deferred call
it would be mixing another logic with the method and I prefer to keep the logic separated
Sample use
I'm using this extension method in LightSwitch to obtain all controls on the screen:
public static class ScreenObjectExtensions
{
public static IEnumerable<IContentItemProxy> FindControls(this IScreenObject screen)
{
var model = screen.Details.GetModel();
return model.GetChildItems()
.SelectManyRecursive(c => c.GetChildItems())
.OfType<IContentItemDefinition>()
.Select(c => screen.FindControl(c.Name));
}
}

Here is a modified Jon Skeet's answer to allow more than "one level":
static IEnumerable Flatten(IEnumerable enumerable)
{
foreach (object element in enumerable)
{
IEnumerable candidate = element as IEnumerable;
if (candidate != null)
{
foreach (object nested in Flatten(candidate))
{
yield return nested;
}
}
else
{
yield return element;
}
}
}
disclaimer: I don't know C#.
The same in Python:
#!/usr/bin/env python
def flatten(iterable):
for item in iterable:
if hasattr(item, '__iter__'):
for nested in flatten(item):
yield nested
else:
yield item
if __name__ == '__main__':
for item in flatten([1,[2, 3, [[4], 5]], 6, [[[7]]], [8]]):
print(item, end=" ")
It prints:
1 2 3 4 5 6 7 8

Isn't that what [SelectMany][1] is for?
enum1.SelectMany(
a => a.SelectMany(
b => b.SelectMany(
c => c.Select(
d => d.Name
)
)
)
);

Function:
public static class MyExtentions
{
public static IEnumerable<T> RecursiveSelector<T>(this IEnumerable<T> nodes, Func<T, IEnumerable<T>> selector)
{
if(nodes.Any() == false)
{
return nodes;
}
var descendants = nodes
.SelectMany(selector)
.RecursiveSelector(selector);
return nodes.Concat(descendants);
}
}
Usage:
var ar = new[]
{
new Node
{
Name = "1",
Chilren = new[]
{
new Node
{
Name = "11",
Children = new[]
{
new Node
{
Name = "111",
}
}
}
}
}
};
var flattened = ar.RecursiveSelector(x => x.Children).ToList();

Okay here's another version which is combined from about 3 answers above.
Recursive. Uses yield. Generic. Optional filter predicate. Optional selection function. About as concise as I could make it.
public static IEnumerable<TNode> Flatten<TNode>(
this IEnumerable<TNode> nodes,
Func<TNode, bool> filterBy = null,
Func<TNode, IEnumerable<TNode>> selectChildren = null
)
{
if (nodes == null) yield break;
if (filterBy != null) nodes = nodes.Where(filterBy);
foreach (var node in nodes)
{
yield return node;
var children = (selectChildren == null)
? node as IEnumerable<TNode>
: selectChildren(node);
if (children == null) continue;
foreach (var child in children.Flatten(filterBy, selectChildren))
{
yield return child;
}
}
}
Usage:
// With filter predicate, with selection function
var flatList = nodes.Flatten(n => n.IsDeleted == false, n => n.Children);

The SelectMany extension method does this already.
Projects each element of a sequence to
an IEnumerable<(Of <(T>)>) and
flattens the resulting sequences into
one sequence.

I had to implement mine from scratch because all of the provided solutions would break in case there is a loop i.e. a child that points to its ancestor. If you have the same requirements as mine please take a look at this (also let me know if my solution would break in any special circumstances):
How to use:
var flattenlist = rootItem.Flatten(obj => obj.ChildItems, obj => obj.Id)
Code:
public static class Extensions
{
/// <summary>
/// This would flatten out a recursive data structure ignoring the loops. The end result would be an enumerable which enumerates all the
/// items in the data structure regardless of the level of nesting.
/// </summary>
/// <typeparam name="T">Type of the recursive data structure</typeparam>
/// <param name="source">Source element</param>
/// <param name="childrenSelector">a function that returns the children of a given data element of type T</param>
/// <param name="keySelector">a function that returns a key value for each element</param>
/// <returns>a faltten list of all the items within recursive data structure of T</returns>
public static IEnumerable<T> Flatten<T>(this IEnumerable<T> source,
Func<T, IEnumerable<T>> childrenSelector,
Func<T, object> keySelector) where T : class
{
if (source == null)
throw new ArgumentNullException("source");
if (childrenSelector == null)
throw new ArgumentNullException("childrenSelector");
if (keySelector == null)
throw new ArgumentNullException("keySelector");
var stack = new Stack<T>( source);
var dictionary = new Dictionary<object, T>();
while (stack.Any())
{
var currentItem = stack.Pop();
var currentkey = keySelector(currentItem);
if (dictionary.ContainsKey(currentkey) == false)
{
dictionary.Add(currentkey, currentItem);
var children = childrenSelector(currentItem);
if (children != null)
{
foreach (var child in children)
{
stack.Push(child);
}
}
}
yield return currentItem;
}
}
/// <summary>
/// This would flatten out a recursive data structure ignoring the loops. The end result would be an enumerable which enumerates all the
/// items in the data structure regardless of the level of nesting.
/// </summary>
/// <typeparam name="T">Type of the recursive data structure</typeparam>
/// <param name="source">Source element</param>
/// <param name="childrenSelector">a function that returns the children of a given data element of type T</param>
/// <param name="keySelector">a function that returns a key value for each element</param>
/// <returns>a faltten list of all the items within recursive data structure of T</returns>
public static IEnumerable<T> Flatten<T>(this T source,
Func<T, IEnumerable<T>> childrenSelector,
Func<T, object> keySelector) where T: class
{
return Flatten(new [] {source}, childrenSelector, keySelector);
}
}

Since yield is not available in VB and LINQ provides both deferred execution and a concise syntax, you can also use.
<Extension()>
Public Function Flatten(Of T)(ByVal objects As Generic.IEnumerable(Of T), ByVal selector As Func(Of T, Generic.IEnumerable(Of T))) As Generic.IEnumerable(Of T)
If(objects.Any()) Then
Return objects.Union(objects.Select(selector).Where(e => e != null).SelectMany(e => e)).Flatten(selector))
Else
Return objects
End If
End Function
public static class Extensions{
public static IEnumerable<T> Flatten<T>(this IEnumerable<T> objects, Func<T, IEnumerable<T>> selector) where T:Component{
if(objects.Any()){
return objects.Union(objects.Select(selector).Where(e => e != null).SelectMany(e => e).Flatten(selector));
}
return objects;
}
}
edited to include:
empty enumerable per https://stackoverflow.com/a/30325216/107683,
null enumerable per https://stackoverflow.com/a/39338919/107683
C# implementation.

static class EnumerableExtensions
{
public static IEnumerable<T> Flatten<T>(this IEnumerable<IEnumerable<T>> sequence)
{
foreach(var child in sequence)
foreach(var item in child)
yield return item;
}
}
Maybe like this? Or do you mean that it could potentially be infintly deep?

class PageViewModel {
public IEnumerable<PageViewModel> ChildrenPages { get; set; }
}
Func<IEnumerable<PageViewModel>, IEnumerable<PageViewModel>> concatAll = null;
concatAll = list => list.SelectMany(l => l.ChildrenPages.Any() ?
concatAll(l.ChildrenPages).Union(new[] { l }) : new[] { l });
var allPages = concatAll(source).ToArray();

Basicly, you need to have a master IENumerable that is outside of your recursive function, then in your recursive function (Psuedo-code)
private void flattenList(IEnumerable<T> list)
{
foreach (T item in list)
{
masterList.Add(item);
if (item.Count > 0)
{
this.flattenList(item);
}
}
}
Though I'm really not sure what you mean by IEnumerable nested in an IEnumerable...whats within that? How many levels of nesting? Whats the final type? obviously my code isn't correct, but I hope it gets you thinking.