I need an easy way to iterate over multiple collections without actually merging them, and I couldn't find anything built into .NET that looks like it does that. It feels like this should be a somewhat common situation. I don't want to reinvent the wheel. Is there anything built in that does something like this:
public class MultiCollectionEnumerable<T> : IEnumerable<T>
{
private MultiCollectionEnumerator<T> enumerator;
public MultiCollectionEnumerable(params IEnumerable<T>[] collections)
{
enumerator = new MultiCollectionEnumerator<T>(collections);
}
public IEnumerator<T> GetEnumerator()
{
enumerator.Reset();
return enumerator;
}
IEnumerator IEnumerable.GetEnumerator()
{
enumerator.Reset();
return enumerator;
}
private class MultiCollectionEnumerator<T> : IEnumerator<T>
{
private IEnumerable<T>[] collections;
private int currentIndex;
private IEnumerator<T> currentEnumerator;
public MultiCollectionEnumerator(IEnumerable<T>[] collections)
{
this.collections = collections;
this.currentIndex = -1;
}
public T Current
{
get
{
if (currentEnumerator != null)
return currentEnumerator.Current;
else
return default(T);
}
}
public void Dispose()
{
if (currentEnumerator != null)
currentEnumerator.Dispose();
}
object IEnumerator.Current
{
get
{
return Current;
}
}
public bool MoveNext()
{
if (currentIndex >= collections.Length)
return false;
if (currentIndex < 0)
{
currentIndex = 0;
if (collections.Length > 0)
currentEnumerator = collections[0].GetEnumerator();
else
return false;
}
while (!currentEnumerator.MoveNext())
{
currentEnumerator.Dispose();
currentEnumerator = null;
currentIndex++;
if (currentIndex >= collections.Length)
return false;
currentEnumerator = collections[currentIndex].GetEnumerator();
}
return true;
}
public void Reset()
{
if (currentEnumerator != null)
{
currentEnumerator.Dispose();
currentEnumerator = null;
}
this.currentIndex = -1;
}
}
}
Try the SelectMany extension method added in 3.5.
IEnumerable<IEnumerable<int>> e = ...;
foreach ( int cur in e.SelectMany(x => x)) {
Console.WriteLine(cur);
}
The code SelectMany(x => x) has the effect of flattening a collection of collections into a single collection. This is done in a lazy fashion and allows for straight forward processing as shown above.
If you only have C# 2.0 available, you can use an iterator to achieve the same results.
public static IEnumerable<T> Flatten<T>(IEnumerable<IEnumerable<T>> enumerable) {
foreach ( var inner in enumerable ) {
foreach ( var value in inner ) {
yield return value;
}
}
}
Just use the Enumerable.Concat() extension method to "concatenate" two IEnumerables. Don't worry, it doesn't actually copy them into a single array (as you might infer from the name), it simply allows you to enumerate over them all as if they were one IEnumerable.
If you have more than two then Enumerable.SelectMany() would be better.
Related
Consider the following trivial example:
class Filter<T, U>
where T : IEnumerable<U>
where U : IEquatable<U>
{
private readonly U id;
public Filter(U id) => this.id = id;
public T Eval_A(T items)
{
foreach (var item in items)
if (item.Equals(id))
yield return item;
}
public IEnumerable<U> Eval_B(IEnumerable<U> items)
{
foreach (var item in items)
if (item.Equals(id))
yield return item;
}
}
Even though T is of type IEnumerable<U>, an error still occurs for T Eval_A(T items) as follows.
The body of 'accessor' cannot be an iterator block because 'type' is not an iterator interface type
MSDN said:
This error occurs if an iterator accessor is used but the return type is not one of the iterator interface types: IEnumerable, IEnumerable<T>, IEnumerator, IEnumerator<T>. To avoid this error, use one of the iterator interface types as a return type.
I think the return type does conforms to the requirement.
What am I missing here?
The return type of the method has to be exactly one of IEnumerable, IEnumerable<T>, IEnumerator or IEnumerator<T>, and not T, which is some type that implements IEnumerable<U>.
This is because the way that iterator methods work, is that the compiler generates a type implementing one of the above interfaces, with the MoveNext and Current correctly implemented so that it matches the behaviour of your method, and then it rewrites your method to return an instance of that type instead.
For example, for:
public IEnumerable<int> GetSingleDigitNumbersLoop()
{
int index = 0;
while (index < 10)
yield return index++;
}
The following is generated (seen using SharpLab.io):
[IteratorStateMachine(typeof(<GetSingleDigitNumbersLoop>d__0))]
public IEnumerable<int> GetSingleDigitNumbersLoop()
{
return new <GetSingleDigitNumbersLoop>d__0(-2);
}
[CompilerGenerated]
private sealed class <GetSingleDigitNumbersLoop>d__0 : IEnumerable<int>, IEnumerable, IEnumerator<int>, IDisposable, IEnumerator
{
private int <>1__state;
private int <>2__current;
private int <>l__initialThreadId;
private int <index>5__2;
int IEnumerator<int>.Current
{
[DebuggerHidden]
get
{
return <>2__current;
}
}
object IEnumerator.Current
{
[DebuggerHidden]
get
{
return <>2__current;
}
}
[DebuggerHidden]
public <GetSingleDigitNumbersLoop>d__0(int <>1__state)
{
this.<>1__state = <>1__state;
<>l__initialThreadId = Environment.CurrentManagedThreadId;
}
[DebuggerHidden]
void IDisposable.Dispose()
{
}
private bool MoveNext()
{
int num = <>1__state;
if (num != 0)
{
if (num != 1)
{
return false;
}
<>1__state = -1;
}
else
{
<>1__state = -1;
<index>5__2 = 0;
}
if (<index>5__2 < 10)
{
<>2__current = <index>5__2++;
<>1__state = 1;
return true;
}
return false;
}
bool IEnumerator.MoveNext()
{
//ILSpy generated this explicit interface implementation from .override directive in MoveNext
return this.MoveNext();
}
[DebuggerHidden]
void IEnumerator.Reset()
{
throw new NotSupportedException();
}
[DebuggerHidden]
IEnumerator<int> IEnumerable<int>.GetEnumerator()
{
if (<>1__state == -2 && <>l__initialThreadId == Environment.CurrentManagedThreadId)
{
<>1__state = 0;
return this;
}
return new <GetSingleDigitNumbersLoop>d__0(0);
}
[DebuggerHidden]
IEnumerator IEnumerable.GetEnumerator()
{
return ((IEnumerable<int>)this).GetEnumerator();
}
}
<GetSingleDigitNumbersLoop>d__0 is the type that implements IEnumerable<int> that GetSingleDigitNumbersLoop is rewritten to return instead.
Now, if GetSingleDigitNumbersLoop were declared to return some other type, like a type parameter T for example, the following wouldn't work:
public T GetSingleDigitNumbersLoop()
{
return new <GetSingleDigitNumbersLoop>d__0(-2);
}
Because <GetSingleDigitNumbersLoop>d__0 is not necessarily a T. In fact it almost never would be, because it is a compiler generated type!
See also: yield statement implementation
I am making a prototype application and for that I designed a class that behaves like an infinite looping list. That is, if my internal list contains 100 values, when I ask for the 101st value, I get the first, the 102nd yields the second, and so on, repeating.
So I would like to write the following code:
var slice = loopingListInstance.Skip(123).Take(5);
And for that I need to implement IEnumerable suitable, as I understand.
Here is my current code:
public class InfiniteLoopingList : IEnumerable<double>
{
double[] _values = File.ReadLines(#"c:\file.txt")
.Select(s => double.Parse(s, CultureInfo.InvariantCulture))
.ToArray();
int _size;
public InfiniteLoopingList()
{
_size = _values.Length;
}
public double this[int i]
{
get { return _values[i % _size]; }
set { _values[i % _size] = value; }
}
public IEnumerator<double> GetEnumerator()
{
return this.GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
// ???? now what ?? :(
}
}
Since you implemented the indexer property, you could do it via the simplest way as follows:
public IEnumerator<double> GetEnumerator()
{
int i = 0;
while (true)
yield return this[i++];
}
IEnumerator IEnumerable.GetEnumerator()
{
return GetEnumerator();
}
EDIT
Please notice, that this is not really infinite loop. This approach will only work until i = int.MaxValue. Thanks to #oleksii.
You don't need a class for this...
An extension method will do the trick:
public static class InfEx
{
public static IEnumerable<T> LoopForever<T>(this IEnumerable<T> src)
{
var data = new List<T>();
foreach(var item in src)
{
data.Add(item);
yield return item;
}
for(;;)
{
foreach(var item in data)
{
yield return item;
}
}
}
}
Now you can take a sequence and make it a looping, infinite sequence:
IEnumerable<Foo> mySeq = ...;
IEnumerable<Foo> infMySeq = mySeq.LoopForver();
IEnumerable<Foo> aSelectionOfInfMySeq = infMySeq.Skip(101).Take(5);
You can implement the IEnumerator interface:
class InifniteEnumerator<T> : IEnumerator<T> {
private int index = -1;
private IList<T> innerList;
private int repeatPos;
public InifniteEnumerator(IList<T> innerList, int repeatPos) {
this.innerList = innerList;
this.repeatPos = repeatPos;
}
public T Current {
get {
if (index == -1) {
throw new InvalidOperationException();
}
return this.innerList[index];
}
}
object IEnumerator.Current {
get {
return this.Current;
}
}
public void Dispose() {
}
public bool MoveNext() {
this.index++;
if (this.index == repeatPos) {
this.index = 0;
}
return true;
}
public void Reset() {
this.index = -1;
}
}
and then return an instance of it in the GetEnumerator methods:
IEnumerator IEnumerable.GetEnumerator() {
return this.GetEnumerator();
}
public IEnumerator<T> IEnumerable<T>.GetEnumerator() {
return new InifniteEnumerator(this, 100);
}
After making a search at the Google, I found this discussion:
Possible to iterate backwards through a foreach?
But in the answers there is used extension method .Reverse(). With reverse, the list of objects, for ie. List of Strings, wtill be reversed first, and foreach doesn't reverse the list with my understoodment? If I got list "Cat", "Dog", and use .Reverse() -method, the list will be "Dog", "Cat", and foreach starts from the 0 element till the lenght-1 -element and that's what I'm not looking for. I would want to know, if there was any way to reverse foreach iteration order, to start from lenght-1 down to 0.
if there was any way to reverse foreach iteration order, to start from length-1 down to 0
Not for a List<T>. The implementation of GetEnumerator() returns an enumerator that enumerates from beginning to end - there's no way to override that.
With a custom collection, then you'd just have to use a different enumerator that could go backwards, but there's no way to override the implementation that List<T> uses.
The Reverse method will copy the list first:
public static IEnumerable<TSource> Reverse<TSource>(this IEnumerable<TSource> source) {
if (source == null) throw Error.ArgumentNull("source");
return ReverseIterator<TSource>(source);
}
static IEnumerable<TSource> ReverseIterator<TSource>(IEnumerable<TSource> source) {
Buffer<TSource> buffer = new Buffer<TSource>(source);
for (int i = buffer.count - 1; i >= 0; i--) yield return buffer.items[i];
}
But you can do an extension method yourself:
public static IEnumerable<TSource> Backwards<TSource>(this IList<TSource> source) {
for (var i = source.Count - 1; i >= 0; --i)
yield return source[i];
}
And then use it like that:
foreach (var item in array.Backwards())
Console.WriteLine(item); // Or whatever else
Or, of course, you could just do the equivalent:
for (var i = array.Length - 1; i >= 0; --i)
Console.WriteLine(array[i]); // Or whatever else
You can implement an enumerator that iterates though a list backwards. That way you can use foreach without changing the original list or creating a copy of it.
public class ReverseEnumerator<T> : IEnumerator<T> {
private IList<T> _list;
private int _index;
private T _current;
public ReverseEnumerator(IList<T> list) {
_list = list;
Reset();
}
public IEnumerator<T> GetEnumerator() {
return this;
}
public T Current {
get {
if (_index < 0 && _index >= _list.Count) throw new InvalidOperationException("Enumeration has not started. Call MoveNext.");
return _current;
}
}
public void Dispose() { }
object IEnumerator.Current { get { return Current; } }
public bool MoveNext() {
bool ok = --_index >= 0;
if (ok) _current = _list[_index];
return ok;
}
public void Reset() {
_index = _list.Count;
}
}
Usage example:
int[] a = { 1, 2, 3, 4, 5 };
foreach (int x in new ReverseEnumerator<int>(a)) {
Console.WriteLine(x);
}
I've found myself in a position where I have to roll my own dynamic array implementation, due to various large performance benefits (in my case). However, after creating an enumerator for my version, and comparing the efficiency with the one List uses, I'm a bit bewildered; the List one is aproximately 30-40% faster than my version, even though it's much more complex.
Here's the important part of the List enumerator implementation:
public struct Enumerator : IEnumerator<T>, IDisposable, IEnumerator
{
private List<T> list;
private int index;
private int version;
private T current;
internal Enumerator(List<T> list)
{
this.list = list;
this.index = 0;
this.version = list._version;
this.current = default(T);
return;
}
public bool MoveNext()
{
List<T> list;
list = this.list;
if (this.version != list._version)
{
goto Label_004A;
}
if (this.index >= list._size)
{
goto Label_004A;
}
this.current = list._items[this.index];
this.index += 1;
return 1;
Label_004A:
return this.MoveNextRare();
}
public T Current
{
get { return this.current; }
}
}
And here's my very barebone version:
internal struct DynamicArrayEnumerator<T> : IEnumerator<T> where T : class
{
private readonly T[] internalArray;
private readonly int lastIndex;
private int currentIndex;
internal DynamicArrayEnumerator(DynamicArray<T> dynamicArray)
{
internalArray = dynamicArray.internalArray;
lastIndex = internalArray.Length - 1;
currentIndex = -1;
}
public T Current
{
get { return internalArray[currentIndex]; }
}
public bool MoveNext()
{
return (++currentIndex <= lastIndex);
}
}
I know this is micro-optimization, but I'm actually interested in understanding why the List enumerator is so much faster than mine. Any ideas? Thanks!
Edit:
As requested; the DynamicArray class (the relevant parts):
The enumerator is an inner class in this.
public struct DynamicArray<T> : IEnumerable<T> where T : class
{
private T[] internalArray;
private int itemCount;
internal T[] Data
{
get { return internalArray; }
}
public int Count
{
get { return itemCount; }
}
public DynamicArray(int count)
{
this.internalArray = new T[count];
this.itemCount = 0;
}
public IEnumerator<T> GetEnumerator()
{
return new DynamicArrayEnumerator<T>(this);
}
IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
}
As for how I'm testing:
List<BaseClass> list = new List<BaseClass>(1000000);
DynamicArray<BaseClass> dynamicArray = new DynamicArray<BaseClass>(1000000);
// Code for filling with data omitted.
int numberOfRuns = 0;
float p1Total = 0;
float p2Total = 0;
while (numberOfRuns < 100)
{
PerformanceAnalyzer p1 = new PerformanceAnalyzer(() =>
{
int u = 0;
foreach (BaseClass b in list)
{
if (b.B > 100) // Some trivial task
u++;
}
});
p1.ExecuteAndClock();
p1Total += p1.TotalElapsedTicks;
PerformanceAnalyzer p2 = new PerformanceAnalyzer(() =>
{
int u = 0;
foreach (BaseClass b in dynamicArray)
{
if (b.B > 100) // Some trivial task
u++;
}
});
p2.ExecuteAndClock();
p2Total += p2.TotalElapsedTicks;
numberOfRuns++;
}
Console.WriteLine("List enumeration: " + p1Total / totalRuns + "\n");
Console.WriteLine("Dynamic array enumeration: " + p2Total / totalRuns + "\n");
The PerformanceAnalyzer class basically starts a Stopwatch, execute the supplied Action delegate, and then stop the Stopwatch afterwards.
Edit 2 (Quick answer to Ryan Gates):
There's a few reasons why I would want to roll my own, most importantly I need a very fast RemoveAt(int index) method.
Since I don't have to worry about the order of the list elements in my particular case, I can avoid the .Net built-in list's way of doing it:
public void RemoveAt(int index)
{
T local;
if (index < this._size)
{
goto Label_000E;
}
ThrowHelper.ThrowArgumentOutOfRangeException();
Label_000E:
this._size -= 1;
if (index >= this._size)
{
goto Label_0042;
}
Array.Copy(this._items, index + 1, this._items, index, this._size - index);
Label_0042:
this._items[this._size] = default(T);
this._version += 1;
return;
}
And instead using something along the lines of:
public void RemoveAt(int index)
{
// overwrites the element at the specified index with the last element in the array and decreases the item count.
internalArray[index] = internalArray[itemCount];
itemCount--;
}
Potencially saving enormous amounts of time in my case, if say the first 1000 elements in a long list have to be removed by index.
Okay, aside from benchmarking problems, here's how you can make your DynamicArray class more like List<T>:
public DynamicArrayEnumerator<T> GetEnumerator()
{
return new DynamicArrayEnumerator<T>(this);
}
IEnumerator<T> IEnumerable<T>.GetEnumerator()
{
return GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
Now, code which knows it's working with a dynamic array can iterate with a DynamicArrayEnumerator<T> without any boxing, and without virtual dispatch. This is exactly what List<T> does. The compiler notices when a type implements the pattern in a custom manner, and will use the types involved instead of the interfaces.
With your current code, you're getting no benefit from creating a struct - because you're boxing it in GetEnumerator().
Try the above change and fix the benchmark to work for longer. I'd expect to see a big difference.
I have to implement a call graph for expressions like Id = Id(Param); and that wasn't a problem.
Now I have to implement an enumerator which lists one at a time all topological orderings among the calls that satisfy the order of dependencies.
And here's the trouble.
This is a simple node for the call graph:
class CallGraphNode
{
private string name;
public List<CallGraphNode> dependents = new List<CallGraphNode>();
public int dependencies;
private bool executed = false;
public bool Executable { get { return dependencies == 0; } }
public bool Executed { get { return executed; } set { executed = value; } }
public CallGraphNode(string name)
{
this.name = name;
dependencies = 0;
}
public override string ToString()
{
return name;
}
public void AddDependent(CallGraphNode n)
{
dependents.Add(n);
}
}
And this is the call graph class itself:
class CallGraph : IEnumerable<List<CallGraphNode>>
{
public List<CallGraphNode> nodes = new List<CallGraphNode>();
public void AddNode(CallGraphNode n)
{
nodes.Add(n);
}
public static void Show(IEnumerable<CallGraphNode> n)
{
foreach (CallGraphNode node in n)
{
Console.Write("{0} ", node);
}
Console.WriteLine();
}
static IEnumerable<List<CallGraphNode>> EnumerateFunctions(List<CallGraphNode> executable, List<CallGraphNode> res)
{
if (executable.Count == 0)
yield return res;
else foreach (CallGraphNode n in executable)
{
if (!n.Executed)
res.Add(n);
List<CallGraphNode> next_executable = new List<CallGraphNode>(executable);
executable.Remove(n);
foreach (CallGraphNode m in n.dependents)
if (--m.dependencies == 0)
next_executable.Add(m);
foreach (List<CallGraphNode> others in EnumerateFunctions(next_executable, res))
yield return others;
foreach (CallGraphNode m in n.dependents)
m.dependencies++;
if (!n.Executed)
res.Remove(n);
}
}
IEnumerator<List<CallGraphNode>> IEnumerable<List<CallGraphNode>>.GetEnumerator()
{
List<CallGraphNode> executable = new List<CallGraphNode>();
foreach (CallGraphNode n in nodes)
if (n.Executable || n.Executed)
executable.Add(n);
List<CallGraphNode> output = new List<CallGraphNode>();
foreach (List<CallGraphNode> list in EnumerateFunctions(executable, output))
yield return list;
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator()
{ throw new NotImplementedException(); }
}
Now, the problem is it just won't work. When I try to create an IEnumerator and assign it the GetEnumerator() return value, I get a casting error and that's honestly what I expected when trying to do so:
IEnumerator<List<CallGraphNode>> lt = cg.GetEnumerator();
Then I've tried:
System.Collections.Generic.List<CallGraphNode>.Enumerator en = cg.nodes.GetEnumerator();
This works, but the method EnumerateFunctions is never called and the enumerator just contains the original list of graph nodes.
Any ideas?
The problem is that you're implementing both IEnumerable<T> and IEnumerable using explicit interface implementation.
You probably want to change this declaration:
IEnumerator<List<CallGraphNode>> IEnumerable<List<CallGraphNode>>.GetEnumerator()
to be a "normal" interface implementation:
public IEnumerator<List<CallGraphNode>> GetEnumerator()
Alternatively, you could stick with explicit interface implementation, but use:
IEnumerable<List<CallGraphNode>> sequence = cg;
IEnumerator<List<CallGraphNode>> lt = sequence.GetEnumerator();