Related
(Sorry for the vague title; couldn't think of anything better. Feel free to rephrase.)
So let's say my function or property returns an IEnumerable<T>:
public IEnumerable<Person> Adults
{
get
{
return _Members.Where(i => i.Age >= 18);
}
}
If I run a foreach on this property without actually materializing the returned enumerable:
foreach(var Adult in Adults)
{
//...
}
Is there a rule that governs whether IEnumerable<Person> will be materialized to array or list or something else?
Also is it safe to cast Adults to List<Person> or Array without calling ToList() or ToArray()?
Edit
Many people have spent a lot of effort into answering this question. Thanks to all of them. However, the gist of this question still remains unanswered. Let me put in some more details:
I understand that foreach doesn't require the target object to be an array or list. It doesn't even need to be a collection of any kind. All it needs the target object to do is to implement enumeration. However if I place inspect the value of target object, it reveals that the actual underlying object is List<T> (just like it shows object (string) when you inspect a boxed string object). This is where the confusion starts. Who performed this materialization? I inspected the underlying layers (Where() function's source) and it doesn't look like those functions are doing this.
So my problem lies at two levels.
First one is purely theoretical. Unlike many other disciplines like physics and biology, in computer sciences we always know precisely how something works (answering #zzxyz's last comment); so I was trying to dig about the agent who created List<T> and how it decided it should choose a List and not an Array and if there is a way of influencing that decision from our code.
My second reason was practical. Can I rely on the type of actual underlying object and cast it to List<T>? I need to use some List<T> functionality and I was wondering if for example ((List<Person>)Adults).BinarySearch() is as safe as Adults.ToList().BinarySearch()?
I also understand that it isn't going to create any performance penalty even if I do call ToList() explicitly. I was just trying to understand how it is working. Anyway, thanks again for the time; I guess I have spent just too much time on it.
In general terms all you need for a foreach to work is to have an object with an accessible GetEnumerator() method that returns an object that has the following methods:
void Reset()
bool MoveNext()
T Current { get; private set; } // where `T` is some type.
You don't even need an IEnumerable or IEnumerable<T>.
This code works as the compiler figures out everything it needs:
void Main()
{
foreach (var adult in new Adults())
{
Console.WriteLine(adult.ToString());
}
}
public class Adult
{
public override string ToString() => "Adult!";
}
public class Adults
{
public class Enumerator
{
public Adult Current { get; private set; }
public bool MoveNext()
{
if (this.Current == null)
{
this.Current = new Adult();
return true;
}
this.Current = null;
return false;
}
public void Reset() { this.Current = null; }
}
public Enumerator GetEnumerator() { return new Enumerator(); }
}
Having a proper enumerable makes the process work more easily and more robustly. The more idiomatic version of the above code is:
public class Adults
{
private class Enumerator : IEnumerator<Adult>
{
public Adult Current { get; private set; }
object IEnumerator.Current => this.Current;
public void Dispose() { }
public bool MoveNext()
{
if (this.Current == null)
{
this.Current = new Adult();
return true;
}
this.Current = null;
return false;
}
public void Reset()
{
this.Current = null;
}
}
public IEnumerator<Adult> GetEnumerator()
{
return new Enumerator();
}
}
This enables the Enumerator to be a private class, i.e. private class Enumerator. The interface then does all of the hard work - it's not even possible to get a reference to the Enumerator class outside of Adults.
The point is that you do not know at compile-time what the concrete type of the class is - and if you did you may not even be able to cast to it.
The interface is all you need, and even that isn't strictly true if you consider my first example.
If you want a List<Adult> or an Adult[] you must call .ToList() or .ToArray() respectively.
There is no such thing as a default concrete type for any interface.
The entire point of an interface is to guarantee properties, methods, events or indexers, without the user need of any knowledge of the concrete type that implements it.
When using an interface, all you can know is the properties, methods, events and indexers this interface declares, and that's all you actually need to know. That's just another aspect of encapsulation - same as when you are using a method of a class you don't need to know the internal implementation of that method.
To answer your question in the comments:
who decides that concrete type in case we don't, just as I did above?
That's the code that created the instance that's implementing the interface.
Since you can't do var Adults = new IEnumerable<Person> - it has to be a concrete type of some sort.
As far as I see in the source code for linq's Enumerable extensions - the where returns either an instance of Iterator<TSource> or an instance of WhereEnumerableIterator<TSource>. I didn't bother checking further what exactly are those types, but I can pretty much guarantee they both implement IEnumerable, or the guys at Microsoft are using a different c# compiler then the rest of us... :-)
The following code hopefully highlights why neither you nor the compiler can assume an underlying collection:
public class OneThroughTen : IEnumerable<int>
{
private static int bar = 0;
public IEnumerator<int> GetEnumerator()
{
while (true)
{
yield return ++bar;
if (bar == 10)
{ yield break; }
}
}
IEnumerator IEnumerable.GetEnumerator() { return GetEnumerator(); }
}
class Program
{
static void Main(string[] args)
{
IEnumerable<int> x = new OneThroughTen();
foreach (int i in x)
{ Console.Write("{0} ", i); }
}
}
Output being, of course:
1 2 3 4 5 6 7 8 9 10
Note, the code above behaves extremely poorly in the debugger. I don't know why. This code behaves just fine:
public IEnumerator<int> GetEnumerator()
{
while (bar < 10)
{
yield return ++bar;
}
bar = 0;
}
(I used static for bar to highlight that not only does the OneThroughTen not have a specific collection, it doesn't have any collection, and in fact has no instance data whatsoever. We could just as easily return 10 random numbers, which would've been a better example, now that I think on it :))
From your edited question and comments it sounds like you understand the general concept of using IEnumerable, and that you cannot assume that "a list object backs all IEnumerable objects". Your real question is about something that has confused you in the debugger, but we've not really been able to understand exactly what it is you are seeing. Perhaps a screenshot would help?
Here I have 5 IEnumerable<int> variables which I assign in various ways, along with how the "Watch" window describes them. Does this show the confusion you are having? If not, can you construct a similarly short program and screenshot that does?
Coming a bit late into the party here :)
Actually Linq's "Where" decides what's going to be the underlying implementation of IEnumerable's GetEnumerator.
Look at the source code:
https://github.com/dotnet/runtime/blob/918e6a9a278bc66fb191c43d4db4a71e63ffad31/src/libraries/System.Linq/src/System/Linq/Where.cs#L59
You'll see that based on the "source" type, the methods return "WhereSelectArrayIterator" or "WhereSelectListIterator" or a more generic "WhereSelectEnumerableSelector".
Each of this objects implement the GetEnumerator over an Array, or a List, so I'm pretty sure that's why you see the underlying object type being one of these on VS inspector.
Hope this helps clarifying.
I have been digging into this myself. I believe the 'underlying type' is an iterator method, not an actual data structure type.
An iterator method defines how to generate the objects in a sequence
when requested.
https://learn.microsoft.com/en-us/dotnet/csharp/iterators#enumeration-sources-with-iterator-methods
In my usecase/testing, the iterator is System.Linq.Enumerable.SelectManySingleSelectorIterator. I don't think this is a collection data type. It is a method that can enumerate IEnumerables.
Here is a snippet:
public IEnumerable<Item> ItemsToBuy { get; set; }
...
ItemsToBuy = Enumerable.Range(1, rng.Next(1, 20))
.Select(RandomItem(rng, market))
.SelectMany(e => e);
The property is IEnumerable and .SelectMany returns IEnumerable. So what is the actual collection data structure? I don't think there is one in how I am interpreting 'collection data structure'.
Also is it safe to cast Adults to List or Array without
calling ToList() or ToArray()?
Not for me. When attempting to cast ItemsToBuy collection in a foreach loop I get the following runtime exception:
{"Unable to cast object of type
'SelectManySingleSelectorIterator2[System.Collections.Generic.IEnumerable1[CashMart.Models.Item],CashMart.Models.Item]'
to type 'CashMart.Models.Item[]'."}
So I could not cast, but I could .ToArray(). I do suspect there is a performance hit as I would think that the IEnumerable would have to 'do things' to make it an array, including memory allocation for the array even if the entities are already in memory.
However if I place inspect the value of target object, it reveals that
the actual underlying object is List
This was not my experience and I think it may depend on the IEnumerable source as well as the LinQ provider. If I add a where, the returned iterator is:
System.Linq.Enumerable.WhereEnumerableIterator
I am unsure what your _Member source is, but using LinQ-to-Objects, I get an iterator. LinQ-to-Entities must call the database and store the result set in memory somehow and then enumerate on that result. I would doubt that it internally makes it a List, but I don't know much. I suspect instead that _Members may be a List somewhere else in your code thus, even after the .Where, it shows as a List.
I am wandering about the more in-depth functionality of the IEnumerable<T> interface.
Basically, it works as an intermediary step in execution. For example, if you write:
IEnumerable<int> temp = new int[]{1,2,3}.Select(x => 2*x);
The result of the Select function will not be calculated (enumerated) until something is done with temp to allow it (such as List<int> list = temp.ToList()).
However, what puzzles me is, since IEnumerable<T> is an interface, it cannot, by definition, be instantiated. So, what is the collection the actual items (in the example 2*x items) reside in?
Moreover, if we were to write IEnumerable<int> temp = Enumerable.Repeat(1, 10);, what would be the underlying collection where the 1s are stored (array, list, something else)?
I cannot seem to find a thorough (more in-depth) explanation as to the actual implementation of this interface and its functionality (for example, if there is an underlying collection, how does the yield keyword work).
Basically, what I am asking for is a more elaborate explanation on the functionality of IEnumerable<T>.
Implementation shouldn't matter. All these (LINQ) methods return IEnumerable<T>, interface members are the only members you can access, and that should be enough to use them.
However, if you really have to know, you can find actual implementations on http://sourceof.net.
Enumerable.cs
But, for some of the methods you won't be able to find explicit class declaration, because some of them use yield return, which means proper class (with state machine) is generated by compiler during compilation. e.g. Enumerable.Repeat is implemented that way:
public static IEnumerable<int> Range(int start, int count) {
long max = ((long)start) + count - 1;
if (count < 0 || max > Int32.MaxValue)
throw Error.ArgumentOutOfRange("count");
return RangeIterator(start, count);
}
static IEnumerable<int> RangeIterator(int start, int count) {
for (int i = 0; i < count; i++)
yield return start + i;
}
You can read more about that on MSDN: Iterators (C# and Visual Basic)
Not all objects that implement IEnumerable defer execution in some way. The API of the interface makes it possible to defer execution, but it doesn't require it. There are likewise implementations that don't defer execution in any way.
So, what is the collection the actual items (in the example 2*x items) reside in?
There is none. Whenever the next value is requested it computes that one value on demand, gives it to the caller, and then forgets the value. It doesn't store it anywhere else.
Moreover, if we were to write IEnumerable<int> temp = Enumerable.Repeat(1, 10);, what would be the underlying collection where the 1s are stored (array, list, something else)?
There wouldn't be one. It would compute each new value immediately when you ask for the next value and it won't remember it afterward. It only stores enough information to be able to compute the next value, which means it only needs to store the element and the number of values left to yield.
While the actual .NET implementations will use much more concise means of creating such a type, creating an enumerable that defers execution is not particularly hard. Doing so even the long way is more tedious than difficult. You simply compute the next value in the MoveNext method of the iterator. In the example you asked of, Repeat, this is easy as you only need to compute if there is another value, not what it is:
public class Repeater<T> : IEnumerator<T>
{
private int count;
private T element;
public Repeater(T element, int count)
{
this.element = element;
this.count = count;
}
public T Current { get { return element; } }
object IEnumerator.Current
{
get { return Current; }
}
public void Dispose() { }
public bool MoveNext()
{
if (count > 0)
{
count--;
return true;
}
else
return false;
}
public void Reset()
{
throw new NotSupportedException();
}
}
(I've omitted an IEnumerable type that just returns a new instance of this type, or a static Repeat method that creates a new instance of that enumerable. There isn't anything particularly interesting to see there.)
A slightly more interesting example would be something like Count:
public class Counter : IEnumerator<int>
{
private int remaining;
public Counter(int start, int count)
{
Current = start;
this.remaining = count;
}
public int Current { get; private set; }
object IEnumerator.Current
{
get { return Current; }
}
public void Dispose() { }
public bool MoveNext()
{
if (remaining > 0)
{
remaining--;
Current++;
return true;
}
else
return false;
}
public void Reset()
{
throw new NotSupportedException();
}
}
Here we're not only computing if we have another value, but what that next value is, each time a new value is requested of us.
So, what is the collection the actual items (in the example 2*x items) reside in?
It is not residing anywhere. There is code that will produce the individual items "on demand" when you iterate, but the 2*x numbers are not computed upfront. They are also not stored anywhere, unless you call ToList or ToArray.
Moreover, if we were to write IEnumerable temp = Enumerable.Repeat(1, 10);, what would be the underlying collection where the 1s are stored (array, list, something else)?
The same picture is here: the returned implementation of IEnumerable is not public, and it returns its items on demand, without storing them anywhere.
C# compiler provides a convenient way to implement IEnumerable without defining a class for it. All you need is to declare your method return type as IEnumerable<T>, and use yield return to supply values on as-needed basis.
I'm trying to implement the Iterator pattern.
Basically, from what I understand, it makes a class "foreachble" and makes the code more secure by not revealing the exact collection type to the user.
I have been experimenting a bit and I found out that if I implement
IEnumerator GetEnumerator() in my class, I get the desired result ... seemingly sparing the headache of messing around with realizing interfaces.
Here is a glimpse to what I mean:
public class ListUserLoggedIn
{
/*
stuff
*/
public List<UserLoggedIn> UserList { get; set; }
public IEnumerator<UserLoggedIn> GetEnumerator()
{
foreach (UserLoggedIn user in this.UserList)
{
yield return user;
}
}
public void traverse()
{
foreach (var item in ListUserLoggedIn.Instance)
{
Console.Write(item.Id);
}
}
}
I guess my question is, is this a valid example of Iterator?
If yes, why is this working, and what can I do to make the iterator return only a part or an anonymous object via "var".
If not, what is the correct way ...
First a smaller and simplified self-contained version:
class Program
{
public IEnumerator<int> GetEnumerator() // IEnumerable<int> works too.
{
for (int i = 0; i < 5; i++)
yield return i;
}
static void Main(string[] args)
{
var p = new Program();
foreach (int x in p)
{
Console.WriteLine(x);
}
}
}
And the 'strange' thing here is that class Program does not implement IEnumerable.
The specs from Ecma-334 say:
§ 8.18 Iterators
The foreach statement is used to iterate over the elements of an enumerable collection. In order to be enumerable, a collection shall have a parameterless GetEnumerator method that returns an enumerator.
So that's why foreach() works on your class. No mention of IEnumerable. But how does the GetEnumerator() produce something that implements Current and MoveNext ? From the same section:
An iterator is a statement block that yields an ordered sequence of values. An iterator is distinguished from a normal statement block by the presence of one or more yield statements
It is important to understand that an iterator is not a kind of member, but is a means of implementing a function member
So the body of your method is an iterator-block, the compiler checks a number of constraints (the method must return an IEnumerable or IEnumerator) and then implements the IEnumerator members for you.
And to the deeper "why", I just learned something too. Based on an annotation by Eric Lippert in "The C# Programming Language 3rd", page 369:
This is called the "pattern-based approach" and it dates from before generics. An interface based approach in C#1 would have been totally based on passing object around and value types would always have had to be boxed. The pattern approach allows
foreach (int x in myIntCollection)
without generics and without boxing. Neat.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
Can anyone explain IEnumerable and IEnumerator to me?
For example, when to use it over foreach? what's the difference between IEnumerable and IEnumerator? Why do we need to use it?
for example, when to use it over foreach?
You don't use IEnumerable "over" foreach. Implementing IEnumerable makes using foreach possible.
When you write code like:
foreach (Foo bar in baz)
{
...
}
it's functionally equivalent to writing:
IEnumerator bat = baz.GetEnumerator();
while (bat.MoveNext())
{
bar = (Foo)bat.Current
...
}
By "functionally equivalent," I mean that's actually what the compiler turns the code into. You can't use foreach on baz in this example unless baz implements IEnumerable.
IEnumerable means that baz implements the method
IEnumerator GetEnumerator()
The IEnumerator object that this method returns must implement the methods
bool MoveNext()
and
Object Current()
The first method advances to the next object in the IEnumerable object that created the enumerator, returning false if it's done, and the second returns the current object.
Anything in .Net that you can iterate over implements IEnumerable. If you're building your own class, and it doesn't already inherit from a class that implements IEnumerable, you can make your class usable in foreach statements by implementing IEnumerable (and by creating an enumerator class that its new GetEnumerator method will return).
The IEnumerable and IEnumerator Interfaces
To begin examining the process of implementing existing .NET interfaces, let’s first look at the role of
IEnumerable and IEnumerator. Recall that C# supports a keyword named foreach that allows you to
iterate over the contents of any array type:
// Iterate over an array of items.
int[] myArrayOfInts = {10, 20, 30, 40};
foreach(int i in myArrayOfInts)
{
Console.WriteLine(i);
}
While it might seem that only array types can make use of this construct, the truth of the matter is
any type supporting a method named GetEnumerator() can be evaluated by the foreach construct.To
illustrate, follow me!
Suppose we have a Garage class:
// Garage contains a set of Car objects.
public class Garage
{
private Car[] carArray = new Car[4];
// Fill with some Car objects upon startup.
public Garage()
{
carArray[0] = new Car("Rusty", 30);
carArray[1] = new Car("Clunker", 55);
carArray[2] = new Car("Zippy", 30);
carArray[3] = new Car("Fred", 30);
}
}
Ideally, it would be convenient to iterate over the Garage object’s subitems using the foreach
construct, just like an array of data values:
// This seems reasonable ...
public class Program
{
static void Main(string[] args)
{
Console.WriteLine("***** Fun with IEnumerable / IEnumerator *****\n");
Garage carLot = new Garage();
// Hand over each car in the collection?
foreach (Car c in carLot)
{
Console.WriteLine("{0} is going {1} MPH",
c.PetName, c.CurrentSpeed);
}
Console.ReadLine();
}
}
Sadly, the compiler informs you that the Garage class does not implement a method named
GetEnumerator(). This method is formalized by the IEnumerable interface, which is found lurking within the System.Collections namespace.
Classes or structures that support this behavior advertise that they are able to expose contained
subitems to the caller (in this example, the foreach keyword itself). Here is the definition of this standard .NET interface:
// This interface informs the caller
// that the object's subitems can be enumerated.
public interface IEnumerable
{
IEnumerator GetEnumerator();
}
As you can see, the GetEnumerator() method returns a reference to yet another interface named
System.Collections.IEnumerator. This interface provides the infrastructure to allow the caller to traverse the internal objects contained by the IEnumerable-compatible container:
// This interface allows the caller to
// obtain a container's subitems.
public interface IEnumerator
{
bool MoveNext (); // Advance the internal position of the cursor.
object Current { get;} // Get the current item (read-only property).
void Reset (); // Reset the cursor before the first member.
}
If you want to update the Garage type to support these interfaces, you could take the long road and
implement each method manually. While you are certainly free to provide customized versions of
GetEnumerator(), MoveNext(), Current, and Reset(), there is a simpler way. As the System.Array type (as well as many other collection classes) already implements IEnumerable and IEnumerator, you can simply delegate the request to the System.Array as follows:
using System.Collections;
...
public class Garage : IEnumerable
{
// System.Array already implements IEnumerator!
private Car[] carArray = new Car[4];
public Garage()
{
carArray[0] = new Car("FeeFee", 200);
carArray[1] = new Car("Clunker", 90);
carArray[2] = new Car("Zippy", 30);
carArray[3] = new Car("Fred", 30);
}
public IEnumerator GetEnumerator()
{
// Return the array object's IEnumerator.
return carArray.GetEnumerator();
}
}
After you have updated your Garage type, you can safely use the type within the C# foreach construct. Furthermore, given that the GetEnumerator() method has been defined publicly, the object user could also interact with the IEnumerator type:
// Manually work with IEnumerator.
IEnumerator i = carLot.GetEnumerator();
i.MoveNext();
Car myCar = (Car)i.Current;
Console.WriteLine("{0} is going {1} MPH", myCar.PetName, myCar.CurrentSpeed);
However, if you prefer to hide the functionality of IEnumerable from the object level, simply make
use of explicit interface implementation:
IEnumerator IEnumerable.GetEnumerator()
{
// Return the array object's IEnumerator.
return carArray.GetEnumerator();
}
By doing so, the casual object user will not find the Garage’s GetEnumerator() method, while the
foreach construct will obtain the interface in the background when necessary.
Adapted from the Pro C# 5.0 and the .NET 4.5 Framework
Implementing IEnumerable means your class returns an IEnumerator object:
public class People : IEnumerable
{
IEnumerator IEnumerable.GetEnumerator()
{
// return a PeopleEnumerator
}
}
Implementing IEnumerator means your class returns the methods and properties for iteration:
public class PeopleEnumerator : IEnumerator
{
public void Reset()...
public bool MoveNext()...
public object Current...
}
That's the difference anyway.
Explanation via Analogy + Code Walkthrough
Analogy: Imagine you are a detective on an aeroplane . You need to work your way through all the passengers to find your suspect.
An aeroplane can only do this, if it is:
countable, and
if it has a counter.
What does countable mean?
If an airline is "countable", this means that there MUST be a flight attendant present on the plane, whose sole job is to count:
The counter/flight attendant MUST start before the first passenger 2. (i.e. the flight attendant) MUST "move next" up the aisle to the first seat.
He/she is to then record: (i) who the person is in the seat, and (ii) their current location in the aisle.
The counter keeps going till he reaches the end of the plane.
Let's tie this with the IEnumerables
foreach (Passenger passenger in Plane)
// the airline hostess is now at the front of the plane
// and slowly making her way towards the back
// when she get to a particular passenger she gets some information
// about the passenger and then immediately heads to the cabin
// to let the captain decide what to do with it
{ // <---------- Note the curly bracket that is here.
// we are now cockpit of the plane with the captain.
// the captain wants to give the passenger free
// champaign if they support manchester city
if (passenger.supports_mancestercity())
{
passenger.getFreeChampaign();
}
else
{
// you get nothing! GOOD DAY SIR!
}
} // <---- Note the curly bracket that is here!
// the hostess has delivered the information
// to the captain and goes to the next person
// on the plane (if she has not reached the
// end of the plane)
Summary
In other words, something is countable if it has a counter. And counter must (basically): (i) remember its place (state), (ii) be able to move next, (iii) and know about the current person he is dealing with.
Enumerable is just a fancy word for "countable".
IEnumerable implements GetEnumerator. When called, that method will return an IEnumerator which implements MoveNext, Reset and Current.
Thus when your class implements IEnumerable, you are saying that you can call a method (GetEnumerator) and get a new object returned (an IEnumerator) you can use in a loop such as foreach.
Implementing IEnumerable enables you to get an IEnumerator for a list.
IEnumerator allows foreach style sequential access to the items in the list, using the yield keyword.
Before foreach implementation (in Java 1.4, for example), the way to iterate a list was to get an enumerator from the list, then ask it for the "next" item in the list, for as long as the value returned as the next item is not null. Foreach simply does that implicitly as a language feature, in the same way that lock() implements the Monitor class behind the scenes.
I expect foreach works on lists because they implement IEnumerable.
An object implementing IEnumerable allows others to visit each of its items (by an enumerator).
An object implementing IEnumerator is the doing the iteration. It's looping over an enumerable object.
Think of enumerable objects as of lists, stacks, trees.
IEnumerable and IEnumerator (and their generic counterparts IEnumerable<T> and IEnumerator<T>) are base interfaces of iterator implementations in .Net Framework Class Libray collections.
IEnumerable is the most common interface you would see in the majority of the code out there. It enables the foreach loop, generators (think yield) and because of its tiny interface, it's used to create tight abstractions. IEnumerable depends on IEnumerator.
IEnumerator, on the other hand, provides a slightly lower level iteration interface. It's referred to as the explicit iterator which gives the programmer more control over the iteration cycle.
IEnumerable
IEnumerable is a standard interface that enables iterating over collections that supports it (in fact, all collection types I can think of today implements IEnumerable). Compiler support allows language features like foreach. In general terms, it enables this implicit iterator implementation.
foreach Loop
foreach (var value in list)
Console.WriteLine(value);
I think foreach loop is one of the main reasons for using IEnumerable interfaces. foreach has a very succinct syntax and very easy to understand compared to classic C style for loops where you need to check the various variables to see what it was doing.
yield Keyword
Probably a lesser known feature is that IEnumerable also enables generators in C# with the use of yield return and yield break statements.
IEnumerable<Thing> GetThings() {
if (isNotReady) yield break;
while (thereIsMore)
yield return GetOneMoreThing();
}
Abstractions
Another common scenario in practice is using IEnumerable to provide minimalistic abstractions. Because it is a minuscule and read-only interface, you are encouraged to expose your collections as IEnumerable (rather than List for example). That way you are free to change your implementation without breaking your client's code (change List to a LinkedList for instance).
Gotcha
One behaviour to be aware of is that in streaming implementations (e.g. retrieving data row by row from a database, instead of loading all the results in memory first) you cannot iterate over the collection more than once. This is in contrast to in-memory collections like List, where you can iterate multiple times without problems. ReSharper, for example, has a code inspection for Possible multiple enumeration of IEnumerable.
IEnumerator
IEnumerator, on the other hand, is the behind the scenes interface which makes IEnumerble-foreach-magic work. Strictly speaking, it enables explicit iterators.
var iter = list.GetEnumerator();
while (iter.MoveNext())
Console.WriteLine(iter.Current);
In my experience IEnumerator is rarely used in common scenarios due to its more verbose syntax and slightly confusing semantics (at least to me; e.g. MoveNext() returns a value as well, which the name doesn't suggest at all).
Use case for IEnumerator
I only used IEnumerator in particular (slightly lower level) libraries and frameworks where I was providing IEnumerable interfaces. One example is a data stream processing library which provided series of objects in a foreach loop even though behind the scenes data was collected using various file streams and serialisations.
Client code
foreach(var item in feed.GetItems())
Console.WriteLine(item);
Library
IEnumerable GetItems() {
return new FeedIterator(_fileNames)
}
class FeedIterator: IEnumerable {
IEnumerator GetEnumerator() {
return new FeedExplicitIterator(_stream);
}
}
class FeedExplicitIterator: IEnumerator {
DataItem _current;
bool MoveNext() {
_current = ReadMoreFromStream();
return _current != null;
}
DataItem Current() {
return _current;
}
}
Differences between IEnumerable and IEnumerator :
IEnumerable uses IEnumerator internally.
IEnumerable doesn't know which item/object is executing.
Whenever we pass IEnumerator to another function, it knows the current position of item/object.
Whenever we pass an IEnumerable collection to another function, it
doesn't know the current position of item/object (doesn't know which item its executing)
IEnumerable have one method GetEnumerator()
public interface IEnumerable<out T> : IEnumerable
{
IEnumerator<T> GetEnumerator();
}
IEnumerator has one property called Current and two methods, Reset() and MoveNext() (which is useful for knowing the current position of an item in a list).
public interface IEnumerator
{
object Current { get; }
bool MoveNext();
void Reset();
}
Implementing IEnumerable essentially means that the object can be iterated over. This doesn't necessarily mean it is an array as there are certain lists that can't be indexed but you can enumerate them.
IEnumerator is the actual object used to perform the iterations. It controls moving from one object to the next in the list.
Most of the time, IEnumerable & IEnumerator are used transparently as part of a foreach loop.
IEnumerable is a box that contains Ienumerator. IEnumerable is base interface for all the collections. foreach loop can operate if the collection implements IEnumerable. In the below code it explains the step of having our own Enumerator. Lets first define our Class of which we are going to make the collection.
public class Customer
{
public String Name { get; set; }
public String City { get; set; }
public long Mobile { get; set; }
public double Amount { get; set; }
}
Now we will define the Class which will act as a collection for our class Customer. Notice that it is implementing the interface IEnumerable. So that we have to implement the method GetEnumerator. This will return our custom Enumerator.
public class CustomerList : IEnumerable
{
Customer[] customers = new Customer[4];
public CustomerList()
{
customers[0] = new Customer { Name = "Bijay Thapa", City = "LA", Mobile = 9841639665, Amount = 89.45 };
customers[1] = new Customer { Name = "Jack", City = "NYC", Mobile = 9175869002, Amount = 426.00 };
customers[2] = new Customer { Name = "Anil min", City = "Kathmandu", Mobile = 9173694005, Amount = 5896.20 };
customers[3] = new Customer { Name = "Jim sin", City = "Delhi", Mobile = 64214556002, Amount = 596.20 };
}
public int Count()
{
return customers.Count();
}
public Customer this[int index]
{
get
{
return customers[index];
}
}
public IEnumerator GetEnumerator()
{
return customers.GetEnumerator(); // we can do this but we are going to make our own Enumerator
return new CustomerEnumerator(this);
}
}
Now we are going to create our own custom Enumerator as follow. So, we have to implement method MoveNext.
public class CustomerEnumerator : IEnumerator
{
CustomerList coll;
Customer CurrentCustomer;
int currentIndex;
public CustomerEnumerator(CustomerList customerList)
{
coll = customerList;
currentIndex = -1;
}
public object Current => CurrentCustomer;
public bool MoveNext()
{
if ((currentIndex++) >= coll.Count() - 1)
return false;
else
CurrentCustomer = coll[currentIndex];
return true;
}
public void Reset()
{
// we dont have to implement this method.
}
}
Now we can use foreach loop over our collection like below;
class EnumeratorExample
{
static void Main(String[] args)
{
CustomerList custList = new CustomerList();
foreach (Customer cust in custList)
{
Console.WriteLine("Customer Name:"+cust.Name + " City Name:" + cust.City + " Mobile Number:" + cust.Amount);
}
Console.Read();
}
}
An understanding of the Iterator pattern will be helpful for you. I recommend reading the same.
Iterator Pattern
At a high level the iterator pattern can be used to provide a standard way of iterating through collections of any type.
We have 3 participants in the iterator pattern, the actual collection (client), the aggregator and the iterator. The aggregate is an interface/abstract class that has a method that returns an iterator. Iterator is an interface/abstract class that has methods allowing us to iterate through a collection.
In order to implement the pattern we first need to implement an iterator to produce a concrete that can iterate over the concerned collection (client)
Then the collection (client) implements the aggregator to return an instance of the above iterator.
Here is the UML diagram
So basically in c#, IEnumerable is the abstract aggregate and IEnumerator is the abstract Iterator. IEnumerable has a single method GetEnumerator that is responsible for creating an instance of IEnumerator of the desired type. Collections like Lists implement the IEnumerable.
Example.
Lets suppose that we have a method getPermutations(inputString) that returns all the permutations of a string and that the method returns an instance of IEnumerable<string>
In order to count the number of permutations we could do something like the below.
int count = 0;
var permutations = perm.getPermutations(inputString);
foreach (string permutation in permutations)
{
count++;
}
The c# compiler more or less converts the above to
using (var permutationIterator = perm.getPermutations(input).GetEnumerator())
{
while (permutationIterator.MoveNext())
{
count++;
}
}
If you have any questions please don't hesitate to ask.
A Minor contribution.
As many of them explain about 'when to use' and 'use with foreach'.
I thought of adding Another States Difference here as requested in question about the difference between both IEnumerable an IEnumerator.
I created the below code sample based on the below discussion threads.
IEnumerable , IEnumerator vs foreach, when to use what
What is the difference between IEnumerator and IEnumerable?
Enumerator preserves the state (iteration position) between function calls while iterations the other hand Enumerable does not.
Here is the tested example with comments to understand.
Experts please add/correct me.
static void EnumerableVsEnumeratorStateTest()
{
IList<int> numList = new List<int>();
numList.Add(1);
numList.Add(2);
numList.Add(3);
numList.Add(4);
numList.Add(5);
numList.Add(6);
Console.WriteLine("Using Enumerator - Remembers the state");
IterateFrom1to3(numList.GetEnumerator());
Console.WriteLine("Using Enumerable - Does not Remembers the state");
IterateFrom1to3Eb(numList);
Console.WriteLine("Using Enumerable - 2nd functions start from the item 1 in the collection");
}
static void IterateFrom1to3(IEnumerator<int> numColl)
{
while (numColl.MoveNext())
{
Console.WriteLine(numColl.Current.ToString());
if (numColl.Current > 3)
{
// This method called 3 times for 3 items (4,5,6) in the collection.
// It remembers the state and displays the continued values.
IterateFrom3to6(numColl);
}
}
}
static void IterateFrom3to6(IEnumerator<int> numColl)
{
while (numColl.MoveNext())
{
Console.WriteLine(numColl.Current.ToString());
}
}
static void IterateFrom1to3Eb(IEnumerable<int> numColl)
{
foreach (int num in numColl)
{
Console.WriteLine(num.ToString());
if (num>= 5)
{
// The below method invokes for the last 2 items.
//Since it doesnot persists the state it will displays entire collection 2 times.
IterateFrom3to6Eb(numColl);
}
}
}
static void IterateFrom3to6Eb(IEnumerable<int> numColl)
{
Console.WriteLine();
foreach (int num in numColl)
{
Console.WriteLine(num.ToString());
}
}
I have noticed these differences:
A. We iterate the list in different way, foreach can be used for IEnumerable and while loop for IEnumerator.
B. IEnumerator can remember the current index when we pass from one method to another (it start working with current index) but IEnumerable can't remember the index and it reset the index to beginning. More in this video https://www.youtube.com/watch?v=jd3yUjGc9M0
IEnumerable and IEnumerator both are interfaces in C#.
IEnumerable is an interface defining a single method GetEnumerator() that returns an IEnumerator interface.
This works for read-only access to a collection that implements that IEnumerable can be used with a foreach statement.
IEnumerator has two methods, MoveNext and Reset. It also has a property called Current.
The following shows the implementation of IEnumerable and IEnumerator.
using System;
using System.Collections;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
namespace Enudemo
{
class Person
{
string name = "";
int roll;
public Person(string name, int roll)
{
this.name = name;
this.roll = roll;
}
public override string ToString()
{
return string.Format("Name : " + name + "\t Roll : " + roll);
}
}
class Demo : IEnumerable
{
ArrayList list1 = new ArrayList();
public Demo()
{
list1.Add(new Person("Shahriar", 332));
list1.Add(new Person("Sujon", 333));
list1.Add(new Person("Sumona", 334));
list1.Add(new Person("Shakil", 335));
list1.Add(new Person("Shruti", 336));
}
IEnumerator IEnumerable.GetEnumerator()
{
return list1.GetEnumerator();
}
}
class Program
{
static void Main(string[] args)
{
Demo d = new Demo(); // Notice here. it is simple object but for
//IEnumerator you can get the collection data
foreach (Person X in d)
{
Console.WriteLine(X);
}
Console.ReadKey();
}
}
}
/*
Output :
Name : Shahriar Roll : 332
Name : Sujon Roll : 333
Name : Sumona Roll : 334
Name : Shakil Roll : 335
Name : Shruti Roll : 336
*/
This is in C#, I have a class that I am using from some else's DLL. It does not implement IEnumerable but has 2 methods that pass back a IEnumerator. Is there a way I can use a foreach loop on these. The class I am using is sealed.
foreach does not require IEnumerable, contrary to popular belief. All it requires is a method GetEnumerator that returns any object that has the method MoveNext and the get-property Current with the appropriate signatures.
/EDIT: In your case, however, you're out of luck. You can trivially wrap your object, however, to make it enumerable:
class EnumerableWrapper {
private readonly TheObjectType obj;
public EnumerableWrapper(TheObjectType obj) {
this.obj = obj;
}
public IEnumerator<YourType> GetEnumerator() {
return obj.TheMethodReturningTheIEnumerator();
}
}
// Called like this:
foreach (var xyz in new EnumerableWrapper(yourObj))
…;
/EDIT: The following method, proposed by several people, does not work if the method returns an IEnumerator:
foreach (var yz in yourObj.MethodA())
…;
Re: If foreach doesn't require an explicit interface contract, does it find GetEnumerator using reflection?
(I can't comment since I don't have a high enough reputation.)
If you're implying runtime reflection then no. It does it all compiletime, another lesser known fact is that it also check to see if the returned object that might Implement IEnumerator is disposable.
To see this in action consider this (runnable) snippet.
using System;
using System.Collections.Generic;
using System.Text;
namespace ConsoleApplication3
{
class FakeIterator
{
int _count;
public FakeIterator(int count)
{
_count = count;
}
public string Current { get { return "Hello World!"; } }
public bool MoveNext()
{
if(_count-- > 0)
return true;
return false;
}
}
class FakeCollection
{
public FakeIterator GetEnumerator() { return new FakeIterator(3); }
}
class Program
{
static void Main(string[] args)
{
foreach (string value in new FakeCollection())
Console.WriteLine(value);
}
}
}
According to MSDN:
foreach (type identifier in expression) statement
where expression is:
Object collection or array expression.
The type of the collection element
must be convertible to the identifier
type. Do not use an expression that
evaluates to null. Evaluates to a type
that implements IEnumerable or a type
that declares a GetEnumerator method.
In the latter case, GetEnumerator
should either return a type that
implements IEnumerator or declares all
the methods defined in IEnumerator.
Short answer:
You need a class with a method named GetEnumerator, which returns the IEnumerator you already have. Achieve this with a simple wrapper:
class ForeachWrapper
{
private IEnumerator _enumerator;
public ForeachWrapper(Func<IEnumerator> enumerator)
{
_enumerator = enumerator;
}
public IEnumerator GetEnumerator()
{
return _enumerator();
}
}
Usage:
foreach (var element in new ForeachWrapper(x => myClass.MyEnumerator()))
{
...
}
From the C# Language Specification:
The compile-time processing of a
foreach statement first determines the
collection type, enumerator type and
element type of the expression. This
determination proceeds as follows:
If the type X of expression is an array type then there is an implicit
reference conversion from X to the
System.Collections.IEnumerable
interface (since System.Array
implements this interface). The
collection type is the
System.Collections.IEnumerable
interface, the enumerator type is the
System.Collections.IEnumerator
interface and the element type is the
element type of the array type X.
Otherwise, determine whether the type X has an appropriate
GetEnumerator method:
Perform member lookup on the type X with identifier GetEnumerator and no
type arguments. If the member lookup
does not produce a match, or it
produces an ambiguity, or produces a
match that is not a method group,
check for an enumerable interface as
described below. It is recommended
that a warning be issued if member
lookup produces anything except a
method group or no match.
Perform overload resolution using the resulting method group and an
empty argument list. If overload
resolution results in no applicable
methods, results in an ambiguity, or
results in a single best method but
that method is either static or not
public, check for an enumerable
interface as described below. It is
recommended that a warning be issued
if overload resolution produces
anything except an unambiguous public
instance method or no applicable
methods.
If the return type E of the GetEnumerator method is not a class,
struct or interface type, an error is
produced and no further steps are
taken.
Member lookup is performed on E with the identifier Current and no
type arguments. If the member lookup
produces no match, the result is an
error, or the result is anything
except a public instance property that
permits reading, an error is produced
and no further steps are taken.
Member lookup is performed on E with the identifier MoveNext and no
type arguments. If the member lookup
produces no match, the result is an
error, or the result is anything
except a method group, an error is
produced and no further steps are
taken.
Overload resolution is performed on the method group with an empty
argument list. If overload resolution
results in no applicable methods,
results in an ambiguity, or results in
a single best method but that method
is either static or not public, or its
return type is not bool, an error is
produced and no further steps are
taken.
The collection type is X, the enumerator type is E, and the element
type is the type of the Current
property.
Otherwise, check for an enumerable interface:
If there is exactly one type T such that there is an implicit
conversion from X to the interface
System.Collections.Generic.IEnumerable<T>,
then the collection type is this
interface, the enumerator type is the
interface
System.Collections.Generic.IEnumerator<T>,
and the element type is T.
Otherwise, if there is more than one such type T, then an error is
produced and no further steps are
taken.
Otherwise, if there is an implicit conversion from X to the
System.Collections.IEnumerable
interface, then the collection type is
this interface, the enumerator type is
the interface
System.Collections.IEnumerator, and
the element type is object.
Otherwise, an error is produced and no further steps are taken.
Not strictly. As long as the class has the required GetEnumerator, MoveNext, Reset, and Current members, it will work with foreach
No, you don't and you don't even need an GetEnumerator method, e.g.:
class Counter
{
public IEnumerable<int> Count(int max)
{
int i = 0;
while (i <= max)
{
yield return i;
i++;
}
yield break;
}
}
which is called this way:
Counter cnt = new Counter();
foreach (var i in cnt.Count(6))
{
Console.WriteLine(i);
}
You could always wrap it, and as an aside to be "foreachable" you only need to have a method called "GetEnumerator" with the proper signature.
class EnumerableAdapter
{
ExternalSillyClass _target;
public EnumerableAdapter(ExternalSillyClass target)
{
_target = target;
}
public IEnumerable GetEnumerator(){ return _target.SomeMethodThatGivesAnEnumerator(); }
}
Given class X with methods A and B that both return IEnumerable, you could use a foreach on the class like this:
foreach (object y in X.A())
{
//...
}
// or
foreach (object y in X.B())
{
//...
}
Presumably the meaning for the enumerables returned by A and B are well-defined.
#Brian: Not sure you try to loop over the value return from method call or the class itself,
If what you want is the class then by make it an array you can use with foreach.
For a class to be usable with foeach all it needs to do is have a public method that returns and IEnumerator named GetEnumerator(), that's it:
Take the following class, it doesn't implement IEnumerable or IEnumerator :
public class Foo
{
private int[] _someInts = { 1, 2, 3, 4, 5, 6 };
public IEnumerator GetEnumerator()
{
foreach (var item in _someInts)
{
yield return item;
}
}
}
alternatively the GetEnumerator() method could be written:
public IEnumerator GetEnumerator()
{
return _someInts.GetEnumerator();
}
When used in a foreach ( Note that the no wrapper is used, just a class instance ):
foreach (int item in new Foo())
{
Console.Write("{0,2}",item);
}
prints:
1 2 3 4 5 6
The type only requires to have a public/non-static/non-generic/parameterless method named GetEnumerator which should return something that has a public MoveNext method and a public Current property. As I recollect Mr Eric Lippert somewhere, this was designed so as to accommodate pre generic era for both type safety and boxing related performance issues in case of value types.
For instance this works:
class Test
{
public SomethingEnumerator GetEnumerator()
{
}
}
class SomethingEnumerator
{
public Something Current //could return anything
{
get { }
}
public bool MoveNext()
{
}
}
//now you can call
foreach (Something thing in new Test()) //type safe
{
}
This is then translated by the compiler to:
var enumerator = new Test().GetEnumerator();
try {
Something element; //pre C# 5
while (enumerator.MoveNext()) {
Something element; //post C# 5
element = (Something)enumerator.Current; //the cast!
statement;
}
}
finally {
IDisposable disposable = enumerator as System.IDisposable;
if (disposable != null) disposable.Dispose();
}
From 8.8.4 section of the spec.
Something worth noting is the enumerator precedence involved - it goes like if you have a public GetEnumerator method, then that is the default choice of foreach irrespective of who is implementing it. For example:
class Test : IEnumerable<int>
{
public SomethingEnumerator GetEnumerator()
{
//this one is called
}
IEnumerator<int> IEnumerable<int>.GetEnumerator()
{
}
}
(If you don't have a public implementation (ie only explicit implementation), then precedence goes like IEnumerator<T> > IEnumerator.)