Consider the following example:
class YieldTest
{
static void Main(string[] args)
{
var res = Create(new string[] { "1 12 123", "1234", "12345" });
}
static IEnumerable<int> Create(IEnumerable<string> strings)
{
foreach(string s in strings)
{
yield return s.Length;
if(s.Contains(' '))
{
string[] tokens = s.Split(' ');
foreach(string t in tokens)
{
yield return t.Length;
}
}
}
}
}
The call to Create returns {8, 1, 2, 3, 4, 5}.
What really confuses me is that the code after the yield return statement is executed.
(Why would they name it yield return and not just yield ??)
The documentation tells us
When a yield return statement is reached in the iterator method,
expression is returned, and the current location in code is retained.
What does that mean? Where does a return occur? What is an iterator method?
It means your code is transformed into a state machine by the compiler.
When you call Create method, you'll get an IEnumerable<T>. You can then call GetEnumerator() on it and obtain an IEnumerator<T>.
Now, every time you call MoveNext on this iterator, your code will execute until it finds the first yield statement (whether it be yield return or yield break).
If it hits a yield return x, MoveNext returns true and the iterator's Current property will be set to x, and the rest of the code will be executed the next time you call MoveNext.
This happens until either there's no more code to run or the code hits a yield break. At that point, MoveNext will return false.
yield return 1;
yield return 2;
yield return 3;
Calling MoveNext on this piece of code will return true the first three times, and at each iteration, Current will be set to 1, 2 and 3.
Edit 2:
Regarding the yield return ?? syntax, from Eric Lippert's Ambiguous Optional Parentheses, Part Three
When they designed C# 2.0 they had this problem:
yield(x);
Does that mean "yield x in an iterator" or "call the yield method with argument x?" By changing it to
yield return(x);
it is now unambiguous; it cannot possibly mean "call the yield method".
The most important part of your quote is:
and the current location in code is retained.
The iterator method does not return in a sense of exiting the method, otherwise you could only enumerate one item. yield return returns one element that is enumerated and then continues to run the code of the iterator method at the spot of the yield return in order to return the next elements when it encounters the next yield return.
An iterator method iterates over a collection and provides the single items by using yield return. These statements allow for a simplified implementation of the IEnumerable interface. In earlier versions of .NET, you had to implement a specific class as the enumerator. As this meant to introduce a lot of boilerplate code for each implementation of IEnumerable, the yield statements were introduced to create an iterator method in a simpler way. The yield return statements in the iterator method provide a signal to the compiler that a new element is to be returned. When building the project, the compiler transfers the iterator method to an implementation of IEnumerable.
An iterator method like this will run until it finds a yield return whenever you fetch a new value from the iterator that it produces. I believe technically it is when you call the MoveNext method of the Enumerator.
When you call MoveNext the next time execution will resume from where it left off in the iterator method and keep going until it gets to the next yield return.
Essentially every call to yield return in your iterator method will yield one of the values in your final enumerable.
Related
I had a problem in my code where a Lazy initializer was called more frequently than what I expected. From the documentation, I expected that using LazyThreadSafetyMode.ExecutionAndPublication would ensure my initializer function was only ever called once, for example if accessing numbers.Value after defining:
numbers = new Lazy<IEnumerable<int>>(
() => GetNumbers(),
LazyThreadSafetyMode.ExecutionAndPublication
);
However, what I have found is that if the initialization function yields its results, the initialization function gets called more than once. I presume this has to with yields delayed execution but I only have a fuzzy sense why.
Question:
In the code below, why do the respective initialization functions get executed a different number of times?
void Main()
{
var foo = new foo();
var tasks = new List<Task>();
for (int i = 0; i < 10; ++i) tasks.Add(Task.Run(() => {foreach (var number in foo.Numbers) Debug.WriteLine(number);}));
Task.WaitAll(tasks.ToArray());
tasks.Clear();
for (int i = 0; i < 10; ++i) tasks.Add(Task.Run(() => {foreach (var letter in foo.Letters) Debug.WriteLine(letter);}));
Task.WaitAll(tasks.ToArray());
}
public class foo
{
public IEnumerable<int> Numbers => numbers.Value;
public IEnumerable<char> Letters => letters.Value;
readonly Lazy<IEnumerable<int>> numbers;
readonly Lazy<IEnumerable<char>> letters;
public foo()
{
numbers = new Lazy<IEnumerable<int>>(
() => GetNumbers(),
LazyThreadSafetyMode.ExecutionAndPublication
);
letters = new Lazy<IEnumerable<char>>(
() => GetLetters().ToList(), //ToList enumerates all yielded letters, creating the expected call once behavior
LazyThreadSafetyMode.ExecutionAndPublication
);
}
protected IEnumerable<char> GetLetters()
{
Debug.WriteLine($"{nameof(GetLetters)} Called");
yield return 'a';
yield return 'b';
yield return 'c';
yield break;
}
protected IEnumerable<int> GetNumbers()
{
Debug.WriteLine($"{nameof(GetNumbers)} Called");
yield return 1;
yield return 2;
yield return 3;
yield break;
}
}
I had a problem in my code where a Lazy initializer was called more frequently than what I expected.
No, the initializer of the lazy is called once. The initializer of the lazy is
() => GetNumbers()
and that is called exactly once.
GetNumbers returns an IEnumerable<int> -- a sequence of integers.
When you foreach that sequence, it calls GetEnumerator to get an enumerator, and then calls MoveNext on the enumerator object until MoveNext returns false.
You've said that you want the sequence to be enumerated as:
the first time MoveNext is called, do a writeline and produce 1
the second time MoveNext is called, produce 2
the third time MoveNext is called, produce 3
the fourth time MoveNext is called, produce 4
Every subsequent time MoveNext is called, return false
So every time you enumerate the sequence, that's what happens.
Can you explain what you expected to happen? I am interested to learn why people have false beliefs about computer programs.
Also it is not clear to me why you are using Lazy at all. You would typically use Lazy to avoid expensive work until it is needed, but sequences already defer work until they are enumerated.
I am trying to understand use of Yield to enumerate the collection. I have written this basic code:
static void Main(string[] args)
{
Iterate iterate = new Iterate();
foreach (int i in iterate.EnumerateList())
{
Console.Write("{0}", i);
}
Console.ReadLine();
}
class Iterate
{
public IEnumerable<int> EnumerateList()
{
List<int> lstNumbers = new List<int>();
lstNumbers.Add(1);
lstNumbers.Add(2);
lstNumbers.Add(3);
lstNumbers.Add(4);
lstNumbers.Add(5);
foreach (int i in lstNumbers)
{
yield return i;
}
}
}
(1) What if I use simply return i instead of yield return i?
(2) What are the advantages of using Yield and when to prefer using it?
Edited **
In the above code, I think it is an overhead to use foreach two times. First in the main function and the second in the EnumerateList method.
Using yield return i makes this method an iterator. It will create an IEnumerable<int> sequence of values from your entire loop.
If you used return i, it would just return a single int value. In your case, this would cause a compiler error, as the return type of your method is IEnumerable<int>, not int.
In this specific example, I would personally just return lstNumbers instead of using the iterator. You could rewrite this without the list, though, as:
public IEnumerable<int> EnumerateList()
{
yield return 1;
yield return 2;
yield return 3;
yield return 4;
yield return 5;
}
Or even:
public IEnumerable<int> EnumerateList()
{
for (int i=1;i<=5;++i)
yield return i;
}
This is very handy when you're making a class which you want to act like a collection. Implementing IEnumerable<T> by hand for a custom type often requires making a custom class, etc. Prior to iterators, this required a lot of code, as shown in this old sample for implementing a custom collection in C#.
As Reed says using yield allows you to implement an iterator. The major advantage of an iterator is that it allows lazy evaluation. I.e. it doesn't have to materialize the entire result unless needed.
Consider Directory.GetFiles from the BCL. It returns string[]. I.e. it has to get all the files and put the names in an array before it returns. In contrast Directory.EnumerateFiles returns IEnumerable<string>. I.e. the caller is responsible for handling the result set. That means that the caller can opt out of enumerating the collection at any point.
The yield keyword will make the compiler turn the function into an enumerator object.
The yield return statement doesn't simply exit the function and return one value, it leaves the code in a state so that it can be resumed at that point when the next value is requested from the enumerator.
You can't return just i as i is an int and not an IEnumerable. It won't even compile. You could have returned lstNumbers as that implements the interface of IEnumerable.
I prefer yield return since the compiler will handle building the enumerable and not having to build the list in the first place. So for me if I have something that is already implementing the interface then I return that if I have to build it and not use it else where then I yield return.
it seems this code is valid. I am wondering what if getTest(1)
IEnumerable<int> getTest(int n)
{
if (n == 0)
yield return 1;
else
;
}
It will return an empty but non-null IEnumberable<int>.
Once you've marked a method as an iterator (by using yield anywhere in the method), it can execute zero or more yield returns to return data in the sequence.
In fact, the simplest way to get an empty IEnumberable is IEnumerable<T> Empty() { yield break; }.
Or just call Enumerable.Empty<T>().
It just returns an empty, although fully "valid", enumerable. Everything went exactly as expected.
i am reading C# 2010 Accelerated. i dont get what is yield
When GetEnumerator is called, the code
in the method that contains the yield
statement is not actually executed at
that point in time. Instead, the
compiler generates an enumerator
class, and that class contains the
yield block code
public IEnumerator<T> GetEnumerator() {
foreach( T item in items ) {
yield return item;
}
}
i also read from Some help understanding “yield”
yield is a lazy producer of data, only
producing another item after the first
has been retrieved, whereas returning
a list will return everything in one
go.
does this mean that each call to GetEnumerator will get 1 item from the collection? so 1st call i get 1st item, 2nd, i get the 2nd and so on ... ?
Best way to think of it is when you first request an item from an IEnumerator (for example in a foreach), it starts running trough the method, and when it hits a yield return it pauses execution and returns that item for you to use in your foreach. Then you request the next item, it resumes the code where it left and repeats the cycle until it encounters either yield break or the end of the method.
public IEnumerator<string> enumerateSomeStrings()
{
yield return "one";
yield return "two";
var array = new[] { "three", "four" }
foreach (var item in array)
yield return item;
yield return "five";
}
Take a look at the IEnumerator<T> interface; that may well to clarify what's happening. The compiler takes your code and turns it into a class that implements both IEnumerable<T> and IEnumerator<T>. The call to GetEnumerator() simply returns the class itself.
The implementation is basically a state machine, which, for each call to MoveNext(), executes the code up until the next yield return and then sets Current to the return value. The foreach loop uses this enumerator to walk through the enumerated items, calling MoveNext() before each iteration of the loop. The compiler is really doing some very cool things here, making yield return one of the most powerful constructs in the language. From the programmer's perspective, it's just an easy way to lazily return items upon request.
Yes thats right, heres the example from MSDN that illustrates how to use it
public class List
{
//using System.Collections;
public static IEnumerable Power(int number, int exponent)
{
int counter = 0;
int result = 1;
while (counter++ < exponent)
{
result = result * number;
yield return result;
}
}
static void Main()
{
// Display powers of 2 up to the exponent 8:
foreach (int i in Power(2, 8))
{
Console.Write("{0} ", i);
}
}
}
/*
Output:
2 4 8 16 32 64 128 256
*/
If I understand your question correct then your understanding is incorrect I'm affraid. The yield statements (yield return and yield break) is a very clever compiler trick. The code in you method is actually compiled into a class that implements IEnumerable. An instance of this class is what the method will return. Let's Call the instance 'ins' when calling ins.GetEnumerator() you get an IEnumerator that for each Call to MoveNext() produced the next element in the collection (the yield return is responsible for this part) when the sequence has no more elements (e.g. a yield break is encountered) MoveNext() returns false and further calls results in an exception. So it is not the Call to GetEnumerator that produced the (next) element but the Call to MoveNext
It looks like you understand it.
yield is used in your class's GetEnumerator as you describe so that you can write code like this:
foreach (MyObject myObject in myObjectCollection)
{
// Do something with myObject
}
By returning the first item from the 1st call the second from the 2nd and so on you can loop over all elements in the collection.
yield is defined in MyObjectCollection.
The Simple way to understand yield keyword is we do not need extra class to hold the result of iteration when return using
yield return keyword. Generally when we iterate through the collection and want to return the result, we use collection object
to hold the result. Let's look at example.
public static List Multiplication(int number, int times)
{
List<int> resultList = new List<int>();
int result = number;
for(int i=1;i<=times;i++)
{
result=number*i;
resultList.Add(result);
}
return resultList;
}
static void Main(string[] args)
{
foreach(int i in Multiplication(2,10))
{
Console.WriteLine(i);
}
Console.ReadKey();
}
In the above example, I want to return the result of multiplication of 2 ten times. So I Create a method Multiplication
which returns me the multiplication of 2 ten times and i store the result in the list and when my main method calls the
multiplication method, the control iterates through the loop ten times and store result result in the list. This is without
using yield return. Suppose if i want to do this using yield return it looks like
public static IEnumerable Multiplication(int number, int times)
{
int result = number;
for(int i=1;i<=times;i++)
{
result=number*i;
yield return result;
}
}
static void Main(string[] args)
{
foreach(int i in Multiplication(2,10))
{
Console.WriteLine(i);
}
Console.ReadKey();
}
Now there is slight changes in Multiplication method, return type is IEnumerable and there is no other list to hold the
result because to work with Yield return type must be IEnumerable or IEnumerator and since Yield provides stateful iteration
we do not need extra class to hold the result. So in the above example, when Multiplication method is called from Main
method, it calculates the result in for 1st iteration and return the result to main method and come backs to the loop and
calculate the result for 2nd iteration and returns the result to main method.In this way Yield returns result to calling
method one by one in each iteration.There is other Keyword break used in combination with Yield that causes the iteration
to stop. For example in the above example if i want to calculate multiplication for only half number of times(10/2=5) then
the method looks like this:
public static IEnumerable Multiplication(int number, int times)
{
int result = number;
for(int i=1;i<=times;i++)
{
result=number*i;
yield return result;
if (i == times / 2)
yield break;
}
}
This method now will result multiplication of 2, 5 times.Hope this will help you understand the concept of Yield. For more
information please visit http://msdn.microsoft.com/en-us/library/9k7k7cf0.aspx
I have seen the yield keyword being used quite a lot on Stack Overflow and blogs. I don't use LINQ. Can someone explain the yield keyword?
I know that similar questions exist.
But none really explain what is its use in plain simple language.
By far the best explanation of this (that I've seen) is Jon Skeet's book - and that chapter is free! Chapter 6, C# in Depth. There is nothing I can add here that isn't covered.
Then buy the book; you will be a better C# programmer for it.
Q: Why didn't I write a longer answer here (paraphrased from comments); simple. As Eric Lippert observes (here), the yield construct (and the magic that goes behind it) is the single most complex bit of code in the C# compiler, and to try and describe it in a brief reply here is naïve at best. There are so many nuances to yield that IMO it is better to refer to a pre-existing (and fully qualified) resource.
Eric's blog now has 7 entries (and that is just the recent ones) discussing yield. I have a vast amount of respect for Eric, but his blog is probably more appropriate as a "further information" for people who are comfortable with the subject (yield in this case), as it typically describes a lot of the background design considerations. Best done in the context of a reasonable foundation.
(and yes, chapter 6 does download; I verified...)
The yield keyword is used with methods that return IEnumerable<T> or IEnumerator<T> and it makes the compiler generate a class that implements the necessary plumbing for using the iterator. E.g.
public IEnumerator<int> SequenceOfOneToThree() {
yield return 1;
yield return 2;
yield return 3;
}
Given the above the compiler will generate a class that implements IEnumerator<int>, IEnumerable<int> and IDisposable (actually it will also implement the non-generic versions of IEnumerable and IEnumerator).
This allows you to call the method SequenceOfOneToThree in a foreach loop like this
foreach(var number in SequenceOfOneToThree) {
Console.WriteLine(number);
}
An iterator is a state machine, so each time yield is called the position in the method is recorded. If the iterator is moved to the next element, the method resumes right after this position. So the first iteration returns 1 and marks that position. The next iterator resumes right after one and thus returns 2 and so forth.
Needless to say you can generate the sequence in any way you like, so you don't have to hard code the numbers like I did. Also, if you want to break the loop you can use yield break.
In an effort to demystify I'll avoid talking about iterators, since they could be part of the mystery themselves.
the yield return and yield break statements are most often used to provide "deferred evaluation" of the collection.
What this means is that when you get the value of a method that uses yield return, the collection of things you are trying to get don't exist together yet (it's essentially empty). As you loop through them (using foreach) it will execute the method at that time and get the next element in the enumeration.
Certain properties and methods will cause the entire enumeration to be evaluated at once (such as "Count").
Here's a quick example of the difference between returning a collection and returning yield:
string[] names = { "Joe", "Jim", "Sam", "Ed", "Sally" };
public IEnumerable<string> GetYieldEnumerable()
{
foreach (var name in names)
yield return name;
}
public IEnumerable<string> GetList()
{
var list = new List<string>();
foreach (var name in names)
list.Add(name);
return list;
}
// we're going to execute the GetYieldEnumerable() method
// but the foreach statement inside it isn't going to execute
var yieldNames = GetNamesEnumerable();
// now we're going to execute the GetList() method and
// the foreach method will execute
var listNames = GetList();
// now we want to look for a specific name in yieldNames.
// only the first two iterations of the foreach loop in the
// GetYieldEnumeration() method will need to be called to find it.
if (yieldNames.Contains("Jim")
Console.WriteLine("Found Jim and only had to loop twice!");
// now we'll look for a specific name in listNames.
// the entire names collection was already iterated over
// so we've already paid the initial cost of looping through that collection.
// now we're going to have to add two more loops to find it in the listNames
// collection.
if (listNames.Contains("Jim"))
Console.WriteLine("Found Jim and had to loop 7 times! (5 for names and 2 for listNames)");
This can also be used if you need to get a reference to the Enumeration before the source data has values. For example if the names collection wasn't complete to start with:
string[] names = { "Joe", "Jim", "Sam", "Ed", "Sally" };
public IEnumerable<string> GetYieldEnumerable()
{
foreach (var name in names)
yield return name;
}
public IEnumerable<string> GetList()
{
var list = new List<string>();
foreach (var name in names)
list.Add(name);
return list;
}
var yieldNames = GetNamesEnumerable();
var listNames = GetList();
// now we'll change the source data by renaming "Jim" to "Jimbo"
names[1] = "Jimbo";
if (yieldNames.Contains("Jimbo")
Console.WriteLine("Found Jimbo!");
// Because this enumeration was evaluated completely before we changed "Jim"
// to "Jimbo" it isn't going to be found
if (listNames.Contains("Jimbo"))
// this can't be true
else
Console.WriteLine("Couldn't find Jimbo, because he wasn't there when I was evaluated.");
The yield keyword is a convenient way to write an IEnumerator. For example:
public static IEnumerator<int> Range(int from, int to)
{
for (int i = from; i < to; i++)
{
yield return i;
}
}
is transformed by the C# compiler to something similiar to:
public static IEnumerator<int> Range(int from, int to)
{
return new RangeEnumerator(from, to);
}
class RangeEnumerator : IEnumerator<int>
{
private int from, to, current;
public RangeEnumerator(int from, int to)
{
this.from = from;
this.to = to;
this.current = from;
}
public bool MoveNext()
{
this.current++;
return this.current < this.to;
}
public int Current
{
get
{
return this.current;
}
}
}
Take a look at the MSDN documentation and the example. It is essentially an easy way to create an iterator in C#.
public class List
{
//using System.Collections;
public static IEnumerable Power(int number, int exponent)
{
int counter = 0;
int result = 1;
while (counter++ < exponent)
{
result = result * number;
yield return result;
}
}
static void Main()
{
// Display powers of 2 up to the exponent 8:
foreach (int i in Power(2, 8))
{
Console.Write("{0} ", i);
}
}
}
Eric White's series on functional programming it well worth the read in it's entirety, but the entry on Yield is as clear an explanation as I've seen.
yield is not directly related to LINQ, but rather to iterator blocks. The linked MSDN article gives great detail on this language feature. See especially the Using Iterators section. For deep details of iterator blocks, see Eric Lippert's recent blog posts on the feature. For the general concept, see the Wikipedia article on iterators.
I came up with this to overcome a .NET shortcoming having to manually deep copy List.
I use this:
static public IEnumerable<SpotPlacement> CloneList(List<SpotPlacement> spotPlacements)
{
foreach (SpotPlacement sp in spotPlacements)
{
yield return (SpotPlacement)sp.Clone();
}
}
And at another place:
public object Clone()
{
OrderItem newOrderItem = new OrderItem();
...
newOrderItem._exactPlacements.AddRange(SpotPlacement.CloneList(_exactPlacements));
...
return newOrderItem;
}
I tried to come up with oneliner that does this, but it's not possible, due to yield not working inside anonymous method blocks.
EDIT:
Better still, use a generic List cloner:
class Utility<T> where T : ICloneable
{
static public IEnumerable<T> CloneList(List<T> tl)
{
foreach (T t in tl)
{
yield return (T)t.Clone();
}
}
}
Let me add to all of this. Yield is not a keyword.
It will only work if you use "yield return" other than that it will work like a normal variable.
It's uses to return iterator from a function. You can search further on that.
I recommend searching for "Returning Array vs Iterator"