We Keep Coding

Testing an infinite sequence with NUnit, but only a finite sequence must be checked

I have an Extension Method that extract the numbers in a list of strings (bbb1, da21ds, dsa231djsla90 ==> 1, 21, 23190):
public static int[] GetContainedNumbers(this string[] source)
{
if (source == null) throw new ArgumentNullException();
int[] result = new int[] {};
foreach (var s in source)
{
string onlyNumbers = new String(s.Where(Char.IsDigit).ToArray());
if (String.IsNullOrEmpty(onlyNumbers)) throw new ArgumentException();
int extractedNumber = Int32.Parse(onlyNumbers);
result = result.Concat(new int[] { extractedNumber }).ToArray();
}
return result;
}
I need to make a test with NUnit. The request is to make a test with an infinite sequence of strings (a1z, a2z, a3z, ...), but the output only needs to check the first 100 numbers.
Currently I have no idea what I'm supposed to do. My starting idea was to create a test like this:
int[] expectedResult = Enumerable.Range(0, 99).ToArray();
string[] source = new string[] {};
int n = 0;
while (true)
{
if (n == 99) break;
source = source.Concat(new string[] { "a" + n + "z" }).ToArray();
n++;
}
Assert.AreEqual(expectedResult, source.GetContainedNumbers());
But it doesn't really make sense since the array is finite and not infinite.
I don't know what it means to create an infinite list of things and how I should create it nor testing it.
I can edit the Extension Method if needed. Thanks in advance for any help and sorry if my english is quite broken.
The first task is to make the Extension method itself, which I have resolved above. It can be modified, but it must resolve this piece of code (which I can't edit):
foreach (var d in new[] { "1qui7", "q8u8o", "-1024", "0q0ua0" }.GetContainedNumbers())
{
Console.WriteLine("{0}, ", d);
var strings = new [ ] { "1qui7" , " q8u8o " , " −1024" , " 0 q0ua0 " };
}
Expected output: 17, 88, 1024, 0,

I believe the purpose of this exercise is to investigate lazy evaluation.
Currently your method accepts a string[] and returns an int[] - but there's nothing in the question that says it has to do that.
If instead you were to write an extension method accepting an IEnumerable<string> and returning an IEnumerable<int>, then you could accept an infinite sequence of elements and return a corresponding infinite sequence of outputs, that is lazily evaluated: when the caller requests the next output element, you in turn request the next input element, process it, and yield an output. C# makes all of this quite straightforward with iterator blocks.
So I would expect a method like this:
public static IEnumerable<int> GetContainedNumbers(IEnumerable<string> source)
{
foreach (string inputElement)
{
int outputElement = ...; // TODO: your logic here
yield return output;
}
}
Now to test this, you can either use iterator blocks again to generate genuinely infinite sequences, or you could test against an effectively infinite sequence using LINQ, e.g.
IEnumerable<string> veryLargeInput = Enumerable
.Range(0, int.MaxValue)
.Select(x => $"a{x}z");
The truly infinite version might be something like:
IEnumerable<string> infiniteSequence = GetInfiniteSequence();
...
private static IEnumerable<string> GetInfiniteSequence()
{
int value = 0;
while (true)
{
yield return $"x{value}z";
// Eventually this will loop round from int.MaxValue to
// int.MinValue. You could use BigInteger if you really wanted.
value++;
}
}
When testing the result, you'll want to take the first 100 elements of the output. For example:
var input = GetInfiniteSequence(); // Or the huge Enumerable.Range
var output = input.GetContainedNumbers(); // This is still infinite
var first100Elements = output.Take(100).ToList();
// Now make assertions against first100Elements

Why can't we debug a method with yield return for the following code? [duplicate]

This question already has answers here:
Cannot step into a method returning IEnumerable<T>?
(4 answers)
Closed 7 years ago.
Following is my code:
class Program {
static List<int> MyList;
static void Main(string[] args) {
MyList = new List<int>() { 1,24,56,7};
var sn = FilterWithYield();
}
static IEnumerable<int> FilterWithYield() {
foreach (int i in MyList) {
if (i > 3)
yield return i;
}
}
}
I have a break point in FilterWithYield Method but its not at all hitting the break point. I have one break at the calling point i.e var sn = FilterWithYield(); Control hits this point and shows the result correctly in debugging window. But why isn't the control stopping in the FilterWithYield method?
One more question. I read that yield returns data to the caller..if that is so if changed return type of FilterWithYield method to int it through error.Does the yield key word always need IEnumerable<T> as return type?

You can debug the method. The problem is, the code that you are trying to reach is never executed.
IEnumerable methods with yield return produce code that makes your sequence lazily, as you go through enumeration. However, when you do this
var sn = FilterWithYield();
you prepare to enumerate the sequence, but you do not start enumerating it.
If, on the other hand, you add a foreach loop or call ToList() on the result, your breakpoint would get hit:
foreach (var n in FilterWithYield()) {
Console.WriteLine(n);
}
or
var sn = FilterWithYield().ToList();

How yield return works in c#

I have following piece of code.
public static void main(string []args)
{
var intergers = GetCollection();
foreach(var val in intergers)
{
console.writeline(val);
}
}
public IEnumerable<int> GetCollection()
{
yield return 10;
var obj = new MyClass();
obj.performLargeAction();
yield return 1;
var obj = new MyClass();
obj.perform();
yield return 2;
console.writeline("I am finished now");
}
Now, when I debug the code and see the foreach iteration, then I see that first time method executes till yield return 10 and then control goes back to foreach loop, on next iteration code after yield return 10 and till yield return 1 is executed and in next iteration it executes from line right after yield return 1 and till yield return and in next iteration it
calls console.writeline();
So, with this behavior, I am keen to know does .net pause the execution of the method which is returning IEnumerable and save its location counter or state to the stack and next iteration again pops the location and set it to program counter or location counter and all these thing or
I have read something yield is a c# feature, so this above piece of code is converted to IL and that yield part is replaced with a class called YieldEnumerator which stores state, current and other thing. but still I could not understand its actual functioning. So, I would like to know its internal working form starting to end.

Need help understanding C# yield in IEnumerable

i am reading C# 2010 Accelerated. i dont get what is yield
When GetEnumerator is called, the code
in the method that contains the yield
statement is not actually executed at
that point in time. Instead, the
compiler generates an enumerator
class, and that class contains the
yield block code
public IEnumerator<T> GetEnumerator() {
foreach( T item in items ) {
yield return item;
}
}
i also read from Some help understanding “yield”
yield is a lazy producer of data, only
producing another item after the first
has been retrieved, whereas returning
a list will return everything in one
go.
does this mean that each call to GetEnumerator will get 1 item from the collection? so 1st call i get 1st item, 2nd, i get the 2nd and so on ... ?

Best way to think of it is when you first request an item from an IEnumerator (for example in a foreach), it starts running trough the method, and when it hits a yield return it pauses execution and returns that item for you to use in your foreach. Then you request the next item, it resumes the code where it left and repeats the cycle until it encounters either yield break or the end of the method.
public IEnumerator<string> enumerateSomeStrings()
{
yield return "one";
yield return "two";
var array = new[] { "three", "four" }
foreach (var item in array)
yield return item;
yield return "five";
}

Take a look at the IEnumerator<T> interface; that may well to clarify what's happening. The compiler takes your code and turns it into a class that implements both IEnumerable<T> and IEnumerator<T>. The call to GetEnumerator() simply returns the class itself.
The implementation is basically a state machine, which, for each call to MoveNext(), executes the code up until the next yield return and then sets Current to the return value. The foreach loop uses this enumerator to walk through the enumerated items, calling MoveNext() before each iteration of the loop. The compiler is really doing some very cool things here, making yield return one of the most powerful constructs in the language. From the programmer's perspective, it's just an easy way to lazily return items upon request.

Yes thats right, heres the example from MSDN that illustrates how to use it
public class List
{
//using System.Collections;
public static IEnumerable Power(int number, int exponent)
{
int counter = 0;
int result = 1;
while (counter++ < exponent)
{
result = result * number;
yield return result;
}
}
static void Main()
{
// Display powers of 2 up to the exponent 8:
foreach (int i in Power(2, 8))
{
Console.Write("{0} ", i);
}
}
}
/*
Output:
2 4 8 16 32 64 128 256
*/

If I understand your question correct then your understanding is incorrect I'm affraid. The yield statements (yield return and yield break) is a very clever compiler trick. The code in you method is actually compiled into a class that implements IEnumerable. An instance of this class is what the method will return. Let's Call the instance 'ins' when calling ins.GetEnumerator() you get an IEnumerator that for each Call to MoveNext() produced the next element in the collection (the yield return is responsible for this part) when the sequence has no more elements (e.g. a yield break is encountered) MoveNext() returns false and further calls results in an exception. So it is not the Call to GetEnumerator that produced the (next) element but the Call to MoveNext

It looks like you understand it.
yield is used in your class's GetEnumerator as you describe so that you can write code like this:
foreach (MyObject myObject in myObjectCollection)
{
// Do something with myObject
}
By returning the first item from the 1st call the second from the 2nd and so on you can loop over all elements in the collection.
yield is defined in MyObjectCollection.

The Simple way to understand yield keyword is we do not need extra class to hold the result of iteration when return using
yield return keyword. Generally when we iterate through the collection and want to return the result, we use collection object
to hold the result. Let's look at example.
public static List Multiplication(int number, int times)
{
List<int> resultList = new List<int>();
int result = number;
for(int i=1;i<=times;i++)
{
result=number*i;
resultList.Add(result);
}
return resultList;
}
static void Main(string[] args)
{
foreach(int i in Multiplication(2,10))
{
Console.WriteLine(i);
}
Console.ReadKey();
}
In the above example, I want to return the result of multiplication of 2 ten times. So I Create a method Multiplication
which returns me the multiplication of 2 ten times and i store the result in the list and when my main method calls the
multiplication method, the control iterates through the loop ten times and store result result in the list. This is without
using yield return. Suppose if i want to do this using yield return it looks like
public static IEnumerable Multiplication(int number, int times)
{
int result = number;
for(int i=1;i<=times;i++)
{
result=number*i;
yield return result;
}
}
static void Main(string[] args)
{
foreach(int i in Multiplication(2,10))
{
Console.WriteLine(i);
}
Console.ReadKey();
}
Now there is slight changes in Multiplication method, return type is IEnumerable and there is no other list to hold the
result because to work with Yield return type must be IEnumerable or IEnumerator and since Yield provides stateful iteration
we do not need extra class to hold the result. So in the above example, when Multiplication method is called from Main
method, it calculates the result in for 1st iteration and return the result to main method and come backs to the loop and
calculate the result for 2nd iteration and returns the result to main method.In this way Yield returns result to calling
method one by one in each iteration.There is other Keyword break used in combination with Yield that causes the iteration
to stop. For example in the above example if i want to calculate multiplication for only half number of times(10/2=5) then
the method looks like this:
public static IEnumerable Multiplication(int number, int times)
{
int result = number;
for(int i=1;i<=times;i++)
{
result=number*i;
yield return result;
if (i == times / 2)
yield break;
}
}
This method now will result multiplication of 2, 5 times.Hope this will help you understand the concept of Yield. For more
information please visit http://msdn.microsoft.com/en-us/library/9k7k7cf0.aspx

Some help understanding "yield"

In my everlasting quest to suck less I'm trying to understand the "yield" statement, but I keep encountering the same error.
The body of [someMethod] cannot be an iterator block because
'System.Collections.Generic.List< AClass>' is not an iterator interface type.
This is the code where I got stuck:
foreach (XElement header in headersXml.Root.Elements()){
yield return (ParseHeader(header));
}
What am I doing wrong? Can't I use yield in an iterator? Then what's the point?
In this example it said that List<ProductMixHeader> is not an iterator interface type.
ProductMixHeader is a custom class, but I imagine List is an iterator interface type, no?
--Edit--
Thanks for all the quick answers.
I know this question isn't all that new and the same resources keep popping up.
It turned out I was thinking I could return List<AClass> as a return type, but since List<T> isn't lazy, it cannot. Changing my return type to IEnumerable<T> solved the problem :D
A somewhat related question (not worth opening a new thread): is it worth giving IEnumerable<T> as a return type if I'm sure that 99% of the cases I'm going to go .ToList() anyway? What will the performance implications be?

A method using yield return must be declared as returning one of the following two interfaces:
IEnumerable<SomethingAppropriate>
IEnumerator<SomethingApropriate>
(thanks Jon and Marc for pointing out IEnumerator)
Example:
public IEnumerable<AClass> YourMethod()
{
foreach (XElement header in headersXml.Root.Elements())
{
yield return (ParseHeader(header));
}
}
yield is a lazy producer of data, only producing another item after the first has been retrieved, whereas returning a list will return everything in one go.
So there is a difference, and you need to declare the method correctly.
For more information, read Jon's answer here, which contains some very useful links.

It's a tricky topic. In a nutshell, it's an easy way of implementing IEnumerable and its friends. The compiler builds you a state machine, transforming parameters and local variables into instance variables in a new class. Complicated stuff.
I have a few resources on this:
Chapter 6 of C# in Depth (free download from that page)
Iterators, iterator blocks and data pipelines (article)
Iterator block implementation details (article)

"yield" creates an iterator block - a compiler generated class that can implement either IEnumerable[<T>] or IEnumerator[<T>]. Jon Skeet has a very good (and free) discussion of this in chapter 6 of C# in Depth.
But basically - to use "yield" your method must return an IEnumerable[<T>] or IEnumerator[<T>]. In this case:
public IEnumerable<AClass> SomeMethod() {
// ...
foreach (XElement header in headersXml.Root.Elements()){
yield return (ParseHeader(header));
}
}

List implements Ienumerable.
Here's an example that might shed some light on what you are trying to learn. I wrote this about 6 months
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace YieldReturnTest
{
public class PrimeFinder
{
private Boolean isPrime(int integer)
{
if (0 == integer)
return false;
if (3 > integer)
return true;
for (int i = 2; i < integer; i++)
{
if (0 == integer % i)
return false;
}
return true;
}
public IEnumerable<int> FindPrimes()
{
int i;
for (i = 1; i < 2147483647; i++)
{
if (isPrime(i))
{
yield return i;
}
}
}
}
class Program
{
static void Main(string[] args)
{
PrimeFinder primes = new PrimeFinder();
foreach (int i in primes.FindPrimes())
{
Console.WriteLine(i);
Console.ReadLine();
}
Console.ReadLine();
Console.ReadLine();
}
}
}

I highly recommend using Reflector to have a look at what yield actually does for you. You'll be able to see the full code of the class that the compiler generates for you when using yield, and I've found that people understand the concept much more quickly when they can see the low-level result (well, mid-level I guess).

To understand yield, you need to understand when to use IEnumerator and IEnumerable (because you have to use either of them). The following examples help you to understand the difference.
First, take a look at the following class, it implements two methods - one returning IEnumerator<int>, one returning IEnumerable<int>. I'll show you that there is a big difference in usage, although the code of the 2 methods is looking similar:
// 2 iterators, one as IEnumerator, one as IEnumerable
public class Iterator
{
public static IEnumerator<int> IterateOne(Func<int, bool> condition)
{
for(var i=1; condition(i); i++) { yield return i; }
}
public static IEnumerable<int> IterateAll(Func<int, bool> condition)
{
for(var i=1; condition(i); i++) { yield return i; }
}
}
Now, if you're using IterateOne you can do the following:
// 1. Using IEnumerator allows to get item by item
var i=Iterator.IterateOne(x => true); // iterate endless
// 1.a) get item by item
i.MoveNext(); Console.WriteLine(i.Current);
i.MoveNext(); Console.WriteLine(i.Current);
// 1.b) loop until 100
int j; while (i.MoveNext() && (j=i.Current)<=100) { Console.WriteLine(j); }
1.a) prints:
1
2
1.b) prints:
3
4
...
100
because it continues counting right after the 1.a) statements have been executed.
You can see that you can advance item by item using MoveNext().
In contrast, IterateAll allows you to use foreach and also LINQ statements for bigger comfort:
// 2. Using IEnumerable makes looping and LINQ easier
var k=Iterator.IterateAll(x => x<100); // limit iterator to 100
// 2.a) Use a foreach loop
foreach(var x in k){ Console.WriteLine(x); } // loop
// 2.b) LINQ: take 101..200 of endless iteration
var lst=Iterator.IterateAll(x=>true).Skip(100).Take(100).ToList(); // LINQ: take items
foreach(var x in lst){ Console.WriteLine(x); } // output list
2.a) prints:
1
2
...
99
2.b) prints:
101
102
...
200
Note: Since IEnumerator<T> and IEnumerable<T> are Generics, they can be used with any type. However, for simplicity I have used int in my examples for type T.
This means, you can use one of the return types IEnumerator<ProductMixHeader> or IEnumerable<ProductMixHeader> (the custom class you have mentioned in your question).
The type List<ProductMixHeader> does not implement any of these interfaces, which is the reason why you can't use it that way. But Example 2.b) is showing how you can create a list from it.
If you're creating a list by appending .ToList() then the implication is, that it will create a list of all elements in memory, while an IEnumerable allows lazy creation of its elements - in terms of performance, it means that elements are enumerated just in time - as late as possible, but as soon as you're using .ToList(), then all elements are created in memory. LINQ tries to optimize performance this way behind the scenes.
DotNetFiddle of all examples

#Ian P´s answer helped me a lot to understand yield and why it is used. One (major) use case for yield is in "foreach" loops after the "in" keyword not to return a fully completed list. Instead of returning a complete list at once, in each "foreach" loop only one item (the next item) is returned. So you will gain performance with yield in such cases.
I have rewritten #Ian P´s code for my better understanding to the following:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
namespace YieldReturnTest
{
public class PrimeFinder
{
private Boolean isPrime(int integer)
{
if (0 == integer)
return false;
if (3 > integer)
return true;
for (int i = 2; i < integer; i++)
{
if (0 == integer % i)
return false;
}
return true;
}
public IEnumerable<int> FindPrimesWithYield()
{
int i;
for (i = 1; i < 2147483647; i++)
{
if (isPrime(i))
{
yield return i;
}
}
}
public IEnumerable<int> FindPrimesWithoutYield()
{
var primes = new List<int>();
int i;
for (i = 1; i < 2147483647; i++)
{
if (isPrime(i))
{
primes.Add(i);
}
}
return primes;
}
}
class Program
{
static void Main(string[] args)
{
PrimeFinder primes = new PrimeFinder();
Console.WriteLine("Finding primes until 7 with yield...very fast...");
foreach (int i in primes.FindPrimesWithYield()) // FindPrimesWithYield DOES NOT iterate over all integers at once, it returns item by item
{
if (i > 7)
{
break;
}
Console.WriteLine(i);
//Console.ReadLine();
}
Console.WriteLine("Finding primes until 7 without yield...be patient it will take lonkg time...");
foreach (int i in primes.FindPrimesWithoutYield()) // FindPrimesWithoutYield DOES iterate over all integers at once, it returns the complete list of primes at once
{
if (i > 7)
{
break;
}
Console.WriteLine(i);
//Console.ReadLine();
}
Console.ReadLine();
Console.ReadLine();
}
}
}

What does the method you're using this in look like? I don't think this can be used in just a loop by itself.
For example...
public IEnumerable<string> GetValues() {
foreach(string value in someArray) {
if (value.StartsWith("A")) { yield return value; }
}
}