We Keep Coding

Could locking an enumerable potentially cause multiple enumeration?

I think ReSharper is lying to me.
I have this extension method that (hopefully) returns an xor of two enumerations:
public static IEnumerable<T> Xor<T>(this IEnumerable<T> first, IEnumerable<T> second)
{
lock (first)
{
lock (second)
{
var firstAsList = first.ToList();
var secondAsList = second.ToList();
return firstAsList.Except(secondAsList).Union(secondAsList.Except(firstAsList));
}
}
}
ReSharper thinks I'm performing a multiple enumeration of an IEnumerable, as you can see, on both the arguments. If I remove the locks, then it's satisfied that I'm not.
Is ReSharper right or wrong? I believe it's wrong.
edit: I do realize that I'm enumerating the lists multiple times, but ReSharper is saying I'm enumerating over the original arguments multiple times, which I don't think is true. I'm enumerating both arguments once into a list so I may then perform the actual set manipulation, but as I see it, I'm not actually iterating over the arguments passed multiple times.
For example, if the passed arguments are actually query results, my belief is this method won't cause a storm of queries to be executed by the set manipulation. I do understand what ReSharper means by warning of multiple enumeration: if the enumerables passed are heavy to generate, then if they're enumerated multiple times, then performing multiple enumerations on them will be much slower.
Also, removing the locks definitely makes ReSharper happier:

You are indeed enumerating both of the lists multiple times. You are not enumerating the enumerables passed as parameters multiple times. Both lists are enumerated once for each call to Except. The call to Union is not enumerating either sequence an additional time, but rather is enumerating the results of the two calls to Except.
Of course, iterating a List multiple times in a context like this isn't really a problem; there aren't negative consequences to iterating an unchanging list multiple times.
The lock statements have nothing whatsoever to do with enumeration of the sequences. Locking on an IEnumerable does not iterate it. Of course, locking on two objects like this, specifically two objects that are not limited in scope to this section of code, is very dangerous. It's quite possible to end up deadlocking the program with locks used in this manor if code elsewhere (such as another invocation of this method) ends up taking locks on the same objects in the opposite order).

This is a bit of a funny one.
First things first: as you've correctly identified, R# is raising this inspection not against the multiple usages of the Lists - there is of course nothing to worry about in multiply enumerating a List - but against (what R' sees as multiple usages of) the IEnumerable arguments. I'm presuming you already know why this would be potentially bad, so I'll skip that.
Now to the question of whether R# is right to complain here. To quote the C# spec,
A lock statement of the form
lock (x) ...
where x is an expression of a reference-type, is precisely
equivalent to
System.Threading.Monitor.Enter(x);
try {
...
}
finally {
System.Threading.Monitor.Exit(x);
}
except that x is only evaluated once.
(I've put in this emphasis because I like this wording; it avoids debates (that I'm definitely not qualified to enter) about whether this is "syntatic sugar" or not.)
Taking a minimal example which produces this R# inspection:
private static void Method(IEnumerable<int> enumerable)
{
lock (enumerable)
{
var list = enumerable.ToList();
}
}
and replacing it by what I think is the precisely equivalent version as mandated by the spec:
private static void Method(IEnumerable<int> enumerable)
{
var x = enumerable;
System.Threading.Monitor.Enter(x);
try
{
var list = enumerable.ToList();
}
finally
{
System.Threading.Monitor.Exit(x);
}
}
also produces the inspection.
The question then is: is R# right to produce this inspection? And this is where I think we get into a grey area. When I pass the following enumerable to either of these methods:
static IEnumerable<int> MyEnumerable()
{
Console.WriteLine("Enumerable enumerated");
yield return 1;
yield return 2;
}
it is not multiply enumerated, which would suggest that R# is wrong to warn here; however, I can't actually find anything in documentation that guarantees this behaviour of either lock or Monitor.Enter. So for me it's not quite as clear-cut as this R# bug I reported, where use of GetType flagged this inspection; but nonetheless I'd guess you're safe.
If you raise this on the R# bug tracker, you can get JetBrains' finest looking at a) whether this behaviour is indeed guaranteed. and b) whether R# can be adjusted to either not warn, or provide a justification for warning.
That said, of course, using locking here probably isn't actually achieving what you want to achieve, as stated in other answers and comments...

Is there a way to determine whether IEnumerable<T> is a sequence generator or true collection?

Let me provide an example:
Suppose i have a method:
public void DoStuff(IEnumerable<T> sequence)
{
if (/* is lazy sequence */) throw ...
// Do stuff...
}
And I want to guard against potentially infinite sequences.
Edit:
To elaborate, guarding against infinite collection is only one of the uses. As Jon mentioned. You can easily have infinite IList. Good point.
Other use might be to detect whether data is potentially unrepeatable. Like a random generator. True collection has data already in memory and iterating it twice will give me same data.

There's nothing that's guaranteed, no. You could see whether the sequence also implements IList<T> - that would prohibit iterator block implementations, but could still be a "virtual" list which continues forever, and it would also fail for some other finite non-iterator-block collections.
In terms of what you're trying to protect against, would a 2-billion-long sequence be okay for what you're trying to do? Would an extension method which threw an exception if you ended up with more than some "limit" (but do so lazily) work for you? (Something like Take, but which blew up at the end.)

This is impossible.
The IEnumerable API cannot possibly tell you whether it is infinite or not. You can try casting it to ICollection to catch the common case where someone has passed you one of those, but if that's what you want then why don't you take an ICollection<...> object in the first place?

Iterators don't support the IEnumerable<>.Reset() method:
public void DoStuff(IEnumerable<T> sequence)
{
sequence.Reset();
// etc..
}
You get a NotSupportedException, which is good, with the Message "Specified method is not supported", which is tolerable.

When NOT to use yield (return) [duplicate]

This question already has answers here:
Closed 12 years ago.
This question already has an answer here:
Is there ever a reason to not use 'yield return' when returning an IEnumerable?
There are several useful questions here on SO about the benefits of yield return. For example,
Can someone demystify the yield
keyword
Interesting use of the c# yield
keyword
What is the yield keyword
I'm looking for thoughts on when NOT to use yield return. For example, if I expect to need to return all items in a collection, it doesn't seem like yield would be useful, right?
What are the cases where use of yield will be limiting, unnecessary, get me into trouble, or otherwise should be avoided?

What are the cases where use of yield will be limiting, unnecessary, get me into trouble, or otherwise should be avoided?
It's a good idea to think carefully about your use of "yield return" when dealing with recursively defined structures. For example, I often see this:
public static IEnumerable<T> PreorderTraversal<T>(Tree<T> root)
{
if (root == null) yield break;
yield return root.Value;
foreach(T item in PreorderTraversal(root.Left))
yield return item;
foreach(T item in PreorderTraversal(root.Right))
yield return item;
}
Perfectly sensible-looking code, but it has performance problems. Suppose the tree is h deep. Then there will at most points be O(h) nested iterators built. Calling "MoveNext" on the outer iterator will then make O(h) nested calls to MoveNext. Since it does this O(n) times for a tree with n items, that makes the algorithm O(hn). And since the height of a binary tree is lg n <= h <= n, that means that the algorithm is at best O(n lg n) and at worst O(n^2) in time, and best case O(lg n) and worse case O(n) in stack space. It is O(h) in heap space because each enumerator is allocated on the heap. (On implementations of C# I'm aware of; a conforming implementation might have other stack or heap space characteristics.)
But iterating a tree can be O(n) in time and O(1) in stack space. You can write this instead like:
public static IEnumerable<T> PreorderTraversal<T>(Tree<T> root)
{
var stack = new Stack<Tree<T>>();
stack.Push(root);
while (stack.Count != 0)
{
var current = stack.Pop();
if (current == null) continue;
yield return current.Value;
stack.Push(current.Left);
stack.Push(current.Right);
}
}
which still uses yield return, but is much smarter about it. Now we are O(n) in time and O(h) in heap space, and O(1) in stack space.
Further reading: see Wes Dyer's article on the subject:
http://blogs.msdn.com/b/wesdyer/archive/2007/03/23/all-about-iterators.aspx

What are the cases where use of yield
will be limiting, unnecessary, get me
into trouble, or otherwise should be
avoided?
I can think of a couple of cases, IE:
Avoid using yield return when you return an existing iterator. Example:
// Don't do this, it creates overhead for no reason
// (a new state machine needs to be generated)
public IEnumerable<string> GetKeys()
{
foreach(string key in _someDictionary.Keys)
yield return key;
}
// DO this
public IEnumerable<string> GetKeys()
{
return _someDictionary.Keys;
}
Avoid using yield return when you don't want to defer execution code for the method. Example:
// Don't do this, the exception won't get thrown until the iterator is
// iterated, which can be very far away from this method invocation
public IEnumerable<string> Foo(Bar baz)
{
if (baz == null)
throw new ArgumentNullException();
yield ...
}
// DO this
public IEnumerable<string> Foo(Bar baz)
{
if (baz == null)
throw new ArgumentNullException();
return new BazIterator(baz);
}

The key thing to realize is what yield is useful for, then you can decide which cases do not benefit from it.
In other words, when you do not need a sequence to be lazily evaluated you can skip the use of yield. When would that be? It would be when you do not mind immediately having your entire collection in memory. Otherwise, if you have a huge sequence that would negatively impact memory, you would want to use yield to work on it step by step (i.e., lazily). A profiler might come in handy when comparing both approaches.
Notice how most LINQ statements return an IEnumerable<T>. This allows us to continually string different LINQ operations together without negatively impacting performance at each step (aka deferred execution). The alternative picture would be putting a ToList() call in between each LINQ statement. This would cause each preceding LINQ statement to be immediately executed before performing the next (chained) LINQ statement, thereby forgoing any benefit of lazy evaluation and utilizing the IEnumerable<T> till needed.

There are a lot of excellent answers here. I would add this one: Don't use yield return for small or empty collections where you already know the values:
IEnumerable<UserRight> GetSuperUserRights() {
if(SuperUsersAllowed) {
yield return UserRight.Add;
yield return UserRight.Edit;
yield return UserRight.Remove;
}
}
In these cases the creation of the Enumerator object is more expensive, and more verbose, than just generating a data structure.
IEnumerable<UserRight> GetSuperUserRights() {
return SuperUsersAllowed
? new[] {UserRight.Add, UserRight.Edit, UserRight.Remove}
: Enumerable.Empty<UserRight>();
}
Update
Here's the results of my benchmark:
These results show how long it took (in milliseconds) to perform the operation 1,000,000 times. Smaller numbers are better.
In revisiting this, the performance difference isn't significant enough to worry about, so you should go with whatever is the easiest to read and maintain.
Update 2
I'm pretty sure the above results were achieved with compiler optimization disabled. Running in Release mode with a modern compiler, it appears performance is practically indistinguishable between the two. Go with whatever is most readable to you.

Eric Lippert raises a good point (too bad C# doesn't have stream flattening like Cw). I would add that sometimes the enumeration process is expensive for other reasons, and therefore you should use a list if you intend to iterate over the IEnumerable more than once.
For example, LINQ-to-objects is built on "yield return". If you've written a slow LINQ query (e.g. that filters a large list into a small list, or that does sorting and grouping), it may be wise to call ToList() on the result of the query in order to avoid enumerating multiple times (which actually executes the query multiple times).
If you are choosing between "yield return" and List<T> when writing a method, consider: is each single element expensive to compute, and will the caller need to enumerate the results more than once? If you know the answers are yes and yes, you shouldn't use yield return (unless, for example, the List produced is very large and you can't afford the memory it would use. Remember, another benefit of yield is that the result list doesn't have to be entirely in memory at once).
Another reason not to use "yield return" is if interleaving operations is dangerous. For example, if your method looks something like this,
IEnumerable<T> GetMyStuff() {
foreach (var x in MyCollection)
if (...)
yield return (...);
}
this is dangerous if there is a chance that MyCollection will change because of something the caller does:
foreach(T x in GetMyStuff()) {
if (...)
MyCollection.Add(...);
// Oops, now GetMyStuff() will throw an exception
// because MyCollection was modified.
}
yield return can cause trouble whenever the caller changes something that the yielding function assumes does not change.

I would avoid using yield return if the method has a side effect that you expect on calling the method. This is due to the deferred execution that Pop Catalin mentions.
One side effect could be modifying the system, which could happen in a method like IEnumerable<Foo> SetAllFoosToCompleteAndGetAllFoos(), which breaks the single responsibility principle. That's pretty obvious (now...), but a not so obvious side effect could be setting a cached result or similar as an optimisation.
My rules of thumb (again, now...) are:
Only use yield if the object being returned requires a bit of processing
No side effects in the method if I need to use yield
If have to have side effects (and limiting that to caching etc), don't use yield and make sure the benefits of expanding the iteration outweigh the costs

Yield would be limiting/unnecessary when you need random access. If you need to access element 0 then element 99, you've pretty much eliminated the usefulness of lazy evaluation.

One that might catch you out is if you are serialising the results of an enumeration and sending them over the wire. Because the execution is deferred until the results are needed, you will serialise an empty enumeration and send that back instead of the results you want.

I have to maintain a pile of code from a guy who was absolutely obsessed with yield return and IEnumerable. The problem is that a lot of third party APIs we use, as well as a lot of our own code, depend on Lists or Arrays. So I end up having to do:
IEnumerable<foo> myFoos = getSomeFoos();
List<foo> fooList = new List<foo>(myFoos);
thirdPartyApi.DoStuffWithArray(fooList.ToArray());
Not necessarily bad, but kind of annoying to deal with, and on a few occasions it's led to creating duplicate Lists in memory to avoid refactoring everything.

When you don't want a code block to return an iterator for sequential access to an underlying collection, you dont need yield return. You simply return the collection then.

If you're defining a Linq-y extension method where you're wrapping actual Linq members, those members will more often than not return an iterator. Yielding through that iterator yourself is unnecessary.
Beyond that, you can't really get into much trouble using yield to define a "streaming" enumerable that is evaluated on a JIT basis.

Which is preferable and less expensive: class matching vs exception?

Which is less expensive and preferable: put1 or put2?
Map<String, Animal> map = new Map<String, Animal>();
void put1(){
for (.....)
if (Animal.class.isAssignableFrom(item[i].getClass())
map.put(key[i], item[i]);
void put2(){
for (.....)
try{
map.put(key[i], item[i]);}
catch (...){}
Question revision:
The question wasn't that clear. Let me revise the question a little. I forgot the casting so that put2 depends on cast exception failure. isAssignableFrom(), isInstanceOf() and instanceof are similar functionally and therefore incur the same expense just one is a method to include subclasses,while the 2nd is for exact type matching and the 3rd is the operator version. Both reflective methods and exceptions are expensive operations.
My question is for those who have done some benchmarking in this area - which is less expensive and preferable: instanceof/isassignablefrom vs cast exception?
void put1(){
for (.....)
if (Animal.class.isAssignableFrom(item[i].getClass())
map.put(key[i], (Animal)item[i]);
void put2(){
for (.....)
try{
map.put(key[i], (Animal)item[i]);}
catch (...){}

Probably you want:
if (item[i] instanceof Animal)
map.put(key[i], (Animal) item[i]);
This is almost certainly much better than calling isAssignableFrom.
Or in C# (since you added the c# tag):
var a = item[i] as Animal;
if (a != null)
map[key[i]] = a;
EDIT: The updated question is which is better: instanceof or cast-and-catch. The functionality is basically the same. The performance difference might not be significant and I would have to measure it; generally throwing an exception is slow, but I don't know about the rest. So I would decide based on style. Say what you mean.
If you always expect expect item[i] to be an Animal, and you're just being extra careful, cast-and-catch. Otherwise I find it much clearer to use instanceof, because that plainly says what you mean: "if this object is an Animal, put it in the map".

I'm confused. If item[i] is not an Animal, then how does map.put(key[i], item[i]) even compile?
That said, the first method says what you're intending to do, although I believe instanceof would be an even better check.

Typically exception handling will be significantly slower because, since it is supposed to be used for exceptional things (rarely occurring) not much work is spent by VM makers on speeding it up.
The tr/catch version of your code I would consider to be abuse of exception handling and would never consider doing it. The fact that you are thinking of doing something like this probably means you have a poor design, items should probably an Animal[] not something else, in which case you don't need to check at runtime at all. Let the compiler do the work for you.

I agree with a previous answer - this will not compile.
But, in my opinion, whether it is an exception or a check depends on the purpose of the function.
Is item[i] not being a Animal an error/exceptional case? Is it expected to happen rarely? In this case, it should be an exception.
If it is part of the logic - meaning you expect item[i] to be many things - and only if it is an Animal you want to put in a map. In this case, the instanceof check is the right way.
UPDATE :
I'll also add an example (bit lame) :
Which is better :
(1)
if ( aNumber < 100 ) {
processNumber(aNumber);
}
or (2)
try {
processNumber(aNumber); //Throws exception if aNumber >= 100
} catch () {
}
This depends on what the program does. (1) may be used for counting numbers < 100 for any integer input. (2) will be used if processNumber expects a percentage value which cannot be greater than 100.
The difference is, it is an error for program (2) to get aNumber > 100. However, for program (1) aNumber > 100 is valid, but "something" happens only when aNumber is < 100.
PS - This may not be helpful to you at all, and I apologize if this is the case.

Your two alternatives are not really equivalent. Which one to choose, depends totally on what your code is supposed to do:
If the item is expected to always be
an Animal, then you should use
put2 (which will throw, if
that's not the case...)
If the item may or may not be an
Animal, you should use put1 (which
checks a condition, not an error...)
Never care about performance in the first place, if you're writing code!

Which of these two GetLargestValue C# implementations is better, and why?

I'm having a disagreement with someone over how best to implement a simple method that takes an array of integers, and returns the highest integer (using C# 2.0).
Below are the two implementations - I have my own opinion of which is better, and why, but I'd appreciate any impartial opinions.
Option A
public int GetLargestValue(int[] values)
{
try {
Array.Sort(values);
return values[values.Length - 1];
}
catch (Exception){ return -1;}
}
Option B
public int GetLargestValue(int[] values)
{
if(values == null)
return -1;
if(values.Length < 1)
return -1;
int highestValue = values[0];
foreach(int value in values)
if(value > highestValue)
highestValue = value;
return highestValue;
}

Ôption B of course.
A is ugly :
Catch(Exception) is a really bad practice
You shoul not rely on exception for null ref, out of range,...
Sorting is way complexier than iteration
Complexity :
A will be O(n log(n)) and even O(n²) in worst case
B worst case is O(n)

A has the side effect that it sorts the array. This might be unexpected by the caller.
Edit: I don't like to return -1 for empty or null array (in both solutions), since -1 might be a legal value in the array. This should really generate an exception (perhaps ArgumentException).

I prefer Option B as it only traverses the collection exactly once.
In Option A, you may have to access many elements more than once (the number of times is dependant upon the implementation of the sort alogrithm).
The Option A is an inefficent implementation, but results in a fairly clear algorithm. It does however use a fairly ugly Exception catch which would only ever be triggered if an empty array is passed in (so could probably be written clearer with a pre-sort check).
PS, you should never simply catch "Exception" and then correct things. There are many types of Exceptions and generally you should catch each possible one and handle accordingly.

The second one is better.
The complexity of the first is O(N LogN), and for the second it is O(N).

I have to choose option B - not that it's perfect but because option A uses exceptions to represent logic.

I would say that it depends on what your goal is, speed or readability.
If processing speed is your goal, I'd say the second solution, but if readability is the goal, I'd pick the first one.
I'd probably go for speed for this type of function, so I'd pick the second one.

There are many factors to consider here. Both options should include the bounds checks that are in option B and do away with using Exception handling in that manner. The second option should perform better most of the time as it only needs to traverse the array once. However, if the data was already sorted or needed to be sorted; then Option A would be preferable.
No sorting algorithm performs in n time, so Option B will be the fastest on average.
Edit: Article on sorting

I see two points here:
Parameter testing as opposed to exception handling: Better use explicit checking, should also be faster.
Sorting and picking the largest value as opposed to walking the whole array. Since sorting involves handling each integer in the array at least once, it will not perform as well as walking the whole array (only) once.
Which is better? For the first point, definitely explicit checking. For the second, it depends...
The first example is shorter, makes it quicker to write and read/understand. The second is faster. So: If runtime efficiency is an issue, choose the second option. If fast coding is your goal, use the first one.