We Keep Coding

Does Linq's IEnumerable.Select return a reference to the original IEnumerable?

I was trying to clone an List in my code, because I needed to output that List to some other code, but the original reference was going to be cleared later on. So I had the idea of using the Select extension method to create a new reference to an IEnumerable of the same elements, for example:
List<int> ogList = new List<int> {1, 2, 3};
IEnumerable<int> enumerable = ogList.Select(s => s);
Now after doing ogList.Clear(), I was surprised to see that my new enumerable was also empty.
So I started fiddling around in LINQPad, and saw that even if my Select returned different objects entirely, the behaviour was the same.
List<int> ogList = new List<int> {1, 2, 3};
IEnumerable<int> enumerable = ogList.Select(s => 5); // Doesn't return the original int
enumerable.Count().Dump(); // Count is 3
ogList.Clear();
enumerable.Count().Dump(); // Count is 0!
Note that in LINQPad, the Dump()s are equivalent to Console.WriteLine().
Now probably my need to clone the list in the first place was due to bad design, and even if I didn't want to rethink the design I could easily clone it properly. But this got me thinking about what the Select extension method actually does.
According to the documentation for Select:
This method is implemented by using deferred execution. The immediate return value is an object that stores all the information that is required to perform the action. The query represented by this method is not executed until the object is enumerated either by calling its GetEnumerator method directly or by using foreach in Visual C# or For Each in Visual Basic.
So then I tried adding this code before clearing:
foreach (int i in enumerable)
{
i.Dump();
}
The result was still the same.
Finally, I tried one last thing to figure out if the reference in my new enumerable was the same as the old one. Instead of clearing the original List, I did:
ogList.Add(4);
Then I printed out the contents of my enumerable (the "cloned" one), expecting to see '4' appended to the end of it. Instead, I got:
5
5
5
5 // Huh?
Now I have no choice but to admit that I have no idea how the Select extension method works behind the scenes. What's going on?

List/List<T> are for all intents and purposes fancy resizable arrays. They own and hold the data for value types such as your ints or references to the data for reference types in memory and they always know how many items they have.
IEnumerable/IEnumerable<T> are different beasts. They provide a different service/contract. An IEnumerable is fictional, it does not exist. It can create data out of thin air, with no physical backing. Their only promise is that they have a public method called GetEnumerator() that returns an IEnumerator/IEnumerator<T>. The promise that an IEnumerator makes is simple:
some item could be available or not at a time when you decide you need it. This is achieved through a simple method that the IEnumerator interface has: bool MoveNext() - which returns false when the enumeration is completed or true if there was in fact a new item that needed to be returned. You can read the data through a property that the IEnumerator interface has, conveniently called Current.
To get back to your observations/question: as far as the IEnumerable in your example is concerned, it does not even think about the data unless your code tells it to fetch some data.
When you are writing:
List<int> ogList = new List<int> {1, 2, 3};
IEnumerable<int> enumerable = ogList.Select(s => s);
You are saying: Listen here IEnumerable, I might come to you asking for some items at some point in the future. I'll tell you when I will need them, for now sit still and do nothing. With Select(s => s) you are conceptually defining an identity projection of int to int.
A very rough simplified, non-real-life implementation of the select you've written is:
IEnumerable<T> Select(this IEnumerable<int> source, Func<int,T> transformer) something like
{
foreach (var i in source) //create an enumerator for source and starts enumeration
{
yield return transformer(i); //yield here == return an item and wait for orders
}
}
(this explains why you got a 5 when expecting a for, your transform was s => 5)
For value types, such as the ints in your case: If you want to clone the list, clone the whole list or part of it for future enumeration by using the result of an enumeration materialized through a List. This way you create a list that is a clone of the original list, entirely detached from its original list:
IEnumerable<int> cloneOfEnumerable = ogList.Select(s => s).ToList();
Later edit: Of course ogList.Select(s => s) is equivalent to ogList. I'm leaving the projection here, as it was in the question.
What you are creating here is: a list from the result of an enumerable, further consumed through the IEnumerable<int> interface. Considering what I've said above about the nature of IList vs IEnumerable, I would prefer to write/read:
IList<int> cloneOfEnumerable = ogList.ToList();
CAUTION: Be careful with reference types. IList/List make no promise of keeping the objects "safe", they can mutate to null for all IList cares. Keyword if you ever need it: deep cloning.
CAUTION: Beware of infinite or non-rewindable IEnumerables

Provided answers explain why you are not obtaining a cloned list (due to deferred execution of some LINQ extension methods).
However, keep in mind that list.Select(e => e).ToList() will get a real clone only when dealing with value types such as int.
If you have a list of reference types you will receive a cloned list of references to existent objects. In this case you should consider one of the solutions provided here for deep-cloning or my favorite from here (which might be limited by object inner structure).

You have to be aware that an object that implements IEnumerable does not have to be a collection itself. It is an object that makes it possible to get an object that implements IEnumerator. Once you have the enumerator you can ask for the first element and for the next element until there are no more next elements.
Every LINQ function that returns an IEnumerable is not the sequence itself, it only enables you to ask for the enumerator. If you want a sequence, you'll have to use ToList.
There are several other LINQ functions that do not return an IEnumerable, but for instance a Dictionary, or only one element (FirstOrDefault(), Max(), Single(), Any(). These functions will get the enumerator from the IEnumerable and start enumerating until they have the result. Any will only have to check if you can start enumerating. Max will enumerate over all elements and remember the largest one. etc.
You'll have to be aware: as long as your LINQ statement is an IEnumerable of something, your source sequence is not accessed yet. If you change your source sequence before you start enumerating, the enumeration is over your changed source sequence.
If you don't want this, you'll have to do the enumeration before you change your source. Usually this will be ToList, but this can be any of the non-deferred function: Max(), Any(), FirstOrDefault(), etc.
List<TSource> sourceItems = ...
var myEnumerable = sourceItems
.Where(sourceItem => ...)
.GroupBy(sourceItem => ...)
.Select(group => ...);
// note: myEnumerable is an IEnumerable, it is not a sequence yet.
var list1 = sourceItems.ToList(); // Enumerate over the sequence
var first = sourceItems.FirstOrDefault(); // Enumerate and stop after the first
// now change the source, and to the same things again
sourceItems.Clear();
var list1 = sourceItems.ToList(); // returns empty list
var first = sourceItems.FirstOrDefault(); // return null: there is no first element
So every LINQ function that does not return IEnumerable, will start enumerating over sourceItems as the sequence is at the moment that you start enumerating. The IEnumerable is not the sequence itself.

This is an enumerable.
var enumerable = ogList.Select(s => s);
If you iterate through this enumerable, LINQ will in turn iterate over the original resultset. Each and every time. If you do anything to the original enumerable, the results will also be reflected in your LINQ calls.
If you need to freeze the data, store it in a list instead:
var enumerable = ogList.Select(s => s).ToList();
Now you've made a copy. Iterating over this list will not touch the original enumerable.

c# Find an item in 2 / multiple lists

I have the presumably common problem of having elements that I wish to place in 2 (or more) lists. However sometimes I want to find an element that could be in one of the lists. Now there is more than one way of doing this eg using linq or appending, but all seem to involve the unnecessary creation of an extra list containing all the elements of the separate lists and hence waste processing time.
So I was considering creating my own generic FindinLists class which would take 2 lists as its constructor parameters would provide a Find() and an Exists() methods. The Find and Exists methods would only need to search the second or subsequent lists if the item was not found in the first list. The FindInLists class could be instantiated in the getter of a ( no setter)property. A second constructor for the FindInLists class could take an array of lists as its parameter.
Is this useful or is there already a way to search multiple lists without incurring the wasteful overhead of the creation of a super list?

You could use the LINQ Concat function.
var query = list1.Concat(list2).Where(x => x.Category=="my category");

Linq already has this functionality by virtue of the FirstOrDefault method. It uses deferred execution so will stream from any input and will short circuit the return when a matching element is found.
var matched = list1.Concat(list2).FirstOrDefault(e => element.Equals(e));
Update
BaseType matched = list1.Concat(list2).Concat(list3).FirstOrDefault(e => element.Equals(e));

I believe IEnumerable<T>.Concat() is what you need. It doesn't create an extra list, it only iterates through the given pair of collections when queried
Concat() uses deferred execution, so at the time it's called it only creates an iterator which stores the reference to both concatenated IEnumerables. At the time the resulting collection is enumerated, it iterates through first and then through the second.
Here's the decompiled code for the iterator - no rocket science going on there:
private static IEnumerable<TSource> ConcatIterator<TSource>(IEnumerable<TSource> first, IEnumerable<TSource> second)
{
foreach (TSource iteratorVariable0 in first)
{
yield return iteratorVariable0;
}
foreach (TSource iteratorVariable1 in second)
{
yield return iteratorVariable1;
}
}
When looking to the docs for Concat(), I've stumbled across another alternative I didn't know - SelectMany. Given a collection of collections it allows you to work with the children of all parent collections at once like this:
IEnumerable<string> concatenated = new[] { firstColl, secondColl }
.SelectMany(item => item);

you can do something like this:
var list1 = new List<int>{1,2,3,4,5,6,7};
var list2 = new List<int>{0,-3,-4,2};
int elementToPush = 4;//value to find among available lists
var exist = list1.Exists(i=>i==elementToPush) || list2.Exists(j=>j==elementToPush);
If at least one collection required element exists, result is false, otherwise it's true.
One row and no external storage creation.
Hope this helps.

You could probably just create a List of lists and then use linq on that list. It is still creating a new List but it is a list of references rather than duplicating the contents of all the lists.
List<string> a = new List<string>{"apple", "aardvark"};
List<string> b = new List<string>{"banana", "bananananana", "bat"};
List<string> c = new List<string>{"cat", "canary"};
List<string> d = new List<string>{"dog", "decision"};
List<List<string>> super = new List<List<string>> {a,b,c,d};
super.Any(x=>x.Contains("apple"));
the Any call should return after the first list returns true so as requested will not process later lists if it finds it in an earlier list.
Edit: Having written this I prefer the answers using Concat but I leave this here as an alternative if you want something that might be more aesthetically pleasing. ;-)

Is this "listMerging" code improvable?

I have a method who merge two lists. The two merged lists are lists of subtypes objects of the returned list. By "merging" I mean "Allowing duplicates".
So Sub1 and Sub2 types are subtypes from Sup1 type.
Here is my code
var listSub1 = new List<Sub1>(); //With some content added..
var listSub2 = new List<Sub2>(); //With content too..
var listToReturn = new List<Sup1>();
listToReturn.AddRange(listSub1.Select(item => item as Sup1).ToList());
listToReturn.AddRange(listSub2.Select(item => item as Sup1).ToList());
return listeToReturn;
It is working fine but I wonder if it is the best way to merge and cast the lists.

you could write
var newList =
listSub1.Cast<Sup1>()
.Concat(listSub2.Cast<Sup1>()).ToList();
Note that Union is not correct to be used, since it will make sure that objects that are "equal" will come out only once.

return listSub1.Cast<Sup1>().Concat(list2.Cast<Sup1>()).ToList()

var result = listSub1.AddRang(listSub2 ).Cast<Sup1>().ToList()

Why not just use LINQ's Union? That should to the trick.

Assuming I don't have to keep binary compatibility with an older released version, I'd redefine the return type as IEnumerable<Sub1> and code it as:
return listSub1.Concat(listSub2);
If a caller really needs a list, it can call .ToList() itself, but a caller that doesn't need a list (e.g. it's just going to foreach through it all) needn't suffer the delay and memory use of .ToList().
Using .ToList() only when needed is a micro-opt as a single change, but as a coding habit can begin to add up significantly.

you can do with linq:
var listToReturn = listSub1.Cast<Sup1>().Concat(listSub2).ToList();
edit: to match the edited question

How can I retrieve first n elements from Dictionary<string, int>?

Is there a way to retrieve first n elements from a Dictionary in C#?

Dictionaries are not ordered per se, you can't rely on the "first" actually meaning that. From MSDN: "For enumeration... The order in which the items are returned is undefined."
You may be able to use an OrderedDictionary depending on your platform version, and it's not a particularly complex thing to create as a custom descendant class of Dictionary.

Oftentimes omitting the cast to dictionary won't work:
dictionary = dictionary.Take(n);
And neither will a simple case like this:
dictionary = dictionary.Take(n).ToDictionary();
The surest method is an explicit cast:
dictionary = dictionary.Take(n).ToDictionary(kvp => kvp.Key, kvp => kvp.Value);

Note that there's no explicit ordering for a Dictionary, so although the following code will return n items, there's no guarantee as to how the framework will determine which n items to return.
using System.Linq;
yourDictionary.Take(n);
The above code returns an IEnumerable<KeyValuePair<TKey,TValue>> containing n items. You can easily convert this to a Dictionary<TKey,TValue> like so:
yourDictionary.Take(n).ToDictionary();

You can't really take the first N elements from a Dictionary<TKey,TValue> because it is not an ordered collection. So it really has no concept of First, Last, etc ... But as others have pointed out, if you just want to take N elements regardless of order the LINQ take function works fine
var map = GetTheDictionary();
var firstFive = map.Take(5);

Could use Linq for example?
var dictionary = new Dictionary<string, int>();
/// Add items to dictionary
foreach(var item in dictionary.Take(5))
{
// Do something with the first 5 pairs in the dictionary
}

How do I get all the values of a Dictionary<TKey, TValue> as an IList<TValue>?

I have a the following dictionary:
IDictionary<int, IList<MyClass>> myDictionary
and I am wanting to get all the values in the dictionary as an IList....
Just to add a bit of a background as to how I've gotten into this situation....
I have a method that gets me a list of MyClass. I then have another method that converts that list into a dictionary where they key is the id for MyClass. Later on...and without access to that original list...I'm needing to obtain the original ungrouped list of MyClass.
When I pass myDictionary.Values.ToList() to a method that takes an IList I get a compile error that says that it can't convert from
System.Collections.Generic.List<System.Collections.Generic.IList<MyClass>>
to:
System.Collections.Generic.IList<MyClass>
Now, I can understand that its gone and added each of the groups of IList to the new list as separate elements of the list....but in this instance its not really what I'm after. I just want a list of all the values in the entire dictionary.
How then can I get what I'm after without looping through each of the key values in the dictionary and creating the list I want?

Noticed a lot of answer were quite old.
This will also work:
using System.Linq;
dict.Values.ToList();

Because of how a dictionary (or hash table) is maintained this is what you would do. Internally the implementation contains keys, buckets (for collision handling) and values. You might be able to retrieve the internal value list but you're better of with something like this:
IDictionary<int, IList<MyClass>> dict;
var flattenList = dict.SelectMany( x => x.Value );
It should do the trick ;) SelectMany flattens the result which means that every list gets concatenated into one long sequence (IEnumerable`1).

A variation on John's suggestion:
var flattenedValues = dict.Values.SelectMany(x => x);
If you need them in a list, you can of course call ToList:
var flattenedList = dict.Values.SelectMany(x => x).ToList();

dictionary.values.toList();
if You want to get Sum just do
myDictionary.values.sum();

Values gets a ICollection containing the values of your dictionary. As implied by the definition of your dictionary, it can be defined as a ICollection<IList<MyClass>> collection. So if you really want a IList<IList<MyClass>>, use spacedog's solution.
If what you really want is a flat `IList', then there is no other solution than looping through each value :
IList<MyClass> l=new List<MyClass>();
foreach (IList<MyClass> v in myDictionary.Values)
l.AddRange(v);
Note that this is so grossly inefficient that you should think again about using a dictionary for what you are trying to achieve.