We Keep Coding

Removerange method after the use of take method from the list<T> has unexpected behavior

Does anyone knows why the length after leaving the while loop body in the below code is zero? Please do not provide me chunking algorithm I am already aware of several such algorithms and I know how to solve this problem. My question is only about the wired behavior of RemoverRange or perhaps Take.
static void Main(string[] args)
{
var list = new List<int>();
for (int i = 0; i < 1000; i++)
{
list.Add(i);
}
var chunks = new List<IEnumerable<int>>();
while (list.Count > 0)
{
chunks.Add(list.Take(10));
list.RemoveRange(0, 10);
}
int length = chunks.ToList()[0].Count();
}

At the following line:
var chunks = new List<IEnumerable<int>>();
you create a list, whose Count is 0, since there aren't any items in the list. Then at the foreach statement at each step you add list.Take(10) and after this you Remove the first 10 items from list. The important this here is to realize that list.Take(10) is lazy evaluated (more often you will hear the term deferred execution). The time that this would first be evaluated is at the following line:
int length = chunks.ToList()[0].Count();
At this line list.Take(10) would be evaluated and since you have removed all the items form the list, there aren't any elements in list. For this reason, list.Take(10) return an empty sequence and consequently the chunks.ToList()[0] would be an empty list.
Update deferred execution explanation:
Let that we have the following list:
var list = new List<int> {1, 2, 3, 4, 5 };
var firstThree = list.Take(3);
The variable firstThree holds a reference to an enumerator - specifically an IEnumerable<int>. It does not hold an array or a list of 1,2,3. The first time you will use this iterator you will start to "fetch" data from the list.
For instance you could call the ToList or ToArray methods:
var firstThreeList = firstThree.ToList();
var firstThreeArray = firstThree.ToArray();
Both the above calls would put the iterator in action - in general terms would force the execution of your query in LINQ -. At this very moment, you will traverse the list and you will fetch the first three items.
That being said, it is clear that if in the meanwhile you have modified the list by removing all the numbers from iist, there wouldn't be any elements in the list and you will get nothing.
As a test I would suggest you run the above code once and then run it again but before ToList and ToArray calls to make this call:
list.RemoveAll(x => true);
You will notice now that both firstThreeArray and firstThreeList are empty.

The line
chunks.Add(list.Take(10));
does not actually take 10 items from list, it only tells to take 10 when chunks[i] is first referenced (through e.g. Count()). Since you are altering the list, the reference points to an empty list. You can force the evaluation of the list using
chunks.Add(list.Take(10).ToList());

The problem you're experiencing is that LINQ is kind of like a view. It only actually iterates through the collection when you call a method that requires it to produce a specific value (First(), Last(), Count(), etc.). So the list is only evaluated at the point where you call one of these methods.
chunks.Add(list.Take(10));
This code effectively says "take a reference to list, and when somebody iterates you, only go as far as the first 10 items". To resolve this, you can convert that small section to a list (evaluate those 10 items, and create a new list from them):
chunks.Add(list.Take(10).ToList());
Consider this code:
List<string> names = new List<string>() { "a", "b", "c" };
IEnumerable<string> skip2 = names.Skip(2);
Console.WriteLine(string.Join(", ", skip2)); // "c"
names.Add("d");
names.Add("e");
Console.WriteLine(string.Join(", ", skip2)); // "c, d, e"
Because you use the iterator (IEnumerable<string> skip2) each time you call string.Join(", ", skip2) it will iterate through the list each time, even if the list has changed.
As such, you will get "c" on the first run, and "c, d, e" on the second run.
In fact, this would be perfectly valid (although harder to read):
List<int> list = new List<int>();
IEnumerable<int> values = list.DefaultIfEmpty(0);
list.Add(5);
list.Add(10);
list.Add(15);
Console.WriteLine(values.Average()); // 10

There is no weird behaviour at all. That is exactly the expected behaviour for IEnumerable. What you should keep in mind is that IEnumerable is lazily evaluated, which means it is evaluated when is enumerated, i.e when you actually access the said IEnumerable. What you are doing is basically→
①Get the reference to list object
②Prepare to take the first 10 elements of the said list object, do not yet evaluate!
③Add the not yet evaluated object, in this case→(LINQ.Take(10)) into chunks list.
④Remove the first 10 elements of list
⑤Rinse and Repeat until there are no more items in list
⑥Create a list, which is all made up not yet evaluated items of list.Take(10).
⑦You take the first element of the said chunks, which is not yet evaluate but is a reference to the first 10 elements of list, which is empty!!!
⑧You call Count on IEnumerable instance, which finally evaluates the first ten elements of an enmpy list

IEnumerable<T>.Union(IEnumerable<T>) overwrites contents instead of unioning

I've got a collection of items (ADO.NET Entity Framework), and need to return a subset as search results based on a couple different criteria. Unfortunately, the criteria overlap in such a way that I can't just take the collection Where the criteria are met (or drop Where the criteria are not met), since this would leave out or duplicate valid items that should be returned.
I decided I would do each check individually, and combine the results. I considered using AddRange, but that would result in duplicates in the results list (and my understanding is it would enumerate the collection every time - am I correct/mistaken here?). I realized Union does not insert duplicates, and defers enumeration until necessary (again, is this understanding correct?).
The search is written as follows:
IEnumerable<MyClass> Results = Enumerable.Empty<MyClass>();
IEnumerable<MyClass> Potential = db.MyClasses.Where(x => x.Y); //Precondition
int parsed_key;
//For each searchable value
foreach(var selected in SelectedValues1)
{
IEnumerable<MyClass> matched = Potential.Where(x => x.Value1 == selected);
Results = Results.Union(matched); //This is where the problem is
}
//Ellipsed....
foreach(var selected in SelectedValuesN) //Happens to be integer
{
if(!int.TryParse(selected, out parsed_id))
continue;
IEnumerable<MyClass> matched = Potential.Where(x => x.ValueN == parsed_id);
Results = Results.Union(matched); //This is where the problem is
}
It seems, however, that Results = Results.Union(matched) is working more like Results = matched. I've stepped through with some test data and a test search. The search asks for results where the first field is -1, 0, 1, or 3. This should return 4 results (two 0s, a 1 and a 3). The first iteration of the loops works as expected, with Results still being empty. The second iteration also works as expected, with Results containing two items. After the third iteration, however, Results contains only one item.
Have I just misunderstood how .Union works, or is there something else going on here?

Because of deferred execution, by the time you eventually consume Results, it is the union of many Where queries all of which are based on the last value of selected.
So you have
Results = Potential.Where(selected)
.Union(Potential.Where(selected))
.Union(potential.Where(selected))...
and all the selected values are the same.
You need to create a var currentSelected = selected inside your loop and pass that to the query. That way each value of selected will be captured individually and you won't have this problem.

You can do this much more simply:
Reuslts = SelectedValues.SelectMany(s => Potential.Where(x => x.Value == s));
(this may return duplicates)
Or
Results = Potential.Where(x => SelectedValues.Contains(x.Value));

As pointed out by others, your LINQ expression is a closure. This means your variable selected is captured by the LINQ expression in each iteration of your foreach-loop. The same variable is used in each iteration of the foreach, so it will end up having whatever the last value was. To get around this, you will need to declare a local variable within the foreach-loop, like so:
//For each searchable value
foreach(var selected in SelectedValues1)
{
var localSelected = selected;
Results = Results.Union(Potential.Where(x => x.Value1 == localSelected));
}
It is much shorter to just use .Contains():
Results = Results.Union(Potential.Where(x => SelectedValues1.Contains(x.Value1)));
Since you need to query multiple SelectedValues collections, you could put them all inside their own collection and iterate over that as well, although you'd need some way of matching the correct field/property on your objects.
You could possibly do this by storing your lists of selected values in a Dictionary with the name of the field/property as the key. You would use Reflection to look up the correct field and perform your check. You could then shorten the code to the following:
// Store each of your searchable lists here
Dictionary<string, IEnumerable<MyClass>> DictionaryOfSelectedValues = ...;
Type t = typeof(MyType);
// For each list of searchable values
foreach(var selectedValues in DictionaryOfSelectedValues) // Returns KeyValuePair<TKey, TValue>
{
// Try to get a property for this key
PropertyInfo prop = t.GetProperty(selectedValues.Key);
IEnumerable<MyClass> localSelected = selectedValues.Value;
if( prop != null )
{
Results = Results.Union(Potential.Where(x =>
localSelected.Contains(prop.GetValue(x, null))));
}
else // If it's not a property, check if the entry is for a field
{
FieldInfo field = t.GetField(selectedValues.Key);
if( field != null )
{
Results = Results.Union(Potential.Where(x =>
localSelected.Contains(field.GetValue(x, null))));
}
}
}

No, your use of union is absoloutely correct.
The only thing to keep in mind is it excludes duplicates as based on the equality operator. Do you have sample data?

Okay, I think you are are haveing a problem because Union uses deferred execution.
What happens if you do,
var unionResults = Results.Union(matched).ToList();
Results = unionResults;

Efficient Hierarchal Linq query using multiple properties

I have a fairly large collection of foo { int id, int parentid, string name}.
I am looking to collect a list of foo objects where the object has a name of "bar3", and is a child of an object named "bar2", which is a child of an object with an ID of 1.
What sort of collection should I use (I've been playing with lookups and dictionaries with not a whole lot of success) and how should I write this to make an efficient function out of this? There are approximately 30K foo objects and my method is choking to death.
Thanks!

If I really had to stick with this layout for foo, and I really had to make lookups as fast as possible (I don't care about memory size, and will be reusing the same objects repeatedly, so the cost of setting up a set of large structures in memory would be worth it), then I would do:
var byNameAndParentLookup = fooSource.ToLookup(f => Tuple.Create(f.parentid, f.name)); //will reuse this repeatedly
var results = byNameAndParentLookup[Tuple.Create(1, "bar2")].SelectMany(f => byNameAndParentLookup[Tuple.Create(f.id, "bar3")]);
That said, if I was going to store tree data in memory, I'd prefer to create a tree-structure, where each foo had a children collection (perhaps a dictionary keyed on name).
Edit: To explain a bit.
fooSource.ToLookup(f => Tuple.Create(f.parentid, f.name))
Goes through all the items in fooSource (wherever our foo objects are coming from), and for each one creates a tuple of the parentid and the name. This is used as a key for a lookup, so for each parentid-name combination we can retrieve 0 or more foo objects with that combo. (This will use the default string comparison, if you want something else such as case-insensitive, create an IEqualityComparer<Tuple<int, string>> implementation that does the comparison you want and use .ToLookup(f => Tuple.Create(f.parentid, f.name), new MyTupleComparer())).
The second line can be broken down into:
var partWayResults = byNameAndParentLookup[Tuple.Create(1, "bar2")];
var results = partWayResults.SelectMany(f => byNameAndParentLookup[Tuple.Create(f.id, "bar3")]);
The first line simply does a search on our lookup, so it returns an enumeration of those foo objects which have a parentid of 1 and a name of "bar2".
SelectMany takes each item of an enumeration or queryable, and computes an expression that returns an enumeration, which is then flattened into a single enumeration.
In other words, it works a bit like this:
public static SelectMany<TSource, TResult>(this IEnumerable<TSource> source, Func<TSource, IEnumerable<TResult>> func)
{
foreach(TSource item in source)
foreach(TResult producedItem in func(item))
yield return producedItem;
}
In our case, the expression passed through takes the id of the element found in the first lookup, and then looks for any elements that have that as their parentid and have the name "bar2".
Hence, for every item with parentid 1 and name bar2, we find every item with that first item's id as its parentid and the name bar3. Which is what was wanted.

Check this out: QuickGraph
I've never actually used it but it seems well documented.
Alternatively you can try the C5 Generic Collection Library
I got this from this tread

I can suggest you to group all items by parentId first then apply conditions on it. First you will need to find group with bar1 element, than you should select all its childs and try to find element with name bar 2...
I can suggest such solution, it not the best but it work (thirdLevelElements will contain needed elements). I've used foreachs to make it clear, this logic could be written in linq statements but for me it will be complicated to understand.
var items = new[]
{
new Foo{id=1,parentid = 0, name="bar1"},
new Foo{id=2,parentid = 1, name="bar2"},
new Foo{id=3,parentid = 2, name="bar3"},
new Foo{id=4,parentid = 0, name="bar12"},
new Foo{id=5,parentid = 1, name="bar13"},
new Foo{id=6,parentid = 2, name="bar14"},
new Foo{id=7,parentid = 2, name="bar3"}
};
var groups = items.GroupBy(item => item.parentid).ToList();
var firstLevelElements = items.Where(item => item.name == "bar1");
List<Foo> secondLevelElements = new List<Foo>();
foreach (var firstLevelElement in firstLevelElements)
{
secondLevelElements.AddRange(groups[firstLevelElement.id]
.Where(item => item.name == "bar2"));
}
List<Foo> thirdLevelElements = new List<Foo>();
foreach (var secondLevelElement in secondLevelElements)
{
thirdLevelElements.AddRange(groups[secondLevelElement.id]
.Where(item => item.name == "bar3"));
}

How would I remove items from a List<T>?

I have a list of items.
The problem is the returned items (which I have no control over) return the same items THREE time.
So while the actual things that should be in the list are:
A
B
C
I get
A
B
C
A
B
C
A
B
C
How can I cleanly and easily remove the duplicates? Maybe count the items, divide by three and delete anything from X to list.Count?

The quickest, simplest thing to do is to not remove the items but run a distinct query
var distinctItems = list.Distinct();
If it's a must that you have a list, you can always append .ToList() to the call. If it's a must that you continue to work with the same list, then you'd just have to iterate over it and keep track of what you already have and remove any duplicates.
Edit: "But I'm working with a class"
If you have a list of a given class, to use Distinct you need to either (a) override Equals and GetHashCode inside your class so that appropriate equality comparisons can be made. If you do not have access to the source code (or simply don't want to override these methods for whatever reason), then you can (b) provide an IEqualityComparer<YourClass> implementation as an argument to the Distinct method. This will also allow you to specify the Equals and GetHashCode implementations without having to modify the source of the actual class.
public class MyObjectComparer : IEqualityComparer<MyObject>
{
public bool Equals(MyObject a, MyObject b)
{
// code to determine equality, usually based on one or more properties
}
public int GetHashCode(MyObject a)
{
// code to generate hash code, usually based on a property
}
}
// ...
var distinctItems = myList.Distinct(new MyObjectComparer());

if you are 100% sure that you receive everything you need 3 times, then just
var newList = oldList.Take(oldList.Count / 3).ToList()

Linq has a Distinct() method which does exactly this. Or put the items in a HashSet if you want to avoid duplicated completely.

If you're using C# 3 or up:
var newList = dupList.Distinct().ToList();
If not then sort the list and do the following:
var lastItem = null;
foreach( var item in dupList )
{
if( item != lastItem )
{
newItems.Add(item);
}
lastItem = item;
}

you could simply create a new list and add items to it that are not already there.

Remove items from list 1 not in list 2

I am learning to write lambda expressions, and I need help on how to remove all elements from a list which are not in another list.
var list = new List<int> {1, 2, 2, 4, 5};
var list2 = new List<int> { 4, 5 };
// Remove all list items not in List2
// new List Should contain {4,5}
// The lambda expression is the Predicate.
list.RemoveAll(item => item. /*solution expression here*/ );
// Display results.
foreach (int i in list)
{
Console.WriteLine(i);
}

You can do this via RemoveAll using Contains:
list.RemoveAll( item => !list2.Contains(item));
Alternatively, if you just want the intersection, using Enumerable.Intersect would be more efficient:
list = list.Intersect(list2).ToList();
The difference is, in the latter case, you will not get duplicate entries. For example, if list2 contained 2, in the first case, you'd get {2,2,4,5}, in the second, you'd get {2,4,5}.

Solution for objects (maybe easier than horaces solution):
If your list contains objects, rather than scalars, it is that simple, by removing by one selected property of the objects:
var a = allActivePatientContracts.RemoveAll(x => !allPatients.Select(y => y.Id).Contains(x.PatientId));

list = list.Except(list2).ToList();

This question has been marked as answered, but there is a catch. If your list contains an object, rather than a scalar, you need to do a bit more work.
I tried this over and over with Remove() and RemoveAt() and all sorts of things and none of them worked correctly. I couldn't even get a Contains() to work correctly. Never matched anything. I was stumped until I got the suspicion that maybe it could not match up the item correctly.
When I realized this, I refactored the item class to implement IEquatable, and then it started working.
Here is my solution:
class GenericLookupE : IEquatable<GenericLookupE>
{
public string ID { get; set; }
public bool Equals( GenericLookupE other )
{
if ( this.ID == other.ID ) return true;
return false;
}
}
After I did this, the above RemoveAll() answer by Reed Copsey worked perfectly for me.
See: http://msdn.microsoft.com/en-us/library/bhkz42b3.aspx