I would like to know how to program what Microsoft is suggesting from the MSDN guidelines for collections, which state the following:
AVOID using ICollection<T> or ICollection as a parameter just to access the
Count property. Instead, consider using IEnumerable<T> or IEnumerable and
dynamically checking whether the object implements ICollection<T> or ICollection.
In short, how do I implement ICollection on an IEnumerable? Microsoft has links all over that article, but no "And here is how you do this" link.
Here is my scenario. I have an MVC web app with a grid that will paginate and have sorting capability on some of the collections. For instance, on an Employee administration screen I display a list of employees in a grid.
Initially I returned the collection as IEnumerable. That was convenient when I didn't need to paginate. But now I'm faced with paginating and needing to extract the Count of employee records to do that. One workaround was to pass an employeeCount integer by ref to my getEmployeeRecords() method and assign the value within that method, but that's just messy.
Based on what I've seen here on StackOverflow, the general recommendation is to use IEnumerable instead of ICollection, or Collection, or IList, or List. So I'm not trying to open up a conversation about that topic. All I want to know is how to make an IEnumerable implement an ICollection, and extract the record count, so my code is more aligned with Microsoft's recommendation. A code sample or clear article demonstrating this would be helpful.
Thanks for your help!
One thing to note is that if you use LINQ's Count() method, it already does the type checking for you:
public static int Count<TSource>(this IEnumerable<TSource> source)
{
if (source == null) throw Error.ArgumentNull("source");
ICollection<TSource> collectionoft = source as ICollection<TSource>;
if (collectionoft != null) return collectionoft.Count;
ICollection collection = source as ICollection;
if (collection != null) return collection.Count;
int count = 0;
using (IEnumerator<TSource> e = source.GetEnumerator())
{
checked
{
while (e.MoveNext()) count++;
}
}
return count;
}
Initially I returned the collection as IEnumerable.
Well there's half your problem. Return types should be as explicit as possible. If you have a collection, make the return type that collection. (I forget where, but this is mentioned in the guidelines.)
Based on what I've seen here on StackOverflow, the general recommendation is to use IEnumerable instead of ICollection, or Collection, or IList, or List.
Some developers have an obsession with casting everything as IEnumerable. I have no idea why, as there is no guidance anywhere from Microsoft that says that is a good idea. (I do know that some think it somehow makes the return value immutable, but really anyone can cast it back to the base type and make changes to it. Or just use dynamic and never even notice you gave them an IEnumerable.)
That's the rule for return types and local variables. For parameters you should be as accepting as possible. In practice that means accepting either IEnumerable or IList depending on whether or not you need to access it by index.
AVOID using ICollection or ICollection as a parameter just to access the
Count property.
The reason for this is that if you need the Count, you probably need to access it by index as well. If not today, then tomorrow. So go ahead and use IList just in case.
(I'm not sure I agree, but it does make some sense.)
In short, how do I implement ICollection on an IEnumerable?
Short answer: the .Count() extension method. Make sure you import System.Linq.
Long answer:
int count = 0;
if (x is ICollection)
count = ((ICollection)x).Count;
else
foreach (var c in x)
count ++;
IEnumerable is an interface, and so is ICollection. It's the object's type that implements one or the other or both. You can check if an object implements ICollection with obj is ICollection.
Example:
public class MyCollection<T> : IEnumerable<T>, ICollection<T>
{
// ... Implemented methods
}
// ...
void Foo(IEnumerable<int> elements)
{
int count;
if (elements is ICollection<int>) {
count = ((ICollection<int>)elements).Count;
}
else {
// Use Linq to traverse the whole enumerable; less efficient, but correct
count = elements.Count();
}
}
// ...
MyCollection<int> myStuff;
Foo(myStuff);
Doesn't ICollection implement IEnumerable already? If you need a collection then you need a collection.
Related
To be able to treat ArraySegment directly like an array, you must cast it to IList with "as". This is described as the proper behavior here:
use of ArraySegment class?
and here:
dotNet ArraySegment Struct
Specifically, Microsoft document says:
If you want to retrieve an element by its index in the ArraySegment
object, you must cast it to an IList object and retrieve it or
modify it by using the IList.Item[Int32] property. The following
example retrieves the element in an ArraySegment object that
delimits a section of a string array.
What's perplexing is that IList has methods like "Remove" and "RemoveAt". You would expect those to not work on an arraysegment cast as a List. I tested this, and in fact calling Remove throws a runtime error. But the compiler doesn't catch the problem.
I'm surprised that Microsoft considered this acceptable behavior in the design of ArraySegment.
I was trying to brainstorm a wrapper class or some way to hide the List methods that shouldn't be called on the ArraySegment as List. But I couldn't come up with anything.
Does anyone have a suggestion on how to fix this?
EDIT: IReadOnlyList has been suggested instead of IList.
IReadOnlyList causes the List to completely read-only, preventing you from modifying the value of elements stored in underlying array. I want to be able to modify the original array values. I just don't want to be able to write list.Remove() or list.Add() since it's clearly wrong and the compiler shouldn't be allowing it.
To anyone who might suggest Span as an alternative:
I am aware of Span, but Span currently has limitations in .NET Framework and Standard. Specifically, it can only be used as a local variable, and thus cannot be passed to other methods.
And to be honest, I actually think Microsoft's IEnumerable heirarchy leaves a bit to be desired -- I can't figure out any way to make an Indexable sequence like List without it offering Add/Remove functionality. ICollection doesn't support Indexing. If anyone has suggestions on that issue itself, I'm all ears.
Turns out, in .NET 4.7.2, the ArraySegment<T> doesn't expose an indexer unless if it's cast to the IList<T> interface, but it does in .NET Core 2.1.
You may cast to the IReadOnlyList<T> interface; note that it doesn't prevent you from changing the contained objects themselves if they are mutable:
The IReadOnlyList<T> represents a list in which the number and order of list elements is read-only. The content of list elements is not guaranteed to be read-only.
So, it only guarantees that the collection itself (the container) is immutable (no adds, removes, replacements). You can't replace an element because the indexer doesn't have a setter.
There's also the ReadOnlyCollection<T> wrapper class (add using System.Collections.ObjectModel;). Also no setter on the indexer though.
If none of these work for you, and you want to implement a wrapper, just create a class that takes in an IList<T> in the constructor, and implement your own indexer. I'd also implement IEnumerable<T>, so that you get LINQ support, and it will also make it work with the foreach loop.
// Add these...
using System.Collections;
using System.Collections.Generic;
//...
public class MyWrapper<T> : IEnumerable<T>
{
private IList<T> _internalList;
public MyWrapper(IList<T> list)
{
_internalList = list;
}
public int Count => _internalList.Count;
// the indexer
public T this[int index]
{
get => _internalList[index];
set => _internalList[index] = value;
}
public IEnumerator<T> GetEnumerator()
{
return _internalList.GetEnumerator();
}
IEnumerator IEnumerable.GetEnumerator()
{
return this.GetEnumerator();
}
}
And then just pass the array segment to the constructor (without casting).
P.S. If this is some internal code, used in few places only, writing a wrapper may or may not be worth the trouble; it's OK to use an IList<T> as long as its clear what the intention was.
Also, if you don't mind working with a copy of the array that's only limited to the range defined by the array segment, and if the copy operation cost is not a concern, you can call either the ToArray or ToList extension method on the segment. But I suspect you want the original array to be updated and you just need a "window" into a segment.
You can still use ArraySegment, which will prevent adding/removing items and access/update items by index through .Array property
var numbers = new[] { 1, 2, 3, 4 };
var segment = new ArraySegment<int>(numbers);
segment.Array[2] = 101;
// segment contains now { 1, 2, 101, 4}
You can always create extension methods to improve readability
public static void UpdateAt<T>(this ArraySegment<T> segment, int index, T value)
{
segment.Array[index] = value;
}
You can create extension methods that allow calling the indexer directly, without unnecessary boxing:
public static T Get<T>(this ArraySegment<T> segment, int index)
{
return GetList<ArraySegment<T>, T>(segment, index);
}
private static T GetList<TList, T>(TList list, int index) where TList : struct, IReadOnlyList<T>
{
return list[index];
}
public static void Set<T>(this ArraySegment<T> segment, int index, T value)
{
SetList<ArraySegment<T>, T>(segment, index, value);
}
private static void SetList<TList, T>(TList list, int index, T value) where TList : struct, IReadOnlyList<T>
{
list[index] = value;
}
Sadly extension indexers are not yet possible, but this at least allows to use the provided indexer in a convenient and fast way.
Given the below two samples of LINQ, at what point is a LINQ data source determined?
int[] numbers = new int[7] { 0, 1, 2, 3, 4, 5, 6 };
IEnumerable<int> linqToOjects = numbers.Where(x => true);
XElement root = XElement.Load("PurchaseOrder.xml");
IEnumerable<XElement> linqToXML = root.Elements("Address").Where(x => true);
My understanding is that the underlying code used to query these two different data sources lives within the IEnumerable object produced by the LINQ methods.
My question is, at what point exactly is it determined whether code will be generated to use the Linq To Objects library or the Linq To XML library?
I would assume that the underlying code (the code which actually does the work of querying the data) used to query these data sources exist within their own libraries and are called upon dependent on the data source. I have looked at https://referencesource.microsoft.com/ to look at the code of the Where clause/extension method thinking that the call to the desired provider might be in there, but it appears to be generic.
How is the magic which goes into the IEnumerable determined?
The "data source" is determined immediately. For example, in your first example, the return value of Where is an object that implements IEnumerable<int> (the Enumerable.WhereArrayIterator<int> class in particular) that has a dependency on the numbers object (stored as a field). And the return value of Where in the second example is an enumerable object that has an dependency on the xml element object. So even before you start enumerating, the resulting enumerable knows where to get the data from.
My question is, at what point exactly is it determined whether code
will be generated to use the Linq To Objects library or the Linq To
XML library?
I think there is no code generation. LINQ just uses the datasource enumerator.
You have a class that implement IEnumerable
Exposes the enumerator, which supports a simple iteration over a
collection of a specified type.
So you can use the method GetEnumerator.
Returns an enumerator that iterates through the collection.
And this all LINQ needs to work, an enumerator.
In your example you use the Where LINQ extension method to apply some filter.
IEnumerable<T> Where(this IEnumerable<T> source, Func<T, bool> predicate)
In the implementation we need to:
- get the enumerator (source.GetEnumerator())
- iterate through the collection and apply the filter (predicate)
In the Enumerable reference source you have the implementation of the method Where. You can see that he uses some specific implementation for array (TSource[]) and list (List), but he uses WhereEnumerableIterator for all the other classes that implement IEnumerable.
So there is no code generation, the code is there.
I think you can understand the implementation of the class WhereEnumerableIterator, you only need to understand first how to implement IEnumerator.
Here you can see the implementation of MoveNext. They call source.GetEnumerator() and then they iterate through the collection (enumerator.MoveNext()) and apply the filter (predicate(item)).
public override bool MoveNext() {
switch (state) {
case 1:
enumerator = source.GetEnumerator();
state = 2;
goto case 2;
case 2:
while (enumerator.MoveNext()) {
TSource item = enumerator.Current;
if (predicate(item)) {
current = item;
return true;
}
}
Dispose();
break;
}
return false;
}
XContainer.GetElement returns an IEnumerable using the yield keyword.
When you use the yield keyword in a statement, you indicate that the
method, operator, or get accessor in which it appears is an iterator.
Using yield to define an iterator removes the need for an explicit
extra class (the class that holds the state for an enumeration, see
IEnumerator for an example) when you implement the IEnumerable and
IEnumerator pattern for a custom collection type.
Thanks to the magic of yield keyword we can obtain an IEnumerable, and we can enumerate the collection. And this is the only thing LINQ needs.
I recently learned that the objects created by .NET's LINQ implementation is inefficient for specific enumeration types.
Take a look at this code:
public class DummyCollection : ICollection<int>
{
public IEnumerator<int> GetEnumerator()
{
throw new Exception();
}
public int Count
{
get
{
return 10;
}
}
//some more interface methods
}
basically, instances of DummyCollection have a size of 10, but throws an exception if it is actually enumerated.
now here:
var d = new DummyCollection();
Console.WriteLine(d.Count());
A 10 is printed without error, but this piece of code:
var l = d.Select(a=> a);
Console.WriteLine(l.Count());
throws an exception, despite it being trivial to say that l's size is 10 as well (since Select offers 1-to-1 mapping). What this basically means is, that when checking the length of an Ienumerable, the input might be a Select-wrapped Collection, thus extending the computation time from an O(1) to a staggering O(n) (could be even worse, if the selection function is particularly cumbersome).
I know that you sacrifice efficiency when you ask for LINQ's generics, but this seems like such a simple problem to fix. I checked online and couldn't find anyone addressing this. Is there a way to bypass this shortcoming? Is anyone looking into this? Is anyone fixing this? Is this just an edge case that isn't that much of a problem? Any insight is appreciated.
You can see how Count() extension method is implemented here. Basically is something like this:
public static int Count<TSource>(this IEnumerable<TSource> source)
{
if (source == null) throw Error.ArgumentNull("source");
ICollection<TSource> collectionoft = source as ICollection<TSource>;
if (collectionoft != null) return collectionoft.Count;
ICollection collection = source as ICollection;
if (collection != null) return collection.Count;
int count = 0;
using (IEnumerator<TSource> e = source.GetEnumerator()) {
checked {
while (e.MoveNext()) count++;
}
}
return count;
}
As you can see the method check first is the source is of type ICollection<TSource> or ICollection, if that is the case then there is no need to iterate counting the elements, just return Count property.
In your first case Count property is called returning 10 and GetEnumerator() method is never called.
When you use Select() method you're wrapping the collection into another type that isn't an ICollection (in above link you can also see Select() implementation), therefore the iteration is necessary.
In your second case, when you call Count() your GetEnumerator() method is called and the exception is thrown.
IEnumerable<T> doesn't have a concept of Count. This exists in implementations, which (apart from the odd shortcut here and there) have no role in LINQ to Objects. If you project an implementation of IEnumerable<T> (such as ICollection<T>), with Select, the only real guarantee you have is that the output will be IEnumerable<T>... which has no Count.
LINQ should be thought of as dealing with sequences of items, one at a time, only with a concept of current and next item (or the end of sequence). Knowing about the number of items is a (potentially) costly operation that requires iteration of all the items being counted, other than in a few, optimized cases.
Given that LINQ relies on iteration in preference to indexes and counts means that an IEnumerable that errors when you try to iterate it is going to need some super weird special-casing to fly. To me, it wouldn't be a very useful use-case.
I am looking at some legacy code. The class uses an ArrayList to keep the items. The items are fetched from Database table and can be up to 6 million. The class exposes a method called 'ListCount' to get the count of the items in the Arraylist.
Class Settings
{
ArrayList settingsList ;
public Settings()
{
settingsList = GetSettings();//Get the settings from the DB. Can also return null
}
public int ListCount
{
get
{
if (settingsList == null )
return 0;
else
return settingsList.Count;
}
}
}
The ListCount is used to check if there are items in the list. I am wondering to introduce 'Any' method to the class.
public bool Any(Func<vpSettings, bool> predicate)
{
return settingsList !=null && settingsList.Cast<vpSettings>().Any(predicate);
}
The question is does the framework do some kind of optimization and maintains a count of the items or does it iterate over the Arraylist to get the count? Would it be advisable to add the 'Any' method as above.
Marc Gravel in the following question advises to use Any for IEnumerable
Which method performs better: .Any() vs .Count() > 0?
The .NET reference source says that ArrayList.Count returns a cached private variable.
For completeness, the source also lists the implementation of the Any() extension method here. Essentially the extension method does a null check and then tries to get the first element via the IEnumerable's enumerator.
The ArrayList is actually implementing IList, which should be faster than the .Any(). The reason though because it is implementing the Count Property not the Count Method. The Count Property should do a quick check then grab the proper property.
Which looks similar to:
ICollection<TSource> collection1 = source as ICollection<TSource>;
if (collection1 != null)
return collection1.Count;
ICollection collection2 = source as ICollection;
if (collection2 != null)
return collection2.Count;
Marc Gravel advises to use Any() over Count() (the extension method), but not necessarily over Count (the property).
The Count property is always going to be faster, because it's just looking up an int that's stored on the heap. Using linq requires a (relatively) expensive object allocation to create the IEnumerator, plus whatever overhead there is in MoveNext (which, if the list is not empty, will needlessly copy the value of the ArrayList's first member to the Current property before returning true).
Now this is all pretty trivial for performance, but the code to do it is more complex, so it should only be used if there is a compelling performance benefit. Since there's actually a trivial performance penalty, we should choose the simpler code. I would therefore implement Any() as return Count > 0;.
However, your example is implementing the parameterized overload of Any. In that case, your solution, delegating to the parameterized Any extension method seems best. There's no relationship between the parameterized Any extension method and the Count property.
ArrayList implements IList so it does have a Count property. Using that would be faster than Any(), if all you care is check the container (non-)emptiness.
I'm aware that .Count() is an extension method in LINQ, and that fundamentally it uses the .Count, so I'm wondering, when should I use Count() and when should I use .Count? Is .Count() predominately better saved for queryable collections that are yet to be executed, and therefore don't have an enumeration yet? Am I safer simply always using .Count() extension method, or vice versa for the property? Or is this solely conditional depending on the collection?
Any advice, or articles, are greatly appreciated.
Update 1
After decompiling the .Count() extension method in LINQ it appears to be using the .Count property if the IEnumerable<T> is an ICollection<T> or ICollection, which is what most answers have suggested. The only real overhead now that I can see is the additional null and type checks, which isn't huge I suppose, but could still make a small amount of difference if performance were of the utmost importance.
Here's the decompiled LINQ .Count() extension method in .NET 4.0.
public static int Count<TSource>(this IEnumerable<TSource> source)
{
if (source == null)
{
throw Error.ArgumentNull("source");
}
ICollection<TSource> collection = source as ICollection<TSource>;
if (collection != null)
{
return collection.Count;
}
ICollection collection2 = source as ICollection;
if (collection2 != null)
{
return collection2.Count;
}
int num = 0;
checked
{
using (IEnumerator<TSource> enumerator = source.GetEnumerator())
{
while (enumerator.MoveNext())
{
num++;
}
}
return num;
}
}
The extension method works on any IEnumerable<T> but it is costly because it counts the sequence by iterating it. There is an optimization if the sequence is ICollection<T> meaning that the length of the collection is known. Then the Count property is used but that is an implementation detail.
The best advice is to use the Count property if available for performance reasons.
Is .Count() predominately better saved for queryable collections that are yet to be executed, and therefore don't have an enumeration yet?
If your collection is IQueryable<T> and not IEnumerable<T> then the query provider may be able to return the count in some efficient maner. In that case you will not suffer a performance penalty but it depends on the query provider.
An IQueryable<T> will not have a Count property so there is no choice between using the extension method and the property. However, if you query provider does not provide an efficient way of computing Count() you might consider using .ToList() to pull the collection to the client side. It really depends on how you intend to use it.
Count retrieves the property from a List (already calculated). Count() is an aggregation, like Sum(), Average(), etc. What it does is to count the items in the enumerable (I believe it internally uses the Count property if the enumerable is a list).
This is an example of the concrete use of the Count() method, when it doesn't just use the Count property:
var list = new List {1,2,3,4,5,6,7,8,9,10};
var count = list.Where(x => x > 5).Count();
Also, Count() has an overload that will count the items matching the predicate:
var count = list.Count(x => x > 5);
You should use Count() when all you have is an interface that doesn't expose a Count or Length property, such as IEnumerabe<T>.
If you're dealing with a collection or collection interface (such as List<T> or ICollection) then you can simply use the Count property, likewise if you have an array use the Length property.
The implementation of the Count() extension property will use the underlying collection's Count property if it is available. Otherwise the collection will be enumerated to calculate the count.
Agreed with comments of .Count if it is available (i.e. an object that implements ICollection<T> under the bonnet).
But they are wrong about .Count() being 'costly'. Enumerable.Count() will check if the object implements ICollection<T>.Count before it enumerates the elements and count them.
I.e. something like,
public int Enumerable.Count<TSource>(IEnumerable<TSource> source)
{
var collection = source as ICollection
if (collection != null)
{
return collection.Count;
}
}
I'm not sure it matters since Count() probably just reads the Count property. The performance difference is truly negligible. Use whichever you like. I use Count() when possible just to be consistent.
As others have said, if you have an ICollection<T>, use the Count property.
I would suggest that the IEnumerable.Count() method is really intended for use when the only thing you want to do with the elements of an enumeration is count them. The equivalent of SQL "SELECT COUNT(...".
If in addition you want to do something else with the elements of an enumeration, it makes more sense to generate a collection (usually a list using ToList()), then you can use the Count property and do whatever else you want to do.