C# console application threading issue with Tasks

C# console application threading issue with Tasks - c#

I have the following code snippets:
lock (lockObject)
{
foreach (KeyValuePair<string, List<string>> entry in productDictionary)
{
string writePath = String.Format(#"{0}\{1}-{2}.txt", directoryPath, hour,entry.Key);
Task writeFileTask = Task.Factory.StartNew(() => WriteProductFile(writePath, entry.Value));
}
}
productDictionary is a ConcurrentDictionary of <string, List<string>> which I am trying to iterate over. For each key value pair I am attempting to construct a file path based on the Key and then write out the list of strings stored in the Value. For this purpose I start up a new task that calls the following method:
public static void WriteProductFile(string filePath, List<string> productQuotes)
{
using(StreamWriter streamWriter = new StreamWriter(filePath))
{
productQuotes.ForEach(x => streamWriter.WriteLine(x));
}
}
Stepping through the code everything looks fine at first. Putting a breakpoint at the method call for WriteProductFile shows that the correct parameters are being passed into the method through the Task. However, when my program actually makes it down to the WriteProductFile method the parameters have become mismatched. Ie, a List of strings that does not match the filepath has been passed in and so my data is no good once the program completes. No errors are thrown and the program executes fine but the wrong information is being written out to the wrong files.
I thought that the ConcurrentDictionary and Lock would take care of any threading issues that would arise but apparently I missed something. Any ideas?

You are capturing a loop variable. You must declare a local in inside the loop.
foreach (KeyValuePair<string, List<string>> entry in productDictionary)
{
string writePath = String.Format(#"{0}\{1}-{2}.txt", directoryPath, hour, entry.Key);
List<string> list = entry.Value; // must add this.
Task writeFileTask = Task.Factory.StartNew(() => WriteProductFile(writePath, list));
}
Prior to C# 5, a single loop variable is used for all iterations of the loop. Each closure references the same variable and therefore the value is updated when a new cycle of the loop starts. For a definitive explanation you should read Eric Lippert's post: Closing over the loop variable considered harmfuil.

Related

For Each Row: The collection has been changed, the enumeration operation may not be performed [duplicate]

I can't get to the bottom of this error, because when the debugger is attached, it does not seem to occur.
Collection was modified; enumeration operation may not execute
Below is the code.
This is a WCF server in a Windows service. The method NotifySubscribers() is called by the service whenever there is a data event (at random intervals, but not very often - about 800 times per day).
When a Windows Forms client subscribes, the subscriber ID is added to the subscribers dictionary, and when the client unsubscribes, it is deleted from the dictionary. The error happens when (or after) a client unsubscribes. It appears that the next time the NotifySubscribers() method is called, the foreach() loop fails with the error in the subject line. The method writes the error into the application log as shown in the code below. When a debugger is attached and a client unsubscribes, the code executes fine.
Do you see a problem with this code? Do I need to make the dictionary thread-safe?
[ServiceBehavior(InstanceContextMode=InstanceContextMode.Single)]
public class SubscriptionServer : ISubscriptionServer
{
private static IDictionary<Guid, Subscriber> subscribers;
public SubscriptionServer()
{
subscribers = new Dictionary<Guid, Subscriber>();
}
public void NotifySubscribers(DataRecord sr)
{
foreach(Subscriber s in subscribers.Values)
{
try
{
s.Callback.SignalData(sr);
}
catch (Exception e)
{
DCS.WriteToApplicationLog(e.Message,
System.Diagnostics.EventLogEntryType.Error);
UnsubscribeEvent(s.ClientId);
}
}
}
public Guid SubscribeEvent(string clientDescription)
{
Subscriber subscriber = new Subscriber();
subscriber.Callback = OperationContext.Current.
GetCallbackChannel<IDCSCallback>();
subscribers.Add(subscriber.ClientId, subscriber);
return subscriber.ClientId;
}
public void UnsubscribeEvent(Guid clientId)
{
try
{
subscribers.Remove(clientId);
}
catch(Exception e)
{
System.Diagnostics.Debug.WriteLine("Unsubscribe Error " +
e.Message);
}
}
}

What's likely happening is that SignalData is indirectly changing the subscribers dictionary under the hood during the loop and leading to that message. You can verify this by changing
foreach(Subscriber s in subscribers.Values)
To
foreach(Subscriber s in subscribers.Values.ToList())
If I'm right, the problem will disappear.
Calling subscribers.Values.ToList() copies the values of subscribers.Values to a separate list at the start of the foreach. Nothing else has access to this list (it doesn't even have a variable name!), so nothing can modify it inside the loop.

When a subscriber unsubscribes you are changing contents of the collection of Subscribers during enumeration.
There are several ways to fix this, one being changing the for loop to use an explicit .ToList():
public void NotifySubscribers(DataRecord sr)
{
foreach(Subscriber s in subscribers.Values.ToList())
{
^^^^^^^^^
...

A more efficient way, in my opinion, is to have another list that you declare that you put anything that is "to be removed" into. Then after you finish your main loop (without the .ToList()), you do another loop over the "to be removed" list, removing each entry as it happens. So in your class you add:
private List<Guid> toBeRemoved = new List<Guid>();
Then you change it to:
public void NotifySubscribers(DataRecord sr)
{
toBeRemoved.Clear();
...your unchanged code skipped...
foreach ( Guid clientId in toBeRemoved )
{
try
{
subscribers.Remove(clientId);
}
catch(Exception e)
{
System.Diagnostics.Debug.WriteLine("Unsubscribe Error " +
e.Message);
}
}
}
...your unchanged code skipped...
public void UnsubscribeEvent(Guid clientId)
{
toBeRemoved.Add( clientId );
}
This will not only solve your problem, it will prevent you from having to keep creating a list from your dictionary, which is expensive if there are a lot of subscribers in there. Assuming the list of subscribers to be removed on any given iteration is lower than the total number in the list, this should be faster. But of course feel free to profile it to be sure that's the case if there's any doubt in your specific usage situation.

Why this error?
In general .Net collections do not support being enumerated and modified at the same time. If you try to modify the collection list during enumeration, it raises an exception. So the issue behind this error is, we can not modify the list/dictionary while we are looping through the same.
One of the solutions
If we iterate a dictionary using a list of its keys, in parallel we can modify the dictionary object, as we are iterating through the key-collection and
not the dictionary(and iterating its key collection).
Example
//get key collection from dictionary into a list to loop through
List<int> keys = new List<int>(Dictionary.Keys);
// iterating key collection using a simple for-each loop
foreach (int key in keys)
{
// Now we can perform any modification with values of the dictionary.
Dictionary[key] = Dictionary[key] - 1;
}
Here is a blog post about this solution.
And for a deep dive in StackOverflow: Why this error occurs?

Okay so what helped me was iterating backwards. I was trying to remove an entry from a list but iterating upwards and it screwed up the loop because the entry didn't exist anymore:
for (int x = myList.Count - 1; x > -1; x--)
{
myList.RemoveAt(x);
}

The accepted answer is imprecise and incorrect in the worst case . If changes are made during ToList(), you can still end up with an error. Besides lock, which performance and thread-safety needs to be taken into consideration if you have a public member, a proper solution can be using immutable types.
In general, an immutable type means that you can't change the state of it once created.
So your code should look like:
public class SubscriptionServer : ISubscriptionServer
{
private static ImmutableDictionary<Guid, Subscriber> subscribers = ImmutableDictionary<Guid, Subscriber>.Empty;
public void SubscribeEvent(string id)
{
subscribers = subscribers.Add(Guid.NewGuid(), new Subscriber());
}
public void NotifyEvent()
{
foreach(var sub in subscribers.Values)
{
//.....This is always safe
}
}
//.........
}
This can be especially useful if you have a public member. Other classes can always foreach on the immutable types without worrying about the collection being modified.

I want to point out other case not reflected in any of the answers. I have a Dictionary<Tkey,TValue> shared in a multi threaded app, which uses a ReaderWriterLockSlim to protect the read and write operations. This is a reading method that throws the exception:
public IEnumerable<Data> GetInfo()
{
List<Data> info = null;
_cacheLock.EnterReadLock();
try
{
info = _cache.Values.SelectMany(ce => ce.Data); // Ad .Tolist() to avoid exc.
}
finally
{
_cacheLock.ExitReadLock();
}
return info;
}
In general, it works fine, but from time to time I get the exception. The problem is a subtlety of LINQ: this code returns an IEnumerable<Info>, which is still not enumerated after leaving the section protected by the lock. So, it can be changed by other threads before being enumerated, leading to the exception. The solution is to force the enumeration, for example with .ToList() as shown in the comment. In this way, the enumerable is already enumerated before leaving the protected section.
So, if using LINQ in a multi-threaded application, be aware to always materialize the queries before leaving the protected regions.

InvalidOperationException-
An InvalidOperationException has occurred. It reports a "collection was modified" in a foreach-loop
Use break statement, Once the object is removed.
ex:
ArrayList list = new ArrayList();
foreach (var item in list)
{
if(condition)
{
list.remove(item);
break;
}
}

Actually the problem seems to me that you are removing elements from the list and expecting to continue to read the list as if nothing had happened.
What you really need to do is to start from the end and back to the begining. Even if you remove elements from the list you will be able to continue reading it.

I had the same issue, and it was solved when I used a for loop instead of foreach.
// foreach (var item in itemsToBeLast)
for (int i = 0; i < itemsToBeLast.Count; i++)
{
var matchingItem = itemsToBeLast.FirstOrDefault(item => item.Detach);
if (matchingItem != null)
{
itemsToBeLast.Remove(matchingItem);
continue;
}
allItems.Add(itemsToBeLast[i]);// (attachDetachItem);
}

I've seen many options for this but to me this one was the best.
ListItemCollection collection = new ListItemCollection();
foreach (ListItem item in ListBox1.Items)
{
if (item.Selected)
collection.Add(item);
}
Then simply loop through the collection.
Be aware that a ListItemCollection can contain duplicates. By default there is nothing preventing duplicates being added to the collection. To avoid duplicates you can do this:
ListItemCollection collection = new ListItemCollection();
foreach (ListItem item in ListBox1.Items)
{
if (item.Selected && !collection.Contains(item))
collection.Add(item);
}

This way should cover a situation of concurrency when the function is called again while is still executing (and items need used only once):
while (list.Count > 0)
{
string Item = list[0];
list.RemoveAt(0);
// do here what you need to do with item
}
If the function get called while is still executing items will not reiterate from the first again as they get deleted as soon as they get used.
Should not affect performance much for small lists.

There is one link where it elaborated very well & solution is also given.
Try it if you got proper solution please post here so other can understand.
Given solution is ok then like the post so other can try these solution.
for you reference original link :-
https://bensonxion.wordpress.com/2012/05/07/serializing-an-ienumerable-produces-collection-was-modified-enumeration-operation-may-not-execute/
When we use .Net Serialization classes to serialize an object where its definition contains an Enumerable type, i.e.
collection, you will be easily getting InvalidOperationException saying "Collection was modified;
enumeration operation may not execute" where your coding is under multi-thread scenarios.
The bottom cause is that serialization classes will iterate through collection via enumerator, as such,
problem goes to trying to iterate through a collection while modifying it.
First solution, we can simply use lock as a synchronization solution to ensure that
the operation to the List object can only be executed from one thread at a time.
Obviously, you will get performance penalty that
if you want to serialize a collection of that object, then for each of them, the lock will be applied.
Well, .Net 4.0 which makes dealing with multi-threading scenarios handy.
for this serializing Collection field problem, I found we can just take benefit from ConcurrentQueue(Check MSDN)class,
which is a thread-safe and FIFO collection and makes code lock-free.
Using this class, in its simplicity, the stuff you need to modify for your code are replacing Collection type with it,
use Enqueue to add an element to the end of ConcurrentQueue, remove those lock code.
Or, if the scenario you are working on do require collection stuff like List, you will need a few more code to adapt ConcurrentQueue into your fields.
BTW, ConcurrentQueue doesnât have a Clear method due to underlying algorithm which doesnât permit atomically clearing of the collection.
so you have to do it yourself, the fastest way is to re-create a new empty ConcurrentQueue for a replacement.

Here is a specific scenario that warrants a specialized approach:
The Dictionary is enumerated frequently.
The Dictionary is modified infrequently.
In this scenario creating a copy of the Dictionary (or the Dictionary.Values) before every enumeration can be quite costly. My idea about solving this problem is to reuse the same cached copy in multiple enumerations, and watch an IEnumerator of the original Dictionary for exceptions. The enumerator will be cached along with the copied data, and interrogated before starting a new enumeration. In case of an exception the cached copy will be discarded, and a new one will be created. Here is my implementation of this idea:
using System;
using System.Collections;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Linq;
public class EnumerableSnapshot<T> : IEnumerable<T>, IDisposable
{
private IEnumerable<T> _source;
private IEnumerator<T> _enumerator;
private ReadOnlyCollection<T> _cached;
public EnumerableSnapshot(IEnumerable<T> source)
{
_source = source ?? throw new ArgumentNullException(nameof(source));
}
public IEnumerator<T> GetEnumerator()
{
if (_source == null) throw new ObjectDisposedException(this.GetType().Name);
if (_enumerator == null)
{
_enumerator = _source.GetEnumerator();
_cached = new ReadOnlyCollection<T>(_source.ToArray());
}
else
{
var modified = false;
if (_source is ICollection collection) // C# 7 syntax
{
modified = _cached.Count != collection.Count;
}
if (!modified)
{
try
{
_enumerator.MoveNext();
}
catch (InvalidOperationException)
{
modified = true;
}
}
if (modified)
{
_enumerator.Dispose();
_enumerator = _source.GetEnumerator();
_cached = new ReadOnlyCollection<T>(_source.ToArray());
}
}
return _cached.GetEnumerator();
}
public void Dispose()
{
_enumerator?.Dispose();
_enumerator = null;
_cached = null;
_source = null;
}
IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}
public static class EnumerableSnapshotExtensions
{
public static EnumerableSnapshot<T> ToEnumerableSnapshot<T>(
this IEnumerable<T> source) => new EnumerableSnapshot<T>(source);
}
Usage example:
private static IDictionary<Guid, Subscriber> _subscribers;
private static EnumerableSnapshot<Subscriber> _subscribersSnapshot;
//...(in the constructor)
_subscribers = new Dictionary<Guid, Subscriber>();
_subscribersSnapshot = _subscribers.Values.ToEnumerableSnapshot();
// ...(elsewere)
foreach (var subscriber in _subscribersSnapshot)
{
//...
}
Unfortunately this idea cannot be used currently with the class Dictionary in .NET Core 3.0, because this class does not throw a Collection was modified exception when enumerated and the methods Remove and Clear are invoked. All other containers I checked are behaving consistently. I checked systematically these classes:
List<T>, Collection<T>, ObservableCollection<T>, HashSet<T>, SortedSet<T>, Dictionary<T,V> and SortedDictionary<T,V>. Only the two aforementioned methods of the Dictionary class in .NET Core are not invalidating the enumeration.
Update: I fixed the above problem by comparing also the lengths of the cached and the original collection. This fix assumes that the dictionary will be passed directly as an argument to the EnumerableSnapshot's constructor, and its identity will not be hidden by (for example) a projection like: dictionary.Select(e => e).ΤοEnumerableSnapshot().
Important: The above class is not thread safe. It is intended to be used from code running exclusively in a single thread.

You can copy subscribers dictionary object to a same type of temporary dictionary object and then iterate the temporary dictionary object using foreach loop.

So a different way to solve this problem would be instead of removing the elements create a new dictionary and only add the elements you didnt want to remove then replace the original dictionary with the new one. I don't think this is too much of an efficiency problem because it does not increase the number of times you iterate over the structure.

Avoid collection was modified enumeration operation may not execute [duplicate]

I can't get to the bottom of this error, because when the debugger is attached, it does not seem to occur.
Collection was modified; enumeration operation may not execute
Below is the code.
This is a WCF server in a Windows service. The method NotifySubscribers() is called by the service whenever there is a data event (at random intervals, but not very often - about 800 times per day).
When a Windows Forms client subscribes, the subscriber ID is added to the subscribers dictionary, and when the client unsubscribes, it is deleted from the dictionary. The error happens when (or after) a client unsubscribes. It appears that the next time the NotifySubscribers() method is called, the foreach() loop fails with the error in the subject line. The method writes the error into the application log as shown in the code below. When a debugger is attached and a client unsubscribes, the code executes fine.
Do you see a problem with this code? Do I need to make the dictionary thread-safe?
[ServiceBehavior(InstanceContextMode=InstanceContextMode.Single)]
public class SubscriptionServer : ISubscriptionServer
{
private static IDictionary<Guid, Subscriber> subscribers;
public SubscriptionServer()
{
subscribers = new Dictionary<Guid, Subscriber>();
}
public void NotifySubscribers(DataRecord sr)
{
foreach(Subscriber s in subscribers.Values)
{
try
{
s.Callback.SignalData(sr);
}
catch (Exception e)
{
DCS.WriteToApplicationLog(e.Message,
System.Diagnostics.EventLogEntryType.Error);
UnsubscribeEvent(s.ClientId);
}
}
}
public Guid SubscribeEvent(string clientDescription)
{
Subscriber subscriber = new Subscriber();
subscriber.Callback = OperationContext.Current.
GetCallbackChannel<IDCSCallback>();
subscribers.Add(subscriber.ClientId, subscriber);
return subscriber.ClientId;
}
public void UnsubscribeEvent(Guid clientId)
{
try
{
subscribers.Remove(clientId);
}
catch(Exception e)
{
System.Diagnostics.Debug.WriteLine("Unsubscribe Error " +
e.Message);
}
}
}

What's likely happening is that SignalData is indirectly changing the subscribers dictionary under the hood during the loop and leading to that message. You can verify this by changing
foreach(Subscriber s in subscribers.Values)
To
foreach(Subscriber s in subscribers.Values.ToList())
If I'm right, the problem will disappear.
Calling subscribers.Values.ToList() copies the values of subscribers.Values to a separate list at the start of the foreach. Nothing else has access to this list (it doesn't even have a variable name!), so nothing can modify it inside the loop.

When a subscriber unsubscribes you are changing contents of the collection of Subscribers during enumeration.
There are several ways to fix this, one being changing the for loop to use an explicit .ToList():
public void NotifySubscribers(DataRecord sr)
{
foreach(Subscriber s in subscribers.Values.ToList())
{
^^^^^^^^^
...

A more efficient way, in my opinion, is to have another list that you declare that you put anything that is "to be removed" into. Then after you finish your main loop (without the .ToList()), you do another loop over the "to be removed" list, removing each entry as it happens. So in your class you add:
private List<Guid> toBeRemoved = new List<Guid>();
Then you change it to:
public void NotifySubscribers(DataRecord sr)
{
toBeRemoved.Clear();
...your unchanged code skipped...
foreach ( Guid clientId in toBeRemoved )
{
try
{
subscribers.Remove(clientId);
}
catch(Exception e)
{
System.Diagnostics.Debug.WriteLine("Unsubscribe Error " +
e.Message);
}
}
}
...your unchanged code skipped...
public void UnsubscribeEvent(Guid clientId)
{
toBeRemoved.Add( clientId );
}
This will not only solve your problem, it will prevent you from having to keep creating a list from your dictionary, which is expensive if there are a lot of subscribers in there. Assuming the list of subscribers to be removed on any given iteration is lower than the total number in the list, this should be faster. But of course feel free to profile it to be sure that's the case if there's any doubt in your specific usage situation.

Why this error?
In general .Net collections do not support being enumerated and modified at the same time. If you try to modify the collection list during enumeration, it raises an exception. So the issue behind this error is, we can not modify the list/dictionary while we are looping through the same.
One of the solutions
If we iterate a dictionary using a list of its keys, in parallel we can modify the dictionary object, as we are iterating through the key-collection and
not the dictionary(and iterating its key collection).
Example
//get key collection from dictionary into a list to loop through
List<int> keys = new List<int>(Dictionary.Keys);
// iterating key collection using a simple for-each loop
foreach (int key in keys)
{
// Now we can perform any modification with values of the dictionary.
Dictionary[key] = Dictionary[key] - 1;
}
Here is a blog post about this solution.
And for a deep dive in StackOverflow: Why this error occurs?

Okay so what helped me was iterating backwards. I was trying to remove an entry from a list but iterating upwards and it screwed up the loop because the entry didn't exist anymore:
for (int x = myList.Count - 1; x > -1; x--)
{
myList.RemoveAt(x);
}

The accepted answer is imprecise and incorrect in the worst case . If changes are made during ToList(), you can still end up with an error. Besides lock, which performance and thread-safety needs to be taken into consideration if you have a public member, a proper solution can be using immutable types.
In general, an immutable type means that you can't change the state of it once created.
So your code should look like:
public class SubscriptionServer : ISubscriptionServer
{
private static ImmutableDictionary<Guid, Subscriber> subscribers = ImmutableDictionary<Guid, Subscriber>.Empty;
public void SubscribeEvent(string id)
{
subscribers = subscribers.Add(Guid.NewGuid(), new Subscriber());
}
public void NotifyEvent()
{
foreach(var sub in subscribers.Values)
{
//.....This is always safe
}
}
//.........
}
This can be especially useful if you have a public member. Other classes can always foreach on the immutable types without worrying about the collection being modified.

I want to point out other case not reflected in any of the answers. I have a Dictionary<Tkey,TValue> shared in a multi threaded app, which uses a ReaderWriterLockSlim to protect the read and write operations. This is a reading method that throws the exception:
public IEnumerable<Data> GetInfo()
{
List<Data> info = null;
_cacheLock.EnterReadLock();
try
{
info = _cache.Values.SelectMany(ce => ce.Data); // Ad .Tolist() to avoid exc.
}
finally
{
_cacheLock.ExitReadLock();
}
return info;
}
In general, it works fine, but from time to time I get the exception. The problem is a subtlety of LINQ: this code returns an IEnumerable<Info>, which is still not enumerated after leaving the section protected by the lock. So, it can be changed by other threads before being enumerated, leading to the exception. The solution is to force the enumeration, for example with .ToList() as shown in the comment. In this way, the enumerable is already enumerated before leaving the protected section.
So, if using LINQ in a multi-threaded application, be aware to always materialize the queries before leaving the protected regions.

InvalidOperationException-
An InvalidOperationException has occurred. It reports a "collection was modified" in a foreach-loop
Use break statement, Once the object is removed.
ex:
ArrayList list = new ArrayList();
foreach (var item in list)
{
if(condition)
{
list.remove(item);
break;
}
}

Actually the problem seems to me that you are removing elements from the list and expecting to continue to read the list as if nothing had happened.
What you really need to do is to start from the end and back to the begining. Even if you remove elements from the list you will be able to continue reading it.

I had the same issue, and it was solved when I used a for loop instead of foreach.
// foreach (var item in itemsToBeLast)
for (int i = 0; i < itemsToBeLast.Count; i++)
{
var matchingItem = itemsToBeLast.FirstOrDefault(item => item.Detach);
if (matchingItem != null)
{
itemsToBeLast.Remove(matchingItem);
continue;
}
allItems.Add(itemsToBeLast[i]);// (attachDetachItem);
}

I've seen many options for this but to me this one was the best.
ListItemCollection collection = new ListItemCollection();
foreach (ListItem item in ListBox1.Items)
{
if (item.Selected)
collection.Add(item);
}
Then simply loop through the collection.
Be aware that a ListItemCollection can contain duplicates. By default there is nothing preventing duplicates being added to the collection. To avoid duplicates you can do this:
ListItemCollection collection = new ListItemCollection();
foreach (ListItem item in ListBox1.Items)
{
if (item.Selected && !collection.Contains(item))
collection.Add(item);
}

This way should cover a situation of concurrency when the function is called again while is still executing (and items need used only once):
while (list.Count > 0)
{
string Item = list[0];
list.RemoveAt(0);
// do here what you need to do with item
}
If the function get called while is still executing items will not reiterate from the first again as they get deleted as soon as they get used.
Should not affect performance much for small lists.

Here is a specific scenario that warrants a specialized approach:
The Dictionary is enumerated frequently.
The Dictionary is modified infrequently.
In this scenario creating a copy of the Dictionary (or the Dictionary.Values) before every enumeration can be quite costly. My idea about solving this problem is to reuse the same cached copy in multiple enumerations, and watch an IEnumerator of the original Dictionary for exceptions. The enumerator will be cached along with the copied data, and interrogated before starting a new enumeration. In case of an exception the cached copy will be discarded, and a new one will be created. Here is my implementation of this idea:
using System;
using System.Collections;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Linq;
public class EnumerableSnapshot<T> : IEnumerable<T>, IDisposable
{
private IEnumerable<T> _source;
private IEnumerator<T> _enumerator;
private ReadOnlyCollection<T> _cached;
public EnumerableSnapshot(IEnumerable<T> source)
{
_source = source ?? throw new ArgumentNullException(nameof(source));
}
public IEnumerator<T> GetEnumerator()
{
if (_source == null) throw new ObjectDisposedException(this.GetType().Name);
if (_enumerator == null)
{
_enumerator = _source.GetEnumerator();
_cached = new ReadOnlyCollection<T>(_source.ToArray());
}
else
{
var modified = false;
if (_source is ICollection collection) // C# 7 syntax
{
modified = _cached.Count != collection.Count;
}
if (!modified)
{
try
{
_enumerator.MoveNext();
}
catch (InvalidOperationException)
{
modified = true;
}
}
if (modified)
{
_enumerator.Dispose();
_enumerator = _source.GetEnumerator();
_cached = new ReadOnlyCollection<T>(_source.ToArray());
}
}
return _cached.GetEnumerator();
}
public void Dispose()
{
_enumerator?.Dispose();
_enumerator = null;
_cached = null;
_source = null;
}
IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}
public static class EnumerableSnapshotExtensions
{
public static EnumerableSnapshot<T> ToEnumerableSnapshot<T>(
this IEnumerable<T> source) => new EnumerableSnapshot<T>(source);
}
Usage example:
private static IDictionary<Guid, Subscriber> _subscribers;
private static EnumerableSnapshot<Subscriber> _subscribersSnapshot;
//...(in the constructor)
_subscribers = new Dictionary<Guid, Subscriber>();
_subscribersSnapshot = _subscribers.Values.ToEnumerableSnapshot();
// ...(elsewere)
foreach (var subscriber in _subscribersSnapshot)
{
//...
}
Unfortunately this idea cannot be used currently with the class Dictionary in .NET Core 3.0, because this class does not throw a Collection was modified exception when enumerated and the methods Remove and Clear are invoked. All other containers I checked are behaving consistently. I checked systematically these classes:
List<T>, Collection<T>, ObservableCollection<T>, HashSet<T>, SortedSet<T>, Dictionary<T,V> and SortedDictionary<T,V>. Only the two aforementioned methods of the Dictionary class in .NET Core are not invalidating the enumeration.
Update: I fixed the above problem by comparing also the lengths of the cached and the original collection. This fix assumes that the dictionary will be passed directly as an argument to the EnumerableSnapshot's constructor, and its identity will not be hidden by (for example) a projection like: dictionary.Select(e => e).ΤοEnumerableSnapshot().
Important: The above class is not thread safe. It is intended to be used from code running exclusively in a single thread.

You can copy subscribers dictionary object to a same type of temporary dictionary object and then iterate the temporary dictionary object using foreach loop.

So a different way to solve this problem would be instead of removing the elements create a new dictionary and only add the elements you didnt want to remove then replace the original dictionary with the new one. I don't think this is too much of an efficiency problem because it does not increase the number of times you iterate over the structure.

checking list of monsters error Unity [duplicate]

I can't get to the bottom of this error, because when the debugger is attached, it does not seem to occur.
Collection was modified; enumeration operation may not execute
Below is the code.
This is a WCF server in a Windows service. The method NotifySubscribers() is called by the service whenever there is a data event (at random intervals, but not very often - about 800 times per day).
When a Windows Forms client subscribes, the subscriber ID is added to the subscribers dictionary, and when the client unsubscribes, it is deleted from the dictionary. The error happens when (or after) a client unsubscribes. It appears that the next time the NotifySubscribers() method is called, the foreach() loop fails with the error in the subject line. The method writes the error into the application log as shown in the code below. When a debugger is attached and a client unsubscribes, the code executes fine.
Do you see a problem with this code? Do I need to make the dictionary thread-safe?
[ServiceBehavior(InstanceContextMode=InstanceContextMode.Single)]
public class SubscriptionServer : ISubscriptionServer
{
private static IDictionary<Guid, Subscriber> subscribers;
public SubscriptionServer()
{
subscribers = new Dictionary<Guid, Subscriber>();
}
public void NotifySubscribers(DataRecord sr)
{
foreach(Subscriber s in subscribers.Values)
{
try
{
s.Callback.SignalData(sr);
}
catch (Exception e)
{
DCS.WriteToApplicationLog(e.Message,
System.Diagnostics.EventLogEntryType.Error);
UnsubscribeEvent(s.ClientId);
}
}
}
public Guid SubscribeEvent(string clientDescription)
{
Subscriber subscriber = new Subscriber();
subscriber.Callback = OperationContext.Current.
GetCallbackChannel<IDCSCallback>();
subscribers.Add(subscriber.ClientId, subscriber);
return subscriber.ClientId;
}
public void UnsubscribeEvent(Guid clientId)
{
try
{
subscribers.Remove(clientId);
}
catch(Exception e)
{
System.Diagnostics.Debug.WriteLine("Unsubscribe Error " +
e.Message);
}
}
}

What's likely happening is that SignalData is indirectly changing the subscribers dictionary under the hood during the loop and leading to that message. You can verify this by changing
foreach(Subscriber s in subscribers.Values)
To
foreach(Subscriber s in subscribers.Values.ToList())
If I'm right, the problem will disappear.
Calling subscribers.Values.ToList() copies the values of subscribers.Values to a separate list at the start of the foreach. Nothing else has access to this list (it doesn't even have a variable name!), so nothing can modify it inside the loop.

When a subscriber unsubscribes you are changing contents of the collection of Subscribers during enumeration.
There are several ways to fix this, one being changing the for loop to use an explicit .ToList():
public void NotifySubscribers(DataRecord sr)
{
foreach(Subscriber s in subscribers.Values.ToList())
{
^^^^^^^^^
...

A more efficient way, in my opinion, is to have another list that you declare that you put anything that is "to be removed" into. Then after you finish your main loop (without the .ToList()), you do another loop over the "to be removed" list, removing each entry as it happens. So in your class you add:
private List<Guid> toBeRemoved = new List<Guid>();
Then you change it to:
public void NotifySubscribers(DataRecord sr)
{
toBeRemoved.Clear();
...your unchanged code skipped...
foreach ( Guid clientId in toBeRemoved )
{
try
{
subscribers.Remove(clientId);
}
catch(Exception e)
{
System.Diagnostics.Debug.WriteLine("Unsubscribe Error " +
e.Message);
}
}
}
...your unchanged code skipped...
public void UnsubscribeEvent(Guid clientId)
{
toBeRemoved.Add( clientId );
}
This will not only solve your problem, it will prevent you from having to keep creating a list from your dictionary, which is expensive if there are a lot of subscribers in there. Assuming the list of subscribers to be removed on any given iteration is lower than the total number in the list, this should be faster. But of course feel free to profile it to be sure that's the case if there's any doubt in your specific usage situation.

Why this error?
In general .Net collections do not support being enumerated and modified at the same time. If you try to modify the collection list during enumeration, it raises an exception. So the issue behind this error is, we can not modify the list/dictionary while we are looping through the same.
One of the solutions
If we iterate a dictionary using a list of its keys, in parallel we can modify the dictionary object, as we are iterating through the key-collection and
not the dictionary(and iterating its key collection).
Example
//get key collection from dictionary into a list to loop through
List<int> keys = new List<int>(Dictionary.Keys);
// iterating key collection using a simple for-each loop
foreach (int key in keys)
{
// Now we can perform any modification with values of the dictionary.
Dictionary[key] = Dictionary[key] - 1;
}
Here is a blog post about this solution.
And for a deep dive in StackOverflow: Why this error occurs?

Okay so what helped me was iterating backwards. I was trying to remove an entry from a list but iterating upwards and it screwed up the loop because the entry didn't exist anymore:
for (int x = myList.Count - 1; x > -1; x--)
{
myList.RemoveAt(x);
}

The accepted answer is imprecise and incorrect in the worst case . If changes are made during ToList(), you can still end up with an error. Besides lock, which performance and thread-safety needs to be taken into consideration if you have a public member, a proper solution can be using immutable types.
In general, an immutable type means that you can't change the state of it once created.
So your code should look like:
public class SubscriptionServer : ISubscriptionServer
{
private static ImmutableDictionary<Guid, Subscriber> subscribers = ImmutableDictionary<Guid, Subscriber>.Empty;
public void SubscribeEvent(string id)
{
subscribers = subscribers.Add(Guid.NewGuid(), new Subscriber());
}
public void NotifyEvent()
{
foreach(var sub in subscribers.Values)
{
//.....This is always safe
}
}
//.........
}
This can be especially useful if you have a public member. Other classes can always foreach on the immutable types without worrying about the collection being modified.

I want to point out other case not reflected in any of the answers. I have a Dictionary<Tkey,TValue> shared in a multi threaded app, which uses a ReaderWriterLockSlim to protect the read and write operations. This is a reading method that throws the exception:
public IEnumerable<Data> GetInfo()
{
List<Data> info = null;
_cacheLock.EnterReadLock();
try
{
info = _cache.Values.SelectMany(ce => ce.Data); // Ad .Tolist() to avoid exc.
}
finally
{
_cacheLock.ExitReadLock();
}
return info;
}
In general, it works fine, but from time to time I get the exception. The problem is a subtlety of LINQ: this code returns an IEnumerable<Info>, which is still not enumerated after leaving the section protected by the lock. So, it can be changed by other threads before being enumerated, leading to the exception. The solution is to force the enumeration, for example with .ToList() as shown in the comment. In this way, the enumerable is already enumerated before leaving the protected section.
So, if using LINQ in a multi-threaded application, be aware to always materialize the queries before leaving the protected regions.

InvalidOperationException-
An InvalidOperationException has occurred. It reports a "collection was modified" in a foreach-loop
Use break statement, Once the object is removed.
ex:
ArrayList list = new ArrayList();
foreach (var item in list)
{
if(condition)
{
list.remove(item);
break;
}
}

Actually the problem seems to me that you are removing elements from the list and expecting to continue to read the list as if nothing had happened.
What you really need to do is to start from the end and back to the begining. Even if you remove elements from the list you will be able to continue reading it.

I had the same issue, and it was solved when I used a for loop instead of foreach.
// foreach (var item in itemsToBeLast)
for (int i = 0; i < itemsToBeLast.Count; i++)
{
var matchingItem = itemsToBeLast.FirstOrDefault(item => item.Detach);
if (matchingItem != null)
{
itemsToBeLast.Remove(matchingItem);
continue;
}
allItems.Add(itemsToBeLast[i]);// (attachDetachItem);
}

I've seen many options for this but to me this one was the best.
ListItemCollection collection = new ListItemCollection();
foreach (ListItem item in ListBox1.Items)
{
if (item.Selected)
collection.Add(item);
}
Then simply loop through the collection.
Be aware that a ListItemCollection can contain duplicates. By default there is nothing preventing duplicates being added to the collection. To avoid duplicates you can do this:
ListItemCollection collection = new ListItemCollection();
foreach (ListItem item in ListBox1.Items)
{
if (item.Selected && !collection.Contains(item))
collection.Add(item);
}

This way should cover a situation of concurrency when the function is called again while is still executing (and items need used only once):
while (list.Count > 0)
{
string Item = list[0];
list.RemoveAt(0);
// do here what you need to do with item
}
If the function get called while is still executing items will not reiterate from the first again as they get deleted as soon as they get used.
Should not affect performance much for small lists.

Here is a specific scenario that warrants a specialized approach:
The Dictionary is enumerated frequently.
The Dictionary is modified infrequently.
In this scenario creating a copy of the Dictionary (or the Dictionary.Values) before every enumeration can be quite costly. My idea about solving this problem is to reuse the same cached copy in multiple enumerations, and watch an IEnumerator of the original Dictionary for exceptions. The enumerator will be cached along with the copied data, and interrogated before starting a new enumeration. In case of an exception the cached copy will be discarded, and a new one will be created. Here is my implementation of this idea:
using System;
using System.Collections;
using System.Collections.Generic;
using System.Collections.ObjectModel;
using System.Linq;
public class EnumerableSnapshot<T> : IEnumerable<T>, IDisposable
{
private IEnumerable<T> _source;
private IEnumerator<T> _enumerator;
private ReadOnlyCollection<T> _cached;
public EnumerableSnapshot(IEnumerable<T> source)
{
_source = source ?? throw new ArgumentNullException(nameof(source));
}
public IEnumerator<T> GetEnumerator()
{
if (_source == null) throw new ObjectDisposedException(this.GetType().Name);
if (_enumerator == null)
{
_enumerator = _source.GetEnumerator();
_cached = new ReadOnlyCollection<T>(_source.ToArray());
}
else
{
var modified = false;
if (_source is ICollection collection) // C# 7 syntax
{
modified = _cached.Count != collection.Count;
}
if (!modified)
{
try
{
_enumerator.MoveNext();
}
catch (InvalidOperationException)
{
modified = true;
}
}
if (modified)
{
_enumerator.Dispose();
_enumerator = _source.GetEnumerator();
_cached = new ReadOnlyCollection<T>(_source.ToArray());
}
}
return _cached.GetEnumerator();
}
public void Dispose()
{
_enumerator?.Dispose();
_enumerator = null;
_cached = null;
_source = null;
}
IEnumerator IEnumerable.GetEnumerator() => GetEnumerator();
}
public static class EnumerableSnapshotExtensions
{
public static EnumerableSnapshot<T> ToEnumerableSnapshot<T>(
this IEnumerable<T> source) => new EnumerableSnapshot<T>(source);
}
Usage example:
private static IDictionary<Guid, Subscriber> _subscribers;
private static EnumerableSnapshot<Subscriber> _subscribersSnapshot;
//...(in the constructor)
_subscribers = new Dictionary<Guid, Subscriber>();
_subscribersSnapshot = _subscribers.Values.ToEnumerableSnapshot();
// ...(elsewere)
foreach (var subscriber in _subscribersSnapshot)
{
//...
}
Unfortunately this idea cannot be used currently with the class Dictionary in .NET Core 3.0, because this class does not throw a Collection was modified exception when enumerated and the methods Remove and Clear are invoked. All other containers I checked are behaving consistently. I checked systematically these classes:
List<T>, Collection<T>, ObservableCollection<T>, HashSet<T>, SortedSet<T>, Dictionary<T,V> and SortedDictionary<T,V>. Only the two aforementioned methods of the Dictionary class in .NET Core are not invalidating the enumeration.
Update: I fixed the above problem by comparing also the lengths of the cached and the original collection. This fix assumes that the dictionary will be passed directly as an argument to the EnumerableSnapshot's constructor, and its identity will not be hidden by (for example) a projection like: dictionary.Select(e => e).ΤοEnumerableSnapshot().
Important: The above class is not thread safe. It is intended to be used from code running exclusively in a single thread.

You can copy subscribers dictionary object to a same type of temporary dictionary object and then iterate the temporary dictionary object using foreach loop.

So a different way to solve this problem would be instead of removing the elements create a new dictionary and only add the elements you didnt want to remove then replace the original dictionary with the new one. I don't think this is too much of an efficiency problem because it does not increase the number of times you iterate over the structure.

ForEach loop for a Generic List repeated endlessly

After Deserializing a file with just one record
It seems that it's in an infinitive loop
IndexSeries = (List<string>)bFormatter.Deserialize(fsSeriesIndexGet);
IndexSeries.ForEach(name => AddSerie(name));
//IndexSeries.ForEach(delegate(String name)
//{
// AddSerie(name);
//});
AddSerie will be executed infinitively !

You use ambiguous terms. Firstly you mention an infinite loop, and then mention that AddSerie will be executed 'infinitively' [sic]; based on this, I would think that the issue you're bringing up is not with ForEach going on and on forever (as implied/stated), but instead that AddSerie does something once that seems to be taking forever.
This could even amount to something mentioned by Joey: if you're adding an element to a list while within the context of a ForEach call, then you're always one step behind in completion, and hence won't 'complete'. However, getting an OutOfMemoryException would actually occur relatively quickly if, say, AddSerie does nothing but that - it might take longer to get to such a point if AddSerie is a relatively time-consuming method. Then again, you might never get such an exception (in the context discussed) if AddSerie simply takes a dogs age to complete without contributing to the length of the list.
Showing your AddSerie code would be potentially most helpful in determining the actual issue.

If I define:
//class level declaration (in a console app)
static List<string> strings;
static void Loop(string s)
{
Console.WriteLine(s);
strings.Add(s + "!");
}
Then
static void Main(string[] args)
{
strings = new List<string> { "sample" };
strings.ForEach(s => Console.WriteLine(s));
}
executes normally, outputing a single string, while
static void Main(string[] args)
{
strings = new List<string> { "sample" };
strings.ForEach(s => Loop(s));
}
loops indefinitely, adding '!'s in the process, and
static void Main(string[] args)
{
strings = new List<string> { "sample" };
foreach (string s in strings)
{
Loop(s);
}
}
throws an InvalidOperationException (Collection was modified; enumeration operation may not execute), which, in my opinion is the correct behavior. Why the List.ForEach method allows the list to be changed by the action, I do not know, but would like to find out :)

Large static arrays are slowing down class load, need a better/faster lookup method

I have a class with a couple static arrays:
an int[] with 17,720 elements
a string[] with 17,720 elements
I noticed when I first access this class it takes almost 2 seconds to initialize, which causes a pause in the GUI that's accessing it.
Specifically, it's a lookup for Unicode character names. The first array is an index into the second array.
static readonly int[] NAME_INDEX = {
0x0000, 0x0001, 0x0005, 0x002C, 0x003B, ...
static readonly string[] NAMES = {
"Exclamation Mark", "Digit Three", "Semicolon", "Question Mark", ...
The following code is how the arrays are used (given a character code). [Note: This code isn't a performance problem]
int nameIndex = Array.BinarySearch<int>(NAME_INDEX, code);
if (nameIndex > 0)
{
return NAMES[nameIndex];
}
I guess I'm looking at other options on how to structure the data so that 1) The class is quickly loaded, and 2) I can quickly get the "name" for a given character code.
Should I not be storing all these thousands of elements in static arrays?
Update
Thanks for all the suggestions. I've tested out a Dictionary approach and the performance of adding all the entries seems to be really poor.
Here is some code with the Unicode data to test out Arrays vs Dictionaries
http://drop.io/fontspace/asset/fontspace-unicodesupport-zip
Solution Update
I tested out my original dual arrays (which are faster than both dictionary options) with a background thread to initialize and that helped performance a bit.
However, the real surprise is how well the binary files in resource streams works. It is the fastest solution discussed in this thread. Thanks everyone for your answers!

So a couple of observations. Binary Search is only going to work if your array is sorted, and from your above code snippet, it doesn't look to be sorted.
Since your primary goal is to find a specific name, your code is begging for a hash table. I would suggest using a Dictionary, it will give you O(1) (on average) lookup, without much more overhead than just having the arrays.
As for the load time, I agree with Andrey that the best way is going to be by using a separate thread. You are going to have some initialization overhead when using the amount of data you are using. Normal practice with GUIs is to use a separate thread for these activites so you don't lock up the UI.

First
A Dictionary<int, string> is going to perform far better than your duelling arrays will. Putting aside how this data gets into the arrays/Dictionary (hardcoded vs. read in from another location, like a resource file), this is still a better and more intuitive storage mechanism
Second
As others have suggested, do your loading in another thread. I'd use a helper function to help you deal with this. You could use an approach like this:
public class YourClass
{
private static Dictionary<int, string> characterLookup;
private static ManualResetEvent lookupCreated;
static YourClass()
{
lookupCreated = new ManualResetEvent(false);
ThreadPool.QueueUserWorkItem(LoadLookup);
}
static void LoadLookup(object garbage)
{
// add your pairs by calling characterLookup.Add(...)
lookupCreated.Set();
}
public static string GetDescription(int code)
{
if (lookupCreated != null)
{
lookupCreated.WaitOne();
lookupCreated.Close();
lookupCreated = null;
}
string output;
if(!characterLookup.TryGetValue(code, out output)) output = null;
return output;
}
}
In your code, call GetDescription in order to translate your integer into the corresponding string. If the UI doesn't call this until later, then you should see a marked decrease in startup time. To be safe, though, I've included a ManualResetEvent that will cause any calls to GetDescription to block until the dictionary has been fully loaded.

"Should I not be storing all these thousands of elements in static arrays?"
A much better way would be to store your data as binary stream in resources in the assembly and then load from the resources. Will be some more programming overhead but therefore doesn't need any object initialization.
Basic idea would be (no real code):
// Load data (two streams):
indices = ResourceManager.GetStream ("indexData");
strings = ResourceManager.GetStream ("stringData");
// Retrieving an entry:
stringIndex = indices.GetIndexAtPosition (char);
string = strings.GetStringFromPosition (stringIndex);
If you want a really good solution (for even some more work) look into using memmapped data files.

Initialize your arrays in separate thread that will not lock the UI
http://msdn.microsoft.com/en-us/library/hz49h034.aspx

if you store the arrays in a file you could do a lazy load
public class Class1
{
const int CountOfEntries = 17700; //or what ever the count is
IEnumerable<KeyValuePair<int, string>> load()
{
using (var reader = File.OpenText("somefile"))
{
while (!reader.EndOfStream)
{
var line = reader.ReadLine();
var pair = line.Split(',');
yield return new KeyValuePair<int, string>(int.Parse(pair[0]), pair[1]);
}
}
}
private static Dictionary<int, string> _lookup = new Dictionary<int, string>();
private static IEnumerator<KeyValuePair<int, string>> _loader = null;
private string LookUp(int index)
{
if (_lookup.Count < CountOfEntries && !_lookup.ContainsKey(index))
{
if(_loader == null)
{
_loader = load().GetEnumerator();
}
while(_loader.MoveNext())
{
var pair = _loader.Current;
_lookup.Add(pair.Key,pair.Value);
if (pair.Key == index)
{
return index;
}
}
}
string name;
if (_lookup.TryGetValue(index,out name))
{
return return name;
}
throw new KeyNotFoundException("The given index was not found");
}
}
the code expectes the file to have one pair on each line like so:
index0,name0
index1,name1
If the first index sought is at the end this will perform slower probably (due to IO mainly) but if the access is random the average case woul be reading half of the values the first time if the access is not random make sure to keep the most used in the top of the file
there are a few more issues to considere. The above code is not threadsafe for the load operation and to increase responsiveness of the rest of the code keep the loading in a background thread
hope this helps

What about using a dictionary instead of two arrays? You could initialize the dictionary asynchronously using a thread or thread pool. The lookup would be O(1) instead of O(log(n)) as well.
public static class Lookup
{
private static readonly ManualResetEvent m_Initialized = new ManualResetEvent(false);
private static readonly Dictionary<int, string> m_Dictionary = new Dictionary<int, string>();
public static Lookup()
{
// Start an asynchronous operation to intialize the dictionary.
// You could use ThreadPool.QueueUserWorkItem instead of creating a new thread.
Thread thread = new Thread(() => { Initialize(); });
thread.Start();
}
public static string Lookup(int code)
{
m_Initialized.WaitOne();
lock (m_Dictionary)
{
return m_Dictionary[code];
}
}
private static void Initialize()
{
lock (m_Dictionary)
{
m_Dictionary.Add(0x0000, "Exclamation Point");
// Keep adding items to the dictionary here.
}
m_Initialized.Set();
}
}

We Keep Coding

C# (C-Sharp) is a programming language developed by Microsoft that runs on the .NET Framework.