I don't know why I have an IndexOutOfRangeException when I am clearing a System.Collections.Generic.List<T>. Does this make sense?
List<MyObject> listOfMyObject = new List<MyObject>();
listOfMyObject.Clear();
This typically happens if multiple threads are accessing the list simultaneously. If one thread deletes an element while another calls Clear(), this exception can occur.
The "answer" in this case is to synchronize this appropriately, locking around all of your List access.
Edit:
In order to handle this, the simplest method is to encapsulate your list within a custom class, and expose the methods you need, but lock as needed. You'll need to add locking to anything that alters the collection.
This would be a simple option:
public class MyClassCollection
{
// Private object for locking
private readonly object syncObject = new object();
private readonly List<MyObject> list = new List<MyObject>();
public this[int index]
{
get { return list[index]; }
set
{
lock(syncObject) {
list[index] = value;
}
}
}
public void Add(MyObject value)
{
lock(syncObject) {
list.Add(value);
}
}
public void Clear()
{
lock(syncObject) {
list.Clear();
}
}
// Do any other methods you need, such as remove, etc.
// Also, you can make this class implement IList<MyObject>
// or IEnumerable<MyObject>, but make sure to lock each
// of the methods appropriately, in particular, any method
// that can change the collection needs locking
}
Are you sure that that code throws an exception? I have
using System.Collections.Generic;
class MyObject { }
class Program {
static void Main(string[] args) {
List<MyObject> listOfMyObject = new List<MyObject>();
listOfMyObject.Clear();
}
}
and I do not get an exception.
Is your real-life example more complex? Perhaps you have multiple threads simultaneously accessing the list? Can we see a stack trace?
List<T>.Clear is really quite simple. Using Reflector:
public void Clear() {
if (this._size > 0) {
Array.Clear(this._items, 0, this._size);
this._size = 0;
}
this._version++;
}
In the case when the list already empty, that is not going to ever throw an exception. However, if you are modifying the list on another thread, Array.Clear could throw an IndexOutOfRangeException exception. So if another thread removes an item from the list then this._size (the number of items to clear) will be too big.
The documentation doesn't mention any Exception this method throws, your problem is probably elsewhere.
List<T>.Clear
Related
private static readonly object MyMethodLockobject = new object();
private static readonly HashSet<long> ActiveWorkItem = new HashSet<long>();
public async Task MyMethod(long id)
{
lock (MyMethodLockobject)
{
if (ActiveWorkItem.Contains(id))
{
throw new AnotherRequestAlreadyInProgressException();
}
ActiveWorkItem.Add(id);
}
try
{
return await DoWork(id);
}
finally
{
ActiveWorkItem.Remove(id);
}
}
ActiveWorkItem purpose is preventing concurrent calls on same id. only add contain and remove are needed.
MyMethod is the only place with ActiveWorkItem access.
my concern is this line:
finally
{
ActiveWorkItem.Remove(id);
}
or is is necessary to change to
finally
{
lock (MyMethodLockobject)
{
ActiveWorkItem.Remove(id);
}
}
better alternative also appreciated
Yes, it is necessary to protect the HashSet inside the finally block too.
finally
{
lock (MyMethodLockobject) ActiveWorkItem.Remove(id);
}
Otherwise one thread may be adding items to the HashSet while multiple threads are concurrently removing items from the same HashSet, potentially putting the object into a corrupted state, and resulting to undefined behavior.
I use ConcurrentDictionary to collect data in memory in web api application. Using api methods I add and update objects in ConcurrentDictionary. And there is background thread which analyze and clean up this dictionary based on object properties. Now I'm considering two approaches:
1. use lock on dictionary item in updateValueFactory in AddOrUpdate method, but question is how to read properties properly to be sure I have the latest version of it and that I'm not reading property in not stable state.
public class ThreadsafeService2
{
private readonly ConcurrentDictionary<string, ThreadSafeItem2> _storage =
new ConcurrentDictionary<string, ThreadSafeItem2>();
public void AddOrUpdate(string name)
{
var newVal = new ThreadSafeItem2();
_storage.AddOrUpdate(name, newVal, (key, oldVal) =>
{
//use lock
lock (oldVal)
{
oldVal.Increment();
}
return oldVal;
});
}
public void Analyze()
{
foreach (var key in _storage.Keys)
{
if (_storage.TryGetValue(key, out var item))
{
//how to read it properly?
long ticks = item.ModifiedTicks;
}
}
}
}
public class ThreadSafeItem2
{
private long _modifiedTicks;
private int _counter;
public void Increment()
{
//no interlocked here
_modifiedTicks = DateTime.Now.Ticks;
_counter++;
}
//now interlocked here
public long ModifiedTicks => _modifiedTicks;
public int Counter => _counter;
}
2. use Interlocked and memory barriers on property level without lock, looks a bit verbose for me.
public class ThreadsafeService1
{
private readonly ConcurrentDictionary<string, ThreadSafeItem1> _storage =
new ConcurrentDictionary<string, ThreadSafeItem1>();
public void AddOrUpdate(string name)
{
var newVal = new ThreadSafeItem1();
_storage.AddOrUpdate(name, newVal, (key, oldVal) =>
{
//no lock here
oldVal.Increment();
return oldVal;
});
}
public void Analyze()
{
foreach(var key in _storage.Keys)
{
if(_storage.TryGetValue(key, out var item))
{
//reading through interloacked
long ticks = item.ModifiedTicks;
}
}
}
}
public class ThreadSafeItem1
{
private long _modifiedTicks;
private int _counter;
public void Increment()
{
//make changes in atomic manner
Interlocked.Exchange(ref _modifiedTicks, DateTime.Now.Ticks);
Interlocked.Increment(ref _counter);
}
public long ModifiedTicks => Interlocked.Read(ref _modifiedTicks);
public int Counter => Thread.VolatileRead(ref _counter);
}
What is the best practices here?
So both of your implementations have major problems. The first solution locks when incrementing, but doesn't lock when reading, meaning the other places accessing the data can read invalid state.
A non-technical problem, but a major issue nonetheless, is that you've named your class ThreadSaveItem and yet it's not actually designed to be accessed safely from multiple threads. It's the callers responsibility, in this implementation, to ensure that the item isn't accessed from multiple threads. If I see a class called ThreadSafeItem I'm going to assume it's safe to access it from multiple threads, and that I don't need to synchronize my access to it so long as each operation I perform is the only thing that needs to be logically atomic.
Your Interlocked solution is problematic in that you have to fields that you're modifying, that are conceptually tied together, but you don't synchronize their changes together, meaning someone can observe a modification to one and not the other, which is a problem for that code.
Next, your use of AddOrUpdate in both solutions isn't really appropriate. The whole point of the method call is to add an item or replace it with another item, not to mutate the provided item (that's why it takes a return value; you're supposed to produce a new item). If you want to go with the approach of getting a mutable item and mutating it, the way to go would be to call GetOrAdd to either get an existing item or create a new one, and then to mutate it in a thread safe manner using the returned value.
The whole solution is radically simplified by simply making ThreadSafeItem immutable. It lets you use AddOrUpdate on the ConcurrentDictionary for the update, and it means that the only synchronization that needs to be done is the updating of the value of the ConcurrentDictionary, and it already handles synchronization of its own state, no synchronization needs to be done at all when accessing ThreadSafeItem, because all access to the data is inherently thread safe because it's immutable. This means that you never actually need to write any synchronization code at all, which is exactly what you want to strive for whenever possible.
And finally, we have the actual code:
public class ThreadsafeService3
{
private readonly ConcurrentDictionary<string, ThreadSafeItem3> _storage =
new ConcurrentDictionary<string, ThreadSafeItem3>();
public void AddOrUpdate(string name)
{
_storage.AddOrUpdate(name, _ => new ThreadSafeItem3(), (_, oldValue) => oldValue.Increment());
}
public void Analyze()
{
foreach (var pair in _storage)
{
long ticks = pair.Value.ModifiedTicks;
//Note, the value may have been updated since we checked;
//you've said you don't care and it's okay for a newer item to be removed here if it loses the race.
if (isTooOld(ticks))
_storage.TryRemove(pair.Key, out _);
}
}
}
public class ThreadSafeItem3
{
public ThreadSafeItem3()
{
Counter = 0;
}
private ThreadSafeItem3(int counter)
{
Counter = counter;
}
public ThreadSafeItem3 Increment()
{
return new ThreadSafeItem3(Counter + 1);
}
public long ModifiedTicks { get; } = DateTime.Now.Ticks;
public int Counter { get; }
}
The solution proposed by Servy (using an immutable Item type) is probably the best solution for your scenario. I would also suggest switching from class to readonly struct for reducing the allocations, although the ConcurrentDictionary is probably going to wrap the struct in a reference-type Node internally, so you might not gain anything from this.
For the sake of completeness I will propose an alternative solution, which is to use the GetOrAdd instead of the AddOrUpdate, and lock on the Item whenever you are doing anything with it:
public class Item // Mutable and thread-unsafe
{
public long ModifiedTicks { get; private set; }
public int Counter { get; private set; }
public void Increment()
{
ModifiedTicks = DateTime.Now.Ticks;
Counter++;
}
}
public class Service
{
private readonly ConcurrentDictionary<string, Item> _storage = new();
public void AddOrUpdate(string name)
{
Item item = _storage.GetOrAdd(name, _ => new());
lock (item) item.Increment(); // Dont't forget to lock!
}
public void Analyze()
{
foreach (var (key, item) in _storage.ToArray())
{
lock (item) // Dont't forget to lock!
{
long ticks = item.ModifiedTicks;
}
}
}
}
This solution offers probably the best performance, but the burden of remembering to lock correctly everywhere cannot be underestimated.
I can't comment on the specifics of what exactly you are doing, but interlock and Concurrent dictionary is better than locks you do yourself.
I would question this approach though. Your data is important enough, but not so important to persist it? Depending on the usage of the application this approach will slow it down by some degree. Again, not knowing exactly what you are doing, you could throw each "Add" into an MSMQ, and then have an external exe run at some schedule to process the items. The website will just fire and forget, with no threading requirements.
I have the following custom observable collection (The code is taken in parts from Dean Chalk's blog http://www.deanchalk.me.uk/post/Thread-Safe-Dispatcher-Safe-Observable-Collection-for-WPF.aspx and slightly altered):
public class ThreadSaveObservableCollection <T> : IList<T>, INotifyCollectionChanged {
private IList<T> collection;
private Dispatcher uiDispatcher;
private ReaderWriterLock rwLock;
public ThreadSaveObservableCollection () {
collection = new List<T>();
rwLock = new ReaderWriterLock();
uiDispatcher = Dispatcher.CurrentDispatcher;
}
public void Insert (int index, T item) {
if (Thread.CurrentThread == uiDispatcher.Thread) {
insert_(index, item);
} else {
uiDispatcher.BeginInvoke(new Action<int, T>(insert_), DispatcherPriority.Normal, new object[] {index, item});
}
}
private void insert_ (int index, T item) {
rwLock.AcquireWriterLock(Timeout.Infinite);
collection.Insert(index, item);
CollectionChanged(this, new NotifyCollectionChangedEventArgs(NotifyCollectionChangedAction.Add, item));
rwLock.ReleaseWriterLock();
}
public IEnumerator<T> GetEnumerator () {
rwLock.AcquireReaderLock(Timeout.Infinite);
IEnumerator<T> enumerator = collection.GetEnumerator();
rwLock.ReleaseReaderLock();
return enumerator;
}
System.Collections.IEnumerator System.Collections.IEnumerable.GetEnumerator () {
rwLock.AcquireReaderLock(Timeout.Infinite);
IEnumerator<T> enumerator = collection.GetEnumerator();
rwLock.ReleaseReaderLock();
return enumerator;
}
public event NotifyCollectionChangedEventHandler CollectionChanged;
... // the remaining methods of the IList<> interface
}
Further I have a ViewModel which holds an instance of this class:
public class ViewModel {
private ThreadSaveObservableCollection<string> Collection {get; set;}
public ViewModel () {
Collection = new ThreadSaveObservableCollection<string>();
}
public void Insert (string item) {
Collection.Insert(0, item);
}
}
I apply data binding in code-behind because I create the corresponding WPF control (an ordinary List control) with name "LogList" dynamically:
wpfContainer.LogList.ItemsSource = viewModel.Collection;
Everything works quite fine except the fact that the order of items in the wpf list control is reversed with respect to the items in the Collection object of the ViewModel.
With the statement Collection.Insert(0, intem) I expect to add the new item at the top of the list but what I get is the same result as I would use Collection.Add(item).
When I step into the code during runtime I can verify that the items inside my Collection are in the correct order but on the surface inside the wpf list control the order is altered i.e. reversed.
What am I making wrong ?
I guess the problem must be found somewhere around the data binding because it's the 'wire' that connects my ObservableCollection with the wpf control and it seems that a correct order is getting into the wire and an incorrect is leaving it.
Maybe it has something to do with the GetEnumerator() methods of the IList interface since the ItemSource property of the wpf control is awaiting an Enumerator ?
I have no clue and I am really stuck ...
Thank you in advance for any help ...
Can you try to do this:
http://msdn.microsoft.com/en-us/library/ms653208.aspx
CollectionChanged(this, new NotifyCollectionChangedEventArgs(NotifyCollectionChangedAction.Add, new List<object>() { item }, 0));
I think that this event is the problem.
Couple notes about your code:
Naming: ThreadSafe, not ThreadSave
Race conditions: you're acquiring a lock to call .GetEnumerator. Then releasing the lock, and returning that enumerator. That is not safe, and will throw an exception at runtime if the thread conditions are right. What you should do here is create a copy of the list while under lock, then return an enumerator to that copy.
ReaderWriterLock has some known performance, scalability, and error-prone usage (e.g. re-entrance) concerns. Use ReaderWriterLocksSlim instead.
The whole idea here is to marshal all operations to the UI thread. If everything happens on the UI thread, there's no need for any locking at all.
Finally, rather than re-invent the wheel, I suggest using one of the existing thread-safe ObservableCollections.
Let's say I have a list in a class which will be used in a multi threading scenario.
public class MyClass
{
List<MyItem> _list= new List<MyItem>();
protected object SyncRoot {
get {
return ((IList)_list).SyncRoot;
}
}
public void Execute1()
{
lock(SyncRoot)
{
foreach(var item in _list) DoSomething(item);
}
}
public void Execute2()
{
Item[] list;
lock(SyncRoot)
{
list=_list.ToArray();
}
for(var i=0;i<list.Length;i++) DoSomething(list[i]);
}
}
The method Execute1 is the 'normal' way to enumerate the list in a thread-safe way. But what about Execute2? Is this approach still thread-safe?
Access to the (copy of the) List is threadsafe in both scenarios. But of course the MyItem elements are not synchronized in any way.
The second form looks a little more expensive but it will allow Add/Remove on the original while the DoSomething()s are running. The array acts like a kind of snapshot, if that matches your requirements it could be useful. Note that you might as well use ToList().
It's safe as long as every other use of _list is also protected with the same lock statement. You are taking exclusive access to the list, copying its contents and then working on the copy (to which you also have exclusive access due to scoping). A bit wasteful at first sight, but a legitimate approach under certain circumstances.
I have List with objects of strings and doubles, and I try to call different methods based on the itemtype and their value. In the debugger I can see that the first iteration works fine, but an error shows when entering the second time after a method is called.
If i comment out the methods and put in simple methods it works, so I understand that it something with how I call the methods.
What do I do wrong, and what can I do to make it work?
If there is easier ways to do what I'm trying, please let me know.
public double evaluateExpressionUsingVariableValues(List<Object> anExpression, Dictionary<String, double> variables)
{
foreach (object element in anExpression)
{
if(element.GetType()!=typeof(string))
{
setOperand((double)element);
}
else if (element.GetType() == typeof(string))
{
if (!element.ToString().StartsWith("%"))
performOperation((string)element);
else
setOperand(variables[element.ToString()]);
}
}
return this.operand;
}
If your methods (setOperand, performOperation) modify the collection at all, you will get an exception. You can't modify the collection while you are iterating over it. One method is to create a result collection and add items to it as you change them, rather than trying to modify the collection in-place.
private void Foo() {
foreach(var item in items) {
if (item.IsBad) {
DeleteItem(item); // will throw an exception as it tries to modify items
}
}
}
private void DeleteItem(Item item) {
items.Remove(item);
}
Instead, try:
private void Foo() {
List<Item> result = new List<Item>();
foreach(var item in items) {
if (!item.IsBad) {
result.Add(item); // we are adding to a collection other
// than the one we are iterating through
}
}
items = result; // we are no longer iterating, so we can modify
// this collection
}
Are you sure that none of the methods you are calling is modifying the collection (anExpression) ? This kind of problem is often the result of that. Try replacing the foreach by a for loop and see if you still get the same issue.