I have a singleton object and have a dictionary defined in it.
public class MyClass
{
public static readonly MyClass Instance = new MyClass();
private MyClass
{}
public Dictionary<int, int> MyDictionary = new Dictionary<int, int>();
}
Now, I have two System.Timers.Timer objects updating MyDictionary.
System.Timers.Timer timer1 = new System.Timers.Timer(5);
timer1.AutoReset = false;
timer1.Elapsed += new System.Timers.ElapsedEventHandler(MyTimer1Handler);
timer1.Enabled = true;
timer1.Start();
System.Timers.Timer timer2 = new System.Timers.Timer(5);
timer2.AutoReset = false;
timer2.Elapsed += new System.Timers.ElapsedEventHandler(MyTimer2Handler);
timer2.Enabled = true;
timer2.Start();
private void MyTimer1Handler(object sender, ElapsedEventArgs e)
{
MyClass.Instance.MyDictonary[1] = 100;
}
private void MyTimer1Handler(object sender, ElapsedEventArgs e)
{
MyClass.Instance.MyDictonary[2] = 100;
}
My question is now, considering the elapsed event handler of timers operate uniquely on index 1 and index 2 of MyDictionary, do I need any lock on MyDictionary ?
Yes, you have to.
http://msdn.microsoft.com/en-us/library/xfhwa508.aspx
That says reading is thread safe, but editing is not. It also says it isn't really safe to iterate the Dictionary.
If you are able to use .NET 4, you can use a ConcurrentDictionary, which is thread safe.
http://msdn.microsoft.com/en-us/library/dd287191.aspx
For this specific example that you post, yes you have to, but strictly speaking, it is not always necessary depending on your usage pattern.
For example, If you have 2 keys predetermined, then you are not modifying shared state of the dictionary if one thread operation is not affecting state of the other thread operation. For example, if you know that you are not adding/removing keys and that each thread will be accessing a specific key.
Lets consider the following simplified example where we are simply incrementing the previous value of 2 given keys in parallel:
class Program
{
static Dictionary<string, int> _dictionary = new Dictionary<string, int>();
static void Main(string[] args)
{
_dictionary["key1"] = 0;
_dictionary["key2"] = 0;
Action<string> updateEntry = (key) =>
{
for (int i = 0; i < 10000000; i++)
{
_dictionary[key] = _dictionary[key] + 1;
}
};
var task1 = Task.Factory.StartNew(() =>
{
updateEntry("key1");
});
var task2 = Task.Factory.StartNew(() =>
{
updateEntry("key2");
});
Task.WaitAll(task1, task2);
Console.WriteLine("Key1 = {0}", _dictionary["key1"]);
Console.WriteLine("Key2 = {0}", _dictionary["key2"]);
Console.ReadKey();
}
}
What do you think the value of each of the keys of the dictionary will be after iterating in 2 separate threads simultaneously on the same dictionary for more than 10 million times within a loop?
Well you get
Key1 = 10000000
Key2 = 10000000
No extra synchronization is necessary in the above example simply to assign values to existing keys in a dictionary.
Of course, if you wanted to add or remove keys then you need to consider synchronizing or using data structures such as ConcurrentDictionary<TKey,TValue>
In your case you are actually adding values to the dictionary, so you have to use some form of synchronization.
Related
I know of and have used the System.Collections.Concurrent.ConcurrentBag<T> for building thread safe code in the past. I have some legacy code that I'm trying to thread to increase performance however there is a non static List object that is being written with different sources of data. All of the writing is being done prior to any reading of the list and my initial tests show that multiple threads appear to write to the object without any issues.
Sample Windows App
Does a non static C# List object have thread safety for writing across multiple threads prior to reading? How can this be tested?
BackgroundWorker backgroundWorkerA, backgroundWorkerB;
System.Threading.ManualResetEvent manualReset;
List<string> _shardList = new List<string>(0);
public UserControl1()
{
InitializeComponent();
manualReset = new System.Threading.ManualResetEvent(false);
backgroundWorkerA = new BackgroundWorker();
backgroundWorkerA.WorkerSupportsCancellation = true;
backgroundWorkerA.DoWork += BackgroundWorkerA_DoWork;
backgroundWorkerB = new BackgroundWorker();
backgroundWorkerB.WorkerSupportsCancellation = true;
backgroundWorkerB.DoWork += BackgroundWorkerB_DoWork;
this.HandleCreated += UserControl1_HandleCreated;
}
private void UserControl1_HandleCreated(object sender, EventArgs e)
{
backgroundWorkerA.RunWorkerAsync(_shardList);
backgroundWorkerB.RunWorkerAsync(_shardList);
manualReset.Set();
}
private void BackgroundWorkerB_DoWork(object sender, DoWorkEventArgs e)
{
List<string> _shardList = (List<string>)e.Argument;
manualReset.WaitOne();
int _i = 0;
while(!this.backgroundWorkerB.CancellationPending)
{
_shardList.Add("b" + _i++.ToString());
System.Diagnostics.Debug.WriteLine("b is running");
}
thread2.Invoke(new MethodInvoker(delegate { thread2.Text = string.Join(System.Environment.NewLine, _shardList.ToArray()); }));
}
private void button1_Click(object sender, EventArgs e)
{
backgroundWorkerA.CancelAsync();
backgroundWorkerB.CancelAsync();
}
private void BackgroundWorkerA_DoWork(object sender, DoWorkEventArgs e)
{
List<string> _shardList = (List<string>)e.Argument;
manualReset.WaitOne();
int _i = 0;
while (!this.backgroundWorkerA.CancellationPending)
{
_shardList.Add("a" + _i++.ToString());
System.Diagnostics.Debug.WriteLine("a is running");
}
thread1.Invoke(new MethodInvoker(delegate { thread1.Text = string.Join(System.Environment.NewLine, _shardList.ToArray()); }));
}
There are multiple things that make concurrent writes to a List<T> unsafe.
First let's have a look at the code of the Add method:
public void Add(T item) {
if (_size == _items.Length) EnsureCapacity(_size + 1);
_items[_size++] = item;
_version++;
}
The first issue is EnsureCapacity. If the list's inner array isn't big enough to receive the new element, it will create a new, bigger array, and copy the elements from the old one to the new one. If a thread writes in the old list after the copy but before the swap, the element will be lost
The second issue is the non-atomic increment of size. If two threads try to write at the same time, they may write at the same index in the array, thus losing one item
Those race conditions are not very likely, but they'll eventually happen if you keep writing in the same list from multiple threads.
When you modify a list, it has to modify the backing array. If one operation is making a change to the backing array at the same time as another, this can put the list into a broken state. You won't see this often unless you're doing very high frequency concurrent operations, but it's a lot better to use a concurrent collection then to discover the issue in production a few weeks or months later.
The following code just executes 1000000 writes in a row simultaneously on each core. On a multi-core machine, this will almost certainly throw an exception because the underlying array gets modified when another concurrent call is not expecting it.
static void Main(string[] args)
{
var list = new List<string>();
void mutateList()
{
for (var i = 0; i < 1000000; i++)
{
list.Add("foo");
}
}
for (var i = 0; i < Environment.ProcessorCount; i++)
{
new Thread(mutateList).Start();
}
Thread.Sleep(-1);
}
I want to know if the following code is thread safe, which I assume it is not. And how I could possibly make it thread safe?
Basically I have a ConcurrentDictionary which acts as a cache for a database table. I want to query the DB every 10 seconds and update the db cache. There will be other threads querying this dictionary the whole time.
I can't just use TryAdd as there may also me elements which have been removed. So I decided instead of searching through the entire dictionary to possibly update, add or remove. I would just reinitialize the dictionary. Please do tell me if this is a silly idea.
My concern is that when I reinitialize the dictionary the querying threads will not longer by thread safe for the instance when the initialization takes place. For that reason I have used a lock for the dictionary when updating it, However I am not sure if this is correct as the object changes in the lock?
private static System.Timers.Timer updateTimer;
private static volatile Boolean _isBusyUpdating = false;
private static ConcurrentDictionary<int, string> _contactIdNames;
public Constructor()
{
// Setup Timers for data updater
updateTimer = new System.Timers.Timer();
updateTimer.Interval = new TimeSpan(0, 0, 10, 0).TotalMilliseconds;
updateTimer.Elapsed += OnTimedEvent;
// Start the timer
updateTimer.Enabled = true;
}
private void OnTimedEvent(Object source, System.Timers.ElapsedEventArgs e)
{
if (!_isBusyUpdating)
{
_isBusyUpdating = true;
// Get new data values and update the list
try
{
var tmp = new ConcurrentDictionary<int, string>();
using (var db = new DBEntities())
{
foreach (var item in db.ContactIDs.Select(x => new { x.Qualifier, x.AlarmCode, x.Description }).AsEnumerable())
{
int key = (item.Qualifier * 1000) + item.AlarmCode;
tmp.TryAdd(key, item.Description);
}
}
if (_contactIdNames == null)
{
_contactIdNames = tmp;
}
else
{
lock (_contactIdNames)
{
_contactIdNames = tmp;
}
}
}
catch (Exception e)
{
Debug.WriteLine("Error occurred in update ContactId db store", e);
}
_isBusyUpdating = false;
}
}
/// Use the dictionary from another Thread
public int GetIdFromClientString(string Name)
{
try
{
int pk;
if (_contactIdNames.TryGetValue(Name, out pk))
{
return pk;
}
}
catch { }
//If all else fails return -1
return -1;
}
You're right your code is not thread safe.
You need to lock _isBusyUpdating variable.
You need to lock _contactIdNames every time, not only when its not null.
Also this code is similar to singleton pattern and it has the same problem with initialization. You can solve it with Double checked locking. However you also need double checked locking when accessing entries.
In the case when you updating whole dictionary at once you need to lock current value every time when accessing. Otherwise you can access it while it's still changing and get error. So you either need to lock variable each time or use Interlocked.
As MSDN says volatile should do the trick with _isBusyUpdating, it should be thread safe.
If you don't want to keep track of _contactIdNames thread safety try to implement update of each entry on the same dictionary. The problem will be in difference detection between DB and current values (what entries have been removed or added, others can be simple rewritten), but not in thread safety, since ConcurrentDictionary is already thread safe.
You seem to be making a lot of work for yourself. Here's how I would tackle this task:
public class Constructor
{
private volatile Dictionary<int, string> _contactIdNames;
public Constructor()
{
Observable
.Interval(TimeSpan.FromSeconds(10.0))
.StartWith(-1)
.Select(n =>
{
using (var db = new DBEntities())
{
return db.ContactIDs.ToDictionary(
x => x.Qualifier * 1000 + x.AlarmCode,
x => x.Description);
}
})
.Subscribe(x => _contactIdNames = x);
}
public string TryGetValue(int key)
{
string value = null;
_contactIdNames.TryGetValue(key, out value);
return value;
}
}
I'm using Microsoft's Reactive Extensions (Rx) Framework - NuGet "Rx-Main" - for the timer to update the dictionary.
The Rx should be fairly straightforward. If you haven't seen it before in very simple terms it's like LINQ meets events.
If you don't like Rx then just go with your current timer model.
All this code does is create a new dictionary every 10 seconds from the DB. I'm just using a plain dictionary since it is only being created from one thread. Since reference assignment is atomic then you can just re-assign the dictionary when you like with complete thread-safety.
Multiple threads can safely read from a dictionary as long as the elements don't change.
I want to know if the following code is thread safe, which I assume it
is not. And how I could possibly make it thread safe?
I believe it's not. First of all i'd create property for ConcurrentDictionary and check if update is underway inside get method, and if it is, i'd return the previous version of object :
private object obj = new object();
private ConcurrentDictionary<int, string> _contactIdNames;
private ConcurrentDictionary<int, string> _contactIdNamesOld;
private volatile bool _isBusyUpdating = false;
public ConcurrentDictionary<int, string> ContactIdNames
{
get
{
if (!_isBusyUpdating) return _contactIdNames;
return _contactIdNamesOld;
}
private set
{
if(_isBusyUpdating) _contactIdNamesOld =
new ConcurrentDictionary<int, string>(_contactIdNames);
_contactIdNames = value;
}
}
And your method can be :
private static void OnTimedEvent(Object source, System.Timers.ElapsedEventArgs e)
{
if (_isBusyUpdating) return;
lock (obj)
{
_isBusyUpdating = true;
// Get new data values and update the list
try
{
ContactIdNames = new ConcurrentDictionary<int, string>();
using (var db = new DBEntities())
{
foreach (var item in db.ContactIDs.Select(x => new { x.Qualifier, x.AlarmCode, x.Description }).AsEnumerable())
{
int key = (item.Qualifier * 1000) + item.AlarmCode;
_contactIdNames.TryAdd(key, item.Description);
}
}
}
catch (Exception e)
{
Debug.WriteLine("Error occurred in update ContactId db store", e);
_contactIdNames = _contactIdNamesOld;
}
finally
{
_isBusyUpdating = false;
}
}
}
P.S.
My concern is that when I reinitialize the dictionary the querying
threads will not longer by thread safe for the instance when the
initialization takes place. For that reason I have used a lock for the
dictionary when updating it, However I am not sure if this is correct
as the object changes in the lock?
It's ConcurrentDictionary<T> type is threadsafe and not the instance of it, so even if you create new instance and change the reference to it - it's not something to worry about.
I have a background worker in a web page which processes a large file import. I have a static property containing a Dictionary of values which I need my background worker to access. To prevent issues with garbage collection, I stringify the Dictionary when passing it into the background worker. The problem is, 1 out of 20 or so times, the Dictionary appears to be garbage collected before it is stringified.
static readonly Dictionary<int, int> myDictionary = new Dictionary<int, int>();
// When button is clicked, I fire the background worker
// Assume for posterity, I've filled the Dictionary with a list of values and those values exist at the time the worker is queued up.
protected void OnProcessClickHandler(object sender, EventArgs e)
{
ThreadPool.QueueUserWorkItem(ProcessInBackground, new object[] {
DictionaryIntIntToCsv(myDictionary)
});
}
// Example of the Background Process
private void ProcessInBackground(object state)
{
object[] parms = state as object[];
if (parms != null && parms.Length > 0)
{
var MyNewDictionary = DictionaryIntIntFromCsv(parms[0] as string);
//... Doing something with the Dictionary
}
}
// Here are some helper methods I am using to stringify the Dictionary. You can ignore these unless you think they have something to do with the issue at hand.
public static Dictionary<int, int> DictionaryIntIntFromCsv(string csv)
{
var dictionary = new Dictionary<int, int>();
foreach (var pair in csv.Split(','))
{
var arrNameValue = pair.Split(':');
if (arrNameValue.Count() != 2) continue;
var key = 0;
var val = 0;
int.TryParse(arrNameValue[0], out key);
int.TryParse(arrNameValue[1], out val);
if (key > 0 && val > 0)
{
dictionary.Add(key, val);
}
}
return dictionary;
}
public static string DictionaryIntIntToCsv(Dictionary<int, int> dictionary)
{
var str = "";
foreach (var key in dictionary.Keys)
{
var value = 0;
dictionary.TryGetValue(key, out value);
if (key == 0 || value == 0) continue;
var item = key + ":" + value;
str += (item + ",");
}
return str;
}
I know there is an issue with Garbage Collection. My theory is sometimes the main thread completes and garbage collection is run before the background worker has a chance to stringify the Dictionary. Would I be correct in assuming I could avoid issues with Garbage Collection if I stringify the Dictionary before queuing the background worker? Like so:
protected void OnProcessClickHandler(object sender, EventArgs e)
{
var MyString = DictionaryIntIntToCsv(MyDictionary);
ThreadPool.QueueUserWorkItem(ProcessInBackground, new object[] {
MyString
});
}
NOTE: The page is interactive and does several postbacks before firing off the background worker.
There is really a lot of misinformation and bizarre implementation in this question, so much so that it cannot actually be answered without further clarification.
What leads you to believe the dictionary will be collected once you've "stringified" it? What are you actually doing with the dictionary values in the ProcessInBackground() method? Is the processing actually more expensive than serializing and deserializing a dictionary to and from a string for no reason? If so, why is there a background worker being used at all? Why is the string passed in inside an object array instead of simply the string itself? Further on that point, why is the dictionary being serialized at all, is there any good reason can't it be passed in as the state argument directly?
You are likely initializing the property on page load. The reference to the property is tied to the instance of the page which existed on page load. After the server delivered the initial page to you, the class was eligible for garbage collection.
I believe you are seeing a race condition between how long it takes the user to do the postback on the page and how long it takes the server to collect the first instance of the class.
If the property were non-static, the values would not be there on postback. However, since it is a static property, it will exist in memory until the Garbage collector cleans it up.
Here you create a local variable with the dictionary:
if (parms != null && parms.Length > 0)
{
var MyNewDictionary = DictionaryIntIntFromCsv(parms[0] as string);
}
The above does not affect the below in any way. No where else in your code do you ever populate the static field MyDictionary.
The above local variable is completely separate from the below static field you have here, so the above assignment does not affect the below property in any way:
static readonly Dictionary<int, int> MyDictionary = new Dictionary<int, int>();
I have a main thread which is controlling a windows form, upon pressing a button two threads are executed. One is used for recording information, the other is used for reading it. The idea behind putting these in threads is to enable the user to interact with the interface while they are executing.
Here is the creating of the two threads;
Thread recordThread = new Thread(() => RecordData(data));
recordThread.Name = "record";
recordThread.Start();
Thread readThread = new Thread(() => ReadData(data));
readThread.Name = "read";
readThread.Start();
The data is simply a Data-object that stores the data that is recorded during the recording.
The problem that I am facing is that the first thread is executed fine, the second refuses to run until the first one completes. Putting a breakpoint in the second threads function, ReadData lets me know that it is only called after the first thread is done with all of its recording.
I have been trying to solve this for a few hours now and I can't get my head around why it would do this. Adding a;
while(readThread.IsAlive) { }
right after the start will halt the execution of anything after that, and it's state is Running. But it will not go to the given method.
Any ideas?
Edit:
The two functions that are called upon by the threads are;
private void RecordData(Data d)
{
int i = 0;
while (i < time * freq)
{
double[] data = daq.Read();
d.AddData(data);
i++;
}
}
private void ReadData(Data d)
{
UpdateLabelDelegate updateData =
new UpdateLabelDelegate(UpdateLabel);
int i = 0;
while (i < time * freq)
{
double[] data = d.ReadLastData();
this.Invoke(updateData, new object[] { data });
i++;
}
}
The data object has locking in both the functions that are called upon; ReadLastData and Read.
Here are the methods in the Data object.
public void AddData(double[] data)
{
lock (this)
{
int i = 0;
foreach (double d in data)
{
movementData[i].Add(d);
i++;
}
}
}
public double[] ReadLastData()
{
double[] data = new double[channels];
lock (this)
{
int i = 0;
foreach (List<double> list in movementData)
{
data[i] = list[list.Count - 1];
}
}
return data;
}
Looks like you have a race condition between your reading/writing. In your first thread you lock down the object whilst you add data to it and in the second thread you attempt to get an exclusive lock on it to start reading. However, the problem is the first thread is executing so fast that the second thread never really gets a chance to acquire the lock.
The solution to this problem really depends on what sort of behaviour you are after here. If you expect after every write you get a consecutive read then what you need to do is control the execution between the reading/writing operations e.g.
static AutoResetEvent canWrite = new AutoResetEvent(true); // default to true so the first write happens
static AutoResetEvent canRead = new AutoResetEvent(false);
...
private void RecordData(Data d)
{
int i = 0;
while (i < time * freq)
{
double[] data = daq.Read();
canWrite.WaitOne(); // wait for the second thread to finish reading
d.AddData(data);
canRead.Set(); // let the second thread know we have finished writing
i++;
}
}
private void ReadData(Data d)
{
UpdateLabelDelegate updateData =
new UpdateLabelDelegate(UpdateLabel);
int i = 0;
while (i < time * freq)
{
canRead.WaitOne(); // wait for the first thread to finish writing
double[] data = d.ReadLastData();
canWrite.Set(); // let the first thread know we have finished reading
this.Invoke(updateData, new object[] { data });
i++;
}
}
Could you try adding a Sleep inside RecordData?
Maybe it's just your (mono cpu??) windows operating system that doesn't let the second thread get its hand on cpu resources.
Don't do this:
lock (this)
Do something like this instead:
private object oLock = new object();
[...]
lock (this.oLock)
EDIT:
Could you try calls like this:
Thread recordThread = new Thread((o) => RecordData((Data)o));
recordThread.Name = "record";
recordThread.Start(data);
I have the following piece of code:
private Dictionary<object, object> items = new Dictionary<object, object>;
public IEnumerable<object> Keys
{
get
{
foreach (object key in items.Keys)
{
yield return key;
}
}
}
Is this thread-safe? If not do I have to put a lock around the loop or the yield return?
Here is what I mean:
Thread1 accesses the Keys property while Thread2 adds an item to the underlying dictionary. Is Thread1 affected by the add of Thread2?
What exactly do you mean by thread-safe?
You certainly shouldn't change the dictionary while you're iterating over it, whether in the same thread or not.
If the dictionary is being accessed in multiple threads in general, the caller should take out a lock (the same one covering all accesses) so that they can lock for the duration of iterating over the result.
EDIT: To respond to your edit, no it in no way corresponds to the lock code. There is no lock automatically taken out by an iterator block - and how would it know about syncRoot anyway?
Moreover, just locking the return of the IEnumerable<TKey> doesn't make it thread-safe either - because the lock only affects the period of time when it's returning the sequence, not the period during which it's being iterated over.
Check out this post on what happens behind the scenes with the yield keyword:
Behind the scenes of the C# yield keyword
In short - the compiler takes your yield keyword and generates an entire class in the IL to support the functionality. You can check out the page after the jump and check out the code that gets generated...and that code looks like it tracks thread id to keep things safe.
OK, I did some testing and got an interesting result.
It seems that it is more an issue of the enumerator of the underlying collection than the yield keyword. The enumerator (actually its MoveNext method) throws (if implemented correctly) an InvalidOperationException because the enumeration has changed. According to the MSDN documentation of the MoveNext method this is the expected behavior.
Because enumerating through a collection is usually not thread-safe a yield return is not either.
I believe it is, but I cannot find a reference that confirms it. Each time any thread calls foreach on an iterator, a new thread local* instance of the underlying IEnumerator should get created, so there should not be any "shared" memory state that two threads can conflict over...
Thread Local - In the sense that it's reference variable is scoped to a method stack frame on that thread
I believe yield implementation is thread-safe. Indeed, you can run that simple program at home and you will notice that the state of the listInt() method is correctly saved and restored for each thread without edge effect from other threads.
public class Test
{
public void Display(int index)
{
foreach (int i in listInt())
{
Console.WriteLine("Thread {0} says: {1}", index, i);
Thread.Sleep(1);
}
}
public IEnumerable<int> listInt()
{
for (int i = 0; i < 5; i++)
{
yield return i;
}
}
}
class MainApp
{
static void Main()
{
Test test = new Test();
for (int i = 0; i < 4; i++)
{
int x = i;
Thread t = new Thread(p => { test.Display(x); });
t.Start();
}
// Wait for user
Console.ReadKey();
}
}
class Program
{
static SomeCollection _sc = new SomeCollection();
static void Main(string[] args)
{
// Create one thread that adds entries and
// one thread that reads them
Thread t1 = new Thread(AddEntries);
Thread t2 = new Thread(EnumEntries);
t2.Start(_sc);
t1.Start(_sc);
}
static void AddEntries(object state)
{
SomeCollection sc = (SomeCollection)state;
for (int x = 0; x < 20; x++)
{
Trace.WriteLine("adding");
sc.Add(x);
Trace.WriteLine("added");
Thread.Sleep(x * 3);
}
}
static void EnumEntries(object state)
{
SomeCollection sc = (SomeCollection)state;
for (int x = 0; x < 10; x++)
{
Trace.WriteLine("Loop" + x);
foreach (int item in sc.AllValues)
{
Trace.Write(item + " ");
}
Thread.Sleep(30);
Trace.WriteLine("");
}
}
}
class SomeCollection
{
private List<int> _collection = new List<int>();
private object _sync = new object();
public void Add(int i)
{
lock(_sync)
{
_collection.Add(i);
}
}
public IEnumerable<int> AllValues
{
get
{
lock (_sync)
{
foreach (int i in _collection)
{
yield return i;
}
}
}
}
}