We Keep Coding

Thread Safety C# List<T>

I know of and have used the System.Collections.Concurrent.ConcurrentBag<T> for building thread safe code in the past. I have some legacy code that I'm trying to thread to increase performance however there is a non static List object that is being written with different sources of data. All of the writing is being done prior to any reading of the list and my initial tests show that multiple threads appear to write to the object without any issues.
Sample Windows App
Does a non static C# List object have thread safety for writing across multiple threads prior to reading? How can this be tested?
BackgroundWorker backgroundWorkerA, backgroundWorkerB;
System.Threading.ManualResetEvent manualReset;
List<string> _shardList = new List<string>(0);
public UserControl1()
{
InitializeComponent();
manualReset = new System.Threading.ManualResetEvent(false);
backgroundWorkerA = new BackgroundWorker();
backgroundWorkerA.WorkerSupportsCancellation = true;
backgroundWorkerA.DoWork += BackgroundWorkerA_DoWork;
backgroundWorkerB = new BackgroundWorker();
backgroundWorkerB.WorkerSupportsCancellation = true;
backgroundWorkerB.DoWork += BackgroundWorkerB_DoWork;
this.HandleCreated += UserControl1_HandleCreated;
}
private void UserControl1_HandleCreated(object sender, EventArgs e)
{
backgroundWorkerA.RunWorkerAsync(_shardList);
backgroundWorkerB.RunWorkerAsync(_shardList);
manualReset.Set();
}
private void BackgroundWorkerB_DoWork(object sender, DoWorkEventArgs e)
{
List<string> _shardList = (List<string>)e.Argument;
manualReset.WaitOne();
int _i = 0;
while(!this.backgroundWorkerB.CancellationPending)
{
_shardList.Add("b" + _i++.ToString());
System.Diagnostics.Debug.WriteLine("b is running");
}
thread2.Invoke(new MethodInvoker(delegate { thread2.Text = string.Join(System.Environment.NewLine, _shardList.ToArray()); }));
}
private void button1_Click(object sender, EventArgs e)
{
backgroundWorkerA.CancelAsync();
backgroundWorkerB.CancelAsync();
}
private void BackgroundWorkerA_DoWork(object sender, DoWorkEventArgs e)
{
List<string> _shardList = (List<string>)e.Argument;
manualReset.WaitOne();
int _i = 0;
while (!this.backgroundWorkerA.CancellationPending)
{
_shardList.Add("a" + _i++.ToString());
System.Diagnostics.Debug.WriteLine("a is running");
}
thread1.Invoke(new MethodInvoker(delegate { thread1.Text = string.Join(System.Environment.NewLine, _shardList.ToArray()); }));
}

There are multiple things that make concurrent writes to a List<T> unsafe.
First let's have a look at the code of the Add method:
public void Add(T item) {
if (_size == _items.Length) EnsureCapacity(_size + 1);
_items[_size++] = item;
_version++;
}
The first issue is EnsureCapacity. If the list's inner array isn't big enough to receive the new element, it will create a new, bigger array, and copy the elements from the old one to the new one. If a thread writes in the old list after the copy but before the swap, the element will be lost
The second issue is the non-atomic increment of size. If two threads try to write at the same time, they may write at the same index in the array, thus losing one item
Those race conditions are not very likely, but they'll eventually happen if you keep writing in the same list from multiple threads.

When you modify a list, it has to modify the backing array. If one operation is making a change to the backing array at the same time as another, this can put the list into a broken state. You won't see this often unless you're doing very high frequency concurrent operations, but it's a lot better to use a concurrent collection then to discover the issue in production a few weeks or months later.
The following code just executes 1000000 writes in a row simultaneously on each core. On a multi-core machine, this will almost certainly throw an exception because the underlying array gets modified when another concurrent call is not expecting it.
static void Main(string[] args)
{
var list = new List<string>();
void mutateList()
{
for (var i = 0; i < 1000000; i++)
{
list.Add("foo");
}
}
for (var i = 0; i < Environment.ProcessorCount; i++)
{
new Thread(mutateList).Start();
}
Thread.Sleep(-1);
}

Safely clear List (with concurrency)

Good day!
I have List<byte> soundBuffer - to get audio signal from microphone.
void _waveInStream_DataAvailable(object sender, WaveInEventArgs e)
{
lock(_lockObject)
{
for (int i = 0; i < e.BytesRecorded; i++)
{
_soundBuffer.Add(e.Buffer[i]);
}
}
}
And if user waits a long type- buffer will be very big (2 mb per minute).
So, i create a timer:
_timerSoundCount = new System.Timers.Timer();
_timerSoundCount.Interval = 10000; // check every 10 second
_timerSoundCount.Enabled = true;
_timerSoundCount.Elapsed += _timerSoundCount_Elapsed;
_timerSoundCount.Start();
And:
void _timerSoundCount_Elapsed(object sender, ElapsedEventArgs e)
{
if(_soundBuffer.Count>2*1024*1024)
{
var energy = GetSignalEnergy(_soundBuffer);
if(energy<1) //if energy of signal is small- clear buffer.
{
lock (_lockObject)
_soundBuffer.Clear();
}
}
else if(_soundBuffer.Count>=5*1024*1024)
{... the same operation}
else if(_sounfBuffer.Count>=10*1024*1024)
{ _soundBuffer.Clear();//very big buffer }
}
Every 10 seconds i check the buffer size. If it is too big- i just clear buffer, because i can detect Speech\Silence and clear buffer at that code.
So, point is: can it be that when i execute _soundBuffer.Clear() at timer and at the same time at _waveInStream_DataAvailable i will add new bytes to buffer- can it be currupt write?
Can it be a deadlock?
If so, can you help me how to safetely clear buffer?
Thank you!

If the two actions are being undertaken from the same thread, there is no chance of a deadlock occurring.
If there are multiple threads writing/reading the list at the same time, then lock should be used (https://msdn.microsoft.com/en-us/library/c5kehkcz.aspx) to prevent multiple threads accessing the object simultaneously. See here (use the same lock object at two different code block?) for a simple example.
Alternatively, you can use a concurrent collection from the System.Collections.Concurrent namespace (https://msdn.microsoft.com/en-us/library/system.collections.concurrent(v=vs.110).aspx) Perhaps a ConcurrentQueue would be appropriate if the data is not being randomly accessed. You can also implement your own concurrent collection, although this is much more complex.

Freeing memory of object

I've got some code like this:
private void button1_Click(object sender, EventArgs e)
{
Someclass object= new Someclass();
foreach (conditioin)
{
some methods();
}
object= null;
}
I need to free memory of object becuase every time that I press the button it shows me the same results. But methods inside should be forget and initialised with new result all over again.
Can you help me, please?
$$ UPDATE $$
OK so I'll provide the specific code
I use NativeWiFi
private void button1_Click(object sender, EventArgs e)
{
listView1.Items.Clear();
WlanClient client = new WlanClient();
foreach (WlanClient.WlanInterface wlanIface in client.Interfaces)
{
Wlan.WlanBssEntry[] wlanBssEntries = wlanIface.GetNetworkBssList();
foreach (Wlan.WlanBssEntry network in wlanBssEntries)
{
byte[] macAddr = network.dot11Bssid;
string tMac = "";
for (int i = 0; i < macAddr.Length; i++)
{
tMac += macAddr[i].ToString("x2").PadLeft(2, '0').ToUpper();
}
listView1.Items.Add(String.Format("{0} Signal: {1}% ", (GetStringForSSID(network.dot11Ssid)), network.linkQuality));
/*
Console.WriteLine("Signal: {0}%.", network.linkQuality);
Console.WriteLine("BSS Type: {0}.", network.dot11BssType);
Console.WriteLine("MAC: {0}.", tMac);
Console.WriteLine("");*/
}
}
client = null;
}
static string GetStringForSSID(Wlan.Dot11Ssid ssid)
{
return System.Text.Encoding.ASCII.GetString(ssid.SSID, 0, (int)ssid.SSIDLength);
}

It is a common misunderstanding that setting a local variable to null frees up any memory. As the variable goes out of scope when leaving the method anyway, the object = null; line doesn't make any sense.
Also in .NET you can not force memory to be freed. The garbage collection does its job whenever it thinks it's time.

Generally spoken, every time you execute the constructor Someclass() a new object is created. In plain C# the location in memory is irrelevant. All members will be initialized, either implicitly e.g. to null or by your own code. If you obtain the same results in each loop iteration that is because the initialization is the same as in the previous runs.

Simple ConsoleProgram with IDisposable - No memory decrease - we have a leak?

I have got a simple ConsoleProgram which is creating a list of 80 IDisposable objects. This object hold two System.Threading.ManualResetEvent which are closed in the Dispose() method. Please consider the code:
public class Program
{
private static void Main(string[] args)
{
Console.ReadLine();
Test(80);
Console.WriteLine("end.");
Console.ReadLine();
}
private static void Test(int c)
{
Console.WriteLine("Test start.");
List<TestObject> list = new List<TestObject>();
for (int i = 0; i < c; i++)
list.Add(new TestObject());
Console.WriteLine("End of adding. Added: {0} items.", c);
Console.ReadLine();
foreach (TestObject obj in list)
obj.Dispose();
list = null;
Console.WriteLine("Dispose end.");
}
public class TestObject : IDisposable
{
public ManualResetEvent mr1 = new ManualResetEvent(true);
public ManualResetEvent mr2 = new ManualResetEvent(false);
public void Dispose()
{
mr1.Close();
mr2.Close();
}
}
}
I have tested my program in case of consumed memory and memory leaks. I supposed that consumed memory will increase after creating all list objects, but it will decrease after calling the Dispose() method and setting null value to the list object. Unfortunately I have observer different behaviour. Please consider following result of my test:
Program starts (nothing created). WorkingSet = 6.700K
List of 80 objects was created. WorkingSet = 7.160K (memory grow: 460K)
Program called the Dispose() method an set null to list object. WorkingSet = 7.164K (memory grow form last point: 4K)
Program is hanging for more than 20 minutes. WorkingSet = 7.296K (memory grow from last point: 105K, memory grow from the beginning: 596K)
Program was closed.
I am really confused about the point 3 and 4. Why the memory was not released? I think that this is a memory leak, because the total memory grow was equal to 596K and it was never released.
Thank you very much for any answer.
PS. Changing the number of object, for example to 9000, cause a memory growth of more than 2MB.
PS2. At the end of my program I am calling GC.Collect() to force the clean-up. But after that amount of used memory is still the same and it is not decreasing - I am confused.

Calling Dispose() and setting a value to null does not free the memory up immediately. Your memory should be released when the Garbage Collector next runs, but this time is not deterministic. Typically, it will happen when the application is under memory pressure, usually as a result of object creation requesting more memory. Without that pressure, and with the application idle, the GC might never collect your memory.
In short, this is not a memory leak.

problem disposing class in Dictionary it is Still in the heap memory although using GC.Collect

i have a problem disposing class in Dictionary
this is my code
private Dictionary<string, MyProcessor> Processors = new Dictionary<string, MyProcessor>();
private void button1_Click(object sender, EventArgs e)
{
if (!Processors.ContainsKey(textBox1.Text))
{
Processors.Add(textBox1.Text, new MyProcessor());
}
}
private void button2_Click(object sender, EventArgs e)
{
MyProcessor currnt_processor = Processors[textBox2.Text];
Processors.Remove(textBox2.Text);
currnt_processor.Dispose();
currnt_processor = null;
GC.Collect();
GC.WaitForFullGCComplete();
}
public class MyProcessor: IDisposable
{
private bool isDisposed = false;
string x = "";
public MyProcessor()
{
for (int i = 0; i < 20000; i++)
{
//this line only to increase the memory usage to know if the class is dispose or not
x = x + "gggggggggggg";
}
}
public void Dispose()
{
x=null;
this.Dispose(true);
GC.SuppressFinalize(this);
}
public void Dispose(bool disposing)
{
if (!this.isDisposed)
{
isDisposed = true;
this.Dispose();
}
}
~MyProcessor()
{
Dispose(false);
}
}
i use "ANTS Memory Profiler" to monitor heap memory
the disposing work only when i remove all keys from dictionary
how can i destroy the class from heap memory ?
this a video link for the problem
http://www.youtube.com/watch?v=ePorlksv2QY
thanks in advance

I think you are seeing ghosts - keep in mind that the .NET garbage collection is a generational garbage collection based on memory pressure. If there is no memory pressure your resources will not be garbage collected. Also calling GC.Collect() is just a bad idea, I hope you are only doing this for your profiling tests.
As an aside what resources exactly are your disposing in your Dispose method? Doesn't look like you need one.
In the implementation you provided not a single Dispose() method call nor the finalizer ~MyProcessor() is needed at all.

I think you might want to call a GC.WaitForFullGCComplete(); as GC.Collect(); just kicks off the GC.
It is possible to have a reference leak (ie you inadvertently keep a whole load of references to objects in a list somewhere, preventing their collection). I doubt that the dictionary has such a "leak" as it is such a widely used class that it would be a known problem. Most likely if you have problems with your code most likely the problem is somewhere else in your code.
If this is not causing you actual problems, lay off it, grab your towel and don't panic, the GC does work :)

You are removing item with one name but disposing item with another name from dictionary. So item that disposed is still referenced from dictionary (Processors.Remove(textBox2.Text); while current = dict[textBox1.Text]).
Take a note of Eric's comment - Dispose does not make object ready for garbage collection, if it has references it still will be in the memory.