We Keep Coding

Thread Safety C# List<T>

I know of and have used the System.Collections.Concurrent.ConcurrentBag<T> for building thread safe code in the past. I have some legacy code that I'm trying to thread to increase performance however there is a non static List object that is being written with different sources of data. All of the writing is being done prior to any reading of the list and my initial tests show that multiple threads appear to write to the object without any issues.
Sample Windows App
Does a non static C# List object have thread safety for writing across multiple threads prior to reading? How can this be tested?
BackgroundWorker backgroundWorkerA, backgroundWorkerB;
System.Threading.ManualResetEvent manualReset;
List<string> _shardList = new List<string>(0);
public UserControl1()
{
InitializeComponent();
manualReset = new System.Threading.ManualResetEvent(false);
backgroundWorkerA = new BackgroundWorker();
backgroundWorkerA.WorkerSupportsCancellation = true;
backgroundWorkerA.DoWork += BackgroundWorkerA_DoWork;
backgroundWorkerB = new BackgroundWorker();
backgroundWorkerB.WorkerSupportsCancellation = true;
backgroundWorkerB.DoWork += BackgroundWorkerB_DoWork;
this.HandleCreated += UserControl1_HandleCreated;
}
private void UserControl1_HandleCreated(object sender, EventArgs e)
{
backgroundWorkerA.RunWorkerAsync(_shardList);
backgroundWorkerB.RunWorkerAsync(_shardList);
manualReset.Set();
}
private void BackgroundWorkerB_DoWork(object sender, DoWorkEventArgs e)
{
List<string> _shardList = (List<string>)e.Argument;
manualReset.WaitOne();
int _i = 0;
while(!this.backgroundWorkerB.CancellationPending)
{
_shardList.Add("b" + _i++.ToString());
System.Diagnostics.Debug.WriteLine("b is running");
}
thread2.Invoke(new MethodInvoker(delegate { thread2.Text = string.Join(System.Environment.NewLine, _shardList.ToArray()); }));
}
private void button1_Click(object sender, EventArgs e)
{
backgroundWorkerA.CancelAsync();
backgroundWorkerB.CancelAsync();
}
private void BackgroundWorkerA_DoWork(object sender, DoWorkEventArgs e)
{
List<string> _shardList = (List<string>)e.Argument;
manualReset.WaitOne();
int _i = 0;
while (!this.backgroundWorkerA.CancellationPending)
{
_shardList.Add("a" + _i++.ToString());
System.Diagnostics.Debug.WriteLine("a is running");
}
thread1.Invoke(new MethodInvoker(delegate { thread1.Text = string.Join(System.Environment.NewLine, _shardList.ToArray()); }));
}

There are multiple things that make concurrent writes to a List<T> unsafe.
First let's have a look at the code of the Add method:
public void Add(T item) {
if (_size == _items.Length) EnsureCapacity(_size + 1);
_items[_size++] = item;
_version++;
}
The first issue is EnsureCapacity. If the list's inner array isn't big enough to receive the new element, it will create a new, bigger array, and copy the elements from the old one to the new one. If a thread writes in the old list after the copy but before the swap, the element will be lost
The second issue is the non-atomic increment of size. If two threads try to write at the same time, they may write at the same index in the array, thus losing one item
Those race conditions are not very likely, but they'll eventually happen if you keep writing in the same list from multiple threads.

When you modify a list, it has to modify the backing array. If one operation is making a change to the backing array at the same time as another, this can put the list into a broken state. You won't see this often unless you're doing very high frequency concurrent operations, but it's a lot better to use a concurrent collection then to discover the issue in production a few weeks or months later.
The following code just executes 1000000 writes in a row simultaneously on each core. On a multi-core machine, this will almost certainly throw an exception because the underlying array gets modified when another concurrent call is not expecting it.
static void Main(string[] args)
{
var list = new List<string>();
void mutateList()
{
for (var i = 0; i < 1000000; i++)
{
list.Add("foo");
}
}
for (var i = 0; i < Environment.ProcessorCount; i++)
{
new Thread(mutateList).Start();
}
Thread.Sleep(-1);
}

How do I call a method with a reference to the class parameter?

So I've been trying to figure out how to get my code to work all night. I've been reading up on all kinds of stuff and trying to identify what I'm doing wrong, but everything I try I end up at the same issue. I'm trying to change a variable in my class by referencing it in a method so it will change in the class and not just locally. But I don't know what to put as a parameter for the ref Storyboard SB. Can someone tell me what should be done, I've tried setting it to null, even through a variable and it doesn't work. Also 'StoryBoard' is the class that I'm writing the code in.
public class StoryBoard
{
public string[] TextBoxes = new string[10];
public int Counter = 0;
private void RtClickButton_Click(object sender, EventArgs e)
{
RtClickButton_ClickImpl(sender, e, "what would I put here?");
}
private void RtClickButton_ClickImpl(object sender, EventArgs e, ref StoryBoard SB)
{
string TBT = TxtBox.Text;
switch(Counter)
{
case 0:
TextBoxes[Counter] = TBT;
break;
}
SB.Counter++; // Adds 1 to the counter.
LtClickButton.Enabled = true;
TxtBox.Clear(); // Clears the text box.
}
}

Try simply
Counter++;
or if in doubt you can use the this keyword to refer to instance members of this class, e.g
this.Counter++; // Adds 1 to the counter.
To expand upon this, all variables from the current object will always be accessible in a normal method (i.e. not static) unless a variable of the same name exists in the same scope, where the scope can be the method or a single block between curly braces.
If you use the this keyword it will always reference the variable that belongs to the object/class and not an inline variable that is defined in a different scope.

But I don't know what to put as a parameter for the ref Storyboard SB.
Keep a private member variable for SB:
private StoryBoard _SB = null; //A member variable to hold the StoryBoard object
public class Form1WhatEver
{
public Form1WhatEver()
{
//Instantiate a reference to the StoryBoard and hold it in the private member variable
_SB = new StoryBoard();
}
public string[] TextBoxes = new string[10];
public int Counter = 0;
private void RtClickButton_Click(object sender, EventArgs e)
{
RtClickButton_ClickImpl(sender, e, ref _SB); //Pass the instance of StoryBoard byRef.
//Check that our _SB Counter variable was incremented (+1)
System.Diagnostics.Debug.WriteLine(_SB.Counter.ToString());
}
private void RtClickButton_ClickImpl(object sender, EventArgs e, ref StoryBoard SB)
{
string TBT = TxtBox.Text;
switch(Counter)
{
case 0:
TextBoxes[Counter] = TBT;
break;
}
SB.Counter++; // Adds 1 to the counter.
LtClickButton.Enabled = true;
TxtBox.Clear(); // Clears the text box.
}

Is Garbage Collection occurring on my static property before my background worker has fully consumed it?

I have a background worker in a web page which processes a large file import. I have a static property containing a Dictionary of values which I need my background worker to access. To prevent issues with garbage collection, I stringify the Dictionary when passing it into the background worker. The problem is, 1 out of 20 or so times, the Dictionary appears to be garbage collected before it is stringified.
static readonly Dictionary<int, int> myDictionary = new Dictionary<int, int>();
// When button is clicked, I fire the background worker
// Assume for posterity, I've filled the Dictionary with a list of values and those values exist at the time the worker is queued up.
protected void OnProcessClickHandler(object sender, EventArgs e)
{
ThreadPool.QueueUserWorkItem(ProcessInBackground, new object[] {
DictionaryIntIntToCsv(myDictionary)
});
}
// Example of the Background Process
private void ProcessInBackground(object state)
{
object[] parms = state as object[];
if (parms != null && parms.Length > 0)
{
var MyNewDictionary = DictionaryIntIntFromCsv(parms[0] as string);
//... Doing something with the Dictionary
}
}
// Here are some helper methods I am using to stringify the Dictionary. You can ignore these unless you think they have something to do with the issue at hand.
public static Dictionary<int, int> DictionaryIntIntFromCsv(string csv)
{
var dictionary = new Dictionary<int, int>();
foreach (var pair in csv.Split(','))
{
var arrNameValue = pair.Split(':');
if (arrNameValue.Count() != 2) continue;
var key = 0;
var val = 0;
int.TryParse(arrNameValue[0], out key);
int.TryParse(arrNameValue[1], out val);
if (key > 0 && val > 0)
{
dictionary.Add(key, val);
}
}
return dictionary;
}
public static string DictionaryIntIntToCsv(Dictionary<int, int> dictionary)
{
var str = "";
foreach (var key in dictionary.Keys)
{
var value = 0;
dictionary.TryGetValue(key, out value);
if (key == 0 || value == 0) continue;
var item = key + ":" + value;
str += (item + ",");
}
return str;
}
I know there is an issue with Garbage Collection. My theory is sometimes the main thread completes and garbage collection is run before the background worker has a chance to stringify the Dictionary. Would I be correct in assuming I could avoid issues with Garbage Collection if I stringify the Dictionary before queuing the background worker? Like so:
protected void OnProcessClickHandler(object sender, EventArgs e)
{
var MyString = DictionaryIntIntToCsv(MyDictionary);
ThreadPool.QueueUserWorkItem(ProcessInBackground, new object[] {
MyString
});
}
NOTE: The page is interactive and does several postbacks before firing off the background worker.

There is really a lot of misinformation and bizarre implementation in this question, so much so that it cannot actually be answered without further clarification.
What leads you to believe the dictionary will be collected once you've "stringified" it? What are you actually doing with the dictionary values in the ProcessInBackground() method? Is the processing actually more expensive than serializing and deserializing a dictionary to and from a string for no reason? If so, why is there a background worker being used at all? Why is the string passed in inside an object array instead of simply the string itself? Further on that point, why is the dictionary being serialized at all, is there any good reason can't it be passed in as the state argument directly?

You are likely initializing the property on page load. The reference to the property is tied to the instance of the page which existed on page load. After the server delivered the initial page to you, the class was eligible for garbage collection.
I believe you are seeing a race condition between how long it takes the user to do the postback on the page and how long it takes the server to collect the first instance of the class.
If the property were non-static, the values would not be there on postback. However, since it is a static property, it will exist in memory until the Garbage collector cleans it up.

Here you create a local variable with the dictionary:
if (parms != null && parms.Length > 0)
{
var MyNewDictionary = DictionaryIntIntFromCsv(parms[0] as string);
}
The above does not affect the below in any way. No where else in your code do you ever populate the static field MyDictionary.
The above local variable is completely separate from the below static field you have here, so the above assignment does not affect the below property in any way:
static readonly Dictionary<int, int> MyDictionary = new Dictionary<int, int>();

New instance every time?

In the following code, which is better? To call add page from within CardPanelDesigner_AddPage? Or use the Func TransactionFunction??
Basically I want to know if doing the inner func will create a "new function" every time :S I don't even know what I'm asking.
Is there an overhead to doing the inner function or should I use the addpage?
private object AddPage(IDesignerHost Host, object Sender)
{
return null;
}
private void CardPanelDesigner_AddPage(object sender, EventArgs e)
{
IDesignerHost DesignerHost = (IDesignerHost)GetService(typeof(IDesignerHost));
if (DesignerHost != null)
{
Func<IDesignerHost, object, object> TransactionFunction = (Host, Param) =>
{
return null;
};
TransactionInfo("Add Page", DesignerHost, AddPage); //Add page? OR TransactionFunction? :S
}
}

Yes, TransactionFunction will create a new object each time CardPanelDesigner_AddPage is called. The performance overhead of this however will likely be negligible. You should do whatever reads best to you (and your team).

How to restore memory usage of my program?

I'm creating app which uses very large amount of memory. When I finish computation the program remains with high memory as I can notice in Task Manager.
How to reset RAM memory usage of my program?
I'm creating a large array in which I store MB worth of random characters. Then print all characters into a variable and then write the content of the variable to the file.
namespace Fake_File_Creator
{
public partial class MainWindow : Form
{
private string text;
private List<char> stringChars;
public MainWindow()
{
InitializeComponent();
}
private void btnNewFile_Click(object sender, EventArgs e)
{
sfdNewFile.Filter = "All Files|*.*";
if (sfdNewFile.ShowDialog() == DialogResult.OK)
{
lblMessage.Text = "Generating data...";
bwCreateData.RunWorkerAsync((int)nudSize.Value * 1024000);
}
}
private void bwCreateData_DoWork(object sender, DoWorkEventArgs e)
{
var random = new Random();
var chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz 0123456789 ~!##$%^&*()_+ /.,'[];{}|:<>?";
stringChars = new List<char>();
for (int i = 0; i < (int)e.Argument; i++)
{
stringChars.Add(chars[random.Next(chars.Length)]);
}
text = new string(stringChars.ToArray());
}
void bwCreateData_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
lblMessage.Text = "Saving file...";
bwCreateFile.RunWorkerAsync();
}
private void bwCreateFile_DoWork(object sender, DoWorkEventArgs e)
{
using (StreamWriter outfile = new StreamWriter(sfdNewFile.FileName))
{
outfile.Write(text);
}
}
void bwCreateFile_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
lblMessage.Text = "File succesfully created...";
text = string.Empty;
stringChars.Clear();
stringChars.TrimExcess();
}
}
}

There is little you can do about this. The memory consumed by unused object gets freed when garbage collection happens. And you can not force garbage collection. You can read more about it on MSDN also check GC Class
Example from MSDN
private const long maxGarbage = 1000;
static void Main()
{
MyGCCollectClass myGCCol = new MyGCCollectClass();
// Determine the maximum number of generations the system
// garbage collector currently supports.
Console.WriteLine("The highest generation is {0}", GC.MaxGeneration);
myGCCol.MakeSomeGarbage();
// Determine which generation myGCCol object is stored in.
Console.WriteLine("Generation: {0}", GC.GetGeneration(myGCCol));
// Determine the best available approximation of the number
// of bytes currently allocated in managed memory.
Console.WriteLine("Total Memory: {0}", GC.GetTotalMemory(false));
// Perform a collection of generation 0 only.
GC.Collect(0);
// Determine which generation myGCCol object is stored in.
Console.WriteLine("Generation: {0}", GC.GetGeneration(myGCCol));
Console.WriteLine("Total Memory: {0}", GC.GetTotalMemory(false));
// Perform a collection of all generations up to and including 2.
GC.Collect(2);
// Determine which generation myGCCol object is stored in.
Console.WriteLine("Generation: {0}", GC.GetGeneration(myGCCol));
Console.WriteLine("Total Memory: {0}", GC.GetTotalMemory(false));
Console.Read();
}
void MakeSomeGarbage()
{
Version vt;
for(int i = 0; i < maxGarbage; i++)
{
// Create objects and release them to fill up memory
// with unused objects.
vt = new Version();
}
}

Types with resources should implement IDisposable, and freeing these resources is done by calling Dispose (or wrapping them in a using statement). GC.Collect is only necessary for objects that have resources but don't implement IDisposable.

You can't force GC
But if you want to set the memory free as soon as you are done with your array.
//I assume stringChars is the array
stringChars.Clear();
stringChars = null;
GC.Collect();

It depends on what type of memory you want to be freed.
Really calling GC.Collect(); may not work. Why?
It collects objects but you don't know which objects. Your object is large. Usually garbage collectors don't collect large objects or they do it rarely (but rarely for machine time measurement not for human).
It is very likely that virtual memory allocated for your array wouldn't be released to the OS. Because virtual machine memory managers usually reuse freed memory.
Also they perform preallocation of memory. They allocate large piece of memory and then use it. It allows not to perform a lot of system calls to get memory for new objects because .NET languages usually create a lot of them (in C++ copying is used a lot and heap is "dummy" in compare with .NET memory manager).