We Keep Coding

Async with part in UI

I have a task, which executed async, in part of this task add items in UI run via Dispatcher.BeginInvoke where i update a ObservebleCollection. For thread safe access to collection, i use a semaphoreSlim, but as request to Collection proceed in UI thread and Dispatcher.BeginInvoke also work in UI thread, i receive a dead lock.
private readonly ObservebleCollection<String> parameters = new ObservebleCollection<String>();
private readonly SemaphoreSlim semaphore = new SemaphoreSlim(0, 1);
//Called from UI
public ObservebleCollection<String> Parameters
{
get
{
semaphore.Wait();
var result = this.parameters;
semaphore.Release();
return result;
}
}
public async Task Operation()
{
await semaphore.WaitAsync();
List<String> stored = new List<String>();
foreach (var parameter in currentRobot.GetParametersProvider().GetParameters())
{
stored.Add(parameter.PropertyName);
}
//Can't do add all items in UI at once, because it's take a long time, and ui started lag
foreach (var model in stored)
{
await UIDispatcher.BeginInvoke(new Action(() =>
{
this.parameters.Add(model);
}), System.Windows.Threading.DispatcherPriority.Background);
}
semaphore.Release();
}
And how i received a dead lock:
When i click a button in my program, Operation executed.
When i click a another button, program try access to Parameters property.
And i received a dead lock =D
Problem: in async operation i fill a observeblecollection via Dispatcher.BeginInvoke for each item separately, because if i add all items at once using Dispatcher, UI will lag. So i need a synchronization method for access to a Parameters property, which will wait until Operation ends.

await ensures the code after it will run in the original Synchronization context, in this case, in the UI thread. This makes BeginInvoke unnecessary as the code already runs on the correct thread.
The result is that you are trying to acquire two locks on the same object from the same thread, resulting in deadlock.
If you want thread-safe access to a collection of objects, avoid manually creating locks and use a thread-safe collection like ConcurrentQueue or ConcurrentDictionary.
Apart from that, I can't say I understand what the code tries to achieve as it does nothing in the background or asynchronously. It could easily be a simple method that copies parameters from one collection to another and it would still be thread safe if written properly. You could just write:
var _parameters=new ConcurrentQueue<string>();
....
public void CopyParameters()
{
foreach (var parameter in currentRobot.GetParametersProvider().GetParameters())
{
_parameters.Enqueue(parameter.PropertyName);
}
}
If you use databinding on the Parameters property, just raise PropertyChanged after you add all entries
What is the real problem you are trying to solve?
UPDATE
It seems the real problem is the UI freezes if you try to add too many items at a time. This isn't a threading problem, it's a WPF problem. There are various solutions, all of which involve raising PropertyChanged only after you finish adding all properties.
If you don't need the old values, just create a list with the new values, replace the old values then raise the PropertyChanged event, eg:
private ObservebleCollection<String> _parameters = new ObservebleCollection<String>();
public ObservebleCollection<String> Parameters
{
get
{
return _parameters;
}
private set
{
_parameters=value;
PropertyChanged("Parameters");
}
public void CopyParameters()
{
var newParameters=currentRobot.GetParametersProvider()
.GetParameters()
.Select(p=>p.PropertyName);
Parameters=new ObservableCollection<string>(newParameters);
}
Unless you have code that modified Parameters one item at a time though, you could easily swap ObservableCollection for any other collection type, even a string[] array.
Another option is to subclass ObservableCollection to add support for AddRange, as shown in this SO question

Is there such a synchronization tool as "single-item-sized async task buffer"?

Many times in UI development I handle events in such a way that when an event first comes - I immediately start processing, but if there is one processing operation in progress - I wait for it to complete before I process another event. If more than one event occurs before the operation completes - I only process the most recent one.
The way I typically do that my process method has a loop and in my event handler I check a field that indicates if I am currently processing something and if I am - I put my current event arguments in another field that is basically a one item sized buffer and when current processing pass completes - I check if there is some other event to process and I loop until I am done.
Now this seems a bit too repetitive and possibly not the most elegant way to do it, though it seems to otherwise work fine for me. I have two questions then:
Does what I need to do have a name?
Is there some reusable synchronization type out there that could do that for me?
I'm thinking of adding something to the set of async coordination primitives by Stephen Toub that I included in my toolkit.

So first, we'll handle the case that you described in which the method is always used from the UI thread, or some other synchronization context. The Run method can itself be async to handle all of the marshaling through the synchronization context for us.
If we're running we just set the next stored action. If we're not, then we indicate that we're now running, await the action, and then continue to await the next action until there is no next action. We ensure that whenever we're done we indicate that we're done running:
public class EventThrottler
{
private Func<Task> next = null;
private bool isRunning = false;
public async void Run(Func<Task> action)
{
if (isRunning)
next = action;
else
{
isRunning = true;
try
{
await action();
while (next != null)
{
var nextCopy = next;
next = null;
await nextCopy();
}
}
finally
{
isRunning = false;
}
}
}
private static Lazy<EventThrottler> defaultInstance =
new Lazy<EventThrottler>(() => new EventThrottler());
public static EventThrottler Default
{
get { return defaultInstance.Value; }
}
}
Because the class is, at least generally, going to be used exclusively from the UI thread there will generally need to be only one, so I added a convenience property of a default instance, but since it may still make sense for there to be more than one in a program, I didn't make it a singleton.
Run accepts a Func<Task> with the idea that it would generally be an async lambda. It might look like:
public class Foo
{
public void SomeEventHandler(object sender, EventArgs args)
{
EventThrottler.Default.Run(async () =>
{
await Task.Delay(1000);
//do other stuff
});
}
}
Okay, so, just to be verbose, here is a version that handles the case where the event handlers are called from different threads. I know you said that you assume they're all called from the UI thread, but I generalized it a bit. This means locking over all access to instance fields of the type in a lock block, but not actually executing the function inside of a lock block. That last part is important not just for performance, to ensure we're not blocking items from just setting the next field, but also to avoid issues with that action also calling run, so that it doesn't need to deal with re-entrancy issues or potential deadlocks. This pattern, of doing stuff in a lock block and then responding based on conditions determined in the lock means setting local variables to indicate what should be done after the lock ends.
public class EventThrottlerMultiThreaded
{
private object key = new object();
private Func<Task> next = null;
private bool isRunning = false;
public void Run(Func<Task> action)
{
bool shouldStartRunning = false;
lock (key)
{
if (isRunning)
next = action;
else
{
isRunning = true;
shouldStartRunning = true;
}
}
Action<Task> continuation = null;
continuation = task =>
{
Func<Task> nextCopy = null;
lock (key)
{
if (next != null)
{
nextCopy = next;
next = null;
}
else
{
isRunning = false;
}
}
if (nextCopy != null)
nextCopy().ContinueWith(continuation);
};
if (shouldStartRunning)
action().ContinueWith(continuation);
}
}

Does what I need to do have a name?
What you're describing sounds a bit like a trampoline combined with a collapsing queue. A trampoline is basically a loop that iteratively invokes thunk-returning functions. An example is the CurrentThreadScheduler in the Reactive Extensions. When an item is scheduled on a CurrentThreadScheduler, the work item is added to the scheduler's thread-local queue, after which one of the following things will happen:
If the trampoline is already running (i.e., the current thread is already processing the thread-local queue), then the Schedule() call returns immediately.
If the trampoline is not running (i.e., no work items are queued/running on the current thread), then the current thread begins processing the items in the thread-local queue until it is empty, at which point the call to Schedule() returns.
A collapsing queue accumulates items to be processed, with the added twist that if an equivalent item is already in the queue, then that item is simply replaced with the newer item (resulting in only the most recent of the equivalent items remaining in the queue, as opposed to both). The idea is to avoid processing stale/obsolete events. Consider a consumer of market data (e.g., stock ticks). If you receive several updates for a frequently traded security, then each update renders the earlier updates obsolete. There is likely no point in processing earlier ticks for the same security if a more recent tick has already arrived. Thus, a collapsing queue is appropriate.
In your scenario, you essentially have a trampoline processing a collapsing queue with for which all incoming events are considered equivalent. This results in an effective maximum queue size of 1, as every item added to a non-empty queue will result in the existing item being evicted.
Is there some reusable synchronization type out there that could do that for me?
I do not know of an existing solution that would serve your needs, but you could certainly create a generalized trampoline or event loop capable of supporting pluggable scheduling strategies. The default strategy could use a standard queue, while other strategies might use a priority queue or a collapsing queue.

What you're describing sounds very similar to how TPL Dataflow's BrodcastBlock behaves: it always remembers only the last item that you sent to it. If you combine it with ActionBlock that executes your action and has capacity only for the item currently being processed, you get what you want (the method needs a better name):
// returns send delegate
private static Action<T> CreateProcessor<T>(Action<T> executedAction)
{
var broadcastBlock = new BroadcastBlock<T>(null);
var actionBlock = new ActionBlock<T>(
executedAction, new ExecutionDataflowBlockOptions { BoundedCapacity = 1 });
broadcastBlock.LinkTo(actionBlock);
return item => broadcastBlock.Post(item);
}
Usage could be something like this:
var processor = CreateProcessor<int>(
i =>
{
Console.WriteLine(i);
Thread.Sleep(i);
});
processor(100);
processor(1);
processor(2);
Output:
100
2

Exit a loop if another thread enter in the

I've a multi-threading issue.
I've a method that is called to make refresh on several items.
In this method, I iterate on a list of items and refresh one of it's property.
The list has a lot of elements and we have to do some math to compute it's property.
The current code of this operation look like this:
public void AddItemsWithLayoutRefresh(IEnumerable<MyItem> items){
_control.Invoke(()=>{
AddItems(items);
for(int i =0;i<_guiItems.Count;i++){
//The goal is to have a condition here to "break" the loop and let the next call to RefreshLayout proceed
_guiItems[i].Propriety = ComputePropriety(_guiItems[i]);
}
});
}
The problem is that I may have 4 call, which are currently just blocking on the Invoke.
I've to finish the "AddItems" methods, but concerning everything that is in the "for" loop, I can abort this without any issue if I know that it will be executed just after.
But how to do this in a thread-safe way?
If I put a private bool _isNewRefreshHere;, set to true before entering the Invoke, then checking in the Invoke, I've no warranty that there is not already two call that have reach the Invoke BEFORE I check it in the for loop.
So how can I break when being in my loop when a new call is made to my method?
Solution
Based on Andrej Mohar's answer, I did the following:
private long m_refreshQueryCount;
public void AddItemsWithLayoutRefresh(IEnumerable<MyItem> items){
Interlocked.Increment(ref m_refreshQueryCount);
_control.Invoke(()=>{
Interlocked.Decrement(ref m_refreshQueryCount);
AddItems(items);
for(int i =0;i<_guiItems.Count;i++){
if (Interlocked.Read(ref m_refreshQueryCount) > 0)
{
break;
}
_guiItems[i].Propriety = ComputePropriety(_guiItems[i]);
}
});
}
Which seems to work very nicely

If I were you, I'd try to make a thread-safe waiting counter. You can use Interlocked methods like Increment and Decrement. What these basically do is they increment the value as an atomic operation, which is considered to be thread-safe. So you increase the variable before the Invoke call. This will allow you to know how many threads are in the waiting queue. You decrement the variable after the for loop finishes and before the ending of the Invoke block. You can then check inside the for statement for the number of waiting threads and break the for if the number is greater than 1. This way you should know exactly how many threads are in the execution chain.

I would do it in the following way:
private readonly object _refresherLock = new object();
private bool _isNewRefreshHere = false;
private AutoResetEvent _refresher = new AutoResetEvent(true);
public void AddItemsWithLayoutRefresh(IEnumerable<MyItem> items)
{
lock (_refresherLock)
{
if (_isNewRefreshHere)
{
return;
}
_isNewRefreshHere = true;
}
_refresher.WaitOne();
_isNewRefreshHere = false;
_control.Invoke(() =>
{
AddItems(items);
for (int i = 0; i < _guiItems.Count && !_isNewRefreshHere; i++)
{
_guiItems[i].Propriety = ComputePropriety(_guiItems[i]);
}
_refresher.Set();
});
}
That is:
You can always cancel the current updation with a new one.
You cannot queue up more than one updation at a time.
You are guaranteed to have no cross-threading conflicts.
You should test that code since I did not. :)

Share a List<T> between multiple threads

Am I right in saying that I only need to use lock to Add/Remove/Change the List, or do I also need to lock it when iterating over it?
So am I thread safe by doing this:
class ItemsList
{
List<int> items = new List<int>();
object listLock = new object();
public void Add(int item)
{
lock (listLock)
{
items.Add(item);
}
}
public void Remove(int item)
{
lock (listLock)
{
items.Remove(item);
}
}
public void IncrementAll()
{
foreach (var item in items)
{
item += 1;
}
}
}

You should definitely lock when iterating over it too - if the list is changed while you're iterating over it, an exception will be thrown.
From the docs for List<T>.GetEnumerator:
The enumerator does not have exclusive access to the collection; therefore, enumerating through a collection is intrinsically not a thread-safe procedure. To guarantee thread safety during enumeration, you can lock the collection during the entire enumeration. To allow the collection to be accessed by multiple threads for reading and writing, you must implement your own synchronization.
Additionally, even a single read from a List<T> isn't thread-safe if you could be writing to it as well - even if it doesn't fail, there's no guarantee that you'll get the most recent value.
Basically, List<T> is only safe for multiple threads if it's not written to after the last point at which its state becomes visible to all threads.
If you want a thread-safe collection, and if you're using .NET 4 or higher, take a look at the System.Collections.Concurrent namespace.

List<T> is not thread-safe generally. Having multiple readers will not cause any issues, however, you cannot write to the list while it is being read. So you would need to lock on both read and write or use something like a System.Threading.ReaderWriterLock (which allows multiple readers but only one writer). If you are developing under .NET 4.0 or bigger, you could use a BlockingCollection instead, which is a thread safe collection.

No, that isn't safe. You will get a "collection was modified" kind of exception if another thread modifies it while you are reading it.
The most efficient way to fix this is to use a ReaderWriterLockSlim to control access, so that multiple threads can be reading it simultaneously, and it will only get locked when something tries to modify it.

You're not even thread safe with what you have if you never iterate it.
You need to define what types of operations you are doing with the data structure before we can discuss whether or not it will work as intended.
In the general case though, you do need to lock while reading. As it is, someone could add an item while you're in the middle of iterating and it would break all kinds of things. Even reading a single item could be broken if you added an item in the middle of the read.
Also note that this would, at best, make each operation logically atomic. If you're ever performing multiple operations and making assumptions about the state of the data structure then that won't be enough.
In many cases, to resolve this issue, you need to do your locking on the caller side, rather than just wrapping each operation in a lock.

You should probably use a ReaderWriterLockSlim so that multiple threads can read the collection, but only one can modify it.

On IncrementAll you will catch InvalidOperationException because of the changes, made in collection. You can see it in test unit, like this:
ItemsList il = new ItemsList();
Task ts = new Task(() =>
{
for (int i = 0; i < 100000; i++)
{
il.Add(i);
System.Threading.Thread.Sleep(100);
}
}
);
ts.Start();
Task ts2 = new Task(() =>
{
//DoSomeActivity
il.IncrementAll();
}
);
ts2.Start();
Console.Read();
Iteration must be locked also!!!

You might want to take a look at ConcurrentQueue<>();
This is basically a thread safe list (As far as i'm aware), that's rather handy. You can use it a bit like this;
public ConcurrentQueue<yourType> alarmQueue = new ConcurrentQueue<yourType>();
System.Timers.Timer timer;
public QueueManager()
{
timer = new System.Timers.Timer(1000);
timer.Elapsed += new System.Timers.ElapsedEventHandler(timer_Elapsed);
timer.Enabled = true;
}
void timer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
DeQueueAlarm();
}
private void DeQueueAlarm()
{
yourType yourtype;
while (alarmQueue.TryDequeue(out yourtype))
{
//dostuff
}
}
edit: Just as John said, this is available in .Net4 onwards. Read more here; http://msdn.microsoft.com/en-us/library/dd267265.aspx

How to visualize a blockingcollection

I think i have some problems with logic right now.
I have used a blocking collection to make thread safe calls to other PCs. In general it looks like this:
public class MyClass
{
private BlockingCollection<workUnit> workUnits = new BlockingCollection<workUnit>();
public void EnQueue(workUnit item)
{
workUnits.Add(item);
}
private void DeQueue()
{
while (!stopFlag)
{
workUnit item = workUnits.Take();
DoLongRunningDBStuff(workUnit);
}
}
}
Now I want to visualize this to a user.
A user should see that
items are in Queue
item processing has started
result of processing (mainly passed/failed/exception)
And now I got some headache.
I was thinking to do the following:
Have a Grid to display the items to users.
If item is Enqueued add it to workunits and additionally to a list bound to the datagrid
If item is Dequeued (consumed) update the item in the list for the grid.
What makes the headache, is how to make this thread safe, and which parts are needed to be thread safe.
If I put something which takes time behind workUnit.Add I think it could be possible, that data gets mixed.
Would something like this be feasible?
If item is Enqueued add it to workunits and an additional BlockingCollection for UI
If item is Dequeued, make a tryget on 2. BlockingCollection and remove it there, update the status and attach it to second list again.
Would I need an additional lock aound 1 and 2? If so, wouldn't it block completly the add if waiting for Take?
Is there an easy solution or approach to visualize, what is going on?

I will try do it this way:
public class MyClass
{
private BlockingCollection<workUnit> workUnits = new BlockingCollection<workUnit>();
public void EnQueue(workUnit item)
{
workUnits.Add(item);
}
private void DeQueue()
{
while (!stopFlag)
{
workUnit item = workUnits.Take();
item.SetState("Processing Started");
try
{
DoLongRunningDBStuff(workUnit);
item.SetState("Processing Successful");
}
catch
{
item.SetState("Processing Failed");
}
}
}
}
in this example I would then make workItem.SetState(...) fire an event that will update UI for the particular item. However, because the event is raised in a non-UI thread, it will be the handler of the event (the form displaying the grid I would assume) that would need to post the update into the context of the UI thread (e.g. If you are using WinForms you would call the Invoke method of the control displaying the data).
In another (preferred) suggestion I would do the following (if you can use the TPL in .NET 4.0 and later):
public class MyClass
{
public Task EnQueue(workUnit item)
{
// Schedule the work on the thread pool.
// If you need limited concurrency here, there are schedulers to enable this.
return Task.Run(() => DoLongRunningDBStuff(item));
}
}
And if you use .NET 4.5 you would be able to use the await feature that would automatically synchronise the continuation of the task in the context of the UI thread. E.g. in the on the caller's side (assuming it is initiated on the UI thread) you would simply do the following:
private async void btnAddItem_Click(object sender, EventArgs e)
{
var item = new workUnit();
// TODO: Add item on UI here
try
{
await myClass.EnQueue(item);
// TODO: Update UI with success result here (no context synchronisation is needed here it is already in the UI context)
}
catch
{
// TODO: Update UI with error result here (no context synchronisation is needed here it is already in the UI context)
}
}
In both examples you do not even need any locking, you simply need to have the updates posted to the correct context (and in the last example that is not even explicitly needed, the compiler takes care of it for you)