We Keep Coding

BlockingCollection where the consumers are also producers

I have a bunch of requests to process, and during the processing of those requests, more "sub-requests" can be generated and added to the same blocking collection. The consumers add sub-requests to the queue.
It's hard to know when to exit the consuming loop: clearly no thread can call BlockingCollection.CompleteAdding as the other threads may add something to the collection. You also cannot exit the consuming loop just because the BlockingCollection is empty as another thread may have just read the final remaining request from the BlockingCollection and will be about to start generating more requests - the Count of the BlockingCollection will then increase from zero again.
My only idea on this so far is to use a Barrier - when all threads reach the Barrier, there can't be anything left in the BlockingCollection and no thread can be generating new requests. Here is my code - is this an acceptable approach? (and please note: this is highly contrived block of code modelling a much more complex situation: no programmer really writes code that processes random strings 😊 )
using System;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Collections.Concurrent;
using System.Threading;
namespace Barrier1
{
class Program
{
private static readonly Random random = new Random();
private static void Main()
{
var bc = new BlockingCollection<string>();
AddRandomStringsToBc(bc, 1000, true);
int nTasks = 4;
var barrier = new Barrier(nTasks);
Action a = () => DoSomething(bc, barrier);
var actions = Enumerable.Range(0, nTasks).Select(x => a).ToArray();
Parallel.Invoke(actions);
}
private static IEnumerable<char> GetC(bool includeA)
{
var startChar = includeA ? 'A' : 'B';
var add = includeA ? 24 : 25;
while (true)
{
yield return (char)(startChar + random.Next(add));
}
}
private static void DoSomething(BlockingCollection<string> bc, Barrier barrier)
{
while (true)
{
if (bc.TryTake(out var str))
{
Console.WriteLine(str);
if (str[0] == 'A')
{
Console.WriteLine("Adding more strings...");
AddRandomStringsToBc(bc, 100);
}
}
else
{
// Can't exit the loop here just because there is nothing in the collection.
// A different thread may be just about to call AddRandomStringsToBc:
if (barrier.SignalAndWait(100))
{
break;
}
}
}
}
private static void AddRandomStringsToBc(BlockingCollection<string> bc, int n, bool startWithA = false, bool sleep = false)
{
var collection = Enumerable.Range(0, n).Select(x => string.Join("", GetC(startWithA).Take(5)));
foreach (var c in collection)
{
bc.Add(c);
}
}
}
}

Here is a collection similar to the BlockingCollection<T>, with the difference that it completes automatically instead of relying on manually calling the CompleteAdding method. The condition for the automatic completion is that the collection is empty, and all the consumers are in a waiting state.
The implementation is based on your clever idea of using a Barrier as a mechanism for checking the auto-complete condition. It's not perfect because it relies on pooling, which is taking place when the collection becomes empty and has some consumers that are still active. On the other hand it allows to exploit all the existing functionality of the BlockingCollection<T> class, instead of rewriting it from scratch:
/// <summary>
/// A blocking collection that completes automatically when it's empty, and all
/// consuming enumerables are in a waiting state.
/// </summary>
public class AutoCompleteBlockingCollection<T> : IEnumerable<T>, IDisposable
{
private readonly BlockingCollection<T> _queue;
private readonly Barrier _barrier;
private volatile bool _autoCompleteStarted;
private volatile int _intervalMilliseconds = 500;
public AutoCompleteBlockingCollection(int boundedCapacity = -1)
{
_queue = boundedCapacity == -1 ? new() : new(boundedCapacity);
_barrier = new(0, _ => _queue.CompleteAdding());
}
public int Count => _queue.Count;
public int BoundedCapacity => _queue.BoundedCapacity;
public bool IsAddingCompleted => _queue.IsAddingCompleted;
public bool IsCompleted => _queue.IsCompleted;
/// <summary>
/// Begin observing the condition for automatic completion.
/// </summary>
public void BeginObservingAutoComplete() => _autoCompleteStarted = true;
/// <summary>
/// Gets or sets how frequently to check for the auto-complete condition.
/// </summary>
public TimeSpan CheckAutoCompleteInterval
{
get { return TimeSpan.FromMilliseconds(_intervalMilliseconds); }
set
{
int milliseconds = checked((int)value.TotalMilliseconds);
if (milliseconds < 0) throw new ArgumentOutOfRangeException();
_intervalMilliseconds = milliseconds;
}
}
public void Add(T item, CancellationToken cancellationToken = default)
=> _queue.Add(item, cancellationToken);
public bool TryAdd(T item) => _queue.TryAdd(item);
public IEnumerable<T> GetConsumingEnumerable(
CancellationToken cancellationToken = default)
{
_barrier.AddParticipant();
try
{
while (true)
{
if (!_autoCompleteStarted)
{
if (_queue.TryTake(out var item, _intervalMilliseconds,
cancellationToken))
yield return item;
}
else
{
if (_queue.TryTake(out var item, 0, cancellationToken))
yield return item;
else if (_barrier.SignalAndWait(_intervalMilliseconds,
cancellationToken))
break;
}
}
}
finally { _barrier.RemoveParticipant(); }
}
IEnumerator<T> IEnumerable<T>.GetEnumerator()
=> ((IEnumerable<T>)_queue).GetEnumerator();
IEnumerator IEnumerable.GetEnumerator()
=> ((IEnumerable<T>)_queue).GetEnumerator();
public void Dispose() { _barrier.Dispose(); _queue.Dispose(); }
}
The BeginObservingAutoComplete method should be called after adding the initial items in the collection. Before calling this method, the auto-complete condition is not checked.
The CheckAutoCompleteInterval is 500 milliseconds by default, and it can be configured at any time.
The Take and TryTake methods are missing on purpose. The collection is intended to be consumed via the GetConsumingEnumerable method. This way the collection keeps track of the currently subscribed consumers, in order to know when to auto-complete. Consumers can be added and removed at any time. A consumer can be removed by exiting the foreach loop, either by break/return etc, or by an exception.
Usage example:
private static void Main()
{
var bc = new AutoCompleteBlockingCollection<string>();
AddRandomStringsToBc(bc, 1000, true);
bc.BeginObservingAutoComplete();
Action action = () => DoSomething(bc);
var actions = Enumerable.Repeat(action, 4).ToArray();
Parallel.Invoke(actions);
}
private static void DoSomething(AutoCompleteBlockingCollection<string> bc)
{
foreach (var str in bc.GetConsumingEnumerable())
{
Console.WriteLine(str);
if (str[0] == 'A')
{
Console.WriteLine("Adding more strings...");
AddRandomStringsToBc(bc, 100);
}
}
}
The collection is thread-safe, with the exception of the Dispose method.

ZeroMQ PUB/SUB Pattern with Multi-Threaded Poller Cancellation

I have two applications, a C++ server, and a C# WPF UI. The C++ code takes requests (from anywhere/anyone) via a ZeroMQ messaging [PUB/SUB] service. I use my C# code for back testing and to create "back tests" and execute them. These back tests can be made up of many "unit tests" and each of these sending/receiving thousands of messages from the C++ server.
Currently individual back tests work well can send off N unit tests each with thousands of requests and captures. My problem is architecture; when I dispatch another back test (following the first) I get a problem with event subscription being done a second time due to the polling thread not being cancelled and disposed. This results in erroneous output. This may seem like a trivial problem (perhaps it is for some of you), but the cancellation of this polling Task under my current configuration is proving troublesome. Some code...
My message broker class is simple and looks like
public class MessageBroker : IMessageBroker<Taurus.FeedMux>, IDisposable
{
private Task pollingTask;
private NetMQContext context;
private PublisherSocket pubSocket;
private CancellationTokenSource source;
private CancellationToken token;
private ManualResetEvent pollerCancelled;
public MessageBroker()
{
this.source = new CancellationTokenSource();
this.token = source.Token;
StartPolling();
context = NetMQContext.Create();
pubSocket = context.CreatePublisherSocket();
pubSocket.Connect(PublisherAddress);
}
public void Dispatch(Taurus.FeedMux message)
{
pubSocket.Send(message.ToByteArray<Taurus.FeedMux>());
}
private void StartPolling()
{
pollerCancelled = new ManualResetEvent(false);
pollingTask = Task.Run(() =>
{
try
{
using (var context = NetMQContext.Create())
using (var subSocket = context.CreateSubscriberSocket())
{
byte[] buffer = null;
subSocket.Options.ReceiveHighWatermark = 1000;
subSocket.Connect(SubscriberAddress);
subSocket.Subscribe(String.Empty);
while (true)
{
buffer = subSocket.Receive();
MessageRecieved.Report(buffer.ToObject<Taurus.FeedMux>());
if (this.token.IsCancellationRequested)
this.token.ThrowIfCancellationRequested();
}
}
}
catch (OperationCanceledException)
{
pollerCancelled.Set();
}
}, this.token);
}
private void CancelPolling()
{
source.Cancel();
pollerCancelled.WaitOne();
pollerCancelled.Close();
}
public IProgress<Taurus.FeedMux> MessageRecieved { get; set; }
public string PublisherAddress { get { return "tcp://127.X.X.X:6500"; } }
public string SubscriberAddress { get { return "tcp://127.X.X.X:6501"; } }
private bool disposed = false;
protected virtual void Dispose(bool disposing)
{
if (!disposed)
{
if (disposing)
{
if (this.pollingTask != null)
{
CancelPolling();
if (this.pollingTask.Status == TaskStatus.RanToCompletion ||
this.pollingTask.Status == TaskStatus.Faulted ||
this.pollingTask.Status == TaskStatus.Canceled)
{
this.pollingTask.Dispose();
this.pollingTask = null;
}
}
if (this.context != null)
{
this.context.Dispose();
this.context = null;
}
if (this.pubSocket != null)
{
this.pubSocket.Dispose();
this.pubSocket = null;
}
if (this.source != null)
{
this.source.Dispose();
this.source = null;
}
}
disposed = true;
}
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
~MessageBroker()
{
Dispose(false);
}
}
The backtesting "engine" use to execute each back test, first constructs a Dictionary containing each Test (unit test) and the messages to dispatch to the C++ application for each test.
The DispatchTests method, here it is
private void DispatchTests(ConcurrentDictionary<Test, List<Taurus.FeedMux>> feedMuxCollection)
{
broker = new MessageBroker();
broker.MessageRecieved = new Progress<Taurus.FeedMux>(OnMessageRecieved);
testCompleted = new ManualResetEvent(false);
try
{
// Loop through the tests.
foreach (var kvp in feedMuxCollection)
{
testCompleted.Reset();
Test t = kvp.Key;
t.Bets = new List<Taurus.Bet>();
foreach (Taurus.FeedMux mux in kvp.Value)
{
token.ThrowIfCancellationRequested();
broker.Dispatch(mux);
}
broker.Dispatch(new Taurus.FeedMux()
{
type = Taurus.FeedMux.Type.PING,
ping = new Taurus.Ping() { event_id = t.EventID }
});
testCompleted.WaitOne(); // Wait until all messages are received for this test.
}
testCompleted.Close();
}
finally
{
broker.Dispose(); // Dispose the broker.
}
}
The PING message at the end, it to tell the C++ that we are finished. We then force a wait, so that the next [unit] test is not dispatched before all of the returns are received from the C++ code - we do this using a ManualResetEvent.
When the C++ receives the PING message, it sends the message straight back. We handle the received messages via OnMessageRecieved and the PING tells us to set the ManualResetEvent.Set() so that we can continue the unit testing; "Next Please"...
private async void OnMessageRecieved(Taurus.FeedMux mux)
{
string errorMsg = String.Empty;
if (mux.type == Taurus.FeedMux.Type.MSG)
{
// Do stuff.
}
else if (mux.type == Taurus.FeedMux.Type.PING)
{
// Do stuff.
// We are finished reciving messages for this "unit test"
testCompleted.Set();
}
}
My problem is that, broker.Dispose() in the finally above is never hit. I appreciate that finally blocks that are executed on background threads are not guaranteed to get executed.
The crossed out text above was due to me messing about with the code; I was stopping a parent thread before the child had completed. However, there are still problems...
Now broker.Dispose() is called correctly, and broker.Dispose() is called, in this method I attempt to cancell the poller thread and dispose of the Task correctly to avoid any multiple subscriptions.
To cancel the thread I use the CancelPolling() method
private void CancelPolling()
{
source.Cancel();
pollerCancelled.WaitOne(); <- Blocks here waiting for cancellation.
pollerCancelled.Close();
}
but in the StartPolling() method
while (true)
{
buffer = subSocket.Receive();
MessageRecieved.Report(buffer.ToObject<Taurus.FeedMux>());
if (this.token.IsCancellationRequested)
this.token.ThrowIfCancellationRequested();
}
ThrowIfCancellationRequested() is never called and the thread is never cancelled, thus never properly disposed. The poller thread is being blocked by the subSocket.Receive() method.
Now, it is not clear to me how to achieve what I want, I need to invoke the broker.Dispose()/PollerCancel() on a thread other than that used to poll for messages and some how force the cancellation. Thread abort is not what I want to get into at any cost.
Essentially, I want to properly dispose of the broker before executing the next back test, how do I correctly handle this, split out the polling and run it in a separate Application Domain?
I have tried, disposing inside the OnMessageRecived handler, but this is clearly executed on the same thread as the poller and is not the way to do this, without invoking additional threads, it blocks.
What is the best way to achieve what I want and is there a pattern for this sort of case that I can follow?
Thanks for your time.

This is how I eventually got around this [although I am open to a better solution!]
public class FeedMuxMessageBroker : IMessageBroker<Taurus.FeedMux>, IDisposable
{
// Vars.
private NetMQContext context;
private PublisherSocket pubSocket;
private Poller poller;
private CancellationTokenSource source;
private CancellationToken token;
private ManualResetEvent pollerCancelled;
/// <summary>
/// Default ctor.
/// </summary>
public FeedMuxMessageBroker()
{
context = NetMQContext.Create();
pubSocket = context.CreatePublisherSocket();
pubSocket.Connect(PublisherAddress);
pollerCancelled = new ManualResetEvent(false);
source = new CancellationTokenSource();
token = source.Token;
StartPolling();
}
#region Methods.
/// <summary>
/// Send the mux message to listners.
/// </summary>
/// <param name="message">The message to dispatch.</param>
public void Dispatch(Taurus.FeedMux message)
{
pubSocket.Send(message.ToByteArray<Taurus.FeedMux>());
}
/// <summary>
/// Start polling for messages.
/// </summary>
private void StartPolling()
{
Task.Run(() =>
{
using (var subSocket = context.CreateSubscriberSocket())
{
byte[] buffer = null;
subSocket.Options.ReceiveHighWatermark = 1000;
subSocket.Connect(SubscriberAddress);
subSocket.Subscribe(String.Empty);
subSocket.ReceiveReady += (s, a) =>
{
buffer = subSocket.Receive();
if (MessageRecieved != null)
MessageRecieved.Report(buffer.ToObject<Taurus.FeedMux>());
};
// Poll.
poller = new Poller();
poller.AddSocket(subSocket);
poller.PollTillCancelled();
token.ThrowIfCancellationRequested();
}
}, token).ContinueWith(ant =>
{
pollerCancelled.Set();
}, TaskContinuationOptions.OnlyOnCanceled);
}
/// <summary>
/// Cancel polling to allow the broker to be disposed.
/// </summary>
private void CancelPolling()
{
source.Cancel();
poller.Cancel();
pollerCancelled.WaitOne();
pollerCancelled.Close();
}
#endregion // Methods.
#region Properties.
/// <summary>
/// Event that is raised when a message is recived.
/// </summary>
public IProgress<Taurus.FeedMux> MessageRecieved { get; set; }
/// <summary>
/// The address to use for the publisher socket.
/// </summary>
public string PublisherAddress { get { return "tcp://127.0.0.1:6500"; } }
/// <summary>
/// The address to use for the subscriber socket.
/// </summary>
public string SubscriberAddress { get { return "tcp://127.0.0.1:6501"; } }
#endregion // Properties.
#region IDisposable Members.
private bool disposed = false;
/// <summary>
/// Dispose managed resources.
/// </summary>
/// <param name="disposing">Is desposing.</param>
protected virtual void Dispose(bool disposing)
{
if (!disposed)
{
if (disposing)
{
CancelPolling();
if (pubSocket != null)
{
pubSocket.Disconnect(PublisherAddress);
pubSocket.Dispose();
pubSocket = null;
}
if (poller != null)
{
poller.Dispose();
poller = null;
}
if (context != null)
{
context.Terminate();
context.Dispose();
context = null;
}
if (source != null)
{
source.Dispose();
source = null;
}
}
// Shared cleanup logic.
disposed = true;
}
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
/// <summary>
/// Finalizer.
/// </summary>
~FeedMuxMessageBroker()
{
Dispose(false);
}
#endregion // IDisposable Members.
}
So we poll in the same way, but using the Poller class from NetMQ. In the Task continuation we set so we are sure that both the Poller and Task are cancelled. We are then safe to dispose...

A higher-level view on subject
Your focus and efforts, dedicated to creating a testing framework, signal that your will aims at developing a rigorous and professional-grade approach, which has made me first raise my hat in a salute of admiration to such brave undertaking.
While testing is an important activity for providing a reasonable quantitative evidence, that a System Under Test is meeting defined expectations, the success in this depends on how close the testing environment meets the real-deployment's conditions.
One may agree, that testing on another, different, bases does not prove the real-deployment will run as expected in an environment, that is principally different from the tested one(s).
Element-wise control or just a state-wise control, that's the question.
Your efforts ( at least at the time of OP was posted ) concentrate on code-architecture, that tries to keep instances in-place and tries to re-set an internal state of a Poller instance before a next test-battery starts.
In my view, testing has a few principles to follow, should you strive for professional testing:
Principle of a Test Repeatability ( tests' re-runs shall serve same results, thus avoiding a quasi-testing that provides just a result-"lottery" )
Principle of Non-Intervening Testing ( tests' re-runs shall not be subject of "external"-interference, not controlled by the test scenario )
Having said this, let me bring a few notes inspired by Harry Markowitz, Nobelist awarded for his remarkable quantitative portfolio optimisation studies.
Rather move one step back to get control over elements' full life-cycle
CACI Simulations, Inc., ( one of Harry Markowitz's companies ) developed in the early 90s their flagship software framework COMET III - an exceptionally powerful simulation engine for large, complex design-prototyping and performance-simulations of processes operated in large-scale computing / networking / telco networks.
The greatest impression from COMET III was it's capability to generate testing scenarios including a configurable pre-test "warm-up" pre-load(s), that have made the tested elements get into a state similar to what "fatigue" means in mechanical torture-test experiments or what hydrogen-diffusion fragility means to nuclear power-plant metallurgists.
Yes, once you go into low-level details on how algorithms, node-buffers, memory-allocations, pipe-lined / load-balanced / grid-processing architecture selections, fault-resilience overheads, garbage collection policies and limited resource-sharing algorithms work and impact ( under real-use work-load patterns "pressure" ) end-to-end performance / latencies, this feature is simply indispensable.
This means, that an individual instance-related simple state-wise control is not sufficient, as it does not provide means for either the test-repeatability and test-isolation/non-intervening behaviour. Simply put, even if you find a way to "reset" a Poller instance, this will not get you into realistic testing with guaranteed test-repeatability with pre-test warm-up(s) possible.
A step-back and a higher layer of abstraction and test-scenario controls is needed.
How does this apply to the OP problem?
Instead of just state-control
Create a multi-layer-ed architecture / control-plane(s) / separate signalling
A ZeroMQ way of supporting this goal
Create super-structures as non-trivial patterns
Use full life-cycle controls of instances used inside testing scenarios
Keep ZeroMQ-maxims: Zero-sharing, Zero-blocking, ...
Benefit from Multi-Context()

Hot Concat in Rx

Observable.Concat is an implementation that joins observables but the second IObservable<T> only makes the subscription when the first is completed.
http://www.introtorx.com/content/v1.0.10621.0/12_CombiningSequences.html#Concat
Is there any implementation of a "HotConcat"? Similar to Observable.Merge, but keeping the delivery order, first pushing the elements of initial subscription and then the subsequents. Something like:
I know that is possible to use the ReplaySubject<T>, but it doesn't seems so good, because of performance and memory usage impacts..

Here's the implementation I've been using for a while. This implementation introduces a BufferUntilSubscribed operator that turns an IObservable into a IConnectableObservable that will start buffering whenever you call Connect and will deliver the buffered results to the first subscriber. Once the first subscriber has "caught up", then the buffering will stop and the subscriber will just be given the live events as they arrive.
Once you have that, you can write HotConcat as something like:
public static IObservable<T> HotConcat<T>(params IObservable<T>[] sources)
{
var s2 = sources.Select(s => s.BufferUntilSubscribed());
var subscriptions = new CompositeDisposable(s2.Select(s2 => s2.Connect()).ToArray());
return Observable.Create<T>(observer =>
{
var s = new SingleAssignmentDisposable();
var d = new CompositeDisposable(subscriptions);
d.Add(s);
s.Disposable = s2.Concat().Subscribe(observer);
return d;
});
}
Here's the implemementation of BufferUntilSubscribed:
private class BufferUntilSubscribedObservable<T> : IConnectableObservable<T>
{
private readonly IObservable<T> _source;
private readonly IScheduler _scheduler;
private readonly Subject<T> _liveEvents;
private bool _observationsStarted;
private Queue<T> _buffer;
private readonly object _gate;
public BufferUntilSubscribedObservable(IObservable<T> source, IScheduler scheduler)
{
_source = source;
_scheduler = scheduler;
_liveEvents = new Subject<T>();
_buffer = new Queue<T>();
_gate = new object();
_observationsStarted = false;
}
public IDisposable Subscribe(IObserver<T> observer)
{
lock (_gate)
{
if (_observationsStarted)
{
return _liveEvents.Subscribe(observer);
}
_observationsStarted = true;
var bufferedEvents = GetBuffers().Concat().Finally(RemoveBuffer); // Finally clause to remove the buffer if the first observer stops listening.
return Observable.Merge(_liveEvents, bufferedEvents).Subscribe(observer);
}
}
public IDisposable Connect()
{
return _source.Subscribe(OnNext, _liveEvents.OnError, _liveEvents.OnCompleted);
}
private void RemoveBuffer()
{
lock (_gate)
{
_buffer = null;
}
}
/// <summary>
/// Acquires a lock and checks the buffer. If it is empty, then replaces it with null and returns null. Else replaces it with an empty buffer and returns the old buffer.
/// </summary>
/// <returns></returns>
private Queue<T> GetAndReplaceBuffer()
{
lock (_gate)
{
if (_buffer == null)
{
return null;
}
if (_buffer.Count == 0)
{
_buffer = null;
return null;
}
var result = _buffer;
_buffer = new Queue<T>();
return result;
}
}
/// <summary>
/// An enumerable of buffers that will complete when a call to GetAndReplaceBuffer() returns a null, e.g. when the observer has caught up with the incoming source data.
/// </summary>
/// <returns></returns>
private IEnumerable<IObservable<T>> GetBuffers()
{
Queue<T> buffer;
while ((buffer = GetAndReplaceBuffer()) != null)
{
yield return buffer.ToObservable(_scheduler);
}
}
private void OnNext(T item)
{
lock (_gate)
{
if (_buffer != null)
{
_buffer.Enqueue(item);
return;
}
}
_liveEvents.OnNext(item);
}
}
/// <summary>
/// Returns a connectable observable, that once connected, will start buffering data until the observer subscribes, at which time it will send all buffered data to the observer and then start sending new data.
/// Thus the observer may subscribe late to a hot observable yet still see all of the data. Later observers will not see the buffered events.
/// </summary>
/// <param name="source"></param>
/// <param name="scheduler">Scheduler to use to dump the buffered data to the observer.</param>
/// <returns></returns>
public static IConnectableObservable<T> BufferUntilSubscribed<T>(this IObservable<T> source, IScheduler scheduler)
{
return new BufferUntilSubscribedObservable<T>(source, scheduler);
}
/// <summary>
/// Returns a connectable observable, that once connected, will start buffering data until the observer subscribes, at which time it will send all buffered data to the observer and then start sending new data.
/// Thus the observer may subscribe late to a hot observable yet still see all of the data. Later observers will not see the buffered events.
/// </summary>
/// <param name="source"></param>
/// <returns></returns>
public static IConnectableObservable<T> BufferUntilSubscribed<T>(this IObservable<T> source)
{
return new BufferUntilSubscribedObservable<T>(source, Scheduler.Immediate);
}

I don't know of such a composition function, but you can write one that matches your needs.
Here's my attempt at writing one. It will keep elements in memory only until they are replayed once. But I think there should be a way to make a cleaner implementation though.
public static IObservable<T> HotConcat<T>(this IObservable<T> first, IObservable<T> second)
{
return Observable.Create<T>(observer =>
{
var queue = new Queue<Notification<T>>();
var secondSubscription = second.Materialize().Subscribe(item =>
{
if (queue == null)
return;
lock (queue)
{
queue.Enqueue(item);
}
});
var secondReplay = Observable.Create<T>(secondObserver =>
{
while (true)
{
Notification<T> item = null;
lock (queue)
{
if (queue.Count > 0)
{
item = queue.Dequeue();
}
else
{
secondObserver.OnCompleted();
secondSubscription.Dispose();
queue = null;
break;
}
}
if (item != null)
item.Accept(secondObserver);
}
return secondSubscription;
});
return first.Concat(secondReplay).Concat(second).Subscribe(observer);
});
}

Efficient signaling Tasks for TPL completions on frequently reoccuring events

I'm working on a simulation system that, among other things, allows for the execution of tasks in discrete simulated time steps. Execution all occurs in the context of the simulation thread, but, from the perspective of an 'operator' using the system, they wish to behave asynchronously. Thankfully the TPL, with the handy 'async/await' keywords, makes this fairly straightforward. I have a primitive method on the Simulation like this:
public Task CycleExecutedEvent()
{
lock (_cycleExecutedBroker)
{
if (!IsRunning) throw new TaskCanceledException("Simulation has been stopped");
return _cycleExecutedBroker.RegisterForCompletion(CycleExecutedEventName);
}
}
This is basically creating a new TaskCompletionSource and then returning a Task. The purpose of this Task is to execute its continuation when the new 'ExecuteCycle' on the simulation occurs.
I then have some extension methods like this:
public static async Task WaitForDuration(this ISimulation simulation, double duration)
{
double startTime = simulation.CurrentSimulatedTime;
do
{
await simulation.CycleExecutedEvent();
} while ((simulation.CurrentSimulatedTime - startTime) < duration);
}
public static async Task WaitForCondition(this ISimulation simulation, Func<bool> condition)
{
do
{
await simulation.CycleExecutedEvent();
} while (!condition());
}
These are very handy, then, for building sequences from an 'operator' perspective, taking actions based on conditions and waiting for periods of simulated time. The issue I'm running into is that CycleExecuted occurs very frequently (roughly every few milliseconds if I'm running at fully accelerated speed). Because these 'wait' helper methods register a new 'await' on each cycle, this causes a large turnover in TaskCompletionSource instances.
I've profiled my code and I've found that roughly 5.5% of my total CPU time is spent within these completions, of which only a negligible percentage is spent in the 'active' code. Effectively all of the time is spent registering new completions while waiting for the triggering conditions to be valid.
My question: how can I improve performance here while still retaining the convenience of the async/await pattern for writing 'operator behaviors'? I'm thinking I need something like a lighter-weight and/or reusable TaskCompletionSource, given that the triggering event occurs so frequently.
I've been doing a bit more research and it sounds like a good option would be to create a custom implementation of the Awaitable pattern, which could tie directly into the event, eliminating the need for a bunch of TaskCompletionSource and Task instances. The reason it could be useful here is that there are a lot of different continuations awaiting the CycleExecutedEvent and they need to await it frequently. So ideally I'm looking at a way to just queue up continuation callbacks, then call back everything in the queue whenever the event occurs. I'll keep digging, but I welcome any help if folks know a clean way to do this.
For anybody browsing this question in the future, here is the custom awaiter I put together:
public sealed class CycleExecutedAwaiter : INotifyCompletion
{
private readonly List<Action> _continuations = new List<Action>();
public bool IsCompleted
{
get { return false; }
}
public void GetResult()
{
}
public void OnCompleted(Action continuation)
{
_continuations.Add(continuation);
}
public void RunContinuations()
{
var continuations = _continuations.ToArray();
_continuations.Clear();
foreach (var continuation in continuations)
continuation();
}
public CycleExecutedAwaiter GetAwaiter()
{
return this;
}
}
And in the Simulator:
private readonly CycleExecutedAwaiter _cycleExecutedAwaiter = new CycleExecutedAwaiter();
public CycleExecutedAwaiter CycleExecutedEvent()
{
if (!IsRunning) throw new TaskCanceledException("Simulation has been stopped");
return _cycleExecutedAwaiter;
}
It's a bit funny, as the awaiter never reports Complete, but fires continues to call completions as they are registered; still, it works well for this application. This reduces the CPU overhead from 5.5% to 2.1%. It will likely still require some tweaking, but it's a nice improvement over the original.

The await keyword doesn't work just on Tasks, it works on anything that follows the awaitable pattern. For details, see Stephen Toub's article await anything;.
The short version is that the type has to have a method GetAwaiter() that returns a type that implements INotifyCompletion and also has IsCompleted property and GetResult() method (void-returning, if the await expression shouldn't have a value). For an example, see TaskAwaiter.
If you create your own awaitable, you could return the same object every time, avoiding the overhead of allocating many TaskCompletionSources.

Here is my version of ReusableAwaiter simulating TaskCompletionSource
public sealed class ReusableAwaiter<T> : INotifyCompletion
{
private Action _continuation = null;
private T _result = default(T);
private Exception _exception = null;
public bool IsCompleted
{
get;
private set;
}
public T GetResult()
{
if (_exception != null)
throw _exception;
return _result;
}
public void OnCompleted(Action continuation)
{
if (_continuation != null)
throw new InvalidOperationException("This ReusableAwaiter instance has already been listened");
_continuation = continuation;
}
/// <summary>
/// Attempts to transition the completion state.
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
public bool TrySetResult(T result)
{
if (!this.IsCompleted)
{
this.IsCompleted = true;
this._result = result;
if (_continuation != null)
_continuation();
return true;
}
return false;
}
/// <summary>
/// Attempts to transition the exception state.
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
public bool TrySetException(Exception exception)
{
if (!this.IsCompleted)
{
this.IsCompleted = true;
this._exception = exception;
if (_continuation != null)
_continuation();
return true;
}
return false;
}
/// <summary>
/// Reset the awaiter to initial status
/// </summary>
/// <returns></returns>
public ReusableAwaiter<T> Reset()
{
this._result = default(T);
this._continuation = null;
this._exception = null;
this.IsCompleted = false;
return this;
}
public ReusableAwaiter<T> GetAwaiter()
{
return this;
}
}
And here is the test code.
class Program
{
static readonly ReusableAwaiter<int> _awaiter = new ReusableAwaiter<int>();
static void Main(string[] args)
{
Task.Run(() => Test());
Console.ReadLine();
_awaiter.TrySetResult(22);
Console.ReadLine();
_awaiter.TrySetException(new Exception("ERR"));
Console.ReadLine();
}
static async void Test()
{
int a = await AsyncMethod();
Console.WriteLine(a);
try
{
await AsyncMethod();
}
catch(Exception ex)
{
Console.WriteLine(ex.Message);
}
}
static ReusableAwaiter<int> AsyncMethod()
{
return _awaiter.Reset();
}
}

Do you really need to receive the WaitForDuration-event on a different thread? If not, you could just register a callback (or an event) with _cycleExecutedBroker and receive notification synchronously. In the callback you can test any condition you like and only if that condition turns out to be true, notify a different thread (using a task or message or whatever mechanism). I understand the condition you test for rarely evaluates to true, so you avoid most cross-thread calls that way.
I guess the gist of my answer is: Try to reduce the amount of cross-thread messaging by moving computation to the "source" thread.

Wait for pooled threads to complete

I'm sorry for a redundant question. However, I've found many solutions to my problem but none of them are very well explained. I'm hoping that it will be made clear, here.
My C# application's main thread spawns 1..n background workers using the ThreadPool. I wish for the original thread to lock until all of the workers have completed. I have researched the ManualResetEvent in particular but I'm not clear on it's use.
In pseudo:
foreach( var o in collection )
{
queue new worker(o);
}
while( workers not completed ) { continue; }
If necessary, I will know the number of workers that are about to be queued before hand.

Try this. The function takes in a list of Action delegates. It will add a ThreadPool worker entry for each item in the list. It will wait for every action to complete before returning.
public static void SpawnAndWait(IEnumerable<Action> actions)
{
var list = actions.ToList();
var handles = new ManualResetEvent[actions.Count()];
for (var i = 0; i < list.Count; i++)
{
handles[i] = new ManualResetEvent(false);
var currentAction = list[i];
var currentHandle = handles[i];
Action wrappedAction = () => { try { currentAction(); } finally { currentHandle.Set(); } };
ThreadPool.QueueUserWorkItem(x => wrappedAction());
}
WaitHandle.WaitAll(handles);
}

Here's a different approach - encapsulation; so your code could be as simple as:
Forker p = new Forker();
foreach (var obj in collection)
{
var tmp = obj;
p.Fork(delegate { DoSomeWork(tmp); });
}
p.Join();
Where the Forker class is given below (I got bored on the train ;-p)... again, this avoids OS objects, but wraps things up quite neatly (IMO):
using System;
using System.Threading;
/// <summary>Event arguments representing the completion of a parallel action.</summary>
public class ParallelEventArgs : EventArgs
{
private readonly object state;
private readonly Exception exception;
internal ParallelEventArgs(object state, Exception exception)
{
this.state = state;
this.exception = exception;
}
/// <summary>The opaque state object that identifies the action (null otherwise).</summary>
public object State { get { return state; } }
/// <summary>The exception thrown by the parallel action, or null if it completed without exception.</summary>
public Exception Exception { get { return exception; } }
}
/// <summary>Provides a caller-friendly wrapper around parallel actions.</summary>
public sealed class Forker
{
int running;
private readonly object joinLock = new object(), eventLock = new object();
/// <summary>Raised when all operations have completed.</summary>
public event EventHandler AllComplete
{
add { lock (eventLock) { allComplete += value; } }
remove { lock (eventLock) { allComplete -= value; } }
}
private EventHandler allComplete;
/// <summary>Raised when each operation completes.</summary>
public event EventHandler<ParallelEventArgs> ItemComplete
{
add { lock (eventLock) { itemComplete += value; } }
remove { lock (eventLock) { itemComplete -= value; } }
}
private EventHandler<ParallelEventArgs> itemComplete;
private void OnItemComplete(object state, Exception exception)
{
EventHandler<ParallelEventArgs> itemHandler = itemComplete; // don't need to lock
if (itemHandler != null) itemHandler(this, new ParallelEventArgs(state, exception));
if (Interlocked.Decrement(ref running) == 0)
{
EventHandler allHandler = allComplete; // don't need to lock
if (allHandler != null) allHandler(this, EventArgs.Empty);
lock (joinLock)
{
Monitor.PulseAll(joinLock);
}
}
}
/// <summary>Adds a callback to invoke when each operation completes.</summary>
/// <returns>Current instance (for fluent API).</returns>
public Forker OnItemComplete(EventHandler<ParallelEventArgs> handler)
{
if (handler == null) throw new ArgumentNullException("handler");
ItemComplete += handler;
return this;
}
/// <summary>Adds a callback to invoke when all operations are complete.</summary>
/// <returns>Current instance (for fluent API).</returns>
public Forker OnAllComplete(EventHandler handler)
{
if (handler == null) throw new ArgumentNullException("handler");
AllComplete += handler;
return this;
}
/// <summary>Waits for all operations to complete.</summary>
public void Join()
{
Join(-1);
}
/// <summary>Waits (with timeout) for all operations to complete.</summary>
/// <returns>Whether all operations had completed before the timeout.</returns>
public bool Join(int millisecondsTimeout)
{
lock (joinLock)
{
if (CountRunning() == 0) return true;
Thread.SpinWait(1); // try our luck...
return (CountRunning() == 0) ||
Monitor.Wait(joinLock, millisecondsTimeout);
}
}
/// <summary>Indicates the number of incomplete operations.</summary>
/// <returns>The number of incomplete operations.</returns>
public int CountRunning()
{
return Interlocked.CompareExchange(ref running, 0, 0);
}
/// <summary>Enqueues an operation.</summary>
/// <param name="action">The operation to perform.</param>
/// <returns>The current instance (for fluent API).</returns>
public Forker Fork(ThreadStart action) { return Fork(action, null); }
/// <summary>Enqueues an operation.</summary>
/// <param name="action">The operation to perform.</param>
/// <param name="state">An opaque object, allowing the caller to identify operations.</param>
/// <returns>The current instance (for fluent API).</returns>
public Forker Fork(ThreadStart action, object state)
{
if (action == null) throw new ArgumentNullException("action");
Interlocked.Increment(ref running);
ThreadPool.QueueUserWorkItem(delegate
{
Exception exception = null;
try { action(); }
catch (Exception ex) { exception = ex;}
OnItemComplete(state, exception);
});
return this;
}
}

First, how long do the workers execute? pool threads should generally be used for short-lived tasks - if they are going to run for a while, consider manual threads.
Re the problem; do you actually need to block the main thread? Can you use a callback instead? If so, something like:
int running = 1; // start at 1 to prevent multiple callbacks if
// tasks finish faster than they are started
Action endOfThread = delegate {
if(Interlocked.Decrement(ref running) == 0) {
// ****run callback method****
}
};
foreach(var o in collection)
{
var tmp = o; // avoid "capture" issue
Interlocked.Increment(ref running);
ThreadPool.QueueUserWorkItem(delegate {
DoSomeWork(tmp); // [A] should handle exceptions internally
endOfThread();
});
}
endOfThread(); // opposite of "start at 1"
This is a fairly lightweight (no OS primitives) way of tracking the workers.
If you need to block, you can do the same using a Monitor (again, avoiding an OS object):
object syncLock = new object();
int running = 1;
Action endOfThread = delegate {
if (Interlocked.Decrement(ref running) == 0) {
lock (syncLock) {
Monitor.Pulse(syncLock);
}
}
};
lock (syncLock) {
foreach (var o in collection) {
var tmp = o; // avoid "capture" issue
ThreadPool.QueueUserWorkItem(delegate
{
DoSomeWork(tmp); // [A] should handle exceptions internally
endOfThread();
});
}
endOfThread();
Monitor.Wait(syncLock);
}
Console.WriteLine("all done");

I have been using the new Parallel task library in CTP here:
Parallel.ForEach(collection, o =>
{
DoSomeWork(o);
});

Here is a solution using the CountdownEvent class.
var complete = new CountdownEvent(1);
foreach (var o in collection)
{
var capture = o;
ThreadPool.QueueUserWorkItem((state) =>
{
try
{
DoSomething(capture);
}
finally
{
complete.Signal();
}
}, null);
}
complete.Signal();
complete.Wait();
Of course, if you have access to the CountdownEvent class then you have the whole TPL to work with. The Parallel class takes care of the waiting for you.
Parallel.ForEach(collection, o =>
{
DoSomething(o);
});

I think you were on the right track with the ManualResetEvent. This link has a code sample that closely matches what your trying to do. The key is to use the WaitHandle.WaitAll and pass an array of wait events. Each thread needs to set one of these wait events.
// Simultaneously calculate the terms.
ThreadPool.QueueUserWorkItem(
new WaitCallback(CalculateBase));
ThreadPool.QueueUserWorkItem(
new WaitCallback(CalculateFirstTerm));
ThreadPool.QueueUserWorkItem(
new WaitCallback(CalculateSecondTerm));
ThreadPool.QueueUserWorkItem(
new WaitCallback(CalculateThirdTerm));
// Wait for all of the terms to be calculated.
WaitHandle.WaitAll(autoEvents);
// Reset the wait handle for the next calculation.
manualEvent.Reset();
Edit:
Make sure that in your worker thread code path you set the event (i.e. autoEvents1.Set();). Once they are all signaled the waitAll will return.
void CalculateSecondTerm(object stateInfo)
{
double preCalc = randomGenerator.NextDouble();
manualEvent.WaitOne();
secondTerm = preCalc * baseNumber *
randomGenerator.NextDouble();
autoEvents[1].Set();
}

I've found a good solution here :
http://msdn.microsoft.com/en-us/magazine/cc163914.aspx
May come in handy for others with the same issue

Using .NET 4.0 Barrier class:
Barrier sync = new Barrier(1);
foreach(var o in collection)
{
WaitCallback worker = (state) =>
{
// do work
sync.SignalAndWait();
};
sync.AddParticipant();
ThreadPool.QueueUserWorkItem(worker, o);
}
sync.SignalAndWait();

Try using CountdownEvent
// code before the threads start
CountdownEvent countdown = new CountdownEvent(collection.Length);
foreach (var o in collection)
{
ThreadPool.QueueUserWorkItem(delegate
{
// do something with the worker
Console.WriteLine("Thread Done!");
countdown.Signal();
});
}
countdown.Wait();
Console.WriteLine("Job Done!");
// resume the code here
The countdown would wait until all threads have finished execution.

Wait for completion of all threads in thread pool there is no inbuilt method available.
Using count no. of threads are active, we can achieve it...
{
bool working = true;
ThreadPool.GetMaxThreads(out int maxWorkerThreads, out int maxCompletionPortThreads);
while (working)
{
ThreadPool.GetAvailableThreads(out int workerThreads, out int completionPortThreads);
//Console.WriteLine($"{workerThreads} , {maxWorkerThreads}");
if (workerThreads == maxWorkerThreads)
{ working = false; }
}
//when all threads are completed then 'working' will be false
}
void xyz(object o)
{
console.writeline("");
}