I'm working on a simulation system that, among other things, allows for the execution of tasks in discrete simulated time steps. Execution all occurs in the context of the simulation thread, but, from the perspective of an 'operator' using the system, they wish to behave asynchronously. Thankfully the TPL, with the handy 'async/await' keywords, makes this fairly straightforward. I have a primitive method on the Simulation like this:
public Task CycleExecutedEvent()
{
lock (_cycleExecutedBroker)
{
if (!IsRunning) throw new TaskCanceledException("Simulation has been stopped");
return _cycleExecutedBroker.RegisterForCompletion(CycleExecutedEventName);
}
}
This is basically creating a new TaskCompletionSource and then returning a Task. The purpose of this Task is to execute its continuation when the new 'ExecuteCycle' on the simulation occurs.
I then have some extension methods like this:
public static async Task WaitForDuration(this ISimulation simulation, double duration)
{
double startTime = simulation.CurrentSimulatedTime;
do
{
await simulation.CycleExecutedEvent();
} while ((simulation.CurrentSimulatedTime - startTime) < duration);
}
public static async Task WaitForCondition(this ISimulation simulation, Func<bool> condition)
{
do
{
await simulation.CycleExecutedEvent();
} while (!condition());
}
These are very handy, then, for building sequences from an 'operator' perspective, taking actions based on conditions and waiting for periods of simulated time. The issue I'm running into is that CycleExecuted occurs very frequently (roughly every few milliseconds if I'm running at fully accelerated speed). Because these 'wait' helper methods register a new 'await' on each cycle, this causes a large turnover in TaskCompletionSource instances.
I've profiled my code and I've found that roughly 5.5% of my total CPU time is spent within these completions, of which only a negligible percentage is spent in the 'active' code. Effectively all of the time is spent registering new completions while waiting for the triggering conditions to be valid.
My question: how can I improve performance here while still retaining the convenience of the async/await pattern for writing 'operator behaviors'? I'm thinking I need something like a lighter-weight and/or reusable TaskCompletionSource, given that the triggering event occurs so frequently.
I've been doing a bit more research and it sounds like a good option would be to create a custom implementation of the Awaitable pattern, which could tie directly into the event, eliminating the need for a bunch of TaskCompletionSource and Task instances. The reason it could be useful here is that there are a lot of different continuations awaiting the CycleExecutedEvent and they need to await it frequently. So ideally I'm looking at a way to just queue up continuation callbacks, then call back everything in the queue whenever the event occurs. I'll keep digging, but I welcome any help if folks know a clean way to do this.
For anybody browsing this question in the future, here is the custom awaiter I put together:
public sealed class CycleExecutedAwaiter : INotifyCompletion
{
private readonly List<Action> _continuations = new List<Action>();
public bool IsCompleted
{
get { return false; }
}
public void GetResult()
{
}
public void OnCompleted(Action continuation)
{
_continuations.Add(continuation);
}
public void RunContinuations()
{
var continuations = _continuations.ToArray();
_continuations.Clear();
foreach (var continuation in continuations)
continuation();
}
public CycleExecutedAwaiter GetAwaiter()
{
return this;
}
}
And in the Simulator:
private readonly CycleExecutedAwaiter _cycleExecutedAwaiter = new CycleExecutedAwaiter();
public CycleExecutedAwaiter CycleExecutedEvent()
{
if (!IsRunning) throw new TaskCanceledException("Simulation has been stopped");
return _cycleExecutedAwaiter;
}
It's a bit funny, as the awaiter never reports Complete, but fires continues to call completions as they are registered; still, it works well for this application. This reduces the CPU overhead from 5.5% to 2.1%. It will likely still require some tweaking, but it's a nice improvement over the original.
The await keyword doesn't work just on Tasks, it works on anything that follows the awaitable pattern. For details, see Stephen Toub's article await anything;.
The short version is that the type has to have a method GetAwaiter() that returns a type that implements INotifyCompletion and also has IsCompleted property and GetResult() method (void-returning, if the await expression shouldn't have a value). For an example, see TaskAwaiter.
If you create your own awaitable, you could return the same object every time, avoiding the overhead of allocating many TaskCompletionSources.
Here is my version of ReusableAwaiter simulating TaskCompletionSource
public sealed class ReusableAwaiter<T> : INotifyCompletion
{
private Action _continuation = null;
private T _result = default(T);
private Exception _exception = null;
public bool IsCompleted
{
get;
private set;
}
public T GetResult()
{
if (_exception != null)
throw _exception;
return _result;
}
public void OnCompleted(Action continuation)
{
if (_continuation != null)
throw new InvalidOperationException("This ReusableAwaiter instance has already been listened");
_continuation = continuation;
}
/// <summary>
/// Attempts to transition the completion state.
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
public bool TrySetResult(T result)
{
if (!this.IsCompleted)
{
this.IsCompleted = true;
this._result = result;
if (_continuation != null)
_continuation();
return true;
}
return false;
}
/// <summary>
/// Attempts to transition the exception state.
/// </summary>
/// <param name="result"></param>
/// <returns></returns>
public bool TrySetException(Exception exception)
{
if (!this.IsCompleted)
{
this.IsCompleted = true;
this._exception = exception;
if (_continuation != null)
_continuation();
return true;
}
return false;
}
/// <summary>
/// Reset the awaiter to initial status
/// </summary>
/// <returns></returns>
public ReusableAwaiter<T> Reset()
{
this._result = default(T);
this._continuation = null;
this._exception = null;
this.IsCompleted = false;
return this;
}
public ReusableAwaiter<T> GetAwaiter()
{
return this;
}
}
And here is the test code.
class Program
{
static readonly ReusableAwaiter<int> _awaiter = new ReusableAwaiter<int>();
static void Main(string[] args)
{
Task.Run(() => Test());
Console.ReadLine();
_awaiter.TrySetResult(22);
Console.ReadLine();
_awaiter.TrySetException(new Exception("ERR"));
Console.ReadLine();
}
static async void Test()
{
int a = await AsyncMethod();
Console.WriteLine(a);
try
{
await AsyncMethod();
}
catch(Exception ex)
{
Console.WriteLine(ex.Message);
}
}
static ReusableAwaiter<int> AsyncMethod()
{
return _awaiter.Reset();
}
}
Do you really need to receive the WaitForDuration-event on a different thread? If not, you could just register a callback (or an event) with _cycleExecutedBroker and receive notification synchronously. In the callback you can test any condition you like and only if that condition turns out to be true, notify a different thread (using a task or message or whatever mechanism). I understand the condition you test for rarely evaluates to true, so you avoid most cross-thread calls that way.
I guess the gist of my answer is: Try to reduce the amount of cross-thread messaging by moving computation to the "source" thread.
Related
The question may sound a bit abstract, here's an example of a non async piece of code that does some lazy loading and ensures that the lazy loading is only done once by the first thread:
public class LazyExample
{
private object ExpensiveResource = null;
private object ExpensiveResourceSyncRoot = new object();
public object GetExpensiveResource()
{
if (ExpensiveResource != null) // Checkpoint 1
return ExpensiveResource;
lock (ExpensiveResourceSyncRoot) // initially there will be a queue of threads waiting here that have already passed Checkpoint 1
{
if (ExpensiveResource != null) // prevent re-retrieval by all subsequent threads that passed Checkpoint 1
return ExpensiveResource;
// create the expensive resource but do not assign yet...
object expensiveResource = new object();
// initialize the expensive resource, for example:
// - Call an API...
// - Do some Database lookups...
Thread.Sleep(1);
// finally as last step before releasing the lock, assign the fully initialized expensive object
ExpensiveResource = expensiveResource;
}
return ExpensiveResource;
}
}
In our lib, the async virus has started to infest many calls. Since awaiting is not allowed directly inside a lock we now wrap the async stuff in a new method like so:
public class LazyExample
{
private object ExpensiveResource = null;
private object ExpensiveResourceSyncRoot = new object();
public async Task<object> GetExpensiveResourceAsync()
{
if (ExpensiveResource != null) // Checkpoint 1
return ExpensiveResource;
lock (ExpensiveResourceSyncRoot) // initially there will be a queue of threads waiting here that have already passed Checkpoint 1
{
if (ExpensiveResource != null) // prevent re-retrieval by all subsequent threads that passed Checkpoint 1
return ExpensiveResource;
// assign the fully initialized expensive object
ExpensiveResource = CreateAndInitializeExpensiveResourceAsync().Result;
}
return ExpensiveResource;
}
private async Task<object> CreateAndInitializeExpensiveResourceAsync()
{
object expensiveResource = new object();
// initialize the expensive resource, this time async:
await Task.Delay(1);
return expensiveResource;
}
}
This however feels like putting a zip-tie around a safety latch and defeating the cause.
In seemingly random cases we need to wrap a call in order to prevent deadlocks like so:
ExpensiveResource = Task.Run(CreateAndInitializeExpensiveResourceAsync).Result;
This will force the method to run in a different thread and cause the current thread to go idle until it joins (extra overhead for no good reason as far as I can tell).
So my question is: is it safe to offload async stuff to a separate method (a new stack frame if you will) inside a lock or are we indeed defeating a safety latch?
Synchronously waiting for your async operation to finish, putting a lock around that, and then punting that off to the ThreadPool to make it async again is a solution, but it's just about the worst option out there.
First, why not use an async lock? SemaphoreSlim is the classic:
public class LazyExample
{
private object ExpensiveResource = null;
private readonly SemaphoreSlim ExpensiveResourceSyncRoot = new(1, 1);
public async Task<object> GetExpensiveResourceAsync()
{
if (ExpensiveResource != null) // Checkpoint 1
return ExpensiveResource;
await ExpensiveResourceSyncRoot.WaitAsync();
try
{
if (ExpensiveResource != null) // prevent re-retrieval by all subsequent threads that passed Checkpoint 1
return ExpensiveResource;
// finally as last step before releasing the lock, assign the fully initialized expensive object
ExpensiveResource = await CreateAndInitializeExpensiveResourceAsync();
}
finally
{
ExpensiveResourceSyncRoot.Release();
}
return ExpensiveResource;
}
}
Secondly, there's a much better solution to this problem of lazy async initialisation. Instead of caching ExpensiveResource, cache a Task which represents the initialisation of ExpensiveResource. When a consumer awaits that Task, either they'll find that it's already complete, in which case the resource is available immediately, or they'll have to wait, in which case the resource is still being created, and by awaiting the Task they'll be notified when it is available.
public class LazyExample
{
private Task<object> ExpensiveResource = null;
private readonly object ExpensiveResourceSyncRoot = new();
// Note this is not async, and does not contain awaits
public Task<object> GetExpensiveResourceAsync()
{
if (ExpensiveResource != null)
return ExpensiveResource;
lock (ExpensiveResourceSyncRoot)
{
if (ExpensiveResource != null)
return ExpensiveResource;
// Note that we don't await this.
// ExpensiveResource holds the Task which represents the creation of the resource
ExpensiveResource = CreateAndInitializeExpensiveResourceAsync();
}
return ExpensiveResource;
}
}
We can further simplify with a Lazy<T>:
public class LazyExample
{
private readonly Lazy<Task<object>> ExpensiveResource;
public LazyExample()
{
ExpensiveResource = new(CreateAndInitializeExpensiveResourceAsync);
}
public Task<object> GetExpensiveResourceAsync() => ExpensiveResource.Value;
}
When trying to answer the following question, I wrote this piece of code :
using static MyNameSpace.Locker; //So that we don't need to specify the static class name before each call.
public class MainClass
{
public MainMethod()
{
Lock(new object()).Lock(new object()).RunAction(() => Console.WriteLine("Finished"));
}
}
public static class Locker
{
public static async Task<List<object>> Lock(object toLock, int timeout = -1)
{
await Task.Run(() => TryEnter(toLock, timeout));
return new List<object>() { toLock };
}
public static async Task<List<object>> Lock(
this Task<List<object>> lockedChain,
object toLock,
int timeout = -1)
{
await Task.Run(() => TryEnter(toLock, timeout));
await lockedChain;
lockedChain.Result.Add(toLock)
return lockedChain.Result;
}
public static async void RunAction(this Task<List<object>> lockChain, Action toRun)
{
await lockChain;
try
{
toRun.Invoke();
}
finally
{
foreach (var chainMember in lockChain.Result)
{
Monitor.Exit(chainMember);
}
}
}
private static void TryEnter(object toLock, int timeout = -1)
{
var success = false;
if (timeout > 0)
{
success = Monitor.TryEnter(toLock, timeout);
}
else
{
success = Monitor.TryEnter(toLock);
}
if (!success)
{
throw new TimeoutException();
}
}
}
But as some user rightfully remarked, this won't work for a very simple reason : Since the methods are async, they may not run on the same thread, thus throwing an exception when trying to release the Monitor.
How would one go to ensure the Enter and Exit method of the monitor a run on the same thread ?
Instead of forcing the lock operations onto the same thread which is nearly impossible, use a lock that is not thread-affine: SemaphoreSlim. It has native async support as well (as opposed to blocking).
In the original question that you linked to I'd go with this answer instead. Seems cleaner than the chain solution which contains a lot of artificial complexity. Code quality is not so much about the specific call syntax being used. Just by putting things in a syntactic chain you cannot reduce complexity much.
In particular the chain solution is just a complicated way of saying Lock(new [] { lock1, lock2 }, () => ...); I think. All the chain does it build up a list. using makes this even simpler because it does away with the lambda. Lambdas are less composable because you can't return from the lambda like you can from using. I think you should target this:
using (MultiLock(new [] { lock1, lock2 }, timeout)) {
//...
}
(Apologies if this was answered elsewhere; it seems like it would be a common problem, but it turns out to be hard to search for since terms like "threading" and "cache" produce overwhelming results.)
I have an expensive computation whose result is accessed frequently but changes infrequently. Thus, I cache the resulting value. Here's some c# pseudocode of what I mean:
int? _cachedResult = null;
int GetComputationResult()
{
if(_cachedResult == null)
{
// Do the expensive computation.
_cachedResult = /* Result of expensive computation. */;
}
return _cachedResult.Value;
}
Elsewhere in my code, I will occasionally set _cachedResult back to null because the input to the computation has changed and thus the cached result is no longer valid and needs to be re-computed. (Which means I can't use Lazy<T> since Lazy<T> doesn't support being reset.)
This works fine for single-threaded scenarios, but of course it's not at all thread-safe. So my question is: What is the most performant way to make GetComputationResult thread-safe?
Obviously I could just put the whole thing in a lock() block, but I suspect there might be a better way? (Something that would do an atomic check to see if the result needs to be recomputed and only lock if it does?)
Thanks a lot!
You can use the double-checked locking pattern:
// Thread-safe (uses double-checked locking pattern for performance)
public class Memoized<T>
{
Func<T> _compute;
volatile bool _cached;
volatile bool _startedCaching;
volatile StrongBox<T> _cachedResult; // Need reference type
object _cacheSyncRoot = new object();
public Memoized(Func<T> compute)
{
_compute = compute;
}
public T Value {
get {
if (_cached) // Fast path
return _cachedResult.Value;
lock (_cacheSyncRoot)
{
if (!_cached)
{
_startedCaching = true;
_cachedResult = new StrongBox<T>(_compute());
_cached = true;
}
}
return _cachedResult.Value;
}
}
public void Invalidate()
{
if (!_startedCaching)
{
// Fast path: already invalidated
Thread.MemoryBarrier(); // need to release
if (!_startedCaching)
return;
}
lock (_cacheSyncRoot)
_cached = _startedCaching = false;
}
}
This particular implementation matches your description of what it should do in corner cases: If the cache has been invalidated, the value should only be computed once, by a single thread, and other threads should wait. However, if the cache is invalidated concurrently with the cached value being accessed, the stale cached value may be returned.
perhaps this will provide some food for thought:).
Generic class.
The class can compute data asynchronously or synchronously.
Allows fast reads thanks to the spinlock.
Does not perform heavy stuff inside the spinlock, just returning Task and if necessary, creating and starting Task on default TaskScheduler, to avoid inlining.
Task with Spinlock is pretty powerful combination, that can solve some problems in lock-free way.
using System;
using System.Threading;
using System.Threading.Tasks;
namespace Example
{
class OftenReadSometimesUpdate<T>
{
private Task<T> result_task = null;
private SpinLock spin_lock = new SpinLock(false);
private TResult LockedFunc<TResult>(Func<TResult> locked_func)
{
TResult t_result = default(TResult);
bool gotLock = false;
if (locked_func == null) return t_result;
try
{
spin_lock.Enter(ref gotLock);
t_result = locked_func();
}
finally
{
if (gotLock) spin_lock.Exit();
gotLock = false;
}
return t_result;
}
public Task<T> GetComputationAsync()
{
return
LockedFunc(GetComputationTaskLocked)
;
}
public T GetComputationResult()
{
return
LockedFunc(GetComputationTaskLocked)
.Result
;
}
public OftenReadSometimesUpdate<T> InvalidateComputationResult()
{
return
this
.LockedFunc(InvalidateComputationResultLocked)
;
}
public OftenReadSometimesUpdate<T> InvalidateComputationResultLocked()
{
result_task = null;
return this;
}
private Task<T> GetComputationTaskLocked()
{
if (result_task == null)
{
result_task = new Task<T>(HeavyComputation);
result_task.Start(TaskScheduler.Default);
}
return result_task;
}
protected virtual T HeavyComputation()
{
//a heavy computation
return default(T);//return some result of computation
}
}
}
You could simply reassign the Lazy<T> to achieve a reset:
Lazy<int> lazyResult = new Lazy<int>(GetComputationResult);
public int Result { get { return lazyResult.Value; } }
public void Reset()
{
lazyResult = new Lazy<int>(GetComputationResult);
}
I have two applications, a C++ server, and a C# WPF UI. The C++ code takes requests (from anywhere/anyone) via a ZeroMQ messaging [PUB/SUB] service. I use my C# code for back testing and to create "back tests" and execute them. These back tests can be made up of many "unit tests" and each of these sending/receiving thousands of messages from the C++ server.
Currently individual back tests work well can send off N unit tests each with thousands of requests and captures. My problem is architecture; when I dispatch another back test (following the first) I get a problem with event subscription being done a second time due to the polling thread not being cancelled and disposed. This results in erroneous output. This may seem like a trivial problem (perhaps it is for some of you), but the cancellation of this polling Task under my current configuration is proving troublesome. Some code...
My message broker class is simple and looks like
public class MessageBroker : IMessageBroker<Taurus.FeedMux>, IDisposable
{
private Task pollingTask;
private NetMQContext context;
private PublisherSocket pubSocket;
private CancellationTokenSource source;
private CancellationToken token;
private ManualResetEvent pollerCancelled;
public MessageBroker()
{
this.source = new CancellationTokenSource();
this.token = source.Token;
StartPolling();
context = NetMQContext.Create();
pubSocket = context.CreatePublisherSocket();
pubSocket.Connect(PublisherAddress);
}
public void Dispatch(Taurus.FeedMux message)
{
pubSocket.Send(message.ToByteArray<Taurus.FeedMux>());
}
private void StartPolling()
{
pollerCancelled = new ManualResetEvent(false);
pollingTask = Task.Run(() =>
{
try
{
using (var context = NetMQContext.Create())
using (var subSocket = context.CreateSubscriberSocket())
{
byte[] buffer = null;
subSocket.Options.ReceiveHighWatermark = 1000;
subSocket.Connect(SubscriberAddress);
subSocket.Subscribe(String.Empty);
while (true)
{
buffer = subSocket.Receive();
MessageRecieved.Report(buffer.ToObject<Taurus.FeedMux>());
if (this.token.IsCancellationRequested)
this.token.ThrowIfCancellationRequested();
}
}
}
catch (OperationCanceledException)
{
pollerCancelled.Set();
}
}, this.token);
}
private void CancelPolling()
{
source.Cancel();
pollerCancelled.WaitOne();
pollerCancelled.Close();
}
public IProgress<Taurus.FeedMux> MessageRecieved { get; set; }
public string PublisherAddress { get { return "tcp://127.X.X.X:6500"; } }
public string SubscriberAddress { get { return "tcp://127.X.X.X:6501"; } }
private bool disposed = false;
protected virtual void Dispose(bool disposing)
{
if (!disposed)
{
if (disposing)
{
if (this.pollingTask != null)
{
CancelPolling();
if (this.pollingTask.Status == TaskStatus.RanToCompletion ||
this.pollingTask.Status == TaskStatus.Faulted ||
this.pollingTask.Status == TaskStatus.Canceled)
{
this.pollingTask.Dispose();
this.pollingTask = null;
}
}
if (this.context != null)
{
this.context.Dispose();
this.context = null;
}
if (this.pubSocket != null)
{
this.pubSocket.Dispose();
this.pubSocket = null;
}
if (this.source != null)
{
this.source.Dispose();
this.source = null;
}
}
disposed = true;
}
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
~MessageBroker()
{
Dispose(false);
}
}
The backtesting "engine" use to execute each back test, first constructs a Dictionary containing each Test (unit test) and the messages to dispatch to the C++ application for each test.
The DispatchTests method, here it is
private void DispatchTests(ConcurrentDictionary<Test, List<Taurus.FeedMux>> feedMuxCollection)
{
broker = new MessageBroker();
broker.MessageRecieved = new Progress<Taurus.FeedMux>(OnMessageRecieved);
testCompleted = new ManualResetEvent(false);
try
{
// Loop through the tests.
foreach (var kvp in feedMuxCollection)
{
testCompleted.Reset();
Test t = kvp.Key;
t.Bets = new List<Taurus.Bet>();
foreach (Taurus.FeedMux mux in kvp.Value)
{
token.ThrowIfCancellationRequested();
broker.Dispatch(mux);
}
broker.Dispatch(new Taurus.FeedMux()
{
type = Taurus.FeedMux.Type.PING,
ping = new Taurus.Ping() { event_id = t.EventID }
});
testCompleted.WaitOne(); // Wait until all messages are received for this test.
}
testCompleted.Close();
}
finally
{
broker.Dispose(); // Dispose the broker.
}
}
The PING message at the end, it to tell the C++ that we are finished. We then force a wait, so that the next [unit] test is not dispatched before all of the returns are received from the C++ code - we do this using a ManualResetEvent.
When the C++ receives the PING message, it sends the message straight back. We handle the received messages via OnMessageRecieved and the PING tells us to set the ManualResetEvent.Set() so that we can continue the unit testing; "Next Please"...
private async void OnMessageRecieved(Taurus.FeedMux mux)
{
string errorMsg = String.Empty;
if (mux.type == Taurus.FeedMux.Type.MSG)
{
// Do stuff.
}
else if (mux.type == Taurus.FeedMux.Type.PING)
{
// Do stuff.
// We are finished reciving messages for this "unit test"
testCompleted.Set();
}
}
My problem is that, broker.Dispose() in the finally above is never hit. I appreciate that finally blocks that are executed on background threads are not guaranteed to get executed.
The crossed out text above was due to me messing about with the code; I was stopping a parent thread before the child had completed. However, there are still problems...
Now broker.Dispose() is called correctly, and broker.Dispose() is called, in this method I attempt to cancell the poller thread and dispose of the Task correctly to avoid any multiple subscriptions.
To cancel the thread I use the CancelPolling() method
private void CancelPolling()
{
source.Cancel();
pollerCancelled.WaitOne(); <- Blocks here waiting for cancellation.
pollerCancelled.Close();
}
but in the StartPolling() method
while (true)
{
buffer = subSocket.Receive();
MessageRecieved.Report(buffer.ToObject<Taurus.FeedMux>());
if (this.token.IsCancellationRequested)
this.token.ThrowIfCancellationRequested();
}
ThrowIfCancellationRequested() is never called and the thread is never cancelled, thus never properly disposed. The poller thread is being blocked by the subSocket.Receive() method.
Now, it is not clear to me how to achieve what I want, I need to invoke the broker.Dispose()/PollerCancel() on a thread other than that used to poll for messages and some how force the cancellation. Thread abort is not what I want to get into at any cost.
Essentially, I want to properly dispose of the broker before executing the next back test, how do I correctly handle this, split out the polling and run it in a separate Application Domain?
I have tried, disposing inside the OnMessageRecived handler, but this is clearly executed on the same thread as the poller and is not the way to do this, without invoking additional threads, it blocks.
What is the best way to achieve what I want and is there a pattern for this sort of case that I can follow?
Thanks for your time.
This is how I eventually got around this [although I am open to a better solution!]
public class FeedMuxMessageBroker : IMessageBroker<Taurus.FeedMux>, IDisposable
{
// Vars.
private NetMQContext context;
private PublisherSocket pubSocket;
private Poller poller;
private CancellationTokenSource source;
private CancellationToken token;
private ManualResetEvent pollerCancelled;
/// <summary>
/// Default ctor.
/// </summary>
public FeedMuxMessageBroker()
{
context = NetMQContext.Create();
pubSocket = context.CreatePublisherSocket();
pubSocket.Connect(PublisherAddress);
pollerCancelled = new ManualResetEvent(false);
source = new CancellationTokenSource();
token = source.Token;
StartPolling();
}
#region Methods.
/// <summary>
/// Send the mux message to listners.
/// </summary>
/// <param name="message">The message to dispatch.</param>
public void Dispatch(Taurus.FeedMux message)
{
pubSocket.Send(message.ToByteArray<Taurus.FeedMux>());
}
/// <summary>
/// Start polling for messages.
/// </summary>
private void StartPolling()
{
Task.Run(() =>
{
using (var subSocket = context.CreateSubscriberSocket())
{
byte[] buffer = null;
subSocket.Options.ReceiveHighWatermark = 1000;
subSocket.Connect(SubscriberAddress);
subSocket.Subscribe(String.Empty);
subSocket.ReceiveReady += (s, a) =>
{
buffer = subSocket.Receive();
if (MessageRecieved != null)
MessageRecieved.Report(buffer.ToObject<Taurus.FeedMux>());
};
// Poll.
poller = new Poller();
poller.AddSocket(subSocket);
poller.PollTillCancelled();
token.ThrowIfCancellationRequested();
}
}, token).ContinueWith(ant =>
{
pollerCancelled.Set();
}, TaskContinuationOptions.OnlyOnCanceled);
}
/// <summary>
/// Cancel polling to allow the broker to be disposed.
/// </summary>
private void CancelPolling()
{
source.Cancel();
poller.Cancel();
pollerCancelled.WaitOne();
pollerCancelled.Close();
}
#endregion // Methods.
#region Properties.
/// <summary>
/// Event that is raised when a message is recived.
/// </summary>
public IProgress<Taurus.FeedMux> MessageRecieved { get; set; }
/// <summary>
/// The address to use for the publisher socket.
/// </summary>
public string PublisherAddress { get { return "tcp://127.0.0.1:6500"; } }
/// <summary>
/// The address to use for the subscriber socket.
/// </summary>
public string SubscriberAddress { get { return "tcp://127.0.0.1:6501"; } }
#endregion // Properties.
#region IDisposable Members.
private bool disposed = false;
/// <summary>
/// Dispose managed resources.
/// </summary>
/// <param name="disposing">Is desposing.</param>
protected virtual void Dispose(bool disposing)
{
if (!disposed)
{
if (disposing)
{
CancelPolling();
if (pubSocket != null)
{
pubSocket.Disconnect(PublisherAddress);
pubSocket.Dispose();
pubSocket = null;
}
if (poller != null)
{
poller.Dispose();
poller = null;
}
if (context != null)
{
context.Terminate();
context.Dispose();
context = null;
}
if (source != null)
{
source.Dispose();
source = null;
}
}
// Shared cleanup logic.
disposed = true;
}
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
/// <summary>
/// Finalizer.
/// </summary>
~FeedMuxMessageBroker()
{
Dispose(false);
}
#endregion // IDisposable Members.
}
So we poll in the same way, but using the Poller class from NetMQ. In the Task continuation we set so we are sure that both the Poller and Task are cancelled. We are then safe to dispose...
A higher-level view on subject
Your focus and efforts, dedicated to creating a testing framework, signal that your will aims at developing a rigorous and professional-grade approach, which has made me first raise my hat in a salute of admiration to such brave undertaking.
While testing is an important activity for providing a reasonable quantitative evidence, that a System Under Test is meeting defined expectations, the success in this depends on how close the testing environment meets the real-deployment's conditions.
One may agree, that testing on another, different, bases does not prove the real-deployment will run as expected in an environment, that is principally different from the tested one(s).
Element-wise control or just a state-wise control, that's the question.
Your efforts ( at least at the time of OP was posted ) concentrate on code-architecture, that tries to keep instances in-place and tries to re-set an internal state of a Poller instance before a next test-battery starts.
In my view, testing has a few principles to follow, should you strive for professional testing:
Principle of a Test Repeatability ( tests' re-runs shall serve same results, thus avoiding a quasi-testing that provides just a result-"lottery" )
Principle of Non-Intervening Testing ( tests' re-runs shall not be subject of "external"-interference, not controlled by the test scenario )
Having said this, let me bring a few notes inspired by Harry Markowitz, Nobelist awarded for his remarkable quantitative portfolio optimisation studies.
Rather move one step back to get control over elements' full life-cycle
CACI Simulations, Inc., ( one of Harry Markowitz's companies ) developed in the early 90s their flagship software framework COMET III - an exceptionally powerful simulation engine for large, complex design-prototyping and performance-simulations of processes operated in large-scale computing / networking / telco networks.
The greatest impression from COMET III was it's capability to generate testing scenarios including a configurable pre-test "warm-up" pre-load(s), that have made the tested elements get into a state similar to what "fatigue" means in mechanical torture-test experiments or what hydrogen-diffusion fragility means to nuclear power-plant metallurgists.
Yes, once you go into low-level details on how algorithms, node-buffers, memory-allocations, pipe-lined / load-balanced / grid-processing architecture selections, fault-resilience overheads, garbage collection policies and limited resource-sharing algorithms work and impact ( under real-use work-load patterns "pressure" ) end-to-end performance / latencies, this feature is simply indispensable.
This means, that an individual instance-related simple state-wise control is not sufficient, as it does not provide means for either the test-repeatability and test-isolation/non-intervening behaviour. Simply put, even if you find a way to "reset" a Poller instance, this will not get you into realistic testing with guaranteed test-repeatability with pre-test warm-up(s) possible.
A step-back and a higher layer of abstraction and test-scenario controls is needed.
How does this apply to the OP problem?
Instead of just state-control
Create a multi-layer-ed architecture / control-plane(s) / separate signalling
A ZeroMQ way of supporting this goal
Create super-structures as non-trivial patterns
Use full life-cycle controls of instances used inside testing scenarios
Keep ZeroMQ-maxims: Zero-sharing, Zero-blocking, ...
Benefit from Multi-Context()
We have an application that regularly receives multimedia messages, and should reply to them.
We currently do this with a single thread, first receiving messages, and then processing them one by one. This does the job, but is slow.
So we're now thinking of doing the same process but with multiple threads sumultaneously.
Any simple way to allow parallel processing of the incoming records, yet avoid erroneously processing the same record by two threads?
Any simple way to allow parallel processing of the incoming records, yet avoid erroneously processing the same record by two threads?
Yes it is actually not too hard, what you are wanting to do is called the "Producer-Consumer model"
If your message receiver could only handle one thread at a time but your message "processor" can work on multiple messages at once you just need to use a BlockingCollection to store the work that needs to be processed
public sealed class MessageProcessor : IDisposable
{
public MessageProcessor()
: this(-1)
{
}
public MessageProcessor(int maxThreadsForProcessing)
{
_maxThreadsForProcessing = maxThreadsForProcessing;
_messages = new BlockingCollection<Message>();
_cts = new CancellationTokenSource();
_messageProcessorThread = new Thread(ProcessMessages);
_messageProcessorThread.IsBackground = true;
_messageProcessorThread.Name = "Message Processor Thread";
_messageProcessorThread.Start();
}
public int MaxThreadsForProcessing
{
get { return _maxThreadsForProcessing; }
}
private readonly BlockingCollection<Message> _messages;
private readonly CancellationTokenSource _cts;
private readonly Thread _messageProcessorThread;
private bool _disposed = false;
private readonly int _maxThreadsForProcessing;
/// <summary>
/// Add a new message to be queued up and processed in the background.
/// </summary>
public void ReceiveMessage(Message message)
{
_messages.Add(message);
}
/// <summary>
/// Signals the system to stop processing messages.
/// </summary>
/// <param name="finishQueue">Should the queue of messages waiting to be processed be allowed to finish</param>
public void Stop(bool finishQueue)
{
_messages.CompleteAdding();
if(!finishQueue)
_cts.Cancel();
//Wait for the message processor thread to finish it's work.
_messageProcessorThread.Join();
}
/// <summary>
/// The background thread that processes messages in the system
/// </summary>
private void ProcessMessages()
{
try
{
Parallel.ForEach(_messages.GetConsumingEnumerable(),
new ParallelOptions()
{
CancellationToken = _cts.Token,
MaxDegreeOfParallelism = MaxThreadsForProcessing
},
ProcessMessage);
}
catch (OperationCanceledException)
{
//Don't care that it happened, just don't want it to bubble up as a unhandeled exception.
}
}
private void ProcessMessage(Message message, ParallelLoopState loopState)
{
//Here be dragons! (or your code to process a message, your choice :-))
//Use if(_cts.Token.IsCancellationRequested || loopState.ShouldExitCurrentIteration) to test if
// we should quit out of the function early for a graceful shutdown.
}
public void Dispose()
{
if(!_disposed)
{
if(_cts != null && _messages != null && _messageProcessorThread != null)
Stop(true); //This line will block till all queued messages have been processed, if you want it to be quicker you need to call `Stop(false)` before you dispose the object.
if(_cts != null)
_cts.Dispose();
if(_messages != null)
_messages.Dispose();
GC.SuppressFinalize(this);
_disposed = true;
}
}
~MessageProcessor()
{
//Nothing to do, just making FXCop happy.
}
}
I highly recommend you read the free book Patterns for Parallel Programming, it goes in to some detail about this. There is a entire section explaining the Producer-Consumer model in detail.
UPDATE: There are some performance issues with GetConsumingEnumerable() and Parallel.ForEach(, instead use the library ParallelExtensionsExtras and it's new extension method GetConsumingPartitioner()
public static Partitioner<T> GetConsumingPartitioner<T>(
this BlockingCollection<T> collection)
{
return new BlockingCollectionPartitioner<T>(collection);
}
private class BlockingCollectionPartitioner<T> : Partitioner<T>
{
private BlockingCollection<T> _collection;
internal BlockingCollectionPartitioner(
BlockingCollection<T> collection)
{
if (collection == null)
throw new ArgumentNullException("collection");
_collection = collection;
}
public override bool SupportsDynamicPartitions {
get { return true; }
}
public override IList<IEnumerator<T>> GetPartitions(
int partitionCount)
{
if (partitionCount < 1)
throw new ArgumentOutOfRangeException("partitionCount");
var dynamicPartitioner = GetDynamicPartitions();
return Enumerable.Range(0, partitionCount).Select(_ =>
dynamicPartitioner.GetEnumerator()).ToArray();
}
public override IEnumerable<T> GetDynamicPartitions()
{
return _collection.GetConsumingEnumerable();
}
}