I'm attempting to figure out an issue that has been raised with my ImageProcessor library here where I am getting intermittent file access errors when adding items to the cache.
System.IO.IOException: The process cannot access the file 'D:\home\site\wwwroot\app_data\cache\0\6\5\f\2\7\065f27fc2c8e843443d210a1e84d1ea28bbab6c4.webp' because it is being used by another process.
I wrote a class designed to perform an asynchronous lock based upon a key generated by a hashed url but it seems I have missed something in the implementation.
My locking class
public sealed class AsyncDuplicateLock
{
/// <summary>
/// The collection of semaphore slims.
/// </summary>
private static readonly ConcurrentDictionary<object, SemaphoreSlim> SemaphoreSlims
= new ConcurrentDictionary<object, SemaphoreSlim>();
/// <summary>
/// Locks against the given key.
/// </summary>
/// <param name="key">
/// The key that identifies the current object.
/// </param>
/// <returns>
/// The disposable <see cref="Task"/>.
/// </returns>
public IDisposable Lock(object key)
{
DisposableScope releaser = new DisposableScope(
key,
s =>
{
SemaphoreSlim locker;
if (SemaphoreSlims.TryRemove(s, out locker))
{
locker.Release();
locker.Dispose();
}
});
SemaphoreSlim semaphore = SemaphoreSlims.GetOrAdd(key, new SemaphoreSlim(1, 1));
semaphore.Wait();
return releaser;
}
/// <summary>
/// Asynchronously locks against the given key.
/// </summary>
/// <param name="key">
/// The key that identifies the current object.
/// </param>
/// <returns>
/// The disposable <see cref="Task"/>.
/// </returns>
public Task<IDisposable> LockAsync(object key)
{
DisposableScope releaser = new DisposableScope(
key,
s =>
{
SemaphoreSlim locker;
if (SemaphoreSlims.TryRemove(s, out locker))
{
locker.Release();
locker.Dispose();
}
});
Task<IDisposable> releaserTask = Task.FromResult(releaser as IDisposable);
SemaphoreSlim semaphore = SemaphoreSlims.GetOrAdd(key, new SemaphoreSlim(1, 1));
Task waitTask = semaphore.WaitAsync();
return waitTask.IsCompleted
? releaserTask
: waitTask.ContinueWith(
(_, r) => (IDisposable)r,
releaser,
CancellationToken.None,
TaskContinuationOptions.ExecuteSynchronously,
TaskScheduler.Default);
}
/// <summary>
/// The disposable scope.
/// </summary>
private sealed class DisposableScope : IDisposable
{
/// <summary>
/// The key
/// </summary>
private readonly object key;
/// <summary>
/// The close scope action.
/// </summary>
private readonly Action<object> closeScopeAction;
/// <summary>
/// Initializes a new instance of the <see cref="DisposableScope"/> class.
/// </summary>
/// <param name="key">
/// The key.
/// </param>
/// <param name="closeScopeAction">
/// The close scope action.
/// </param>
public DisposableScope(object key, Action<object> closeScopeAction)
{
this.key = key;
this.closeScopeAction = closeScopeAction;
}
/// <summary>
/// Disposes the scope.
/// </summary>
public void Dispose()
{
this.closeScopeAction(this.key);
}
}
}
Usage - within a HttpModule
private readonly AsyncDuplicateLock locker = new AsyncDuplicateLock();
using (await this.locker.LockAsync(cachedPath))
{
// Process and save a cached image.
}
Can anyone spot where I have gone wrong? I'm worried that I am misunderstanding something fundamental.
The full source for the library is stored on Github here
As the other answerer noted, the original code is removing the SemaphoreSlim from the ConcurrentDictionary before it releases the semaphore. So, you've got too much semaphore churn going on - they're being removed from the dictionary when they could still be in use (not acquired, but already retrieved from the dictionary).
The problem with this kind of "mapping lock" is that it's difficult to know when the semaphore is no longer necessary. One option is to never dispose the semaphores at all; that's the easy solution, but may not be acceptable in your scenario. Another option - if the semaphores are actually related to object instances and not values (like strings) - is to attach them using ephemerons; however, I believe this option would also not be acceptable in your scenario.
So, we do it the hard way. :)
There are a few different approaches that would work. I think it makes sense to approach it from a reference-counting perspective (reference-counting each semaphore in the dictionary). Also, we want to make the decrement-count-and-remove operation atomic, so I just use a single lock (making the concurrent dictionary superfluous):
public sealed class AsyncDuplicateLock
{
private sealed class RefCounted<T>
{
public RefCounted(T value)
{
RefCount = 1;
Value = value;
}
public int RefCount { get; set; }
public T Value { get; private set; }
}
private static readonly Dictionary<object, RefCounted<SemaphoreSlim>> SemaphoreSlims
= new Dictionary<object, RefCounted<SemaphoreSlim>>();
private SemaphoreSlim GetOrCreate(object key)
{
RefCounted<SemaphoreSlim> item;
lock (SemaphoreSlims)
{
if (SemaphoreSlims.TryGetValue(key, out item))
{
++item.RefCount;
}
else
{
item = new RefCounted<SemaphoreSlim>(new SemaphoreSlim(1, 1));
SemaphoreSlims[key] = item;
}
}
return item.Value;
}
public IDisposable Lock(object key)
{
GetOrCreate(key).Wait();
return new Releaser { Key = key };
}
public async Task<IDisposable> LockAsync(object key)
{
await GetOrCreate(key).WaitAsync().ConfigureAwait(false);
return new Releaser { Key = key };
}
private sealed class Releaser : IDisposable
{
public object Key { get; set; }
public void Dispose()
{
RefCounted<SemaphoreSlim> item;
lock (SemaphoreSlims)
{
item = SemaphoreSlims[Key];
--item.RefCount;
if (item.RefCount == 0)
SemaphoreSlims.Remove(Key);
}
item.Value.Release();
}
}
}
Here is a KeyedLock class that is less convenient and more error prone, but also less allocatey than Stephen Cleary's AsyncDuplicateLock. It maintains internally a pool of SemaphoreSlims, that can be reused by any key after they are released by the previous key. The capacity of the pool is configurable, and by default is 10.
This class is not allocation-free, because the SemaphoreSlim class allocates memory (quite a lot actually) every time the semaphore cannot be acquired synchronously because of contention.
The lock can be requested both synchronously and asynchronously, and can also be requested with cancellation and timeout. These features are provided by exploiting the existing functionality of the SemaphoreSlim class.
public class KeyedLock<TKey>
{
private readonly Dictionary<TKey, (SemaphoreSlim, int)> _perKey;
private readonly Stack<SemaphoreSlim> _pool;
private readonly int _poolCapacity;
public KeyedLock(IEqualityComparer<TKey> keyComparer = null, int poolCapacity = 10)
{
_perKey = new Dictionary<TKey, (SemaphoreSlim, int)>(keyComparer);
_pool = new Stack<SemaphoreSlim>(poolCapacity);
_poolCapacity = poolCapacity;
}
public async Task<bool> WaitAsync(TKey key, int millisecondsTimeout,
CancellationToken cancellationToken = default)
{
var semaphore = GetSemaphore(key);
bool entered = false;
try
{
entered = await semaphore.WaitAsync(millisecondsTimeout,
cancellationToken).ConfigureAwait(false);
}
finally { if (!entered) ReleaseSemaphore(key, entered: false); }
return entered;
}
public Task WaitAsync(TKey key, CancellationToken cancellationToken = default)
=> WaitAsync(key, Timeout.Infinite, cancellationToken);
public bool Wait(TKey key, int millisecondsTimeout,
CancellationToken cancellationToken = default)
{
var semaphore = GetSemaphore(key);
bool entered = false;
try { entered = semaphore.Wait(millisecondsTimeout, cancellationToken); }
finally { if (!entered) ReleaseSemaphore(key, entered: false); }
return entered;
}
public void Wait(TKey key, CancellationToken cancellationToken = default)
=> Wait(key, Timeout.Infinite, cancellationToken);
public void Release(TKey key) => ReleaseSemaphore(key, entered: true);
private SemaphoreSlim GetSemaphore(TKey key)
{
SemaphoreSlim semaphore;
lock (_perKey)
{
if (_perKey.TryGetValue(key, out var entry))
{
int counter;
(semaphore, counter) = entry;
_perKey[key] = (semaphore, ++counter);
}
else
{
lock (_pool) semaphore = _pool.Count > 0 ? _pool.Pop() : null;
if (semaphore == null) semaphore = new SemaphoreSlim(1, 1);
_perKey[key] = (semaphore, 1);
}
}
return semaphore;
}
private void ReleaseSemaphore(TKey key, bool entered)
{
SemaphoreSlim semaphore; int counter;
lock (_perKey)
{
if (_perKey.TryGetValue(key, out var entry))
{
(semaphore, counter) = entry;
counter--;
if (counter == 0)
_perKey.Remove(key);
else
_perKey[key] = (semaphore, counter);
}
else
{
throw new InvalidOperationException("Key not found.");
}
}
if (entered) semaphore.Release();
if (counter == 0)
{
Debug.Assert(semaphore.CurrentCount == 1);
lock (_pool) if (_pool.Count < _poolCapacity) _pool.Push(semaphore);
}
}
}
Usage example:
var locker = new KeyedLock<string>();
await locker.WaitAsync("Hello");
try
{
await DoSomethingAsync();
}
finally
{
locker.Release("Hello");
}
The implementation uses tuple deconstruction, that requires at least C# 7.
The KeyedLock class could be easily modified to become a KeyedSemaphore, that would allow more than one concurrent operations per key. It would just need a maximumConcurrencyPerKey parameter in the constructor, that would be stored and passed to the constructor of the SemaphoreSlims.
Note: The SemaphoreSlim class when misused it throws a SemaphoreFullException. This happens when the semaphore is released more times than it has been acquired. The KeyedLock implementation of this answer behaves differently in case of misuse: it throws an InvalidOperationException("Key not found."). This happens because when a key is released as many times as it has been acquired, the associated semaphore is removed from the dictionary. If this implementation ever throw a SemaphoreFullException, it would be an indication of a bug.
I wrote a library called AsyncKeyedLock to fix this common problem. The library currently supports using it with the type object (so you can mix different types together) or using generics to get a more efficient solution. It allows for timeouts, cancellation tokens, and also pooling so as to reduce allocations. Underlying it uses a ConcurrentDictionary and also allows for setting the initial capacity and concurrency for this dictionary.
I have benchmarked this against the other solutions provided here and it is more efficient, in terms of speed, memory usage (allocations) as well as scalability (internally it uses the more scalable ConcurrentDictionary). It's being used in a number of systems in production and used by a number of popular libraries.
The source code is available on GitHub and packaged at NuGet.
The approach here is to basically use the ConcurrentDictionary to store an IDisposable object which has a counter on it and a SemaphoreSlim. Once this counter reaches 0, it is removed from the dictionary and either disposed or returned to the pool (if pooling is used). Monitor is used to lock this object when either the counter is being incremented or decremented.
Usage example:
var locker = new AsyncKeyedLocker<string>(o =>
{
o.PoolSize = 20;
o.PoolInitialFill = 1;
});
string key = "my key";
// asynchronous code
using (await locker.LockAsync(key, cancellationToken))
{
...
}
// synchronous code
using (locker.Lock(key))
{
...
}
Download from NuGet.
For a given key,
Thread 1 calls GetOrAdd and adds a new semaphore and acquires it via Wait
Thread 2 calls GetOrAdd and gets the existing semaphore and blocks on Wait
Thread 1 releases the semaphore, only after having called TryRemove, which removed the semaphore from the dictionary
Thread 2 now acquires the semaphore.
Thread 3 calls GetOrAdd for the same key as thread 1 and 2. Thread 2 is still holding the semaphore, but the semaphore is not in the dictionary, so thread 3 creates a new semaphore and both threads 2 and 3 access the same protected resource.
You need to adjust your logic. The semaphore should only be removed from the dictionary when it has no waiters.
Here is one potential solution, minus the async part:
public sealed class AsyncDuplicateLock
{
private class LockInfo
{
private SemaphoreSlim sem;
private int waiterCount;
public LockInfo()
{
sem = null;
waiterCount = 1;
}
// Lazily create the semaphore
private SemaphoreSlim Semaphore
{
get
{
var s = sem;
if (s == null)
{
s = new SemaphoreSlim(0, 1);
var original = Interlocked.CompareExchange(ref sem, null, s);
// If someone else already created a semaphore, return that one
if (original != null)
return original;
}
return s;
}
}
// Returns true if successful
public bool Enter()
{
if (Interlocked.Increment(ref waiterCount) > 1)
{
Semaphore.Wait();
return true;
}
return false;
}
// Returns true if this lock info is now ready for removal
public bool Exit()
{
if (Interlocked.Decrement(ref waiterCount) <= 0)
return true;
// There was another waiter
Semaphore.Release();
return false;
}
}
private static readonly ConcurrentDictionary<object, LockInfo> activeLocks = new ConcurrentDictionary<object, LockInfo>();
public static IDisposable Lock(object key)
{
// Get the current info or create a new one
var info = activeLocks.AddOrUpdate(key,
(k) => new LockInfo(),
(k, v) => v.Enter() ? v : new LockInfo());
DisposableScope releaser = new DisposableScope(() =>
{
if (info.Exit())
{
// Only remove this exact info, in case another thread has
// already put its own info into the dictionary
((ICollection<KeyValuePair<object, LockInfo>>)activeLocks)
.Remove(new KeyValuePair<object, LockInfo>(key, info));
}
});
return releaser;
}
private sealed class DisposableScope : IDisposable
{
private readonly Action closeScopeAction;
public DisposableScope(Action closeScopeAction)
{
this.closeScopeAction = closeScopeAction;
}
public void Dispose()
{
this.closeScopeAction();
}
}
}
I rewrote the #StephenCleary answer with this:
public sealed class AsyncLockList {
readonly Dictionary<object, SemaphoreReferenceCount> Semaphores = new Dictionary<object, SemaphoreReferenceCount>();
SemaphoreSlim GetOrCreateSemaphore(object key) {
lock (Semaphores) {
if (Semaphores.TryGetValue(key, out var item)) {
item.IncrementCount();
} else {
item = new SemaphoreReferenceCount();
Semaphores[key] = item;
}
return item.Semaphore;
}
}
public IDisposable Lock(object key) {
GetOrCreateSemaphore(key).Wait();
return new Releaser(Semaphores, key);
}
public async Task<IDisposable> LockAsync(object key) {
await GetOrCreateSemaphore(key).WaitAsync().ConfigureAwait(false);
return new Releaser(Semaphores, key);
}
sealed class SemaphoreReferenceCount {
public readonly SemaphoreSlim Semaphore = new SemaphoreSlim(1, 1);
public int Count { get; private set; } = 1;
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public void IncrementCount() => Count++;
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public void DecrementCount() => Count--;
}
sealed class Releaser : IDisposable {
readonly Dictionary<object, SemaphoreReferenceCount> Semaphores;
readonly object Key;
[MethodImpl(MethodImplOptions.AggressiveInlining)]
public Releaser(Dictionary<object, SemaphoreReferenceCount> semaphores, object key) {
Semaphores = semaphores;
Key = key;
}
public void Dispose() {
lock (Semaphores) {
var item = Semaphores[Key];
item.DecrementCount();
if (item.Count == 0)
Semaphores.Remove(Key);
item.Semaphore.Release();
}
}
}
}
Inspired by this previous answer, here is a version that supports async wait:
public class KeyedLock<TKey>
{
private readonly ConcurrentDictionary<TKey, LockInfo> _locks = new();
public int Count => _locks.Count;
public async Task<IDisposable> WaitAsync(TKey key, CancellationToken cancellationToken = default)
{
// Get the current info or create a new one.
var info = _locks.AddOrUpdate(key,
// Add
k => new LockInfo(),
// Update
(k, v) => v.Enter() ? v : new LockInfo());
try
{
await info.Semaphore.WaitAsync(cancellationToken);
return new Releaser(() => Release(key, info, true));
}
catch (OperationCanceledException)
{
// The semaphore wait was cancelled, release the lock.
Release(key, info, false);
throw;
}
}
private void Release(TKey key, LockInfo info, bool isCurrentlyLocked)
{
if (info.Leave())
{
// This was the last lock for the key.
// Only remove this exact info, in case another thread has
// already put its own info into the dictionary
// Note that this call to Remove(entry) is in fact thread safe.
var entry = new KeyValuePair<TKey, LockInfo>(key, info);
if (((ICollection<KeyValuePair<TKey, LockInfo>>)_locks).Remove(entry))
{
// This exact info was removed.
info.Dispose();
}
}
else if (isCurrentlyLocked)
{
// There is another waiter.
info.Semaphore.Release();
}
}
private class LockInfo : IDisposable
{
private SemaphoreSlim _semaphore = null;
private int _refCount = 1;
public SemaphoreSlim Semaphore
{
get
{
// Lazily create the semaphore.
var s = _semaphore;
if (s is null)
{
s = new SemaphoreSlim(1, 1);
// Assign _semaphore if its current value is null.
var original = Interlocked.CompareExchange(ref _semaphore, s, null);
// If someone else already created a semaphore, return that one
if (original is not null)
{
s.Dispose();
return original;
}
}
return s;
}
}
// Returns true if successful
public bool Enter()
{
if (Interlocked.Increment(ref _refCount) > 1)
{
return true;
}
// This lock info is not valid anymore - its semaphore is or will be disposed.
return false;
}
// Returns true if this lock info is now ready for removal
public bool Leave()
{
if (Interlocked.Decrement(ref _refCount) <= 0)
{
// This was the last lock
return true;
}
// There is another waiter
return false;
}
public void Dispose() => _semaphore?.Dispose();
}
private sealed class Releaser : IDisposable
{
private readonly Action _dispose;
public Releaser(Action dispose) => _dispose = dispose;
public void Dispose() => _dispose();
}
}
I have a Silverlight 5 browser application.
There is a class
public class ActivityControl:UserControl {
public void LoadSubControls() {
//Creates Other UserControls, does calculations and is very slow..No refactoring..
}
}
I need to create multiple instances of this class and call the method LoadSubControls on runtime.
public class BasicContainer:UserControl {
public void CreateMultipleActivityControls() {
for (int i = 0; i < 1000; i++) {
ActivityControl c = new ActivityControl(); ====> I need to call this in a different thread but causes Invalid Cross Thread Exception
c.LoadSubControls();
}
}
}
Is there any way to create multiple UI Threads in order to avoid invalid cross thread exception?
I need multithreading for performance reasons and because the method call is very slow and the UI freezes.
Is there any way to call method SetSyncronizationContext (which is [SecurityCritical]) in Silverlight?
There is no avoiding creating these controls on the UI thread, but you could take advantage of System.Threading.Tasks.Task from Task Parallel Library (TPL) to allow for asynchronous operations.
I've been able to do something like this in silverlight 5 with a structure like this. Got the original idea looking at the source for Caliburn.Micro.
The following is a subset that applies to what you want.
public interface IPlatformProvider {
/// <summary>
/// Executes the action on the UI thread asynchronously.
/// </summary>
/// <param name = "action">The action to execute.</param>
System.Threading.Tasks.Task OnUIThreadAsync(Action action);
}
Here is the implementation.
/// <summary>
/// A <see cref="IPlatformProvider"/> implementation for the XAML platfrom (Silverlight).
/// </summary>
public class XamlPlatformProvider : IPlatformProvider {
private Dispatcher dispatcher;
public XamlPlatformProvider() {
dispatcher = System.Windows.Deployment.Current.Dispatcher;
}
private void validateDispatcher() {
if (dispatcher == null)
throw new InvalidOperationException("Not initialized with dispatcher.");
}
/// <summary>
/// Executes the action on the UI thread asynchronously.
/// </summary>
/// <param name = "action">The action to execute.</param>
public Task OnUIThreadAsync(System.Action action) {
validateDispatcher();
var taskSource = new TaskCompletionSource<object>();
System.Action method = () => {
try {
action();
taskSource.SetResult(null);
} catch (Exception ex) {
taskSource.SetException(ex);
}
};
dispatcher.BeginInvoke(method);
return taskSource.Task;
}
}
You could either go down the constructor DI route to pass in the provider or use a static locator pattern like this.
/// <summary>
/// Access the current <see cref="IPlatformProvider"/>.
/// </summary>
public static class PlatformProvider {
private static IPlatformProvider current = new XamlPlatformProvider();
/// <summary>
/// Gets or sets the current <see cref="IPlatformProvider"/>.
/// </summary>
public static IPlatformProvider Current {
get { return current; }
set { current = value; }
}
}
Now you should be able to make your calls without blocking the main thread and freezing the UI
public class BasicContainer : UserControl {
public async Task CreateMultipleActivityControls() {
var platform = PlatformProvider.Current;
for (var i = 0; i < 1000; i++) {
await platform.OnUIThreadAsync(() => {
var c = new ActivityControl();
c.LoadSubControls();
});
}
}
}
if making multiple calls to the dispatcher caused any performance issues you could instead move the entire process to one acync call.
public class BasicContainer : UserControl {
public async Task CreateMultipleActivityControls() {
var platform = PlatformProvider.Current;
await platform.OnUIThreadAsync(() => {
for (var i = 0; i < 1000; i++) {
var c = new ActivityControl();
c.LoadSubControls();
}
});
}
}
Observable.Concat is an implementation that joins observables but the second IObservable<T> only makes the subscription when the first is completed.
http://www.introtorx.com/content/v1.0.10621.0/12_CombiningSequences.html#Concat
Is there any implementation of a "HotConcat"? Similar to Observable.Merge, but keeping the delivery order, first pushing the elements of initial subscription and then the subsequents. Something like:
I know that is possible to use the ReplaySubject<T>, but it doesn't seems so good, because of performance and memory usage impacts..
Here's the implementation I've been using for a while. This implementation introduces a BufferUntilSubscribed operator that turns an IObservable into a IConnectableObservable that will start buffering whenever you call Connect and will deliver the buffered results to the first subscriber. Once the first subscriber has "caught up", then the buffering will stop and the subscriber will just be given the live events as they arrive.
Once you have that, you can write HotConcat as something like:
public static IObservable<T> HotConcat<T>(params IObservable<T>[] sources)
{
var s2 = sources.Select(s => s.BufferUntilSubscribed());
var subscriptions = new CompositeDisposable(s2.Select(s2 => s2.Connect()).ToArray());
return Observable.Create<T>(observer =>
{
var s = new SingleAssignmentDisposable();
var d = new CompositeDisposable(subscriptions);
d.Add(s);
s.Disposable = s2.Concat().Subscribe(observer);
return d;
});
}
Here's the implemementation of BufferUntilSubscribed:
private class BufferUntilSubscribedObservable<T> : IConnectableObservable<T>
{
private readonly IObservable<T> _source;
private readonly IScheduler _scheduler;
private readonly Subject<T> _liveEvents;
private bool _observationsStarted;
private Queue<T> _buffer;
private readonly object _gate;
public BufferUntilSubscribedObservable(IObservable<T> source, IScheduler scheduler)
{
_source = source;
_scheduler = scheduler;
_liveEvents = new Subject<T>();
_buffer = new Queue<T>();
_gate = new object();
_observationsStarted = false;
}
public IDisposable Subscribe(IObserver<T> observer)
{
lock (_gate)
{
if (_observationsStarted)
{
return _liveEvents.Subscribe(observer);
}
_observationsStarted = true;
var bufferedEvents = GetBuffers().Concat().Finally(RemoveBuffer); // Finally clause to remove the buffer if the first observer stops listening.
return Observable.Merge(_liveEvents, bufferedEvents).Subscribe(observer);
}
}
public IDisposable Connect()
{
return _source.Subscribe(OnNext, _liveEvents.OnError, _liveEvents.OnCompleted);
}
private void RemoveBuffer()
{
lock (_gate)
{
_buffer = null;
}
}
/// <summary>
/// Acquires a lock and checks the buffer. If it is empty, then replaces it with null and returns null. Else replaces it with an empty buffer and returns the old buffer.
/// </summary>
/// <returns></returns>
private Queue<T> GetAndReplaceBuffer()
{
lock (_gate)
{
if (_buffer == null)
{
return null;
}
if (_buffer.Count == 0)
{
_buffer = null;
return null;
}
var result = _buffer;
_buffer = new Queue<T>();
return result;
}
}
/// <summary>
/// An enumerable of buffers that will complete when a call to GetAndReplaceBuffer() returns a null, e.g. when the observer has caught up with the incoming source data.
/// </summary>
/// <returns></returns>
private IEnumerable<IObservable<T>> GetBuffers()
{
Queue<T> buffer;
while ((buffer = GetAndReplaceBuffer()) != null)
{
yield return buffer.ToObservable(_scheduler);
}
}
private void OnNext(T item)
{
lock (_gate)
{
if (_buffer != null)
{
_buffer.Enqueue(item);
return;
}
}
_liveEvents.OnNext(item);
}
}
/// <summary>
/// Returns a connectable observable, that once connected, will start buffering data until the observer subscribes, at which time it will send all buffered data to the observer and then start sending new data.
/// Thus the observer may subscribe late to a hot observable yet still see all of the data. Later observers will not see the buffered events.
/// </summary>
/// <param name="source"></param>
/// <param name="scheduler">Scheduler to use to dump the buffered data to the observer.</param>
/// <returns></returns>
public static IConnectableObservable<T> BufferUntilSubscribed<T>(this IObservable<T> source, IScheduler scheduler)
{
return new BufferUntilSubscribedObservable<T>(source, scheduler);
}
/// <summary>
/// Returns a connectable observable, that once connected, will start buffering data until the observer subscribes, at which time it will send all buffered data to the observer and then start sending new data.
/// Thus the observer may subscribe late to a hot observable yet still see all of the data. Later observers will not see the buffered events.
/// </summary>
/// <param name="source"></param>
/// <returns></returns>
public static IConnectableObservable<T> BufferUntilSubscribed<T>(this IObservable<T> source)
{
return new BufferUntilSubscribedObservable<T>(source, Scheduler.Immediate);
}
I don't know of such a composition function, but you can write one that matches your needs.
Here's my attempt at writing one. It will keep elements in memory only until they are replayed once. But I think there should be a way to make a cleaner implementation though.
public static IObservable<T> HotConcat<T>(this IObservable<T> first, IObservable<T> second)
{
return Observable.Create<T>(observer =>
{
var queue = new Queue<Notification<T>>();
var secondSubscription = second.Materialize().Subscribe(item =>
{
if (queue == null)
return;
lock (queue)
{
queue.Enqueue(item);
}
});
var secondReplay = Observable.Create<T>(secondObserver =>
{
while (true)
{
Notification<T> item = null;
lock (queue)
{
if (queue.Count > 0)
{
item = queue.Dequeue();
}
else
{
secondObserver.OnCompleted();
secondSubscription.Dispose();
queue = null;
break;
}
}
if (item != null)
item.Accept(secondObserver);
}
return secondSubscription;
});
return first.Concat(secondReplay).Concat(second).Subscribe(observer);
});
}
We have an application that regularly receives multimedia messages, and should reply to them.
We currently do this with a single thread, first receiving messages, and then processing them one by one. This does the job, but is slow.
So we're now thinking of doing the same process but with multiple threads sumultaneously.
Any simple way to allow parallel processing of the incoming records, yet avoid erroneously processing the same record by two threads?
Any simple way to allow parallel processing of the incoming records, yet avoid erroneously processing the same record by two threads?
Yes it is actually not too hard, what you are wanting to do is called the "Producer-Consumer model"
If your message receiver could only handle one thread at a time but your message "processor" can work on multiple messages at once you just need to use a BlockingCollection to store the work that needs to be processed
public sealed class MessageProcessor : IDisposable
{
public MessageProcessor()
: this(-1)
{
}
public MessageProcessor(int maxThreadsForProcessing)
{
_maxThreadsForProcessing = maxThreadsForProcessing;
_messages = new BlockingCollection<Message>();
_cts = new CancellationTokenSource();
_messageProcessorThread = new Thread(ProcessMessages);
_messageProcessorThread.IsBackground = true;
_messageProcessorThread.Name = "Message Processor Thread";
_messageProcessorThread.Start();
}
public int MaxThreadsForProcessing
{
get { return _maxThreadsForProcessing; }
}
private readonly BlockingCollection<Message> _messages;
private readonly CancellationTokenSource _cts;
private readonly Thread _messageProcessorThread;
private bool _disposed = false;
private readonly int _maxThreadsForProcessing;
/// <summary>
/// Add a new message to be queued up and processed in the background.
/// </summary>
public void ReceiveMessage(Message message)
{
_messages.Add(message);
}
/// <summary>
/// Signals the system to stop processing messages.
/// </summary>
/// <param name="finishQueue">Should the queue of messages waiting to be processed be allowed to finish</param>
public void Stop(bool finishQueue)
{
_messages.CompleteAdding();
if(!finishQueue)
_cts.Cancel();
//Wait for the message processor thread to finish it's work.
_messageProcessorThread.Join();
}
/// <summary>
/// The background thread that processes messages in the system
/// </summary>
private void ProcessMessages()
{
try
{
Parallel.ForEach(_messages.GetConsumingEnumerable(),
new ParallelOptions()
{
CancellationToken = _cts.Token,
MaxDegreeOfParallelism = MaxThreadsForProcessing
},
ProcessMessage);
}
catch (OperationCanceledException)
{
//Don't care that it happened, just don't want it to bubble up as a unhandeled exception.
}
}
private void ProcessMessage(Message message, ParallelLoopState loopState)
{
//Here be dragons! (or your code to process a message, your choice :-))
//Use if(_cts.Token.IsCancellationRequested || loopState.ShouldExitCurrentIteration) to test if
// we should quit out of the function early for a graceful shutdown.
}
public void Dispose()
{
if(!_disposed)
{
if(_cts != null && _messages != null && _messageProcessorThread != null)
Stop(true); //This line will block till all queued messages have been processed, if you want it to be quicker you need to call `Stop(false)` before you dispose the object.
if(_cts != null)
_cts.Dispose();
if(_messages != null)
_messages.Dispose();
GC.SuppressFinalize(this);
_disposed = true;
}
}
~MessageProcessor()
{
//Nothing to do, just making FXCop happy.
}
}
I highly recommend you read the free book Patterns for Parallel Programming, it goes in to some detail about this. There is a entire section explaining the Producer-Consumer model in detail.
UPDATE: There are some performance issues with GetConsumingEnumerable() and Parallel.ForEach(, instead use the library ParallelExtensionsExtras and it's new extension method GetConsumingPartitioner()
public static Partitioner<T> GetConsumingPartitioner<T>(
this BlockingCollection<T> collection)
{
return new BlockingCollectionPartitioner<T>(collection);
}
private class BlockingCollectionPartitioner<T> : Partitioner<T>
{
private BlockingCollection<T> _collection;
internal BlockingCollectionPartitioner(
BlockingCollection<T> collection)
{
if (collection == null)
throw new ArgumentNullException("collection");
_collection = collection;
}
public override bool SupportsDynamicPartitions {
get { return true; }
}
public override IList<IEnumerator<T>> GetPartitions(
int partitionCount)
{
if (partitionCount < 1)
throw new ArgumentOutOfRangeException("partitionCount");
var dynamicPartitioner = GetDynamicPartitions();
return Enumerable.Range(0, partitionCount).Select(_ =>
dynamicPartitioner.GetEnumerator()).ToArray();
}
public override IEnumerable<T> GetDynamicPartitions()
{
return _collection.GetConsumingEnumerable();
}
}
Using the command pattern returning tasks that are just sleeps for 5 seconds the total for the 3 tasks to complete is ~15 seconds.
What am I doing that is keeping this code from executing in "parallel"?
The calling code
var timer = new Stopwatch();
timer.Start();
var task = CommandExecutor.ExecuteCommand(new Fake());
var task2 = CommandExecutor.ExecuteCommand(new Fake());
var task3 = CommandExecutor.ExecuteCommand(new Fake());
Task.WaitAll(new Task[]
{
task, task2, task3
});
timer.Stop();
Debug.Print("{0}ms for {1},{2},{3} records", timer.ElapsedMilliseconds, task.Result.Count, task2.Result.Count(), task3.Result.Count());
The Commands being executed
public class Fake : Command<Task<Dictionary<string, string[]>>>
{
public override string ToString()
{
return string.Format("Fake");
}
protected override void Execute()
{
new System.Threading.ManualResetEvent(false).WaitOne(5000);
Result = Task.Factory.StartNew(() => new Dictionary<string, string[]>());
}
}
The Command Abstraction
public abstract class Command
{
public void Run()
{
try
{
var timer = new Stopwatch();
timer.Start();
//Debug.Print("{0}-{1}", ToString(), "Executing");
Execute();
timer.Stop();
Debug.Print("{0}-{1} Duration: {2}ms", ToString(), "Done", timer.ElapsedMilliseconds.ToString(CultureInfo.InvariantCulture));
}
catch (Exception ex)
{
Debug.Print("Error processing task:" + ToString(), ex);
}
}
public abstract override string ToString();
protected abstract void Execute();
}
/// <summary>
/// A command with a return value
/// </summary>
/// <typeparam name="T"></typeparam>
public abstract class Command<T> : Command
{
public T Result { get; protected set; }
public T GetResult()
{
Run();
return Result;
}
}
the Command Executor
public class CommandExecutor
{
/// <summary>
/// Executes the command.
/// </summary>
/// <param name="cmd">The CMD.</param>
public static void ExecuteCommand(Command cmd)
{
cmd.Run();
}
/// <summary>
/// Executes the command for commands with a result.
/// </summary>
/// <typeparam name="TResult">The type of the result.</typeparam>
/// <param name="cmd">The CMD.</param>
/// <returns></returns>
public static TResult ExecuteCommand<TResult>(Command<TResult> cmd)
{
ExecuteCommand((Command) cmd);
return cmd.Result;
}
The problem is that you aren't waiting inside the actual Task object, you're waiting in the method that creates the task before it actually provides that task:
protected override void Execute()
{
new System.Threading.ManualResetEvent(false).WaitOne(5000);
Result = Task.Factory.StartNew(() => new Dictionary<string, string[]>());
}
should instead be:
protected override void Execute()
{
Result = Task.Factory.StartNew(() =>
{
new System.Threading.ManualResetEvent(false).WaitOne(5000);
return new Dictionary<string, string[]>();
});
}