I have an interleaved stream which I split into separate sequential streams.
Producer
int streamCount = 3;
new MyIEnumerable<ElementType>()
.ToObservable(Scheduler.ThreadPool)
.Select((x,i) => new { Key = (i % streamCount), Value = x })
.Subscribe(x => outputs[x.Key].OnNext(x.Value));
Where outputs[] are Subjects which process the streams are defined below. The .ObserveOn() is used to process the streams concurrently (multi-threaded).
Consumers
var outputs = Enumerable.Repeat(0, streamCount).Select(_ => new Subject<char>()).ToArray();
outputs[0].ObserveOn(Scheduler.ThreadPool).Subscribe(x => Console.WriteLine("stream 0: {0}", x));
outputs[1].ObserveOn(Scheduler.ThreadPool).Subscribe(x => Console.WriteLine("stream 1: {0}", x));
outputs[2].ObserveOn(Scheduler.ThreadPool).Subscribe(x => Console.WriteLine("stream 2: {0}", x));
The problem with this code is that it will read the entire enumerable as fast as possible, even if the output streams cannot catch up. In my case the enumerable is a file stream so this might cause using a lot of memory. Therefore, I would like the reading to block if the buffer(s) reach some threshold.
I have solved this by using a semaphore on the producer and consumers like shown below. However, I am not sure that this is considered a good solution (in terms of Rx contracts, programming style, etc).
var semaphore = new SemaphoreSlim(MAX_BUFFERED_ELEMENTS);
// add to producer (before subscribe)
.Do(_ => semaphore.Wait());
// add to consumer (before subscribe)
.Do(_ => semaphore.Release()))
It might be a good idea to pass a CancelationToken to the call to Wait() and make sure it is cancelled when the stream stops abnormally?
I think your solution is very reasonable. The biggest problem (having some background to the previous question) is that the 'insides' of your solution are currently exposed everywhere. Just make sure that when you code this properly you clean up the following:
Wrap everything into a class that exposes a single method: IDisposable Subscribe(<index>, Action) or alternatively IObservable<element> ToObservable(<index>)). Either the returned subscription or the returned observable will have all the 'work' already done to them, namely the added Do actions and so forth. The fact that there's a dictionary or list under it all should be completely irrelevant to the user, otherwise any change to your code here will require changes all over the place.
A CancelationToken is a great idea, make sure to cancel it on either OnCompleted or OnError, which you can do using overloads to Do.
Related
I'm using the Reactive .NET extensions and I wonder about its disposal. I know in some cases it's good to dispose it like that: .TakeUntil(Observable.Timer(TimeSpan.FromMinutes(x))). I
First case
In this case, I have a timer that triggers after x seconds and then it completes and should be disposed.
public void ScheduleOrderCancellationIfNotFilled(string pair, long orderId, int waitSecondsBeforeCancel)
{
Observable.Timer(TimeSpan.FromSeconds(waitSecondsBeforeCancel))
.Do(e =>
{
var result = _client.Spot.Order.GetOrder(pair, orderId);
if (result.Success)
{
if (result.Data?.Status != OrderStatus.Filled)
{
_client.Spot.Order.CancelOrder(pair, orderId);
}
}
})
.Subscribe();
}
Second case
In this case, the timer runs on the first second and then it repeats itself on each 29 minutes. This should live until its defining class is disposed. I believe this one should be disposed with IDisposable implementation. How?
var keepAliveListenKey = Observable.Timer(TimeSpan.FromSeconds(1), TimeSpan.FromMinutes(29))
.Do(async e =>
{
await KeepAliveListenKeyAsync().ConfigureAwait(false);
})
.Subscribe();
Edit
I also want it to be using a Subject<T> which makes it easier to dispose and to reset the subscription.
For ex. Reset and Dispose observable subscriber, Reactive Extensions (#Enigmativity)
public class UploadDicomSet : ImportBaseSet
{
IDisposable subscription;
Subject<IObservable<long>> subject = new Subject<IObservable<long>>();
public UploadDicomSet()
{
subscription = subject.Switch().Subscribe(s => CheckUploadSetList(s));
subject.OnNext(Observable.Interval(TimeSpan.FromMinutes(2)));
}
void CheckUploadSetList(long interval)
{
subject.OnNext(Observable.Never<long>());
// Do other things
}
public void AddDicomFile(SharedLib.DicomFile dicomFile)
{
subject.OnNext(Observable.Interval(TimeSpan.FromMinutes(2)));
// Reset the subscription to go off in 2 minutes from now
// Do other things
}
}
In the first case it gonna be disposed automatically. It is, actually, a common way to achieve automatic subscription management and that's definitely nice and elegant way to deal with rx.
In the second case you have over-engineered. Observable.Timer(TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1)) is itself sufficient to generate a sequence of ascending longs over time. Since this stream is endless by its nature, you right - explicit subscription management is required. So it is enough to have:
var sub = Observable.Timer(TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1)).Subscribe()
...and sub.Dispose() it later.
P.S. Note that in your code you .Do async/await. Most probably that is not what you want. You want SelectMany to ensure that async operation is properly awaited and exceptions handled.
Answering your questions in the comments section:
What about disposing using Subject instead?
Well, nothing so special about it. Both IObserver<>, IObservable<> is implemented by this class such that it resembles classical .NET events (list of callbacks to be called upon some event). It does not differ in any sense with respect to your question and use-case.
May you give an example about the .Do with exception handling?
Sure. The idea is that you want translate your async/await encapsulated into some Task<T> to IObservable<T> such that is preserves both cancellation and error signals. For that .SelectMany method must be used (like SelectMany from LINQ, the same idea). So just change your .Do to .SelectMany.
Observable
.Timer(TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1))
.SelectMany(_ => Observable.FromAsync(() => /* that's the point where your Task<> becomes Observable */ myTask))
I'm confused again. Do I need IObservable<IObservable> (Select) or IObservable (SelectMany)
Most probably, you don't need switch. Why? Because it was created mainly to avoid IO race conditions, such that whenever new event is emitted, the current one (which might be in progress due to natural parallelism or asynchronous workflow) is guaranteed to be cancelled (i.e. unsubscribed). Otherwise race conditions can (and will) damage your state.
SelectMany, on the contrary, will make sure all of them are happen sequentially, in some total order they have indeed arrived. Nothing will be cancelled. You will finish (await, if you wish) current callback and then trigger the next one. Of course, such behavior can be altered by means of appropriate IScheduler, but that is another story.
Reactive Observable Subscription Disposal (#Enigmativity)
The disposable returned by the Subscribe extension methods is returned solely to allow you to manually unsubscribe from the observable before the observable naturally ends.
If the observable completes - with either OnCompleted or OnError - then the subscription is already disposed for you.
One important thing to note: the garbage collector never calls .Dispose() on observable subscriptions, so you must dispose of your subscriptions if they have not (or may not have) naturally ended before your subscription goes out of scope.
First case
Looks like I don't need to manually .Dispose() the subscription in the first case scenario because it ends naturally.
Dispose is being triggered at the end.
var xs = Observable.Create<long>(o =>
{
var d = Observable.Timer(TimeSpan.FromSeconds(5))
.Do(e =>
{
Console.WriteLine("5 seconds elapsed.");
})
.Subscribe(o);
return Disposable.Create(() =>
{
Console.WriteLine("Disposed!");
d.Dispose();
});
});
var subscription = xs.Subscribe(x => Console.WriteLine(x));
Second case
but in the second case, where it doesn't end "naturally", I should dispose it.
Dispose is not triggered unless manually disposed.
var xs = Observable.Create<long>(o =>
{
var d = Observable.Timer(TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1))
.Do(e =>
{
Console.WriteLine("Test.");
})
.Subscribe(o);
return Disposable.Create(() =>
{
Console.WriteLine("Disposed!");
d.Dispose();
});
});
var subscription = xs.Subscribe(x => Console.WriteLine(x));
Conclusion
He gave such a nice examples, it's worth seeing if you are asking yourself the same question.
I have a function like such:
static void AddResultsToDb(IEnumerable<int> numbers)
{
foreach (int number in numbers)
{
int result = ComputeResult(number); // This takes a long time, but is thread safe.
AddResultToDb(number, result); // This is quick but not thread safe.
}
}
I could solve this problem by using, for example, Parallel.ForEach to compute the results, and then use a regular foreach to add the results to the database.
However, for educational purposes, I would like a solution that revolves around await/async. But no matter how much I read about it, I cannot wrap my mind around it. If await/async is not applicable in this context, I would like to understand why.
As others have suggested, this isn't a case of using async/await as that is for asynchrony. What you're doing is concurrency. Microsoft has a framework specifically for that and it solves this problem nicely.
So for learning purposes, you should use Microsoft's Reactive Framework (aka Rx) - NuGet System.Reactive and add using System.Reactive.Linq; - then you can do this:
static void AddResultsToDb(IEnumerable<int> numbers)
{
numbers
.ToObservable()
.SelectMany(n => Observable.Start(() => new { n, r = ComputeResult(n) }))
.Do(x => AddResultToDb(x.n, x.r))
.Wait();
}
The SelectMany/Observable.Start combination allows as many ComputeResult calls to occur as possible concurrently. The nice thing about Rx is that it then serializes the results so that only one call at a time goes to AddResultToDb.
To control the degrees of parallelism you can change the SelectMany to a Select/Merge like this:
static void AddResultsToDb(IEnumerable<int> numbers)
{
numbers
.ToObservable()
.Select(n => Observable.Start(() => new { n, r = ComputeResult(n) }))
.Merge(maxConcurrent: 2)
.Do(x => AddResultToDb(x.n, x.r))
.Wait();
}
The async and await pattern is not really suitable for your first method. It's well suited for IO Bound workloads to achieve scalability, or for frameworks that have UI's for responsiveness. It's less suited for raw CPU workloads.
However you could still get benefits from parallel processing because your first method is expensive and thread safe.
In the following example I used Parallel LINQ (PLINQ) for a fluent expression of the results without worrying about a pre-sized array / concurrent collection / locking, though you could use other TPL functionality, like Parallel.For/ForEach
// Potentially break up the workloads in parallel
// return the number and result in a ValueTuple
var results = numbers.AsParallel()
.Select(x => (number: x, result: ComputeResult(x)))
.ToList();
// iterate through the number and results and execute them serially
foreach (var (number, result) in results)
AddResultToDb(number, result);
Note : The assumption here is the order is not important
Supplemental
Your method AddResultToDb looks like it's just inserting results into a database, which is IO Bound and is worthy of async, furthermore could probably take all results at once and insert them in bulk/batch saving round trips
From Comments credit #TheodorZoulias
To preserve the order you could use the method AsOrdered, at
the cost of some performance penalty. A possible performance
improvement is to remove the ToList(), so that the results are added
to the DB concurrently with the computations.
To make the results available as fast as possible it's probably a good
idea to disable the partial buffering that happens by default, by
chaining the method
.WithMergeOptions(ParallelMergeOptions.NotBuffered) in the query
var results = numbers.AsParallel()
.Select(x => (number: x, result: ComputeResult(x)))
.WithMergeOptions(ParallelMergeOptions.NotBuffered)
.AsOrdered();
Example
Additional resources
ParallelEnumerable.AsOrdered Method
Enables treatment of a data source as if it were ordered, overriding
the default of unordered. AsOrdered may only be invoked on non-generic
sequences
ParallelEnumerable.WithMergeOptions
Sets the merge options for this query, which specify how the query
will buffer output.
ParallelMergeOptions Enum
NotBuffered Use a merge without output buffers. As soon as result elements have been computed, make that element available to the
consumer of the query.
This isn't really a case for async/await because it sounds like ComputeResult is expensive computationally, as opposed to just taking a long, indeterminate amount of time. aync/await is better for tasks you are truly waiting on. Parallel.ForEach will actually thread your workload.
If anything, AddResultToDb is what you would want to async/await - you would be waiting on an external action to complete.
Good in-depth explanation: https://stackoverflow.com/a/35485780/127257
Using Parallel.For honestly seems like the simplest solution, since your computations are likely to be CPU-bound. Async/await is better for I/O bound operations since it does not require another thread to wait for an I/O operation to complete (see there is no thread).
That being said, you can still use async/await for tasks that you put on the thread pool. So here's how you could do it.
static void AddResultToDb(int number)
{
int result = ComputeResult(number);
AddResultToDb(number, result);
}
static async Task AddResultsToDb(IEnumerable<int> numbers)
{
var tasks = numbers.Select
(
number => Task.Run( () => AddResultToDb(number) )
)
.ToList();
await Task.WhenAll(tasks);
}
Well, the title is self explanatory.
My code is something like this:
var clients = new ConcurrentBag<SharingClient>();
var peers = new ConcurrentDictionary<Task, SharingClient>();
clients.Add(...); // This may happen on another thread
clients.Add(...); // This may happen on another thread
var token = cts.Token;
while (!token.IsCancellationRequested)
{
if (clients.Count == 0)
{
// Waiting for more clients
await TaskEx.Delay(500, token); // Using BCL.Async as I need to support Windows XP (.Net 4)
}
else if (clients.Count == 1)
{
// Close the left-alone client
clients.FirstOrDefault()?.Close(); // Close may remove or modify **_clients**.
}
else if (clients.Any(c => c.DataAvailable))
{
// There is some data, let see if we have timed out peers from last time
foreach (var p in peers.Where(peer => !peer.Key.IsCompleted))
{
// Close the timed out clients, will also terminate the task
p.Value.Close(); // Close may remove or modify **_clients**.
}
peers.Clear();
// Checking for possible operations and filling the array
Parallel.ForEach(clients.Where(c => c.DataAvailable), (item) =>
{
Parallel.ForEach(clients.Where(c => !c.Equals(item)), (item2) =>
{
peers.TryAdd(item2.Broadcast(item.Data), item2); //Broadcast may add, remove or modify **_clients**.
});
});
// Minimum of 10 secs for operations to run before getting timedout
await TaskEx.WhenAny(TaskEx.WhenAll(peers.Keys.ToArray()), TaskEx.Delay(10000, token)); // Using BCL.Async as I need to support Windows XP (.Net 4)
// Even tho some of them may have timed out by now, as we may have no data for the next operation, we will wait for data before deciding to close them
}
else
{
// Waiting for some data to appear - Recheck as soon as OS allows us
await TaskEx.Delay(1, token); // Using BCL.Async as I need to support Windows XP (.Net 4)
}
}
As you can clearly see, I used ConcurrentBag and ConcurrentDictionary along with await, Linq, Simple foreach, Parallel.ForEach and BCL.Async methods.
As this is the first time I am using Parallel, ConcurrentBag and ConcurrentDictionary; I want to ask the people more familiar with these parts of the Framework to tell me if there is a problem in logic, thread safety and/or new ways of doing things.
I am all ears, Thank you in advance
EDIT
Based on one MSDN documentation mentioned in Ivan's answer, for more safety I should remove the .Where() method and merge it into the Parallel.ForEach's body as an if.
But still, there is little about .FirstOrDefault() and .Any() methods and their thread safety situation. I know that this is a good practice to use lock every time a variable is going to get accessed. But in this case as I am using a ConcurrentBag, I want to be sure about the necessity of copying the variable before running this lines of code:
if (clients.Any(client => client.IsConnected))
{
clients.FirstOrDefault()?.Close(); // Close may remove or modify **_clients**.
EDIT 2
By decompiling the .FirstOrDefault() and .Any() methods, I found both of them using a simple foreach. So I believe they should be thread safe after all. So now the question is:
Is running a Parallel.ForEach inside an other Parallel.ForEach with same source and then modifying this source, alright? Or should I change the logic here?
Parallel.ForEach(clients, (item) =>
{
if (item.DataAvailable)
{
Parallel.ForEach(clients, (item2) =>
{
if (!item2.Equals(item))
{
// Modify **clients**
From ConcurrentBag Class documentation:
Thread Safety
All public and protected members of ConcurrentBag<T> are thread-safe and may be used concurrently from multiple threads. However, members accessed through one of the interfaces the ConcurrentBag<T> implements, including extension methods, are not guaranteed to be thread safe and may need to be synchronized by the caller.
Note the bold italic paragraph. Since LINQ is based on one of the interface implemented (IEnumerable<T>), your usage is not thread safe.
I have a
List<object> listOfData;
in C# that gets data every X minute from a getDataBackgroundWorker.
I have another processDataBackgroundWorker that waits for the data from listOfData and processes it.
How can I make sure that I only get unique data from listOfData ( new data is added always ) and how can I ask the processDataBackgroundWorker to pause when there is no new data in listOfData?
List<> isn't a great choice in concurrency - there are out of the box alternatives, like ConcurrentBag, ConcurrentQueue which already have a lot of hard work done for you.
Here's an implementation of a producer-consumer pattern, using a BlockingCollection implementation as per MSDN,
The BlockingCollection is backed with a ConcurrentQueue, assuming that we are serially pull data in sequence on the consumer.
Methods which iterate over BlockingCollection block (with little overhead) until an item is available (i.e. your 'pause' is inherent - no need for a looped check with Thread.Sleeps on the consumer).
Termination is inherently built in, when the producer calls CompletedAdding
If you have more than one concurrent (competing) consumer, only one consumer will get an item, i.e. the duplication condition shouldn't be a concern (unless you mean the producer actually adds duplicates in the first place).
var queue = new BlockingCollection<string>(new ConcurrentQueue<string>());
var producer = Task.Run(() =>
{
// Produce some random messages, with delays in between
queue.Add("Hello!");
Thread.Sleep(1000);
queue.Add("World!");
Thread.Sleep(2000);
Enumerable.Range(0, 100)
.ToList()
.ForEach(x => queue.Add(x.ToString()));
queue.CompleteAdding();
});
var consumer = Task.Run(() =>
{
while (!queue.IsCompleted)
{
try
{
Debug.WriteLine(queue.Take());
}
catch (InvalidOperationException)
{
}
}
});
Task.WaitAll(producer, consumer);
I've been doing some work lately with the Reactive Framework and have been absolutely loving it so far. I'm looking at replacing a traditional polling message queue with some filtered IObservables to clean up my server operations. In the old way, I dealt with messages coming into the server like so:
// Start spinning the process message loop
Task.Factory.StartNew(() =>
{
while (true)
{
Command command = m_CommandQueue.Take();
ProcessMessage(command);
}
}, TaskCreationOptions.LongRunning);
Which results in a continuously polling thread that delegates commands from clients out to the ProcessMessage method where I have a series of if/else-if statements that determine the type of the command and delegate work based on its type
I am replacing this with an event driven system using Reactive for which I've written the following code:
private BlockingCollection<BesiegedMessage> m_MessageQueue = new BlockingCollection<BesiegedMessage>();
private IObservable<BesiegedMessage> m_MessagePublisher;
m_MessagePublisher = m_MessageQueue
.GetConsumingEnumerable()
.ToObservable(TaskPoolScheduler.Default);
// All generic Server messages (containing no properties) will be processed here
IDisposable genericServerMessageSubscriber = m_MessagePublisher
.Where(message => message is GenericServerMessage)
.Subscribe(message =>
{
// do something with the generic server message here
}
My question is that while this works, is it good practice to use a blocking collection as the backing for an IObservable like this? I don't see where Take() is ever called this way which makes me think that the Messages will pile off on the queue without being removed after they have been processed?
Would it be more efficient to look into Subjects as the backing collection to drive the filtered IObservables that will be picking up these messages? Is there anything else I'm missing here that might benefit the architecture of this system?
Here is a complete worked example, tested under Visual Studio 2012.
Create a new C# console app.
Right click on your project, select "Manage NuGet Packages", and add "Reactive Extensions - Main
Library".
Add this C# code:
using System;
using System.Collections.Concurrent;
using System.Reactive.Concurrency;
using System.Reactive.Linq;
namespace DemoRX
{
class Program
{
static void Main(string[] args)
{
BlockingCollection<string> myQueue = new BlockingCollection<string>();
{
IObservable<string> ob = myQueue.
GetConsumingEnumerable().
ToObservable(TaskPoolScheduler.Default);
ob.Subscribe(p =>
{
// This handler will get called whenever
// anything appears on myQueue in the future.
Console.Write("Consuming: {0}\n",p);
});
}
// Now, adding items to myQueue will trigger the item to be consumed
// in the predefined handler.
myQueue.Add("a");
myQueue.Add("b");
myQueue.Add("c");
Console.Write("[any key to exit]\n");
Console.ReadKey();
}
}
}
You will see this on the console:
[any key to exit]
Consuming: a
Consuming: b
Consuming: c
The really nice thing about using RX is that you can use the full power of LINQ to filter out any unwanted messages. For example, add a .Where clause to filter by "a", and observe what happens:
ob.Where(o => (o == "a")).Subscribe(p =>
{
// This will get called whenever something appears on myQueue.
Console.Write("Consuming: {0}\n",p);
});
Philosophical notes
The advantage of this method over starting up a dedicated thread to poll the queue, is that you don't have to worry about disposing of the thread properly once the program has exited. This means you don't have to bother with IDisposable or CancellationToken (which is always required when dealing with a BlockingCollection or else your program might hang on exit with a thread that refuses to die).
Believe me, its not as easy as you think to write completely robust code to consume events coming out of a BlockingCollection. I much prefer using the RX method, as shown above as its cleaner, more robust, has less code, and you can filter using LINQ.
Latency
I was surprised at how fast this method is.
On my Xeon X5650 # 2.67Ghz, it takes 5 seconds to process 10 million events, which works out at approximately 0.5 microseconds per event. It took 4.5 seconds to put the items into the BlockingCollection, so RX was taking them out and processing them almost as fast as they were going in.
Threading
In all of my tests, RX only spun up one thread to handle the tasks on the queue.
This means that we have a very nice pattern: we can use RX to collect incoming data from multiple threads, place them into a shared queue, then process the queue contents on a single thread (which is, by definition, thread safe).
This pattern eliminates a huge amount of headaches when dealing with multithreaded code, by decoupling the producer and consumer of data via a queue, where the producer could be multi-threaded and the consumer is single-threaded and thus thread-safe. This is the concept that makes Erlang so robust. For more information on this pattern, see Multi-threading made ridiculously simple.
Here's something pulled directly from my posterior - any real solution would be very much dependent on your actual usage, but here's "The cheapest pseudo Message Queue system ever":
Thoughts/motivations:
Deliberate exposure of IObservable<T> such that subscribers can do any filtering/cross subscriptions they want to
The overall Queue is typeless, but Register and Publish are type-safe(ish)
YMMV with the Publish() where it is - try experimenting with moving it around
Generally Subject is a no-no, although in this case it does make for some SIMPLE code.
One could "internalize" the registration to actually do the subscription as well, but then the queue would need to manage the IDisposables created - bah, let your consumers deal with it!
The Code:
public class TheCheapestPubSubEver
{
private Subject<object> _inner = new Subject<object>();
public IObservable<T> Register<T>()
{
return _inner.OfType<T>().Publish().RefCount();
}
public void Publish<T>(T message)
{
_inner.OnNext(message);
}
}
Usage:
void Main()
{
var queue = new TheCheapestPubSubEver();
var ofString = queue.Register<string>();
var ofInt = queue.Register<int>();
using(ofInt.Subscribe(i => Console.WriteLine("An int! {0}", i)))
using(ofString.Subscribe(s => Console.WriteLine("A string! {0}", s)))
{
queue.Publish("Foo");
queue.Publish(1);
Console.ReadLine();
}
}
Output:
A string! Foo
An int! 1
HOWEVER, this doesn't strictly enforce "consuming consumers" - multiple Registers of a specific type would result in multiple observer calls - that is:
var queue = new TheCheapestPubSubEver();
var ofString = queue.Register<string>();
var anotherOfString = queue.Register<string>();
var ofInt = queue.Register<int>();
using(ofInt.Subscribe(i => Console.WriteLine("An int! {0}", i)))
using(ofString.Subscribe(s => Console.WriteLine("A string! {0}", s)))
using(anotherOfString.Subscribe(s => Console.WriteLine("Another string! {0}", s)))
{
queue.Publish("Foo");
queue.Publish(1);
Console.ReadLine();
}
Results in:
A string! Foo
Another string! Foo
An int! 1
I haven't used BlockingCollection in this context - so I'm 'conjecturing' - you should run it to approve, disprove.
BlockingCollection might only further complicate things here (or provide little help). Take a look at this post from Jon - simply to confirm. GetConsumingEnumerable will provide you with 'per subscriber' enumerable. Exhausting them down eventually - something to have in mind with Rx.
Also the the IEnumerable<>.ToObservable further flattens out the 'source'. As it works (you can lookup the source - I'd recommend w/ Rx more than anything) - each subscribe creates an own 'enumerator' - so all will be getting their own versions of the feed. I'm really not sure, how that pans out in the Observable scenario like this.
Anyhow - if you want to provide app-wide messages - IMO you'd need to introduce Subject or state in some other form (e.g. Publish etc.). And in that sense, I don't think BlockingCollection will help any - but again, it's best that you try it out yourself.
Note (a philosophical one)
If you want to combine message types, or combine different sources - e.g. in a more 'real world' scenario - it gets more complex. And it gets quite interesting I must say.
Keep an eye on having them 'rooted' into a single-shared stream (and avoid what Jer suggested rightly).
I'd recommend that you don't try to evade using Subject. For what you need, that's your friend - no matter all the no-state related discussions and how Subject is bad - you effectively have a state (and you need a 'state') - Rx kicks in 'after the fact', so you enjoy benefits from it regardless.
I encourage you to go that way, as I love it how it turned out.
My issue here is that we have turned a Queue (which I normally associate with destructive reads by one consumer especially if you are using BlockingCollection) into a broadcast (send to anyone and everyone listening right now).
These seem two conflicting ideas.
I have seen this done, but it then was thrown away as it was the "right solution to the wrong question".