We Keep Coding

Subscribe/Listen to "side events" of the stream with System.Reactive

I'm starting with reactive extensions and I'm having a problem where I'm not sure if I'm on the right track.
I'm using an Observable to create and consume a listener for an event broker with .NET. I created a "IncomingMessage" class which contains the messages from the eventbroker as they come in and I start creating the listener in the Observerable.Create function. That works very well.
Now I also want to get status notification from the listener as in "Connecting...", "Connected.", "Closing..." which are not an IncomingMessage so I created a class "BrokerEvent" with a "Message" property and an interface for "IncomingMessage" and "BrokerEvent". Now I send both via observer.OnNext(...) as they occur. That also works well so far.
However on the Subscriber side I'm now having a bit of a problem to filter the events I need.
I do:
GetObservable().Where(x => x is BrokerEvent ||
(x is IncomingMessage msg &&
msg.User == "test")).Subscribe(...)
That works however I then need to figure out the type in Subscribe again which seems a bit ugly.
After trying a bit I ended up doing this now...
var observable = GetObservable().Publish();
observable.OfType<BrokerEvent>().Subscribe(...);
observable.OfType<IncomingMessage>().Where(x=>x.User == "test").Subscribe(...);
var disposable = observable.Connect();
This also seems to work but as I'm new to reactive extensions I'm not quite sure if that has any unwanted sideeffects. I'm also not sure if it's the "right" way to include status messages into the stream at all. Is there any better way to handle that (possible without using Publish) or is that the way to go?
And to stop listening is it enough to just dispose the disposable I get from .Connect() or do I have to also dispose both disposables I get from .Subscribe()?
Thanks!

I'm assuming GetObservable returns IObservable<object>, which isn't ideal.
The best way to do the code you have above is as follows:
var observable = GetObservable().Publish();
var sub1 = observable.OfType<BrokerEvent>().Subscribe(_ => { });
var sub2 = observable.OfType<IncomingMessage>().Where(x => x.User == "test").Subscribe(_ => { });
var connectSub = observable.Connect();
var disposable = new CompositeDisposable(sub1, sub2, connectSub);
The composite disposable will then dispose of all the children when it is disposed.
If the two message streams have nothing to do with each other, that approach will work. However, since you basically have a control-message stream, and a data-message stream, I'm assuming the messages from one may be important when handling the messages in the other. In this case you may want to treat it like one stream.
In that case, you may want to create a Discriminated-Union type for your observable, which may make handling easier.

What you can do is create an 'event handler' class with three overloads of a 'process message'. One for object (default) that does nothing, one for the status type, one for incoming message. In .Subscribe use this syntax
m=>processor.Process((dynamic)m)
This will call the correct implementation or do nothing, as required.
If you want to filter before calling the Process, you can introduce a common class (ProcessableMesage or some such), or you can call .Merge on your OfType streams, or you can take the same approach as above by having a dynamic MessageFilter class.

Reactive Extensions unfold / scan approach for nested hierarchy

I'm currently building a wizard system for an application, and we're using ReactiveUI and as a result Rx.
Each step in the wizard implements IWizardStep<T> where T is just the data type that the wizard ultimately produces.
Each step has the capability of surfacing which step should be available for the user to move to next, so as to enable branching based on data entry.
The step can be thought of having a similar structure to:
public interface IWizardStep<T>
{
IObservable<IStepChoice<T>> NextStepChoice {get;}
}
With IStepChoice<T> simply being:
public interface IStepChoice<T>
{
IWizardStep<T> Step {get;}
string Reason {get;}
}
In order to calculate the current path from the start to the end, for display to the user, I need to be able to go from the starting step, and walk along the NextStepChoice chain recursively until it hits a null (it's valid behaviour for the NextStepChoice observable to emit a null to indicate the end of the wizard).
I've taken a look at Observable.Scan but can't for the life of me work out how to get this working properly recursively.
I've also taken a look at Observable.Generate which looks promising, as this is a classic unfold-esque problem; the only issue is that Generate takes a function to determine when to break the loop, but I need to evaluate the inner observable to work this out.
Observable.Generate(
new WizardStepChoice<T>(start, null),
choice => choice != null,
choice => choice.ChosenStep.NextStepChoice,
choice => choice);
This would be ideal, and produce the output I'm after, but the NextStepChoice selector there obviously doesn't compile because it's an IObservable<IWizardStepChoice<T>> rather than an IWizardStepChoice<T>.
I've looked at using Aggregate and Scan but as these are more fold-driven operations, and I've only got the starting element, it's an unfold I'm looking for ala Generate, but I need it to be able to evaluate the nested observable.
Would Observable.Create perhaps be something I could utilise? I've tried it and come up with:
Path = CurrentStep.Select(_ => Observable.Create<IWizardStep<T>>(async observer =>
{
IWizardStepChoice<T> next = new WizardStepChoice<T>(start, null);
observer.OnNext(next.ChosenStep);
while (next != null)
{
next = await next.ChosenStep.NextStepChoice;
observer.OnNext(next.ChosenStep);
}
observer.OnCompleted();
return Disposable.Empty;
}).Aggregate(new List<IWizardStep<T>>(),
(l, s) =>
{
l.Add(s);
return l;
})).Switch().Publish().RefCount();
Which has all the right signature I want IWizardStep<T>->IReadOnlyList<IWizardStep<T>>, so at first glance it looks right, but it doesn't work; it fires, and I can step through, but it hangs once it hits the await and doesn't come back.
I've got a feeling I'm close, and this is a scheduling issue, so my question really is this:
What is the best approach to solve this, am I close?
If this is right, why is there an issue with the await, and how might I solve it?
Update
After a little bit of tinkering I noticed that the await was likely hanging as that observable hadn't yet (and wasn't going to) emit a value (duh), which I've now resolved by initialising each step with a value at the beginning of the wizard.
I've even sanity-checked this by adding a property to IWizardStep<T> - IWizardStepChoice<T> LatestStepChoice {get;} which is just hooked up with:
NextStepChoice.Subscribe(c => _latestStepChoice = c);
This is done on the step class itself, and I can confirm it's working just fine.
Yet still the await hangs, so I tried:
Making it Replay(1) so the await calling .Subscribe() would get the value - this didn't work
Making it Repeat() so even if something is subscribed it'll see the new value - this just made the whole thing hang.
Clearly I'm missing something here, I want it so that when the await queries the observable, it will be given the latest value seen, which is what I thought Replay(1) would achieve; I've also tried PublishLast() but then future updates don't get honoured because of the AsyncSubject<T> behaviour.
For now I've switched to using the self-subscribed property, but it's not ideal, I'd rather not have to break out of querying the observables if I can help it, it feels "hacky".

A recursive walk can transform the tree of observables into a single observable:
static IObservable<IWizardStep<T>> Walk<T>(IWizardStep<T> step)
{
if (step?.NextStepChoice == null)
return Observable.Return(step);
return step.NextStepChoice.SelectMany(choice => Walk(choice.Step)).StartWith(step);
}
Usage:
var steps = await Walk(step).ToArray();

How would I write this code with Reactive Programming?

I just started messing around with reactive programming, and I know just enough to write code but not enough to figure out what's happening when I don't get what I expect. I don't really have a mentor available other than blog posts. I haven't found a very good solution to a situation I'm having, and I'm curious about the right approach.
The problem:
I need to get a Foo, which is partially composed of an array of Bar objects. I fetch the Bar objects from web services. So I represented each web service call as an IObservable from which I expect 0 or 1 elements before completion. I want to make an IObservable that will:
Subscribe to each of the IObservable instances.
Wait for up to a 2 second Timeout.
When either both sequences complete or the timeout happens:
Create an array with any of the Bar objects that were generated (there may be 0.)
Produce the Foo object using that Bar[].
I sort of accomplished this with this bit of code:
public Foo CreateFoo() {
var producer1 = webService.BarGenerator()
.Timeout(TimeSpan.FromSeconds(2), Observable.Empty<Bar>());
var producer2 = // similar to above
var pipe = producer1.Concat(producer2);
Bar[] result = pipe.ToEnumerable().ToArray();
...
}
That doesn't seem right, for a lot of reasons. The most obvious is Concat() will start the sequences serially rather than in parallel, so that's a 4-second timeout. I don't really care that it blocks, it's actually convenient for the architecture I'm working with that it does. I'm fine with this method becoming a generator of IObservable, but there's a few extra caveats here that seem to make that challenging when I try:
I need the final array to put producer1 and producer2's result in that order, if they both produce a result.
I'd like to use a TestScheduler to verify the timeout but haven't succeeded at that yet, I apparently don't understand schedulers at all.
This is, ultimately, a pull model, whatever gets the Foo needs it at a distinct point and there's no value to receiving it 'on the fly'. Maybe this tilts the answer to "Don't use Rx". To be honest, I got stuck enough I switched to a Task-based API. But I want to see how one might approach this with Rx, because I want to learn.

var pipe = producer1
.Merge(producer2)
.Buffer(Observable.Timer(TimeSpan.FromSeconds(2), testScheduler))
.Take(1);
var subscription = pipe
.Select(list => new Foo(list.ToArray())
.Subscribe(foo => {} /* Do whatever you want with your foo here.*/);
Buffer takes all elements emitted during a window (in our case in two seconds), and outputs a list.
If you want to stick with your pull model, instead of a subscription you could do:
var list = await pipe;
var foo = new Foo(list.ToArray());
//....

RX Switch()'s Subscribe and Unsubscribe Order

So I have written a rough draft of the problem I'm encountering. I have an IObserverable< IObservable< TResult>> which contains my stream, and I would like to use switch to get the latest items from it, however the problem I have can be neatly demonstrated with the code below:
var obs = Observable.Create<IObservable<int>>(sub =>
{
var item = Observable.Create<int>(innersub =>
{
var count = 0;
return Observable.Interval(TimeSpan.FromSeconds(2)).Subscribe(x => innersub.OnNext(count++));
}).Publish().RefCount();
return Observable.Interval(TimeSpan.FromSeconds(10)).Subscribe(x => sub.OnNext(item));
});
obs.Switch().Subscribe(x => Console.WriteLine(x));
The above test case shows that when used in conjunction with Publish().RefCount() the switch first unsubscribes and then subscribes to the new item.
What I would like is for a continuous stream of numbers going up, however the test shows the "item" is disposed of first before the new subscription hits and I lose that count and have to start again.
If the item is the same, and refcount is used, what I would like is for the subscription to happen first, so refcount is happy, and then the old subscription disposed of. Is this behavior that RX can demonstrate by default or would it require some bodging to get right? I'm confident I could write a simple enough extension method based on a cut down version of the RX source code but if it already exists or there is a better way I'd like to know first.
Edit: The code written was a naive example to demonstrate the problem in a simple way. What I actually have is an observable that publishes a new observable regularly, which has different filters on it, but which ultimately boils down to the same publish/refcount observable at the base of it all. (The where clause changes, or the select does something different. The real use is a .Merge() of several streams so I'm confident in my logic and my conclusion of the problem). I'm well aware that my example could be simplified.

You're going to have to look at the source as the previous observable gets disposed before the current one gets subscribed to. That's how .Switch() works.
If Rx disposed after the new subscription, the intent of your code appears to be the equivalent of simply doing this:
var obs = Observable.Create<int>(innersub =>
{
var count = 0;
return Observable.Interval(TimeSpan.FromSeconds(2))
.Subscribe(x => innersub.OnNext(count++));
});
obs.Subscribe(x => Console.WriteLine(x));
And in this example it then boils down to:
var obs = Observable.Interval(TimeSpan.FromSeconds(2));
obs.Subscribe(x => Console.WriteLine(x));
Perhaps you could let us know what your underlying requirement is and we could work on that?

This operator is designed primarily for cold observables. Switch unsubscribes from the previous observable before subscribing to the new one. Otherwise there would exist a race condition where extra events might slip through during the brief period it was subscribed to both.
Since your underlying observable is hot, you might consider an alternative solution where you just modify the filter/select "on the fly" instead of using Switch to "re-subscribe". Something like:
source
.Where(t => ApplyCurrentFilter(t))
.Select(t => ApplyCurrentProjection(t))
.Subscribe(...);
// do something that changes what `ApplyCurrentFilter` does...
I don't know if this is better or worse than your current solution, but it does avoid the need to unsubscribe/resubscribe from source data.

As has been said, Observable.Create produces a cold Observable, and Publish.RefCount only makes it hot while there are still subscribers there. It would be possible to write your own version of Switch that subscribed the new subscriber before disposing the old. But I'd be very wary of race conditions. And in general, it feels a bit of a weird thing to do, which in Rx usually signals there's another way to do what you want that's cleaner.
In this example, if you have the desired result, there's no purpose to publishing many Observables and Switching them because really you just want to be subscribed to one observable for the full duration. Hence how it boils down to what Enigmativity says.
However, obviously it's a contrived example so let's assume there's a more complicated situation that requires this approach - it may help if you're able to elaborate. From the example, it seems you only want to subscribe to the inner observable once, ever. Based on that requirement, RefCount is not appropriate, but I assume you're using it because you want a shared observable at the core, which you're wrapping with other operators that you want to act differently each time. If that's the case, you could possibly use an approach like this:
var obs = Observable.Create<IObservable<int>>(sub =>
{
var item = Observable.Create<int>(innersub =>
{
var count = 0;
return Observable.Interval(TimeSpan.FromSeconds(2))
.Subscribe(x => innersub.OnNext(count++));
}).Publish();
bool connected = false;
var disposables = new CompositeDisposable();
disposables.Add(Observable.Interval(TimeSpan.FromSeconds(10))
.Subscribe(x =>
{
// push the new stream to the observer first
sub.OnNext(item);
if (!connected)
{
connected = true;
disposables.Add(item.Connect());
}
}));
return disposables;
});
I haven't thought through potential race conditions with this approach, etc, and a lot depends on your real situation. However, in the basic test from the original post, this seems to behave how you want it to.

Rx Framework: execute an action on timeout without interrupting the original observable sequence

Given an observable source, generated by polling the (changes of a) state of a low-level device...
// observable source metacode:
IObservable<DeviceState> source = Observable.Interval(TimeSpan.FromSeconds(0.5))
.Select(tick => new DeviceState(_device.ReadValue()))
.DistinctUntilChanged();
... and a consumer that updates the UI...
// UI metacode:
service.GetObservableDeviceStates()
.Subscribe(state => viewModel.CurrentState = state.ToString());
... I need to execute a custom action after x seconds of source's "inactivity", without interrupting the subscription to source. Something like this:
// UI metacode:
service.GetObservableDeviceStates()
.DoOnTimeout(TimeSpan.FromSeconds(x), () => viewModel.CurrentState = "Idle")
.Subscribe(state => viewModel.CurrentState = state.ToString());
What are the best practices? Possible solutions that come to mind are (I'm a Rx noob):
Buffer (even if it's not so readable)
Playing around this Timeout overload;
Returning something special "service-side" when nothing changes (instead of using DistinctUntilChanged) and dealing with it on the UI code:
service.GetObservableDeviceStates()
.Subscribe(state =>
viewModel.CurrentState = state.Special ? "Idle" : state.ToString());
EDIT: as reported in the answer, the solution is:
service.GetObservableDeviceStates()
.Do(onNext)
.Throttle(TimeSpan.FromSeconds(x))
.Subscribe(onTimeout);
EDIT2 (Warning)
If onNext and onTimeout updates UI components, to avoid CrossThreadExceptions two ObserveOn(uiSynchronizationContext) are needed, since Throttle works on another thread!
service.GetObservableDeviceStates()
.ObserveOn(uiSynchronizationContext)
.Do(onNext)
.Throttle(TimeSpan.FromSeconds(x))
.ObserveOn(uiSynchronizationContext)
.Subscribe(onTimeout);

Timeout is more or less meant for observables which represent single asynchronous operations - for e.g., to return a default value or OnError if said observable hasn't notified you in a certain amount of time.
The operator you're looking for is Throttle, even though it may not seem like it at first. Throttle(p) gives you a stream which produces a value when the source stream has not produced a value for period p.
Parallel to your existing code, you can use source.Throttle(period).Do(...side effect).

I personally would avoid the Do method for this. It does make the code in this example fairly easy, but I find once the use of 'Do' sneaks into the code base you soon have spaghetti.
You could also consider using combinations of Amb, Timer, TakeUntil, Throttle etc to get the result you are looking for and still maintaining the Monad*. Or in simple terms, I assume you ideally want to have a sequence of the status values coming through and not require the need to put a timer in your code (i.e. off load it to the service).
public IObservable<DeviceStatus> GetObservableDeviceStates(TimeSpan silencePeriod)
{
return Observable.Create<DeviceStatus>(
o=>
{
var idle = Observable.Timer(silencePeriod).Select(_=>new DeviceStatus("Idle"));
var polledStatus = Observable.Interval(TimeSpan.FromSeconds(0.5))
.Select(tick => new DeviceStatus(_device.ReadValue()))
.DistinctUntilChanged()
.Publish();
var subscription = (from status in polledStatus
from cont in Observable.Return(status).Concat(idle.TakeUntil(polledStatus))
select cont)
.Subscribe(o);
return new CompositeDisposable(subscription, polledStatus.Connect());
});
}
This code now has the Service returning an Idle status value once the specified period of change silence has occurred.
This means your UI meta code stays simple and the logic related to DeviceStatus stays where it belongs
// UI metacode:
service.GetObservableDeviceStates(TimeSpan.FromSeconds(2))
.Subscribe(state => viewModel.CurrentState = state.ToString());