We Keep Coding

Proper approach to dealing with intermittent errors and Observables

I am using the GraphQL.NET client to subscribe to data on a remote service. The client returns an Observable so when the subscription is created you, as expected, receive new messages in onNext and get errors (both initial connection errors, reconnection errors, and anything else) in onError. The GraphQL client has the ability to automatically reconnect if the initial connection fails or when an established connection drops.
I know that by convention, any messages coming in on onError is supposed to terminate the sequence of messages. However, somehow they are able to continue sending to onNext and onError after that first onError. I have tried reading through the code but it is confusing. There seems to be multiple nesting of Observable and I suspect they are creating a new sequence when they encounter an error.
To clarify my issue, suppose I had the following pseudo Event based wrapper class.
public class PubSubSubscription() {
...
public void CreateSubscription<TResponse>(string topic) {
// GraphQL client
var stream = client
.CreateSubscriptionStream<FixConnectionChangedSubscriptionResult>(...);
stream
.Subscribe(
response => {
// Do stuff with incoming data (validation, mapping, logging, etc.)
// send it on the UI
DataReceived?.Invoke(this, new DataReceivedEventArgs { Message = response });
},
ex => {
// ******************************
// Note that the Observable created by CreateSubscriptionStream()
// will call `onError` over-and-over since it _seems_ like it is
// creating (and re-creating) nested Observables in its own
// classes. In the event of an initial connection failure or
// re-connect it will raise an error and then automatically
// try again.
// ******************************
// send it on to UI
ErrorDetected?.Invoke(this, new ErrorDetectedEventArgs { Exception = ex });
});
}
...
}
I would then call it as follows (or close enough)...
...
var orders = ordersPubSub.CreateSubscription("/orders");
orders.DataReceived += OnDataReceived;
orders.ErrorDetected += OnErrorDetected;
void OnErrorDetected(object sender, ErrorDetectedEventArgs e) {
// Can be called multiple times
// Display message in UI
}
...
I am having trouble converting that event-based wrapper approach to an Observable wrapper approach.
public class PubSubSubscription() {
...
public IObservable<TResponse> CreateSubscription<TResponse>(string topic) {
// Observable that I give back to my UI
var eventSubject = new Subject<TResponse>();
// GraphQL client
var stream = client
.CreateSubscriptionStream<FixConnectionChangedSubscriptionResult>(...);
stream
.Subscribe(
response => {
// Do stuff with incoming data (validation, mapping, logging, etc.)
// send it on the UI
eventSubject.onNext(response);
},
ex => {
// ******************************
// Note that the Observable created by CreateSubscriptionStream()
// will call `onError` over-and-over since it _seems_ like it is
// creating (and re-creating) nested Observables in its own
// classes. In the event of an initial connection failure or
// re-connect it will raise an error and then automatically
// try again.
// ******************************
// send it on to UI
eventSubject.onError(ex);
});
return eventSubject.AsObservable();
}
...
}
This I would then call it as follows (or close enough)...
...
var orders = ordersPubSub.CreateSubscription("/orders");
orders
// Things I have tried...
// Do() by itself does not stop the exception from hitting onError (which makes sense)
.Do(
_ => { },
ex => // display in UI)
// Retry() seems to cause the GraphQL subscription to "go away" because I no longer see connection attempts
.Retry()
// Stops the exception from hitting onError but the sequence still stops since I need to return _something_ from this method
.Catch(() => {
// display in UI
return Observable.Empty<T>();
})
.Subscribe(
msg => // do something with data,
ex => // display in UI);
}
...
Bottom line is what is the proper approach to dealing with sequences that can be "temporarily interrupted"?
I am also unsure of the idea of pushing the responsibility of retries onto the observer. This means that I would need to duplicate the logic each time CreateSubscription() is called. Yet, if I move it into the CreateSubscription() method, I am still unsure how to let the observer know the interruption happened so the UI can be updated.
One approach I am playing with (after reading about it as a possible solution) is to wrap my TResponse in a "fake" SubscriptionResponse<TResponse> which has T Value and Exception Error properties so the outer Observable only has onNext called. Then in my Subscribe I add if/else logic to check if Error is non-null and react accordingly. But this just feels ugly... I would almost want to go back to using events...

If you have an unruly observable - one that produces multiple errors without ended - you can make it workable by doing this:
IObservable<int> unruly = ...;
IObservable<Notification<int>> workable =
unruly
.Materialize();
The Materialize operator turns the IObservable<int> into an IObservable<Notification<int>> where the OnCompleted, OnError, and OnNext messages all get converted to OnNext messages that you can inspect like this:
Now you can deal with the errors without the sequence ending. When you've cleared them you can restore the sequence with Dematerialize like so:
IObservable<int> ruly =
workable
.Where(x => x.Kind != NotificationKind.OnError)
.Dematerialize();

How to handle the exception thrown by the async method with observable?

I have an observable and I would like to subscribe this observable with an async method, however each time the exception thrown by the async method, the subscription disposed immediately even if I put the catch code in the observable definition. The pseudo code as follow to demonstrate this situation:
[Fact]
public async Task Test()
{
var observable = Observable.Create<int>(observer =>
{
try
{
Enumerable.Range(1, 10).ToList().ForEach(x =>
{
observer.OnNext(x);
});
}
catch (Exception ex)
{
// get called after the exception is thrown
_testOutputHelper.WriteLine($"The exception is catch:{ex.ToString()}");
}
return Disposable.Create(() =>
{
// also get called after exception is thrown
_testOutputHelper.WriteLine("Observable Dispose");
});
});
Func<int, Task> handler = async (i) =>
{
// simulate the handler logic
await Task.Delay(TimeSpan.FromSeconds(1));
// throw the exception to test
throw new Exception($"{i}");
};
observable.Subscribe(x=>handler(x).Wait());
await Task.Delay(TimeSpan.FromSeconds(10));
}
From above code, I don`t understand why the dispose delegate get called even the exception is catch (For some reason, I have to deal with the exception inside the observable definition), Is there any way to prevent the subscription being disposed when the exception thrown from async method?

What's happening in your code is a direct consequence of you using Observable.Create and filling the observable with this code:
Enumerable.Range(1, 10).ToList().ForEach(x =>
{
observer.OnNext(x);
});
Observable.Create uses the current thread to create the observable, so the Enumerable.Range(1, 10).ToList().ForEach executes immediately on the current thread and the call to OnNext executes the handler(x).Wait() immediately.
You'll note, though, that the exception occurs in the delegate passed to the Subscribe. Internally there is code like this:
catch (Exception exception)
{
if (!autoDetachObserver.Fail(exception))
{
throw;
}
return autoDetachObserver;
}
That catches the exception in the subscribe, cancels the subscription - hence the "Observable Dispose" message - and then rethrows the exception and that's where your code catches it.
Now, if you wanted to do this properly in Rx, you'd avoid Observable.Create. It's a tempting way to create observables, but it leads to trouble.
Instead do this:
public async Task Test()
{
Func<int, Task> handler = async (i) =>
{
// simulate the handler logic
await Task.Delay(TimeSpan.FromSeconds(1));
// throw the exception to test
throw new Exception($"{i}");
};
await
Observable
.Range(1, 10)
.SelectMany(i => Observable.FromAsync(() => handler(i)))
.LastOrDefaultAsync();
}
But, of course, we want to handle the exception. The simple way is like this:
public async Task Test()
{
Func<int, Task> handler = async (i) =>
{
// simulate the handler logic
await Task.Delay(TimeSpan.FromSeconds(1));
// throw the exception to test
throw new Exception($"{i}");
};
await
Observable
.Range(1, 10)
.SelectMany(i =>
Observable
.FromAsync(() => handler(i))
.Catch<Unit, Exception>(ex =>
{
Console.WriteLine($"The exception is catch:{ex.ToString()}");
return Observable.Empty<Unit>();
}))
.LastOrDefaultAsync();
}
That now outputs the 10 exception errors and completes normally.

Your code is doing exactly what you told it to.
The purpose of catching an exception is so that your program can continue without abruptly stopping. That's exactly what your code is doing: the exception is caught, then execution continues after the catch block.
If you want it to do something else, you have two options.
Rethrow the exception after logging it:
catch (Exception ex)
{
// get called after the exception is thrown
_testOutputHelper.WriteLine($"The exception is catch:{ex.ToString()}");
throw;
}
Then, whatever code called Test() will have the responsibility of catching that exception (or not).
Move the return inside the try block and return something else when the exception is caught:
try
{
Enumerable.Range(1, 10).ToList().ForEach(x =>
{
observer.OnNext(x);
});
return Disposable.Create(() =>
{
// also get called after exception is thrown
_testOutputHelper.WriteLine("Observable Dispose");
});
}
catch (Exception ex)
{
// get called after the exception is thrown
_testOutputHelper.WriteLine($"The exception is catch:{ex.ToString()}");
return //something else
}
You may benefit from reading Microsoft's documentation on Exception Handling.

How to implement a "better" Finally Rx operator?

Recently I become aware that the Rx Finally operator behaves in a way which, at least for me, is unexpected. My expectation was that any error thrown by the finallyAction would be propagated to the operator's observers downstream. Alas this is not what happens. In the reality the operator first propagates the completion (or the failure) of the antecedent sequence to its observers, and then invokes the action, at a point in time when it's not possible to propagate a potential error thrown by the action. So it throws the error on the ThreadPool, and crashes the process. Which is not only unexpected, but also highly problematic. Below is a minimal demonstration of this behavior:
Observable
.Timer(TimeSpan.FromMilliseconds(100))
.Finally(() => throw new ApplicationException("Oops!"))
.Subscribe(_ => { }, ex => Console.WriteLine(ex.Message),
() => Console.WriteLine("Completed"));
Thread.Sleep(1000);
Outcome: Unhandled exception (Fiddle)
The exception thrown by the Finally lambda is not handled by the Subscribe:onError handler, as it would be desirable.
This feature (I am tempted to call it a flaw) limits severely the usefulness of the Finally operator in my eyes. Essentially I can only use it when I want to invoke an action that is expected to never fail, and if it fails it would indicate a catastrophic corruption of the application's state, when no recovery is possible. I could use it for example to Release a SemaphoreSlim (like I've done here for example), which can only fail if my code has a bug. I am OK with my app crashing in this case. But I've also used it recently to invoke an unknown action supplied by the caller, an action that could potentially fail, and crashing the app in this case is unacceptable. Instead, the error should be propagated downstream. So what I am asking here is how to implement a Finally variant (let's call it FinallySafe) with identical signature, and the behavior specified below:
public static IObservable<TSource> FinallySafe<TSource>(
this IObservable<TSource> source, Action finallyAction);
The finallyAction should be invoked after the source sequence has emitted an OnCompleted or an OnError notification, but before this notification is propagated to the observer.
If the finallyAction invocation completed successfully, the original OnCompleted/OnError notification should be propagated to the observer.
If the finallyAction invocation failed, an OnError notification should be propagated to the observer, containing the error that just occurred. In this case the previous error, the one that may have caused the source to complete with failure, should be ignored (not propagated).
The finallyAction should also be invoked when the FinallySafe is unsubscribed before the completion of the source. When a subscriber (observer) disposes a subscription, the finallyAction should by invoked synchronously, and any error should be propagated to the caller of the Dispose method.
If the FinallySafe is subscribed by multiple observers, the finallyAction should be invoked once per subscription, independently for each subscriber, following the rules above. Concurrent invocations are OK.
The finallyAction should never be invoked more than once per subscriber.
Validation: replacing the Finally with the FinallySafe in the code snippet above, should result to a program that doesn't crash with an unhandled exception.
Alternative: I am also willing to accept an answer that provides a reasonable explanation about why the behavior of the built-in Finally operator is better than the behavior of the custom FinallySafe operator, as specified above.

Finally gets called after the sequence has ended, and since the Rx contract only allows one OnError or OnCompleted it can't issue a second one.
But, if you replace the Finally with Do you can get the behaviour that you want.
Try this code:
Observable
.Timer(TimeSpan.FromMilliseconds(100))
.Do(_ => { }, () => throw new ApplicationException("Oops!"))
.Subscribe
(_ => { },
ex => Console.WriteLine(ex.Message),
() => Console.WriteLine("Completed"));
Thread.Sleep(TimeSpan.FromMilliseconds(1000));
That operates as you expect it to.
I get this output:
Oops!
If you want to run something at unsubscribe, then use this extension method:
public static class Ext
{
public static IObservable<T> Unsubscribed<T>(this IObservable<T> source, Action unsubscribed) =>
Observable.Create<T>(o =>
new CompositeDisposable(source.Subscribe(o), Disposable.Create(unsubscribed)));
}
Here's an example of its use:
var source = Observable.Never<int>();
var subscription =
source
.Unsubscribed(() => Console.WriteLine("Unsubscribed"))
.Subscribe();
subscription.Dispose();
That outputs:
Unsubscribed

Here is an implementation of the FinallySafe operator, having the behavior specified in the question:
/// <summary>
/// Invokes a specified action after the source observable sequence terminates
/// successfully or exceptionally. The action is invoked before the propagation
/// of the source's completion, and any exception thrown by the action is
/// propagated to the observer. The action is also invoked if the observer
/// is unsubscribed before the termination of the source sequence.
/// </summary>
public static IObservable<T> FinallySafe<T>(this IObservable<T> source,
Action finallyAction)
{
return Observable.Create<T>(observer =>
{
var finallyOnce = Disposable.Create(finallyAction);
var subscription = source.Subscribe(observer.OnNext, error =>
{
try { finallyOnce.Dispose(); }
catch (Exception ex) { observer.OnError(ex); return; }
observer.OnError(error);
}, () =>
{
try { finallyOnce.Dispose(); }
catch (Exception ex) { observer.OnError(ex); return; }
observer.OnCompleted();
});
return new CompositeDisposable(subscription, finallyOnce);
});
}
The finallyAction is assigned as the Dispose action of a Disposable.Create disposable instance, in order to ensure that the action will be invoked at most once. This disposable is then combined with the disposable subscription of the source, by using a CompositeDisposable instance.
As a side note, I would like to address the question if we could go even further, and propagate downstream a possible error of the finallyAction during the unsubscription. This could be desirable in some cases, but unfortunately it's not possible. First and foremost doing so would violate a guideline, found in The Observable Contract document, that states:
When an observer issues an Unsubscribe notification to an Observable, the Observable will attempt to stop issuing notifications to the observer. It is not guaranteed, however, that the Observable will issue no notifications to the observer after an observer issues it an Unsubscribe notification.
So such an implementation would be non-conforming. Even worse, the Observable.Create method enforces this guideline, by muting the observer immediately after the subscription is disposed. It does so by encapsulating the observer inside an AutoDetachObserver wrapper. And even if we tried to circumvent this limitation by implementing an IObservable<T> type from scratch, any built-in operator that could be attached after our non-conforming Finally operator would mute our post-unsubscription OnError notification anyway. So it's just not possible. An error during the unsubscription cannot be propagated to the subscriber that just requested to unsubscribe.

I read the documentation and now I'm sure. The finally-operator will be called after the completition and should not throw any exception.
Compared to non-reactive programming:
StreamReader file = new StreamReader("file.txt");
string ln;
try {
while ((ln = file.ReadLine()) != null) {
Console.WriteLine(ln);
}
}
finally {
// avoid to throw an exception inside of finally!
if (file != null) {
file.close();
}
}
It is important to not throw an exception inside of finally.
Here is an example howto use it correctly (fiddle):
using System;
using System.Reactive.Linq;
using System.Threading;
public class Program
{
public static void Main()
{
Observable
.Range(1,5) // simulates stream-reader
.Finally(() => Console.WriteLine("Close streamreader"))
.Do(i => {
if (i == 5) {
throw new ApplicationException("Oops!"); // simulates IO-error
}
Console.WriteLine("Read " + i);
})
.Subscribe(_ => { }, ex => Console.WriteLine(ex.Message),
() => Console.WriteLine("Completed"));
Thread.Sleep(1000);
}
}
I'm not sure what you are trying to do (and I'm pretty new to c# reactive), but I think you are using not the right operator.
Edit
But you can patch it, if you want. In this article, they do something familar.
http://introtorx.com/Content/v1.0.10621.0/11_AdvancedErrorHandling.html

How to retry an action OnError reactive X IObservable several times and if successful continue

I have the following problem: I have a subject which uses method OnNext to signalize new events. However, there can occur an exception which is simulated in the code bellow with method call OnError. The cause of the error is time dependent so if I try to repeat the same action again a few times, it will succeed. So I want to use the catch method which accepts a function generating new sequences to repeat the action let's say 5 times. If all repeated attempt fail, then it should throw a final exception and if at least one succeeds it should continue with OnNext(5) and OnNext(6).
Ideally the exception and the subsequent OnError call should be prevented, but that isn't in my case possible.
I have tried multiple scenarios, with methods Retry, Concat, Catch etc... but nothing worked as I wanted.
Subject<int> sub = new Subject<int>();
var seq = sub.Select(x =>
{
//time dependent operation
Console.WriteLine(x);
return x;
}).
Catch((SeqException<int> ex) =>
{
return Observable.Empty(0); // what sequece to return to achieve the desired behaviour
});
seq.Subscribe();
sub.OnNext(1);
sub.OnNext(2);
sub.OnNext(3);
sub.OnError(new SeqException<int>{ Value = 4});
sub.OnNext(5);
sub.OnNext(6);
seq.Wait();
Thanks in advance.

Case 1: If the time dependent operation inside the Select is what you need to retry.
Use Observable.Start to convert a thunk into an observable. Then you get to use all the recovery operators to declare your behavior.
Subject<int> sub = new Subject<int>();
var seq = sub.SelectMany(x =>
Observable.Start(() =>
{
//time dependent failure
if (DateTime.Now.Second % 2 == 0)
throw new Exception();
Console.WriteLine(x);
return x;
})
.Retry(5)
.Catch(Observable.Return(x))
);
//for testing
Observable.Interval(TimeSpan.FromSeconds(1)).Select(x => (int)x).Subscribe(sub);
seq.Wait();
Case 2: You're actually getting signalled through the Subject.
The Rx contract requires that no more notifications occur after OnComplete|OnError. And on receiving that error, the subject disposes all its subscriptions and the pipeline is torn down. You need to bring it around to Case 1 to make it work.

You can always wrap anything with a retry block. Just an example for an action with one argument.
Action<T> RetryAction<T>( Action<T> action, int retries )
{
return arg => {
int count = 0;
while ( true )
{
try
{
action( arg );
return;
}
catch
{
if ( count == retries )
throw;
count++;
}
}
}
}
So you do not have to change the logic of the sequence handling.

Reactive Framework Hello World

This is an easy program to introduce the Reactive Framework. But I want to try the error handler, by modifying the program to be:
var cookiePieces = Observable.Range(1, 10);
cookiePieces.Subscribe(x =>
{
Console.WriteLine("{0}! {0} pieces of cookie!", x);
throw new Exception(); // newly added by myself
},
ex => Console.WriteLine("the exception message..."),
() => Console.WriteLine("Ah! Ah! Ah! Ah!"));
Console.ReadLine();
In this sample the follwing overload is used.
public static IDisposable Subscribe<TSource>(
this IObservable<TSource> source,
Action<TSource> onNext,
Action<Exception> onError,
Action onCompleted);
I hoped I would see the exception message printed, but the console application crashed. What is the reason?

The exception handler is used for exceptions created in the observable itself, not by the observer.
An easy way to provoke the exception handler is something like this:
using System;
using System.Linq;
class Test
{
static void Main(string[] args)
{
var xs = Observable.Range(1, 10)
.Select(x => 10 / (5 - x));
xs.Subscribe(x => Console.WriteLine("Received {0}", x),
ex => Console.WriteLine("Bang! {0}", ex),
() => Console.WriteLine("Done"));
Console.WriteLine("App ending normally");
}
}
Output:
Received 2
Received 3
Received 5
Received 10
Bang! System.DivideByZeroException: Attempted to divide by zero.
at Test.<Main>b__0(Int32 x)
at System.Linq.Observable.<>c__DisplayClass35a`2.<>c__DisplayClass35c.<Select
>b__359(TSource x)
App ending normally

In the Rx library, any user code passed into an operator that works on IObservable (Select, Where, GroupBy etc...) will be caught and send to the OnError handler of observers subscribed to the observable. The reason these are handled is that they are part of the computation.
Exceptions occurring in Observer code will have to be handled by the user. As they're at the end of the computation, it is unclear to Rx how to handle these.

Does it really crash or jumps Visual Studio into and shows you that an exception happened? If the second is true, you should take a look into Debug - Exception within the menu bar and deselect everything on the right.