Consider this Observable:
_listener = Observable.Defer(() => _deviceTypeProvider.GetDeviceTypes().ToObservable()
.SelectMany(CreateUdpListener, CreateMessage)
.OfType<DeviceMessage>()
.SelectMany(InjectTestMode)
.OfType<DeviceMessage>()
.Do(async message => await PublishMessage(message)))
.Retry()
.Subscribe(OnMessageReceive, OnError, OnComplete);
This works fine except when there is an exception thrown in either CreateMessage or InjectTestMode.
I'd like the Observable to skip the item in the sequence which generated the exception and carry on.
I've read about Catch but the example I found allow you to start a new Observable and I would like to carry on with the one I have.
At the moment the entire sequence restarts which includes the UDP ports which I'd like to avoid if possible.
[Update]
A colleague and I re-read some of the comments about using IEnumerable<IObservable<>> or IObservable<IObservable<>> and came up with this which works! But is it right/best practice?
In the event of an exception in the inner observable I'm wondering if it will only drop the packets from the ReceiveAsync event that was in flight.
var listeners = Observable.Defer(() => _deviceTypeProvider.GetDeviceTypes()
.ToObservable()
.Select(UdpListener)
.SelectMany(listener =>
{
return Observable.Defer(() => Observable
.FromAsync(listener.UdpClient.ReceiveAsync)
.Where(x => x.Buffer.Length > 0)
.Repeat()
.Select(result => CreateMessage(listener.DeviceType, result))
.SelectMany(InjectTestMode)
.OfType<DeviceMessage>()
.Do(async message => await PublishMessage(message)))
.Retry();
})).Retry();
_listener = listeners.Subscribe(OnMessageReceive, OnError, OnComplete);
The documentation for IObservable<T> specifies that a sequence must match this grammar:
OnNext* (OnCompleted|OnError)
There cannot be any more values emitted after an exception or completion. If you manually make an observable that violates this grammar, you can expect undefined behavior when you use any of the existing Rx operators. Not good!
Model your query as an IEnumerable<IObservable<T>> or IObservable<IObservable<T>> if you want to get retry behavior, where the outer IEnumerable<*> or IObservable<*> never throws.
Make some static extension function, pass delegate to it, and wrap it up into try{}catch{} inside this function.
Related
Recently I become aware that the Rx Finally operator behaves in a way which, at least for me, is unexpected. My expectation was that any error thrown by the finallyAction would be propagated to the operator's observers downstream. Alas this is not what happens. In the reality the operator first propagates the completion (or the failure) of the antecedent sequence to its observers, and then invokes the action, at a point in time when it's not possible to propagate a potential error thrown by the action. So it throws the error on the ThreadPool, and crashes the process. Which is not only unexpected, but also highly problematic. Below is a minimal demonstration of this behavior:
Observable
.Timer(TimeSpan.FromMilliseconds(100))
.Finally(() => throw new ApplicationException("Oops!"))
.Subscribe(_ => { }, ex => Console.WriteLine(ex.Message),
() => Console.WriteLine("Completed"));
Thread.Sleep(1000);
Outcome: Unhandled exception (Fiddle)
The exception thrown by the Finally lambda is not handled by the Subscribe:onError handler, as it would be desirable.
This feature (I am tempted to call it a flaw) limits severely the usefulness of the Finally operator in my eyes. Essentially I can only use it when I want to invoke an action that is expected to never fail, and if it fails it would indicate a catastrophic corruption of the application's state, when no recovery is possible. I could use it for example to Release a SemaphoreSlim (like I've done here for example), which can only fail if my code has a bug. I am OK with my app crashing in this case. But I've also used it recently to invoke an unknown action supplied by the caller, an action that could potentially fail, and crashing the app in this case is unacceptable. Instead, the error should be propagated downstream. So what I am asking here is how to implement a Finally variant (let's call it FinallySafe) with identical signature, and the behavior specified below:
public static IObservable<TSource> FinallySafe<TSource>(
this IObservable<TSource> source, Action finallyAction);
The finallyAction should be invoked after the source sequence has emitted an OnCompleted or an OnError notification, but before this notification is propagated to the observer.
If the finallyAction invocation completed successfully, the original OnCompleted/OnError notification should be propagated to the observer.
If the finallyAction invocation failed, an OnError notification should be propagated to the observer, containing the error that just occurred. In this case the previous error, the one that may have caused the source to complete with failure, should be ignored (not propagated).
The finallyAction should also be invoked when the FinallySafe is unsubscribed before the completion of the source. When a subscriber (observer) disposes a subscription, the finallyAction should by invoked synchronously, and any error should be propagated to the caller of the Dispose method.
If the FinallySafe is subscribed by multiple observers, the finallyAction should be invoked once per subscription, independently for each subscriber, following the rules above. Concurrent invocations are OK.
The finallyAction should never be invoked more than once per subscriber.
Validation: replacing the Finally with the FinallySafe in the code snippet above, should result to a program that doesn't crash with an unhandled exception.
Alternative: I am also willing to accept an answer that provides a reasonable explanation about why the behavior of the built-in Finally operator is better than the behavior of the custom FinallySafe operator, as specified above.
Finally gets called after the sequence has ended, and since the Rx contract only allows one OnError or OnCompleted it can't issue a second one.
But, if you replace the Finally with Do you can get the behaviour that you want.
Try this code:
Observable
.Timer(TimeSpan.FromMilliseconds(100))
.Do(_ => { }, () => throw new ApplicationException("Oops!"))
.Subscribe
(_ => { },
ex => Console.WriteLine(ex.Message),
() => Console.WriteLine("Completed"));
Thread.Sleep(TimeSpan.FromMilliseconds(1000));
That operates as you expect it to.
I get this output:
Oops!
If you want to run something at unsubscribe, then use this extension method:
public static class Ext
{
public static IObservable<T> Unsubscribed<T>(this IObservable<T> source, Action unsubscribed) =>
Observable.Create<T>(o =>
new CompositeDisposable(source.Subscribe(o), Disposable.Create(unsubscribed)));
}
Here's an example of its use:
var source = Observable.Never<int>();
var subscription =
source
.Unsubscribed(() => Console.WriteLine("Unsubscribed"))
.Subscribe();
subscription.Dispose();
That outputs:
Unsubscribed
Here is an implementation of the FinallySafe operator, having the behavior specified in the question:
/// <summary>
/// Invokes a specified action after the source observable sequence terminates
/// successfully or exceptionally. The action is invoked before the propagation
/// of the source's completion, and any exception thrown by the action is
/// propagated to the observer. The action is also invoked if the observer
/// is unsubscribed before the termination of the source sequence.
/// </summary>
public static IObservable<T> FinallySafe<T>(this IObservable<T> source,
Action finallyAction)
{
return Observable.Create<T>(observer =>
{
var finallyOnce = Disposable.Create(finallyAction);
var subscription = source.Subscribe(observer.OnNext, error =>
{
try { finallyOnce.Dispose(); }
catch (Exception ex) { observer.OnError(ex); return; }
observer.OnError(error);
}, () =>
{
try { finallyOnce.Dispose(); }
catch (Exception ex) { observer.OnError(ex); return; }
observer.OnCompleted();
});
return new CompositeDisposable(subscription, finallyOnce);
});
}
The finallyAction is assigned as the Dispose action of a Disposable.Create disposable instance, in order to ensure that the action will be invoked at most once. This disposable is then combined with the disposable subscription of the source, by using a CompositeDisposable instance.
As a side note, I would like to address the question if we could go even further, and propagate downstream a possible error of the finallyAction during the unsubscription. This could be desirable in some cases, but unfortunately it's not possible. First and foremost doing so would violate a guideline, found in The Observable Contract document, that states:
When an observer issues an Unsubscribe notification to an Observable, the Observable will attempt to stop issuing notifications to the observer. It is not guaranteed, however, that the Observable will issue no notifications to the observer after an observer issues it an Unsubscribe notification.
So such an implementation would be non-conforming. Even worse, the Observable.Create method enforces this guideline, by muting the observer immediately after the subscription is disposed. It does so by encapsulating the observer inside an AutoDetachObserver wrapper. And even if we tried to circumvent this limitation by implementing an IObservable<T> type from scratch, any built-in operator that could be attached after our non-conforming Finally operator would mute our post-unsubscription OnError notification anyway. So it's just not possible. An error during the unsubscription cannot be propagated to the subscriber that just requested to unsubscribe.
I read the documentation and now I'm sure. The finally-operator will be called after the completition and should not throw any exception.
Compared to non-reactive programming:
StreamReader file = new StreamReader("file.txt");
string ln;
try {
while ((ln = file.ReadLine()) != null) {
Console.WriteLine(ln);
}
}
finally {
// avoid to throw an exception inside of finally!
if (file != null) {
file.close();
}
}
It is important to not throw an exception inside of finally.
Here is an example howto use it correctly (fiddle):
using System;
using System.Reactive.Linq;
using System.Threading;
public class Program
{
public static void Main()
{
Observable
.Range(1,5) // simulates stream-reader
.Finally(() => Console.WriteLine("Close streamreader"))
.Do(i => {
if (i == 5) {
throw new ApplicationException("Oops!"); // simulates IO-error
}
Console.WriteLine("Read " + i);
})
.Subscribe(_ => { }, ex => Console.WriteLine(ex.Message),
() => Console.WriteLine("Completed"));
Thread.Sleep(1000);
}
}
I'm not sure what you are trying to do (and I'm pretty new to c# reactive), but I think you are using not the right operator.
Edit
But you can patch it, if you want. In this article, they do something familar.
http://introtorx.com/Content/v1.0.10621.0/11_AdvancedErrorHandling.html
I have tried to write console observable as in the example below, but it doesn't work. There are some issues with subscriptions. How to solve these issues?
static class Program
{
static async Task Main(string[] args)
{
// var observable = Observable.Interval(TimeSpan.FromMilliseconds(1000)).Publish().RefCount(); // works
// var observable = FromConsole().Publish().RefCount(); // doesn't work
var observable = FromConsole(); // doesn't work
observable.Subscribe(Console.WriteLine);
await Task.Delay(1500);
observable.Subscribe(Console.WriteLine);
await new TaskCompletionSource().Task;
}
static IObservable<string> FromConsole()
{
return Observable.Create<string>(async observer =>
{
while (true)
{
observer.OnNext(Console.ReadLine());
}
});
}
}
If I used Observable.Interval, it subscribes two times and I have two outputs for one input. If I used any version of FromConsole, I have one subscription and a blocked thread.
To start with, it is usually best to avoid using Observable.Create to create observables - it's certainly there for that purpose, but it can create observables that don't behave like you think they should because of their blocking nature. As you've discovered!
Instead, when possible, use the built-in operators to create observables. And that can be done in this case.
My version of FromConsole is this:
static IObservable<string> FromConsole() =>
Observable
.Defer(() =>
Observable
.Start(() => Console.ReadLine()))
.Repeat();
Observable.Start effectively is like Task.Run for observables. It calls Console.ReadLine() for us without blocking.
The Observable.Defer/Repeat pair repeatedly calls Observable.Start(() => Console.ReadLine()). Without the Defer it would just call Observable.Start and repeatedly return the one string forever.
That solves that.
Now, the second issue is that you want to see the value from the Console.ReadLine() output by both subscriptions to the FromConsole() observable.
Due to the way Console.ReadLine works, you are getting values from each subscription, but only one at a time. Try this code:
static async Task Main(string[] args)
{
var observable = FromConsole();
observable.Select(x => $"1:{x}").Subscribe(Console.WriteLine);
observable.Select(x => $"2:{x}").Subscribe(Console.WriteLine);
await new TaskCompletionSource<int>().Task;
}
static IObservable<string> FromConsole() =>
Observable
.Defer(() =>
Observable
.Start(() => Console.ReadLine()))
.Repeat();
When I run that I get this kind of output:
1:ddfd
2:dfff
1:dfsdfs
2:sdffdfd
1:sdfsdfsdf
The reason for this is that each subscription starts up a fresh subscription to FromConsole. So you have two calls to Console.ReadLine() they effectively queue and each one only gets each alternate input. Hence the alternation between 1 & 2.
So, to solve this you simply need the .Publish().RefCount() operator pair.
Try this:
static async Task Main(string[] args)
{
var observable = FromConsole().Publish().RefCount();
observable.Select(x => $"1:{x}").Subscribe(Console.WriteLine);
observable.Select(x => $"2:{x}").Subscribe(Console.WriteLine);
await new TaskCompletionSource<int>().Task;
}
static IObservable<string> FromConsole() =>
Observable
.Defer(() =>
Observable
.Start(() => Console.ReadLine()))
.Repeat();
I now get:
1:Hello
2:Hello
1:World
2:World
In a nutshell, it's the combination of the non-blocking FromConsole observable and the use of .Publish().RefCount() that makes this work the way you expect.
The problem is that the Console.ReadLine is a blocking method, so the subscription to the FromConsole sequence blocks indefinitely, so the await Task.Delay(1500); line is never reached. You can solve this problem by reading from the console asynchronously, offloading the blocking call to a ThreadPool thread:
static IObservable<string> FromConsole()
{
return Observable.Create<string>(async observer =>
{
while (true)
{
observer.OnNext(await Task.Run(() => Console.ReadLine()));
}
});
}
You can take a look at this question about why there is no better solution than offloading.
As a side note, subscribing to a sequence without providing an onError handler is not a good idea, unless having the process crash with an unhandled exception is an acceptable behavior for your app. It is especially problematic with sequences produced with Observable.Create<T>(async, because it can lead to weird/buggy behavior like this one: Async Create hanging while publishing observable.
You need to return a observable without the publish. You can then subscribe to it and do your thing further. Here is an example. When I run it i can readline multiple times.
public class Program
{
static void Main(string[] args)
{
FromConsole().Subscribe(x =>
{
Console.WriteLine(x);
});
}
static IObservable<string> FromConsole()
{
return Observable.Create<string>(async observer =>
{
while (true)
{
observer.OnNext(Console.ReadLine());
}
});
}
}
I have a method that do some work asynchronously with use of observable. I would like to know what is the best way to make this method async, so that I will be able to await on it and do some work after observable completes.
My first try was to use await on observable.
public async Task DoWorkAsync()
{
var observable = Observable.Create<int>(o =>
{
Task.Run(() =>
{
Thread.Sleep(1000);
Console.WriteLine("OnNext");
o.OnNext(1);
o.OnError(new Exception("exception in observable logic"));
//o.OnCompleted();
});
return Disposable.Empty;
});
//observable = observable.Publish().RefCount();
observable.Subscribe(i => Console.WriteLine(i));
Console.WriteLine("Awaiting...");
await observable;
Console.WriteLine("After Awaiting...");
}
Depending on the scenario I had different issues with that approach (+/- means that this part of code is uncommented/commented):
+OnNext +OnCompleted -OnError -RefCount: OnNext was invoked 2 times (observable was subscribed 2 times). This is what I would like to avoid.
+OnNext +OnCompleted -OnError +RefCount: When I use RefCount() method the code works.
-OnNext +OnCompleted -OnError +RefCount: "Sequence contains no element" exception is thrown when my observable doesn't raise OnNext.
+OnNext -OnCompleted +OnError -RefCount: OnNext was invoked 2 times. Exception raised.
+OnNext -OnCompleted +OnError +RefCount: Hangs after displaying 1 (probably because it wants to return to thread that is awaited). We can make it work (and raise exception) by using SubscribeOn(ThreadPoolScheduler.Instance)
Anyway in case when observable is empty (no OnNext rised) we get exception even if OnError is not called and we don't have any exception inside observable logic. Thats why awaiting on observable is not good solution.
That is why I tried another solution using TaskCompletionSource
public async Task DoWorkAsync()
{
var observable = Observable.Create<int>(o =>
{
Task.Run(() =>
{
Thread.Sleep(1000);
Console.WriteLine("OnNext");
o.OnNext(1);
o.OnError(new Exception("exception in observable logic"));
//o.OnCompleted();
});
return Disposable.Empty;
});
var tcs = new TaskCompletionSource<bool>();
observable.Subscribe(i => Console.WriteLine(i),
e =>
{
//tcs.TrySetException(e);
tcs.SetResult(false);
},
() => tcs.TrySetResult(true));
Console.WriteLine("Awaiting...");
await tcs.Task;
Console.WriteLine("After Awaiting...");
}
It works ok in all scenarios and in case of OnError is invoked we could either use tcs.SetResult(false) and don't have information about exception details in outside method or we could use tcs.TrySetException(e) and be able to catch the exception in the outside method.
Can you suggest me if there is some better/cleaner solution or my second solution is the way to go?
EDIT
So I would like to know if there is a better solution than my second solution that will:
not require to use .Publish().RefCount()
not require additional subscription (what happens in await observable under the hood - OnNext is invoked 2 times)
Of course I could wrap my solution in some async extension method for subscribing that returns Task
EDIT:
If you remove the subscription you can do the following:
await observable.Do(i => Console.WriteLine(i)).LastOrDefaultAsync();
As for your arbitrary requirements... Not having multiple subscriptions for a cold observable makes sense; so you publish it. Refusing to use .Publish().Refcount() doesn't make sense. I don't understand why you're rejecting a solution that solves your problem.
There's a lot there, but I'm assuming this is your main question:
Anyway in case when observable is empty (no OnNext rised) we get
exception even if OnError is not called and we don't have any
exception inside observable logic. Thats why awaiting on observable is
not good solution.
await observable is the same as await observable.LastAsync(). So if there is no element, you get an exception. Imagine changing that statement to int result = await observable; What should the value of result be if there's no elements?
If you change await observable; to await observable.LastOrDefaultAsync(); everything should run smoothly.
And yes, you should use .Publish().Refcount()
I'd clearly prefer the 2nd solution, because it only subscribes once.
But out of curiosity: what's the purpose of writing a method like this?
If it's to allow for configurable side effects, this would be equivalent:
public async Task DoWorkAsync()
{
Action<int> onNext = Console.WriteLine;
await Task.Delay(1000);
onNext(1);
throw new Exception("exception in DoWork logic"); // ... or don't
}
You could use ToTask extension method:
await observable.ToTask();
I have tried several ways but failing to get my subscription method called:
Method 1:
var buffer = new List<Kpi>();
buffer.ToObservable().Buffer(TimeSpan.FromMinutes(1), 5).Subscribe(
async kpis =>
{
await _retry.ExecuteAsync(() => Process(kpis.ToList())).ConfigureAwait(false);
});
Then buffer.Add(new Kpi()); won't trigger my method.
Method 2: (Note: I have read the definition for the special methods Empty/Never/Throw but other than these I can't seem to find a way to create an observable that emits something other than primitive numbers etc.)
var buffer = Observable.Empty<Kpi>();
buffer.Buffer(TimeSpan.FromMinutes(1), 5).Subscribe(
async kpis =>
{
await _retry.ExecuteAsync(() => Process(kpis.ToList())).ConfigureAwait(false);
});
Then buffer.Publish(new Kpi()) . Again nothing happens
Where am I am going wrong ?
In the first case, calling ToObservable on List won't make the List magically notify of it's changes. List simply does not have that feature.
In the second case, Publish does something completely different than what you are expecting.
If you want to create an observable from events, you are looking for Subject class.
var buffer = new Subject<Kpi>();
buffer.Buffer(TimeSpan.FromMinutes(1), 5).Subscribe(
async kpis =>
{
await _retry.ExecuteAsync(() => Process(kpis.ToList())).ConfigureAwait(false);
});
// notify of new item
buffer.OnNext(new Kpi());
There are many ways to create new observable sequence. I would recommend read through it to see if one is more suited for you. For example turning event into observable.
I have the following code:
var tasks = await taskSeedSource
.Select(taskSeed => GetPendingOrRunningTask(taskSeed, createTask, onFailed, onSuccess, sem))
.ToList()
.ToTask();
if (tasks.Count == 0)
{
return;
}
if (tasks.Contains(null))
{
tasks = tasks.Where(t => t != null).ToArray();
if (tasks.Count == 0)
{
return;
}
}
await Task.WhenAll(tasks);
Where taskSeedSource is a Reactive Observable. It could be that this code have many problems, but I see at least two:
I am collecting tasks whereas I could do without it.
Somehow, the returned tasks list may contain nulls, even though GetPendingOrRunningTask is an async method and hence never returns null. I failed to understand why it happens, so I had to defend against it without understanding the cause of the problem.
I would like to use the AsyncCountdownEvent from the AsyncEx framework instead of collecting the tasks and then awaiting on them.
So, I can pass the countdown event to GetPendingOrRunningTask which will increment it immediately and signal before returning after awaiting for the completion of its internal logic. However, I do not understand how to integrate the countdown event into the monad (that is the Reactive jargon, isn't it?).
What is the right way to do it?
EDIT
Guys, let us forget about the mysterious nulls in the returned list. Suppose everything is green and the code is
var tasks = await taskSeedSource
.Select(taskSeed => GetPendingOrRunningTask(taskSeed, ...))
.ToList()
.ToTask();
await Task.WhenAll(tasks);
Now the question is how do I do it with the countdown event? So, suppose I have:
var c = new AsyncCountdownEvent(1);
and
async Task GetPendingOrRunningTask<T>(AsyncCountdownEvent c, T taskSeed, ...)
{
c.AddCount();
try
{
await ....
}
catch (Exception exc)
{
// The exception is handled
}
c.Signal();
}
My problem is that I no longer need the returned task. These tasks where collected and awaited to get the moment when all the work items are over, but now the countdown event can be used to indicate when the work is over.
My problem is that I am not sure how to integrate it into the Reactive chain. Essentially, the GetPendingOrRunningTask can be async void. And here I am stuck.
EDIT 2
Strange appearance of a null entry in the list of tasks
#Servy is correct that you need to solve the null Task problem at the source. Nobody wants to answer a question about how to workaround a problem that violates the contracts of a method that you've defined yourself and yet haven't provided the source for examination.
As for the issue about collecting tasks, it's easy to avoid with Merge if your method returns a generic Task<T>:
await taskSeedSource
.Select(taskSeed => GetPendingOrRunningTask(taskSeed, createTask, onFailed, onSuccess, sem))
.Where(task => task != null) // According to you, this shouldn't be necessary.
.Merge();
However, unfortunately there's no official Merge overload for the non-generic Task but that's easy enough to define:
public static IObservable<Unit> Merge(this IObservable<Task> sources)
{
return sources.Select(async source =>
{
await source.ConfigureAwait(false);
return Unit.Default;
})
.Merge();
}