I am trying to create an observable pipeline with the following characteristics:
is lazy (does nothing until somebody subscribes)
executes at most once regardless of how many subscriptions are received
replays its resulting value, if any OR
replays its resulting error, if any
For the life of me, I can't figure out the correct semantics to accomplish this. I thought it would be a simple case of doing something like this:
Observable
.Defer(() => Observable
.Start(() => { /* do something */ })
.PublishLast()
.ConnectUntilCompleted());
Where ConnectUntilCompleted just does what it sounds like:
public static IObservable<T> ConnectUntilCompleted<T>(this IConnectableObservable<T> #this)
{
#this.Connect();
return #this;
}
This seems to work when the observable terminates successfully, but not when there's an error. Any subscribers do not receive the error:
[Fact]
public void test()
{
var o = Observable
.Defer(() => Observable
.Start(() => { throw new InvalidOperationException(); })
.PublishLast()
.ConnectUntilCompleted());
// this does not throw!
o.Subscribe();
}
Can anyone tell me what I'm doing wrong? Why doesn't Publish replay any error it receives?
UPDATE: it gets even stranger:
[Fact]
public void test()
{
var o = Observable
.Defer(() => Observable
.Start(() => { throw new InvalidOperationException(); })
.PublishLast()
.ConnectUntilCompleted())
.Do(
_ => { },
ex => { /* this executes */ });
// this does not throw!
o.Subscribe();
o.Subscribe(
_ => { },
ex => { /* even though this executes */ });
}
Try this version of you ConnectUntilCompleted method:
public static IObservable<T> ConnectUntilCompleted<T>(this IConnectableObservable<T> #this)
{
return Observable.Create<T>(o =>
{
var subscription = #this.Subscribe(o);
var connection = #this.Connect();
return new CompositeDisposable(subscription, connection);
});
}
The allows Rx to behave properly.
Now I've added to it to help show what's going on:
public static IObservable<T> ConnectUntilCompleted<T>(this IConnectableObservable<T> #this)
{
return Observable.Create<T>(o =>
{
var disposed = Disposable.Create(() => Console.WriteLine("Disposed!"));
var subscription = Observable
.Defer<T>(() => { Console.WriteLine("Subscribing!"); return #this; })
.Subscribe(o);
Console.WriteLine("Connecting!");
var connection = #this.Connect();
return new CompositeDisposable(disposed, subscription, connection);
});
}
Now your observable looks like this:
var o =
Observable
.Defer(() =>
Observable
.Start(() =>
{
Console.WriteLine("Started.");
throw new InvalidOperationException();
}))
.PublishLast()
.ConnectUntilCompleted();
The final key thing is to actually handle the errors in the subscription - so it's not enough to simply do o.Subscribe().
So do this:
o.Subscribe(
x => Console.WriteLine(x),
e => Console.WriteLine(e.Message),
() => Console.WriteLine("Done."));
o.Subscribe(
x => Console.WriteLine(x),
e => Console.WriteLine(e.Message),
() => Console.WriteLine("Done."));
o.Subscribe(
x => Console.WriteLine(x),
e => Console.WriteLine(e.Message),
() => Console.WriteLine("Done."));
When I run that I get this:
Subscribing!
Connecting!
Subscribing!
Connecting!
Subscribing!
Connecting!
Started.
Operation is not valid due to the current state of the object.
Disposed!
Operation is not valid due to the current state of the object.
Disposed!
Operation is not valid due to the current state of the object.
Disposed!
Note that "Started" only appears once, but the error is reported three times.
(Sometimes Started appears higher up in the list after the first subscription.)
I think this is what you wanted from your description.
Just to support #Engimativity's answer, i want to show how you should be runing your tests so you stop getting these "surprises". Your tests are non-deterministic because they are multi-threaded/concurrent. Your use of Observable.Start without providing an IScheduler is problematic. If you run your tests with a TestScheduler your tests will now be singlethreaded and determinisitic
[Test]
public void Test()
{
var testScheduler = new TestScheduler();
var o = Observable
.Defer(() => Observable
.Start(() => { throw new InvalidOperationException(); }, testScheduler)
.PublishLast()
.ConnectUntilCompleted());
var observer = testScheduler.CreateObserver<Unit>();
o.Subscribe(observer);
testScheduler.Start();
CollectionAssert.IsNotEmpty(observer.Messages);
Assert.AreEqual(NotificationKind.OnError, observer.Messages[0].Value.Kind);
}
An alternative way to achieve your requirements could be:
var lazy = new Lazy<Task>(async () => { /* execute once */ }, isThreadSafe: true);
var o = Observable.FromAsync(() => lazy.Value);
When subscribed for the first time, lazy would create (and execute) the task. For other subscriptions, lazy would return the same (possibly already completed or failed) task.
Related
I'm monitoring a directory with the following setup:
var folder = new Subject();
folder.OnNext("somepath");
folder.SelectMany(FileMonitor)
.Subscribe(x => Console.WriteLine($"Found: {x}"));
public IObservable<string> FileMonitor(string pathToWatch){
return Observable.Create<string>(obs => {
var dfs = CreateAndStartFileWatcher(pathToWatch,obs);
() => dfs.Dispose();
});
}
This works, but if I pass a new path to the subject, the previous FileMonitor is not disposed.
Is there a way to cancel/dispose the previously generated Observable?
It looks like I need: http://reactivex.io/documentation/operators/switch.html but this is not implemented in c#?
Sometimes, asking a question gives yourself new insights.
The solution is to use switch which is available, but only works on a Observable.
So it should be:
var folder = new Subject();
folder.OnNext("somepath");
folder.Select(FileMonitor)
.Switch()
.Subscribe(x => Console.WriteLine($"Found: {x}"));
public IObservable<string> FileMonitor(string pathToWatch){
return Observable.Create<string>(obs => {
var dfs = CreateAndStartFileWatcher(pathToWatch,obs);
() => dfs.Dispose();
});
}
Leaving this question for reference instead of removing it.
I am writing some code using RX in C# that must interface with an older system by emitting events.
In summary, I have an observable and need to emit one event when the observable completes and another event if a timeout exception is detected. The main problem is how best to handle the exception.
I'm relatively new to RX, so although I have found a solution, I can't be sure that there isn't a better or more appropriate way that uses the RX extensions better.
This is not the real code but indicates the pattern of my thinking:
public delegate void SuccessHandler(object sender, SuccessEventArgs e);
public event SuccessHandler OnSuccess;
public delegate void TimeoutHandler(object sender, TimeoutEventArgs e);
public event TimeoutHandler OnTimeout;
var id;
var o = Observable.Return() // <- this would be a fetch from an asynchronous source
.Where(r=>r.status=="OK")
.Timeout(new Timespan(0,0,30)
.Do(r=> {
id=r.Id // <-- Ugh! I know this shouldn't be done!
}
.Subscribe(r => {
var statusResponse= new StatusResponse()
{
Id = r.Id
Name = r.Name
Message = "The operation completed successfully",
Status = Status.Success
};
if (OnSuccess == null) return;
OnSuccess (this, new SuccessEventArgs(statusResponse);
},
e =>
{
_logger.LogError(e, "A matching response was not returned in a timely fashion");
if (OnTimeout == null) return;
OnTimeout(this, new TimeoutEventArgs(id));
});
If I didn't need to detect and act upon the timeout it would be fine; I have already worked out how to substitute the Subscribe for ToEventPattern:
...
.Select(r =>
{
var statusResponse= new StatusResponse()
{
Id = r.Id
Name = r.Name
Message = "The operation completed successfully",
Status = Status.Success
};
return new EventPattern<SuccessEventArgs>(this, new SuccessEventArgs(statusResponse));
})
.ToEventPattern();
However, I'd like to be able to detect the timeout (and possibly other exceptions). my experiments with Catch have been unsuccessful because I can't seem to get the types to line up correctly, probably because I don't really understand what is going on.
I'd very much appreciate opinions on this. Is this an acceptable solution? How can I improve it? Can anyone point me to some good online references that will explain how this kind of flow-control and exception handling can be done (all the examples I've seen so far seem to stop short of the real-world case where you want to emit an event and combine that with exception handling).
Thanks in advance
You can branch from observables quite easily, e.g.
var a = Observable.Range(0, 10);
var b = a.Select(x => x * x);
var c = a.Select(x => x * 10);
A word of warning - if the observable is cold, this will cause the producer function to run for each subscription. Look up the difference between hot and cold observables if this isn't clear.
I've created a solution that creates two branches from the source observable and turns each into an event:
class Program
{
static void Main(string[] args)
{
Console.WriteLine("Hello World!");
var service = new Service();
var apiCall = service.CallApi();
apiCall.OnSuccess.OnNext += (_, __) => Console.WriteLine("Success!");
apiCall.OnTimeout.OnNext += (_, __) => Console.WriteLine("Timeout!");
Console.ReadLine();
}
}
class SuccessEventArgs{}
class TimeoutEventArgs{}
class ApiCall
{
public IEventPatternSource<SuccessEventArgs> OnSuccess {get;}
public IEventPatternSource<TimeoutEventArgs> OnTimeout {get;}
public ApiCall(IEventPatternSource<SuccessEventArgs> onSuccess, IEventPatternSource<TimeoutEventArgs> onTimeout)
{
OnSuccess = onSuccess;
OnTimeout = onTimeout;
}
}
class Service
{
public ApiCall CallApi()
{
var apiCall = Observable
.Timer(TimeSpan.FromSeconds(3))
.Do(_ => Console.WriteLine("Api Called"))
.Select(_ => new EventPattern<SuccessEventArgs>(null, new SuccessEventArgs()))
// .Timeout(TimeSpan.FromSeconds(2)) // uncomment to time out
.Timeout(TimeSpan.FromSeconds(4))
// the following two lines turn the "cold" observable "hot"
// comment them out and see how often "Api Called" is logged
.Publish()
.RefCount();
var success = apiCall
// ignore the TimeoutException and return an empty observable
.Catch<EventPattern<SuccessEventArgs>, TimeoutException>(_ => Observable.Empty<EventPattern<SuccessEventArgs>>())
.ToEventPattern();
var timeout = apiCall
.Materialize() // turn the exception into a call to OnNext rather than OnError
.Where(x => x.Exception is TimeoutException)
.Select(_ => new EventPattern<TimeoutEventArgs>(null, new TimeoutEventArgs()))
.ToEventPattern();
return new ApiCall(success, timeout);
}
}
I'm trying to create an Rx operator that seems pretty useful, but I've suprisingly not found any questions on Stackoverflow that match precisely. I'd like to create a variation on Throttle that lets values through immediately if there's been a period of inactivity. My imagined use case is something like this:
I have a dropdown that kicks off a web request when the value is changed. If the user holds down the arrow key and cycles rapidly through the values, I don't want to kick off a request for each value. But if I throttle the stream then the user has to wait out the throttle duration every time they just select a value from the dropdown in the normal manner.
So whereas a normal Throttle looks like this:
I want to create ThrottleSubsequent that look like this:
Note that marbles 1, 2, and 6 are passed through without delay because they each follow a period of inactivity.
My attempt at this looks like the following:
public static IObservable<TSource> ThrottleSubsequent<TSource>(this IObservable<TSource> source, TimeSpan dueTime, IScheduler scheduler)
{
// Create a timer that resets with each new source value
var cooldownTimer = source
.Select(x => Observable.Interval(dueTime, scheduler)) // Each source value becomes a new timer
.Switch(); // Switch to the most recent timer
var cooldownWindow = source.Window(() => cooldownTimer);
// Pass along the first value of each cooldown window immediately
var firstAfterCooldown = cooldownWindow.SelectMany(o => o.Take(1));
// Throttle the rest of the values
var throttledRest = cooldownWindow
.SelectMany(o => o.Skip(1))
.Throttle(dueTime, scheduler);
return Observable.Merge(firstAfterCooldown, throttledRest);
}
This seems to work, but I'm having a difficult time reasoning about this, and I get the feeling there are some edge cases here where things might get screwy with duplicate values or something. I'd like to get some feedback from more experienced Rx-ers as to whether or not this code is correct, and/or whether there is a more idiomatic way of doing this.
Well, here's a test suite (using nuget Microsoft.Reactive.Testing):
var ts = new TestScheduler();
var source = ts.CreateHotObservable<char>(
new Recorded<Notification<char>>(200.MsTicks(), Notification.CreateOnNext('A')),
new Recorded<Notification<char>>(300.MsTicks(), Notification.CreateOnNext('B')),
new Recorded<Notification<char>>(500.MsTicks(), Notification.CreateOnNext('C')),
new Recorded<Notification<char>>(510.MsTicks(), Notification.CreateOnNext('D')),
new Recorded<Notification<char>>(550.MsTicks(), Notification.CreateOnNext('E')),
new Recorded<Notification<char>>(610.MsTicks(), Notification.CreateOnNext('F')),
new Recorded<Notification<char>>(760.MsTicks(), Notification.CreateOnNext('G'))
);
var target = source.ThrottleSubsequent(TimeSpan.FromMilliseconds(150), ts);
var expectedResults = ts.CreateHotObservable<char>(
new Recorded<Notification<char>>(200.MsTicks(), Notification.CreateOnNext('A')),
new Recorded<Notification<char>>(450.MsTicks(), Notification.CreateOnNext('B')),
new Recorded<Notification<char>>(500.MsTicks(), Notification.CreateOnNext('C')),
new Recorded<Notification<char>>(910.MsTicks(), Notification.CreateOnNext('G'))
);
var observer = ts.CreateObserver<char>();
target.Subscribe(observer);
ts.Start();
ReactiveAssert.AreElementsEqual(expectedResults.Messages, observer.Messages);
and using
public static class TestingHelpers
{
public static long MsTicks(this int i)
{
return TimeSpan.FromMilliseconds(i).Ticks;
}
}
Seems to pass. If you wanted to reduce it, you could turn it into this:
public static IObservable<TSource> ThrottleSubsequent2<TSource>(this IObservable<TSource> source, TimeSpan dueTime, IScheduler scheduler)
{
return source.Publish(_source => _source
.Window(() => _source
.Select(x => Observable.Interval(dueTime, scheduler))
.Switch()
))
.Publish(cooldownWindow =>
Observable.Merge(
cooldownWindow
.SelectMany(o => o.Take(1)),
cooldownWindow
.SelectMany(o => o.Skip(1))
.Throttle(dueTime, scheduler)
)
);
}
EDIT:
Publish forces sharing of a subscription. If you have a bad (or expensive) source observable with subscription side-effects, Publish makes sure you only subscribe once. Here's an example where Publish helps:
void Main()
{
var source = UglyRange(10);
var target = source
.SelectMany(i => Observable.Return(i).Delay(TimeSpan.FromMilliseconds(10 * i)))
.ThrottleSubsequent2(TimeSpan.FromMilliseconds(70), Scheduler.Default) //Works with ThrottleSubsequent2, fails with ThrottleSubsequent
.Subscribe(i => Console.WriteLine(i));
}
static int counter = 0;
public IObservable<int> UglyRange(int limit)
{
var uglySource = Observable.Create<int>(o =>
{
if (counter++ == 0)
{
Console.WriteLine("Ugly observable should only be created once.");
Enumerable.Range(1, limit).ToList().ForEach(i => o.OnNext(i));
}
else
{
Console.WriteLine($"Ugly observable should only be created once. This is the {counter}th time created.");
o.OnError(new Exception($"observable invoked {counter} times."));
}
return Disposable.Empty;
});
return uglySource;
}
I'm trying to solve the following:
a) subscriber receives events from IObservable for some time. Then it unsubscribes, do some stuff and then subscribe again. Here it should start receiving events from exactly the same point where unsubscription was performed.
b) Such behavior is desirable for multiple subscribers model. E.g. when one has unsubscribed, others should continue receiving events.
Are there any suggestions from the RX side?
Thanks in advance!
Here's a reasonably simple Rx way to do what you want copied from my answer to this other question. I've created an extension method called Pausable that takes a source observable and a second observable of boolean that pauses or resumes the observable.
public static IObservable<T> Pausable<T>(
this IObservable<T> source,
IObservable<bool> pauser)
{
return Observable.Create<T>(o =>
{
var paused = new SerialDisposable();
var subscription = Observable.Publish(source, ps =>
{
var values = new ReplaySubject<T>();
Func<bool, IObservable<T>> switcher = b =>
{
if (b)
{
values.Dispose();
values = new ReplaySubject<T>();
paused.Disposable = ps.Subscribe(values);
return Observable.Empty<T>();
}
else
{
return values.Concat(ps);
}
};
return pauser.StartWith(false).DistinctUntilChanged()
.Select(p => switcher(p))
.Switch();
}).Subscribe(o);
return new CompositeDisposable(subscription, paused);
});
}
It can be used like this:
var xs = Observable.Generate(
0,
x => x < 100,
x => x + 1,
x => x,
x => TimeSpan.FromSeconds(0.1));
var bs = new Subject<bool>();
var pxs = xs.Pausable(bs);
pxs.Subscribe(x => { /* Do stuff */ });
Thread.Sleep(500);
bs.OnNext(true);
Thread.Sleep(5000);
bs.OnNext(false);
Thread.Sleep(500);
bs.OnNext(true);
Thread.Sleep(5000);
bs.OnNext(false);
It sounds like you need a "pausable" stream. Assuming that only 1 subscriber will handle the values at a time (while the other subscribers just wait), this solution is probably what you need.
What I'm trying to achieve is to buffer incoming events from some IObservable ( they come in bursts) and release them further, but one by one, in even intervals.
Like this:
-oo-ooo-oo------------------oooo-oo-o-------------->
-o--o--o--o--o--o--o--------o--o--o--o--o--o--o---->
Since I'm quite new to Rx, I'm not sure if there already is a Subject or an operator that does just this. Maybe It can be done by composition?
update:
Thanks to
Richard Szalay for pointing out the Drain operator, I found another example by James Miles of Drain operator usage. Here's how I managed to get it to work in a WPF app:
.Drain(x => {
Process(x);
return Observable.Return(new Unit())
.Delay(TimeSpan.FromSeconds(1), Scheduler.Dispatcher );
}).Subscribe();
I had some fun, because omitting the scheduler parameter causes the app to crash in debug mode without any exception showing up ( I need to learn how to deal with exceptions in Rx).
The Process method modifies the UI state directly, but I guess it's quite simple to make an IObservable out of it (using a ISubject?).
update:
In the meantime I've been experimenting with ISubject, the class below does what I wanted - it lets out buffered Ts in a timely manner:
public class StepSubject<T> : ISubject<T>
{
IObserver<T> subscriber;
Queue<T> queue = new Queue<T>();
MutableDisposable cancel = new MutableDisposable();
TimeSpan interval;
IScheduler scheduler;
bool idle = true;
public StepSubject(TimeSpan interval, IScheduler scheduler)
{
this.interval = interval;
this.scheduler = scheduler;
}
void Step()
{
T next;
lock (queue)
{
idle = queue.Count == 0;
if (!idle)
next = queue.Dequeue();
}
if (!idle)
{
cancel.Disposable = scheduler.Schedule(Step, interval);
subscriber.OnNext(next);
}
}
public void OnNext(T value)
{
lock (queue)
queue.Enqueue(value);
if (idle)
cancel.Disposable = scheduler.Schedule(Step);
}
public IDisposable Subscribe(IObserver<T> observer)
{
subscriber = observer;
return cancel;
}
}
This naive implementation is stripped from OnCompleted and OnError for clarity, also only single subscription allowed.
It's actually tricker than it sounds.
Using Delay doesn't work because the values will still happen in bulk, only slightly delayed.
Using Interval with either CombineLatest or Zip doesn't work, since the former will cause source values to be skipped and the latter will buffer interval values.
I think the new Drain operator (added in 1.0.2787.0), combined with Delay should do the trick:
source.Drain(x => Observable.Empty<int>().Delay(TimeSpan.FromSeconds(1)).StartWith(x));
The Drain operator works like SelectMany, but waits until the previous output completes before calling the selector with the next value. It's still not exactly what you are after (the first value in a block will also be delayed), but it's close: The usage above matches your marble diagram now.
Edit: Apparently the Drain in the framework doesn't work like SelectMany. I'll ask for some advice in the official forums. In the meantime, here's an implementation of Drain that does what you're after:
Edit 09/11: Fixed errors in implementation and updated usage to match your requested marble diagram.
public static class ObservableDrainExtensions
{
public static IObservable<TOut> Drain<TSource, TOut>(this IObservable<TSource> source,
Func<TSource, IObservable<TOut>> selector)
{
return Observable.Defer(() =>
{
BehaviorSubject<Unit> queue = new BehaviorSubject<Unit>(new Unit());
return source
.Zip(queue, (v, q) => v)
.SelectMany(v => selector(v)
.Do(_ => { }, () => queue.OnNext(new Unit()))
);
});
}
}
Just for completeness here is an alterantive (more compact) version of the Drain() method suggested by Richard:
public static IObservable<T2> SelectManySequential<T1, T2>(
this IObservable<T1> source,
Func<T1, IObservable<T2>> selector
)
{
return source
.Select(x => Observable.Defer<T2>(() => selector(x)))
.Concat();
}
See the thread Drain + SelectMany = ? in the Rx forum.
Update:
I realized that the Concat() overload that I used was one of my personal Rx extensions that are (not yet) part of the framework. I am sorry for this mistake .. Of course this makes my solution less elegant than I thought.
Nevertheless for completeness I post here my Conact() extension method overload:
public static IObservable<T> Concat<T>(this IObservable<IObservable<T>> source)
{
return Observable.CreateWithDisposable<T>(o =>
{
var lockCookie = new Object();
bool completed = false;
bool subscribed = false;
var waiting = new Queue<IObservable<T>>();
var pendingSubscription = new MutableDisposable();
Action<Exception> errorHandler = e =>
{
o.OnError(e);
pendingSubscription.Dispose();
};
Func<IObservable<T>, IDisposable> subscribe = null;
subscribe = (ob) =>
{
subscribed = true;
return ob.Subscribe(
o.OnNext,
errorHandler,
() =>
{
lock (lockCookie)
{
if (waiting.Count > 0)
pendingSubscription.Disposable = subscribe(waiting.Dequeue());
else if (completed)
o.OnCompleted();
else
subscribed = false;
}
}
);
};
return new CompositeDisposable(pendingSubscription,
source.Subscribe(
n =>
{
lock (lockCookie)
{
if (!subscribed)
pendingSubscription.Disposable = subscribe(n);
else
waiting.Enqueue(n);
}
},
errorHandler
, () =>
{
lock (lockCookie)
{
completed = true;
if (!subscribed)
o.OnCompleted();
}
}
)
);
});
}
And now beating myself with my own weapons:
The same Concat() method could be written much more elegant in Richard Szalay's brilliant way:
public static IObservable<T> Concat<T>(this IObservable<IObservable<T>> source)
{
return Observable.Defer(() =>
{
BehaviorSubject<Unit> queue = new BehaviorSubject<Unit>(new Unit());
return source
.Zip(queue, (v, q) => v)
.SelectMany(v =>
v.Do(_ => { }, () => queue.OnNext(new Unit()))
);
});
}
So credit belongs to Richard. :-)
Here's how I did this, just using an explicit queue (ReactiveCollection is just a fancy version of WPF's ObservableCollection - ReactiveCollection.ItemsAdded OnNext's for each item added, as you can imagine):
https://github.com/xpaulbettsx/ReactiveXaml/blob/master/ReactiveXaml/ReactiveCollection.cs#L309
public static ReactiveCollection<T> CreateCollection<T>(this IObservable<T> FromObservable, TimeSpan? WithDelay = null)
{
var ret = new ReactiveCollection<T>();
if (WithDelay == null) {
FromObservable.ObserveOn(RxApp.DeferredScheduler).Subscribe(ret.Add);
return ret;
}
// On a timer, dequeue items from queue if they are available
var queue = new Queue<T>();
var disconnect = Observable.Timer(WithDelay.Value, WithDelay.Value)
.ObserveOn(RxApp.DeferredScheduler).Subscribe(_ => {
if (queue.Count > 0) {
ret.Add(queue.Dequeue());
}
});
// When new items come in from the observable, stuff them in the queue.
// Using the DeferredScheduler guarantees we'll always access the queue
// from the same thread.
FromObservable.ObserveOn(RxApp.DeferredScheduler).Subscribe(queue.Enqueue);
// This is a bit clever - keep a running count of the items actually
// added and compare them to the final count of items provided by the
// Observable. Combine the two values, and when they're equal,
// disconnect the timer
ret.ItemsAdded.Scan0(0, ((acc, _) => acc+1)).Zip(FromObservable.Aggregate(0, (acc,_) => acc+1),
(l,r) => (l == r)).Where(x => x != false).Subscribe(_ => disconnect.Dispose());
return ret;
}