I'm monitoring a directory with the following setup:
var folder = new Subject();
folder.OnNext("somepath");
folder.SelectMany(FileMonitor)
.Subscribe(x => Console.WriteLine($"Found: {x}"));
public IObservable<string> FileMonitor(string pathToWatch){
return Observable.Create<string>(obs => {
var dfs = CreateAndStartFileWatcher(pathToWatch,obs);
() => dfs.Dispose();
});
}
This works, but if I pass a new path to the subject, the previous FileMonitor is not disposed.
Is there a way to cancel/dispose the previously generated Observable?
It looks like I need: http://reactivex.io/documentation/operators/switch.html but this is not implemented in c#?
Sometimes, asking a question gives yourself new insights.
The solution is to use switch which is available, but only works on a Observable.
So it should be:
var folder = new Subject();
folder.OnNext("somepath");
folder.Select(FileMonitor)
.Switch()
.Subscribe(x => Console.WriteLine($"Found: {x}"));
public IObservable<string> FileMonitor(string pathToWatch){
return Observable.Create<string>(obs => {
var dfs = CreateAndStartFileWatcher(pathToWatch,obs);
() => dfs.Dispose();
});
}
Leaving this question for reference instead of removing it.
Related
Following would NOT catch the emitted value:
someObservable = Observable.FromEventPattern(this, "MyEvent");
FireMyEvent("foo");
await someObservable.FirstOrDefaultAsync(e => e.Args == "foo");
So I've come across this pattern in my unit tests:
var tsc = new TaskCompletionSource<bool>();
var callback = someObservable.FirstOrDefaultAsync(...);
using (callback.Subscribe(e => tsc.SetResult(e.Args)))
{
FireMyEvent("foo");
tsc.Task.Wait(1000);
}
Assert.AreEqual("foo", tsc.Task.Result);
Basically I need to subscribe, then trigger the action and then await the subscribed observable.
Is it possible to simplify this subscribe/await without introducing new Task, or WaitHandle?
I'm not sure what exactly you're trying to do, but if I understand the basic idea then this works:
var subject = new Subject<Unit>();
var callback = subject.FirstOrDefaultAsync();
using (callback.Subscribe(_ => { }, () => Console.WriteLine("Done.")))
{
subject.OnNext(Unit.Default);
}
Can you explain your use-case more fully?
I don't really understand your assumption, that the await won't catch the result.
You have to await the event before it gets fired
someObservable = Observable.FromEventPattern(this, "MyEvent");
Task.Delay(1000).ContinueWith(_ => FireMyEvent("foo"));
var result = await someObservable.FirstOrDefaultAsync(e => e.Args == "foo");
Your result ist now a EventPattern. However I would recommend a more type safer FromEventPattern-Overload:
var eventObservable = Observable
.FromEventPattern<YourEventArgs>(
h => MyEvent += h,
h => MyEvent -= h);
Assume I have the following Observable. (Note that the parsing logic lives in a different layer, and should be testable, so it must remain a separate method. Note also that the real loop is parsing XML and has various branching and exception handling).
IObservable<string> GetLinesAsync(StreamReader r)
{
return Observable.Create<string>(subscribeAsync: async (observer, ct) =>
{
//essentially force a continuation/callback to illustrate my problem
await Task.Delay(5);
while (!ct.IsCancellationRequested)
{
string readLine = await r.ReadLineAsync();
if (readLine == null)
break;
observer.OnNext(readLine);
}
});
}
I would like to use this, for example with another Observable that produces the StreamReader, as in the below, but in any case I cannot get the disposal to work.
[TestMethod]
public async Task ReactiveTest()
{
var filePath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Windows), "win.ini");
var source1 = Observable.Using(
() => File.OpenRead(filePath),
readFile => Observable.Using(() => new StreamReader(readFile),
reader => Observable.Return(reader)
)
);
//"GetLinesAsync" already exists. How can I use it?
var combined = source1
.SelectMany(GetLinesAsync);
int count = await combined.Count();
}
If you run this a few times (e.g. with breakpoints, etc), you should see that it blows up because the TextReader is closed. (In my actual problem it happens sporadically on ReadLineAsync but the Task.Delay makes it happen much more easily). Apparently the asynchronous nature causes the first observable to dispose the stream, and only after that does the continuation occur, and of course at that point the stream is already closed.
So:
is the first disposable with the usings set up right? I tried it other ways (see below*)
Is that the right way to do an async Observable (i.e. GetLinesAsync)? Is there anything else I need to do for that?
Is this a proper way to chain the observables together? Assume the GetLinesAsync already exists and if possible its signature shouldn't be changed (e.g. to take in IObservable<StreamReader>)
if this is the right way to glue together the observables, is there any way to get it working with async usage?
*this was another way I set up the first observerable
var source3 = Observable.Create<StreamReader>(observer =>
{
FileStream readFile = File.OpenRead(filePath);
StreamReader reader = new StreamReader(readFile);
observer.OnNext(reader);
observer.OnCompleted();
return new CompositeDisposable(readFile, reader);
});
You really need to make good use of the Defer and Using operators here.
Using is specifically for the case where you have a disposable resource that you would like to have created and finally disposed of when the subscription starts and completes respectively.
Defer is a way to ensure that you always create a new pipeline whenever you have a new subscription (read more on MSDN)
Your second approach is the way to go. You got this 100% right:
Observable.Using(
() => File.OpenRead(filePath),
readFile =>
Observable.Using(
() => new StreamReader(readFile),
reader =>
This will open and dispose of the resources at the correct time for each.
It's what goes before this block of code and what's after the reader => that you need to fix.
After the reader => is this:
Observable
.Defer(() => Observable.FromAsync(() => reader.ReadLineAsync()))
.Repeat()
.TakeWhile(x => x != null)));
That's the idiomatic way for Rx to read from a stream until completion.
The "before" block is just another Defer to ensure that you compute Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Windows), "win.ini") with each new subscriber. It's not necessary in this case because we know that the filePath won't change, but it's good practice and quite probably crucial when this value can change.
Here's the full code:
public async Task ReactiveTest()
{
IObservable<string> combined =
Observable.Defer(() =>
{
var filePath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Windows), "win.ini");
return
Observable.Using(
() => File.OpenRead(filePath),
readFile =>
Observable.Using(
() => new StreamReader(readFile),
reader =>
Observable
.Defer(() => Observable.FromAsync(() => reader.ReadLineAsync()))
.Repeat()
.TakeWhile(x => x != null)));
});
int count = await combined.Count();
}
I've tested it and it works superbly.
Given that you have a fixed signature for GetLines you can do this:
public IObservable<string> GetLines(StreamReader reader)
{
return Observable
.Defer(() => Observable.FromAsync(() => reader.ReadLineAsync()))
.Repeat()
.TakeWhile(x => x != null);
}
public async Task ReactiveTest()
{
IObservable<string> combined =
Observable.Defer(() =>
{
var filePath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Windows), "win.ini");
return
Observable.Using(
() => File.OpenRead(filePath),
readFile =>
Observable.Using(
() => new StreamReader(readFile),
GetLines));
});
int count = await combined.Count();
}
It also works and was tested.
The problem you are having is that your sequences return a single item, the reader. Making use of the reader, requires the file stream to be open. The file stream is unfortunately closed immediately after the stream reader is created:
StreamReader reader is created
OnNext(reader) is called
using block exits, disposing of stream
OnComplete is called, terminating the subscription
Oops!
To fix this, you must tie the lifetime of the StreamReader to the lifetime of the consumer rather than the producer. The original fault occurs because Observable.Using disposes the resource as soon as OnCompleted is called upon the source.
// Do not dispose of the reader when it is created
var readerSequence = Observable.Return(new StreamReader(ms));
var combined = readerSequence
.Select(reader =>
{
return Observable.Using(() => reader, resource => GetLines(resource));
})
.Concat();
I'm not a massive fan of this as you now rely on your consumer cleaning up the each StreamReader but I'm yet to formulate a better way!
So far this is the only thing that has worked while allowing me to keep using GetLinesAsync:
//"GetLinesAsync" already exists. How can I use it?
[TestMethod]
public async Task ReactiveTest2()
{
var combined2 = Observable.Create<string>(async observer =>
{
var filePath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Windows), "win.ini");
using (FileStream readFile = File.OpenRead(filePath))
{
using (StreamReader reader = new StreamReader(readFile))
{
await GetLinesAsync(reader)
.ForEachAsync(result => observer.OnNext(result));
}
}
});
int count = await combined2.Count();
}
This does not work reliably:
[TestMethod]
public async Task ReactiveTest3()
{
var filePath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Windows), "win.ini");
var source1 = Observable.Defer(() => Observable.Using(
() => File.OpenRead(filePath),
readFile => Observable.Using(() => new StreamReader(readFile),
reader => Observable.Return(reader)
)
));
//"GetLines" already exists. How can I use it?
var combined = source1
.SelectMany(reader => Observable.Defer(() => GetLinesAsync(reader)));
int count = await combined.Count();
}
It only seems to work if there's one Observable, as per Enigmativity's solution:
[TestMethod]
public async Task ReactiveTest4()
{
var filePath = Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.Windows), "win.ini");
var source1 = Observable.Using(
() => File.OpenRead(filePath),
readFile => Observable.Using(() => new StreamReader(readFile),
reader => GetLinesAsync(reader)
)
);
int count = await source1.Count();
}
But I haven't found a way to preserve the separation between the two Observables (which I do for layering and unit test purposes) and have it work right, so I don't consider the question answered.
I'm trying to create an Rx operator that seems pretty useful, but I've suprisingly not found any questions on Stackoverflow that match precisely. I'd like to create a variation on Throttle that lets values through immediately if there's been a period of inactivity. My imagined use case is something like this:
I have a dropdown that kicks off a web request when the value is changed. If the user holds down the arrow key and cycles rapidly through the values, I don't want to kick off a request for each value. But if I throttle the stream then the user has to wait out the throttle duration every time they just select a value from the dropdown in the normal manner.
So whereas a normal Throttle looks like this:
I want to create ThrottleSubsequent that look like this:
Note that marbles 1, 2, and 6 are passed through without delay because they each follow a period of inactivity.
My attempt at this looks like the following:
public static IObservable<TSource> ThrottleSubsequent<TSource>(this IObservable<TSource> source, TimeSpan dueTime, IScheduler scheduler)
{
// Create a timer that resets with each new source value
var cooldownTimer = source
.Select(x => Observable.Interval(dueTime, scheduler)) // Each source value becomes a new timer
.Switch(); // Switch to the most recent timer
var cooldownWindow = source.Window(() => cooldownTimer);
// Pass along the first value of each cooldown window immediately
var firstAfterCooldown = cooldownWindow.SelectMany(o => o.Take(1));
// Throttle the rest of the values
var throttledRest = cooldownWindow
.SelectMany(o => o.Skip(1))
.Throttle(dueTime, scheduler);
return Observable.Merge(firstAfterCooldown, throttledRest);
}
This seems to work, but I'm having a difficult time reasoning about this, and I get the feeling there are some edge cases here where things might get screwy with duplicate values or something. I'd like to get some feedback from more experienced Rx-ers as to whether or not this code is correct, and/or whether there is a more idiomatic way of doing this.
Well, here's a test suite (using nuget Microsoft.Reactive.Testing):
var ts = new TestScheduler();
var source = ts.CreateHotObservable<char>(
new Recorded<Notification<char>>(200.MsTicks(), Notification.CreateOnNext('A')),
new Recorded<Notification<char>>(300.MsTicks(), Notification.CreateOnNext('B')),
new Recorded<Notification<char>>(500.MsTicks(), Notification.CreateOnNext('C')),
new Recorded<Notification<char>>(510.MsTicks(), Notification.CreateOnNext('D')),
new Recorded<Notification<char>>(550.MsTicks(), Notification.CreateOnNext('E')),
new Recorded<Notification<char>>(610.MsTicks(), Notification.CreateOnNext('F')),
new Recorded<Notification<char>>(760.MsTicks(), Notification.CreateOnNext('G'))
);
var target = source.ThrottleSubsequent(TimeSpan.FromMilliseconds(150), ts);
var expectedResults = ts.CreateHotObservable<char>(
new Recorded<Notification<char>>(200.MsTicks(), Notification.CreateOnNext('A')),
new Recorded<Notification<char>>(450.MsTicks(), Notification.CreateOnNext('B')),
new Recorded<Notification<char>>(500.MsTicks(), Notification.CreateOnNext('C')),
new Recorded<Notification<char>>(910.MsTicks(), Notification.CreateOnNext('G'))
);
var observer = ts.CreateObserver<char>();
target.Subscribe(observer);
ts.Start();
ReactiveAssert.AreElementsEqual(expectedResults.Messages, observer.Messages);
and using
public static class TestingHelpers
{
public static long MsTicks(this int i)
{
return TimeSpan.FromMilliseconds(i).Ticks;
}
}
Seems to pass. If you wanted to reduce it, you could turn it into this:
public static IObservable<TSource> ThrottleSubsequent2<TSource>(this IObservable<TSource> source, TimeSpan dueTime, IScheduler scheduler)
{
return source.Publish(_source => _source
.Window(() => _source
.Select(x => Observable.Interval(dueTime, scheduler))
.Switch()
))
.Publish(cooldownWindow =>
Observable.Merge(
cooldownWindow
.SelectMany(o => o.Take(1)),
cooldownWindow
.SelectMany(o => o.Skip(1))
.Throttle(dueTime, scheduler)
)
);
}
EDIT:
Publish forces sharing of a subscription. If you have a bad (or expensive) source observable with subscription side-effects, Publish makes sure you only subscribe once. Here's an example where Publish helps:
void Main()
{
var source = UglyRange(10);
var target = source
.SelectMany(i => Observable.Return(i).Delay(TimeSpan.FromMilliseconds(10 * i)))
.ThrottleSubsequent2(TimeSpan.FromMilliseconds(70), Scheduler.Default) //Works with ThrottleSubsequent2, fails with ThrottleSubsequent
.Subscribe(i => Console.WriteLine(i));
}
static int counter = 0;
public IObservable<int> UglyRange(int limit)
{
var uglySource = Observable.Create<int>(o =>
{
if (counter++ == 0)
{
Console.WriteLine("Ugly observable should only be created once.");
Enumerable.Range(1, limit).ToList().ForEach(i => o.OnNext(i));
}
else
{
Console.WriteLine($"Ugly observable should only be created once. This is the {counter}th time created.");
o.OnError(new Exception($"observable invoked {counter} times."));
}
return Disposable.Empty;
});
return uglySource;
}
I am trying to create an observable pipeline with the following characteristics:
is lazy (does nothing until somebody subscribes)
executes at most once regardless of how many subscriptions are received
replays its resulting value, if any OR
replays its resulting error, if any
For the life of me, I can't figure out the correct semantics to accomplish this. I thought it would be a simple case of doing something like this:
Observable
.Defer(() => Observable
.Start(() => { /* do something */ })
.PublishLast()
.ConnectUntilCompleted());
Where ConnectUntilCompleted just does what it sounds like:
public static IObservable<T> ConnectUntilCompleted<T>(this IConnectableObservable<T> #this)
{
#this.Connect();
return #this;
}
This seems to work when the observable terminates successfully, but not when there's an error. Any subscribers do not receive the error:
[Fact]
public void test()
{
var o = Observable
.Defer(() => Observable
.Start(() => { throw new InvalidOperationException(); })
.PublishLast()
.ConnectUntilCompleted());
// this does not throw!
o.Subscribe();
}
Can anyone tell me what I'm doing wrong? Why doesn't Publish replay any error it receives?
UPDATE: it gets even stranger:
[Fact]
public void test()
{
var o = Observable
.Defer(() => Observable
.Start(() => { throw new InvalidOperationException(); })
.PublishLast()
.ConnectUntilCompleted())
.Do(
_ => { },
ex => { /* this executes */ });
// this does not throw!
o.Subscribe();
o.Subscribe(
_ => { },
ex => { /* even though this executes */ });
}
Try this version of you ConnectUntilCompleted method:
public static IObservable<T> ConnectUntilCompleted<T>(this IConnectableObservable<T> #this)
{
return Observable.Create<T>(o =>
{
var subscription = #this.Subscribe(o);
var connection = #this.Connect();
return new CompositeDisposable(subscription, connection);
});
}
The allows Rx to behave properly.
Now I've added to it to help show what's going on:
public static IObservable<T> ConnectUntilCompleted<T>(this IConnectableObservable<T> #this)
{
return Observable.Create<T>(o =>
{
var disposed = Disposable.Create(() => Console.WriteLine("Disposed!"));
var subscription = Observable
.Defer<T>(() => { Console.WriteLine("Subscribing!"); return #this; })
.Subscribe(o);
Console.WriteLine("Connecting!");
var connection = #this.Connect();
return new CompositeDisposable(disposed, subscription, connection);
});
}
Now your observable looks like this:
var o =
Observable
.Defer(() =>
Observable
.Start(() =>
{
Console.WriteLine("Started.");
throw new InvalidOperationException();
}))
.PublishLast()
.ConnectUntilCompleted();
The final key thing is to actually handle the errors in the subscription - so it's not enough to simply do o.Subscribe().
So do this:
o.Subscribe(
x => Console.WriteLine(x),
e => Console.WriteLine(e.Message),
() => Console.WriteLine("Done."));
o.Subscribe(
x => Console.WriteLine(x),
e => Console.WriteLine(e.Message),
() => Console.WriteLine("Done."));
o.Subscribe(
x => Console.WriteLine(x),
e => Console.WriteLine(e.Message),
() => Console.WriteLine("Done."));
When I run that I get this:
Subscribing!
Connecting!
Subscribing!
Connecting!
Subscribing!
Connecting!
Started.
Operation is not valid due to the current state of the object.
Disposed!
Operation is not valid due to the current state of the object.
Disposed!
Operation is not valid due to the current state of the object.
Disposed!
Note that "Started" only appears once, but the error is reported three times.
(Sometimes Started appears higher up in the list after the first subscription.)
I think this is what you wanted from your description.
Just to support #Engimativity's answer, i want to show how you should be runing your tests so you stop getting these "surprises". Your tests are non-deterministic because they are multi-threaded/concurrent. Your use of Observable.Start without providing an IScheduler is problematic. If you run your tests with a TestScheduler your tests will now be singlethreaded and determinisitic
[Test]
public void Test()
{
var testScheduler = new TestScheduler();
var o = Observable
.Defer(() => Observable
.Start(() => { throw new InvalidOperationException(); }, testScheduler)
.PublishLast()
.ConnectUntilCompleted());
var observer = testScheduler.CreateObserver<Unit>();
o.Subscribe(observer);
testScheduler.Start();
CollectionAssert.IsNotEmpty(observer.Messages);
Assert.AreEqual(NotificationKind.OnError, observer.Messages[0].Value.Kind);
}
An alternative way to achieve your requirements could be:
var lazy = new Lazy<Task>(async () => { /* execute once */ }, isThreadSafe: true);
var o = Observable.FromAsync(() => lazy.Value);
When subscribed for the first time, lazy would create (and execute) the task. For other subscriptions, lazy would return the same (possibly already completed or failed) task.
I'm trying to solve the following:
a) subscriber receives events from IObservable for some time. Then it unsubscribes, do some stuff and then subscribe again. Here it should start receiving events from exactly the same point where unsubscription was performed.
b) Such behavior is desirable for multiple subscribers model. E.g. when one has unsubscribed, others should continue receiving events.
Are there any suggestions from the RX side?
Thanks in advance!
Here's a reasonably simple Rx way to do what you want copied from my answer to this other question. I've created an extension method called Pausable that takes a source observable and a second observable of boolean that pauses or resumes the observable.
public static IObservable<T> Pausable<T>(
this IObservable<T> source,
IObservable<bool> pauser)
{
return Observable.Create<T>(o =>
{
var paused = new SerialDisposable();
var subscription = Observable.Publish(source, ps =>
{
var values = new ReplaySubject<T>();
Func<bool, IObservable<T>> switcher = b =>
{
if (b)
{
values.Dispose();
values = new ReplaySubject<T>();
paused.Disposable = ps.Subscribe(values);
return Observable.Empty<T>();
}
else
{
return values.Concat(ps);
}
};
return pauser.StartWith(false).DistinctUntilChanged()
.Select(p => switcher(p))
.Switch();
}).Subscribe(o);
return new CompositeDisposable(subscription, paused);
});
}
It can be used like this:
var xs = Observable.Generate(
0,
x => x < 100,
x => x + 1,
x => x,
x => TimeSpan.FromSeconds(0.1));
var bs = new Subject<bool>();
var pxs = xs.Pausable(bs);
pxs.Subscribe(x => { /* Do stuff */ });
Thread.Sleep(500);
bs.OnNext(true);
Thread.Sleep(5000);
bs.OnNext(false);
Thread.Sleep(500);
bs.OnNext(true);
Thread.Sleep(5000);
bs.OnNext(false);
It sounds like you need a "pausable" stream. Assuming that only 1 subscriber will handle the values at a time (while the other subscribers just wait), this solution is probably what you need.