We Keep Coding

Spooling ongoing items generated by Observable.FromEvent

My goal here is to spool all items/notifications going from IObservable<T> for future subscribers.
E.g. if someone subscribes on a message stream, first he receives all messages which came prior to the subscription. Then he starts receiving new messages, whenever there are any. This should occur seamlessly, without repetitions and losses on the "boundary" between old and new messages.
I came up with following extension method:
public static IObservable<T> WithHistory<T>(this IObservable<T> source)
{
var accumulator = new BlockingCollection<T>();
source.Subscribe(accumulator.Add);
return accumulator
.GetConsumingEnumerable()
.ToObservable()
.SubscribeOn(ThreadPoolScheduler.Instance);
}
As far as I tested it, it works:
class Generator<T>
{
event Action<T> onPush;
public IObservable<T> Items =>
Observable.FromEvent<T>(d => onPush += d, d => onPush -= d);
public void Push(T item) => onPush?.Invoke(item);
}
...
private static void Main()
{
var g = new Generator<int>();
var ongoingItems = g.Items;
var allItems = g.Items.WithHistory();
g.Push(1);
g.Push(2);
ongoingItems.Subscribe(x => Console.WriteLine($"Ongoing: got {x}"));
allItems.Subscribe(x => Console.WriteLine($"WithHistory: got {x}"));
g.Push(3);
g.Push(4);
g.Push(5);
Console.ReadLine();
}
The result:
Ongoing: got 3
Ongoing: got 4
Ongoing: got 5
WithHistory: got 1
WithHistory: got 2
WithHistory: got 3
WithHistory: got 4
WithHistory: got 5
However, using BlockingCollection<T> seems to be an overkill. Also the method above does not support completion, error handling and would cause deadlocks without .SubscribeOn(ThreadPoolScheduler.Instance).
Is there any better way to achieve it, without the described flaws?

Best way to do it is with .Replay()
void Main()
{
var g = new Generator<int>();
var ongoingItems = g.Items;
var allItems = g.Items.Replay().RefCount();
using(var tempSubscriber = allItems.Subscribe())
{
g.Push(1);
g.Push(2);
ongoingItems.Subscribe(x => Console.WriteLine($"Ongoing: got {x}"));
allItems.Subscribe(x => Console.WriteLine($"WithHistory: got {x}"));
g.Push(3);
g.Push(4);
g.Push(5);
Console.ReadLine();
}
}
.Replay().RefCount() produces an observable that will keep an internal queue for replaying, as long as there's a subscriber. If you have a persistent subscriber though (like your solution does in the WithHistory method), you have a memory leak. The best way to get around this is to have a temporary subscriber which automatically disconnects after you're no longer interested in the history.

RX terminolgy: Async processing in RX operator when there are frequent observable notifications

The purpose is to do some async work on a scarce resource in a RX operator, Select for example. Issues arise when observable notifications came at a rate that is faster than the time it takes for the async operation to complete.
Now I actually solved the problem. My question would be what is the correct terminology for this particular kind of issue? Does it have a name? Is it backpressure? Research I did until now indicate that this is some kind of a pressure problem, but not necessarily backpressure from my understanding. The most relevant resources I found are these:
https://github.com/ReactiveX/RxJava/wiki/Backpressure-(2.0)
http://reactivex.io/documentation/operators/backpressure.html
Now to the actual code. Suppose there is a scarce resource and it's consumer. In this case exception is thrown when resource is in use. Please note that this code should not be changed.
public class ScarceResource
{
private static bool inUse = false;
public async Task<int> AccessResource()
{
if (inUse) throw new Exception("Resource is alredy in use");
var result = await Task.Run(() =>
{
inUse = true;
Random random = new Random();
Thread.Sleep(random.Next(1, 2) * 1000);
inUse = false;
return random.Next(1, 10);
});
return result;
}
}
public class ResourceConsumer
{
public IObservable<int> DoWork()
{
var resource = new ScarceResource();
return resource.AccessResource().ToObservable();
}
}
Now here is the problem with a naive implementation to consume the resource. Error is thrown because notifications came at a faster rate than the consumer takes to run.
private static void RunIntoIssue()
{
var numbers = Enumerable.Range(1, 10);
var observableSequence = numbers
.ToObservable()
.SelectMany(n =>
{
Console.WriteLine("In observable: {0}", n);
var resourceConsumer = new ResourceConsumer();
return resourceConsumer.DoWork();
});
observableSequence.Subscribe(n => Console.WriteLine("In observer: {0}", n));
}
With the following code the problem is solved. I slow down processing by using a completed BehaviorSubject in conjunction with the Zip operator. Essentially what this code does is to take a sequential approach instead of a parallel one.
private static void RunWithZip()
{
var completed = new BehaviorSubject<bool>(true);
var numbers = Enumerable.Range(1, 10);
var observableSequence = numbers
.ToObservable()
.Zip(completed, (n, c) =>
{
Console.WriteLine("In observable: {0}, completed: {1}", n, c);
var resourceConsumer = new ResourceConsumer();
return resourceConsumer.DoWork();
})
.Switch()
.Select(n =>
{
completed.OnNext(true);
return n;
});
observableSequence.Subscribe(n => Console.WriteLine("In observer: {0}", n));
Console.Read();
}
Question
Is this backpressure, and if not does it have another terminology associated?

You're basically implementing a form of locking, or a mutex. Your code an cause backpressure, it's not really handling it.
Imagine if your source wasn't a generator function, but rather a series of data pushes. The data pushes arrive at a constant rate of every millisecond. It takes you 10 Millis to process each one, and your code forces serial processing. This causes backpressure: Zip will queue up the unprocessed datapushes infinitely until you run out of memory.

Custom Rx operator for throttling only when there's a been a recent value

I'm trying to create an Rx operator that seems pretty useful, but I've suprisingly not found any questions on Stackoverflow that match precisely. I'd like to create a variation on Throttle that lets values through immediately if there's been a period of inactivity. My imagined use case is something like this:
I have a dropdown that kicks off a web request when the value is changed. If the user holds down the arrow key and cycles rapidly through the values, I don't want to kick off a request for each value. But if I throttle the stream then the user has to wait out the throttle duration every time they just select a value from the dropdown in the normal manner.
So whereas a normal Throttle looks like this:
I want to create ThrottleSubsequent that look like this:
Note that marbles 1, 2, and 6 are passed through without delay because they each follow a period of inactivity.
My attempt at this looks like the following:
public static IObservable<TSource> ThrottleSubsequent<TSource>(this IObservable<TSource> source, TimeSpan dueTime, IScheduler scheduler)
{
// Create a timer that resets with each new source value
var cooldownTimer = source
.Select(x => Observable.Interval(dueTime, scheduler)) // Each source value becomes a new timer
.Switch(); // Switch to the most recent timer
var cooldownWindow = source.Window(() => cooldownTimer);
// Pass along the first value of each cooldown window immediately
var firstAfterCooldown = cooldownWindow.SelectMany(o => o.Take(1));
// Throttle the rest of the values
var throttledRest = cooldownWindow
.SelectMany(o => o.Skip(1))
.Throttle(dueTime, scheduler);
return Observable.Merge(firstAfterCooldown, throttledRest);
}
This seems to work, but I'm having a difficult time reasoning about this, and I get the feeling there are some edge cases here where things might get screwy with duplicate values or something. I'd like to get some feedback from more experienced Rx-ers as to whether or not this code is correct, and/or whether there is a more idiomatic way of doing this.

Well, here's a test suite (using nuget Microsoft.Reactive.Testing):
var ts = new TestScheduler();
var source = ts.CreateHotObservable<char>(
new Recorded<Notification<char>>(200.MsTicks(), Notification.CreateOnNext('A')),
new Recorded<Notification<char>>(300.MsTicks(), Notification.CreateOnNext('B')),
new Recorded<Notification<char>>(500.MsTicks(), Notification.CreateOnNext('C')),
new Recorded<Notification<char>>(510.MsTicks(), Notification.CreateOnNext('D')),
new Recorded<Notification<char>>(550.MsTicks(), Notification.CreateOnNext('E')),
new Recorded<Notification<char>>(610.MsTicks(), Notification.CreateOnNext('F')),
new Recorded<Notification<char>>(760.MsTicks(), Notification.CreateOnNext('G'))
);
var target = source.ThrottleSubsequent(TimeSpan.FromMilliseconds(150), ts);
var expectedResults = ts.CreateHotObservable<char>(
new Recorded<Notification<char>>(200.MsTicks(), Notification.CreateOnNext('A')),
new Recorded<Notification<char>>(450.MsTicks(), Notification.CreateOnNext('B')),
new Recorded<Notification<char>>(500.MsTicks(), Notification.CreateOnNext('C')),
new Recorded<Notification<char>>(910.MsTicks(), Notification.CreateOnNext('G'))
);
var observer = ts.CreateObserver<char>();
target.Subscribe(observer);
ts.Start();
ReactiveAssert.AreElementsEqual(expectedResults.Messages, observer.Messages);
and using
public static class TestingHelpers
{
public static long MsTicks(this int i)
{
return TimeSpan.FromMilliseconds(i).Ticks;
}
}
Seems to pass. If you wanted to reduce it, you could turn it into this:
public static IObservable<TSource> ThrottleSubsequent2<TSource>(this IObservable<TSource> source, TimeSpan dueTime, IScheduler scheduler)
{
return source.Publish(_source => _source
.Window(() => _source
.Select(x => Observable.Interval(dueTime, scheduler))
.Switch()
))
.Publish(cooldownWindow =>
Observable.Merge(
cooldownWindow
.SelectMany(o => o.Take(1)),
cooldownWindow
.SelectMany(o => o.Skip(1))
.Throttle(dueTime, scheduler)
)
);
}
EDIT:
Publish forces sharing of a subscription. If you have a bad (or expensive) source observable with subscription side-effects, Publish makes sure you only subscribe once. Here's an example where Publish helps:
void Main()
{
var source = UglyRange(10);
var target = source
.SelectMany(i => Observable.Return(i).Delay(TimeSpan.FromMilliseconds(10 * i)))
.ThrottleSubsequent2(TimeSpan.FromMilliseconds(70), Scheduler.Default) //Works with ThrottleSubsequent2, fails with ThrottleSubsequent
.Subscribe(i => Console.WriteLine(i));
}
static int counter = 0;
public IObservable<int> UglyRange(int limit)
{
var uglySource = Observable.Create<int>(o =>
{
if (counter++ == 0)
{
Console.WriteLine("Ugly observable should only be created once.");
Enumerable.Range(1, limit).ToList().ForEach(i => o.OnNext(i));
}
else
{
Console.WriteLine($"Ugly observable should only be created once. This is the {counter}th time created.");
o.OnError(new Exception($"observable invoked {counter} times."));
}
return Disposable.Empty;
});
return uglySource;
}

Rx: How can I respond immediately, and throttle subsequent requests

I would like to set up an Rx subscription that can respond to an event right away, and then ignore subsequent events that happen within a specified "cooldown" period.
The out of the box Throttle/Buffer methods respond only once the timeout has elapsed, which is not quite what I need.
Here is some code that sets up the scenario, and uses a Throttle (which isn't the solution I want):
class Program
{
static Stopwatch sw = new Stopwatch();
static void Main(string[] args)
{
var subject = new Subject<int>();
var timeout = TimeSpan.FromMilliseconds(500);
subject
.Throttle(timeout)
.Subscribe(DoStuff);
var factory = new TaskFactory();
sw.Start();
factory.StartNew(() =>
{
Console.WriteLine("Batch 1 (no delay)");
subject.OnNext(1);
});
factory.StartNewDelayed(1000, () =>
{
Console.WriteLine("Batch 2 (1s delay)");
subject.OnNext(2);
});
factory.StartNewDelayed(1300, () =>
{
Console.WriteLine("Batch 3 (1.3s delay)");
subject.OnNext(3);
});
factory.StartNewDelayed(1600, () =>
{
Console.WriteLine("Batch 4 (1.6s delay)");
subject.OnNext(4);
});
Console.ReadKey();
sw.Stop();
}
private static void DoStuff(int i)
{
Console.WriteLine("Handling {0} at {1}ms", i, sw.ElapsedMilliseconds);
}
}
The output of running this right now is:
Batch 1 (no delay)
Handling 1 at 508ms
Batch 2 (1s delay)
Batch 3 (1.3s delay)
Batch 4 (1.6s delay)
Handling 4 at 2114ms
Note that batch 2 isn't handled (which is fine!) because we wait for 500ms to elapse between requests due to the nature of throttle. Batch 3 is also not handled, (which is less alright because it happened more than 500ms from batch 2) due to its proximity to Batch 4.
What I'm looking for is something more like this:
Batch 1 (no delay)
Handling 1 at ~0ms
Batch 2 (1s delay)
Handling 2 at ~1000s
Batch 3 (1.3s delay)
Batch 4 (1.6s delay)
Handling 4 at ~1600s
Note that batch 3 wouldn't be handled in this scenario (which is fine!) because it occurs within 500ms of Batch 2.
EDIT:
Here is the implementation for the "StartNewDelayed" extension method that I use:
/// <summary>Creates a Task that will complete after the specified delay.</summary>
/// <param name="factory">The TaskFactory.</param>
/// <param name="millisecondsDelay">The delay after which the Task should transition to RanToCompletion.</param>
/// <returns>A Task that will be completed after the specified duration.</returns>
public static Task StartNewDelayed(
this TaskFactory factory, int millisecondsDelay)
{
return StartNewDelayed(factory, millisecondsDelay, CancellationToken.None);
}
/// <summary>Creates a Task that will complete after the specified delay.</summary>
/// <param name="factory">The TaskFactory.</param>
/// <param name="millisecondsDelay">The delay after which the Task should transition to RanToCompletion.</param>
/// <param name="cancellationToken">The cancellation token that can be used to cancel the timed task.</param>
/// <returns>A Task that will be completed after the specified duration and that's cancelable with the specified token.</returns>
public static Task StartNewDelayed(this TaskFactory factory, int millisecondsDelay, CancellationToken cancellationToken)
{
// Validate arguments
if (factory == null) throw new ArgumentNullException("factory");
if (millisecondsDelay < 0) throw new ArgumentOutOfRangeException("millisecondsDelay");
// Create the timed task
var tcs = new TaskCompletionSource<object>(factory.CreationOptions);
var ctr = default(CancellationTokenRegistration);
// Create the timer but don't start it yet. If we start it now,
// it might fire before ctr has been set to the right registration.
var timer = new Timer(self =>
{
// Clean up both the cancellation token and the timer, and try to transition to completed
ctr.Dispose();
((Timer)self).Dispose();
tcs.TrySetResult(null);
});
// Register with the cancellation token.
if (cancellationToken.CanBeCanceled)
{
// When cancellation occurs, cancel the timer and try to transition to cancelled.
// There could be a race, but it's benign.
ctr = cancellationToken.Register(() =>
{
timer.Dispose();
tcs.TrySetCanceled();
});
}
if (millisecondsDelay > 0)
{
// Start the timer and hand back the task...
timer.Change(millisecondsDelay, Timeout.Infinite);
}
else
{
// Just complete the task, and keep execution on the current thread.
ctr.Dispose();
tcs.TrySetResult(null);
timer.Dispose();
}
return tcs.Task;
}

Here's my approach. It's similar to others that have gone before, but it doesn't suffer the over-zealous window production problem.
The desired function works a lot like Observable.Throttle but emits qualifying events as soon as they arrive rather than delaying for the duration of the throttle or sample period. For a given duration after a qualifying event, subsequent events are suppressed.
Given as a testable extension method:
public static class ObservableExtensions
{
public static IObservable<T> SampleFirst<T>(
this IObservable<T> source,
TimeSpan sampleDuration,
IScheduler scheduler = null)
{
scheduler = scheduler ?? Scheduler.Default;
return source.Publish(ps =>
ps.Window(() => ps.Delay(sampleDuration,scheduler))
.SelectMany(x => x.Take(1)));
}
}
The idea is to use the overload of Window that creates non-overlapping windows using a windowClosingSelector that uses the source time-shifted back by the sampleDuration. Each window will therefore: (a) be closed by the first element in it and (b) remain open until a new element is permitted. We then simply select the first element from each window.
Rx 1.x Version
The Publish extension method used above is not available in Rx 1.x. Here is an alternative:
public static class ObservableExtensions
{
public static IObservable<T> SampleFirst<T>(
this IObservable<T> source,
TimeSpan sampleDuration,
IScheduler scheduler = null)
{
scheduler = scheduler ?? Scheduler.Default;
var sourcePub = source.Publish().RefCount();
return sourcePub.Window(() => sourcePub.Delay(sampleDuration,scheduler))
.SelectMany(x => x.Take(1));
}
}

The solution I found after a lot of trial and error was to replace the throttled subscription with the following:
subject
.Window(() => { return Observable.Interval(timeout); })
.SelectMany(x => x.Take(1))
.Subscribe(i => DoStuff(i));
Edited to incorporate Paul's clean-up.

Awesome solution Andrew! We can take this a step further though and clean up the inner Subscribe:
subject
.Window(() => { return Observable.Interval(timeout); })
.SelectMany(x => x.Take(1))
.Subscribe(DoStuff);

The initial answer I posted has a flaw: namely that the Window method, when used with an Observable.Interval to denote the end of the window, sets up an infinite series of 500ms windows. What I really need is a window that starts when the first result is pumped into the subject, and ends after the 500ms.
My sample data masked this problem because the data broke down nicely into the windows that were already going to be created. (i.e. 0-500ms, 501-1000ms, 1001-1500ms, etc.)
Consider instead this timing:
factory.StartNewDelayed(300,() =>
{
Console.WriteLine("Batch 1 (300ms delay)");
subject.OnNext(1);
});
factory.StartNewDelayed(700, () =>
{
Console.WriteLine("Batch 2 (700ms delay)");
subject.OnNext(2);
});
factory.StartNewDelayed(1300, () =>
{
Console.WriteLine("Batch 3 (1.3s delay)");
subject.OnNext(3);
});
factory.StartNewDelayed(1600, () =>
{
Console.WriteLine("Batch 4 (1.6s delay)");
subject.OnNext(4);
});
What I get is:
Batch 1 (300ms delay)
Handling 1 at 356ms
Batch 2 (700ms delay)
Handling 2 at 750ms
Batch 3 (1.3s delay)
Handling 3 at 1346ms
Batch 4 (1.6s delay)
Handling 4 at 1644ms
This is because the windows begin at 0ms, 500ms, 1000ms, and 1500ms and so each Subject.OnNext fits nicely into its own window.
What I want is:
Batch 1 (300ms delay)
Handling 1 at ~300ms
Batch 2 (700ms delay)
Batch 3 (1.3s delay)
Handling 3 at ~1300ms
Batch 4 (1.6s delay)
After a lot of struggling and an hour banging on it with a co-worker, we arrived at a better solution using pure Rx and a single local variable:
bool isCoolingDown = false;
subject
.Where(_ => !isCoolingDown)
.Subscribe(
i =>
{
DoStuff(i);
isCoolingDown = true;
Observable
.Interval(cooldownInterval)
.Take(1)
.Subscribe(_ => isCoolingDown = false);
});
Our assumption is that calls to the subscription method are synchronized. If they are not, then a simple lock could be introduced.

Use .Scan() !
This is what I use for Throttling when I need the first hit (after a certain period) immediately, but delay (and group/ignore) any subsequent hits.
Basically works like Throttle, but fires immediately if the previous onNext was >= interval ago, otherwise, schedule it at exactly interval from the previous hit. And of course, if within the 'cooling down' period multiple hits come, the additional ones are ignored, just like Throttle does.
The difference with your use case is that if you get an event at 0 ms and 100 ms, they will both be handled (at 0ms and 500ms), which might be what you actually want (otherwise, the accumulator is easy to adapt to ignore ANY hit closer than interval to the previous one).
public static IObservable<T> QuickThrottle<T>(this IObservable<T> src, TimeSpan interval, IScheduler scheduler)
{
return src
.Scan(new ValueAndDueTime<T>(), (prev, id) => AccumulateForQuickThrottle(prev, id, interval, scheduler))
.Where(vd => !vd.Ignore)
.SelectMany(sc => Observable.Timer(sc.DueTime, scheduler).Select(_ => sc.Value));
}
private static ValueAndDueTime<T> AccumulateForQuickThrottle<T>(ValueAndDueTime<T> prev, T value, TimeSpan interval, IScheduler s)
{
var now = s.Now;
// Ignore this completely if there is already a future item scheduled
// but do keep the dueTime for accumulation!
if (prev.DueTime > now) return new ValueAndDueTime<T> { DueTime = prev.DueTime, Ignore = true };
// Schedule this item at at least interval from the previous
var min = prev.DueTime + interval;
var nextTime = (now < min) ? min : now;
return new ValueAndDueTime<T> { DueTime = nextTime, Value = value };
}
private class ValueAndDueTime<T>
{
public DateTimeOffset DueTime;
public T Value;
public bool Ignore;
}

I got another one for your. This one doesn't use Repeat() nor Interval() so it might be what you are after:
subject
.Window(() => Observable.Timer(TimeSpan.FromMilliseconds(500)))
.SelectMany(x => x.Take(1));

Well the most obvious thing will be to use Repeat() here. However, as far as I know Repeat() might introduce problems so that notifications disappear in between the moment when the stream stops and we subscribe again. In practice this has never been a problem for me.
subject
.Take(1)
.Concat(Observable.Empty<long>().Delay(TimeSpan.FromMilliseconds(500)))
.Repeat();
Remember to replace with the actual type of your source.
UPDATE:
Updated query to use Concat instead of Merge

I have stumbled upon this question while trying to re-implement my own solution to the same or similar problem using .Window
Take a look, it seems to be the same as this one and solved quite elegantly:
https://stackoverflow.com/a/3224723/58463

It's an old post, but no answer could really fill my needs, so I'm giving my own solution :
public static IObservable<T> ThrottleOrImmediate<T>(this IObservable<T> source, TimeSpan delay, IScheduler scheduler)
{
return Observable.Create<T>((obs, token) =>
{
// Next item cannot be send before that time
DateTime nextItemTime = default;
return Task.FromResult(source.Subscribe(async item =>
{
var currentTime = DateTime.Now;
// If we already reach the next item time
if (currentTime - nextItemTime >= TimeSpan.Zero)
{
// Following item will be send only after the set delay
nextItemTime = currentTime + delay;
// send current item with scheduler
scheduler.Schedule(() => obs.OnNext(item));
}
// There is still time before we can send an item
else
{
// we schedule the time for the following item
nextItemTime = currentTime + delay;
try
{
await Task.Delay(delay, token);
}
catch (TaskCanceledException)
{
return;
}
// If next item schedule was change by another item then we stop here
if (nextItemTime > currentTime + delay)
return;
else
{
// Set next possible time for an item and send item with scheduler
nextItemTime = currentTime + delay;
scheduler.Schedule(() => obs.OnNext(item));
}
}
}));
});
}
First item is immediately sent, then following items are throttled. Then if a following item is sent after the delayed time, it's immediately sent too.

A way to push buffered events in even intervals

What I'm trying to achieve is to buffer incoming events from some IObservable ( they come in bursts) and release them further, but one by one, in even intervals.
Like this:
-oo-ooo-oo------------------oooo-oo-o-------------->
-o--o--o--o--o--o--o--------o--o--o--o--o--o--o---->
Since I'm quite new to Rx, I'm not sure if there already is a Subject or an operator that does just this. Maybe It can be done by composition?
update:
Thanks to
Richard Szalay for pointing out the Drain operator, I found another example by James Miles of Drain operator usage. Here's how I managed to get it to work in a WPF app:
.Drain(x => {
Process(x);
return Observable.Return(new Unit())
.Delay(TimeSpan.FromSeconds(1), Scheduler.Dispatcher );
}).Subscribe();
I had some fun, because omitting the scheduler parameter causes the app to crash in debug mode without any exception showing up ( I need to learn how to deal with exceptions in Rx).
The Process method modifies the UI state directly, but I guess it's quite simple to make an IObservable out of it (using a ISubject?).
update:
In the meantime I've been experimenting with ISubject, the class below does what I wanted - it lets out buffered Ts in a timely manner:
public class StepSubject<T> : ISubject<T>
{
IObserver<T> subscriber;
Queue<T> queue = new Queue<T>();
MutableDisposable cancel = new MutableDisposable();
TimeSpan interval;
IScheduler scheduler;
bool idle = true;
public StepSubject(TimeSpan interval, IScheduler scheduler)
{
this.interval = interval;
this.scheduler = scheduler;
}
void Step()
{
T next;
lock (queue)
{
idle = queue.Count == 0;
if (!idle)
next = queue.Dequeue();
}
if (!idle)
{
cancel.Disposable = scheduler.Schedule(Step, interval);
subscriber.OnNext(next);
}
}
public void OnNext(T value)
{
lock (queue)
queue.Enqueue(value);
if (idle)
cancel.Disposable = scheduler.Schedule(Step);
}
public IDisposable Subscribe(IObserver<T> observer)
{
subscriber = observer;
return cancel;
}
}
This naive implementation is stripped from OnCompleted and OnError for clarity, also only single subscription allowed.

It's actually tricker than it sounds.
Using Delay doesn't work because the values will still happen in bulk, only slightly delayed.
Using Interval with either CombineLatest or Zip doesn't work, since the former will cause source values to be skipped and the latter will buffer interval values.
I think the new Drain operator (added in 1.0.2787.0), combined with Delay should do the trick:
source.Drain(x => Observable.Empty<int>().Delay(TimeSpan.FromSeconds(1)).StartWith(x));
The Drain operator works like SelectMany, but waits until the previous output completes before calling the selector with the next value. It's still not exactly what you are after (the first value in a block will also be delayed), but it's close: The usage above matches your marble diagram now.
Edit: Apparently the Drain in the framework doesn't work like SelectMany. I'll ask for some advice in the official forums. In the meantime, here's an implementation of Drain that does what you're after:
Edit 09/11: Fixed errors in implementation and updated usage to match your requested marble diagram.
public static class ObservableDrainExtensions
{
public static IObservable<TOut> Drain<TSource, TOut>(this IObservable<TSource> source,
Func<TSource, IObservable<TOut>> selector)
{
return Observable.Defer(() =>
{
BehaviorSubject<Unit> queue = new BehaviorSubject<Unit>(new Unit());
return source
.Zip(queue, (v, q) => v)
.SelectMany(v => selector(v)
.Do(_ => { }, () => queue.OnNext(new Unit()))
);
});
}
}

Just for completeness here is an alterantive (more compact) version of the Drain() method suggested by Richard:
public static IObservable<T2> SelectManySequential<T1, T2>(
this IObservable<T1> source,
Func<T1, IObservable<T2>> selector
)
{
return source
.Select(x => Observable.Defer<T2>(() => selector(x)))
.Concat();
}
See the thread Drain + SelectMany = ? in the Rx forum.
Update:
I realized that the Concat() overload that I used was one of my personal Rx extensions that are (not yet) part of the framework. I am sorry for this mistake .. Of course this makes my solution less elegant than I thought.
Nevertheless for completeness I post here my Conact() extension method overload:
public static IObservable<T> Concat<T>(this IObservable<IObservable<T>> source)
{
return Observable.CreateWithDisposable<T>(o =>
{
var lockCookie = new Object();
bool completed = false;
bool subscribed = false;
var waiting = new Queue<IObservable<T>>();
var pendingSubscription = new MutableDisposable();
Action<Exception> errorHandler = e =>
{
o.OnError(e);
pendingSubscription.Dispose();
};
Func<IObservable<T>, IDisposable> subscribe = null;
subscribe = (ob) =>
{
subscribed = true;
return ob.Subscribe(
o.OnNext,
errorHandler,
() =>
{
lock (lockCookie)
{
if (waiting.Count > 0)
pendingSubscription.Disposable = subscribe(waiting.Dequeue());
else if (completed)
o.OnCompleted();
else
subscribed = false;
}
}
);
};
return new CompositeDisposable(pendingSubscription,
source.Subscribe(
n =>
{
lock (lockCookie)
{
if (!subscribed)
pendingSubscription.Disposable = subscribe(n);
else
waiting.Enqueue(n);
}
},
errorHandler
, () =>
{
lock (lockCookie)
{
completed = true;
if (!subscribed)
o.OnCompleted();
}
}
)
);
});
}
And now beating myself with my own weapons:
The same Concat() method could be written much more elegant in Richard Szalay's brilliant way:
public static IObservable<T> Concat<T>(this IObservable<IObservable<T>> source)
{
return Observable.Defer(() =>
{
BehaviorSubject<Unit> queue = new BehaviorSubject<Unit>(new Unit());
return source
.Zip(queue, (v, q) => v)
.SelectMany(v =>
v.Do(_ => { }, () => queue.OnNext(new Unit()))
);
});
}
So credit belongs to Richard. :-)

Here's how I did this, just using an explicit queue (ReactiveCollection is just a fancy version of WPF's ObservableCollection - ReactiveCollection.ItemsAdded OnNext's for each item added, as you can imagine):
https://github.com/xpaulbettsx/ReactiveXaml/blob/master/ReactiveXaml/ReactiveCollection.cs#L309
public static ReactiveCollection<T> CreateCollection<T>(this IObservable<T> FromObservable, TimeSpan? WithDelay = null)
{
var ret = new ReactiveCollection<T>();
if (WithDelay == null) {
FromObservable.ObserveOn(RxApp.DeferredScheduler).Subscribe(ret.Add);
return ret;
}
// On a timer, dequeue items from queue if they are available
var queue = new Queue<T>();
var disconnect = Observable.Timer(WithDelay.Value, WithDelay.Value)
.ObserveOn(RxApp.DeferredScheduler).Subscribe(_ => {
if (queue.Count > 0) {
ret.Add(queue.Dequeue());
}
});
// When new items come in from the observable, stuff them in the queue.
// Using the DeferredScheduler guarantees we'll always access the queue
// from the same thread.
FromObservable.ObserveOn(RxApp.DeferredScheduler).Subscribe(queue.Enqueue);
// This is a bit clever - keep a running count of the items actually
// added and compare them to the final count of items provided by the
// Observable. Combine the two values, and when they're equal,
// disconnect the timer
ret.ItemsAdded.Scan0(0, ((acc, _) => acc+1)).Zip(FromObservable.Aggregate(0, (acc,_) => acc+1),
(l,r) => (l == r)).Where(x => x != false).Subscribe(_ => disconnect.Dispose());
return ret;
}