Say, I would like to have a stream that emits a new job 1 second after the previous job is done. The job should be emitted as soon as it's created, so the subscriber can track its progress. How could this be achieved?
var currentJob = Observable.Interval(TimeSpan.FromSeconds(1))
.Select(i => new Job(i))
// .MergeWhen(job => job.IsDone)?
Edit: I understand it's easy to do this in a TPL/Rx mashup flavor; I'm mostly curious that if there is a more Rx-native approach. Based on my experience, if I can't express a logic in a Rx-native way, most of the time it's because I'm not thinking in the Rx way correctly, so insights on how to think about this kind of workflow is also welcome.
Considering you have some kind of job request stream, which produces parameters for the job:
var jobRequestStream = Observable.Interval(TimeSpan.FromSeconds(1));
You could build the stream of jobs with a delay between them with something like this:
var jobTrackingStream = jobRequestStream.Select(i => Observable.Defer(() =>
{
var job = new Job(i);
var awaiter = Observable
.FromAsync(() => job.JobCompletionWait(TimeSpan.FromMilliseconds(100), TimeSpan.FromSeconds(1)))
.Where(_ => false); // this silly-looking where clause is to produce an empty observable that completes after Task completes
return Observable.Return(job).Concat(awaiter);
}))
.Concat()
where JobCompletionWait is an extension that does some kind of async spin wait for the job to complete:
public static class JobExtensions
{
public static async Task<Job> JobCompletionWait(this Job job, TimeSpan completionCheckInterval, TimeSpan delayCompletion)
{
while (!job.IsDone)
{
await Task.Delay(completionCheckInterval);
}
await Task.Delay(delayCompletion);
return job;
}
}
To test this I launched LinqPad and wrote a dummy Job class:
public class Job
{
static readonly Random Rand = new Random();
public bool IsDone { get; private set; }
public long I { get; }
public Job(long i)
{
I = i;
$"{DateTime.Now:hh:mm:ss.ffff} Job {i} started".Dump();
Task.Delay(Rand.Next(2000)).ContinueWith(_ =>
{
$"{DateTime.Now:hh:mm:ss.ffff} Job {i} is done.".Dump(); ;
IsDone = true;
});
}
}
and subscribed to the jobTrackingStream with:
jobTrackingStream
.Subscribe(job => $"{DateTime.Now:hh:mm:ss.ffff} Emited job {job.I}".Dump());
The result was:
08:01:34.8062 Job 0 started
08:01:34.8186 Emited job 0
08:01:36.3715 Job 0 is done.
08:01:37.4795 Job 1 started
08:01:37.4797 Emited job 1
08:01:37.6315 Job 1 is done.
08:01:38.7041 Job 2 started
08:01:38.7043 Emited job 2
08:01:39.7325 Job 2 is done.
08:01:40.8508 Job 3 started
08:01:40.8510 Emited job 3
08:01:42.3270 Job 3 is done.
08:01:43.4013 Job 4 started
08:01:43.4015 Emited job 4
08:01:44.2755 Job 4 is done.
08:01:45.3936 Job 5 started
08:01:45.3939 Emited job 5
08:01:45.8429 Job 5 is done.
08:01:46.9792 Job 6 started
08:01:46.9794 Emited job 6
08:01:47.7110 Job 6 is done.
....
Which seems to be the expected behaviour. Maybe there is a more clean solution but this one is just something that came to my mind.
Edit:
I found task-less solution, without subjects:
var jobTrackingStream = jobRequestStream.Select(i => Observable.Defer(() =>
Observable.Generate(
new Job(i),
job => !job.IsDone,
job => job,
job => job,
_ => TimeSpan.FromMilliseconds(100)
)
.Distinct()
.Concat(Observable.Delay(Observable.Empty<Job>(), TimeSpan.FromSeconds(1)))
))
.Concat();
You can create a sequence that produces just one Job, with the Observable.Start method, and then Repeat this sequence. The delay between completing the one Job and starting the other can be injected by Concatenating a Delayed Empty sequence:
IObservable<Job> jobs = Observable
.Defer(() => Observable.Start(() => new Job()))
.Concat(Observable.Empty<Job>().Delay(TimeSpan.FromSeconds(1)))
.Repeat();
The Job constructor is invoked on a ThreadPool thread. The purpose of the Defer is to "cool" the Observable.Start sequence, which is a hot sequence, so that the first Job constructor is invoked when the jobs sequence is subscribed, not when it is created.
Related
So I have this Task:
public async Task ProcessNewMessages()
{
while (true)
{
await _exceptionHandler.WithExceptionHandling(async () =>
{
List<Message> newUncompletedNewMessages = await _newUncompletedMessagesJob.Execute();
// Do stuff
}
Thread.Sleep(30000);
}
}
The Execute returns a list of 10 messages and I want to add a while loop in the while true that runs the whole code again as long as the Execute returns me a full list of 10 items, that runs straight after the items are done rather than waiting 30 seconds every time.
The 30 seconds will be in place to keep checking if there are new messages and is ireelevant to the Task of handling them.
So to rephrase my question: How do I rerun code based on the list returned by Execute() having 10 messages?
public async Task ProcessNewMessages()
{
while (true)
{
IEnumerable<MessageToSend> uncompletedNewMessages = new List<MessageToSend>();
do
{
await _exceptionHandler.WithExceptionHandling(async () =>
{
uncompletedNewMessages = await _newUncompletedMessagesJob.Execute();
// DoStuff
});
} while (uncompletedNewMessages.Count() == 10)
await Task.Delay(30000);
}
}
I have an answer.
I read people not liking the Thread.Sleep, but this is a webjob running continuously to check for new messages. Now this doesn't need to run every second, because there might not be new message every second, but they can keep on coming in throughout the day. Let me know your suggestions how to do that differently.
Edit: Thanks for the explanation Jeroen
My C# application stops responding for a long time, as I break the Debug it stops on a function.
foreach (var item in list)
{
xmldiff.Compare(item, secondary, output);
...
}
I guess the running time of this function is long or it hangs. Anyway, I want to wait for a certain time (e.g. 5 seconds) for the execution of this function, and if it exceeds this time, I skip it and go to the next item in the loop. How can I do it? I found some similar question but they are mostly for processes or asynchronous methods.
You can do it the brutal way: spin up a thread to do the work, join it with timeout, then abort it, if the join didn't work.
Example:
var worker = new Thread( () => { xmlDiff.Compare(item, secondary, output); } );
worker.Start();
if (!worker.Join( TimeSpan.FromSeconds( 1 ) ))
worker.Abort();
But be warned - aborting threads is not considered nice and can make your app unstable. If at all possible try to modify Compare to accept a CancellationToken to cancel the comparison.
I would avoid directly using threads and use Microsoft's Reactive Extensions (NuGet "Rx-Main") to abstract away the management of the threads.
I don't know the exact signature of xmldiff.Compare(item, secondary, output) but if I assume it produces an integer then I could do this with Rx:
var query =
from item in list.ToObservable()
from result in
Observable
.Start(() => xmldiff.Compare(item, secondary, output))
.Timeout(TimeSpan.FromSeconds(5.0), Observable.Return(-1))
select new { item, result };
var subscription =
query
.Subscribe(x =>
{
/* do something with `x.item` and/or `x.result` */
});
This automatically iterates through each item and starts a background computation of xmldiff.Compare, but only allows each computation to take as much as 5.0 seconds before returning a default value of -1.
The subscription variable is an IDisposable, so if you want to abort the entire query before it completes just call .Dispose().
I skip it and go to the next item in the loop
By "skip it", do you mean "leave it there" or "cancel it"? The two scenarios are quite different. But for both two I suggest you use Task.
//generate 10 example tasks
var tasks = Enumerable
.Range(0, 10)
.Select(n => new Task(() => DoSomething(n)))
.ToList();
var maxExecutionTime = TimeSpan.FromSeconds(5);
foreach (var task in tasks)
{
if (task.Wait(maxExecutionTime))
{
//the task is finished in time
}
else
{
// the task is over time
// just leave it there
// the loop continues
// if you want to cancel it, see
// http://stackoverflow.com/questions/4783865/how-do-i-abort-cancel-tpl-tasks
}
}
One thing to improve is "do you really need to run your tasks one by one?" If they are independent you can run them in parallel.
Here is my Interval definition:
m_interval = Observable.Interval(TimeSpan.FromSeconds(5), m_schedulerProvider.EventLoop)
.ObserveOn(m_schedulerProvider.EventLoop)
.Select(l => Observable.FromAsync(DoWork))
.Concat()
.Subscribe();
In the code above, I feed the IScheduler in both Interval & ObserveOn from a SchedulerProvider so that I can unit test faster (TestScheduler.AdvanceBy). Also, DoWork is an async method.
In my particular case, I want the DoWork function to be called every 5 seconds. The issue here is that I want the 5 seconds to be the time between the end of DoWork and the start of the other. So if DoWork takes more than 5 seconds to execute, let's say 10 seconds, the first call would be at 5 seconds and the second call at 15 seconds.
Unfortunately, the following test proves it does not behave like that:
[Fact]
public void MultiPluginStatusHelperShouldWaitForNextQuery()
{
m_queryHelperMock
.Setup(x => x.CustomQueryAsync())
.Callback(() => Thread.Sleep(10000))
.Returns(Task.FromResult(new QueryCompletedEventData()))
.Verifiable()
;
var multiPluginStatusHelper = m_container.GetInstance<IMultiPluginStatusHelper>();
multiPluginStatusHelper.MillisecondsInterval = 5000;
m_testSchedulerProvider.EventLoopScheduler.AdvanceBy(TimeSpan.FromMilliseconds(5000).Ticks);
m_testSchedulerProvider.EventLoopScheduler.AdvanceBy(TimeSpan.FromMilliseconds(5000).Ticks);
m_queryHelperMock.Verify(x => x.CustomQueryAsync(), Times.Once);
}
The DoWork calls the CustomQueryAsync and the test fails saying that is was called twice. It should only be called once because of the delay forced with .Callback(() => Thread.Sleep(1000)).
What am I doing wrong here ?
My actual implementation comes from this example.
This problem comes up a lot, usually when polling some non-observable data source. When I come across it, I use a RepeatAfterDelay operator I wrote a while back:
public static IObservable<T> RepeatAfterDelay<T>(this IObservable<T> source, TimeSpan delay, IScheduler scheduler)
{
var repeatSignal = Observable
.Empty<T>()
.Delay(delay, scheduler);
// when source finishes, wait for the specified
// delay, then repeat.
return source.Concat(repeatSignal).Repeat();
}
And this is how I use it:
// do first set of work immediately, and then every 5 seconds do it again
m_interval = Observable
.FromAsync(DoWork)
.RepeatAfterDelay(TimeSpan.FromSeconds(5), scheduler)
.Subscribe();
// wait 5 seconds, then do first set of work, then again every 5 seconds
m_interval = Observable
.Timer(TimeSpan.FromSeconds(5), scheduler)
.SelectMany(_ => Observable
.FromAsync(DoWork)
.RepeatAfterDelay(TimeSpan.FromSeconds(5), scheduler))
.Subscribe();
Your problem is that your code is mixing lazy (Observable) and non-lazy (Task) constructs. While your first Task is executing the Interval will fire again and create a new task in the Select operator. If you want to avoid this behavior you need to wrap your Observable into a Defer block:
m_interval = Observable.Interval(TimeSpan.FromSeconds(5), m_schedulerProvider.EventLoop)
.ObserveOn(m_schedulerProvider.EventLoop)
//I think `Defer` implicitly wraps Tasks, if not wrap it in `FromAsync` Again
.Select(l => Observable.Defer(() => DoWork()))
.Concat()
.Subscribe();
The result of this is that each Observable will only execute the deferred Task when it is subscribed to, i.e. when the previous completes.
Notably this does have a problem if your producer is producing much faster than you can consume, it will begin to pile up and each your memory. As an alternative I would propose using this GenerateAsync implementation:
public static IObservable<TOut> GenerateAsync<TResult, TOut>(
Func<Task<TResult>> initialState,
Func<TResult, bool> condition,
Func<TResult, Task<TResult>> iterate,
Func<TResult, TimeSpan> timeSelector,
Func<TResult, TOut> resultSelector,
IScheduler scheduler = null)
{
var s = scheduler ?? Scheduler.Default;
return Observable.Create<TOut>(async obs => {
//You have to do your initial time delay here.
var init = await initialState();
return s.Schedule(init, timeSelector(init), async (state, recurse) =>
{
//Check if we are done
if (!condition(state))
{
obs.OnCompleted();
return;
}
//Process the result
obs.OnNext(resultSelector(state));
//Initiate the next request
state = await iterate(state);
//Recursively schedule again
recurse(state, timeSelector(state));
});
});
}
GenerateAsync(DoWork /*Initial state*/,
_ => true /*Forever*/,
_ => DoWork() /*Do your async task*/,
_ => TimeSpan.FromSeconds(5) /*Delay between events*/,
_ => _ /*Any transformations*/,
scheduler)
.Subscribe();
The above removes the issue of producer/consumer races, by not scheduling the next event until after the first one is done.
While #Brandon's solution is nice and clean I discovered that it blocks a thread to wait for the delay timer. Non-blocking alternative can look something like:
public static IObservable<T> DelayRepeat<T>(this IObservable<T> source, TimeSpan delay) =>
source
.Concat(
Observable.Create<T>(async observer =>
{
await Task.Delay(delay);
observer.OnCompleted();
}))
.Repeat();
I'm trying to use Reactive Extensions (Rx) for a task where it seems to be a good fit, polling at a specific interval a web service and display its last x results.
I have a web service that sends me the status of an instrument I want to monitor. I would like to poll this instrument at a specific rate and display in a list the last 20 status that have been polled.
So my list would be like a "moving window" of the service result.
I'm developing a WPF app with Caliburn.Micro, but I don't think this is very relevant.
What I managed to get until now is the following (just a sample app that I hacked quickly, I'm not going to do this in the ShellViewModel in the real app):
public class ShellViewModel : Caliburn.Micro.PropertyChangedBase, IShell
{
private ObservableCollection<string> times;
private string currentTime;
public ShellViewModel()
{
times = new ObservableCollection<string>();
Observable
.Interval(TimeSpan.FromSeconds(1))
.SelectMany(x => this.GetCurrentDate().ToObservable())
.ObserveOnDispatcher()
.Subscribe(x =>
{
this.CurrentTime = x;
this.times.Add(x);
});
}
public IEnumerable<string> Times
{
get
{
return this.times;
}
}
public string CurrentTime
{
get
{
return this.currentTime;
}
set
{
this.currentTime = value;
this.NotifyOfPropertyChange(() => this.CurrentTime);
}
}
private async Task<string> GetCurrentDate()
{
var client = new RestClient("http://www.timeapi.org");
var request = new RestRequest("/utc/now.json");
var response = await client.ExecuteGetTaskAsync(request);
return response.Content;
}
}
In the view I have just a label bound to the CurrentTime property and a list bound to the Times property.
The issue I have is:
It's not limited to the 20 items in the list as I always add items to the ObservableCollection but I can't find a better way to databind
The Interval doesn't work as I'd like. If the querying takes more than 1 second to run, two queries will be run in parallel, which I'd like not to happen. My goal would be that the query repeats indefinitely but at a pace of no more than 1 query every seconds. If a query makes more than 1 second to end, it should wait for it to have finish and directly trigger the new query.
Second edit:
Previous edit below was me being stupid and very confused, it triggers events continuously because Interval is something continuous that never ends. Brandon's solution is correct and works as expected.
Edit:
Based on Brandon's example, I tried to do the following code in LinqPad:
Observable
.Merge(Observable.Interval(TimeSpan.FromSeconds(2)), Observable.Interval(TimeSpan.FromSeconds(10)))
.Repeat()
.Scan(new List<double>(), (list, item) => { list.Add(item); return list; })
.Subscribe(x => Console.Out.WriteLine(x))
And I can see that the write to the console occurs every 2 seconds, and not every 10. So the Repeat doesn't wait for both Observable to be finished before repeating.
Try this:
// timer that completes after 1 second
var intervalTimer = Observable
.Empty<string>()
.Delay(TimeSpan.FromSeconds(1));
// queries one time whenever subscribed
var query = Observable.FromAsync(GetCurrentDate);
// query + interval timer which completes
// only after both the query and the timer
// have expired
var intervalQuery = Observable.Merge(query, intervalTimer);
// Re-issue the query whenever intervalQuery completes
var queryLoop = intervalQuery.Repeat();
// Keep the 20 most recent results
// Note. Use an immutable list for this
// https://www.nuget.org/packages/microsoft.bcl.immutable
// otherwise you will have problems with
// the list changing while an observer
// is still observing it.
var recentResults = queryLoop.Scan(
ImmutableList.Create<string>(), // starts off empty
(acc, item) =>
{
acc = acc.Add(item);
if (acc.Count > 20)
{
acc = acc.RemoveAt(0);
}
return acc;
});
// store the results
recentResults
.ObserveOnDispatcher()
.Subscribe(items =>
{
this.CurrentTime = items[0];
this.RecentItems = items;
});
This should skip the interval messages while a GetCurrentDate is in Progress.
Observable
.Interval(TimeSpan.FromSeconds(1))
.GroupByUntil(p => 1,p => GetCurrentDate().ToObservable().Do(x => {
this.CurrentTime = x;
this.times.Add(x);
}))
.SelectMany(p => p.LastAsync())
.Subscribe();
I would like to set up an Rx subscription that can respond to an event right away, and then ignore subsequent events that happen within a specified "cooldown" period.
The out of the box Throttle/Buffer methods respond only once the timeout has elapsed, which is not quite what I need.
Here is some code that sets up the scenario, and uses a Throttle (which isn't the solution I want):
class Program
{
static Stopwatch sw = new Stopwatch();
static void Main(string[] args)
{
var subject = new Subject<int>();
var timeout = TimeSpan.FromMilliseconds(500);
subject
.Throttle(timeout)
.Subscribe(DoStuff);
var factory = new TaskFactory();
sw.Start();
factory.StartNew(() =>
{
Console.WriteLine("Batch 1 (no delay)");
subject.OnNext(1);
});
factory.StartNewDelayed(1000, () =>
{
Console.WriteLine("Batch 2 (1s delay)");
subject.OnNext(2);
});
factory.StartNewDelayed(1300, () =>
{
Console.WriteLine("Batch 3 (1.3s delay)");
subject.OnNext(3);
});
factory.StartNewDelayed(1600, () =>
{
Console.WriteLine("Batch 4 (1.6s delay)");
subject.OnNext(4);
});
Console.ReadKey();
sw.Stop();
}
private static void DoStuff(int i)
{
Console.WriteLine("Handling {0} at {1}ms", i, sw.ElapsedMilliseconds);
}
}
The output of running this right now is:
Batch 1 (no delay)
Handling 1 at 508ms
Batch 2 (1s delay)
Batch 3 (1.3s delay)
Batch 4 (1.6s delay)
Handling 4 at 2114ms
Note that batch 2 isn't handled (which is fine!) because we wait for 500ms to elapse between requests due to the nature of throttle. Batch 3 is also not handled, (which is less alright because it happened more than 500ms from batch 2) due to its proximity to Batch 4.
What I'm looking for is something more like this:
Batch 1 (no delay)
Handling 1 at ~0ms
Batch 2 (1s delay)
Handling 2 at ~1000s
Batch 3 (1.3s delay)
Batch 4 (1.6s delay)
Handling 4 at ~1600s
Note that batch 3 wouldn't be handled in this scenario (which is fine!) because it occurs within 500ms of Batch 2.
EDIT:
Here is the implementation for the "StartNewDelayed" extension method that I use:
/// <summary>Creates a Task that will complete after the specified delay.</summary>
/// <param name="factory">The TaskFactory.</param>
/// <param name="millisecondsDelay">The delay after which the Task should transition to RanToCompletion.</param>
/// <returns>A Task that will be completed after the specified duration.</returns>
public static Task StartNewDelayed(
this TaskFactory factory, int millisecondsDelay)
{
return StartNewDelayed(factory, millisecondsDelay, CancellationToken.None);
}
/// <summary>Creates a Task that will complete after the specified delay.</summary>
/// <param name="factory">The TaskFactory.</param>
/// <param name="millisecondsDelay">The delay after which the Task should transition to RanToCompletion.</param>
/// <param name="cancellationToken">The cancellation token that can be used to cancel the timed task.</param>
/// <returns>A Task that will be completed after the specified duration and that's cancelable with the specified token.</returns>
public static Task StartNewDelayed(this TaskFactory factory, int millisecondsDelay, CancellationToken cancellationToken)
{
// Validate arguments
if (factory == null) throw new ArgumentNullException("factory");
if (millisecondsDelay < 0) throw new ArgumentOutOfRangeException("millisecondsDelay");
// Create the timed task
var tcs = new TaskCompletionSource<object>(factory.CreationOptions);
var ctr = default(CancellationTokenRegistration);
// Create the timer but don't start it yet. If we start it now,
// it might fire before ctr has been set to the right registration.
var timer = new Timer(self =>
{
// Clean up both the cancellation token and the timer, and try to transition to completed
ctr.Dispose();
((Timer)self).Dispose();
tcs.TrySetResult(null);
});
// Register with the cancellation token.
if (cancellationToken.CanBeCanceled)
{
// When cancellation occurs, cancel the timer and try to transition to cancelled.
// There could be a race, but it's benign.
ctr = cancellationToken.Register(() =>
{
timer.Dispose();
tcs.TrySetCanceled();
});
}
if (millisecondsDelay > 0)
{
// Start the timer and hand back the task...
timer.Change(millisecondsDelay, Timeout.Infinite);
}
else
{
// Just complete the task, and keep execution on the current thread.
ctr.Dispose();
tcs.TrySetResult(null);
timer.Dispose();
}
return tcs.Task;
}
Here's my approach. It's similar to others that have gone before, but it doesn't suffer the over-zealous window production problem.
The desired function works a lot like Observable.Throttle but emits qualifying events as soon as they arrive rather than delaying for the duration of the throttle or sample period. For a given duration after a qualifying event, subsequent events are suppressed.
Given as a testable extension method:
public static class ObservableExtensions
{
public static IObservable<T> SampleFirst<T>(
this IObservable<T> source,
TimeSpan sampleDuration,
IScheduler scheduler = null)
{
scheduler = scheduler ?? Scheduler.Default;
return source.Publish(ps =>
ps.Window(() => ps.Delay(sampleDuration,scheduler))
.SelectMany(x => x.Take(1)));
}
}
The idea is to use the overload of Window that creates non-overlapping windows using a windowClosingSelector that uses the source time-shifted back by the sampleDuration. Each window will therefore: (a) be closed by the first element in it and (b) remain open until a new element is permitted. We then simply select the first element from each window.
Rx 1.x Version
The Publish extension method used above is not available in Rx 1.x. Here is an alternative:
public static class ObservableExtensions
{
public static IObservable<T> SampleFirst<T>(
this IObservable<T> source,
TimeSpan sampleDuration,
IScheduler scheduler = null)
{
scheduler = scheduler ?? Scheduler.Default;
var sourcePub = source.Publish().RefCount();
return sourcePub.Window(() => sourcePub.Delay(sampleDuration,scheduler))
.SelectMany(x => x.Take(1));
}
}
The solution I found after a lot of trial and error was to replace the throttled subscription with the following:
subject
.Window(() => { return Observable.Interval(timeout); })
.SelectMany(x => x.Take(1))
.Subscribe(i => DoStuff(i));
Edited to incorporate Paul's clean-up.
Awesome solution Andrew! We can take this a step further though and clean up the inner Subscribe:
subject
.Window(() => { return Observable.Interval(timeout); })
.SelectMany(x => x.Take(1))
.Subscribe(DoStuff);
The initial answer I posted has a flaw: namely that the Window method, when used with an Observable.Interval to denote the end of the window, sets up an infinite series of 500ms windows. What I really need is a window that starts when the first result is pumped into the subject, and ends after the 500ms.
My sample data masked this problem because the data broke down nicely into the windows that were already going to be created. (i.e. 0-500ms, 501-1000ms, 1001-1500ms, etc.)
Consider instead this timing:
factory.StartNewDelayed(300,() =>
{
Console.WriteLine("Batch 1 (300ms delay)");
subject.OnNext(1);
});
factory.StartNewDelayed(700, () =>
{
Console.WriteLine("Batch 2 (700ms delay)");
subject.OnNext(2);
});
factory.StartNewDelayed(1300, () =>
{
Console.WriteLine("Batch 3 (1.3s delay)");
subject.OnNext(3);
});
factory.StartNewDelayed(1600, () =>
{
Console.WriteLine("Batch 4 (1.6s delay)");
subject.OnNext(4);
});
What I get is:
Batch 1 (300ms delay)
Handling 1 at 356ms
Batch 2 (700ms delay)
Handling 2 at 750ms
Batch 3 (1.3s delay)
Handling 3 at 1346ms
Batch 4 (1.6s delay)
Handling 4 at 1644ms
This is because the windows begin at 0ms, 500ms, 1000ms, and 1500ms and so each Subject.OnNext fits nicely into its own window.
What I want is:
Batch 1 (300ms delay)
Handling 1 at ~300ms
Batch 2 (700ms delay)
Batch 3 (1.3s delay)
Handling 3 at ~1300ms
Batch 4 (1.6s delay)
After a lot of struggling and an hour banging on it with a co-worker, we arrived at a better solution using pure Rx and a single local variable:
bool isCoolingDown = false;
subject
.Where(_ => !isCoolingDown)
.Subscribe(
i =>
{
DoStuff(i);
isCoolingDown = true;
Observable
.Interval(cooldownInterval)
.Take(1)
.Subscribe(_ => isCoolingDown = false);
});
Our assumption is that calls to the subscription method are synchronized. If they are not, then a simple lock could be introduced.
Use .Scan() !
This is what I use for Throttling when I need the first hit (after a certain period) immediately, but delay (and group/ignore) any subsequent hits.
Basically works like Throttle, but fires immediately if the previous onNext was >= interval ago, otherwise, schedule it at exactly interval from the previous hit. And of course, if within the 'cooling down' period multiple hits come, the additional ones are ignored, just like Throttle does.
The difference with your use case is that if you get an event at 0 ms and 100 ms, they will both be handled (at 0ms and 500ms), which might be what you actually want (otherwise, the accumulator is easy to adapt to ignore ANY hit closer than interval to the previous one).
public static IObservable<T> QuickThrottle<T>(this IObservable<T> src, TimeSpan interval, IScheduler scheduler)
{
return src
.Scan(new ValueAndDueTime<T>(), (prev, id) => AccumulateForQuickThrottle(prev, id, interval, scheduler))
.Where(vd => !vd.Ignore)
.SelectMany(sc => Observable.Timer(sc.DueTime, scheduler).Select(_ => sc.Value));
}
private static ValueAndDueTime<T> AccumulateForQuickThrottle<T>(ValueAndDueTime<T> prev, T value, TimeSpan interval, IScheduler s)
{
var now = s.Now;
// Ignore this completely if there is already a future item scheduled
// but do keep the dueTime for accumulation!
if (prev.DueTime > now) return new ValueAndDueTime<T> { DueTime = prev.DueTime, Ignore = true };
// Schedule this item at at least interval from the previous
var min = prev.DueTime + interval;
var nextTime = (now < min) ? min : now;
return new ValueAndDueTime<T> { DueTime = nextTime, Value = value };
}
private class ValueAndDueTime<T>
{
public DateTimeOffset DueTime;
public T Value;
public bool Ignore;
}
I got another one for your. This one doesn't use Repeat() nor Interval() so it might be what you are after:
subject
.Window(() => Observable.Timer(TimeSpan.FromMilliseconds(500)))
.SelectMany(x => x.Take(1));
Well the most obvious thing will be to use Repeat() here. However, as far as I know Repeat() might introduce problems so that notifications disappear in between the moment when the stream stops and we subscribe again. In practice this has never been a problem for me.
subject
.Take(1)
.Concat(Observable.Empty<long>().Delay(TimeSpan.FromMilliseconds(500)))
.Repeat();
Remember to replace with the actual type of your source.
UPDATE:
Updated query to use Concat instead of Merge
I have stumbled upon this question while trying to re-implement my own solution to the same or similar problem using .Window
Take a look, it seems to be the same as this one and solved quite elegantly:
https://stackoverflow.com/a/3224723/58463
It's an old post, but no answer could really fill my needs, so I'm giving my own solution :
public static IObservable<T> ThrottleOrImmediate<T>(this IObservable<T> source, TimeSpan delay, IScheduler scheduler)
{
return Observable.Create<T>((obs, token) =>
{
// Next item cannot be send before that time
DateTime nextItemTime = default;
return Task.FromResult(source.Subscribe(async item =>
{
var currentTime = DateTime.Now;
// If we already reach the next item time
if (currentTime - nextItemTime >= TimeSpan.Zero)
{
// Following item will be send only after the set delay
nextItemTime = currentTime + delay;
// send current item with scheduler
scheduler.Schedule(() => obs.OnNext(item));
}
// There is still time before we can send an item
else
{
// we schedule the time for the following item
nextItemTime = currentTime + delay;
try
{
await Task.Delay(delay, token);
}
catch (TaskCanceledException)
{
return;
}
// If next item schedule was change by another item then we stop here
if (nextItemTime > currentTime + delay)
return;
else
{
// Set next possible time for an item and send item with scheduler
nextItemTime = currentTime + delay;
scheduler.Schedule(() => obs.OnNext(item));
}
}
}));
});
}
First item is immediately sent, then following items are throttled. Then if a following item is sent after the delayed time, it's immediately sent too.