So I have this Task:
public async Task ProcessNewMessages()
{
while (true)
{
await _exceptionHandler.WithExceptionHandling(async () =>
{
List<Message> newUncompletedNewMessages = await _newUncompletedMessagesJob.Execute();
// Do stuff
}
Thread.Sleep(30000);
}
}
The Execute returns a list of 10 messages and I want to add a while loop in the while true that runs the whole code again as long as the Execute returns me a full list of 10 items, that runs straight after the items are done rather than waiting 30 seconds every time.
The 30 seconds will be in place to keep checking if there are new messages and is ireelevant to the Task of handling them.
So to rephrase my question: How do I rerun code based on the list returned by Execute() having 10 messages?
public async Task ProcessNewMessages()
{
while (true)
{
IEnumerable<MessageToSend> uncompletedNewMessages = new List<MessageToSend>();
do
{
await _exceptionHandler.WithExceptionHandling(async () =>
{
uncompletedNewMessages = await _newUncompletedMessagesJob.Execute();
// DoStuff
});
} while (uncompletedNewMessages.Count() == 10)
await Task.Delay(30000);
}
}
I have an answer.
I read people not liking the Thread.Sleep, but this is a webjob running continuously to check for new messages. Now this doesn't need to run every second, because there might not be new message every second, but they can keep on coming in throughout the day. Let me know your suggestions how to do that differently.
Edit: Thanks for the explanation Jeroen
.NET 6 introduced the PeriodicTimer.
I need to do something every minute, at the top of the minute. For example: 09:23:00, 09:24:00, 09:25:00, ...
But with a one minute period - new PeriodicTimer(TimeSpan.FromMinutes(1)) - and starting at 09:23:45, I will get "ticks" at: 09:24:45, 09:25:45, 09:26:45, ...
So it's dependent on the start time.
My workaround is a one-second period, and a check that the current time has seconds equal to 0. Another workaround is to wait for the next minute and then start the timer. Both approaches work but are fiddly and use too fine a resolution.
Is there a built-in or better way to trigger at the top of the minute rather than one-minute-after-start?
AFAIK there is nothing like this available in the standard .NET libraries. And I don't think that it's likely to be added any time soon. My suggestion is to use the third party Cronos library, that does a good job at calculating time intervals¹. You can find a usage example here, by Stephen Cleary. What this library does is to take a DateTime and a Cron expression as input, and calculate the next DateTime that satisfies this expression. It is just a DateTime calculator, not a scheduler.
If you want to get fancy you could include the functionality of the Cronos library in a custom PeriodicTimer-like component, like the one below:
using Cronos;
public sealed class CronosPeriodicTimer : IDisposable
{
private readonly CronExpression _cronExpression; // Also used as the locker
private PeriodicTimer _activeTimer;
private bool _disposed;
private static readonly TimeSpan _minDelay = TimeSpan.FromMilliseconds(500);
public CronosPeriodicTimer(string expression, CronFormat format)
{
_cronExpression = CronExpression.Parse(expression, format);
}
public async ValueTask<bool> WaitForNextTickAsync(
CancellationToken cancellationToken = default)
{
cancellationToken.ThrowIfCancellationRequested();
PeriodicTimer timer;
lock (_cronExpression)
{
if (_disposed) return false;
if (_activeTimer is not null)
throw new InvalidOperationException("One consumer at a time.");
DateTime utcNow = DateTime.UtcNow;
DateTime? utcNext = _cronExpression.GetNextOccurrence(utcNow + _minDelay);
if (utcNext is null)
throw new InvalidOperationException("Unreachable date.");
TimeSpan delay = utcNext.Value - utcNow;
Debug.Assert(delay > _minDelay);
timer = _activeTimer = new(delay);
}
try
{
// Dispose the timer after the first tick.
using (timer)
return await timer.WaitForNextTickAsync(cancellationToken)
.ConfigureAwait(false);
}
finally { Volatile.Write(ref _activeTimer, null); }
}
public void Dispose()
{
PeriodicTimer activeTimer;
lock (_cronExpression)
{
if (_disposed) return;
_disposed = true;
activeTimer = _activeTimer;
}
activeTimer?.Dispose();
}
}
Apart from the constructor, the CronosPeriodicTimer class has identical API and behavior with the PeriodicTimer class. You could use it like this:
var timer = new CronosPeriodicTimer("0 * * * * *", CronFormat.IncludeSeconds);
//...
await timer.WaitForNextTickAsync();
The expression 0 * * * * * means "on the 0 (zero) second of every minute, of every hour, of every day of the month, of every month, and of every day of the week."
You can find detailed documentation about the format of the Cron expressions here.
The 500 milliseconds _minDelay has the intention to prevent the remote possibility of the timer ticking twice by mistake. Also because the PeriodicTimer class has a minimum period of 1 millisecond.
For an implementation that uses the Task.Delay method instead of the PeriodicTimer class, and so it can be used by .NET versions previous than 6.0, you can look at the 3rd revision of this answer.
¹ With the caveat that the Cronos library is currently capped to the year 2099 (version 0.7.1).
For completeness, here are the workarounds mentioned in my question.
Tick every second, and wait for top-of-the-minute:
var timer = new PeriodicTimer(TimeSpan.FromSeconds(1));
while (await timer.WaitForNextTickAsync(cancellationToken)) {
if (DateTime.UtcNow.Second == 0)
await DoSomething();
}
Delay till top-of-the-minute, then tick every minute:
var delay = (60 - DateTime.UtcNow.Second) * 1000; // take milliseconds into account to improve start-time accuracy
await Task.Delay(delay);
var timer = new PeriodicTimer(TimeSpan.FromMinutes(1));
while (await timer.WaitForNextTickAsync(cancellationToken)) {
await DoSomething();
}
Some notes about accuracy:
This will never be perfectly accurate, as discussed in the comments above. But it's good enough in many cases.
Small clock drifts will occur and make the per-minute timer more and more inaccurate, and this will only be fixed when the server is restarted. But the per-second timer "self-corrects" on every tick, so although it's "heavier", it will be more accurate over time.
The per-second timer can sometimes lose a tick or get a double tick (see comments above). The per-minute timer won't lose ticks, but they may be inaccurate. Whether this matters is something for you to consider.
I'm trying to write my own scheduler; the rationale behind it is that all the submitted actions will be executed in order, according to a delay. For example, if at time 0 I schedule action A with delay 5 and at time 1 I schedule action B with delay 2, then B should be executed first at time 3 and A should be executed second, at time 5.
Basically, what I am trying to do is something like:
public class MyScheduler
{
Task _task = new Task(() => { });
public MyScheduler()
{
_task.Start();
}
public void Schedule(Action action, long delay)
{
Task.Delay(TimeSpan.FromTicks(delay)).ContinueWith(_ =>
lock(_task) {
_task = _task.ContinueWith(task => action())
}
);
}
}
A relevant test for this code would be:
var waiter = new Waiter(3);
int _count = 0;
mysched = new MyScheduler();
mysched.Schedule(() => { _count++; waiter.Signal(); });
mysched.Schedule(() => { Task.Delay(100).Wait(); _count *= 3; waiter.Signal(); });
mysched.Schedule(() => { _count++; waiter.Signal(); });
waiter.Await();
Assert.AreEqual(4, _count);
In the above code, Waiter is a class with an internal variable initialized in the constructor; the Signal method decrements that internal variable and the Await method loops (and sleeps 10 ms on each iteration) until the internal variable is less than or equal to zero.
The aim of the test is to show that the scheduled actions have been performed in order.
Most of the times this is true and the test passes, but on few occasions the resulting value for _count is 2 instead of 4. I have spent a lot of time trying to figure out why this happens, but I can't seem to figure it out and my lack of experience with C# is not helping either.
Does anyone have any suggestions?
For one thing, _count is not synchronized for access from different threads.
I recommend that you not use ContinueWith at all; it is a very low-level method and is very easy to get the details wrong (for example, the default scheduler is TaskScheduler.Current, which is almost never what you want). Your general logic code should use await instead of ContinueWith.
Regarding the scheduler, these days it is almost impossible to make a good use case for developing your own. There are better ones available that are developed by geniuses and extremely well-tested. Consider Reactive Extensions: they provide several schedulers, and they all support scheduling.
I am building a windows store app using C# and xaml. I need to refresh the data after certain interval of time (bring the new data from the server). I used ThreadPoolTimer to execute my refresh function periodically as follows:
TimeSpan period = TimeSpan.FromMinutes(15);
ThreadPoolTimer PeriodicTimer = ThreadPoolTimer.CreatePeriodicTimer(async(source)=> {
n++;
Debug.WriteLine("hello" + n);
await dp.RefreshAsync(); //Function to refresh the data
await Dispatcher.RunAsync(CoreDispatcherPriority.High,
() =>
{
bv.Text = "timer thread" + n;
});
}, period);
This is working properly. The only problem is that what if refresh function doesnot complete before its next instance is submitted to the thread pool. Is there some way to specify the gap between its execution.
Step 1 : Refresh function executes (takes any amount of time)
Step 2 : Refresh function completes its execution
Step 3 : Gap for 15mins then go to Step 1
Refresh function executes. 15mins after its execution ends, it executes again.
The AutoResetEvent will solve this problem. Declare a class-level AutoResetEvent instance.
AutoResetEvent _refreshWaiter = new AutoResetEvent(true);
Then inside your code: 1. wait on it till it is signaled, and 2. pass its reference as an argument to RefreshAsync method.
TimeSpan period = TimeSpan.FromMinutes(15);
ThreadPoolTimer PeriodicTimer = ThreadPoolTimer.CreatePeriodicTimer(async(source)=> {
// 1. wait till signaled. execution will block here till _refreshWaiter.Set() is called.
_refreshWaiter.WaitOne();
n++;
Debug.WriteLine("hello" + n);
// 2. pass _refreshWaiter reference as an argument
await dp.RefreshAsync(_refreshWaiter); //Function to refresh the data
await Dispatcher.RunAsync(CoreDispatcherPriority.High,
() =>
{
bv.Text = "timer thread" + n;
});
}, period);
Finally, at the end of dp.RefreshAsync method, call _refreshWaiter.Set(); so that if 15 seconds have passed then the next RefreshAsync may be called. Note that if RefreshAsync method takes less than 15 minutes, the execution proceeds as normal.
I think an easier way to do this is with async:
private async Task PeriodicallyRefreshDataAsync(TimeSpan period)
{
while (true)
{
n++;
Debug.WriteLine("hello" + n);
await dp.RefreshAsync(); //Function to refresh the data
bv.Text = "timer thread" + n;
await Task.Delay(period);
}
}
TimeSpan period = TimeSpan.FromMinutes(15);
Task refreshTask = PeriodicallyRefreshDataAsync(period);
This solution also provides a Task which can be used to detect errors.
I would like to set up an Rx subscription that can respond to an event right away, and then ignore subsequent events that happen within a specified "cooldown" period.
The out of the box Throttle/Buffer methods respond only once the timeout has elapsed, which is not quite what I need.
Here is some code that sets up the scenario, and uses a Throttle (which isn't the solution I want):
class Program
{
static Stopwatch sw = new Stopwatch();
static void Main(string[] args)
{
var subject = new Subject<int>();
var timeout = TimeSpan.FromMilliseconds(500);
subject
.Throttle(timeout)
.Subscribe(DoStuff);
var factory = new TaskFactory();
sw.Start();
factory.StartNew(() =>
{
Console.WriteLine("Batch 1 (no delay)");
subject.OnNext(1);
});
factory.StartNewDelayed(1000, () =>
{
Console.WriteLine("Batch 2 (1s delay)");
subject.OnNext(2);
});
factory.StartNewDelayed(1300, () =>
{
Console.WriteLine("Batch 3 (1.3s delay)");
subject.OnNext(3);
});
factory.StartNewDelayed(1600, () =>
{
Console.WriteLine("Batch 4 (1.6s delay)");
subject.OnNext(4);
});
Console.ReadKey();
sw.Stop();
}
private static void DoStuff(int i)
{
Console.WriteLine("Handling {0} at {1}ms", i, sw.ElapsedMilliseconds);
}
}
The output of running this right now is:
Batch 1 (no delay)
Handling 1 at 508ms
Batch 2 (1s delay)
Batch 3 (1.3s delay)
Batch 4 (1.6s delay)
Handling 4 at 2114ms
Note that batch 2 isn't handled (which is fine!) because we wait for 500ms to elapse between requests due to the nature of throttle. Batch 3 is also not handled, (which is less alright because it happened more than 500ms from batch 2) due to its proximity to Batch 4.
What I'm looking for is something more like this:
Batch 1 (no delay)
Handling 1 at ~0ms
Batch 2 (1s delay)
Handling 2 at ~1000s
Batch 3 (1.3s delay)
Batch 4 (1.6s delay)
Handling 4 at ~1600s
Note that batch 3 wouldn't be handled in this scenario (which is fine!) because it occurs within 500ms of Batch 2.
EDIT:
Here is the implementation for the "StartNewDelayed" extension method that I use:
/// <summary>Creates a Task that will complete after the specified delay.</summary>
/// <param name="factory">The TaskFactory.</param>
/// <param name="millisecondsDelay">The delay after which the Task should transition to RanToCompletion.</param>
/// <returns>A Task that will be completed after the specified duration.</returns>
public static Task StartNewDelayed(
this TaskFactory factory, int millisecondsDelay)
{
return StartNewDelayed(factory, millisecondsDelay, CancellationToken.None);
}
/// <summary>Creates a Task that will complete after the specified delay.</summary>
/// <param name="factory">The TaskFactory.</param>
/// <param name="millisecondsDelay">The delay after which the Task should transition to RanToCompletion.</param>
/// <param name="cancellationToken">The cancellation token that can be used to cancel the timed task.</param>
/// <returns>A Task that will be completed after the specified duration and that's cancelable with the specified token.</returns>
public static Task StartNewDelayed(this TaskFactory factory, int millisecondsDelay, CancellationToken cancellationToken)
{
// Validate arguments
if (factory == null) throw new ArgumentNullException("factory");
if (millisecondsDelay < 0) throw new ArgumentOutOfRangeException("millisecondsDelay");
// Create the timed task
var tcs = new TaskCompletionSource<object>(factory.CreationOptions);
var ctr = default(CancellationTokenRegistration);
// Create the timer but don't start it yet. If we start it now,
// it might fire before ctr has been set to the right registration.
var timer = new Timer(self =>
{
// Clean up both the cancellation token and the timer, and try to transition to completed
ctr.Dispose();
((Timer)self).Dispose();
tcs.TrySetResult(null);
});
// Register with the cancellation token.
if (cancellationToken.CanBeCanceled)
{
// When cancellation occurs, cancel the timer and try to transition to cancelled.
// There could be a race, but it's benign.
ctr = cancellationToken.Register(() =>
{
timer.Dispose();
tcs.TrySetCanceled();
});
}
if (millisecondsDelay > 0)
{
// Start the timer and hand back the task...
timer.Change(millisecondsDelay, Timeout.Infinite);
}
else
{
// Just complete the task, and keep execution on the current thread.
ctr.Dispose();
tcs.TrySetResult(null);
timer.Dispose();
}
return tcs.Task;
}
Here's my approach. It's similar to others that have gone before, but it doesn't suffer the over-zealous window production problem.
The desired function works a lot like Observable.Throttle but emits qualifying events as soon as they arrive rather than delaying for the duration of the throttle or sample period. For a given duration after a qualifying event, subsequent events are suppressed.
Given as a testable extension method:
public static class ObservableExtensions
{
public static IObservable<T> SampleFirst<T>(
this IObservable<T> source,
TimeSpan sampleDuration,
IScheduler scheduler = null)
{
scheduler = scheduler ?? Scheduler.Default;
return source.Publish(ps =>
ps.Window(() => ps.Delay(sampleDuration,scheduler))
.SelectMany(x => x.Take(1)));
}
}
The idea is to use the overload of Window that creates non-overlapping windows using a windowClosingSelector that uses the source time-shifted back by the sampleDuration. Each window will therefore: (a) be closed by the first element in it and (b) remain open until a new element is permitted. We then simply select the first element from each window.
Rx 1.x Version
The Publish extension method used above is not available in Rx 1.x. Here is an alternative:
public static class ObservableExtensions
{
public static IObservable<T> SampleFirst<T>(
this IObservable<T> source,
TimeSpan sampleDuration,
IScheduler scheduler = null)
{
scheduler = scheduler ?? Scheduler.Default;
var sourcePub = source.Publish().RefCount();
return sourcePub.Window(() => sourcePub.Delay(sampleDuration,scheduler))
.SelectMany(x => x.Take(1));
}
}
The solution I found after a lot of trial and error was to replace the throttled subscription with the following:
subject
.Window(() => { return Observable.Interval(timeout); })
.SelectMany(x => x.Take(1))
.Subscribe(i => DoStuff(i));
Edited to incorporate Paul's clean-up.
Awesome solution Andrew! We can take this a step further though and clean up the inner Subscribe:
subject
.Window(() => { return Observable.Interval(timeout); })
.SelectMany(x => x.Take(1))
.Subscribe(DoStuff);
The initial answer I posted has a flaw: namely that the Window method, when used with an Observable.Interval to denote the end of the window, sets up an infinite series of 500ms windows. What I really need is a window that starts when the first result is pumped into the subject, and ends after the 500ms.
My sample data masked this problem because the data broke down nicely into the windows that were already going to be created. (i.e. 0-500ms, 501-1000ms, 1001-1500ms, etc.)
Consider instead this timing:
factory.StartNewDelayed(300,() =>
{
Console.WriteLine("Batch 1 (300ms delay)");
subject.OnNext(1);
});
factory.StartNewDelayed(700, () =>
{
Console.WriteLine("Batch 2 (700ms delay)");
subject.OnNext(2);
});
factory.StartNewDelayed(1300, () =>
{
Console.WriteLine("Batch 3 (1.3s delay)");
subject.OnNext(3);
});
factory.StartNewDelayed(1600, () =>
{
Console.WriteLine("Batch 4 (1.6s delay)");
subject.OnNext(4);
});
What I get is:
Batch 1 (300ms delay)
Handling 1 at 356ms
Batch 2 (700ms delay)
Handling 2 at 750ms
Batch 3 (1.3s delay)
Handling 3 at 1346ms
Batch 4 (1.6s delay)
Handling 4 at 1644ms
This is because the windows begin at 0ms, 500ms, 1000ms, and 1500ms and so each Subject.OnNext fits nicely into its own window.
What I want is:
Batch 1 (300ms delay)
Handling 1 at ~300ms
Batch 2 (700ms delay)
Batch 3 (1.3s delay)
Handling 3 at ~1300ms
Batch 4 (1.6s delay)
After a lot of struggling and an hour banging on it with a co-worker, we arrived at a better solution using pure Rx and a single local variable:
bool isCoolingDown = false;
subject
.Where(_ => !isCoolingDown)
.Subscribe(
i =>
{
DoStuff(i);
isCoolingDown = true;
Observable
.Interval(cooldownInterval)
.Take(1)
.Subscribe(_ => isCoolingDown = false);
});
Our assumption is that calls to the subscription method are synchronized. If they are not, then a simple lock could be introduced.
Use .Scan() !
This is what I use for Throttling when I need the first hit (after a certain period) immediately, but delay (and group/ignore) any subsequent hits.
Basically works like Throttle, but fires immediately if the previous onNext was >= interval ago, otherwise, schedule it at exactly interval from the previous hit. And of course, if within the 'cooling down' period multiple hits come, the additional ones are ignored, just like Throttle does.
The difference with your use case is that if you get an event at 0 ms and 100 ms, they will both be handled (at 0ms and 500ms), which might be what you actually want (otherwise, the accumulator is easy to adapt to ignore ANY hit closer than interval to the previous one).
public static IObservable<T> QuickThrottle<T>(this IObservable<T> src, TimeSpan interval, IScheduler scheduler)
{
return src
.Scan(new ValueAndDueTime<T>(), (prev, id) => AccumulateForQuickThrottle(prev, id, interval, scheduler))
.Where(vd => !vd.Ignore)
.SelectMany(sc => Observable.Timer(sc.DueTime, scheduler).Select(_ => sc.Value));
}
private static ValueAndDueTime<T> AccumulateForQuickThrottle<T>(ValueAndDueTime<T> prev, T value, TimeSpan interval, IScheduler s)
{
var now = s.Now;
// Ignore this completely if there is already a future item scheduled
// but do keep the dueTime for accumulation!
if (prev.DueTime > now) return new ValueAndDueTime<T> { DueTime = prev.DueTime, Ignore = true };
// Schedule this item at at least interval from the previous
var min = prev.DueTime + interval;
var nextTime = (now < min) ? min : now;
return new ValueAndDueTime<T> { DueTime = nextTime, Value = value };
}
private class ValueAndDueTime<T>
{
public DateTimeOffset DueTime;
public T Value;
public bool Ignore;
}
I got another one for your. This one doesn't use Repeat() nor Interval() so it might be what you are after:
subject
.Window(() => Observable.Timer(TimeSpan.FromMilliseconds(500)))
.SelectMany(x => x.Take(1));
Well the most obvious thing will be to use Repeat() here. However, as far as I know Repeat() might introduce problems so that notifications disappear in between the moment when the stream stops and we subscribe again. In practice this has never been a problem for me.
subject
.Take(1)
.Concat(Observable.Empty<long>().Delay(TimeSpan.FromMilliseconds(500)))
.Repeat();
Remember to replace with the actual type of your source.
UPDATE:
Updated query to use Concat instead of Merge
I have stumbled upon this question while trying to re-implement my own solution to the same or similar problem using .Window
Take a look, it seems to be the same as this one and solved quite elegantly:
https://stackoverflow.com/a/3224723/58463
It's an old post, but no answer could really fill my needs, so I'm giving my own solution :
public static IObservable<T> ThrottleOrImmediate<T>(this IObservable<T> source, TimeSpan delay, IScheduler scheduler)
{
return Observable.Create<T>((obs, token) =>
{
// Next item cannot be send before that time
DateTime nextItemTime = default;
return Task.FromResult(source.Subscribe(async item =>
{
var currentTime = DateTime.Now;
// If we already reach the next item time
if (currentTime - nextItemTime >= TimeSpan.Zero)
{
// Following item will be send only after the set delay
nextItemTime = currentTime + delay;
// send current item with scheduler
scheduler.Schedule(() => obs.OnNext(item));
}
// There is still time before we can send an item
else
{
// we schedule the time for the following item
nextItemTime = currentTime + delay;
try
{
await Task.Delay(delay, token);
}
catch (TaskCanceledException)
{
return;
}
// If next item schedule was change by another item then we stop here
if (nextItemTime > currentTime + delay)
return;
else
{
// Set next possible time for an item and send item with scheduler
nextItemTime = currentTime + delay;
scheduler.Schedule(() => obs.OnNext(item));
}
}
}));
});
}
First item is immediately sent, then following items are throttled. Then if a following item is sent after the delayed time, it's immediately sent too.