We Keep Coding

How can I read messages from a queue in parallel?

Situation
We have one message queue. We would like to process messages in parallel and limit the number of simultaneously processed messages.
Our trial code below does process messages in parallel, but it only starts a new batch of processes when the previous one is finished. We would like to restart Tasks as they finish.
In other words: The maximum number of Tasks should always be active as long as the message queue is not empty.
Trial code
static string queue = #".\Private$\concurrenttest";
private static void Process(CancellationToken token)
{
Task.Factory.StartNew(async () =>
{
while (true)
{
IEnumerable<Task> consumerTasks = ConsumerTasks();
await Task.WhenAll(consumerTasks);
await PeekAsync(new MessageQueue(queue));
}
});
}
private static IEnumerable<Task> ConsumerTasks()
{
for (int i = 0; i < 15; i++)
{
Command1 message;
try
{
MessageQueue msMq = new MessageQueue(queue);
msMq.Formatter = new XmlMessageFormatter(new Type[] { typeof(Command1) });
Message msg = msMq.Receive();
message = (Command1)msg.Body;
}
catch (MessageQueueException mqex)
{
if (mqex.MessageQueueErrorCode == MessageQueueErrorCode.IOTimeout)
yield break; // nothing in queue
else throw;
}
yield return Task.Run(() =>
{
Console.WriteLine("id: " + message.id + ", name: " + message.name);
Thread.Sleep(1000);
});
}
}
private static Task<Message> PeekAsync(MessageQueue msMq)
{
return Task.Factory.FromAsync<Message>(msMq.BeginPeek(), msMq.EndPeek);
}

EDIT
I spent a lot of time thinking about reliability of the pump - specifically if a message is received from the MessageQueue, cancellation becomes tricky - so I provided two ways to terminate the queue:
Signaling the CancellationToken stops the pipeline as quickly as possible and will likely result in dropped messages.
Calling MessagePump.Stop() terminates the pump but allows all messages which have already been taken from the queue to be fully processed before the MessagePump.Completion task transitions to RanToCompletion.
The solution uses TPL Dataflow (NuGet: Microsoft.Tpl.Dataflow).
Full implementation:
using System;
using System.Messaging;
using System.Threading;
using System.Threading.Tasks;
using System.Threading.Tasks.Dataflow;
namespace StackOverflow.Q34437298
{
/// <summary>
/// Pumps the message queue and processes messages in parallel.
/// </summary>
public sealed class MessagePump
{
/// <summary>
/// Creates a <see cref="MessagePump"/> and immediately starts pumping.
/// </summary>
public static MessagePump Run(
MessageQueue messageQueue,
Func<Message, Task> processMessage,
int maxDegreeOfParallelism,
CancellationToken ct = default(CancellationToken))
{
if (messageQueue == null) throw new ArgumentNullException(nameof(messageQueue));
if (processMessage == null) throw new ArgumentNullException(nameof(processMessage));
if (maxDegreeOfParallelism <= 0) throw new ArgumentOutOfRangeException(nameof(maxDegreeOfParallelism));
ct.ThrowIfCancellationRequested();
return new MessagePump(messageQueue, processMessage, maxDegreeOfParallelism, ct);
}
private readonly TaskCompletionSource<bool> _stop = new TaskCompletionSource<bool>();
/// <summary>
/// <see cref="Task"/> which completes when this instance
/// stops due to a <see cref="Stop"/> or cancellation request.
/// </summary>
public Task Completion { get; }
/// <summary>
/// Maximum number of parallel message processors.
/// </summary>
public int MaxDegreeOfParallelism { get; }
/// <summary>
/// <see cref="MessageQueue"/> that is pumped by this instance.
/// </summary>
public MessageQueue MessageQueue { get; }
/// <summary>
/// Creates a new <see cref="MessagePump"/> instance.
/// </summary>
private MessagePump(MessageQueue messageQueue, Func<Message, Task> processMessage, int maxDegreeOfParallelism, CancellationToken ct)
{
MessageQueue = messageQueue;
MaxDegreeOfParallelism = maxDegreeOfParallelism;
// Kick off the loop.
Completion = RunAsync(processMessage, ct);
}
/// <summary>
/// Soft-terminates the pump so that no more messages will be pumped.
/// Any messages already removed from the message queue will be
/// processed before this instance fully completes.
/// </summary>
public void Stop()
{
// Multiple calls to Stop are fine.
_stop.TrySetResult(true);
}
/// <summary>
/// Pump implementation.
/// </summary>
private async Task RunAsync(Func<Message, Task> processMessage, CancellationToken ct = default(CancellationToken))
{
using (CancellationTokenSource producerCTS = ct.CanBeCanceled
? CancellationTokenSource.CreateLinkedTokenSource(ct)
: new CancellationTokenSource())
{
// This CancellationToken will either be signaled
// externally, or if our consumer errors.
ct = producerCTS.Token;
// Handover between producer and consumer.
DataflowBlockOptions bufferOptions = new DataflowBlockOptions {
// There is no point in dequeuing more messages than we can process,
// so we'll throttle the producer by limiting the buffer capacity.
BoundedCapacity = MaxDegreeOfParallelism,
CancellationToken = ct
};
BufferBlock<Message> buffer = new BufferBlock<Message>(bufferOptions);
Task producer = Task.Run(async () =>
{
try
{
while (_stop.Task.Status != TaskStatus.RanToCompletion)
{
// This line and next line are the *only* two cancellation
// points which will not cause dropped messages.
ct.ThrowIfCancellationRequested();
Task<Message> peekTask = WithCancellation(PeekAsync(MessageQueue), ct);
if (await Task.WhenAny(peekTask, _stop.Task).ConfigureAwait(false) == _stop.Task)
{
// Stop was signaled before PeekAsync returned. Wind down the producer gracefully
// by breaking out and propagating completion to the consumer blocks.
break;
}
await peekTask.ConfigureAwait(false); // Observe Peek exceptions.
ct.ThrowIfCancellationRequested();
// Zero timeout means that we will error if someone else snatches the
// peeked message from the queue before we get to it (due to a race).
// I deemed this better than getting stuck waiting for a message which
// may never arrive, or, worse yet, let this ReceiveAsync run onobserved
// due to a cancellation (if we choose to abandon it like we do PeekAsync).
// You will have to restart the pump if this throws.
// Omit timeout if this behaviour is undesired.
Message message = await ReceiveAsync(MessageQueue, timeout: TimeSpan.Zero).ConfigureAwait(false);
await buffer.SendAsync(message, ct).ConfigureAwait(false);
}
}
finally
{
buffer.Complete();
}
},
ct);
// Wire up the parallel consumers.
ExecutionDataflowBlockOptions executionOptions = new ExecutionDataflowBlockOptions {
CancellationToken = ct,
MaxDegreeOfParallelism = MaxDegreeOfParallelism,
SingleProducerConstrained = true, // We don't require thread safety guarantees.
BoundedCapacity = MaxDegreeOfParallelism,
};
ActionBlock<Message> consumer = new ActionBlock<Message>(async message =>
{
ct.ThrowIfCancellationRequested();
await processMessage(message).ConfigureAwait(false);
},
executionOptions);
buffer.LinkTo(consumer, new DataflowLinkOptions { PropagateCompletion = true });
if (await Task.WhenAny(producer, consumer.Completion).ConfigureAwait(false) == consumer.Completion)
{
// If we got here, consumer probably errored. Stop the producer
// before we throw so we don't go dequeuing more messages.
producerCTS.Cancel();
}
// Task.WhenAll checks faulted tasks before checking any
// canceled tasks, so if our consumer threw a legitimate
// execption, that's what will be rethrown, not the OCE.
await Task.WhenAll(producer, consumer.Completion).ConfigureAwait(false);
}
}
/// <summary>
/// APM -> TAP conversion for MessageQueue.Begin/EndPeek.
/// </summary>
private static Task<Message> PeekAsync(MessageQueue messageQueue)
{
return Task.Factory.FromAsync(messageQueue.BeginPeek(), messageQueue.EndPeek);
}
/// <summary>
/// APM -> TAP conversion for MessageQueue.Begin/EndReceive.
/// </summary>
private static Task<Message> ReceiveAsync(MessageQueue messageQueue, TimeSpan timeout)
{
return Task.Factory.FromAsync(messageQueue.BeginReceive(timeout), messageQueue.EndPeek);
}
/// <summary>
/// Allows abandoning tasks which do not natively
/// support cancellation. Use with caution.
/// </summary>
private static async Task<T> WithCancellation<T>(Task<T> task, CancellationToken ct)
{
ct.ThrowIfCancellationRequested();
TaskCompletionSource<bool> tcs = new TaskCompletionSource<bool>();
using (ct.Register(s => ((TaskCompletionSource<bool>)s).TrySetResult(true), tcs, false))
{
if (task != await Task.WhenAny(task, tcs.Task).ConfigureAwait(false))
{
// Cancellation task completed first.
// We are abandoning the original task.
throw new OperationCanceledException(ct);
}
}
// Task completed: synchronously return result or propagate exceptions.
return await task.ConfigureAwait(false);
}
}
}
Usage:
using (MessageQueue msMq = GetQueue())
{
MessagePump pump = MessagePump.Run(
msMq,
async message =>
{
await Task.Delay(50);
Console.WriteLine($"Finished processing message {message.Id}");
},
maxDegreeOfParallelism: 4
);
for (int i = 0; i < 100; i++)
{
msMq.Send(new Message());
Thread.Sleep(25);
}
pump.Stop();
await pump.Completion;
}
Untidy but functional unit tests:
https://gist.github.com/KirillShlenskiy/7f3e2c4b28b9f940c3da
ORIGINAL ANSWER
As mentioned in my comment, there are established producer/consumer patterns in .NET, one of which is pipeline. An excellent example of such can be found in "Patterns of Parallel Programming" by Microsoft's own Stephen Toub (full text here: https://www.microsoft.com/en-au/download/details.aspx?id=19222, page 55).
The idea is simple: producers continuously throw stuff in a queue, and consumers pull it out and process (in parallel to producers and possibly one another).
Here's an example of a message pipeline where the consumer uses synchronous, blocking methods to process the items as they arrive (I've parallelised the consumer to suit your scenario):
void MessageQueueWithBlockingCollection()
{
// If your processing is continuous and never stops throughout the lifetime of
// your application, you can ignore the fact that BlockingCollection is IDisposable.
using (BlockingCollection<Message> messages = new BlockingCollection<Message>())
{
Task producer = Task.Run(() =>
{
try
{
for (int i = 0; i < 10; i++)
{
// Hand over the message to the consumer.
messages.Add(new Message());
// Simulated arrival delay for the next message.
Thread.Sleep(10);
}
}
finally
{
// Notify consumer that there is no more data.
messages.CompleteAdding();
}
});
Task consumer = Task.Run(() =>
{
ParallelOptions options = new ParallelOptions {
MaxDegreeOfParallelism = 4
};
Parallel.ForEach(messages.GetConsumingEnumerable(), options, message => {
ProcessMessage(message);
});
});
Task.WaitAll(producer, consumer);
}
}
void ProcessMessage(Message message)
{
Thread.Sleep(40);
}
The above code completes in approx 130-140 ms, which is exactly what you would expect given the parallelisation of the consumers.
Now, in your scenario you are using Tasks and async/await better suited to TPL Dataflow (official Microsoft supported library tailored to parallel and asynchronous sequence processing).
Here's a little demo showing the different types of TPL Dataflow processing blocks that you would use for the job:
async Task MessageQueueWithTPLDataflow()
{
// Set up our queue.
BufferBlock<Message> queue = new BufferBlock<Message>();
// Set up our processing stage (consumer).
ExecutionDataflowBlockOptions options = new ExecutionDataflowBlockOptions {
CancellationToken = CancellationToken.None, // Plug in your own in case you need to support cancellation.
MaxDegreeOfParallelism = 4
};
ActionBlock<Message> consumer = new ActionBlock<Message>(m => ProcessMessageAsync(m), options);
// Link the queue to the consumer.
queue.LinkTo(consumer, new DataflowLinkOptions { PropagateCompletion = true });
// Wire up our producer.
Task producer = Task.Run(async () =>
{
try
{
for (int i = 0; i < 10; i++)
{
queue.Post(new Message());
await Task.Delay(10).ConfigureAwait(false);
}
}
finally
{
// Signal to the consumer that there are no more items.
queue.Complete();
}
});
await consumer.Completion.ConfigureAwait(false);
}
Task ProcessMessageAsync(Message message)
{
return Task.Delay(40);
}
It's not hard to adapt the above to use your MessageQueue and you can be sure that the end result will be free of threading issues. I'll do just that if I get a bit more time today/tomorrow.

You have one collection of things you want to process.
You create another collection for things being processed (this could be your task objects or items of some sort that reference a task).
You create a loop that will repeat as long as you have work to do. That is, work items are waiting to be started or you still have work items being processed.
At the start of the loop you populate your active task collection with as many tasks as you want to run concurrently and you start them as you add them.
You let the things run for a while (like Thread.Sleep(10);).
You create an inner loop that checks all your started tasks for completion. If one has completed, you remove it and report the results or do whatever seems appropriate.
That's it. On the next turn the upper part of your outer loop will add tasks to your running tasks collection until the number equals the maximum you have set, keeping your work-in-progress collection full.
You may want to do all this on a worker thread and monitor cancel requests in your loop.

The task library in .NET is made to execute a number of tasks in parallell. While there are ways to limit the number of active tasks, the library itself will limit the number of active tasks according to the computers CPU.
The first question that needs to be answered is why do you need to create another limit? If the limit imposed by the task library is OK, then you can just keep create tasks and rely on the task library to execute it with good performance.
If this is OK, then as soon as you get a message from MSMQ just start a task to process the message, skip the waiting (WhenAll call), start over and wait for the next message.
You can limit the number of concurrent tasks by using a custom task scheduler. More on MSDN: https://msdn.microsoft.com/en-us/library/system.threading.tasks.taskscheduler%28v=vs.110%29.aspx.

My colleague came up with the solution below. This solution works, but I'll let this code be reviewed on Code Review.
Based on answers given and some research of our own, we've come to a solution. We're using a SemaphoreSlim to limit our number of parallel Tasks.
static string queue = #".\Private$\concurrenttest";
private static async Task Process(CancellationToken token)
{
MessageQueue msMq = new MessageQueue(queue);
msMq.Formatter = new XmlMessageFormatter(new Type[] { typeof(Command1) });
SemaphoreSlim s = new SemaphoreSlim(15, 15);
while (true)
{
await s.WaitAsync();
await PeekAsync(msMq);
Command1 message = await ReceiveAsync(msMq);
Task.Run(async () =>
{
try
{
await HandleMessage(message);
}
catch (Exception)
{
// Exception handling
}
s.Release();
});
}
}
private static Task HandleMessage(Command1 message)
{
Console.WriteLine("id: " + message.id + ", name: " + message.name);
Thread.Sleep(1000);
return Task.FromResult(1);
}
private static Task<Message> PeekAsync(MessageQueue msMq)
{
return Task.Factory.FromAsync<Message>(msMq.BeginPeek(), msMq.EndPeek);
}
public class Command1
{
public int id { get; set; }
public string name { get; set; }
}
private static async Task<Command1> ReceiveAsync(MessageQueue msMq)
{
var receiveAsync = await Task.Factory.FromAsync<Message>(msMq.BeginReceive(), msMq.EndPeek);
return (Command1)receiveAsync.Body;
}

You should look at using Microsoft's Reactive Framework for this.
You code could look like this:
var query =
from command1 in FromQueue<Command1>(queue)
from text in Observable.Start(() =>
{
Thread.Sleep(1000);
return "id: " + command1.id + ", name: " + command1.name;
})
select text;
var subscription =
query
.Subscribe(text => Console.WriteLine(text));
This does all of the processing in parallel, and ensures that the processing is properly distributed across all cores. When one value ends another starts.
To cancel the subscription just call subscription.Dispose().
The code for FromQueue is:
static IObservable<T> FromQueue<T>(string serverQueue)
{
return Observable.Create<T>(observer =>
{
var responseQueue = Environment.MachineName + "\\Private$\\" + Guid.NewGuid().ToString();
var queue = MessageQueue.Create(responseQueue);
var frm = new System.Messaging.BinaryMessageFormatter();
var srv = new MessageQueue(serverQueue);
srv.Formatter = frm;
queue.Formatter = frm;
srv.Send("S " + responseQueue);
var loop = NewThreadScheduler.Default.ScheduleLongRunning(cancel =>
{
while (!cancel.IsDisposed)
{
var msg = queue.Receive();
observer.OnNext((T)msg.Body);
}
});
return new CompositeDisposable(
loop,
Disposable.Create(() =>
{
srv.Send("D " + responseQueue);
MessageQueue.Delete(responseQueue);
})
);
});
}
Just NuGet "Rx-Main" to get the bits.
In order to limit the concurrency you can do this:
int maxConcurrent = 2;
var query =
FromQueue<Command1>(queue)
.Select(command1 => Observable.Start(() =>
{
Thread.Sleep(1000);
return "id: " + command1.id + ", name: " + command1.name;
}))
.Merge(maxConcurrent);

Feeding a cancellationtoken to a Task does nothing?

I have two examples, straight from microsoft, where these examples seem to have nothing to do with cancellation token, because I can remove the token that is fed to the task, and the result is the same. So my question is: What is the cancellation token for, and why the poor examples? Am I missing something..? :)
using System;
using System.Threading;
using System.Threading.Tasks;
namespace Chapter1.Threads
{
    public class Program
    {
        static void Main()
        {
            CancellationTokenSource cancellationTokenSource =
                new CancellationTokenSource();
            CancellationToken token = cancellationTokenSource.Token;
            Task task = Task.Run(() =>
            {
                while (!token.IsCancellationRequested)
                {
                    Console.Write(“*”);
                    Thread.Sleep(1000);
                }
                token.ThrowIfCancellationRequested();
            }, token);
            try
            {
             Console.WriteLine(“Press enter to stop the task”);
             Console.ReadLine();
                cancellationTokenSource.Cancel();
                task.Wait();
            }  
catch (AggregateException e)
            {
                Console.WriteLine(e.InnerExceptions[0].Message);
            }
            Console.WriteLine(“Press enter to end the application”);
            Console.ReadLine();
        }
    }
}
Code example2:
https://msdn.microsoft.com/en-us/library/system.threading.cancellationtoken(v=vs.110).aspx
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
public class Example
{
public static void Main()
{
// Define the cancellation token.
CancellationTokenSource source = new CancellationTokenSource();
CancellationToken token = source.Token;
Random rnd = new Random();
Object lockObj = new Object();
List<Task<int[]>> tasks = new List<Task<int[]>>();
TaskFactory factory = new TaskFactory(token);
for (int taskCtr = 0; taskCtr <= 10; taskCtr++) {
int iteration = taskCtr + 1;
tasks.Add(factory.StartNew( () => {
int value;
int[] values = new int[10];
for (int ctr = 1; ctr <= 10; ctr++) {
lock (lockObj) {
value = rnd.Next(0,101);
}
if (value == 0) {
source.Cancel();
Console.WriteLine("Cancelling at task {0}", iteration);
break;
}
values[ctr-1] = value;
}
return values;
}, token));
}
try {
Task<double> fTask = factory.ContinueWhenAll(tasks.ToArray(),
(results) => {
Console.WriteLine("Calculating overall mean...");
long sum = 0;
int n = 0;
foreach (var t in results) {
foreach (var r in t.Result) {
sum += r;
n++;
}
}
return sum/(double) n;
} , token);
Console.WriteLine("The mean is {0}.", fTask.Result);
}
catch (AggregateException ae) {
foreach (Exception e in ae.InnerExceptions) {
if (e is TaskCanceledException)
Console.WriteLine("Unable to compute mean: {0}",
((TaskCanceledException) e).Message);
else
Console.WriteLine("Exception: " + e.GetType().Name);
}
}
finally {
source.Dispose();
}
}
}

Since cancellation in .Net is cooperative passing a CancellationToken into Task.Run for example is not enough to make sure the task is cancelled.
Passing the token as a parameter only associates the token with the task. It can cancel the task only if it didn't have a chance to start running before the token was cancelled. For example:
var token = new CancellationToken(true); // creates a cancelled token
Task.Run(() => {}, token);
To cancel a task "mid-flight" you need the task itself to observe the token and throw when cancellation is signaled, similar to:
Task.Run(() =>
{
while (true)
{
token.ThrowIfCancellationRequested();
// do something
}
}, token);
Moreover, simply throwing an exception from inside the task only marks the task as Faulted. To mark it as Cancelled the TaskCanceledException.CancellationToken needs to match the token passed to Task.Run.

I was about to ask a similar question until I found this one. The answer from i3arnon makes sense but I'll add this answer as an addition to hopefully help someone along.
I'll start out by saying (in contrast to the comments on the accepted answer) that the examples from Microsoft on the MSDN are horrible. Unless you already know how Cancellation works, they won't help you much. This MSDN article shows you how to pass a CancellationToken to a Task but if you follow the examples, they never actually show you how to cancel your own currently executing Task. The CancellationToken just vanishes into Microsoft code:
await client.GetAsync("http://msdn.microsoft.com/en-us/library/dd470362.aspx", ct);
await response.Content.ReadAsByteArrayAsync();
Here are examples of how I use CancellationToken:
When I have a task that needs to continually repeat:
public class Foo
{
private CancellationTokenSource _cts;
public Foo()
{
this._cts = new CancellationTokenSource();
}
public void StartExecution()
{
Task.Factory.StartNew(this.OwnCodeCancelableTask, this._cts.Token);
Task.Factory.StartNew(this.OwnCodeCancelableTask_EveryNSeconds, this._cts.Token);
}
public void CancelExecution()
{
this._cts.Cancel();
}
/// <summary>
/// "Infinite" loop with no delays. Writing to a database while pulling from a buffer for example.
/// </summary>
/// <param name="taskState">The cancellation token from our _cts field, passed in the StartNew call</param>
private void OwnCodeCancelableTask(object taskState)
{
var token = (CancellationToken) taskState;
while ( !token.IsCancellationRequested )
{
Console.WriteLine("Do your task work in this loop");
}
}
/// <summary>
/// "Infinite" loop that runs every N seconds. Good for checking for a heartbeat or updates.
/// </summary>
/// <param name="taskState">The cancellation token from our _cts field, passed in the StartNew call</param>
private async void OwnCodeCancelableTask_EveryNSeconds(object taskState)
{
var token = (CancellationToken)taskState;
while (!token.IsCancellationRequested)
{
Console.WriteLine("Do the work that needs to happen every N seconds in this loop");
// Passing token here allows the Delay to be cancelled if your task gets cancelled.
await Task.Delay(1000 /*Or however long you want to wait.*/, token);
}
}
}
When I have a task that a user can initiate:
public class Foo
{
private CancellationTokenSource _cts;
private Task _taskWeCanCancel;
public Foo()
{
this._cts = new CancellationTokenSource();
//This is where it's confusing. Passing the token here will only ensure that the task doesn't
//run if it's canceled BEFORE it starts. This does not cancel the task during the operation of our code.
this._taskWeCanCancel = new Task(this.FireTheTask, this._cts.Token);
}
/// <summary>
/// I'm not a fan of returning tasks to calling code, so I keep this method void
/// </summary>
public void FireTheTask()
{
//Check task status here if it's required.
this._taskWeCanCancel.Start();
}
public void CancelTheTask()
{
this._cts.Cancel();
}
/// <summary>
/// Go and get something from the web, process a piece of data, execute a lengthy calculation etc...
/// </summary>
private async void OurTask()
{
Console.WriteLine("Do your work here and check periodically for task cancellation requests...");
if (this._cts.Token.IsCancellationRequested) return;
Console.WriteLine("Do another step to your work here then check the token again if necessary...");
if (this._cts.Token.IsCancellationRequested) return;
Console.WriteLine("Some work that we need to delegate to another task");
await Some.Microsoft.Object.DoStuffAsync();
}
}
Maybe I missed some key feature of Task, but passing the CancellationToken to a Task as anything other than state has never made much sense to me. I have yet to run into a situation where I've passed a CancellationToken to a Task and cancelled the Task before it's run, and even if I did, the first line in every Task I create is always
if (token.IsCancellationRequested) return;

how to cancel Task quicker

I have a Windows Service which starts a task on start up
This task which has a while loop and after performing one iteration it go to sleep for 5 minutes.
When I stop service, the task is cancelled first and later some other operations gets performed
if the task is in sleep, it get cancelled only when it wakes up , i want it to be cancelled even if it is sleeping and don't want to wait for waking it up.
following is the code
Task controllerTask = Task.Factory.StartNew(() =>
{
var interval = 300;
while(true)
{
if (cancellationToken.IsCancellationRequested)
break;
Thread.Sleep(interval * 1000);
if (cancellationToken.IsCancellationRequested)
break;
//SOME WORK HERE
}
}, cancellationToken);
Is there any way?
EDIT:
I am not able to use Task.Delay , I can't find it in System.Threading.Tasks.Task namespace , because I am using .Net Framework 4.0 not 4.5
Is there any other better solution that works with 4.0.

Use Task.Delay instead of Thread.Sleep. It takes a CancellationToken parameter so you can abort it before the end of the delay.
If you're using async code, you could write it like this:
await Task.Delay(duration, cancellationToken);
If it's synchronous code, you can just wait the task:
Task.Delay(duration, cancellationToken).Wait();

This is one blocking solution you can use in C# 4.0, VS2010.
cancellationToken.WaitHandle.WaitOne(TimeSpan.FromMinutes(5));
It will unblock when you cancel the token source or on timeout which is your desired sleep interval.

Inspired by the other answers, simple example of using await for this problem:
public static class TaskExtension
{
/// <summary>
/// Call to highlight fact that you do not want to wait on this task.
///
/// This nicely removes resharper warnings without need for comments.
/// </summary>
/// <param name="task"></param>
public static void FireAndForget(this Task task)
{
}
}
internal class Program
{
private static void Main(string[] args)
{
var cancellationToken = new CancellationTokenSource();
TaskCode(cancellationToken.Token).FireAndForget();
Console.ReadLine();
cancellationToken.Cancel();
Console.WriteLine("End");
Console.ReadLine();
}
private static async Task TaskCode(CancellationToken cancellationToken)
{
while (!cancellationToken.IsCancellationRequested)
{
var interval = TimeSpan.FromSeconds(1);
await Task.Delay(interval, cancellationToken);
//SOME WORK HERE
Console.WriteLine("Tick");
}
}
}

I've broken long sleep into multiple small sleeps, following is the modified code:
Task controllerTask = Task.Factory.StartNew(() =>
{
while(true)
{
if (cancellationToken.IsCancellationRequested) break;
var sleepTime = 10;
if (interval < sleepTime)
interval = sleepTime;
var iteration = (interval / sleepTime);
if ((interval % sleepTime) > 0)
iteration++;
bool cancel = false;
for (int i = 0; i < iteration; i++)
{
Thread.Sleep(sleepTime * 1000);
if (cancellationToken.IsCancellationRequested) { cancel = true; break; };
}
if (cancel) break;
//SOME WORK HERE
}
}

.NET Throttle algorithm

i would like to implement a good throttle algorithm by in .net (C# or VB) but i cannot figure out how could i do that.
The case is my asp.net website should post requests to another website to fetch results.
At maximum 300 requests per minute should be sent.
If the requests exceed the 300 limit the other party Api returns nothing (Which is something i would not like to use as a check in my code).
P.S. I have seen solutions in other languages than .net but i am a newbie and please be kind and keep your answers as simple as 123.
Thank you

You could have a simple application (or session) class and check that for hits. This is something extremely rough just to give you the idea:
public class APIHits {
public int hits { get; private set; }
private DateTime minute = DateTime.Now();
public bool AddHit()
{
if (hits < 300) {
hits++;
return true;
}
else
{
if (DateTime.Now() > minute.AddSeconds(60))
{
//60 seconds later
minute = DateTime.Now();
hits = 1;
return true;
}
else
{
return false;
}
}
}
}

The simplest approach is just to time how long it is between packets and not allow them to be sent at a rate of more than one every 0.2 seconds. That is, record the time when you are called and when you are next called, check that at least 200ms has elasped, or return nothing.
This approach will work, but it will only work for smooth packet flows - if you expect bursts of activity then you may want to allow 5 messages in any 200ms period as long as the average over 1 minute is no more than 300 calls. In this case, you could use an array of values to store the "timestamps" of the last 300 packets, and then each time yoiu receive a call you can look back to "300 calls ago" to check that at least 1 minute has elapsed.
For both of these schemes, the time values returned by Environment.TickCount would be adequate for your needs (spans of not less than 200 milliseconds), as it's accurate to about 15 ms.

Here's an async and sync implementation of a throttle that can limit the number of calls to a method per duration of time. It's based on a simple comparison of the current time to DateTimeOffset and Task.Delay/Thread.Sleep. It should work fine for many implementations that don't need a high degree of time resolution, and should be called BEFORE the methods that you want to throttle.
This solution allows the user to specify the number of calls that are allowed per duration (with the default being 1 call per time period). This allows your throttle to be as "burstable" as you need at the cost of no control over when the callers can continue, or calls can be as evenly spaced as possible.
Let's say let’s say the target is 300 calls/min: you could have a regular throttle with a duration of 200ms that will evenly spread out every call with at least a minimum of 200ms in between, or you could create a throttle that will allow 5 calls every second with no regard to their spacing (first 5 calls win – might be all at once!). Both will keep the rate limit under 300calls/min, but the former is on the extreme end of evenly separated and the latter is more “bursty”. Having things evenly spread out is nice when processing items in a loop, but may not be so good for things running in parallel (like web requests) where the call times are unpredictable and unnecessary delays might actually slow down throughput. Again, your use case and testing will have to be your guide on which is best.
This class is thread-safe and you'll need to keep a reference to an instance of it somewhere that is accessible to the object instances that need to share it. For an ASP.NET web application that would be a field on the application instance, could be a static field on a web page/controller, injected from the DI container of your choice as a singleton, or any other way you could access the shared instance in your particular scenario.
EDIT: Updated to ensure the delay is never longer than the duration.
public class Throttle
{
/// <summary>
/// How maximum time to delay access.
/// </summary>
private readonly TimeSpan _duration;
/// <summary>
/// The next time to run.
/// </summary>
private DateTimeOffset _next = DateTimeOffset.MinValue;
/// <summary>
/// Synchronize access to the throttle gate.
/// </summary>
private readonly SemaphoreSlim _mutex = new SemaphoreSlim(1, 1);
/// <summary>
/// Number of allowed callers per time window.
/// </summary>
private readonly int _numAllowed = 1;
/// <summary>
/// The number of calls in the current time window.
/// </summary>
private int _count;
/// <summary>
/// The amount of time per window.
/// </summary>
public TimeSpan Duration => _duration;
/// <summary>
/// The number of calls per time period.
/// </summary>
public int Size => _numAllowed;
/// <summary>
/// Crates a Throttle that will allow one caller per duration.
/// </summary>
/// <param name="duration">The amount of time that must pass between calls.</param>
public Throttle(TimeSpan duration)
{
if (duration.Ticks <= 0)
throw new ArgumentOutOfRangeException(nameof(duration));
_duration = duration;
}
/// <summary>
/// Creates a Throttle that will allow the given number of callers per time period.
/// </summary>
/// <param name="num">The number of calls to allow per time period.</param>
/// <param name="per">The duration of the time period.</param>
public Throttle(int num, TimeSpan per)
{
if (num <= 0 || per.Ticks <= 0)
throw new ArgumentOutOfRangeException();
_numAllowed = num;
_duration = per;
}
/// <summary>
/// Returns a task that will complete when the caller may continue.
/// </summary>
/// <remarks>This method can be used to synchronize access to a resource at regular intervals
/// with no more frequency than specified by the duration,
/// and should be called BEFORE accessing the resource.</remarks>
/// <param name="cancellationToken">A cancellation token that may be used to abort the stop operation.</param>
/// <returns>The number of actors that have been allowed within the current time window.</returns>
public async Task<int> WaitAsync(CancellationToken cancellationToken = default(CancellationToken))
{
await _mutex.WaitAsync(cancellationToken)
.ConfigureAwait(false);
try
{
var delay = _next - DateTimeOffset.UtcNow;
// ensure delay is never longer than the duration
if (delay > _duration)
delay = _duration;
// continue immediately based on count
if (_count < _numAllowed)
{
_count++;
if (delay.Ticks <= 0) // past time window, reset
{
_next = DateTimeOffset.UtcNow.Add(_duration);
_count = 1;
}
return _count;
}
// over the allowed count within the window
if (delay.Ticks > 0)
{
// delay until the next window
await Task.Delay(delay, cancellationToken)
.ConfigureAwait(false);
}
_next = DateTimeOffset.UtcNow.Add(_duration);
_count = 1;
return _count;
}
finally
{
_mutex.Release();
}
}
/// <summary>
/// Returns a task that will complete when the caller may continue.
/// </summary>
/// <remarks>This method can be used to synchronize access to a resource at regular intervals
/// with no more frequency than specified by the duration,
/// and should be called BEFORE accessing the resource.</remarks>
/// <param name="cancellationToken">A cancellation token that may be used to abort the stop operation.</param>
/// <returns>The number of actors that have been allowed within the current time window.</returns>
public int Wait(CancellationToken cancellationToken = default(CancellationToken))
{
_mutex.Wait(cancellationToken);
try
{
var delay = _next - DateTimeOffset.UtcNow;
// ensure delay is never larger than the duration.
if (delay > _duration)
delay = _duration;
// continue immediately based on count
if (_count < _numAllowed)
{
_count++;
if (delay.Ticks <= 0) // past time window, reset
{
_next = DateTimeOffset.UtcNow.Add(_duration);
_count = 1;
}
return _count;
}
// over the allowed count within the window
if (delay.Ticks > 0)
{
// delay until the next window
Thread.Sleep(delay);
}
_next = DateTimeOffset.UtcNow.Add(_duration);
_count = 1;
return _count;
}
finally
{
_mutex.Release();
}
}
}
This sample shows how the throttle can be used synchronously in a loop, as well as how cancellation behaves. If you think of it like people getting in line for a ride, if the cancellation token is signaled it's as if the person steps out of line and the other people move forward.
var t = new Throttle(5, per: TimeSpan.FromSeconds(1));
var c = new CancellationTokenSource(TimeSpan.FromSeconds(22));
foreach(var i in Enumerable.Range(1,300)) {
var ct = i > 250
? default(CancellationToken)
: c.Token;
try
{
var n = await t.WaitAsync(ct).ConfigureAwait(false);
WriteLine($"{i}: [{n}] {DateTime.Now}");
}
catch (OperationCanceledException)
{
WriteLine($"{i}: Operation Canceled");
}
}

Rx: How can I respond immediately, and throttle subsequent requests

I would like to set up an Rx subscription that can respond to an event right away, and then ignore subsequent events that happen within a specified "cooldown" period.
The out of the box Throttle/Buffer methods respond only once the timeout has elapsed, which is not quite what I need.
Here is some code that sets up the scenario, and uses a Throttle (which isn't the solution I want):
class Program
{
static Stopwatch sw = new Stopwatch();
static void Main(string[] args)
{
var subject = new Subject<int>();
var timeout = TimeSpan.FromMilliseconds(500);
subject
.Throttle(timeout)
.Subscribe(DoStuff);
var factory = new TaskFactory();
sw.Start();
factory.StartNew(() =>
{
Console.WriteLine("Batch 1 (no delay)");
subject.OnNext(1);
});
factory.StartNewDelayed(1000, () =>
{
Console.WriteLine("Batch 2 (1s delay)");
subject.OnNext(2);
});
factory.StartNewDelayed(1300, () =>
{
Console.WriteLine("Batch 3 (1.3s delay)");
subject.OnNext(3);
});
factory.StartNewDelayed(1600, () =>
{
Console.WriteLine("Batch 4 (1.6s delay)");
subject.OnNext(4);
});
Console.ReadKey();
sw.Stop();
}
private static void DoStuff(int i)
{
Console.WriteLine("Handling {0} at {1}ms", i, sw.ElapsedMilliseconds);
}
}
The output of running this right now is:
Batch 1 (no delay)
Handling 1 at 508ms
Batch 2 (1s delay)
Batch 3 (1.3s delay)
Batch 4 (1.6s delay)
Handling 4 at 2114ms
Note that batch 2 isn't handled (which is fine!) because we wait for 500ms to elapse between requests due to the nature of throttle. Batch 3 is also not handled, (which is less alright because it happened more than 500ms from batch 2) due to its proximity to Batch 4.
What I'm looking for is something more like this:
Batch 1 (no delay)
Handling 1 at ~0ms
Batch 2 (1s delay)
Handling 2 at ~1000s
Batch 3 (1.3s delay)
Batch 4 (1.6s delay)
Handling 4 at ~1600s
Note that batch 3 wouldn't be handled in this scenario (which is fine!) because it occurs within 500ms of Batch 2.
EDIT:
Here is the implementation for the "StartNewDelayed" extension method that I use:
/// <summary>Creates a Task that will complete after the specified delay.</summary>
/// <param name="factory">The TaskFactory.</param>
/// <param name="millisecondsDelay">The delay after which the Task should transition to RanToCompletion.</param>
/// <returns>A Task that will be completed after the specified duration.</returns>
public static Task StartNewDelayed(
this TaskFactory factory, int millisecondsDelay)
{
return StartNewDelayed(factory, millisecondsDelay, CancellationToken.None);
}
/// <summary>Creates a Task that will complete after the specified delay.</summary>
/// <param name="factory">The TaskFactory.</param>
/// <param name="millisecondsDelay">The delay after which the Task should transition to RanToCompletion.</param>
/// <param name="cancellationToken">The cancellation token that can be used to cancel the timed task.</param>
/// <returns>A Task that will be completed after the specified duration and that's cancelable with the specified token.</returns>
public static Task StartNewDelayed(this TaskFactory factory, int millisecondsDelay, CancellationToken cancellationToken)
{
// Validate arguments
if (factory == null) throw new ArgumentNullException("factory");
if (millisecondsDelay < 0) throw new ArgumentOutOfRangeException("millisecondsDelay");
// Create the timed task
var tcs = new TaskCompletionSource<object>(factory.CreationOptions);
var ctr = default(CancellationTokenRegistration);
// Create the timer but don't start it yet. If we start it now,
// it might fire before ctr has been set to the right registration.
var timer = new Timer(self =>
{
// Clean up both the cancellation token and the timer, and try to transition to completed
ctr.Dispose();
((Timer)self).Dispose();
tcs.TrySetResult(null);
});
// Register with the cancellation token.
if (cancellationToken.CanBeCanceled)
{
// When cancellation occurs, cancel the timer and try to transition to cancelled.
// There could be a race, but it's benign.
ctr = cancellationToken.Register(() =>
{
timer.Dispose();
tcs.TrySetCanceled();
});
}
if (millisecondsDelay > 0)
{
// Start the timer and hand back the task...
timer.Change(millisecondsDelay, Timeout.Infinite);
}
else
{
// Just complete the task, and keep execution on the current thread.
ctr.Dispose();
tcs.TrySetResult(null);
timer.Dispose();
}
return tcs.Task;
}

Here's my approach. It's similar to others that have gone before, but it doesn't suffer the over-zealous window production problem.
The desired function works a lot like Observable.Throttle but emits qualifying events as soon as they arrive rather than delaying for the duration of the throttle or sample period. For a given duration after a qualifying event, subsequent events are suppressed.
Given as a testable extension method:
public static class ObservableExtensions
{
public static IObservable<T> SampleFirst<T>(
this IObservable<T> source,
TimeSpan sampleDuration,
IScheduler scheduler = null)
{
scheduler = scheduler ?? Scheduler.Default;
return source.Publish(ps =>
ps.Window(() => ps.Delay(sampleDuration,scheduler))
.SelectMany(x => x.Take(1)));
}
}
The idea is to use the overload of Window that creates non-overlapping windows using a windowClosingSelector that uses the source time-shifted back by the sampleDuration. Each window will therefore: (a) be closed by the first element in it and (b) remain open until a new element is permitted. We then simply select the first element from each window.
Rx 1.x Version
The Publish extension method used above is not available in Rx 1.x. Here is an alternative:
public static class ObservableExtensions
{
public static IObservable<T> SampleFirst<T>(
this IObservable<T> source,
TimeSpan sampleDuration,
IScheduler scheduler = null)
{
scheduler = scheduler ?? Scheduler.Default;
var sourcePub = source.Publish().RefCount();
return sourcePub.Window(() => sourcePub.Delay(sampleDuration,scheduler))
.SelectMany(x => x.Take(1));
}
}

The solution I found after a lot of trial and error was to replace the throttled subscription with the following:
subject
.Window(() => { return Observable.Interval(timeout); })
.SelectMany(x => x.Take(1))
.Subscribe(i => DoStuff(i));
Edited to incorporate Paul's clean-up.

Awesome solution Andrew! We can take this a step further though and clean up the inner Subscribe:
subject
.Window(() => { return Observable.Interval(timeout); })
.SelectMany(x => x.Take(1))
.Subscribe(DoStuff);

The initial answer I posted has a flaw: namely that the Window method, when used with an Observable.Interval to denote the end of the window, sets up an infinite series of 500ms windows. What I really need is a window that starts when the first result is pumped into the subject, and ends after the 500ms.
My sample data masked this problem because the data broke down nicely into the windows that were already going to be created. (i.e. 0-500ms, 501-1000ms, 1001-1500ms, etc.)
Consider instead this timing:
factory.StartNewDelayed(300,() =>
{
Console.WriteLine("Batch 1 (300ms delay)");
subject.OnNext(1);
});
factory.StartNewDelayed(700, () =>
{
Console.WriteLine("Batch 2 (700ms delay)");
subject.OnNext(2);
});
factory.StartNewDelayed(1300, () =>
{
Console.WriteLine("Batch 3 (1.3s delay)");
subject.OnNext(3);
});
factory.StartNewDelayed(1600, () =>
{
Console.WriteLine("Batch 4 (1.6s delay)");
subject.OnNext(4);
});
What I get is:
Batch 1 (300ms delay)
Handling 1 at 356ms
Batch 2 (700ms delay)
Handling 2 at 750ms
Batch 3 (1.3s delay)
Handling 3 at 1346ms
Batch 4 (1.6s delay)
Handling 4 at 1644ms
This is because the windows begin at 0ms, 500ms, 1000ms, and 1500ms and so each Subject.OnNext fits nicely into its own window.
What I want is:
Batch 1 (300ms delay)
Handling 1 at ~300ms
Batch 2 (700ms delay)
Batch 3 (1.3s delay)
Handling 3 at ~1300ms
Batch 4 (1.6s delay)
After a lot of struggling and an hour banging on it with a co-worker, we arrived at a better solution using pure Rx and a single local variable:
bool isCoolingDown = false;
subject
.Where(_ => !isCoolingDown)
.Subscribe(
i =>
{
DoStuff(i);
isCoolingDown = true;
Observable
.Interval(cooldownInterval)
.Take(1)
.Subscribe(_ => isCoolingDown = false);
});
Our assumption is that calls to the subscription method are synchronized. If they are not, then a simple lock could be introduced.

Use .Scan() !
This is what I use for Throttling when I need the first hit (after a certain period) immediately, but delay (and group/ignore) any subsequent hits.
Basically works like Throttle, but fires immediately if the previous onNext was >= interval ago, otherwise, schedule it at exactly interval from the previous hit. And of course, if within the 'cooling down' period multiple hits come, the additional ones are ignored, just like Throttle does.
The difference with your use case is that if you get an event at 0 ms and 100 ms, they will both be handled (at 0ms and 500ms), which might be what you actually want (otherwise, the accumulator is easy to adapt to ignore ANY hit closer than interval to the previous one).
public static IObservable<T> QuickThrottle<T>(this IObservable<T> src, TimeSpan interval, IScheduler scheduler)
{
return src
.Scan(new ValueAndDueTime<T>(), (prev, id) => AccumulateForQuickThrottle(prev, id, interval, scheduler))
.Where(vd => !vd.Ignore)
.SelectMany(sc => Observable.Timer(sc.DueTime, scheduler).Select(_ => sc.Value));
}
private static ValueAndDueTime<T> AccumulateForQuickThrottle<T>(ValueAndDueTime<T> prev, T value, TimeSpan interval, IScheduler s)
{
var now = s.Now;
// Ignore this completely if there is already a future item scheduled
// but do keep the dueTime for accumulation!
if (prev.DueTime > now) return new ValueAndDueTime<T> { DueTime = prev.DueTime, Ignore = true };
// Schedule this item at at least interval from the previous
var min = prev.DueTime + interval;
var nextTime = (now < min) ? min : now;
return new ValueAndDueTime<T> { DueTime = nextTime, Value = value };
}
private class ValueAndDueTime<T>
{
public DateTimeOffset DueTime;
public T Value;
public bool Ignore;
}

I got another one for your. This one doesn't use Repeat() nor Interval() so it might be what you are after:
subject
.Window(() => Observable.Timer(TimeSpan.FromMilliseconds(500)))
.SelectMany(x => x.Take(1));

Well the most obvious thing will be to use Repeat() here. However, as far as I know Repeat() might introduce problems so that notifications disappear in between the moment when the stream stops and we subscribe again. In practice this has never been a problem for me.
subject
.Take(1)
.Concat(Observable.Empty<long>().Delay(TimeSpan.FromMilliseconds(500)))
.Repeat();
Remember to replace with the actual type of your source.
UPDATE:
Updated query to use Concat instead of Merge

I have stumbled upon this question while trying to re-implement my own solution to the same or similar problem using .Window
Take a look, it seems to be the same as this one and solved quite elegantly:
https://stackoverflow.com/a/3224723/58463

It's an old post, but no answer could really fill my needs, so I'm giving my own solution :
public static IObservable<T> ThrottleOrImmediate<T>(this IObservable<T> source, TimeSpan delay, IScheduler scheduler)
{
return Observable.Create<T>((obs, token) =>
{
// Next item cannot be send before that time
DateTime nextItemTime = default;
return Task.FromResult(source.Subscribe(async item =>
{
var currentTime = DateTime.Now;
// If we already reach the next item time
if (currentTime - nextItemTime >= TimeSpan.Zero)
{
// Following item will be send only after the set delay
nextItemTime = currentTime + delay;
// send current item with scheduler
scheduler.Schedule(() => obs.OnNext(item));
}
// There is still time before we can send an item
else
{
// we schedule the time for the following item
nextItemTime = currentTime + delay;
try
{
await Task.Delay(delay, token);
}
catch (TaskCanceledException)
{
return;
}
// If next item schedule was change by another item then we stop here
if (nextItemTime > currentTime + delay)
return;
else
{
// Set next possible time for an item and send item with scheduler
nextItemTime = currentTime + delay;
scheduler.Schedule(() => obs.OnNext(item));
}
}
}));
});
}
First item is immediately sent, then following items are throttled. Then if a following item is sent after the delayed time, it's immediately sent too.