We Keep Coding

C# async lock get the same result without code execution

I have a method that returns some value based on an API call, this API limits the amount of calls that you can do per period of time. I need to access the results of this call from multiple threads. Right now i have the following code:
class ReturningSemaphoreLocker<TOutput>
{
private readonly SemaphoreSlim _semaphore = new SemaphoreSlim(1, 1);
public async Task<T> LockAsync<T>(Func<Task<T>> worker)
{
await _semaphore.WaitAsync();
try
{
return await worker();
}
finally
{
_semaphore.Release();
}
}
}
Usage example:
...
private static readonly ReturningSemaphoreLocker<List<int>> LockingSemaphore = new ReturningSemaphoreLocker<List<int>>();
...
public async Task<List<int>> GetStuff()
{
return await LockingSemaphore.LockAsync(async () =>
{
var client = _clientFactory.CreateClient("SomeName");
using (var cts = GetDefaultRequestCts())
{
var resp = await client.GetAsync("API TO QUERY URL", cts.Token);
var jsonString = await resp.Content.ReadAsStringAsync();
var items = JsonConvert.DeserializeObject<List<int>>(jsonString);
return items;
}
});
}
So the question is: how do i get the same result from GetStuff() if it's already running WITHOUT querying the API again and query the API again if the method is not running at this very moment?

The trick here is to hold onto the Task<T> that is the incomplete result; consider the following completely untested approach - the _inProgress field is the key here:
private static readonly ReturningSemaphoreLocker<List<int>> LockingSemaphore = new ReturningSemaphoreLocker<List<int>>();
...
private Task<List<int>> _inProgress;
public Task<List<int>> GetStuffAsync()
{
if (_inProgress != null) return _inProgress;
return _inProgress = GetStuffImplAsync();
}
private async Task<List<int>> GetStuffImplAsync()
{
var result = await LockingSemaphore.LockAsync(async () =>
{
var client = _clientFactory.CreateClient("SomeName");
using (var cts = GetDefaultRequestCts())
{
var resp = await client.GetAsync("API TO QUERY URL", cts.Token);
var jsonString = await resp.Content.ReadAsStringAsync();
var items = JsonConvert.DeserializeObject<List<int>>(jsonString);
return items;
}
});
// this is important so that if everything turns
// out to be synchronous, we don't nuke the _inProgress field *before*
// it has actually been set
await Task.Yield();
// and now wipe the field since we know it is no longer in progress;
// the next caller should actually try to do something interesting
_inProgress = null;
return result;
}

Here is a class that you could use for time-based throttling, instead of the ReturningSemaphoreLocker:
class ThrottledOperation
{
private readonly object _locker = new object();
private readonly Stopwatch _stopwatch = Stopwatch.StartNew();
private Task _task;
public Task<T> GetValueAsync<T>(Func<Task<T>> taskFactory, TimeSpan interval)
{
lock (_locker)
{
if (_task != null && (_stopwatch.Elapsed < interval || !_task.IsCompleted))
{
return (Task<T>)_task;
}
_task = taskFactory();
_stopwatch.Restart();
return (Task<T>)_task;
}
}
}
The GetValueAsync method returns the same task, until the throttling interval has been elapsed and the task has been completed. At that point it creates and returns a new task, using the supplied task-factory method.
Usage example:
private static readonly ThrottledOperation _throttledStuff = new ThrottledOperation();
public Task<List<int>> GetStuffAsync()
{
return _throttledStuff.GetValueAsync(async () =>
{
var client = _clientFactory.CreateClient("SomeName");
using (var cts = GetDefaultRequestCts())
{
var resp = await client.GetAsync("API TO QUERY URL", cts.Token);
var jsonString = await resp.Content.ReadAsStringAsync();
var items = JsonConvert.DeserializeObject<List<int>>(jsonString);
return items;
}
}, TimeSpan.FromSeconds(30));
}

Async seems to be synchronous

I am not sure if I am missing something here but more for loop seems to be executing synchronously even though I await all tasks out side of it.
Here is my code below:
static void Main(string[] args) {
var t = Start();
}
public static async Task < List < Task < TaskInfo >>> Start() {
var listOfTasks = new List < Task < TaskInfo >> ();
for (var i = 0; i <= 100; i++) {
var process = new Processor();
listOfTasks.Add(process.Process(i));
}
await Task.WhenAll(listOfTasks);
return listOfTasks;
}
I pass in the taskId to log out just to see the order the tasks execute.
Am I missing something really obvious here?
EDIT:
Changed code to this based on the answers and comments below and it still appears synchronously:
public class StartWork
{
public int TaskId { get; set; }
public Processor Processor { get;}
public StartWork()
{
Processor = new Processor();
}
}
static void Main(string[] args)
{
var t = Start();
}
public static async Task<TaskInfo[]> Start()
{
var tasks = new List<StartWork>();
for (int i = 1; i < 100; i++)
{
var work = new StartWork
{
TaskId = i
};
tasks.Add(work);
}
return await Task.WhenAll(tasks.Select(i => i.Processor.Process(i.TaskId)));
}
The function I am calling in processor class:
public Task<TaskInfo> Process(int taskId)
{
try
{
taskId = taskId + 1;
stopwatch.Start();
using (var bus = RabbitHutch.CreateBus(xxDev))
{
#event = new AutoResetEvent(false);
var replyTo = Guid.NewGuid().ToString();
var messageQueue = bus.Advanced.QueueDeclare(replyTo, autoDelete: true);
bus.Advanced.Consume(messageQueue, (payload, properties, info) =>
{
ReceivePdf(payload, properties, info);
return Task.FromResult(0);
});
taskInfo.InputFile = inputFile;
var html = File.ReadAllText(inputFile);
taskInfo.Html = html;
var message = PrepareMessage(new RenderRequest()
{
Html = Encoding.UTF8.GetBytes(html),
Options = new RenderRequestOptions()
{
PageSize = "A4",
ImageQuality = 70,
PageLoadRetryAttempts = 3
}
});
var correlation = Guid.NewGuid().ToString();
Console.WriteLine($"CorrelationId: {correlation}, TaskId {taskId}");
var props = new MessageProperties
{
CorrelationId = correlation,
ReplyTo = replyTo,
Expiration = "6000"
};
Publish(bus, props, message);
taskInfo.CorrelationId = Guid.Parse(correlation);
#event.WaitOne();
stopwatch.Stop();
taskInfo.TimeTaken = stopwatch.Elapsed;
return Task.FromResult(taskInfo);
}
}
catch (Exception e)
{
taskInfo.OutputFile = Empty;
return Task.FromResult(taskInfo);
}
}
void ReceivePdf(byte[] payload, MessageProperties properties, MessageReceivedInfo info)
{
var file = Format(outputFile, properties.CorrelationId);
taskInfo.OutputFile = file;
Console.WriteLine("Output written to " + file);
File.WriteAllBytes(file, payload);
var remaining = Interlocked.Decrement(ref outstandingRequests);
if (remaining == 0)
{
#event.Set();
}
}

This is a synchronous task
listOfTasks.Add(process.Process(i));
You are just adding items to the list.
This is also a synchronous task
process.Process(i);
The task that the function above returns is asynchronous and it will execute asynchronously in the whenAll call of your code.
Bear in mind that when all will wait for all tasks to run and if the task is trivial, since they start one after the other, will most times run sequentially by chance.
You will see some difference if the task code executed differentiated in execution time based on input.

First async doesn't mean multithread, async is used to run background task without blocking UI or to run I/O operations without bloking main thread.
Usually the operating system handles async with multithreading, but there is no guarantee.
If you want be sure to start multiple threads use Thread.Start.
Anyway in your code you force your code to run synchronously, because you call the async method start in the Main method without await.
You need to change the code to:
static async void Main(string[] args)
{
var t = await Start();
}
or without waiting to (but the program risk to terminate before the task complete):
static void Main(string[] args)
{
Task.Run(async () => {
var t = await Start();
});
}

Best way to throttle outgoing requests in C# by time and some extra criteria?

I have to call an external HTTP API that only allows one request every 4 seconds by userId. As long as I'm calling this API sending a different userId each time, I can call it at any rate.
In this code I'm able to comply with the external API rate, but I'm not doing it in the optimal way, since some requests get blocked by previous calls even if that userId doesn't need to wait. (Check comments in code)
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;
namespace ConsoleApp2
{
class Program
{
static void Main(string[] args)
{
var caller = new ExternalAPICaller();
caller.RunCalls();
Console.ReadKey();
}
}
public class ExternalAPICaller
{
private static SemaphoreSlim throttler = new SemaphoreSlim(20); // up to 20 concurrent calls
private static List<string> userIds = new List<string>();
private const int rateLimitByUser = 4000;
public async Task CallAPIWithThrottling(string userId)
{
if (userIds.Contains(userId)) Thread.Sleep(rateLimitByUser);
userIds.Add(userId);
await throttler.WaitAsync();
var task = MockHttpCall(userId);
_ = task.ContinueWith(async s =>
{
await Task.Delay(rateLimitByUser);
throttler.Release();
userIds.Remove(userId);
});
}
public Task MockHttpCall(string id)
{
Console.WriteLine("http call for " + id);
Thread.Sleep(300);
return Task.CompletedTask;
}
public async Task RunCalls()
{
await CallAPIWithThrottling("Mike");
await CallAPIWithThrottling("John");
await CallAPIWithThrottling("Sarah");
await CallAPIWithThrottling("Matt");
await CallAPIWithThrottling("John");
await CallAPIWithThrottling("Jacob"); // this should be called right away, but the second John makes it wait
await CallAPIWithThrottling("John");
await CallAPIWithThrottling("Amy"); // this should be called right away, but the thrid John makes it wait
}
}
}

I would try to abstract the throttling functionality, so that I can test it independently. I would make a Throttler class that can be configured with global and per-user concurrency limits and delays. In your case the configuration would be:
Global concurrency limit: 20
Global delay: 0 (simultaneous requests for different users are allowed)
Per user concurrency limit: 1
Per user delay: 4000
Here is an implementation of the Throttler class. The per user concurrency limit is omitted for simplicity (it would require a second SemaphoreSlim per user).
public class Throttler<TKey>
{
private readonly SemaphoreSlim _globalConcurrencySemaphore;
private readonly SemaphoreSlim _globalDelaySemaphore;
private readonly int _globalDelay;
private readonly int _perKeyDelay;
private readonly ConcurrentDictionary<TKey, SemaphoreSlim> _perKeyDelaySemaphores;
public Throttler(int globalConcurrencyLimit, int globalDelay, int perKeyDelay)
{
_globalConcurrencySemaphore = new SemaphoreSlim(globalConcurrencyLimit,
globalConcurrencyLimit);
_globalDelaySemaphore = new SemaphoreSlim(1, 1);
_globalDelay = globalDelay;
_perKeyDelay = perKeyDelay;
_perKeyDelaySemaphores = new ConcurrentDictionary<TKey, SemaphoreSlim>();
}
public async Task<TResult> Execute<TResult>(TKey key,
Func<Task<TResult>> taskFactory)
{
var perKeyDelaySemaphore = _perKeyDelaySemaphores.GetOrAdd(
key, _ => new SemaphoreSlim(1, 1));
await perKeyDelaySemaphore.WaitAsync().ConfigureAwait(false);
ReleaseAsync(perKeyDelaySemaphore, _perKeyDelay);
await _globalDelaySemaphore.WaitAsync().ConfigureAwait(false);
ReleaseAsync(_globalDelaySemaphore, _globalDelay);
await _globalConcurrencySemaphore.WaitAsync().ConfigureAwait(false);
try
{
var task = taskFactory();
return await task.ConfigureAwait(false);
}
finally
{
_globalConcurrencySemaphore.Release();
}
}
private async void ReleaseAsync(SemaphoreSlim semaphore, int delay)
{
await Task.Delay(delay).ConfigureAwait(false);
semaphore.Release();
}
}
The delay is between one semaphore acquirement and the next. The delay of the HTTP call is not taken into account.
Usage example:
var throttler = new Throttler<string>(20, 0, 4000);
var keys = new string[] { "Mike", "John", "Sarah", "Matt", "John", "Jacob",
"John", "Amy" };
var tasks = new List<Task>();
foreach (var key in keys)
{
tasks.Add(throttler.Execute(key, () => MockHttpCall(key)));
}
Task.WaitAll(tasks.ToArray());
async Task<int> MockHttpCall(string id)
{
Console.WriteLine($"{DateTime.Now:HH:mm:ss.fff} HTTP call for " + id);
await Task.Delay(300);
return 0;
}
Output:
11:20:41.635 HTTP call for Mike
11:20:41.652 HTTP call for John
11:20:41.652 HTTP call for Sarah
11:20:41.652 HTTP call for Matt
11:20:41.653 HTTP call for Jacob
11:20:41.654 HTTP call for Amy
11:20:45.965 HTTP call for John
11:20:50.272 HTTP call for John

Wait for a deferred value in a lambda before returning it in a function

I have a function call (in a SignalR hub) that is supposed to return a MyState object.
The MyState object is not quite ready at the time it is needed by the caller and has to be processed first by some background process prior being eligible as being ready to be sent.
That's why I want the action that retrieves the state to be queued in a stack to be processed later. This is part of an existing framework.
I am not good at all with the synchronization methods of the .NET framework, and everything I looked for is in JavaScript. So I would need a bit help to fill what is missing here:
public MyState GetState() {
MyState state;
// var waitMe = someObjectThatCanWait(); ?
actions.Enqueue(sync => {
// This lambda will be executed later by a function that processes the actions queue
state = sync.GetSyncState());
// waitMe.NowYouCanGo() ?
});
// waitMe.Wait() ?
return state;
}
EDIT I have something that works. It is a bit obscure to me I wonder
whether this is ok.
public MyState GetState() {
MyState state;
var waitMe = new Barrier(2);
actions.Enqueue(sync => {
// This lambda will be executed later by a function that processes the actions queue
state = sync.GetSyncState());
waitMe.SignalAndWait();
});
waitMe.SignalAndWait();
return state;
}
EDIT2 I have something more suitable since it does not block the lambda
and the function at the same time
public MyState GetState() {
MyState state;
var waitMe = new ManualResetEventSlim();
actions.Enqueue(sync => {
// This lambda will be executed later by a function that processes the actions queue
state = sync.GetSyncState());
waitMe.Set();
});
waitMe.Wait(1500);
return state;
}

Here is an example of similar workflow using TaskCompletionSource:
class Program
{
private static readonly ConcurrentQueue<(string Data, TaskCompletionSource<string> Tcs)> Queue = new ConcurrentQueue<(string Data, TaskCompletionSource<string> Tcs)>();
static void Main()
{
var cts = new CancellationTokenSource();
Task.Run(() => ProcessQueue(cts.Token), cts.Token);
ListenForTasks().GetAwaiter().GetResult();
cts.Cancel();
}
private static async Task ListenForTasks()
{
while (true)
{
var input = Console.ReadLine();
if (input == "q")
{
break;
}
var tcs = new TaskCompletionSource<string>();
Queue.Enqueue((input, tcs));
Console.WriteLine("waiting...");
var result = await tcs.Task;
Console.WriteLine($"waited: {result}");
}
}
private static void ProcessQueue(CancellationToken ct)
{
while (true)
{
ct.ThrowIfCancellationRequested();
if (!Queue.TryDequeue(out var item))
{
continue;
}
//emulate work
Thread.Sleep(1000);
Console.WriteLine($"processed {item.Data}");
item.Tcs.SetResult(item.Data + "_PROCESSED");
}
}
}

C# HttpWebRequest.BeginGetResponse blocks in one class but not another

I am validating a list of proxies using HttpWebRequest.BeginGetResponse. It works really well, I can validate thousands of proxies in seconds and doesn't block.
In another class within my project, I am calling the same code and it blocks.
Proxy validation method (Doesn't block):
public void BeginTest(IProxyTest test, Action<ProxyStatus> callback, int timeout = 10000)
{
var req = HttpWebRequest.Create(test.URL);
req.Proxy = new WebProxy(this.ToString());
req.Timeout = timeout;
WebHelper.BeginGetResponse(req, new Action<RequestCallbackState>(callbackState =>
{
if (callbackState.Exception != null)
{
callback(ProxyStatus.Invalid);
}
else
{
var responseStream = callbackState.ResponseStream;
using (var reader = new StreamReader(responseStream))
{
var responseString = reader.ReadToEnd();
if (responseString.Contains(test.Validation))
{
callback(ProxyStatus.Valid);
}
else
{
callback(ProxyStatus.Invalid);
}
}
}
}));
}
WebHelper.BeginGetResponse
public static void BeginGetResponse(WebRequest request, Action<RequestCallbackState> responseCallback)
{
Task<WebResponse> asyncTask = Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null);
ThreadPool.RegisterWaitForSingleObject((asyncTask as IAsyncResult).AsyncWaitHandle, new WaitOrTimerCallback(TimeoutCallback), request, request.Timeout, true);
asyncTask.ContinueWith(task =>
{
WebResponse response = task.Result;
Stream responseStream = response.GetResponseStream();
responseCallback(new RequestCallbackState(responseStream));
responseStream.Close();
response.Close();
}, TaskContinuationOptions.NotOnFaulted);
//Handle errors
asyncTask.ContinueWith(task =>
{
var exception = task.Exception;
responseCallback(new RequestCallbackState(exception.InnerException));
}, TaskContinuationOptions.OnlyOnFaulted);
}
Other class with a similar method that also calls WebHelper.BeginGetResponse, but blocks (why?)
public void BeginTest(Action<ProxyStatus> callback, int timeout = 10000)
{
var req = HttpWebRequest.Create(URL);
req.Timeout = timeout;
WebHelper.BeginGetResponse(req, new Action<RequestCallbackState>(callbackState =>
{
if (callbackState.Exception != null)
{
callback(ProxyStatus.Invalid);
}
else
{
var responseStream = callbackState.ResponseStream;
using (var reader = new StreamReader(responseStream))
{
var responseString = reader.ReadToEnd();
if (responseString.Contains(Validation))
{
callback(ProxyStatus.Valid);
}
else
{
callback(ProxyStatus.Invalid);
}
}
}
}));
}
Calling code which blocks
private async void validateTestsButton_Click(object sender, EventArgs e)
{
await Task.Run(() =>
{
foreach (var test in tests)
{
test.BeginTest((status) => test.Status = status);
}
});
}
Calling code which doesn't block:
public static async Task BeginTests(ICollection<Proxy> proxies, ICollection<ProxyJudge> judges, int timeout = 10000, IProgress<int> progress = null)
{
await Task.Run(() =>
{
foreach (var proxy in proxies)
{
proxy.BeginTest(judges.GetRandomItem(), new Action<ProxyStatus>(status =>
{
proxy.Status = status;
}), timeout);
}
});
}

Although this dosnt address your problem exactly, it might help you out a little
Here are a couple of problems
You are using APM (Asynchronous Programming Model)
You are using the ThreadPool class which seems a little old fashioned
You are doing IO bound work and blocking threads on the threadpool
You are using a weird mix of APM and TBA asynchronous models
And seemingly tying up your thread pool waiting for IO
So you are doing IO bound work, the best pattern to use as you might have guess is the TBA async await pattern. basically every time you wait for a an IO Completion port you want to give that thread back to the operating system and be nice to your system inturn freeing up resources for where its needed.
Also you obviously want some degree of parallelism and you are best to at least have some control over it.
I would suggest this is a nice job for TPL Dataflow and an ActionBlock
Given
public class Proxy
{
public ProxyStatus ProxyStatus { get; set; }
public string ProxyUrl { get; set; }
public string WebbUrl { get; set; }
public string Error { get; set; }
}
ActionBlock Example
public static async Task DoWorkLoads(List<Proxy> results)
{
var options = new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 50
};
var block = new ActionBlock<Proxy>(CheckUrlAsync, options);
foreach (var proxy in results)
{
block.Post(proxy);
}
block.Complete();
await block.Completion;
}
CheckUrlAsync Example
// note i havent tested this, add pepper and salt to taste
public static async Task CheckUrlAsync(Proxy proxy)
{
try
{
var request = WebRequest.Create(proxy.Url);
if (proxy.ProxyUrl != null)
request.Proxy = new WebProxy(proxy.ProxyUrl);
using (var response = await request.GetResponseAsync())
{
using (var responseStream = response.GetResponseStream())
{
using (var reader = new StreamReader(responseStream))
{
var responseString = reader.ReadToEnd();
if (responseString.Contains("asdasd"))
proxy.ProxyStatus = ProxyStatus.Valid;
else
proxy.ProxyStatus = ProxyStatus.Invalid;
}
}
}
}
catch (Exception e)
{
proxy.ProxyStatus = ProxyStatus.Error;
proxy.Error = e.Message;
}
}
Usage
await DoWorkLoads(proxies to test);
Summary
The code is neater, you arnt throwing actions all over the place, you using async and await you have ditched APM, you have control of the degrees of parallel and you are being nice to the thread pool

I solved the problem by wrapping the code in the BeginTest method which was blocking mysteriously in an Action, and then calling BeginInvoke on that Action.
I deduced this was caused by not setting the Proxy property on the HttpWebRequest in that method, which seemed to be causing a synchronous lookup of my systems proxy.
public void BeginTest(Action<ProxyStatus> callback, int timeout = 10000)
{
var action = new Action(() =>
{
var req = HttpWebRequest.Create(URL);
req.Timeout = timeout;
WebHelper.BeginGetResponse(req, new Action<RequestCallbackState>(callbackState =>
{
if (callbackState.Exception != null)
{
callback(ProxyStatus.Invalid);
}
else
{
var responseStream = callbackState.ResponseStream;
using (var reader = new StreamReader(responseStream))
{
var responseString = reader.ReadToEnd();
if (responseString.Contains(Validation))
{
callback(ProxyStatus.Valid);
}
else
{
callback(ProxyStatus.Invalid);
}
}
}
}));
});
action.BeginInvoke(null, null);
}