I need to call an API thousands of times as quickly as possible. The API has a limit of 10 calls per second. In order to take full advantage of the 10 calls per second without going over, I'm calling the API asynchronously and throttling the calls with a semaphore and a timer. My code enters the semaphore, calls the API, and then makes sure at least one second has passed before it releases the semaphore.
The API call is actually pretty quick and returns in about a second or less, so my code should move right on to the check time/release semaphore logic. However, what actually happens is after 10 calls the semaphore fills up and is not released at all until the rest of the async Tasks to call the API are created. After that everything works as expected, so I'm not experiencing any real issues. The behavior just seems strange and I would like to understand it.
public static class MyClass
{
SemaphoreSlim semaphore = new SemaphoreSlim(10);
public static async Task CreateCallApiTasks(IList<object> requests)
{
var callApiTasks = requests.Select(x => CallApi(x));
await Task.WhenAll(callApiTasks);
}
private static async Task CallApi(object requestBody)
{
using (var request = new HttpRequestMessage(HttpMethod.Post, <apiUri>))
{
request.Content = new StringContent(requestBody, System.Text.Encoding.UTF8, "application/json");
HttpResponseMessage response = null;
using (var httpClient = new HttpClient())
{
var throttle = new Stopwatch();
ExceptionDispatchInfo capturedException = null;
await semaphore.WaitAsync();
try
{
throttle.Start();
response = await httpClient.SendAsync(request);
while (throttle.ElapsedMilliseconds < 1000)
{
await Task.Delay(1);
}
semaphore.Release();
throttle.Stop();
}
catch (Exception ex)
{
capturedException = ExceptionDispatchInfo.Capture(ex);
}
if (capturedException != null)
{
while (throttle.ElapsedMilliseconds < 1000)
{
await Task.Delay(1);
}
semaphore.Release();
throttle.Stop();
capturedException.Throw();
}
}
}
}
}
Related
So I have been struggling with this issue for like 3 weeks. Here's what I want to do.
So I have like 2000 stock options. I want to fetch 5 of them at a time and process but it all has to be parallel. I'll write them in steps to make it more clear.
Get 5 stock symbols from an array
Send it to fetch its data and process. Don't wait for a response keep on processing.
wait 2.6 seconds (as we are limited to 120 API requests per minute so this delay helps in keeping it throttled to 115 per minute)
Goto 1
All the steps above have to be parallel. I have written the code for it and it all seems to be working fine but randomly it crashes saying
"A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection
failed because connected host has failed to respond".
And sometimes it'll never happen and everything works like a charm.
This error is very random. It could show up on maybe 57th stock or maybe at 1829th stock. I have used HttpClient for it. I have tested this same scenario using Angular and creating custom requests and it never crashes there so it's not third-party server's fault.
What I have already done:
Changed HttpClient class usage from new instances every time to a single instance for the whole project.
Increases Service point manager Connection limit to a different number. (Default for .net core is 2)
Instead of HttpClient Queuing I have used SemaphoreSlim for queue and short-circuiting.
Forced ConnectionLeaseTimeout to 40 seconds to detect DNS changes if any.
Changed Async Tasks to threading.
Tried almost everything from the internet.
My doubts:
I doubt that it has something to do with the HttpClient class. I have read a lot of bad things about its misleading documentation etc.
My friend's doubt:
He said it could be because of concurrent tasks and I should change it to threads.
Here's the code:
// Inside Class Constructor
private readonly HttpClient HttpClient = new HttpClient();
SetMaxConcurrency(ApiBaseUrl, maxConcurrentRequests);
// SetMaxConcurrency function
private void SetMaxConcurrency(string url, int maxConcurrentRequests)
{
ServicePointManager.FindServicePoint(new Uri(url)).ConnectionLimit = maxConcurrentRequests;
ServicePointManager.FindServicePoint(new Uri(url)).ConnectionLeaseTimeout = 40*1000;
}
// code for looping through chunks of symbol each chunk has 5 symbols/stocks in it
foreach(var chunkedSymbol in chunkedSymbols)
{
//getting o auth token
string AuthToken = await OAuth();
if(String.IsNullOrEmpty(AuthToken))
{
throw new ArgumentNullException("Access Token is null!");
}
processingSymbols += chunkSize;
OptionChainReq.symbol = chunkedSymbol.ToArray();
async Task func()
{
//function that makes request
var response = await GetOptionChain(AuthToken, ClientId, OptionChainReq);
// concat the result in main list
appResponses = appResponses.Concat(response).ToList();
}
// if request reaches 115 process the remaning requests first
if(processingSymbols >= 115)
{
await Task.WhenAll(tasks);
processingSymbols = 0;
}
tasks.Add(func());
// 2600 millisecond delay to wait for all the data to process
await Task.Delay(delay);
}
//once the loop is completed process the remaining requests
await Task.WhenAll(tasks);
// This code processes every symbol. this code is inside GetOptionChain()
try{
var tasks = new List<Task>();
foreach (string symbol in OptionChainReq.symbol)
{
List<KeyValuePair<string, string>> Params = new List<KeyValuePair<string, string>>();
string requestParams = string.Empty;
// Converting Request Params to Key Value Pair.
Params.Add(new KeyValuePair<string, string>("apikey" , ClientId));
// URL Request Query parameters.
requestParams = new FormUrlEncodedContent(Params).ReadAsStringAsync().Result;
string endpoint = ApiBaseUrl + "/marketdata/chains?";
HttpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", OAuthToken);
Uri tosUri = new Uri(endpoint + requestParams, UriKind.Absolute);
async Task func()
{
try{
string responseString = await GetTosData(tosUri);
OptionChainResponse OptionChainRes = JsonConvert.DeserializeObject<OptionChainResponse>(responseString);
var mappedOptionAppRes = MapOptionsAppRes( OptionChainRes );
if(mappedOptionAppRes != null)
{
OptionsData.Add( mappedOptionAppRes );
}
}
catch(Exception ex)
{
throw new Exception("Crashed");
}
}
// asyncronusly processing each request
tasks.Add(func());
}
//making sure all 5 requests are processed
await Task.WhenAll(tasks);
}
catch (Exception ex)
{
failedSymbols += " "+ string.Join(",", OptionChainReq.symbol);
}
// The code below is for individual request
public async Task<string> GetTosData(Uri url)
{
try
{
await semaphore.WaitAsync();
if (IsTripped())
{
return UNAVAILABLE;
}
var response = await HttpClient.GetAsync(url);
if(response.StatusCode == System.Net.HttpStatusCode.Unauthorized)
{
string OAuthToken = await OAuth();
HttpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", OAuthToken);
return await GetTosData(url);
}
else if(response.StatusCode != HttpStatusCode.OK)
{
TripCircuit(reason: $"Status not OK. Status={response.StatusCode}");
return UNAVAILABLE;
}
return await response.Content.ReadAsStringAsync();
}
catch(Exception ex) when (ex is OperationCanceledException || ex is TaskCanceledException)
{
Console.WriteLine("Timed out");
TripCircuit(reason: $"Timed out");
return UNAVAILABLE;
}
finally
{
semaphore.Release();
}
}
I am working on a Xamarin.Forms Android application that requires the phone to ping to the server every 15 seconds. All my calls are asynchronous and all of them have the "await" attribute, this includes all the main user functions in the main classes that are not found in Device.StartTimer objects. For example, registering data upon button click, logging in and logging out. For the 15 second ping I am using a Device.StartTimer function and I have not had too many issues but at times I do notice that the responses do overlap, however I thought the "await" declaration would have taken care of responses overlapping because I read that Device.StartTimer works on the main thread. What I'm I doing wrong? Is there a better way to managed timed HTTPClient calls?
I tried applying the await attribute to the function to make sure that the calls do not overlap. There is a note about Device.StartTimer running on the main thread, so I thought that ALL my async await functions would be respected. Including the async function in the main classes.
//Function to ping to the server every 15 seconds
private void StartOfflineTimer()
{
Device.StartTimer(TimeSpan.FromSeconds(15.0), () =>
{
if(timerOffline)
{
Task.Run(async () =>
if(await InformarOfflineAsync(Settings.AccessToken, idRutaOffline))
{
DependencyService.Get<ILogUtils>().GuardarLine("**Device conected..");
}
else
{
DependencyService.Get<ILogUtils>().GuardarLine("**Device disconnected..");
}
);
}
return timerOffline;
});
}
//Default Example of how I handle ALL HTTPClient calls on the app, including calls that are in the main classes, not embedded in a device timer. All of these calls are inside their own public async Task<ExampleObject> function. Once again all of the functions that make these calls have an "await" attribute.
var jsonRequest = await Task.Run(() => JsonConvert.SerializeObject(requestObj));
var httpContent = new StringContent(jsonRequest, Encoding.UTF8, "application/json");
using (var httpClient = new HttpClient())
{
httpClient.Timeout = TimeSpan.FromSeconds(10.0);
var httpResponse = await httpClient.PostAsync(Constants.BaseUrl + "login/validarOffline", httpContent);
ExampleObjectResponseObj exampleObject = new ExampleObjectResponseObj();
var responseContent = await httpResponse.Content.ReadAsStringAsync();
ExampleObjectResponseObj = JsonConvert.DeserializeObject<InformDataResponseObj>(responseContent);
return ExampleObjectResponseObj;
}
The HTTPClient responses may overlap, or at times they double up and send at the exact same time.
If you don't want your calls to overlap, don't use timer but a loop with a delay:
Task.Run(async () =>
{
const TimeSpan checkInterval = TimeSpan.FromSeconds(15);
while (true)
{
var callTime = DateTime.UtcNow;
try
{
await server.Ping();
}
catch (Exception exception)
{
HandleException(exception);
}
var elapsedTime = DateTime.UtcNow - callTime;
var timeToWait = checkInterval - elapsedTime;
if (timeToWait > TimeSpan.Zero)
{
await Task.Delay(timeToWait);
}
}
});
The code above isn't complete and not enough for a very detailed and precise answer, but still most if not all things can be answered:
You run your timer callback in Task.Run, so it is NOT running in the main thread
If you wanted to run HttpClient in the UI thread it may prevent the overlap but expect your app to become completely unresponsive.
To prevent the overlap you may use several methods but most likely you are looking for SemaphoreSlim
I have several asynec methods.
One of them triggers a POST method which start a process. I then need to 'sample' the results of another GET method every 10 minutes, and check if the status has changed from "pending" to "success" .
I tryed usingSystem.Threading.Timer with no luck, complaining about my method being asynced .
Error CS0407 'Task Campaigns.repeat(object)' has the wrong return type Campaigns
This is my code:
public async Task waitForCampaignLoadAsync(string Uri)
{
...........
var container = JsonConvert.DeserializeObject<CampaignTempleteStatus>(json);
if(container.status == "pending")
{
var autoEvent = new AutoResetEvent(false);
//The next row triggers the error
var stateTimer = new Timer(repeat, autoEvent, 1000, (1000 * 60 * 10));
//How can I keep repeating this, until (bool isFinished = true)??
}
public async Task repeat(Object stateInfo)
{
if(...)
isFinished = true;
}
Another thing is , how do I pass extra info inside repeat function? I need to pass the Uri input for inner ussage ?
When an asynchronous method starts getting complicated it's a sure sign something is wrong. Most of the time async code looks almost the same as synchronous code with the addition of await.
A simple polling loop could be as simple as :
public async Task<string> waitForCampaignLoadAsync(string uri)
{
var client=new HttpClient();
for(int i=0;i<30;i++)
{
token.ThrowIfCancellationRequested();
var json = await client.GetStringAsync(uri);
var container = JsonConvert.DeserializeObject<CampaignTempleteStatus>(json);
if (container.status != "pending")
{
return container.status;
}
await Task.Delay(10000);
}
return "Timed out!";
}
Cancellation in managed threads explains how CancellationTokenSource and CancellationToken can be used to cancel threads, tasks and asynchronous functions. Many asynchronous methods already provide overloads that accept a CancellationToken parameter. The polling function could be modified to accept and check a canellation token :
public async Task<string> waitForCampaignLoadAsync(string uri,CancellationToken token=default)
{
var client=new HttpClient();
for(int i=0;i<30;i++)
{
var json = await client.GetStringAsync(uri);
var container = JsonConvert.DeserializeObject<CampaignTempleteStatus>(json);
if (container.status != "pending")
{
return container.status;
}
await Task.Delay(10000,token);
}
return "Timed out!";
}
A CancellationTokenSource can be used to call this method with an overall timeout of eg, 5 minutes :
var cts=new CancellationTokenSource(TimeSpan.FromMinutes(5));
try
{
var result=waitForCampaignLoadAsync(uri,cts.Token);
//Process the result ....
}
catch(OperationCancelledExcepction ex)
{
//Handle the timeout here
}
This code can be improved. For example, GetStringAsync() doesn't accept a cancellation token. The operation can be broken in two steps though, one call to GetAsync() with a cancellation token that waits for the server to send a result
and another to HttpContent.ReadAsStringAsync() to read the response, eg :
var response=await client.GetAsync(uri,token)
response.EnsureSuccessStatusCode();
var json=await response.Content.ReadAsStringAsync();
...
The first parameter of Timer is a TimerCallback delegate, which should return void
var stateTimer = new Timer(Repeat, autoEvent, 1000, (1000 * 60 * 10));
private void Repeat(object state)
{
....
}
I have a WinForms application that has two roles. If no command line parameters are present, the Main function calls Application.Run, and presents the UI. If command line parameters are present, Application.Run is NOT called. Instead, I call an async method like this:
result = HandleCommandLine(args).GetAwaiter().GetResult();
(I am new to async/await, and this form was based on a SO answer).
The end goal is to loop through a list, and for each entry, start a new task. Each of those tasks should run in parallel with the others. The tasks are started like this:
runningTasks.Add(Task.Factory.StartNew((args) => HandlePlayback( (Dictionary<string,string>) ((object[])args)[0]), new object[] { runArgs } ));
The tasks are added to the collection of runningTasks, and I later call:
Task.WaitAll(runningTasks.ToArray());
In each of the runningTasks, I am trying to send web requests using HttpClient:
using (HttpResponseMessage response = await Client.SendAsync(message))
{
using (HttpContent responseContent = response.Content)
{
result = await responseContent.ReadAsStringAsync();
}
}
Once Client.SendAsync is called, the whole thing goes belly up. All of my runningTasks complete, and the application exits. Nothing past the Client.SendAsync executes in any of those tasks.
Since I am new at async/await, I have very few ideas about what exactly might be wrong, and hence few ideas about how to fix it. I imagine it has something to do with the SynchronizationContexts in this situation (WinForms app acting like a console app), but I'm not grasping what I need to do and where to keep the service request and the web request async calls from causing everything to complete too early.
I guess my question then is, why are (only some) 'awaited' calls causing all tasks to complete? What can I do about it?
UPDATE:
Two things. #Joe White: The WindowsFormsSynchronizationContext.Current is always null wherever I check.
#David Pine: Minimal (kind of :) ) complete viable example follows. You will either need to add a command line argument to the project, or force execution to the HandleCommandLine function. In this example, it tries to make a website request for each of three sites. It doesn't appear to matter if they exist. The code reaches the Client.SendAsync some number of times (usually not three), but timing appears to matter.
using System;
using System.Collections.Generic;
using System.Net;
using System.Net.Http;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
namespace WindowsFormsApplication1
{
static class Program
{
static List<Task> runningTasks = new List<Task>();
[STAThread]
static int Main()
{
int result = 1; // true, optimism
Application.EnableVisualStyles();
Application.SetCompatibleTextRenderingDefault(false);
string[] args = Environment.GetCommandLineArgs();
if (args.Length > 1)
{
// do the command line work async, while keeping this thread active
result = HandleCommandLine(args).GetAwaiter().GetResult();
}
else
{
// normal interface mode
Application.Run(new Form1());
}
return result;
}
static async Task<int> HandleCommandLine(string[] args)
{
// headless mode
int result = 1; // true, optimism
result = await HandleControlMode(args);
return result;
}
private static async Task<int> HandleControlMode(string[] Arguments)
{
int result = 1; // optimism
try
{
List<string> sites = new List<string>() { #"http://localhost/site1", #"http://localhost/site2", #"http://localhost/site3" };
foreach (string site in sites)
{
Begin(site); // fire off tasks
// the HandleControlMode method is async because in other circumstances, I do the following:
//await Task.Delay(5000); // sleep 5 seconds
}
// wait while all test running threads complete
try
{
Task.WaitAll(runningTasks.ToArray());
}
catch (Exception)
{
// not really a catch all handler...
}
}
catch (Exception)
{
// not really a catch all handler...
}
return result;
}
private static void Begin(string site)
{
//runningTasks.Add(Task.Factory.StartNew(() => HandlePlayback(runArgs)));
runningTasks.Add(Task.Factory.StartNew((args) => HandlePlayback((string)((object[])args)[0]), new object[] { site }));
}
private static async Task<int> HandlePlayback(string site)
{
int result = 1;
try
{
PlaybackEngine engine = new PlaybackEngine(site);
bool runResult = await engine.RunCommandLine(site);
if (!runResult)
{
result = 0;
}
}
catch (Exception)
{
result = 0;
}
return result;
}
}
public class PlaybackEngine
{
private static HttpClientHandler ClientHandler = new HttpClientHandler()
{
AllowAutoRedirect = false,
AutomaticDecompression = System.Net.DecompressionMethods.GZip | DecompressionMethods.Deflate
};
private static HttpClient Client = new HttpClient(ClientHandler);
public string Target { get; set; }
public PlaybackEngine(string target)
{
Target = target;
}
public async Task<bool> RunCommandLine(string site)
{
bool success = true;
string response = await this.SendRequest();
return success;
}
private async Task<string> SendRequest()
{
string result = string.Empty;
string requestTarget = Target;
HttpMethod method = HttpMethod.Post;
var message = new HttpRequestMessage(method, requestTarget);
StringContent requestContent = null;
requestContent = new StringContent("dummycontent", Encoding.UTF8, "application/x-www-form-urlencoded");
message.Content = requestContent;
try
{
using (HttpResponseMessage response = await Client.SendAsync(message))
{
using (HttpContent responseContent = response.Content)
{
result = await responseContent.ReadAsStringAsync();
System.Diagnostics.Debug.WriteLine(result);
}
}
}
catch (Exception ex)
{
}
return result;
}
}
}
UPDATE2:
I put similar code online at http://rextester.com/CJS33330
It's a straight console app, and I've added .ConfigureAwait(false) to all awaits (with no effect). In separate testing, I tried 4 or 5 other ways to call the first async function from Main - which all worked but had the same behavior.
The problem with this code is that I am not waiting on the Tasks that I thought I was. The runningTasks collection accepts any kind of Task. I didn't realize that Task.Factory.StartNew returned different type than the Task I was trying to start. My function returns
Task<int>
but StartNew returns
Task<Task<int>>
Those tasks completed immediately, and so the main thread did not stay alive long enough for the actual routines to run. You have to wait on the inner task instead:
Task<Task<int>> wrappedTask = Task.Factory.StartNew(...);
Task<int> t = await wrappedTask;
runningTasks.Add(t);
...
Task allTasks = Task.WhenAll(runningTasks.ToArray());
await allTasks;
For some reason, I was not able to use the built in ".Unwrap" function that should be equivalent, but the above code does the job.
I have made a class to handle multiple HTTP GET requests. It looks something like this:
public partial class MyHttpClass : IDisposable
{
private HttpClient theClient;
private string ApiBaseUrl = "https://example.com/";
public MyHttpClass()
{
this.theClient = new HttpClient();
this.theClient.BaseAddress = new Uri(ApiBaseUrl);
this.theClient.DefaultRequestHeaders.Accept.Clear();
this.theClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue("application/json"));
}
public async Task<JObject> GetAsync(string reqUrl)
{
var returnObj = new JObject();
var response = await this.theClient.GetAsync(reqUrl);
if (response.IsSuccessStatusCode)
{
returnObj = await response.Content.ReadAsAsync<JObject>();
Console.WriteLine("GET successful");
}
else
{
Console.WriteLine("GET failed");
}
return returnObj;
}
public void Dispose()
{
theClient.Dispose();
}
}
I am then queueing multiple requets by using a loop over Task.Run() and then after Task.WaitAll() in the manner of:
public async Task Start()
{
foreach(var item in list)
{
taskList.Add(Task.Run(() => this.GetThing(item)));
}
Task.WaitAll(taskList.ToArray());
}
public async Task GetThing(string url)
{
var response = await this.theClient.GetAsync(url);
// some code to process and save response
}
It definitiely works faster than synchonus operation but it is not as fast as I expected. Based on other advice I think the local threadpool is slowing me down. MSDN suggest I should specify it as a long running task but I can't see a way to do that calling it like this.
Right now I haven't got into limiting threads, I am just doing batches and testing speed to discover the right approach.
Can anyone suggest some areas for me to look at to increase the speed?
So, after you've set your DefaultConnectionLimit to a nice high number, or just the ConnectionLimit of the ServicePoint that manages connections to the host you are hitting:
ServicePointManager
.FindServicePoint(new Uri("https://example.com/"))
.ConnectionLimit = 1000;
the only suspect bit of code is where you start everything...
public async Task Start()
{
foreach(var item in list)
{
taskList.Add(Task.Run(() => this.GetThing(item)));
}
Task.WaitAll(taskList.ToArray());
}
This can be reduced to
var tasks = list.Select(this.GetThing);
to create the tasks (your async methods return hot (running) tasks... no need to double wrap with Task.Run)
Then, rather that blocking while waiting for them to complete, wait asynchronously instead:
await Task.WhenAll(tasks);
You are probably hitting some overhead in creating multiple instance-based HttpClient vs using a static instance. Your implementation will not scale. Using a shared HttpClient is actually recommended.
See my answer why - What is the overhead of creating a new HttpClient per call in a WebAPI client?