We Keep Coding

Async HTTP requests inside for loop without await asynchronous operations concurrently

I'm building a solution to find a desired value from an API call inside a for loop.
I basically need to pass to an API the index of a for loop, one of those index will return a desired output, in that moment I want the for loop to break, but I need to efficient this process. I thought making it asynchronous without the await, so that when some API returns the desired output it breaks the loop.
Each API call takes around 10sec, so if I make this async or multithread I would reduce the execution time considerably.
I haven't fount any good orientation of how making async / not await HTTP request.
Any suggestions?
for (int i = 0; i < 60000; i += 256)
{
Console.WriteLine("Incrementing_value: " + i);
string response = await client.GetStringAsync(
"http://localhost:7075/api/Function1?index=" + i.ToString());
Console.WriteLine(response);
if (response != "null")//
{
//found the desired output
break;
}
}

You can run requests in parallel, and cancel them once you have found your desired output:
public class Program {
public static async Task Main() {
var cts = new CancellationTokenSource();
var client = new HttpClient();
var tasks = new List<Task>();
// This is the number of requests that you want to run in parallel.
const int batchSize = 10;
int requestId = 0;
int batchRequestCount = 0;
while (requestId < 60000) {
if (batchRequestCount == batchSize) {
// Batch size reached, wait for current requests to finish.
await Task.WhenAll(tasks);
tasks.Clear();
batchRequestCount = 0;
}
tasks.Add(MakeRequestAsync(client, requestId, cts));
requestId += 256;
batchRequestCount++;
}
if (tasks.Count > 0) {
// Await any remaining tasks
await Task.WhenAll(tasks);
}
}
private static async Task MakeRequestAsync(HttpClient client, int index, CancellationTokenSource cts) {
if (cts.IsCancellationRequested) {
// The desired output was already found, no need for any more requests.
return;
}
string response;
try {
response = await client.GetStringAsync(
"http://localhost:7075/api/Function1?index=" + index.ToString(), cts.Token);
}
catch (TaskCanceledException) {
// Operation was cancelled.
return;
}
if (response != "null") {
// Cancel all current connections
cts.Cancel();
// Do something with the output ...
}
}
}
Note that this solution uses a simple mechanism to limit the amount of concurrent requests, a more advanced solution would make use of semaphores (as mentioned in some of the comments).

There are multiple ways to solve this problem. My personal favorite is to use an ActionBlock<T> from the TPL Dataflow library as a processing engine. This component invokes a provided Action<T> delegate for every data element received, and can also be provided with an asynchronous delegate (Func<T, Task>). It has many useful features, including (among others) configurable degree of parallelism/concurrency, and cancellation via a CancellationToken. Here is an implementation that takes advantage of those features:
async Task<string> GetStuffAsync()
{
var client = new HttpClient();
var cts = new CancellationTokenSource();
string output = null;
// Define the dataflow block
var block = new ActionBlock<string>(async url =>
{
string response = await client.GetStringAsync(url, cts.Token);
Console.WriteLine($"{url} => {response}");
if (response != "null")
{
// Found the desired output
output = response;
cts.Cancel();
}
}, new ExecutionDataflowBlockOptions()
{
CancellationToken = cts.Token,
MaxDegreeOfParallelism = 10 // Configure this to a desirable value
});
// Feed the block with URLs
for (int i = 0; i < 60000; i += 256)
{
block.Post("http://localhost:7075/api/Function1?index=" + i.ToString());
}
block.Complete();
// Wait for the completion of the block
try { await block.Completion; }
catch (OperationCanceledException) { } // Ignore cancellation errors
return output;
}
The TPL Dataflow library is built-in the .NET Core / .NET 5. and it is available as a package for .NET Framework.
The upcoming .NET 6 will feature a new API Parallel.ForEachAsync, that could also be used to solve this problem in a similar fashion.

Nsubstitute or moq an Async method so it will return a faulted task instead of exception

In the below code for each request we are calling rest client in an Async manner. Rest client is basically a http client which is calling some webapis in a batch. If suppose AsyncTaskCount value is 5 then 5 request will be called asynchronously and then in the while block we are getting the result for each call. If any response out of those 5 request has an exception then the response will be faulted and IsFaulted becomes true for that particular request and in the response we can get the inner exception.
private async Task<List<RequestResponse>> ProcessInvestment(List<Request> Requests, List<Result> Results, ILogger log)
{
var requestResponses = new List<RequestResponse>();
var asyncTaskCount = Convert.ToInt32(Environment.GetEnvironmentVariable("AsyncTaskCount"));
log.LogInformation($"Start processing {Requests.Count} in batches of {asyncTaskCount}");
for (int index = 0; index < Requests.Count; index = index + asyncTaskCount)
{
var requestBatch = Requests.Skip(index).Take(asyncTaskCount).ToList();
var requests = requestBatch.Select(x => _restClient.RequestResponse(x)).ToList();
while (requests.Count > 0)
{
// Identify the first task that completes.
Task<RequestResponse> requestResponseTask = await Task.WhenAny(requests);
var requestResponse = new RequestResponse();
// ***Remove the selected task from the list so that you don't process it more than once
requests.Remove(requestResponseTask);
if (!requestResponseTask.IsFaulted)
{
// Await the completed task.
requestResponse = await requestResponseTask;
requestResponses.Add(requestResponse);
}
else
{
if (requestResponseTask.Exception.InnerException != null && requestResponseTask.Exception.InnerException is Exception)
{
var result = new Result();
result = ResponseTransformComponent.ResponseToResult(((Exception)requestResponseTask.Exception.InnerException).Request, null);
result.SetBadRequestErrorDetails(((Exception)RequestResponseTask.Exception.InnerException).BadRequestResponse);
results.Add(Result);
}
else
{
throw requestResponseTask.Exception;
}
}
}
log.LogInformation($"Number of records processed = {requestResponses.Count}");
}
log.LogInformation($"Total invalid and Bad requests count = {results.Count}");
return RequestResponses;
}
Below is the code for restclient which is called from the above method.
public async Task<Response> RequestResponse(Request request)
{
var response = await GetDataFromService("calculation", "CalculateCapital", request);
return JsonConvert.DeserializeObject<Response>(response);
}
public async Task<string> GetDataFromService(string controller, string method, object request)
{
var client = _httpClientFactory.CreateClient(ServiceEnum.DCS);
string baseAddress = client.BaseAddress.ToString();
var requestUrl = $"api/{controller}/{method}";
var content = new StringContent(JsonConvert.SerializeObject(request), Encoding.UTF8, "application/json");
var response = await client.PostAsync(requestUrl, content).ConfigureAwait(false);
if (!response.IsSuccessStatusCode)
{
var responseResult = response.Content.ReadAsStringAsync().Result;
if (response.StatusCode == HttpStatusCode.BadRequest)
{
throw new CalculatorServiceException("Bad Request", JsonConvert.DeserializeObject<BadRequestResponse>(responseResult), (Request)request);
}
throw new Exception($"Status code: {response.StatusCode}. {responseResult}");
}
return response.Content.ReadAsStringAsync().Result;
}
GetDataFromService method is called from Calculate method. And GetDataFromService method will return a custom exception if the request is a Bad Request.
I am trying to write the unit test case for the above method and try to mock a request so that it will return a faulted task and then IsFaulted should become true. Below is the part of my unit test case.
_restClient
.When(a => a.RequestResponse(Arg.Any<Request>()))
.Do(a => {Task.FromException(new CalculatorServiceException(string.Empty, new BadRequestResponse { Message = string.Empty, ModelState = new Dictionary<string, string[]> { { "CalculationDates.StartDate", new string[] { "0002: Duration too short to execute calculations (CalculationDates.StartDate)" } } } }, Arg.Any<Request>())); });
If i mock my restclient method like above then it is throwing the exception instead of giving the response with IsFaulted to true. So how should i mock the restclient method so that it will return a faulted task which has an exception instead of throwing it. Please suggest.
Thanks in advance.

When..Do is for wiring up callbacks for when a member is called.
Try using Returns instead:
_restClient.RequestResponse(Arg.Any<Request>())
.Returns(x => Task.FromException<RequestResponse>(...));
// (assuming RequestResponse(..) returns a Task<RequestResponse>. Tweak as required)

How to wait for a callback before returning

I am trying to port a code from .Net to Unity C# and I am stuck on a syntax including a callback.
Basically, I had to replace the .Net 'HttpClient' library by this one, also called 'HttpClient'. But the 'Get' syntax is not the same and uses a Callback. I am quite new to C# and Http queries and don't know how to deal with this syntax to get the expected return.
The original function written in .Net:
internal static JObject GetToBackOffice(string action)
{
var url = backOfficeUrl;
url = url + action;
HttpClient httpClient = new HttpClient();
var response =
httpClient.GetAsync(url
).Result;
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
var idProcess = Newtonsoft.Json.Linq.JObject.Parse(response.Content.ReadAsStringAsync().Result);
return idProcess;
}
else
return null;
The C# code I am writing for Unity:
internal class Utils
{
internal static JObject GetToBackOffice(string action)
{
var url = backOfficeUrl;
url = url + action;
HttpClient httpClient = new HttpClient();
JObject idProcess = new JObject();
httpClient.GetString(new Uri(url),
(response) =>
{
// Raised when the download completes
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
idProcess = Newtonsoft.Json.Linq.JObject.Parse(response.Data);
}
else
{
idProcess = null;
}
});
// Here I would like to wait for the response, so that idProcess is filled with the received data before returning
return idProcess;
}
public class Action
{
public bool SendData(string id, string secretKey, FakeData data)
{
var idProcess = Utils.GetToBackOffice(String.Format("Events/{0}/infos",id));
//...I do then something with idProcess
//Currently, when I use idProcess here, it is still empty since the GetString response hasn't been received yet when this line is executed
return true;
}
}
public class EventHubSimulator : MonoBehaviour
{
void Start()
{
//Fill the parameters (I skip the details)
string oId = ...;
string secretKey = ...;
var vh = ...;
Action action = new Action();
action.SendData(oId, secretKey, vh);
}
}
My issue is that after the GetToBackOffice function, my code directly uses 'idProcess' for something else but this object is empty because the response was not received yet. I would like to wait for the response before my function returns.
I hope I was clear enough. I know that similar question have already been posted but couldn't find a solution to my specific issue.
Edit:
Finally I used a coroutine as Nain suggested but couldn't get what I expected the way he said. This way seems to work (event though it might not be a good way to do it).
public class EventHubSimulator : MonoBehaviour
{
void Start()
{
//Fill the parameters (I skip the details)
string oId = ...;
string secretKey = ...;
var vh = ...;
Utils utils = new Utils();
StartCoroutine(utils.SendData(oId, secretKey, vh));
}
}
public class Utils: MonoBehaviour
{
private const string backOfficeUrl = "http://myurl/api/";
public CI.HttpClient.HttpResponseMessage<string> response;
public IEnumerator SendData(string id, string secretKey, FakeData data)
{
response = null;
yield return GetToBackOffice(String.Format("Events/{0}/infos", id)); //Make a Http Get request
//The next lines are executed once the response has been received
//Do something with response
Foo(response);
}
IEnumerator GetToBackOffice(string action)
{
var url = backOfficeUrl;
url = url + action;
//Make a Http Get request
HttpClient httpClient = new HttpClient();
httpClient.GetString(new Uri(url), (r) =>
{
// Raised when the download completes
if (r.StatusCode == System.Net.HttpStatusCode.OK)
{
//Once the response has been received, write it in the global variable
response = r;
Debug.Log("Response received : " + response);
}
else
{
Debug.Log("ERROR =============================================");
Debug.Log(r.ReasonPhrase);
throw new Exception(r.ReasonPhrase);
}
});
//Wait for the response to be received
yield return WaitForResponse();
Debug.Log("GetToBackOffice coroutine end ");
}
IEnumerator WaitForResponse()
{
Debug.Log("WaitForResponse Coroutine started");
//Wait for response to become be assigned
while (response == null)
{
yield return new WaitForSeconds(0.02f);
}
Debug.Log("WaitForResponse Coroutine ended");
}
}

One solution is polling for completion as Nain submitted as an answer. If you don't want polling you can use a TaskCompletionSource. This Q&A dives a bit deeper into the why and how.
Your code can then be written like this:
async Task CallerMethod()
{
JObject result = await GetToBackOffice(...);
// Do something with result
}
internal static Task<JObject> GetToBackOffice(string action)
{
var tsc = new TaskCompletionSource<JObject>();
var url = backOfficeUrl;
url = url + action;
HttpClient httpClient = new HttpClient();
JObject idProcess = new JObject();
httpClient.GetString(new Uri(url),
(response) =>
{
// Raised when the download completes
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
tsc.SetResult(Newtonsoft.Json.Linq.JObject.Parse(response.Data));
}
else
{
tsc.SetResult(null);
}
});
return tsc.Task;
}
See also the section Async Method Calls a Coroutine And Wait for Completion of this msdn blogpost.
NOTE Tasks, and async/await support is only available as beta functionality in Unity. See also this post.

Write a coroutine like
//Class scope variable is neede to hold Response other wise it will
//be destroied as soon as function is ended
ResponseType response;
IEnumerator WaitForResponce(ResponseType response)
{
this.response = response;
while(this.response.Data == null)
yield return new WaitForSeconds (0.02f);
//do what you want here
}
and call the coroutine
httpClient.GetString(new Uri(url),
(response) =>
{
// Raised when the download completes
if (response.StatusCode == System.Net.HttpStatusCode.OK)
{
//idProcess = Newtonsoft.Json.Linq.JObject.Parse(response.Data);
StartCoroutine(WaitForResponse(response));
}
else
{
idProcess = null;
}
});

If the GetString method is returning a Task object then you can use the Wait() method to let the Task finished processing before a result is returned.
var task = httpClient.GetString({
impl here..
});
task.Wait();
return idProcess;

DateTime difference inside Thread

I have a for-loop that creates a new Thread each iteration. In short, my loop is creating 20 threads that does some action, at the same time.
My goal inside each of these threads, is to create a DateTime variable with a start time, execute an operation, and create a DateTime variable with an end time. Hereafter I'll take the difference between these two variables to find out, how long this operation took in this SPECIFIC thread. Then log it out.
However that isn't working as expected, and I'm confused on why.
It seems like it justs "adds" the time to the variables, each iteration of a new thread, instead of creating a completely new and fresh version of the variable, only to be taking into consideration in that specific thread.
This is my for-loop code:
for(int i = 0; i < 20; i++)
{
Thread thread = new Thread(() =>
{
Stopwatch sw = new Stopwatch();
sw.Start();
RESTRequest(Method.POST, ....),
sw.Stop();
Console.WriteLine("Result took (" + sw.Elapsed.Seconds + " seconds, " + sw.Elapsed.Milliseconds + " milliseconds)");
});
thread.IsBackground = true;
thread.Start();
}
Long operation function:
public static string RESTRequest(Method method, string endpoint, string resource, string body, SimplytureRESTRequestHeader[] requestHeaders = null, SimplytureRESTResponseHeader[] responseHeaders = null, SimplytureRESTAuthentication authentication = null, SimplytureRESTParameter[] parameters = null)
{
var client = new RestClient(endpoint);
if(authentication != null)
{
client.Authenticator = new HttpBasicAuthenticator(authentication.username, authentication.password);
}
var request = new RestRequest(resource, method);
if (requestHeaders != null)
{
foreach (var header in requestHeaders)
{
request.AddHeader(header.headerType, header.headerValue);
}
}
if(body != null)
{
request.AddParameter("text/json", body, ParameterType.RequestBody);
}
if(parameters != null)
{
foreach (var parameter in parameters)
{
request.AddParameter(parameter.key, parameter.value);
}
}
IRestResponse response = client.Execute(request);
if (responseHeaders != null)
{
foreach (var header in responseHeaders)
{
var par = new Parameter();
par.Name = header.headerType;
par.Value = header.headerValue;
response.Headers.Add(par);
}
}
var content = response.Content;
return content;
}
This is my results:
EDIT:
I also tried using the Stopwatch class, but it didn't do any difference, but definitely more handy. I also Added the long operation for debugging.

There is a limitation for concurrent calls to the same ServicePoint.
The default is 2 concurrent connections for each unique ServicePoint.
Add System.Net.ServicePointManager.DefaultConnectionLimit = 20; to raise that limit to match the thread count.
System.Net.ServicePointManager.DefaultConnectionLimit
You can also set this value in config file
<system.net>
<connectionManagement>
<add address="*" maxconnection="20" />
</connectionManagement>
</system.net>

Creating a load tester for an upload API with UploadFileAsync

What I am trying to accomplish here is to see how much this API can handle as far as requests per second. I am trying to consume the API in a console app that will ultimately be throwaway code. My idea was to make a for loop that would try to upload an xml document every 2 seconds. I've never done this sort of thing before so forgive my ignorance. Here's my Main method:
static void Main()
{
RunAsync().Wait();
}
And the RunAsync method:
static async Task RunAsync()
{
Uri apiUrl = new Uri("http://apiurl.com/upload/files/uploadfiles");
const string file = #"C:\simple.xml";
WebClient client = new WebClient();
for (int i = 0; i <= 100; i++)
{
client.UploadFileCompleted += FileUploadSuccess;
client.UploadFileAsync(apiUrl, file);
await Task.Delay(2000);
Console.WriteLine("Upload waiting 2 seconds...");
}
Console.WriteLine("Loop completed.");
}
And the success method:
private static void FileUploadSuccess(object sender, UploadFileCompletedEventArgs e)
{
string reply = System.Text.Encoding.UTF8.GetString(e.Result);
Console.WriteLine("The file result was: {0}", reply);
}
It throws an exception on the first time through on e.Result. Here's the exception:
An unhandled exception of type 'System.Reflection.TargetInvocationException' occurred in System.dll
After doing some research, apparently I can't call the API method (which returns an async Task) without await'ing it. Unfortunately it seems UploadFileAsync is not "awaitable."
Here's the API method:
public async Task<HttpResponseMessage> UploadFiles()
{
var pilotTokenObject = TokenHelper.CreatePilotTokenObject(Request);
byte[] fileBuffer = null;
HttpResponseMessage retVal = null;
if (pilotTokenObject != null)
{
var content = Request.Content;
if (content == null)
{
throw new PilotApiException("Empty request content", HttpStatusCode.NoContent);
}
if (!content.IsMimeMultipartContent())
{
throw new PilotApiException("Request does not contain not multi-part content");
}
var uploadModelController = new PilotUploadModelController();
//*SAVE STREAMED FILE*
string serverSavePath = ConfigurationManager.AppSettings["PilotUploadApiTempStoragePath"];
if (!Directory.Exists(serverSavePath))
Directory.CreateDirectory(serverSavePath);
var provider = new MultipartFormDataStreamProvider(serverSavePath);
await Request.Content.ReadAsMultipartAsync(provider);
var fileData = provider.FileData;
if (fileData == null || fileData.Count == 0)
{
throw new PilotApiException("No multipart/form file data present.");
}
bool uploaded = false;
//Loop through each file
fileData.ForEach((fileRequest) =>
{
if (RetryUntilFileReadable(Path.Combine(serverSavePath, fileRequest.LocalFileName), 1000, 5))
{
var fileHeader = fileRequest.Headers;
if (fileHeader != null && fileHeader.ContentDisposition != null)
{
var fileName = fileHeader.ContentDisposition.FileName.Replace("\"", "");
var fileBytes = File.ReadAllBytes(Path.Combine(serverSavePath, fileRequest.LocalFileName));
//Save File to DB
var upload = uploadModelController.UploadHelper
.AddUploadFileToDb(pilotTokenObject.CentralUserDbUserId, pilotTokenObject.ClientIp, pilotTokenObject.UserAgentString,
UploadEnums.UploadKind.PilotUploadApi, fileName, fileBytes.Length, fileBytes,
UploadEnums.EncryptionType.None);
if (upload != null)
uploaded = true;
}
}
});
if (uploaded)
{
retVal = Request.CreateResponse(HttpStatusCode.Accepted, new
{
Response = String.Format("file uploaded successfully.")
});
}
}
return retVal;
}
Am I going about this the completely wrong way? Is what I want to do even doable?

It seems to me that the following would work better in your scenario:
byte[] response = await Task.Run(() => client.UploadFile(apiUrl, file));
string reply = System.Text.Encoding.UTF8.GetString(response);
Console.WriteLine("The file result was: {0}", reply);
Console.WriteLine("Upload waiting 2 seconds...");
await Task.Delay(2000);
Trying to mix-and-match the older asynchronous API with the newer async/await doesn't seem fruitful in this case. Better to just wrap the synchronous version of the API with async/await-compatible code.
(Note that it seems to me you could just as well call Thread.Sleep(2000) instead of creating a new delay task to wait on, but the above should work fine too).