We Keep Coding

Asp.Net Core Web API returns the result while client is hanging

I have a simple Asp.Net Core Web API with .NET 6, running on IIS 10, Windows 10. The web API calls another API and returns the results. Below is a simplified version of its code but I tried to keep the most important parts.
[ApiController]
[Produces("application/json")]
public class SomeController
{
private async Task<ApiOutput> RunApiForClientAsync(ApiInput input)
{
try
{
//create a httpclient with a lot of configuration
var response = await client.SendAsync(request);
var content = await response.Content.ReadAsStringAsync();
return new ApiOutput
{
Data = content,
Error = null,
StatusCode = 200,
Input = input,
};
}
catch(Exception ex)
{
return new ApiOutput
{
Data = null,
Error = new ApiError("Error Getting the Result from the Server", ex.Message),
StatusCode = 400,
Input = input,
};
}
}
private async Task<List<ApiOutput>> RunApiCallsAsync(string requestId, IEnumerable<ApiInput> items)
{
var result = new List<ApiOutput>();
var tasks = new List<Task<ApiOutput>>();
foreach(var item in items)
{
tasks.Add(RunApiForAsync(item));
}
var taskResults = await Task.WhenAll(tasks);
result.AddRange(taskResults);
return result;
}
[HttpPost]
[Route("rest/multiple")]
public async Task<IActionResult> PostMultiple(ApiInput[] models, string? requestId)
{
_logger.LogInformation(ApiLoggingEvents.PostMultiple, "Request received with ID {requestId}", requestId);
var result = await RunApiCallsAsync(requestId, models);
try
{
_logger.LogDebug(ApiLoggingEvents.PostMultiple, "Request ID {requestId} Generating JSONs.", requestId);
var resultJson = GetJson(result);
await SaveResultAsync(resultJson, requestId);
_logger.LogDebug(ApiLoggingEvents.PostMultiple, "Request ID {requestId} Everything is finished. Returning....", requestId);
return Content(resultJson, "application/json");
}
catch (Exception ex)
{
_logger.LogDebug(ApiLoggingEvents.PostMultiple, "Exception while returning {requestId}, message {msg}", requestId, ex.Message);
throw new Exception("Try again");
}
}
}
Every once in a while, the caller sends the request to the API but never gets the result back. However, when I read the logs, I see the last line for the request is the line containing the text "Everything is finished. Returning" which means everything was successful. In addition, the output JSON is saved on the server's local drive (the await SaveResultAsync(resultJson, requestId); call is successful too).
I should mention that these types of requests are the ones that take a long while to respond. Usually around 10 minutes. Is there a setting that I need to change on the application or the IIS?
I tried to use the following but it doesn't work with the In-Process model:
builder.WebHost.UseKestrel(o =>
{
o.Limits.MaxConcurrentConnections = 100;
o.Limits.KeepAliveTimeout = TimeSpan.FromMilliseconds(timeout);
o.Limits.MaxRequestBodySize = int.MaxValue;
o.Limits.MaxResponseBufferSize = int.MaxValue;
});
Note:
The requestId is a unique GUID for every request to help me keep track of each request on the log file and see whether it was successful or not and if it has created the output file.
Update:
Upon further investigation, it seems like the requests that have a runtime more than 5 minutes are failing. Any idea what might be related to this number?
Update 2:
I created a very simple endpoint that waits for a specified amount of seconds, then returns back with a simple message:
[HttpPost]
[Route("rest/testpost")]
public IActionResult TestPost(int delay)
{
_logger.LogInformation(1, "Delay for {delay} started.", delay);
Thread.Sleep(delay * 1000);
_logger.LogInformation(1, "Delay for {delay} ended.", delay);
return Ok($"Delay for {delay} worked.");
}
I then added the requestTimeout="00:20:00" to the web.config file, just to make sure.
Interestingly, for values such as 310 seconds, sometimes I get the result, but sometimes I don't. (Postman still hangs)

To your web.config, add the following (the value is in seconds--so this will allow runtimes up to 20 minutes):
<system.web>
<httpRuntime executionTimeout="1200" />
</system.web>
Here is a link to the documentation: https://learn.microsoft.com/en-us/dotnet/api/system.web.configuration.httpruntimesection.executiontimeout?view=netframework-4.8

How to cancel async Task from the client

I have on ASP.Net C# web API with an endpoint for the import. Javascript client sends a list of items to this API and API process this list in another thread (long task) and immediately returns unique id (GUID) of process. Now I need the cancel the background task from the CLIENT. Is possible to somehow send the cancelation token from the client? I have tried to add CancellationToken as a parameter to my controller async action but I don't know how to pass it from the client. For simplification, we can use as the client the Postman app.
Sample server-side
[HttpPost]
[UserContextActionFilter]
[RequestBodyType(typeof(List<List<Item>>))]
[Route("api/bulk/ImportAsync")]
public async Task<IHttpActionResult> ImportAsync()
{
var body = await RequestHelper.GetRequestBody(this);
var queue = JsonConvert.DeserializeObject<List<List<Item>>>(body);
var resultWrapper = new AsynckResultWrapper(queue.Count);
HostingEnvironment.QueueBackgroundWorkItem(async ct =>
{
foreach (var item in queue)
{
var result = await ProcessItemList(item, false);
resultWrapper.AddResultItem(result);
}
});
return Ok(new
{
ProcessId = resultWrapper.ProcessId.ToString()
});
}
private async Task<ItemResult> ProcessItemList(<List<Item>>itemList, bool runInOneTransaction = false)
{
try
{
var result = await PerformBulkOperation(true, itemList);
return new ResultWrapper(result);
}
catch (Exception ex)
{
// process exception
return new ResultWrapper(ex);
}
}

On a high level what you could do is store the process id along with a cancellation token source when you queue the work. Then you can expose a new endpoint that accepts a process id, gets the cancellation token source from the store and cancels the associated token:
[HttpPost]
[UserContextActionFilter]
[RequestBodyType(typeof(List<List<Item>>))]
[Route("api/bulk/ImportAsync")]
public async Task<IHttpActionResult> ImportAsync()
{
var body = await RequestHelper.GetRequestBody(this);
var queue = JsonConvert.DeserializeObject<List<List<Item>>>(body);
var resultWrapper = new AsynckResultWrapper(queue.Count);
HostingEnvironment.QueueBackgroundWorkItem(async ct =>
{
var lts = CancellationTokenSource.CreateLinkedTokenSource(ct);
var ct = lts.Token;
TokenStore.Store(resultWrapper.ProcessId, lts);
foreach (var item in queue)
{
var result = await ProcessItemList(item, ct, false);
resultWrapper.AddResultItem(result);
}
TokenStore.Remove(processId) // remove the cancellation token source from storage when doen, because there is nothing to cancel
});
return Ok(new
{
ProcessId = resultWrapper.ProcessId.ToString()
});
}
private async Task<ItemResult> ProcessItemList(<List<Item>>itemList, CancellationToken token, bool runInOneTransaction = false)
{
try
{
var result = await PerformBulkOperation(true, itemList, token);
return new ResultWrapper(result);
}
catch (Exception ex)
{
// process exception
return new ResultWrapper(ex);
}
}
[Route("api/bulk/CancelImportAsync")]
public async Task<IHttpActionResult> CancelImportAsync(Guid processId)
{
var tokenSource = TokenStore.Get(processId);
tokenSource.Cancel();
TokenStore.Remove(processId) // remove the cancellation token source from storage when cancelled
}
In the above example I modified the ProcessItemList to accept a cancellation token and pass it to PerformBulkOperation, assuming that method has support for cancellation tokens. If not, you can manually call ThrowIfCancellationRequested(); on the cancellation token at certain points in the code to stop when cancellation is requested.
I've added a new endpoint that allows you to cancel a pending operation.
Disclaimer
There are for sure some things you need to think about, especially when it is a public api. You can extend the store to accepts some kind of security token and when cancellation is requested you check whether it matches with the security token that queued the work. My answer is focused on the basics of the question
Also, I left the implementation of the store to your own imagination ;-)

Asynchronous multiple Web Service requests with ordered result

Given a certain number of request objects (max 9), i need to call a web service endpoint the same number of times asynchronously. With .NET 4.0, we used delegate and IAsyncResult to achieve this.
Is there a better way to do this with asyc/await, TPL or both of them combined with .NET 4.6.1?
Will using Parallel.ForEach with ConcurrentBag be optimal as suggested in this answer?
Synchronous Code Example:
public List<WbsResponse> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
List<WbsResponse> results = new List<WbsResponse>();
foreach (var request in requests)
{
//Call same web service endpoint n number of times
var response = CallWebService(request);
results.Add(response);
}
//do something with results
return results;
}
private WbsResponse CallWebService(WbsRequest request)
{
//Call web service
}
Edit/Update 1: Based on #Thierry's answer, i've created a sample code assuming there's an Order property in both the request and response objects to mark the request/response ordering:
public List<WbsResponse> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
List<WbsResponse> results = new List<WbsResponse>();
Parallel.ForEach(requests, (request) => {
var response = CallWebService(request);
response.Order = request.Order;
results.Add(response);
});
results = results.OrderBy(r => r.Order).ToList();
//do something with results
return results;
}
private WbsResponse CallWebService(WbsRequest request)
{
//Call web service
}
Edit/Update 2: Based on this thread, i've made a few changes to Update 1:
await Task.Run(() => {
Parallel.ForEach(requests, (request) => {
var response = CallWebService(request);
response.Order = request.Order;
results.Add(response);
});
});
Requirement Summary:
Make multiple web service requests asynchronously to the same endpoint with different parameters.
Add web service results to a list in the same order as the request was made (as if it was synchronous).

Because each task finish with diffrent moment, I think you should numero the request and ordered the responses by this numero.
In the request, you init a numero and pass this numero for the response associated. Finally, when I have the results, I order it.
Like this:
public async Task<List<WbsResponse>> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
List<Task<WbsResponse>> tasks = new List<Task<WbsResponse>>();
for (var i = 0; i < requests.Count; i++)
{
var task = new Task<WbsResponse>(() => { CallWebService(WbsRequest); });
tasks.Add(task);
}
var responses = await Task.WhenAll(tasks);
var responsesOrdered = responses.OrderBy(r => r.Order)
//do something with results
return results;
}
public List<WbsRequest> CompileWbsRequests()
{
//create requests
foreach(var request in requests)
{
request.Order += 1;
}
}
private WbsResponse CallWebService(WbsRequest request)
{
//Call web service
reponse.order = request.order;
return reponse;
}

I think you can use Task.WaitAll to make the code work in async way and it will look prettier as well:
public List<WbsResponse> GetWbsResults()
{
List<WbsRequest> requests = CompileWbsRequests();
var responses = await Task.WhenAll(requests.Select(CallWebService));
return responses;
}
but you have to modify this method as below to return a task:
private async Task<WbsResponse> CallWebService(WbsRequest request)
{
//Call web service
}

Calling Asynchronous API in ASP.Net Application

I'm a little new to ASP.Net and Asynchronous coding so bear with me. I have written an asynchronous wrapper in C# for a web API that I would like to use in a ASP.Net application.
Here is one of the functions in the C# API wrapper:
public async Task<string> getProducts()
{
Products products = new Products();
products.data = new List<Item>();
string URL = client.BaseAddress + "/catalog/products";
string additionalQuery = "include=images";
HttpResponseMessage response = await client.GetAsync(URL + "?" + additionalQuery);
if (response.IsSuccessStatusCode)
{
Products p = await response.Content.ReadAsAsync<Products>();
products.data.AddRange(p.data);
while (response.IsSuccessStatusCode && p.meta.pagination.links.next != null)
{
response = await client.GetAsync(URL + p.meta.pagination.links.next + "&" + additionalQuery);
if (response.IsSuccessStatusCode)
{
p = await response.Content.ReadAsAsync<Products>();
products.data.AddRange(p.data);
}
}
}
return JsonConvert.SerializeObject(products, Formatting.Indented);
}
I then have a WebMethod in my ASP.Net application (which will be called using Ajax from a Javascript file) which should call the getProducts() function.
[WebMethod]
public static string GetProducts()
{
BigCommerceAPI api = getAPI();
return await api.getProducts();
}
Now of course this will not work as the WebMethod is not an async method. I have tried to change it to an async method which looked like:
[WebMethod]
public static async Task<string> GetProducts()
{
BigCommerceAPI api = getAPI();
return await api.getProducts();
}
This code does run, but as soon as it gets to the HttpResponseMessage response = await client.GetAsync(URL + "?" + additionalQuery); line in the getProducts() function the debugger will stop without any errors or data being returned.
What am I missing? How can I get call this asynchronous API from my ASP application?

So I actually resolved an issue very similar to this last night. It's odd because the call worked in .net 4.5. But we moved to 4.5.2 and the method started deadlocking.
I found these enlightening articles (here, here, and here) on async and asp.net.
So I modified my code to this
public async Task<Member> GetMemberByOrganizationId(string organizationId)
{
var task =
await
// ReSharper disable once UseStringInterpolation
_httpClient.GetAsync(string.Format("mdm/rest/api/members/member?accountId={0}", organizationId)).ConfigureAwait(false);
task.EnsureSuccessStatusCode();
var payload = task.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<Member>(await payload.ConfigureAwait(false),
new JsonSerializerSettings { ContractResolver = new CamelCasePropertyNamesContractResolver() });
}
which resolved my deadlocking issue.
So TLDR: from the Stephen Cleary article
In the overview, I mentioned that when you await a built-in awaitable,
then the awaitable will capture the current “context” and later apply
it to the remainder of the async method. What exactly is that
“context”?
Simple answer:
If you’re on a UI thread, then it’s a UI context. If you’re responding
to an ASP.NET request, then it’s an ASP.NET request context.
Otherwise, it’s usually a thread pool context. Complex answer:
If SynchronizationContext.Current is not null, then it’s the current
SynchronizationContext. (UI and ASP.NET request contexts are
SynchronizationContext contexts). Otherwise, it’s the current
TaskScheduler (TaskScheduler.Default is the thread pool context).
and the solution
In this case, you want to tell the awaiter to not capture the current
context by calling ConfigureAwait and passing false

I am not sure what is [WebMethod] in ASP.NET. I remember it used to be SOAP web services but no one does it anymore as we have Web API with controllers where you can use async/await in action methods.
One way to test your code would be to execute async method synchronously using .Result:
[WebMethod]
public static string GetProducts()
{
BigCommerceAPI api = getAPI();
return api.getProducts().Result;
}
As maccettura pointed out in the comment, it's a synchronous call and it locks the thread. To make sure you don't have dead locks, follow Fran's advice and add .ConfigureAwait(false) at the end of each async call in getProducts() method.

First by convention GetProducts() should be named GetProductsAsync().
Second, async does not magically allocate a new thread for it's method invocation. async-await is mainly about taking advantage of naturally asynchronous APIs, such as a network call to a database or a remote web-service.
When you use Task.Run, you explicitly use a thread-pool thread to execute your delegate.
[WebMethod]
public static string GetProductsAsync()
{
BigCommerceAPI api = getAPI();
return Task.Run(() => api.getProductsAsync().Result);
}
Check this link It's a project sample about how to implement Asynchronous web services call in ASP.NET

I had a very similar issue:
Main webapp is a ASP.NET 4.5 Web forms, but many of its functions implemented as AJAX calls from UI to a [webMethod] decorated function in the aspx.cs code-behind:
The webmethod makes an async call to a proxy. This call was
originally implemented with Task.Run() and I tried to rewrite with
just await ...
[WebMethod]
public static async Task<OperationResponse<CandidatesContainer>> GetCandidates(string currentRoleName, string customerNameFilter, string countryFilter, string currentQuarter)
{
string htmlResult = String.Empty;
List<CandidateEntryDTO> entries = new List<CandidateEntryDTO>();
try
{
entries = await GetCandiatesFromProxy(currentUser, currentRoleName, customerNameFilter, countryFilter, currentQuarter)
.ConfigureAwait(false);
}
catch (Exception ex)
{
log.Error("Error .....", ex);
}
CandidatesContainer payloadContainer = new CandidatesContainer {
CountryMappedCandiates = ...,
GridsHtml = htmlResult };
return new OperationResponse<CandidatesContainer>(payloadContainer, true);
}
3) The call GetCandiatesFromProxy(...) is the top of a chain of several async methods and at the bottom there's finally a HttpClient.GetAsync(...) call:
private async Task<B2PSResponse<string>> GetResponseFromB2PService(string serviceURI)
{
string jsonResultString = String.Empty;
if (_httpClientHandler == null)
{
_httpClientHandler = new HttpClientHandler() { UseDefaultCredentials = true };
}
if (_client == null)
{
_client = new HttpClient(_httpClientHandler);
}
HttpResponseMessage response = await _client.GetAsync(serviceURI).ConfigureAwait(false);
HttpContent content = response.Content;
string json = String.Empty;
if (response.StatusCode == HttpStatusCode.OK)
{
json = await content.ReadAsStringAsync().ConfigureAwait(false);
}
B2PSResponse<string> b2psResponse = new B2PSResponse<string>(response.StatusCode, response.ReasonPhrase, json);
return b2psResponse;
}
The code was not working (was stuck on the lowest level await) until
I started to add .ConfigureAwait(false) to each await call.
Interesting, that I had to add these .ConfigureAwait(false) to all await calls on the chain - all the way to the top call in the webMethod. Removing any of them would break the code - it would hang after the await that does not have the .ConfigureAwait(false).
The last point: I had to modify the Ajax call's SUCCESS path. The default Jason serialization for webmethods makes the result sent to AJAX call as
{data.d.MyObject}
i.e. inserts the {d} field containing the actual payload. After the webmethod return value was changed from MyObject to Task - this no longer worked - my payload was not found in the {data.d}. The result now contains
{data.d.Result.MyObject}
This is simply the result of serializing the Task object - which has the .Result field.
With one small change to the AJAX call is now working.

HttpClient PostAsync not working in CRM plugin

I am trying to send json to a web API using HttpClient.PostAsync. It works from a console application but not from my CRM plugin. Doing some research I noted that it is probably something to do with the context the plugin runs in and threading. Anyway here is my calling code:
public async void Execute(IServiceProvider serviceProvider)
{
IPluginExecutionContext context = (IPluginExecutionContext)serviceProvider.GetService(typeof(IPluginExecutionContext));
if (context.InputParameters.Contains("Target"))
{
if (context.InputParameters["Target"] is Entity)
{
Entity entity = (Entity)context.InputParameters["Target"];
if (entity.LogicalName == "new_product")
{
IOrganizationServiceFactory serviceFactory = (IOrganizationServiceFactory)serviceProvider.GetService(typeof(IOrganizationServiceFactory));
IOrganizationService service = serviceFactory.CreateOrganizationService(context.UserId);
try
{
if (entity.Contains("new_begindate") && entity.Contains("new_expirationdate"))
{
await OnlineSignUp(entity, service);
}
}
catch (InvalidPluginExecutionException)
{
throw;
}
catch (Exception e)
{
throw new InvalidPluginExecutionException(OperationStatus.Failed, "Error signing up: " + e.Message);
}
}
}
}
}
And here is the relevant code for sending the json:
private async Task<HttpResponseMessage> OnlineSignUp(Entity license, IOrganizationService service)
{
...
var json = JsonConvert.Serialize(invitation);
var content = new StringContent(json, Encoding.UTF8, "application/json");
var httpClient = new HttpClient();
httpClient.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Token", "token=7d20f3f09ef24067ae64f4323bc95163");
Uri uri = new Uri("http://signup.com/api/v1/user_invitations");
var response = await httpClient.PostAsync(uri, content).ConfigureAwait(false);
int n = 1;
return response;
}
}
The exception is thrown with a message "Thread was being aborted". Can anyone tell me what I am doing wrong?

I would guess this is going to fail somewhat randomly based on use of async/await. I wouldn't think CRM really supports plugins returning control before they are complete. When it fails, it looks like the thread was in the process of being cleaned up behind the scenes.
CRM is already handling multi-threading of plugins and supports registering plugin steps as asynchronous if they are long running (or don't need to be run in the synchronous pipeline). It would make more sense to use a synchronous HTTP call here like:
var response = httpClient.PostAsync(uri, content).Result;
EDIT: To illustrate, this is an overly-trivialized example of what is most likely happening when CRM goes to kickoff your plugin and you're using async/await (from LinqPad).
static async void CrmInternalFunctionThatKicksOffPlugins()
{
var plugin = new YourPlugin();
//NOTE Crm is not going to "await" your plugin execute method here
plugin.Execute();
"CRM is Done".Dump();
}
public class YourPlugin
{
public async void Execute()
{
await OnlineSignUp();
}
private async Task<HttpResponseMessage> OnlineSignUp()
{
var httpClient = new HttpClient();
var r = await httpClient.PostAsync("http://www.example.com", null);
"My Async Finished".Dump();
return r;
}
}
Which will print:
CRM is Done My Async Finished

looks like you are using Json.NET, when you use external assemblies there are some things to take care of (merging) and not always the result works.
Try to serialize using DataContractJsonSerializer
example: http://www.crmanswers.net/2015/02/json-and-crm-sandbox-plugins.html