We Keep Coding

await makes http request return status code 0

Well, I'm building web parsing app and having some troubles making it async.
I have a method which creates async tasks, and decorator for RestSharp so I can do requests via proxy. Basically in code it just does 5 tries of requesting the webpage.
Task returns RestResponse and it's status code is always 0. And this is the problem, because if I do the same synchronously, it works.
private static async Task<HtmlNode> GetTableAsync(int page)
{
ProxyClient client = new ProxyClient((name) =>ProxyProvider.GetCoreNoCD(name),
serviceName, 10000, 10000);
var task = client.TryGetAsync(new Uri(GetPageUrl(page)), (res) =>
{
return res.IsSuccessStatusCode && res.IsSuccessful;
},5);
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml((await task).Content);
return doc.DocumentNode.SelectSingleNode("//div[#class=\"table_block\"]/table");
}
And this works as expected, but synchronously.
private static async Task<HtmlNode> GetTableAsync(int page)
{
ProxyClient client = new ProxyClient((name) =>ProxyProvider.GetCoreNoCD(name),
serviceName, 10000, 10000);
var task = client.TryGetAsync(new Uri(GetPageUrl(page)), (res) =>
{
return res.IsSuccessStatusCode && res.IsSuccessful;
},5);
task.Wait();
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(task.Result.Content);
return doc.DocumentNode.SelectSingleNode("//div[#class=\"table_block\"]/table");
}
ProxyClient's insides:
public async Task<RestResponse?> TryGetAsync(Uri uri,
Predicate<RestResponse> condition, int tryCount = 15,
List<KeyValuePair<string, string>> query = null,
List<KeyValuePair<string, string>> headers = null,
Method method = Method.Get, string body = null)
{
WebClient? client = null;
RestResponse? res = null;
for(int i = 0; i < tryCount; i++)
{
try
{
client = new WebClient(source.Invoke(serviceName), serviceName, timeout);
res = await client.GetResponseAsync(uri, query, headers, method, body);
if (condition(res))
return res;
}
catch(Exception)
{
///TODO:add log maybe?
}
finally
{
if (client != null)
{
client.SetCDToProxy(new TimeSpan(cd));
client.Dispose();
}
}
}
return res;
}
I have no idea how to make it work with async and don't understand why it doesn't work as expected.

I think it might have to do with the Task.Wait() I would consider changing to await like this.
private static async Task<HtmlNode> GetTableAsync(int page)
{
ProxyClient client = new ProxyClient((name) =>ProxyProvider.GetCoreNoCD(name),
serviceName, 10000, 10000);
var statusOk = false;
var result = await client.GetAsync(new Uri(GetPageUrl(page));
statusOk = result.IsSuccessStatusCode &&
result.StatusCode == HttpStatusCode.OK;
//do what you want based on statusOk
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(result.Content);
return doc.DocumentNode.SelectSingleNode("//div[#class=\"table_block\"]/table");
}

Just decided to try different solutions, and seems like it works only if I return task result
Like this:
ProxyClient client = new ProxyClient((name) => ProxyProvider.GetCoreNoCD(name),
serviceName, 10000, 10000);
return await client.TryGetAsync(new Uri(GetPageUrl(page)), (res) =>
{ return res.IsSuccessStatusCode && res.IsSuccessful; });
I thought it could be some kind of misunderstanding of async/await, but seems like no. Maybe some kind of RestSharp bug.

I think you're just checking the result too early. You need to look at the result after the await:
var task = client.TryGetAsync(...);
// Too early to check
var x = await task;
// Check now
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(x.Content);

PostAsync call vanishes and web page becomes available again

I have a web page that I'm trying to call a web service and I'm having it skip out of the process when called.
Here's what I'm referring to...
When I call this method after the customer enters in their check information:
public CheckInfo SubmitCheck(CheckInfo checkInfo)
{
try
{
var check = new Check();
check.account_number = checkInfo.CheckAccountNumber;
check.transit_number = checkInfo.CheckRoutingNumber;
check.amount = checkInfo.Amount.ToString();
check.check_number = checkInfo.CheckNumber;
check.bill_to_city = checkInfo.City;
check.bill_to_country = "US";
check.bill_to_postal_code = checkInfo.Zip;
check.bill_to_street = checkInfo.Street;
check.bill_to_state = checkInfo.State;
check.name_on_check = checkInfo.NameOnCheck;
check.transaction_type = "sale";
check.account_type = checkInfo.AccountType;
check.check_type = checkInfo.CheckType;
var ent = new SuburbanPortalEntities();
var gatewaySettings = (from x in ent.GatewayUsers
where x.TokenId == CurrentCustomerSession.Current.TokenId &&
x.Gateway.Name == "DirectAch2"
select x).FirstOrDefault();
var credentials = new Authentication();
credentials.password = gatewaySettings.Password;
credentials.username = gatewaySettings.UserName;
var response = Process.SubmitCheck(credentials, check).Result;
The public class that calls the private class:
public static async Task<Response> SubmitCheck(Authentication authentication, Check check)
{
return await Submit(authentication, check, PaymentTypes.Check);
}
The SubmitCheck Method:
private static async Task<Response> Submit(Authentication authentication, Object payment, PaymentTypes paymentType)
{
var resp = new Response();
try
{
var client = new HttpClient();
var bodyjson = JsonConvert.SerializeObject(authentication);
var bodycontent = new StringContent(bodyjson, Encoding.UTF8, "application/json");
var authenticationPost =
await client.PostAsync("https://someplace.com/api/v2/Identity", bodycontent);
var bodyResponseJson = await authenticationPost.Content.ReadAsStringAsync();
When I get to this line, it just returns out of the method and it doesn't continue on with anything, it's like I never executed this method.
var authenticationPost =
await client.PostAsync("https://someplace.com/api/v2/Identity", bodycontent);
No other code is executed after this line. It just stops and the web page becomes available again. I have the method wrapped in a try catch but the catch isn't.. catching anything.
I'm at a loss at this poing, any suggestions?
EDIT#1
I wrapped the line in a try catch as suggested.
try
{
authenticationPost =
await client.PostAsync("https://someplace.com/api/v2/Identity", bodycontent);
}
catch (Exception e)
{
Console.WriteLine(e);
throw;
}
It's stepping out of the try/catch and never executing any more code after that. No exception is caught, nothing, it just runs the line and leaves the method.

Here's your problem:
var response = Process.SubmitCheck(credentials, check).Result;
Don't block on async code, as I describe on my blog. This is a common mistake for those new to async/await. Instead of Result, use await:
public async Task<CheckInfo> SubmitCheck(CheckInfo checkInfo)
{
...
var response = await Process.SubmitCheck(credentials, check);
Note that you'd then need to await the call to SubmitCheck, and so on. It's async all the way.
Side note: I recommend using the standard pattern of *Async suffixes on your method names; it makes it clearer in the code that their return values need to be awaited:
public async Task<CheckInfo> SubmitCheckAsync(CheckInfo checkInfo)
{
...
var response = await Process.SubmitCheckAsync(credentials, check);

Nsubstitute or moq an Async method so it will return a faulted task instead of exception

In the below code for each request we are calling rest client in an Async manner. Rest client is basically a http client which is calling some webapis in a batch. If suppose AsyncTaskCount value is 5 then 5 request will be called asynchronously and then in the while block we are getting the result for each call. If any response out of those 5 request has an exception then the response will be faulted and IsFaulted becomes true for that particular request and in the response we can get the inner exception.
private async Task<List<RequestResponse>> ProcessInvestment(List<Request> Requests, List<Result> Results, ILogger log)
{
var requestResponses = new List<RequestResponse>();
var asyncTaskCount = Convert.ToInt32(Environment.GetEnvironmentVariable("AsyncTaskCount"));
log.LogInformation($"Start processing {Requests.Count} in batches of {asyncTaskCount}");
for (int index = 0; index < Requests.Count; index = index + asyncTaskCount)
{
var requestBatch = Requests.Skip(index).Take(asyncTaskCount).ToList();
var requests = requestBatch.Select(x => _restClient.RequestResponse(x)).ToList();
while (requests.Count > 0)
{
// Identify the first task that completes.
Task<RequestResponse> requestResponseTask = await Task.WhenAny(requests);
var requestResponse = new RequestResponse();
// ***Remove the selected task from the list so that you don't process it more than once
requests.Remove(requestResponseTask);
if (!requestResponseTask.IsFaulted)
{
// Await the completed task.
requestResponse = await requestResponseTask;
requestResponses.Add(requestResponse);
}
else
{
if (requestResponseTask.Exception.InnerException != null && requestResponseTask.Exception.InnerException is Exception)
{
var result = new Result();
result = ResponseTransformComponent.ResponseToResult(((Exception)requestResponseTask.Exception.InnerException).Request, null);
result.SetBadRequestErrorDetails(((Exception)RequestResponseTask.Exception.InnerException).BadRequestResponse);
results.Add(Result);
}
else
{
throw requestResponseTask.Exception;
}
}
}
log.LogInformation($"Number of records processed = {requestResponses.Count}");
}
log.LogInformation($"Total invalid and Bad requests count = {results.Count}");
return RequestResponses;
}
Below is the code for restclient which is called from the above method.
public async Task<Response> RequestResponse(Request request)
{
var response = await GetDataFromService("calculation", "CalculateCapital", request);
return JsonConvert.DeserializeObject<Response>(response);
}
public async Task<string> GetDataFromService(string controller, string method, object request)
{
var client = _httpClientFactory.CreateClient(ServiceEnum.DCS);
string baseAddress = client.BaseAddress.ToString();
var requestUrl = $"api/{controller}/{method}";
var content = new StringContent(JsonConvert.SerializeObject(request), Encoding.UTF8, "application/json");
var response = await client.PostAsync(requestUrl, content).ConfigureAwait(false);
if (!response.IsSuccessStatusCode)
{
var responseResult = response.Content.ReadAsStringAsync().Result;
if (response.StatusCode == HttpStatusCode.BadRequest)
{
throw new CalculatorServiceException("Bad Request", JsonConvert.DeserializeObject<BadRequestResponse>(responseResult), (Request)request);
}
throw new Exception($"Status code: {response.StatusCode}. {responseResult}");
}
return response.Content.ReadAsStringAsync().Result;
}
GetDataFromService method is called from Calculate method. And GetDataFromService method will return a custom exception if the request is a Bad Request.
I am trying to write the unit test case for the above method and try to mock a request so that it will return a faulted task and then IsFaulted should become true. Below is the part of my unit test case.
_restClient
.When(a => a.RequestResponse(Arg.Any<Request>()))
.Do(a => {Task.FromException(new CalculatorServiceException(string.Empty, new BadRequestResponse { Message = string.Empty, ModelState = new Dictionary<string, string[]> { { "CalculationDates.StartDate", new string[] { "0002: Duration too short to execute calculations (CalculationDates.StartDate)" } } } }, Arg.Any<Request>())); });
If i mock my restclient method like above then it is throwing the exception instead of giving the response with IsFaulted to true. So how should i mock the restclient method so that it will return a faulted task which has an exception instead of throwing it. Please suggest.
Thanks in advance.

When..Do is for wiring up callbacks for when a member is called.
Try using Returns instead:
_restClient.RequestResponse(Arg.Any<Request>())
.Returns(x => Task.FromException<RequestResponse>(...));
// (assuming RequestResponse(..) returns a Task<RequestResponse>. Tweak as required)

Xamarin Can't Convert System.Threading.Tasks

I am wanting to make a call to a sever, get back a JSON result and return the result to the calling method. Below is my code which makes the call to the server:
public async Task<Dictionary<string,string>> callAjax(string mthd,Dictionary<string,string> values)
{
HttpClient client = new HttpClient();
var content = new FormUrlEncodedContent(values);
var response = await client.PostAsync("http://dev.adex-intl.com/adex/mvc/receiving/"+mthd, content);
var responseString = await response.Content.ReadAsStringAsync();
Dictionary<string,string> result = JsonConvert.DeserializeObject<Dictionary<string, string>>((string)responseString);
return result;
}
And this is the code that calls the above method:
public void loginPressed(object sender, EventArgs e)
{
if(String.IsNullOrEmpty(badge.Text)) {
DisplayAlert("Error", "Enter your badge number", "Ok");
} else {
IsBusy = true;
var parameters = new Dictionary<string, string>
{
{ "badgeNumber", badge.Text }
};
GlobalMethods globalMethods = new GlobalMethods();
Dictionary<string,string> results = globalMethods.callAjax("login", parameters);
var id = results["userid"].ToString();
}
}
I am getting a compiler error of "Cannot implicitly convert type 'System.Threading.Tasks.Task'" on line "Dictionary results = globalMethods.callAjax("login", parameters);"

callAjax is an async method, so you need to use await when calling it
Dictionary<string,string> results = await globalMethods.callAjax("login", parameters);

asynchronous module or handler completed while an asynchronous operation was still pending

I am working on bot technology, in one of my projet i wrote below lines of code
private async void DeliveryProgressReport(IDialogContext context, Activity message)
{
MessagesController.progressdetails = SQLDatabaseService.getinprogressdetails();
var progress = MessagesController.progressdetails;
if (progress.Count > 0)
{
try
{
Activity replyToConversation = message.CreateReply("**In Progress Report Details**");
replyToConversation.Recipient = message.From;
replyToConversation.Type = "message";
replyToConversation.Attachments = new List<Attachment>();
Dictionary<string, string> progresslist = new Dictionary<string, string>();
foreach (var progressreport in progress)
{
//Invoke the machine learning model for predicting the delivery status of delivery person
//var deliveryStatus= await InvokeRequestResponseServiceOfDeliveryPersonPredictionExp1();
//await Task.Delay(TimeSpan.FromSeconds(5));
var deliveryStatus = await InvokeRequestResponseServiceOfDeliveryPersonPredictionExp(progress[0].Name, progress[0].Mobile_Number);
progresslist.Add(progressreport.Name, progressreport.Mobile_Number);
List<CardImage> cardImages = new List<CardImage>();
cardImages.Add(new CardImage(url: progressreport.Photo_Url));
ThumbnailCard tlcard = new ThumbnailCard()
{
Title = "Name:" + progressreport.Name,
Subtitle = "Call:" + progressreport.Mobile_Number,
Images = cardImages,
Text = "Staus by Using Machine Learning Prediction:" + deliveryStatus
};
Attachment plAttachment = tlcard.ToAttachment();
replyToConversation.Attachments.Add(plAttachment);
}
replyToConversation.AttachmentLayout = AttachmentLayoutTypes.List;
await context.PostAsync(replyToConversation);
} catch (Exception ex)
{
Debug.WriteLine(ex.Message);
}
}
else
{
Activity replyToConversation = message.CreateReply("**There are no in progress deliveries are found**");
await context.PostAsync(replyToConversation);
}
}
private async Task<string> InvokeRequestResponseServiceOfDeliveryPersonPredictionExp(string name, string mobile_Number)
{
string status = "";
//Func<Stream, Task> copyStreamAsync = async stream =>
//{
//await Task.Factory.StartNew(async () =>
//{
//using (stream)
//using (var sourceStream = await sourceContent.Content.ReadAsStreamAsync())
//{
// await sourceStream.CopyToAsync(stream);
//}
//var client = new HttpClient();
using (var client = new HttpClient())
{
var scoreRequest = new
{
Inputs = new Dictionary<string, StringTable>() {
{
"input1",
new StringTable()
{
ColumnNames = new string[] {"Id", "Name", "Mobile_Number", "CourierCompany_Name", "Status", "EmailId"},
Values = new string[,] { { "", name, mobile_Number, "", "","" }, { "", name, mobile_Number, "", "", "" }, }
}
},
},
GlobalParameters = new Dictionary<string, string>()
{
}
};
const string apiKey = "xxxxxxxxxxxxxxxxxxxxxxxxxxxx=="; // Replace this with the API key for the web service
client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", apiKey);
client.BaseAddress = new Uri("My Request URL");
// WARNING: The 'await' statement below can result in a deadlock if you are calling this code from the UI thread of an ASP.Net application.
// One way to address this would be to call ConfigureAwait(false) so that the execution does not attempt to resume on the original context.
// For instance, replace code such as:
// result = await DoSomeTask()
// with the following:
// result = await DoSomeTask().ConfigureAwait(false)
//var status = await PostRequest(scoreRequest,client).ConfigureAwait(false);
HttpResponseMessage response = await client.PostAsJsonAsync("", scoreRequest);//.ConfigureAwait(false);
string correctLocation = "";
string wrongLocation = "";
string notReached = "";
string personMismatch = "";
if (response.IsSuccessStatusCode)
{
string result = await response.Content.ReadAsStringAsync();
var results = JsonConvert.DeserializeObject<RootObject>(result);
foreach (var value in results.Results.output1.value.Values)
{
status = value[8].ToString();
correctLocation = value[4].ToString();
notReached = value[5].ToString();
personMismatch = value[6].ToString();
wrongLocation = value[7].ToString();
}
Debug.WriteLine("Result: {0}", result);
return status;
}
else
{
Debug.WriteLine(string.Format("The request failed with status code: {0}", response.StatusCode));
// Print the headers - they include the requert ID and the timestamp, which are useful for debugging the failure
Debug.WriteLine(response.Headers.ToString());
string responseContent = await response.Content.ReadAsStringAsync();
Debug.WriteLine(responseContent);
return status;
}
};
// return status;
}
After executing this below line I got the exception like asynchronous module or handler completed while an asynchronous operation was still pending
await context.PostAsync(replyToConversation);
Before posting this question I had followed through this below links, but I didn't resolve it.
Async Void, ASP.Net, and Count of Outstanding Operations
Web Api + HttpClient: An asynchronous module or handler completed while an asynchronous operation was still pending
Please tell how to resolve this exception.
-Pradeep

Finally, I resolved the above exception when I am return Task instead of void in DeliveryProgressReport method. and also where ever I was called the await DeliveryProgressReport() method there also I return Task instead of void.
-Pradeep