We Keep Coding

Consume a Web Service in C#: Remove Task.Wait()

I am trying to consume a web service. It's an XML based service. I mean response in XML format. The code is working fine. However, I do not want to use task.Wait(). Please let me know how I can replace it with async/await.
Below is my code :
using System.IO;
using System.Net.Http;
using System.Threading.Tasks;
using System.Xml.Serialization;
namespace ConsoleApp6
{
class Program
{
static void Main(string[] args)
{
Program obj = new Program();
var result = obj.GetData().Result;
}
public async Task<string> GetData()
{
string url =
"https://test.net/info.php?akey=abcd&skey=xyz";
HttpClient client = new HttpClient();
HttpResponseMessage response = client.GetAsync(url).Result;
var responseValue = string.Empty;
if (response != null)
{
Task task = response.Content.ReadAsStreamAsync().ContinueWith(t =>
{
var stream = t.Result;
using (var reader = new StreamReader(stream))
{
responseValue = reader.ReadToEnd();
}
});
task.Wait(); // How I can replace it and use await
}
return responseValue;
}
}
[XmlRoot(ElementName = "Info")]
public class Test
{
[XmlAttribute(AttributeName = "att")]
public string SomeAttribute{ get; set; }
[XmlText]
public string SomeText{ get; set; }
}
}

You already are in an async context, so just use await:
var stream = await response.Content.ReadAsStreamAsync();
using (var reader = new StreamReader(stream))
{
responseValue = reader.ReadToEnd();
}
That said, you should check all your calls:
HttpResponseMessage response = await client.GetAsync(url);
and make your main async, too and while we are at it make the method static:
public static async Task Main)
{
var result = await GetData();
}
where your method signature is:
public static async Task<string> GetData()
The static isn't required, but you will find parallel and/or asynchronous programming is a lot easier if you have as little side effects as possible.

You can make Main method async as well and await GetData
static async Task Main(string[] args)
{
Program obj = new Program();
var result = await obj.GetData();
}

Getting HttpClient to POST/PUT Async

I am attempting to PUT and POST to a URL using HttpClient in C#. I need to do so asynchronously for scaling purposes. However, I am only able to get my PUT and POST to work synchronously. Below is the code I am using to PUT the ZoomData objects in JSON format to the URL:
// takes the dataset to PUT and PUTs to server
public async Task<HttpResponseMessage> JsonPUTcall(ZoomData toPut)
{
string jsonString = JsonConvert.SerializeObject(toPut);
return await client.PutAsync(InsiteDatasetPutURL.Replace("sys_id", toPut.sys_id), new StringContent(jsonString, UnicodeEncoding.UTF8, "application/json"));
}
And here is the code I am using to actually pass ZoomData objects in a queue to JsonPUTcall:
public async void JsonPUTqueueCall(Queue<ZoomData> toPut)
{
if (toPut.Count == 0)
return;
foreach (var zoomData in toPut)
{
var result = await this.JsonPUTcall(zoomData);
}
}
However, when I attempt this, it simply hangs. So, as a test, I replaced "var result = await this.JsonPUTcall(zoomData);" with the following:
public async void JsonPUTqueueCall(Queue<ZoomData> toPut)
{
if (toPut.Count == 0)
return;
foreach (var zoomData in toPut)
{
var result = this.JsonPUTcall(zoomData);
result.Wait();
}
}
That works, but since it is synchronous, it defeats the purpose of using async. What am I missing?

this is my API client which is efficiently uses resources and the methods are async
using System;
using System.Net;
using System.Net.Http;
using System.Net.Http.Headers;
using System.Text;
using System.Threading.Tasks;
//You need to install package Newtonsoft.Json > https://www.nuget.org/packages/Newtonsoft.Json/
using Newtonsoft.Json;
using Newtonsoft.Json.Serialization;
public class MyApiClient : IDisposable
{
private readonly TimeSpan _timeout;
private HttpClient _httpClient;
private HttpClientHandler _httpClientHandler;
private readonly string _baseUrl;
private const string ClientUserAgent = "my-api-client-v1";
private const string MediaTypeJson = "application/json";
public MyApiClient(string baseUrl, TimeSpan? timeout = null)
{
_baseUrl = NormalizeBaseUrl(baseUrl);
_timeout = timeout ?? TimeSpan.FromSeconds(90);
}
public async Task<string> PostAsync(string url, object input)
{
EnsureHttpClientCreated();
using (var requestContent = new StringContent(ConvertToJsonString(input), Encoding.UTF8, MediaTypeJson))
{
using (var response = await _httpClient.PostAsync(url, requestContent))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
}
public async Task<TResult> PostAsync<TResult>(string url, object input) where TResult : class, new()
{
var strResponse = await PostAsync(url, input);
return JsonConvert.DeserializeObject<TResult>(strResponse, new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
});
}
public async Task<TResult> GetAsync<TResult>(string url) where TResult : class, new()
{
var strResponse = await GetAsync(url);
return JsonConvert.DeserializeObject<TResult>(strResponse, new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
});
}
public async Task<string> GetAsync(string url)
{
EnsureHttpClientCreated();
using (var response = await _httpClient.GetAsync(url))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
public async Task<string> PutAsync(string url, object input)
{
return await PutAsync(url, new StringContent(JsonConvert.SerializeObject(input), Encoding.UTF8, MediaTypeJson));
}
public async Task<string> PutAsync(string url, HttpContent content)
{
EnsureHttpClientCreated();
using (var response = await _httpClient.PutAsync(url, content))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
public async Task<string> DeleteAsync(string url)
{
EnsureHttpClientCreated();
using (var response = await _httpClient.DeleteAsync(url))
{
response.EnsureSuccessStatusCode();
return await response.Content.ReadAsStringAsync();
}
}
public void Dispose()
{
_httpClientHandler?.Dispose();
_httpClient?.Dispose();
}
private void CreateHttpClient()
{
_httpClientHandler = new HttpClientHandler
{
AutomaticDecompression = DecompressionMethods.Deflate | DecompressionMethods.GZip
};
_httpClient = new HttpClient(_httpClientHandler, false)
{
Timeout = _timeout
};
_httpClient.DefaultRequestHeaders.UserAgent.ParseAdd(ClientUserAgent);
if (!string.IsNullOrWhiteSpace(_baseUrl))
{
_httpClient.BaseAddress = new Uri(_baseUrl);
}
_httpClient.DefaultRequestHeaders.Accept.Add(new MediaTypeWithQualityHeaderValue(MediaTypeJson));
}
private void EnsureHttpClientCreated()
{
if (_httpClient == null)
{
CreateHttpClient();
}
}
private static string ConvertToJsonString(object obj)
{
if (obj == null)
{
return string.Empty;
}
return JsonConvert.SerializeObject(obj, new JsonSerializerSettings
{
ContractResolver = new CamelCasePropertyNamesContractResolver()
});
}
private static string NormalizeBaseUrl(string url)
{
return url.EndsWith("/") ? url : url + "/";
}
}
Usage;
using ( var client = new MyApiClient("http://localhost:8080"))
{
var response = client.GetAsync("api/users/findByUsername?username=alper").Result;
var userResponse = client.GetAsync<MyUser>("api/users/findByUsername?username=alper").Result;
}

These 2 lines is to make it synchronous :
var result = this.JsonPUTcall(zoomData);
result.Wait();
To make it async you hate to do that :
var result = await this.JsonPUTcall(zoomData);

C# HttpWebRequest.BeginGetResponse blocks in one class but not another

I am validating a list of proxies using HttpWebRequest.BeginGetResponse. It works really well, I can validate thousands of proxies in seconds and doesn't block.
In another class within my project, I am calling the same code and it blocks.
Proxy validation method (Doesn't block):
public void BeginTest(IProxyTest test, Action<ProxyStatus> callback, int timeout = 10000)
{
var req = HttpWebRequest.Create(test.URL);
req.Proxy = new WebProxy(this.ToString());
req.Timeout = timeout;
WebHelper.BeginGetResponse(req, new Action<RequestCallbackState>(callbackState =>
{
if (callbackState.Exception != null)
{
callback(ProxyStatus.Invalid);
}
else
{
var responseStream = callbackState.ResponseStream;
using (var reader = new StreamReader(responseStream))
{
var responseString = reader.ReadToEnd();
if (responseString.Contains(test.Validation))
{
callback(ProxyStatus.Valid);
}
else
{
callback(ProxyStatus.Invalid);
}
}
}
}));
}
WebHelper.BeginGetResponse
public static void BeginGetResponse(WebRequest request, Action<RequestCallbackState> responseCallback)
{
Task<WebResponse> asyncTask = Task.Factory.FromAsync<WebResponse>(request.BeginGetResponse, request.EndGetResponse, null);
ThreadPool.RegisterWaitForSingleObject((asyncTask as IAsyncResult).AsyncWaitHandle, new WaitOrTimerCallback(TimeoutCallback), request, request.Timeout, true);
asyncTask.ContinueWith(task =>
{
WebResponse response = task.Result;
Stream responseStream = response.GetResponseStream();
responseCallback(new RequestCallbackState(responseStream));
responseStream.Close();
response.Close();
}, TaskContinuationOptions.NotOnFaulted);
//Handle errors
asyncTask.ContinueWith(task =>
{
var exception = task.Exception;
responseCallback(new RequestCallbackState(exception.InnerException));
}, TaskContinuationOptions.OnlyOnFaulted);
}
Other class with a similar method that also calls WebHelper.BeginGetResponse, but blocks (why?)
public void BeginTest(Action<ProxyStatus> callback, int timeout = 10000)
{
var req = HttpWebRequest.Create(URL);
req.Timeout = timeout;
WebHelper.BeginGetResponse(req, new Action<RequestCallbackState>(callbackState =>
{
if (callbackState.Exception != null)
{
callback(ProxyStatus.Invalid);
}
else
{
var responseStream = callbackState.ResponseStream;
using (var reader = new StreamReader(responseStream))
{
var responseString = reader.ReadToEnd();
if (responseString.Contains(Validation))
{
callback(ProxyStatus.Valid);
}
else
{
callback(ProxyStatus.Invalid);
}
}
}
}));
}
Calling code which blocks
private async void validateTestsButton_Click(object sender, EventArgs e)
{
await Task.Run(() =>
{
foreach (var test in tests)
{
test.BeginTest((status) => test.Status = status);
}
});
}
Calling code which doesn't block:
public static async Task BeginTests(ICollection<Proxy> proxies, ICollection<ProxyJudge> judges, int timeout = 10000, IProgress<int> progress = null)
{
await Task.Run(() =>
{
foreach (var proxy in proxies)
{
proxy.BeginTest(judges.GetRandomItem(), new Action<ProxyStatus>(status =>
{
proxy.Status = status;
}), timeout);
}
});
}

Although this dosnt address your problem exactly, it might help you out a little
Here are a couple of problems
You are using APM (Asynchronous Programming Model)
You are using the ThreadPool class which seems a little old fashioned
You are doing IO bound work and blocking threads on the threadpool
You are using a weird mix of APM and TBA asynchronous models
And seemingly tying up your thread pool waiting for IO
So you are doing IO bound work, the best pattern to use as you might have guess is the TBA async await pattern. basically every time you wait for a an IO Completion port you want to give that thread back to the operating system and be nice to your system inturn freeing up resources for where its needed.
Also you obviously want some degree of parallelism and you are best to at least have some control over it.
I would suggest this is a nice job for TPL Dataflow and an ActionBlock
Given
public class Proxy
{
public ProxyStatus ProxyStatus { get; set; }
public string ProxyUrl { get; set; }
public string WebbUrl { get; set; }
public string Error { get; set; }
}
ActionBlock Example
public static async Task DoWorkLoads(List<Proxy> results)
{
var options = new ExecutionDataflowBlockOptions
{
MaxDegreeOfParallelism = 50
};
var block = new ActionBlock<Proxy>(CheckUrlAsync, options);
foreach (var proxy in results)
{
block.Post(proxy);
}
block.Complete();
await block.Completion;
}
CheckUrlAsync Example
// note i havent tested this, add pepper and salt to taste
public static async Task CheckUrlAsync(Proxy proxy)
{
try
{
var request = WebRequest.Create(proxy.Url);
if (proxy.ProxyUrl != null)
request.Proxy = new WebProxy(proxy.ProxyUrl);
using (var response = await request.GetResponseAsync())
{
using (var responseStream = response.GetResponseStream())
{
using (var reader = new StreamReader(responseStream))
{
var responseString = reader.ReadToEnd();
if (responseString.Contains("asdasd"))
proxy.ProxyStatus = ProxyStatus.Valid;
else
proxy.ProxyStatus = ProxyStatus.Invalid;
}
}
}
}
catch (Exception e)
{
proxy.ProxyStatus = ProxyStatus.Error;
proxy.Error = e.Message;
}
}
Usage
await DoWorkLoads(proxies to test);
Summary
The code is neater, you arnt throwing actions all over the place, you using async and await you have ditched APM, you have control of the degrees of parallel and you are being nice to the thread pool

I solved the problem by wrapping the code in the BeginTest method which was blocking mysteriously in an Action, and then calling BeginInvoke on that Action.
I deduced this was caused by not setting the Proxy property on the HttpWebRequest in that method, which seemed to be causing a synchronous lookup of my systems proxy.
public void BeginTest(Action<ProxyStatus> callback, int timeout = 10000)
{
var action = new Action(() =>
{
var req = HttpWebRequest.Create(URL);
req.Timeout = timeout;
WebHelper.BeginGetResponse(req, new Action<RequestCallbackState>(callbackState =>
{
if (callbackState.Exception != null)
{
callback(ProxyStatus.Invalid);
}
else
{
var responseStream = callbackState.ResponseStream;
using (var reader = new StreamReader(responseStream))
{
var responseString = reader.ReadToEnd();
if (responseString.Contains(Validation))
{
callback(ProxyStatus.Valid);
}
else
{
callback(ProxyStatus.Invalid);
}
}
}
}));
});
action.BeginInvoke(null, null);
}

Testing Polly retry policy with moq

I'm trying to write a unit test for polly, but it looks like the return is cached.
Method PostAsyncWithRetry:
using Polly;
using System;
using System.Diagnostics;
using System.Net.Cache;
using System.Net.Http;
public class RetryClient
{
private HttpClient httpClient = new HttpClient(new WebRequestHandler()
{ CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.NoCacheNoStore) });
public HttpResponseMessage PostAsyncWithRetry(
String url,
String path,
StringContent httpContent)
{
httpClient.BaseAddress = new Uri(url);
var retryPolicy =
Policy.Handle<HttpRequestException>()
.OrResult<HttpResponseMessage>(r => !r.IsSuccessStatusCode)
.RetryAsync(3, (exception, retryCount, context) =>
{
Debug.WriteLine("RetryCount: {0}", retryCount);
});
var response = retryPolicy.ExecuteAsync(async () =>
{
return await httpClient.PostAsync(path, httpContent);
}
);
return response.Result;
}
}
Test:
[TestFixture]
class Test
{
private HttpClient mockHTTPClient;
private Mock<WebRequestHandler> mockHttpMessageHandler;
private RetryClient testInstance;
private const String URL = "https://aaa.com";
private const String PATH = "/path";
private const String EXPECTED_STRING_CONTENT = "Some return text";
[SetUp]
public void SetUp()
{
testInstance = new RetryClient();
mockHttpMessageHandler = new Mock<WebRequestHandler>();
mockHttpMessageHandler.Object.CachePolicy = new HttpRequestCachePolicy(HttpRequestCacheLevel.NoCacheNoStore);
mockHTTPClient = new HttpClient(mockHttpMessageHandler.Object);
var type = typeof(RetryClient);
var fields = type.GetFields(BindingFlags.NonPublic | BindingFlags.Instance);
fields[0].SetValue(testInstance, mockHTTPClient);
}
[Test]
public void TestMEEEE()
{
var responses = new Queue<Task<HttpResponseMessage>>();
responses.Enqueue(Task.FromResult(new HttpResponseMessage
{
StatusCode = HttpStatusCode.NotFound,
Content = new StringContent(EXPECTED_STRING_CONTENT)
}));
responses.Enqueue(Task.FromResult(new HttpResponseMessage
{
StatusCode = HttpStatusCode.OK,
Content = new StringContent(EXPECTED_STRING_CONTENT)
}));
var postContent = new StringContent(EXPECTED_STRING_CONTENT);
mockHttpMessageHandler.Protected()
.Setup<Task>("SendAsync", ItExpr.IsAny<HttpRequestMessage>(), ItExpr.IsAny<CancellationToken>())
.Returns(responses.Dequeue());
var response = testInstance.PostAsyncWithRetry(
URL, PATH, postContent);
mockHttpMessageHandler.Verify();
Assert.AreEqual(responses.Count, 0, "didn't dequeue");
Assert.AreEqual(HttpStatusCode.OK, response.StatusCode, "Incorrect status code");
}
}
Not sure why, but it looks like the responses queue is only being Dequeue once, this leads me to believe the response is being cache. Does anyone know who is caching the response, and how do I disable it?
Thanks a bunch in advance!

I figured it out. It has nothing to do with caching. During the mock setup, it stores the Dequeue value as a return instead of invoking it every time.
Changing it to () => responses.Dequeue() works now.
Thank you guys!

Mock returns null value when ReturnResult is a custom object but works as expected when it is a primitive type bool

I am using Moq in .net core(1.1) and having a bit of torrid time understanding this behavior as all the examples on interweb points to the fact the this should work with no issues.
I have already tried with:
Returns(Task.FromResult(...)
Returns(Task.FromResult(...)
ReturnsAsync(...)
Mocking a IHttpClient interface to wrap PostAsync, PutAsync and GetAsync. All of these return an ApiResponse object.
var mockClient = new Mock<IHttpClient>();
Does not work:
mockClient.Setup(x => x.PostAsync(url, JsonConvert.SerializeObject(body), null))
.Returns(Task.FromResult(new ApiResponse()));
PostSync definition:
public async Task<ApiResponse> PostAsync(string url, string body, string authToken = null)
Does work:
mockClient.Setup(x => x.PostAsync(url, JsonConvert.SerializeObject(body), null))
.Returns(Task.FromResult(bool));
PostSync definition:
public async Task<bool> PostAsync(string url, string body, string authToken = null)
Usage:
var api = new ApiService(mockClient.Object);
var response = api.LoginAsync(body.Username, body.Password);
UPDATE
[Fact]
public async void TestLogin()
{
var mockClient = new Mock<IHttpClient>();
mockClient.Setup(x => x.PostAsync(url, JsonConvert.SerializeObject(body), null)).Returns(Task.FromResult(new ApiResponse()));
var api = new ApiService(mockClient.Object);
var response = await api.LoginAsync(body.Username, body.Password);
Assert.IsTrue(response);
}
Return Type:
public class ApiResponse
{
public string Content { get; set; }
public HttpStatusCode StatusCode { get; set; }
public string Reason { get; set; }
}
LoginAsync:
public async Task<bool> LoginAsync(string user, string password)
{
var body = new { Username = user, Password = password };
try
{
var response = await _http.PostAsync(_login_url, JsonConvert.SerializeObject(body), null);
return response .State == 1;
}
catch (Exception ex)
{
Logger.Error(ex);
return false;
}
}
PostAsync:
public async Task<object> PostAsync(string url, string body, string authToken = null)
{
var client = new HttpClient();
var content = new StringContent(body, Encoding.UTF8, "application/json");
var response = await client.PostAsync(new Uri(url), content);
var resp = await response.Result.Content.ReadAsStringAsync();
return new ApiResponse
{
Content = resp,
StatusCode = response.Result.StatusCode,
Reason = response.Result.ReasonPhrase
};
}

Assuming a simple method under test like this based on minimal example provided above.
public class ApiService {
private IHttpClient _http;
private string _login_url;
public ApiService(IHttpClient httpClient) {
this._http = httpClient;
}
public async Task<bool> LoginAsync(string user, string password) {
var body = new { Username = user, Password = password };
try {
var response = await _http.PostAsync(_login_url, JsonConvert.SerializeObject(body), null);
return response.StatusCode == HttpStatusCode.OK;
} catch (Exception ex) {
//Logger.Error(ex);
return false;
}
}
}
The following test works when configured correctly
[Fact]
public async Task Login_Should_Return_True() { //<-- note the Task and not void
//Arrange
var mockClient = new Mock<IHttpClient>();
mockClient
.Setup(x => x.PostAsync(It.IsAny<string>(), It.IsAny<string>(), null))
.ReturnsAsync(new ApiResponse() { StatusCode = HttpStatusCode.OK });
var api = new ApiService(mockClient.Object);
//Act
var response = await api.LoginAsync("", "");
//Assert
Assert.IsTrue(response);
}
The above is just for demonstrative purposes only to show that it can work provided the test is configured properly and exercised based on the expected behavior.
Take some time and review the Moq quick start to get a better understanding of how to use the framework.