I am trying to trace a chain of redirects (for an online ad pixel) programmatically, but with a timeout of 2 seconds (in other words, if the redirect chain takes more than 2 seconds to resolve, I want it to abort and return null).
My code is (more or less) running synchronously, so I had to do some acrobatics to do what I wanted, but functionally speaking, it seems to work... except for the timeout part.
I have some asynchronous helpers like so:
public static async Task<TResult> TimeoutAfter<TResult>(this Task<TResult> task, TimeSpan timeout)
{
using (var timeoutCancellationTokenSource = new CancellationTokenSource())
{
var completedTask = await Task.WhenAny(task, Task.Delay(timeout, timeoutCancellationTokenSource.Token));
if (completedTask != task)
{
throw new TimeoutException();
}
timeoutCancellationTokenSource.Cancel();
return await task;
}
}
public static T ToSynchronousResult<T>(this Task<T> task)
{
return Task.Run(async () => await task).Result;
}
The TimeoutAfter() helper method was adapted from the SO article that can be found here. In my service I have a method that resembles this:
public string GetFinalUrl(string url)
{
string finalUrl;
try
{
finalUrl = FollowDestinationUrl(url).TimeoutAfter(TimeSpan.FromSeconds(2)).ToSynchronousResult();
}
catch (TimeoutException)
{
finalUrl = null;
}
return finalUrl;
}
private async Task<string> FollowDestinationUrl(string url)
{
var request = _webRequestFactory.CreateGet(url);
var payload = await request.GetResponseAsync();
return payload.ResponseUri.ToString();
}
The _webRequestFactory here returns an HttpWebRequest abstraction that was written as an IHttpRequest.
In my success case unit test (response under 2 seconds), I get back the result I expect:
private class TestWebResponse : WebResponse
{
public override Uri ResponseUri => new Uri("https://www.mytest.com/responseIsGood");
}
[TestMethod]
public void RedirectUriUnderTimeout()
{
//arrange
var service = GetService();
A.CallTo(() => _httpRequest.GetResponseAsync()).ReturnsLazily(() => new TestWebResponse());
A.CallTo(() => _httpRequest.GetResponseString())
.ReturnsLazily(() => VALID_REQUEST_PAYLOAD);
//act
var url = service.GetFinalUrl("https://someplace.com/testurl");
//assert
Assert.IsNotNull(url);
}
...however, when I try to implement a delay to verify the timeout is working correctly, it's not aborting as I would expect:
[TestMethod]
public void RedirectUriUnderTimeout()
{
//arrange
var service = GetService();
A.CallTo(() => _httpRequest.GetResponseAsync()).ReturnsLazily(() => {
Thread.Sleep(TimeSpan.FromSeconds(3));
return new TestWebResponse();
});
A.CallTo(() => _httpRequest.GetResponseString())
.ReturnsLazily(() => VALID_REQUEST_PAYLOAD);
//act
var url = service.GetFinalUrl("https://someplace.com/testurl");
//assert
Assert.IsNull(url);
}
It seems like it waits for the full three seconds, before returning the TestWebResponse that has a non-null ResponseUri.
I don't know if there's something fundamentally wrong with my implementation, or wrong with my test, but obviously I'm blocking an async call in a way I'm not expecting to.
Can someone help me identify what I've done wrong?
public static T ToSynchronousResult<T>(this Task<T> task)
{
return Task.Run(async () => await task).Result;
}
This part causes to get thread blocked.As you mentioned the method ToSynchronousResult, it will block the thread until task result returned. You should follow "async all the way" rule and you should use await. It is only way to apply async efficiently.
public async Task<string> GetFinalUrl(string url)
{
string finalUrl;
try
{
finalUrl = await FollowDestinationUrl(url).TimeoutAfter(TimeSpan.FromSeconds(2));
}
catch (TimeoutException)
{
finalUrl = null;
}
return finalUrl;
}
OK, it looks like I was way overthinking it. #Stormcloak clued me in that what I was doing wasn't going to work, so I started looking at alternatives, and I realized that while the async/ await pattern weren't appropriate here, the TPL library still came in handy.
I changed my FinalDestinationUrl method to synchronous like so:
private string FollowDestinationUrl(string url)
{
var request = _webRequestFactory.CreateGet(url);
var payload = request.GetResponse();
return payload.ResponseUri.ToString();
}
then I called it like so:
var task = Task.Run(() => FollowDestinationUrl(destinationUrl));
finalUrl = task.Wait(TimeSpan.FromSeconds(2)) ? task.Result : null;
Then I changed my unit test to resemble:
[TestMethod]
public void RedirectUriUnderTimeout()
{
//arrange
var service = GetService();
A.CallTo(() => _httpRequest.GetResponse()).ReturnsLazily(() => {
Thread.Sleep(TimeSpan.FromSeconds(3));
return new TestWebResponse();
});
A.CallTo(() => _httpRequest.GetResponseString())
.ReturnsLazily(() => VALID_REQUEST_PAYLOAD);
//act
var url = service.GetFinalUrl("https://someplace.com/testurl");
//assert
Assert.IsNull(url);
}
The test passed. All is well in the world. Thanks!
Related
I am trying to create a solution with polly where I request an other api.
I have a list of URLs to multiple instances of the same service.
I want that when the first request failes, an other should automaticly start with the next url from my list.
Here is an example where i try this behaviour with two static addresses
The Problem with this solution is that the url does not change until i start the next request.
I want that the urls changes on every retry
public static void ConfigureUserServiceClient(this IServiceCollection services)
{
_userServiceUri = new Uri("https://localhost:5001");
services.AddHttpClient("someService", client =>
{
client.BaseAddress = _userServiceUri;
client.DefaultRequestHeaders.Add("Accept", "application/json");
}).AddPolicyHandler(retryPolicy());
}
private static IAsyncPolicy<HttpResponseMessage> retryPolicy()
{
return Policy.HandleResult<HttpResponseMessage>(r => r.StatusCode == HttpStatusCode.RequestTimeout)
.WaitAndRetryAsync(3, retryAttempt => TimeSpan.FromSeconds(retryAttempt),
onRetry: (result, span, ctx) =>
{
_userServiceUri = new Uri("https://localhost:5002");
});
}
You should consider to use the Fallback policy instead.
Like this:
private static HttpClient client = new HttpClient();
static async Task Main(string[] args)
{
var addressIterator = GetUrls().GetEnumerator();
var retryLikePolicy = Policy<string>
.Handle<HttpRequestException>()
.FallbackAsync(fallbackAction: async (ct) =>
{
if (addressIterator.MoveNext())
return await GetData(addressIterator.Current);
return null;
});
addressIterator.MoveNext();
var data = await retryLikePolicy.ExecuteAsync(
async () => await GetData(addressIterator.Current));
Console.WriteLine("End");
}
static async Task<string> GetData(string uri)
{
Console.WriteLine(uri);
var response = await client.GetAsync(uri);
return await response.Content.ReadAsStringAsync();
}
static IEnumerable<string> GetUrls()
{
yield return "http://localhost:5500/index.html";
yield return "http://localhost:5600/index.html";
yield return "http://localhost:5700/index.html";
}
Please note that this code is just for demonstration.
UPDATE #1: Multiple fallback
If you have more than one fallback urls then you can alter the above code like this:
private static HttpClient client = new HttpClient();
static async Task Main(string[] args)
{
var retryInCaseOfHRE = Policy
.Handle<HttpRequestException>()
.WaitAndRetryForeverAsync(_ => TimeSpan.FromSeconds(1));
var response = await retryInCaseOfHRE.ExecuteAsync(
async () => await GetNewAddressAndPerformRequest());
if (response == null)
{
Console.WriteLine("All requests failed");
Environment.Exit(1);
}
Console.WriteLine("End");
}
static IEnumerable<string> GetAddresses()
{
yield return "http://localhost:5500/index.html";
yield return "http://localhost:5600/index.html";
yield return "http://localhost:5700/index.html";
yield return "http://localhost:5800/index.html";
}
static readonly IEnumerator<string> AddressIterator = GetAddresses().GetEnumerator();
static async Task<string> GetNewAddressAndPerformRequest()
=> AddressIterator.MoveNext() ? await GetData(AddressIterator.Current) : null;
static async Task<string> GetData(string uri)
{
Console.WriteLine(uri);
var response = await client.GetAsync(uri);
return await response.Content.ReadAsStringAsync();
}
The trick: the retry policy wraps a method which is responsible to retrieve the next url and then call the GetData
In other word we need to move the iteration process into the to be wrapped method (GetNewAddressAndPerformRequest)
I've replaced the Fallback policy to Retry since we need to perform (potentially) more than 1 fallback actions
I've used null to indicate we have run out of fallback urls but it might be a better solution to use a custom exception for that
I'm trying to call an Api. If the call fails, I'm retrying with in the configured time out. When I'm writing the unit test for the scenario where the api repeated calls fails and the time out happens. I'm unable to determine what to Assert here. I think it's obvious to verify that the mockedBaseApiClient has never returned the response success for any of it's call in the execution. But I can't figure out how to verify the return type on the mocked object.
public async Task UpdateDocumentPath(UpdateDocument doc)
{
var path = $"v1/documents/{doc.accountId}/{doc.documentId}/paths";
try
{
var response = await _baseApiClient.PutAsync(DocumentsApiKey, path, doc);
if (response.IsSuccessStatusCode)
{
var updatedDocument = await response.Deserialize<UpdateDocument>();
var updatedPath = updatedDocument.documentPath;
}
else
{
await TryToReConnect(doc);
}
}
catch (Exception ex)
{
_exceptionPublisher.PublishException(ex);
await TryToReConnect(doc);
}
}
My Unit Test:
[Fact]
public async Task DocumentPatchAdapter_PatchDocument_RetriesWhenUnSuccessfullStatusCode_ButTimeoutExhausts_Tests()
{
MockConfiguration();
_mockedConfigurationSection.SetupGet(c => c.Value).Returns("20000");
HttpResponseMessage notFoundResponse = new HttpResponseMessage
{
StatusCode = HttpStatusCode.NotFound
};
_mockedBaseApiClient
.SetupSequence(c => c.PutAsync(It.IsAny<string>(), It.IsAny<string>(), It.IsAny<Object>()))
.ReturnsAsync(notFoundResponse)
.ReturnsAsync(notFoundResponse)
.ReturnsAsync(notFoundResponse);
UpdateDocument documentPathToBeUpdated = new UpdateDocument
{
accountId = 34512274,
documentPath = "\\\\sharoi.vliproj.com\\DevL\\DocumentMaker\\Print\\InsuranceScoreRenew_19196100600.pdf",
documentId = 3656261
};
await _documentPatchAdapter.UpdateDocumentPath(documentPathToBeUpdated);
//_mockedBaseApiClient.Verify(c=>c.PutAsync(.....)).NeverReturns(SuccessCode); I want something like this here
}
I am trying to write a function which uses Task and TaskCompletion.
My problem is that after login, the result is not returned. I used similar code before and it was working. I do not know what causes for this situation.
public async Task<byte[]> Sign(byte[] documentContent)
{
var service = new SignServiceWrapper("https://example.com?wsdl");
var loginResult = await Task.Run(() => service.Login(loginRequest));
//....
}
and my SignServiceWrapper class
public class SignServiceWrapper
{
private static string _webServiceUrl;
private BrokerClientClient client;
public SignServiceWrapper(string webServiceUrl)
{
_webServiceUrl = webServiceUrl;
}
public Task<loginResponse> Login(loginRequest request)
{
var tcs = new TaskCompletionSource<loginResponse>();
ClientGenerator.WebServiceUrl = _webServiceUrl;
ClientGenerator.InitializeService();
client = ClientGenerator.ServiceClient;
client.loginCompleted += (sender, loginResult) =>
{
if (loginResult.Error != null)
tcs.SetException(loginResult.Error);
else
tcs.TrySetResult(loginResult.Result);
};
client.loginAsync(request);
return tcs.Task;
}
// ...
}
If I call my login function like that it works
var loginResult = Task.Run(() => service.Login(loginRequest));
loginResult.Wait();
I know that there is kind of a deadlock but I don't know how to solve this here and which object.
Here is a working .NET Fiddle.
I think your .Login method is trying to do too much. The first thing that I noticed (and can only imagine how it's implemented) is the static ClientGenerator, that has static mutable state. This which is alarming and a very specific code smell. I would love to see what the client itself looks like and how that is implemented as that would certainly help to better answer this question.
Based on what you shared thus far (and assuming that the client.loginAsync returns a Task<loginResponse>), I would say that you could do the following:
public class SignServiceWrapper
{
private static string _webServiceUrl;
private BrokerClientClient client;
public SignServiceWrapper(string webServiceUrl)
{
_webServiceUrl = webServiceUrl;
}
public Task<loginResponse> LoginAsync(loginRequest request)
{
ClientGenerator.WebServiceUrl = _webServiceUrl;
ClientGenerator.InitializeService();
client = ClientGenerator.ServiceClient;
return client.loginAsync(request);
}
// ...
}
You could then consume this as such:
public async Task<byte[]> Sign(byte[] documentContent)
{
var service = new SignServiceWrapper("https://example.com?wsdl");
var loginResult = await service.LoginAsync(loginRequest);
//...
}
If the client.loginAsync doesn't return what you're looking for, then you'll need to approach this doing something similar to your current approach. Or if you are locked in to the event-based async pattern, you have other considerations - like whether or not you want to support cancellation, IsBusy, progress, incremental results and if you have the ability to have the event args inherit the System.ComponentModel.AsyncCompletedEventArgs, etc...
One final consideration, if the client.loginAsync is Task returning, even if it doesn't return the loginResponse you need to await it like so:
public async Task<loginResponse> Login(loginRequest request)
{
var tcs = new TaskCompletionSource<loginResponse>();
ClientGenerator.WebServiceUrl = _webServiceUrl;
ClientGenerator.InitializeService();
client = ClientGenerator.ServiceClient;
client.loginCompleted += (sender, loginResult) =>
{
if (loginResult.Error != null)
tcs.SetException(loginResult.Error);
else
tcs.TrySetResult(loginResult.Result);
};
await client.loginAsync(request);
return tcs.Task;
}
Update
After discussion with OP this .NET Fiddle seemed to align with his needs.
Change var loginResult = await Task.Run(() =>service.Login(loginRequest));
To var loginResult = await service.Login(loginRequest);
I am not familiar with threading in .NET.
I have an ansync method MyTest:
public async Task MyTest() {
using (HttpClient httpClient = new HttpClient()) {
httpClient.BaseAddress = new Uri(_uri);
var response = await httpClient.GetAsync("API/GetData");
if(response!=null && response.IsSuccessStatusCode) {
var json = await response.Content.ReadAsStringAsync();
return JsonConvert.DeserializeObject<Dictionary<string, string>>(json);
}
}
}
The problem that I am running into is calling the method to get the results (Dictionary).
When I step though my code I am seeing the IsCompleted is done before the results come back from my rest call.
How do I properly use threading in this case?
My method to call the async method.
public void GetTestData()
{
try
{
ARestService rest = new ARestService();
Task tsk = new Task(rest.MyTest);
if (tsk.IsCompleted)
{
var tst = "Done?";
}
}
catch(Exception ex)
{
string a = ex.Message;
}
}
If you can turn your GetTestData method into an async method simply do this.
public async Task GetTestData()
{
try
{
ARestService rest = new ARestService();
await rest.MyTest();
var tst = "Done?";
}
catch(Exception ex)
{
string a = ex.Message;
}
}
You should also define that your method returns a Task<Dictionary<string, string>> to receive the result of your rest service call, with the following statement.
Dictionary<string, string> dict = await rest.MyTest();
If not, have a look at some workarounds, like using GetAwaiter().GetResult() but as explained in this question Is .GetAwaiter().GetResult(); safe for general use? it can cause some problems, so the best option is to make your calling code async too.
I have the following ServiceStack web service
public class RetreadServices : Service
{
public IRepository Repository { get; set; }
//http://stackoverflow.com/questions/18902135/servicestack-no-server-side-async-support
public async Task<ValidationResponse> Get(RetreadSsnInfoAsync request)
{
return await Task.Factory.StartNew(() =>
{
Tenant.SetTenant(request.TenantId);
return new ValidationResponse { Result = Repository.CannotRetreadSsn(request.Ssn) };
});
}
public async Task<ValidationResponse> Get(RetreadEmailInfoAsync request)
{
return await Task.Factory.StartNew(() =>
{
Tenant.SetTenant(request.TenantId);
return new ValidationResponse { Result = Repository.CannotRetreadEmail(request.Emails) };
});
}
//more methods constructed in the same fashion
}
then in a separate class in my application I have this method (based off of code from #StephenCleary)
private async Task<ValidationResponse[]> DoOperationsConcurrentlyAsync(string tenantId, string[] emails, string ssn)
{
Task<ValidationResponse>[] tasks =
{
ResolveService<RetreadServices>().Get(new RetreadEmailInfoAsync { TenantId = tenantId, Emails = emails }),
ResolveService<RetreadServices>().Get(new RetreadSsnInfoAsync { TenantId = tenantId, Ssn = ssn })
};
await Task.WhenAll(tasks);
ValidationResponse[] result = new ValidationResponse[tasks.Length];
return result;
}
and being invoked like this
synchronous code...
synchronous code...
var result = DoOperationsConcurrentlyAsync(request.TenantId, request.Emails, request.Ssn);
synchronous code...
synchronous code...
The goal of all of this is to wait while the tasks process in parallel and hopefully decrease my over-all time... but the DoOperationsConcurrentlyAsync needs to block, how do I do that? Then as a bonus, how do I pass the return values back from the individual tasks up and out of the method with the async modifier?
I hope I'm understanding this correctly. But don't you just need to add the await keyword?
synchronous code...
synchronous code...
var result = await DoOperationsConcurrentlyAsync(request.TenantId, request.Emails, request.Ssn);
synchronous code...
synchronous code...
EDIT: Bonus question: Again, I hope I'm understanding you correctly...
But I would replace this:
await Task.WhenAll(tasks);
ValidationResponse[] result = new ValidationResponse[tasks.Length];
return result;
...with simply this:
return await Task.WhenAll(tasks);
That should work the way you want it.