I am writing NUnit Test for async methods and am using extent reporting to report the results. The ExtentTest linked to my test get completed as soon as the await step i the test method has completed execution and am no longer able to access the ExtentTest for any logging purpose. Is there any issue with my code or is this expected ?
Here is my Test method:
[Test, RequiresThread]
public async Task GetList()
{
try
{
ReportHelper.ExtentTestInfo("system.readResources() method is called");
Resources resources = await system.readResources();
ReportHelper.ExtentTestInfo("system.readResources() method finished and responded");
//Test Assertions
}
}
Here is my ReportHelper class:
public class ReportHelper
{
private static ExtentReports TestReportHTML = new ExtentReports();
var htmlReporter = new ExtentV3HtmlReporter("Test_Run_Report_" + #".html");
TestReportHTML.AttachReporter(htmlReporter);
[ThreadStatic] private static ExtentTest _extentTest;
_extentTest = TestReportHTML.CreateTest(testName); //testName is passed during [SetUp]
public static void ExtentTestInfo(string testInfo)
{
_extentTest.Info(testInfo);
}
}
Once the await call has been executed the _extentTest status is passed and on the next line I am getting NullReferenceException for the _extentTest
You need to remove the [ThreadStatic] attribute. That stops the variable being shared between threads but await may cause the code to execute the remaining code on a different thread, which is causing your _extentTest to be null after the await.
Related
I've the following test in Nunit with Moq:
[TestFixture]
public class MessageServiceTests
{
private Mock<IFtpClient> _ftpService;
private IMessageService _messageService;
[SetUp]
public void Setup()
{
_ftpService = new Mock<IFtpClient>();
_messageService = new MessageService(_ftpService.Object);
}
[Test]
public void SendAsync_WithSettings_ConnectsWithCredentials()
{
//act
_messageService.SendAsync(It.IsAny<Stream>(), It.IsAny<String>(), It.IsAny<Settings>());
}
}
and the following method that is tested:
public async Task SendAsync(Stream stream, string fileName, Settings settings)
{
throw new NotImplementedException();
}
Is expect the test to fail, but when I run it in Visual Studio it passes. I can't get my head around it, tests should fail when an unexpected exception is thrown, right? So why does it pass?
SendAsync ran successfully, and returned a Task which contained an exception (its IsFaulted property returns true, and its Exception property contains the NotImplementedException).
However, you're not checking the Task which is returned from SendAsync, so you never realise that it contains an exception.
The easiest way to check the Task for exceptions is using await, and make your test method async Task. This also handles the case where the Task doesn't complete straight away.
[Test]
public async Task SendAsync_WithSettings_ConnectsWithCredentials()
{
//act
await _messageService.SendAsync(It.IsAny<Stream>(), It.IsAny<String>(), It.IsAny<Settings>());
}
I have the following four tests and the last one hangs when I run it. Why does this happen:
[Test]
public void CheckOnceResultTest()
{
Assert.IsTrue(CheckStatus().Result);
}
[Test]
public async void CheckOnceAwaitTest()
{
Assert.IsTrue(await CheckStatus());
}
[Test]
public async void CheckStatusTwiceAwaitTest()
{
Assert.IsTrue(await CheckStatus());
Assert.IsTrue(await CheckStatus());
}
[Test]
public async void CheckStatusTwiceResultTest()
{
Assert.IsTrue(CheckStatus().Result); // This hangs
Assert.IsTrue(await CheckStatus());
}
private async Task<bool> CheckStatus()
{
var restClient = new RestClient(#"https://api.test.nordnet.se/next/1");
Task<IRestResponse<DummyServiceStatus>> restResponse = restClient.ExecuteTaskAsync<DummyServiceStatus>(new RestRequest(Method.GET));
IRestResponse<DummyServiceStatus> response = await restResponse;
return response.Data.SystemRunning;
}
I use this extension method for restsharp RestClient:
public static class RestClientExt
{
public static Task<IRestResponse<T>> ExecuteTaskAsync<T>(this RestClient client, IRestRequest request) where T : new()
{
var tcs = new TaskCompletionSource<IRestResponse<T>>();
RestRequestAsyncHandle asyncHandle = client.ExecuteAsync<T>(request, tcs.SetResult);
return tcs.Task;
}
}
public class DummyServiceStatus
{
public string Message { get; set; }
public bool ValidVersion { get; set; }
public bool SystemRunning { get; set; }
public bool SkipPhrase { get; set; }
public long Timestamp { get; set; }
}
Why does the last test hang?
Acquiring a value via an async method:
var result = Task.Run(() => asyncGetValue()).Result;
Syncronously calling an async method
Task.Run( () => asyncMethod()).Wait();
No deadlock issues will occur due to the use of Task.Run.
You're running into the standard deadlock situation that I describe on my blog and in an MSDN article: the async method is attempting to schedule its continuation onto a thread that is being blocked by the call to Result.
In this case, your SynchronizationContext is the one used by NUnit to execute async void test methods. I would try using async Task test methods instead.
You can avoid deadlock adding ConfigureAwait(false) to this line:
IRestResponse<DummyServiceStatus> response = await restResponse;
=>
IRestResponse<DummyServiceStatus> response = await restResponse.ConfigureAwait(false);
I've described this pitfall in my blog post Pitfalls of async/await
You are blocking the UI by using Task.Result property.
In MSDN Documentation they have clearly mentioned that,
"The Result property is a blocking property. If you try to access it
before its task is finished, the thread that's currently active is
blocked until the task completes and the value is available. In most
cases, you should access the value by using Await or await instead of
accessing the property directly."
The best solution for this scenario would be to remove both await & async from methods & use only Task where you're returning result. It won't mess your execution sequence.
An addition to the answer given by #HermanSchoenfeld. Unfortunately the quote below is not true:
No deadlock issues will occur due to the use of Task.Run.
public String GetSqlConnString(RubrikkUser user, RubrikkDb db)
{
// deadlock if called from threadpool,
// works fine on UI thread, works fine from console main
return Task.Run(() =>
GetSqlConnStringAsync(user, db)).Result;
}
The execution is wrapped inside a Task.Run, this will schedule the task on the threadpool the block the calling thread. This is okay, as long as the calling thread is not a threadpool thread. If the calling thread is from the threadpool then the following disaster happens: A new task is queued to the end of the queue, and the threadpool thread which would eventually execute the Task is blocked until the Task is executed.
In library code there is no easy solution as you cannot assume under what context your code is called. The best solution is to only call async code from async code, blocking sync APIs from sync methods, don’t mix them.
Source:
https://medium.com/rubrikkgroup/understanding-async-avoiding-deadlocks-e41f8f2c6f5d
If you don't get any callbacks or the control hangs up, after calling the service/API async function, you have to configure Context to return a result on the same called context.
Use TestAsync().ConfigureAwait(continueOnCapturedContext: false);
You will be facing this issue only in web applications, but not in static void main.
I have the following four tests and the last one hangs when I run it. Why does this happen:
[Test]
public void CheckOnceResultTest()
{
Assert.IsTrue(CheckStatus().Result);
}
[Test]
public async void CheckOnceAwaitTest()
{
Assert.IsTrue(await CheckStatus());
}
[Test]
public async void CheckStatusTwiceAwaitTest()
{
Assert.IsTrue(await CheckStatus());
Assert.IsTrue(await CheckStatus());
}
[Test]
public async void CheckStatusTwiceResultTest()
{
Assert.IsTrue(CheckStatus().Result); // This hangs
Assert.IsTrue(await CheckStatus());
}
private async Task<bool> CheckStatus()
{
var restClient = new RestClient(#"https://api.test.nordnet.se/next/1");
Task<IRestResponse<DummyServiceStatus>> restResponse = restClient.ExecuteTaskAsync<DummyServiceStatus>(new RestRequest(Method.GET));
IRestResponse<DummyServiceStatus> response = await restResponse;
return response.Data.SystemRunning;
}
I use this extension method for restsharp RestClient:
public static class RestClientExt
{
public static Task<IRestResponse<T>> ExecuteTaskAsync<T>(this RestClient client, IRestRequest request) where T : new()
{
var tcs = new TaskCompletionSource<IRestResponse<T>>();
RestRequestAsyncHandle asyncHandle = client.ExecuteAsync<T>(request, tcs.SetResult);
return tcs.Task;
}
}
public class DummyServiceStatus
{
public string Message { get; set; }
public bool ValidVersion { get; set; }
public bool SystemRunning { get; set; }
public bool SkipPhrase { get; set; }
public long Timestamp { get; set; }
}
Why does the last test hang?
Acquiring a value via an async method:
var result = Task.Run(() => asyncGetValue()).Result;
Syncronously calling an async method
Task.Run( () => asyncMethod()).Wait();
No deadlock issues will occur due to the use of Task.Run.
You're running into the standard deadlock situation that I describe on my blog and in an MSDN article: the async method is attempting to schedule its continuation onto a thread that is being blocked by the call to Result.
In this case, your SynchronizationContext is the one used by NUnit to execute async void test methods. I would try using async Task test methods instead.
You can avoid deadlock adding ConfigureAwait(false) to this line:
IRestResponse<DummyServiceStatus> response = await restResponse;
=>
IRestResponse<DummyServiceStatus> response = await restResponse.ConfigureAwait(false);
I've described this pitfall in my blog post Pitfalls of async/await
You are blocking the UI by using Task.Result property.
In MSDN Documentation they have clearly mentioned that,
"The Result property is a blocking property. If you try to access it
before its task is finished, the thread that's currently active is
blocked until the task completes and the value is available. In most
cases, you should access the value by using Await or await instead of
accessing the property directly."
The best solution for this scenario would be to remove both await & async from methods & use only Task where you're returning result. It won't mess your execution sequence.
An addition to the answer given by #HermanSchoenfeld. Unfortunately the quote below is not true:
No deadlock issues will occur due to the use of Task.Run.
public String GetSqlConnString(RubrikkUser user, RubrikkDb db)
{
// deadlock if called from threadpool,
// works fine on UI thread, works fine from console main
return Task.Run(() =>
GetSqlConnStringAsync(user, db)).Result;
}
The execution is wrapped inside a Task.Run, this will schedule the task on the threadpool the block the calling thread. This is okay, as long as the calling thread is not a threadpool thread. If the calling thread is from the threadpool then the following disaster happens: A new task is queued to the end of the queue, and the threadpool thread which would eventually execute the Task is blocked until the Task is executed.
In library code there is no easy solution as you cannot assume under what context your code is called. The best solution is to only call async code from async code, blocking sync APIs from sync methods, don’t mix them.
Source:
https://medium.com/rubrikkgroup/understanding-async-avoiding-deadlocks-e41f8f2c6f5d
If you don't get any callbacks or the control hangs up, after calling the service/API async function, you have to configure Context to return a result on the same called context.
Use TestAsync().ConfigureAwait(continueOnCapturedContext: false);
You will be facing this issue only in web applications, but not in static void main.
I have a this code
public class ClassToTest
{
private readonly IRepository repository;
public ClassToTest(DI GOES HERE){...}
public DoSomething()
{
Task.Run(async () => {
//some code
repository.ExecuteAsync();
}
}
}
public class Repository : IRepository
{
public Task ExecuteAsync()
{
using (var connection = new SqlConnection(DbConfiguration.DatabaseConnection))
{
return connection.ExecuteAsync(storedProcedure, parameters, commandType: CommandType.StoredProcedure, commandTimeout: Configuration.TransactionTimeout);
}
}
}
[Test]
public void TestMethod()
{
var repository = new Mock<IRepository>;
var classToTest = new ClassToTest();
classToTest.DoSomething();
repository.Veryfy(p => p.ExecuteAsync(), Times.Once());
}
The test fails with this message
Expected invocation on the mock once, but was 0 times: p => p.ExecuteAsync()
Does anyone knows why?
Thanks
As others have alluded, because you're calling Task.Run and not waiting for a response, the Unit test will likely complete before the background task is even started, hence the Moq Verify failure.
Also, your code won't compile as is - when asking a Q on StackOverflow, be sure to give a complete, compilable MVP.
Of special importance is the bug in the code you are trying to test. Repository.ExecuteAsync calls connection.ExecuteAsync, inside a using scope, but this isn't awaited. This will mean that the connection will be disposed before the task completes. You'll need to change the method to async and await the call to defer disposal of the connection.
The wrapper method DoSomething method shouldn't use Task.Run(), although, because it adds no value to the repository Task, it doesn't need to repeat the async / return await, either.
The caller (your Unit test, in this instance) can then await DoSomething (or if the caller genuinely wants to do further processing without awaiting the Task, then leave it to the caller to decide. At least this way, the caller gets a handle to the Task, to check on completion).
The final state of your code might look more like this:
public class ClassToTest
{
private readonly IRepository _repository;
public ClassToTest(IRepository repository)
{
_repository = repository;
}
// Doesn't necessarily need to be async
public Task DoSomething()
{
// We're return the wrapped task directly, and adding no additional value.
return repository.ExecuteAsync();
}
}
public class Repository : IRepository
{
public async Task ExecuteAsync()
{
using (var connection = new SqlConnection(DbConfiguration.DatabaseConnection))
{
// Here we do need to await, otherwise we'll dispose the connection
return await connection.ExecuteAsync(storedProcedure, parameters,
commandType: CommandType.StoredProcedure,
commandTimeout: Configuration.TransactionTimeout);
}
}
}
// NUnit has full support for async / await
[Test]
public async Task TestMethod()
{
var repository = new Mock<IRepository>();
var classToTest = new ClassToTest(repository.Object);
repository.Setup(_ => _.ExecuteAsync()).Returns(Task.FromResult((object)null));
// Moq also has support for async, e.g. .ReturnsAsync
// You need to await the test.
await classToTest.DoSomething();
repository.Verify(p => p.ExecuteAsync(), Times.Once());
}
I have a situation where I need to call a web service and, on successful completion, do multiple things with the results returned from the web service. I have developed code that "works" -- just not as I intended. Specifically, I want to take the results from the call to the web service and pass those results onto multiple successive tasks that are to execute in parallel, but what I have at the moment executes the first successive task before starting the second.
I've put together a much simplified example of what I'm currently doing that'll hopefully help illustrate this situation. First, the implementation:
public interface IConfigurationSettings
{
int? ConfigurationSetting { get; set; }
}
public interface IPrintCommandHandler
{
System.Threading.Tasks.Task<bool> ExecuteAsync(byte[] reportContent);
}
public interface ISaveCommandHandler
{
System.Threading.Tasks.Task<bool> ExecuteAsync(byte[] reportContent);
}
public interface IWebService
{
System.Threading.Tasks.Task<object> RetrieveReportAsync(string searchToken, string reportFormat);
}
public class ReportCommandHandler
{
private readonly IConfigurationSettings _configurationSettings;
private readonly IPrintCommandHandler _printCommandHandler;
private readonly ISaveCommandHandler _saveCommandHandler;
private readonly IWebService _webService;
public ReportCommandHandler(IWebService webService, IPrintCommandHandler printCommandHandler, ISaveCommandHandler saveCommandHandler, IConfigurationSettings configurationSettings)
{
_webService = webService;
_printCommandHandler = printCommandHandler;
_saveCommandHandler = saveCommandHandler;
_configurationSettings = configurationSettings;
}
public async Task<bool> ExecuteAsync(string searchToken)
{
var reportTask = _webService.RetrieveReportAsync(searchToken, "PDF");
var nextStepTasks = new List<Task<bool>>();
// Run "print" task after report task.
var printTask = await reportTask.ContinueWith(task => _printCommandHandler.ExecuteAsync((byte[]) task.Result));
nextStepTasks.Add(printTask);
// Run "save" task after report task.
if (_configurationSettings.ConfigurationSetting.HasValue)
{
var saveTask = await reportTask.ContinueWith(task => _saveCommandHandler.ExecuteAsync((byte[]) task.Result));
nextStepTasks.Add(saveTask);
}
var reportTaskResult = await Task.WhenAll(nextStepTasks);
return reportTaskResult.Aggregate(true, (current, result) => current & result);
}
}
So, the web service (third party, nothing to do with me) has an endpoint for doing a search/lookup that, if successful, returns a reference number (I've called it a search token in my example). This reference number is then used to retrieve the results of the lookup (using a different endpoint) in any of several different formats.
The IWebService interface in this example is representative of an application service I created to manage interaction with the web service. The actual implementation has other methods on it for doing a lookup, ping, etc.
Just to make things more interesting, one of the successive tasks is required (will always execute after the primary task) but the other successive task is optional, execution subject to a configuration setting set elsewhere in the application.
To more easily demonstrate the issue, I created a unit test:
public class RhinoMockRepository : IDisposable
{
private readonly ArrayList _mockObjectRepository;
public RhinoMockRepository()
{
_mockObjectRepository = new ArrayList();
}
public T CreateMock<T>() where T : class
{
var mock = MockRepository.GenerateMock<T>();
_mockObjectRepository.Add(mock);
return mock;
}
public T CreateStub<T>() where T : class
{
return MockRepository.GenerateStub<T>();
}
public void Dispose()
{
foreach (var obj in _mockObjectRepository) obj.VerifyAllExpectations();
_mockObjectRepository.Clear();
}
}
[TestFixture]
public class TapTest
{
private const string SearchToken = "F71C8B50-ECD1-4C02-AD3F-6C24F1AF3D9A";
[Test]
public void ReportCommandExecutesPrintAndSave()
{
using (var repository = new RhinoMockRepository())
{
// Arrange
const string reportContent = "This is a PDF file.";
var reportContentBytes = System.Text.Encoding.Default.GetBytes(reportContent);
var retrieveReportResult = System.Threading.Tasks.Task.FromResult<object>(reportContentBytes);
var webServiceMock = repository.CreateMock<IWebService>();
webServiceMock.Stub(x => x.RetrieveReportAsync(SearchToken, "PDF")).Return(retrieveReportResult);
var printCommandHandlerMock = repository.CreateMock<IPrintCommandHandler>();
var printResult = System.Threading.Tasks.Task.FromResult(true);
printCommandHandlerMock
.Expect(x => x.ExecuteAsync(reportContentBytes))
//.WhenCalled(method => System.Threading.Thread.Sleep(TimeSpan.FromSeconds(2)))
.Return(printResult);
var configurationSettingsStub = repository.CreateStub<IConfigurationSettings>();
configurationSettingsStub.ConfigurationSetting = 10;
var saveCommandHandlerMock = repository.CreateMock<ISaveCommandHandler>();
var saveResult = System.Threading.Tasks.Task.FromResult(true);
saveCommandHandlerMock.Expect(x => x.ExecuteAsync(reportContentBytes)).Return(saveResult);
// Act
var reportCommandHandler = new ReportCommandHandler(webServiceMock, printCommandHandlerMock, saveCommandHandlerMock, configurationSettingsStub);
var result = System.Threading.Tasks.Task
.Run(async () => await reportCommandHandler.ExecuteAsync(SearchToken))
.Result;
// Assert
Assert.That(result, Is.True);
}
}
}
Ideally, on completion of the call to RetrieveReportAsync() on IWebService both the "print" and "save" command handlers should be executed simultaneously, having received a copy of the results from RetrieveReportAsync(). However, if the call to WhenCalled... in the unit test is uncommented, and on stepping through the implementation of ReportCommandHandler.ExecuteAsync(), you can see that the "print" command executes and completes before it gets to the "save" command. Now, I am aware that the whole point of await is to suspend execution of the calling async method until the awaited code completes, but it isn't clear to me how to instantiate both the "print" and "save" commands (tasks) as continuations of the "report" task such that they both execute in parallel when the "report" task completes, and the "report" command is then able to return a result that is based on the results from both the "print" and "save" commands (tasks).
Your question really involves addressing two different goals:
How to wait for a task?
How to execute two other tasks concurrently?
I find the mixing of await and ContinueWith() in your code confusing. It's not clear to me why you did that. One of the key things await does for you is to automatically set up a continuation, so you don't have to call ContinueWith() explicitly. Yet, you do anyway.
On the assumption that's simply a mistake, out of lack of full understanding of how to accomplish your goal, here's how I'd have written your method:
public async Task<bool> ExecuteAsync(string searchToken)
{
var reportTaskResult = await _webService.RetrieveReportAsync(searchToken, "PDF");
var nextStepTasks = new List<Task<bool>>();
// Run "print" task after report task.
var printTask = _printCommandHandler.ExecuteAsync((byte[]) reportTaskResult);
nextStepTasks.Add(printTask);
// Run "save" task after report task.
if (_configurationSettings.ConfigurationSetting.HasValue)
{
var saveTask = _saveCommandHandler.ExecuteAsync((byte[]) reportTaskResult);
nextStepTasks.Add(saveTask);
}
var reportTaskResult = await Task.WhenAll(nextStepTasks);
return reportTaskResult.Aggregate(false, (current, result) => current | result);
}
In other words, do await the original task first. Then you know it's done and have its result. At that time, go ahead and start the other tasks, adding their Task object to your list, but not awaiting each one individually. Finally, await the entire list of tasks.