I have a unit test (using MSTest) like so:
[TestMethod]
public void MyTest()
{
var viewModel = new MyViewModel();
viewModel.Run();
//Assert something here
}
Run is an async method that returns void.
Let's say Run is implemented like so:
public async void Run()
{
//Show a busy indicator here
try
{
var result = await myAsyncModelClass.LongRunningOperation();
//Use the results here
}
finally
{
//Hide the busy indicator here
}
}
myAsyncModelClass.LongRunningOperation(), is itself an async method that returns some Task<T> where T is the result my ViewModel is interested in.
My issue, is that my test is running the Run method asynchronously, so the my assertions are called before the Run methods completes. It is odd, b/c the finally block is never reached when I put a breakpoint, since the assertions fail. How can I keep the Run method synchronous to be able to unit test it?
I have a unit test of myAsyncModelClass.LongRunningOperation() also, but I merely call Task<T>.Wait() since it returns a task. This makes it synchronous when unit testing.
Also, I would like to mention, Run() is invoke by an ICommand magically by an MVVM framework. void may or may not be a require return type, I will have to try it out.
Async methods need a context to "return to". Since MSTests run on the thread pool, by default the async methods all continue on a thread pool thread as well (and do not block the MSTest method).
Under the (C# Testing) Unit Testing sample (in your Async CTP install directory), there's a type called GeneralThreadAffineContext, which can be used as such:
[TestMethod]
public void MyTest()
{
MyViewModel viewModel = null;
GeneralThreadAffineContext.Run(() =>
{
viewModel = new MyViewModel();
viewModel.Run();
});
//Assert something here
}
There are also specific WPF and WinForms contexts, but the thread-affine context should work for general ViewModels (that don't make explicit use of Dispatcher).
Update 2012-02-05: If you can change your ViewModel method to return Task, then you have another option: the new AsyncUnitTests library. Install that NuGet package, change your TestClass to AsyncTestClass, and your async unit tests can be written much more naturally:
[TestMethod]
public async void MyTest()
{
MyViewModel viewModel = new MyViewModel();
await viewModel.Run();
//Assert something here
}
Update 2012-09-04: Visual Studio 2012 includes async unit testing, so you don't need the AsyncUnitTests library anymore:
[TestMethod]
public async Task MyTest()
{
MyViewModel viewModel = new MyViewModel();
await viewModel.Run();
//Assert something here
}
Since Visual Studio 2012 MSTest supports async test methods. Just remeber they should return Task instead of void:
[TestMethod]
public async Task MyTest()
{
MyViewModel viewModel = new MyViewModel();
await viewModel.Run();
//Assert something here
}
Related
I am writing NUnit Test for async methods and am using extent reporting to report the results. The ExtentTest linked to my test get completed as soon as the await step i the test method has completed execution and am no longer able to access the ExtentTest for any logging purpose. Is there any issue with my code or is this expected ?
Here is my Test method:
[Test, RequiresThread]
public async Task GetList()
{
try
{
ReportHelper.ExtentTestInfo("system.readResources() method is called");
Resources resources = await system.readResources();
ReportHelper.ExtentTestInfo("system.readResources() method finished and responded");
//Test Assertions
}
}
Here is my ReportHelper class:
public class ReportHelper
{
private static ExtentReports TestReportHTML = new ExtentReports();
var htmlReporter = new ExtentV3HtmlReporter("Test_Run_Report_" + #".html");
TestReportHTML.AttachReporter(htmlReporter);
[ThreadStatic] private static ExtentTest _extentTest;
_extentTest = TestReportHTML.CreateTest(testName); //testName is passed during [SetUp]
public static void ExtentTestInfo(string testInfo)
{
_extentTest.Info(testInfo);
}
}
Once the await call has been executed the _extentTest status is passed and on the next line I am getting NullReferenceException for the _extentTest
You need to remove the [ThreadStatic] attribute. That stops the variable being shared between threads but await may cause the code to execute the remaining code on a different thread, which is causing your _extentTest to be null after the await.
Consider the following highly simplified viewmodel for fetching and showing a list of projects:
public class ProjectListViewModel
{
private readonly IWebService _webService;
public ICommand RefreshCommand { get; }
// INotifyPropertyChanged implementation skipped for brevity
public ObservableCollection<Project> Projects { get; set; }
public ProjectListViewModel(IWebService serverApi)
{
_serverApi = serverApi;
// ICommand implemented by Xamarin.Forms
RefreshCommand = new Command(async () => await RefreshAsync());
}
private async Task RefreshAsync()
{
try
{
Projects = await _webService.GetProjectsAsync();
}
catch (TaskCanceledException)
{
// Empty (task only cancelled when we are navigating away from page)
}
}
}
Using NUnit and Moq, I'm trying test that when GetProjectsAsync throws a TaskCanceledException, the ViewModel will catch it. The closest I get is this:
[Test]
public void When_Refreshing_Catches_TaskCanceledException()
{
// Arrange
webService = new Mock<IServerApi>();
webService.Setup(mock => mock.GetProjectsAsync())
.ThrowsAsync(new TaskCanceledException());
vm = new ProjectListViewModel(webService.Object);
// Act and assert
Assert.That(() => vm.RefreshCommand.Execute(null), Throws.Nothing);
}
The test passes, but unfortunately it's faulty - it still passes if I throw e.g. Exception instead of TaskCanceledException. As far as I know, the reason is that the exception doesn't bubble up past the command lambda, async () => await RefreshAsync(), so no exception thrown by GetProjectsAsync will ever be detected by the test. (When running the actual app however, the TaskCanceledException will bubble up and crash the app if not caught. I suspect this is related to synchronization contexts, of which I have very limited understanding.)
It works if I debug the test - if I mock it to throw Exception, it will break on the line with the command/lambda definition, and if I throw TaskCanceledException, the test will pass.
Note that the results are the same if I use Throws instead of ThrowsAsync. And in case it's relevant, I'm using the test runner in ReSharper 2016.2.
Using nUnit, is it possible at all to unit test exceptions thrown when executing "async" commands like this? Is it possible without writing a custom Command implementation?
Your problem is here:
new Command(async () => await RefreshAsync())
This async lambda is converted to an async void method by the compiler.
In my article on async best practices, I explain why the exception cannot be caught like this. async methods cannot propagate their exceptions directly (since their stack can be gone by the time the exception happens). async Task methods solve this naturally by placing the exception on their returned task. async void methods are unnatural, and they have nowhere to place the exception, so they raise it directly on the SynchronizationContext that was current at the time the method started.
In your application, this is the UI context, so it's just like it was thrown directly in an event handler. In your unit test, there is no context, so it's thrown on a thread pool thread. I think NUnit's behavior in this situation is to catch the exception and dump it to the console.
Personally, I prefer using my own asynchronous-compatible ICommand such as AsyncCommand in my Mvvm.Async library (also see my article on asynchronous MVVM commands):
new AsyncCommand(_ => RefreshAsync())
which can then be naturally unit tested:
await vm.RefreshCommand.ExecuteAsync(null); // does not throw
Alternatively, you can provide your own synchronization context in the unit test (using, e.g., my AsyncContext):
// Arrange
webService = new Mock<IServerApi>();
webService.Setup(mock => mock.GetProjectsAsync())
.ThrowsAsync(new TaskCanceledException());
vm = new ProjectListViewModel(webService.Object);
// Act/Assert
AsyncContext.Run(() => vm.RefreshCommand.Execute(null));
In this case, if there was an exception, Run would propagate it.
Since async void (which is what the handler to your command is) is basically "fire and forget" and you can't await for it in the test I would suggest unit testing the RefreshAsync() method (you may want to make it internal or public), this can be easily done in NUnit:
if you are asserting exceptions being thrown:
[Test]
public async Task Test_Exception_RefreshAsync(){
try
{
await vm.RefreshAsync();
Assert.Fail("No exception was thrown");
}
catch (NotImplementedException e)
{
// Pass
}
}
or simply
[Test]
public async Task Test_RefreshAsync(){
var vm = new ProjectListViewModel(...);
await vm.RefreshAsync();
//Assertions here
}
or as other answer state you can create your own AsyncCommand that you can await on.
I am just looking at best practices for unit testing (NUnit) ICommand and specifically the MvxCommand implementation within MVVMCross
View Model
public ICommand GetAuthorisationCommand
{
get { return new MvxCommand(
async () => await GetAuthorisationToken(),
() => !string.IsNullOrWhiteSpace(UserName) && !string.IsNullOrWhiteSpace(Password)); }
}
private async Task GetAuthorisationToken()
{
// ...Do something async
}
Unit Test
[Test]
public async Task DoLogonCommandTest()
{
//Arrange
ViewModel vm = new ViewModel(clubCache, authorisationCache, authorisationService);
//Act
await Task.Run(() => vm.GetAuthorisationToken.Execute(null));
//Assert
Assert.Greater(MockDispatcher.Requests.Count, 0);
}
Now the problem I have is that the tests drop through with out awaiting the async operations and this feels a little hacky in calling the async method from the ICommand.
Are there any best practices in unit testing these kind of ICommands and async methods?
You can use MvxAsyncCommand (see: implementation) instead of MvxCommand and change the published type of GetAuthorisationCommand from ICommand to IMvxAsyncCommand (but that interface is not available via nuget, yet) and then you can call
await vm.GetAuthorisationToken.ExecuteAsync();
I think the answer with MvxAsyncCommand is the best long-term solution.
However, if you want something that works today without depending on prerelease software, you can follow this pattern which I have found helpful when dealing with asynchronous MVVM commands.
First, define an IAsyncCommand:
interface IAsyncCommand: ICommand
{
Task ExecuteAsync(object parameter);
}
Then you can define an AsyncCommand implementation as such:
public class AsyncCommand: MvxCommand, IAsyncCommand
{
private readonly Func<Task> _execute;
public AsyncCommand(Func<Task> execute)
: this(execute, null)
{
}
public AsyncCommand(Func<Task> execute, Func<bool> canExecute)
: base(async () => await execute(), canExecute)
{
_execute = execute;
}
public Task ExecuteAsync()
{
_execute();
}
}
And then use await command.ExecuteAsync() instead of command.Execute() in your unit tests.
As a command is a fire and forget event you dont get back a completion directly.
I would suggest splitting the test into two actions (or even creating two Unit test).
Test if the Command can be executed
Test if the Async Task return expected result
Something along the lines of:
//Act
var canExecute = vm.GetAuthorisationToken.CanExecute();
var result = await vm.GetAuthorisationToken();
However, is would require GetAuthorisationToken to change its protection level from private inorder to expose it for the unit test.
Alternatively
You can make use of a library such as AsyncEx, which can allow you to await the completion of the async call.
[Test]
public async Task DoLogonCommandTest()
{
AsyncContext.Run(() =>
{
//Arrange
ViewModel vm = new ViewModel(clubCache, authorisationCache, authorisationService);
//Act
await Task.Run(() => vm.GetAuthorisationToken.Execute(null));
});
//Assert
Assert.Greater(MockDispatcher.Requests.Count, 0);
}
I come across on this test method in NUnit services tests in project I am working at a moment
[Test]
public void FindAsync_By_Using_Some_Condition()
{
//code omitted for clarity ...
var results = Repository.FindAsync(workClients);
results.Wait(TimeSpan.FromSeconds(1));
Assert.AreEqual(2, results.Result.Count);
}
public async Task<ICollection<T>> FindAsync( FindOptions<T> options )
{
var output = new List<T>();
//code omitted for clarity ...
return await Task.FromResult(output);
}
where they have been using Task.Wait before they did assertion. Any particular reason why it was done this way? What would have happened if they omitted Task.Wait (TimeSpan.FromSeconds(1))?
And why FindAsync_By_Using_Some_Condition() is not used in conjunction with async / await keywords?
The test was probably written before NUnit supported async. With NUnit 2.6.4 or 3.x, it could now be rewritten,
[Test]
public async Task FindAsync_By_Using_Some_Condition()
{
var results = await Repository.FindAsync(workClients);
Assert.AreEqual(2, results.Count);
}
I am trying to discover how to apply the async and await keywords to my xUnit tests. I am using xUnit 1.9 and Async CTP 1.3. Here is my test case
I have an interface which specifies one asynchronous method call
public interface IDoStuffAsync
{
Task AnAsyncMethod(string value);
}
I have a class which consumes the interface and calls the async method
public class UseAnAsyncThing
{
private readonly IDoStuffAsync _doStuffAsync;
public UseAnAsyncThing(IDoStuffAsync doStuffAsync)
{
_doStuffAsync = doStuffAsync;
}
public async Task DoThatAsyncOperation(string theValue)
{
await _doStuffAsync.AnAsyncMethod(theValue);
}
}
In my tests I wish to check that the method DoThatAsyncOperation is calling the method with the correct value so I mock the interface and use the Moq to verify the call
[Fact]
public async void The_test_will_pass_even_though_it_should_fail()
{
var mock = new Mock<IDoStuffAsync>();
var sut = new UseAnAsyncThing(mock.Object);
mock.Setup(x => x.AnAsyncMethod(It.IsAny<string>()));
await sut.DoThatAsyncOperation("test");
// This won't throw a Moq.MockExcpetion so the test appears to pass
// However it does not run
mock.Verify(x => x.AnAsyncMethod("fail"));
}
This test is using the async and await keywords. When it runs it erroneously passes as Moq should assert that the verify fails. Any code after the call to sut.DoThatAsyncOperation("test"); does not run
[Fact]
public void This_will_work_and_assert_the_reslt()
{
var mock = new Mock<IDoStuffAsync>();
var sut = new UseAnAsyncThing(mock.Object);
mock.Setup(x => x.AnAsyncMethod(It.IsAny<string>()));
sut.DoThatAsyncOperation("test").ContinueWith(y => { });
// This won't throw a Moq.MockExcpetion so the test appears to pass
// However it does not run
mock.Verify(x => x.AnAsyncMethod("fail"));
}
This test is setup without the await and async keywords and passes fine.
Is this expected behavior for xUnit and Moq?
Update
Thanks for Stephen's comment I managed to fix the first test by making two changes. The test now returns a Task instead of void and the Mock also returns a Task.
[Fact]
public async Task The_test_will_pass_even_though_it_should_fail()
{
var mock = new Mock<IDoStuffAsync>();
var sut = new UseAnAsyncThing(mock.Object);
mock.Setup(x => x.AnAsyncMethod(It.IsAny<string>())).ReturnAsync(true);
await sut.DoThatAsyncOperation("test");
// This now fails as it should
mock.Verify(x => x.AnAsyncMethod("fail"));
}
Change your unit test method to return Task instead of void, and it should work. Support for async void unit tests is being considered for a future release.
I describe in detail why async unit tests don't work by default on my blog. (My blog examples use MSTest, but the same problems existed in every other test runner, including xUnit pre-1.9).
I tried to use the code from your 'Update', but it was stopping at the async method that I was mocking.
var tcs = new TaskCompletionSource<T>();
tcs.SetResult(default(T));
mock.Setup(x => x.AnAsyncMethod(It.IsAny<T>())).Returns(tcs.Task);
So to fix that I had to change the 'Return' method:
mock.Setup(x => x.AnAsyncMethod(It.IsAny<T>())).Returns(()=> { return tcs.Task; } );