I am trying to discover how to apply the async and await keywords to my xUnit tests. I am using xUnit 1.9 and Async CTP 1.3. Here is my test case
I have an interface which specifies one asynchronous method call
public interface IDoStuffAsync
{
Task AnAsyncMethod(string value);
}
I have a class which consumes the interface and calls the async method
public class UseAnAsyncThing
{
private readonly IDoStuffAsync _doStuffAsync;
public UseAnAsyncThing(IDoStuffAsync doStuffAsync)
{
_doStuffAsync = doStuffAsync;
}
public async Task DoThatAsyncOperation(string theValue)
{
await _doStuffAsync.AnAsyncMethod(theValue);
}
}
In my tests I wish to check that the method DoThatAsyncOperation is calling the method with the correct value so I mock the interface and use the Moq to verify the call
[Fact]
public async void The_test_will_pass_even_though_it_should_fail()
{
var mock = new Mock<IDoStuffAsync>();
var sut = new UseAnAsyncThing(mock.Object);
mock.Setup(x => x.AnAsyncMethod(It.IsAny<string>()));
await sut.DoThatAsyncOperation("test");
// This won't throw a Moq.MockExcpetion so the test appears to pass
// However it does not run
mock.Verify(x => x.AnAsyncMethod("fail"));
}
This test is using the async and await keywords. When it runs it erroneously passes as Moq should assert that the verify fails. Any code after the call to sut.DoThatAsyncOperation("test"); does not run
[Fact]
public void This_will_work_and_assert_the_reslt()
{
var mock = new Mock<IDoStuffAsync>();
var sut = new UseAnAsyncThing(mock.Object);
mock.Setup(x => x.AnAsyncMethod(It.IsAny<string>()));
sut.DoThatAsyncOperation("test").ContinueWith(y => { });
// This won't throw a Moq.MockExcpetion so the test appears to pass
// However it does not run
mock.Verify(x => x.AnAsyncMethod("fail"));
}
This test is setup without the await and async keywords and passes fine.
Is this expected behavior for xUnit and Moq?
Update
Thanks for Stephen's comment I managed to fix the first test by making two changes. The test now returns a Task instead of void and the Mock also returns a Task.
[Fact]
public async Task The_test_will_pass_even_though_it_should_fail()
{
var mock = new Mock<IDoStuffAsync>();
var sut = new UseAnAsyncThing(mock.Object);
mock.Setup(x => x.AnAsyncMethod(It.IsAny<string>())).ReturnAsync(true);
await sut.DoThatAsyncOperation("test");
// This now fails as it should
mock.Verify(x => x.AnAsyncMethod("fail"));
}
Change your unit test method to return Task instead of void, and it should work. Support for async void unit tests is being considered for a future release.
I describe in detail why async unit tests don't work by default on my blog. (My blog examples use MSTest, but the same problems existed in every other test runner, including xUnit pre-1.9).
I tried to use the code from your 'Update', but it was stopping at the async method that I was mocking.
var tcs = new TaskCompletionSource<T>();
tcs.SetResult(default(T));
mock.Setup(x => x.AnAsyncMethod(It.IsAny<T>())).Returns(tcs.Task);
So to fix that I had to change the 'Return' method:
mock.Setup(x => x.AnAsyncMethod(It.IsAny<T>())).Returns(()=> { return tcs.Task; } );
Related
I have a this code
public class ClassToTest
{
private readonly IRepository repository;
public ClassToTest(DI GOES HERE){...}
public DoSomething()
{
Task.Run(async () => {
//some code
repository.ExecuteAsync();
}
}
}
public class Repository : IRepository
{
public Task ExecuteAsync()
{
using (var connection = new SqlConnection(DbConfiguration.DatabaseConnection))
{
return connection.ExecuteAsync(storedProcedure, parameters, commandType: CommandType.StoredProcedure, commandTimeout: Configuration.TransactionTimeout);
}
}
}
[Test]
public void TestMethod()
{
var repository = new Mock<IRepository>;
var classToTest = new ClassToTest();
classToTest.DoSomething();
repository.Veryfy(p => p.ExecuteAsync(), Times.Once());
}
The test fails with this message
Expected invocation on the mock once, but was 0 times: p => p.ExecuteAsync()
Does anyone knows why?
Thanks
As others have alluded, because you're calling Task.Run and not waiting for a response, the Unit test will likely complete before the background task is even started, hence the Moq Verify failure.
Also, your code won't compile as is - when asking a Q on StackOverflow, be sure to give a complete, compilable MVP.
Of special importance is the bug in the code you are trying to test. Repository.ExecuteAsync calls connection.ExecuteAsync, inside a using scope, but this isn't awaited. This will mean that the connection will be disposed before the task completes. You'll need to change the method to async and await the call to defer disposal of the connection.
The wrapper method DoSomething method shouldn't use Task.Run(), although, because it adds no value to the repository Task, it doesn't need to repeat the async / return await, either.
The caller (your Unit test, in this instance) can then await DoSomething (or if the caller genuinely wants to do further processing without awaiting the Task, then leave it to the caller to decide. At least this way, the caller gets a handle to the Task, to check on completion).
The final state of your code might look more like this:
public class ClassToTest
{
private readonly IRepository _repository;
public ClassToTest(IRepository repository)
{
_repository = repository;
}
// Doesn't necessarily need to be async
public Task DoSomething()
{
// We're return the wrapped task directly, and adding no additional value.
return repository.ExecuteAsync();
}
}
public class Repository : IRepository
{
public async Task ExecuteAsync()
{
using (var connection = new SqlConnection(DbConfiguration.DatabaseConnection))
{
// Here we do need to await, otherwise we'll dispose the connection
return await connection.ExecuteAsync(storedProcedure, parameters,
commandType: CommandType.StoredProcedure,
commandTimeout: Configuration.TransactionTimeout);
}
}
}
// NUnit has full support for async / await
[Test]
public async Task TestMethod()
{
var repository = new Mock<IRepository>();
var classToTest = new ClassToTest(repository.Object);
repository.Setup(_ => _.ExecuteAsync()).Returns(Task.FromResult((object)null));
// Moq also has support for async, e.g. .ReturnsAsync
// You need to await the test.
await classToTest.DoSomething();
repository.Verify(p => p.ExecuteAsync(), Times.Once());
}
I come across on this test method in NUnit services tests in project I am working at a moment
[Test]
public void FindAsync_By_Using_Some_Condition()
{
//code omitted for clarity ...
var results = Repository.FindAsync(workClients);
results.Wait(TimeSpan.FromSeconds(1));
Assert.AreEqual(2, results.Result.Count);
}
public async Task<ICollection<T>> FindAsync( FindOptions<T> options )
{
var output = new List<T>();
//code omitted for clarity ...
return await Task.FromResult(output);
}
where they have been using Task.Wait before they did assertion. Any particular reason why it was done this way? What would have happened if they omitted Task.Wait (TimeSpan.FromSeconds(1))?
And why FindAsync_By_Using_Some_Condition() is not used in conjunction with async / await keywords?
The test was probably written before NUnit supported async. With NUnit 2.6.4 or 3.x, it could now be rewritten,
[Test]
public async Task FindAsync_By_Using_Some_Condition()
{
var results = await Repository.FindAsync(workClients);
Assert.AreEqual(2, results.Count);
}
I'm using Microsoft Fakes to Shim an async method that invokes another method to get an implemented DbContext. Because database connection string is not supplied in the Unit Test while the method being invoked inside the async method needs it. Shim will not only skip the method that uses the connection string, but returns a customizable DbContext.
Here is the aysnc method implementation:
public async Task<AccountDataDataContext> GetAccountDataInstance(int accountId)
{
var account = await this.Accounts.FindAsync(accountId);
return AccountDataDataContext.GetInstance(account.AccountDataConnectionString);
}
However, I'm not familiar with Shim async method. What I did look like this:
ConfigurationEntities.Fakes.ShimConfigurationDataContext.AllInstances.GetAccountDataInstanceInt32NullableOfInt32 = (x, y, z) => new Task<AccountDataEntities.AccountDataDataContext>(() =>
{
return new SampleContext();// This is the fake context I created for replacing the AccountDataDataContext.
});
And SampleContext is implementing AccountDataDataContext as follows:
public class SampleContext: AccountDataDataContext
{
public SampleContext()
{
this.Samples = new TestDbSet<Sample>();
var data = new AccountDataRepository();
foreach (var item in data.GetFakeSamples())
{
this.Samples.Add(item);
}
}
}
Below is the code snippet for the test case:
[TestMethod]
public async Task SampleTest()
{
using (ShimsContext.Create())
{
//Arrange
SamplesController controller = ArrangeHelper(1);// This invokes the Shim code pasted in the second block and returns SamplesController object in this test class
var accountId = 1;
var serviceId = 2;
//Act
var response = await controller.GetSamples(accountId, serviceId);// The async method is invoked in the GetSamples(int32, int32) method.
var result = response.ToList();
//Assert
Assert.AreEqual(1, result.Count);
Assert.AreEqual("body 2", result[0].Body);
}
}
As the result, my test case is running forever. I think I might write the Shim lamdas expression completely wrong.
Any suggestion? Thank you.
You don't want to return a new Task. In fact, you should never, ever use the Task constructor. As I describe on my blog, it has no valid use cases at all.
Instead, use Task.FromResult:
ConfigurationEntities.Fakes.ShimConfigurationDataContext.AllInstances.GetAccountDataInstanceInt32NullableOfInt32 =
(x, y, z) => Task.FromResult(new SampleContext());
Task also has several other From* methods that are useful for unit testing (e.g., Task.FromException).
I am trying to check an async method throws concrete exception.
For that I am using MSTEST and FluentAssertions 2.0.1.
I have checked this Discussion on Codeplex and to see how it works with async-exception methods this another one link about FluentAssertions async tests:
After a while trying to work with my 'production' code I have switched off to the Fluentassertions fake aync class and my resulting code is like this (put this code inside a [TestClass]:
[TestMethod]
public void TestThrowFromAsyncMethod()
{
var asyncObject = new AsyncClass();
Action action = () =>
{
Func<Task> asyncFunction = async () =>
{
await asyncObject.ThrowAsync<ArgumentException>();
};
asyncFunction.ShouldNotThrow();
};
}
internal class AsyncClass
{
public async Task ThrowAsync<TException>()
where TException : Exception, new()
{
await Task.Factory.StartNew(() =>
{
throw new TException();
});
}
public async Task SucceedAsync()
{
await Task.FromResult(0);
}
}
The problem is that ShouldNotThrow is not valid:
ShouldNotThrow method is not recognised by the code. If I try to
compile, it gives me this error:
'System.Func' does not contain a
definition for 'ShouldNotThrow' and the best extension method overload
'FluentAssertions.AssertionExtensions.ShouldNotThrow(System.Action,
string, params object[])' has some invalid arguments
Thanks.
SOLUTION
2.0.1 FA version doesn't support this ShouldNotThrow functionality and it will be included in the next reléase 2.1 (near next week).
Note: ShouldThrow is already supported in 2.0.1 versión.
You don't need the encompassing Action. That is only used in the unit tests to verify that the API is throwing the right exception. This should be sufficient:
[TestMethod]
public void TestThrowFromAsyncMethod()
{
Func<Task> asyncFunction = async () =>
{
await asyncObject.ThrowAsync<ArgumentException>();
};
asyncFunction.ShouldNotThrow();
}
Unfortunately the ShoudlNotThrow() on a Func is missing from .NET 4.5. I've fixed this in release 2.1 (currently dogfooding).
If you look at AssertionExtensions.cs class you will see that the ShouldNotThrow extension method on Func is only defined for net45 or winrt compilation targets.
Check this:
Your unit tests project is on .net 4.5 or winrt
The referenced assertion library is the .net 4.5 one, if not try changing the
referenced FluentAssertions library to the right one.
Also after doing this, I think you need to call the action method to do the assertion, otherwise the inner lambda will not be called:
[TestMethod]
public void TestThrowFromAsyncMethod()
{
var asyncObject = new AsyncClass();
Action action = () =>
{
Func<Task> asyncFunction = async () =>
{
await asyncObject.ThrowAsync<ArgumentException>();
};
asyncFunction.ShouldNotThrow();
};
action.ShouldNotThrow();
}
I have a unit test (using MSTest) like so:
[TestMethod]
public void MyTest()
{
var viewModel = new MyViewModel();
viewModel.Run();
//Assert something here
}
Run is an async method that returns void.
Let's say Run is implemented like so:
public async void Run()
{
//Show a busy indicator here
try
{
var result = await myAsyncModelClass.LongRunningOperation();
//Use the results here
}
finally
{
//Hide the busy indicator here
}
}
myAsyncModelClass.LongRunningOperation(), is itself an async method that returns some Task<T> where T is the result my ViewModel is interested in.
My issue, is that my test is running the Run method asynchronously, so the my assertions are called before the Run methods completes. It is odd, b/c the finally block is never reached when I put a breakpoint, since the assertions fail. How can I keep the Run method synchronous to be able to unit test it?
I have a unit test of myAsyncModelClass.LongRunningOperation() also, but I merely call Task<T>.Wait() since it returns a task. This makes it synchronous when unit testing.
Also, I would like to mention, Run() is invoke by an ICommand magically by an MVVM framework. void may or may not be a require return type, I will have to try it out.
Async methods need a context to "return to". Since MSTests run on the thread pool, by default the async methods all continue on a thread pool thread as well (and do not block the MSTest method).
Under the (C# Testing) Unit Testing sample (in your Async CTP install directory), there's a type called GeneralThreadAffineContext, which can be used as such:
[TestMethod]
public void MyTest()
{
MyViewModel viewModel = null;
GeneralThreadAffineContext.Run(() =>
{
viewModel = new MyViewModel();
viewModel.Run();
});
//Assert something here
}
There are also specific WPF and WinForms contexts, but the thread-affine context should work for general ViewModels (that don't make explicit use of Dispatcher).
Update 2012-02-05: If you can change your ViewModel method to return Task, then you have another option: the new AsyncUnitTests library. Install that NuGet package, change your TestClass to AsyncTestClass, and your async unit tests can be written much more naturally:
[TestMethod]
public async void MyTest()
{
MyViewModel viewModel = new MyViewModel();
await viewModel.Run();
//Assert something here
}
Update 2012-09-04: Visual Studio 2012 includes async unit testing, so you don't need the AsyncUnitTests library anymore:
[TestMethod]
public async Task MyTest()
{
MyViewModel viewModel = new MyViewModel();
await viewModel.Run();
//Assert something here
}
Since Visual Studio 2012 MSTest supports async test methods. Just remeber they should return Task instead of void:
[TestMethod]
public async Task MyTest()
{
MyViewModel viewModel = new MyViewModel();
await viewModel.Run();
//Assert something here
}