[Edit (May 2020)] - This issue has been reportedly addressed in newer releases of NUnit. Please see Nunit.ThrowsAsync. (Ref this answer, thanks #James-Ross)
I have a controller UserController with this action
// GET /blah
public Task<User> Get(string domainUserName)
{
if (string.IsNullOrEmpty(domainUserName))
{
throw new ArgumentException("No username specified.");
}
return Task.Factory.StartNew(
() =>
{
var user = userRepository.GetByUserName(domainUserName);
if (user != null)
{
return user;
}
throw new HttpResponseException(Request.CreateErrorResponse(HttpStatusCode.NotFound, string.Format("{0} - username does not exist", domainUserName)));
});
}
I am trying to write a test for the case where I throw a 404 exception.
Here is what I have tried, with the output -
1)
[Test]
public void someTest()
{
var mockUserRepository = new Mock<IUserRepository>();
mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));
var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };
Assert.That(async () => await userController.Get("foo"), Throws.InstanceOf<HttpResponseException>());
}
Result
Test Failed
Expected: instance of <System.Web.Http.HttpResponseException>
But was: no exception thrown
[Test]
public void someTest()
{
var mockUserRepository = new Mock();
mockUserRepository.Setup(x => x.GetByUserName(It.IsAny())).Returns(default(User));
var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };
var httpResponseException = Assert.Throws<HttpResponseException>(() => userController.Get("foo").Wait());
Assert.That(httpResponseException.Response.StatusCode, Is.EqualTo(HttpStatusCode.NotFound));
}
Result
Test failed
Expected: <System.Web.Http.HttpResponseException>
But was: <System.AggregateException> (One or more errors occurred.)
[Test]
public void someTest()
{
var mockUserRepository = new Mock<IUserRepository>();
mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));
var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };
var httpResponseException = Assert.Throws<HttpResponseException>(async () => await userController.Get("foo"));
Assert.That(httpResponseException.Response.StatusCode, Is.EqualTo(HttpStatusCode.NotFound));
}
Result
Test Failed
Expected: <System.Web.Http.HttpResponseException>
But was: null
[Test]
[ExpectedException(typeof(HttpResponseException))]
public async void ShouldThrow404WhenNotFound()
{ var mockUserRepository = new Mock<IUserRepository>();
mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));
var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };
var task = await userController.Get("foo");
}
Result
Test passes
Questions -
Why would Assert.Throws not handle HttpResponseException, when ExpectedException does?
I don't want to just test that exception is thrown. I want to assert on the Status Code of the response. What's the way to do this?
Any comparision on these behaviour and its cause(s) would be great!
You're seeing problems due to async void.
In particular:
async () => await userController.Get("foo") is converted into TestDelegate, which returns void, so your lambda expression is treated as async void. So the test runner will begin executing the lambda but not wait for it to complete. The lambda returns before Get completes (because it's async), and the test runner sees that it returned without an exception.
Wait wraps any exceptions in an AggregateException.
Again, the async lambda is being treated as async void, so the test runner is not waiting for its completion.
I recommend you make this async Task rather than async void, but in this case the test runner does wait for completion, and thus sees the exception.
According to this bug report, there is a fix for this coming in the next build of NUnit. In the meantime, you can build your own ThrowsAsync method; an example for xUnit is here.
I'm not sure when it was added, but the current version of Nunit (3.4.1 at time of writing) includes a ThrowsAsync method
see https://github.com/nunit/docs/wiki/Assert.ThrowsAsync
Example:
[Test]
public void ShouldThrow404WhenNotFound()
{
var mockUserRepository = new Mock<IUserRepository>();
mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));
var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };
var exception = Assert.ThrowsAsync<HttpResponseException>(() => userController.Get("foo"));
Assert.That(exception.Response.StatusCode, Is.EqualTo(HttpStatusCode.NotFound));
}
This blog talks about problems similar to mine.
I followed the recommendation proposed there, and have a test like this -
[Test]
public void ShouldThrow404WhenNotFound()
{
var mockUserRepository = new Mock<IUserRepository>();
mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));
var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };
var aggregateException = Assert.Throws<AggregateException>(() => userController.Get("foo").Wait());
var httpResponseException = aggregateException.InnerExceptions
.FirstOrDefault(x => x.GetType() == typeof(HttpResponseException)) as HttpResponseException;
Assert.That(httpResponseException, Is.Not.Null);
Assert.That(httpResponseException.Response.StatusCode, Is.EqualTo(HttpStatusCode.NotFound));
}
I am not too pleased with it, but this works.
EDIT 1
Inspired by #StephenCleary, I added a static helper class that does the asserts that I am looking for. It looks like this -
public static class AssertEx
{
public static async Task ThrowsAsync<TException>(Func<Task> func) where TException : class
{
await ThrowsAsync<TException>(func, exception => { });
}
public static async Task ThrowsAsync<TException>(Func<Task> func, Action<TException> action) where TException : class
{
var exception = default(TException);
var expected = typeof(TException);
Type actual = null;
try
{
await func();
}
catch (Exception e)
{
exception = e as TException;
actual = e.GetType();
}
Assert.AreEqual(expected, actual);
action(exception);
}
}
I can now have a test like -
[Test]
public async void ShouldThrow404WhenNotFound()
{
var mockUserRepository = new Mock<IUserRepository>();
mockUserRepository.Setup(x => x.GetByUserName(It.IsAny<string>())).Returns(default(User));
var userController = new UserController(mockUserRepository.Object) { Request = new HttpRequestMessage() };
Action<HttpResponseException> asserts = exception => Assert.That(exception.Response.StatusCode, Is.EqualTo(HttpStatusCode.NotFound));
await AssertEx.ThrowsAsync(() => userController.Get("foo"), asserts);
}
If you await a Task then Exceptions that are thrown are aggregated into AggregateException. You can inspect the inner exceptions of AggregateException. This could be the reason why you case 2 does not work.
Unhandled exceptions that are thrown by user code that is running inside a task are propagated back to the joining thread, except in certain scenarios that are described later in this topic. Exceptions are propagated when you use one of the static or instance Task.Wait or Task.Wait methods, and you handle them by enclosing the call in a try-catch statement. If a task is the parent of attached child tasks, or if you are waiting on multiple tasks, then multiple exceptions could be thrown. To propagate all the exceptions back to the calling thread, the Task infrastructure wraps them in an AggregateException instance. The AggregateException has an InnerExceptions property that can be enumerated to examine all the original exceptions that were thrown, and handle (or not handle) each one individually. Even if only one exception is thrown, it is still wrapped in an AggregateException.
Link to MSDN
I have a similar issue that you have in scenario 3 Test case failed due to following result
Expected: <UserDefineException>
But was: null
by using Assert.ThrowAsync<> problem gets solved
My Web API action method and Unit test case method as below
public async Task<IHttpActionResult> ActionMethod(RequestModel requestModel)
{
throw UserDefineException();
}
[Test]
public void Test_Contrller_Method()
{
Assert.ThrowsAsync<UserDefineException>(() => _controller.ActionMethod(new RequestModel()));
}
This is an example from documentation:
var ex = Assert.ThrowsAsync<ArgumentException>(async () => await MethodThatThrows());
Use ThrowsAsync
Use async / await
https://docs.nunit.org/articles/nunit/writing-tests/assertions/classic-assertions/Assert.ThrowsAsync.html
Related
I am working on a autocomplete component. When typing, a search function is invoked to return results. This search function e.g. calls a api endpoint. Since loading the results may take some time, I want to be able to cancel previous search requests when there are new requests.
Code:
public Func<string, CancellationToken, Task<IEnumerable<T>>> SearchFuncWithCancel { get; set; }
private CancellationTokenSource _cancellationTokenSrc;
private void CancelToken()
{
_cancellationTokenSrc?.Cancel();
_cancellationTokenSrc = new CancellationTokenSource();
}
private async Task OnSearchAsync()
{
IEnumerable<T> searched_items = Array.Empty<T>();
CancelToken();
try
{
searched_items = (await SearchFuncWithCancel(Text, _cancellationTokenSrc.Token)) ?? Array.Empty<T>();
{
catch (Exception e)
{
Console.WriteLine("The search function failed to return results: " + e.Message);
}
//...
}
For every search any previous (and still active) search is cancelled. I would like to unit test this behaviour.
What I've tried so far:
[Test]
public async Task Autocomplete_Should_Cancel_Search_On_Search()
{
// Arrange
var mockHttp = new MockHttpMessageHandler();
mockHttp.When("http://localhost/autocomplete")
.Respond("application/json", "[\"Foo\", \"Bar\"]");
var client = mockHttp.ToHttpClient();
var comp = Context.RenderComponent<AutocompleteTest1>();
var autocompletecomp = comp.FindComponent<MudAutocomplete<string>>();
autocompletecomp.SetParam(p => p.SearchFuncWithCancel, new Func<string, System.Threading.CancellationToken, Task<IEnumerable<string>>>(async (s, cancellationToken) => {
var result = await client.GetAsync("http://localhost/autocomplete", cancellationToken);
var content = await result.Content.ReadAsStringAsync(cancellationToken);
return JsonConvert.DeserializeObject<IEnumerable<string>>(content);
}));
// Test
autocompletecomp.Find("input").Input("Foo");
comp.WaitForAssertion(() => comp.Find("div.mud-popover").ToMarkup().Should().Contain("Foo"));
autocompletecomp.Find("input").Input("Bar");
comp.WaitForAssertion(() => comp.Find("div.mud-popover").ToMarkup().Should().Contain("Bar"));
}
This test passes, however it does not test the cancellation behaviour yet. I am not sure how to do it. Any ideas?
As long as you can mock the API-call to return something else you should be able to use taskCompletionSources to simulate whatever behavior you desire. I'm not familiar with your specific mock framework, but perhaps something like this should work
// Setup the API method for the first call
CancellationToken cancelToken= null;
var tcs1= new TaskCompletionSource<IEnumerable<T>>();
subjectUnderTest.SearchFuncWithCancel = (t, c) => {
cancelToken= c;
return tcs.Task;
};
var call1 = subjectUnderTest.DoSearch("call1");
Assert.IsFalse(cancelToken.IsCancellationRequested);
// Setup the API-method for the second call
var tcs2= new TaskCompletionSource<IEnumerable<T>>();
subjectUnderTest.SearchFuncWithCancel = (t, c) => tcs2.Task;
var call2 = subjectUnderTest.DoSearch("call2");
Assert.IsTrue(cancelToken.IsCancellationRequested);
// Check that the first call handles a cancelled task correctly
tcs1.SetCanceled();
Assert.IsTrue(call1.IsCanceled);
// Second call succeed
tcs2.SetResult(...);
Assert.IsTrue(call2.IsCompletedSuccessfully);
The first call to the API will return a task that never completes. So when the second call occurs we can check that the cancellation token for the first call is cancelled. I hope the pseudocode is possible to translate to your particular mock framework.
I need to unit test a method that looks roughly like this:
public async Task DoStuffAsync()
{
var tasks = { dependency.FooAsync(), dependency.BarAsync() };
await Task.WhenAll(tasks);
}
My first approach (using Moq) was like this:
dependency.Setup(d => d.FooAsync()).Returns(Task.FromResult(default(object)));
dependency.Setup(d => d.BarAsync()).Returns(Task.FromResult(default(object)));
await systemUnderTest.DoStuffAsync();
dependency.Verify(d => d.FooAsync(), Times.Once);
dependency.Verify(d => d.BarAsync(), Times.Once);
But this won't do, since if I change the method to something dumb like this:
public async Task DoStuffAsync()
{
var tasks = { dependency.FooAsync(), dependency.BarAsync() };
await tasks[0];
}
the test won't fail. How do I assert that the tasks were awaited?
EDIT:
I think this solution to this problem could be analogous to mine:
let's say I want to test this method for failure, i.e. one of the tasks throws and I want to assert that the exception is propagated. I need to check that for all the tasks, but there could be a dozen tasks awaited by Task.WhenAll. I'd then have to write a dozen tests, each of them would have a different, single task throw an exception, to make sure that all of them are correctly propagated. And writing a dozen same-y tests doesn't sound like a good thing. If I could just assert that all the tasks are awaited - that would solve my problem, as I can trust the async framework to propagate if an error occurs.
You give very little context (programming is all about context :)).
In you particular case if you want check that DoStuffAsync await all tasks returned by dependencies throw exceptions and await for them
For .NET 4.5.1 create helper method to create task with an exception.
static Task CreateTaskWithException(string exceptionMessage)
{
var taskSource = new TaskCompletionSource<object>();
var exception = new Exception(exceptionMessage);
taskSource.SetException(exception);
return taskSource.Task;
}
If you have multiple dependencies you can test for await - you can simple test all of them at once
dependency.Setup(d => d.FooAsync()).Returns(CreateTaskWithException("Foo"));
dependency.Setup(d => d.BarAsync()).Returns(CreateTaskWithException("Bar"));
dependency.Setup(d => d.FizzBuzzAsync()).Returns(CreateTaskWithException("FizzBuzz"));
var expectedErrors = new[] { "Foo", "Bar", "FizzBuzz" };
string[] actualErrors = null;
try
{
DoStuffAsync().Wait();
Assert.Fail("DoStuffAsync should throw exception");
}
catch(AggregateException exception)
{
actualErrors = exception.InnerExceptions.Select(ex => ex.Message);
}
actualErrors.Should().BeEquivalentTo(expected);
Test will fail if you not awaiting all tasks.
public async Task DoStuffAsync()
{
var tasks = { dependency.FooAsync(), dependency.BarAsync() };
reutrn Task.WhenAll(tasks.Skip(1));
// Test fail with message:
// Expected subject to be a collection with 2 item(s), but {"Bar"}
// contains 1 item(s) less than {"Foo", "Bar"}.
}
I have some problem with my unit tests. I want to check that my remove method in service is invoke remove method in repository.
My test method:
[Fact]
public async Task Remove_room_async_should_invoke_remove_room_async_on_room_repository()
{
//Arrange
var room = new Room(Guid.NewGuid(), "A-11");
var roomRepositoryMock = new Mock<IRoomRepository>();
var mapperMock = new Mock<IMapper>();
var roomService = new RoomService(roomRepositoryMock.Object, mapperMock.Object);
//Act
await roomService.RemoveAsync(room.RoomId);
//Assert
roomRepositoryMock.Verify(x => x.RemoveAsync(It.IsAny<Room>()), Times.Once());
}
Currently it return me FAIL with exception becouse room with this id doesn't exist.
My repository remove method implementation:
public async Task RemoveAsync(Room room)
{
_rooms.Remove(room);
await Task.CompletedTask;
}
And service
public async Task RemoveAsync(Guid roomId)
{
var room = await _roomRepository.GetOrFailAsync(roomId);
await _roomRepository.RemoveAsync(room);
}
You need to set up the GetOrFailAsync method on your mock IRoomRepository, otherwise the mock is not going to return anything.
So your arrange section becomes:
var room = new Room(Guid.NewGuid(), "A-11");
var roomRepositoryMock = new Mock<IRoomRepository>();
roomRepositoryMock.Setup(r => r.GetOrFailAsync).Returns(room);
var mapperMock = new Mock<IMapper>();
var roomService = new RoomService(roomRepositoryMock.Object, mapperMock.Object);
You also probably want to verify this call as well:
roomRepositoryMock.Verify(r => r.GetOrFailAsync(room.Id), Times.Once());
And consider changing your other verification to match exactly the room you want to delete, rather than IsAny:
roomRepositoryMock.Verify(x => x.RemoveAsync(room), Times.Once());
I am currently using Moq to help with my unit testing however I ran in to an issue that I do not know how to resolve.
For example, say I would like to validate that CancellationToken.ThrowIfCancellationRequested() was called once per Upload( call
public UploadEngine(IUploader uploader)
{
_uploader = uploader;
}
public void PerformUpload(CancellationToken token)
{
token.ThrowIfCancellationRequested();
_uploader.Upload(token, "Foo");
token.ThrowIfCancellationRequested();
_uploader.Upload(token, "Bar");
}
If token was a reference type I normally would do something like
[TestMethod()]
public void PerformUploadTest()
{
var uploader = new Mock<IUploader>();
var token = new Mock<CancellationToken>();
int callCount = 0;
uploader.Setup(a => a.Upload(token.Object, It.IsAny<string>())).Callback(() => callCount++);
token.Setup(a => a.ThrowIfCancellationRequested());
var engine = new UploadEngine(uploader.Object);
engine.PerformUpload(token.Object);
token.Verify(a => a.ThrowIfCancellationRequested(), Times.Exactly(callCount));
}
However from what I can tell Moq does not support value types. What would be the correct way to test this, or is there no way to do what I want through Moq without boxing the CancellationToken inside a container first to be passed in to PerformUpload(?
You've probably moved on from this, but it occurred to me that what you're trying to test doesn't really seem to make sense. Testing that ThrowIfCancellationRequested is called the same number of times as Upload does nothing to ensure that they have been called in the right sequence, which I'm assuming would actually be relevant in this case. You wouldn't want code like this to pass, but I'm pretty sure it would:
_uploader.Upload(token, "Foo");
token.ThrowIfCancellationRequested();
token.ThrowIfCancellationRequested();
_uploader.Upload(token, "Bar");
As has been said in the comments, the easiest way to get round this would be to push the token.ThrowIfCancellationRequested call into the Upload call. Assuming that for whatever reason this isn't possible, I might take the following approach to testing your scenario.
Firstly, I'd encapsulate the functionality of checking to see if the cancellation had been requested and if not, calling an action into something testable. At first thought, this might look like this:
public interface IActionRunner {
void ExecIfNotCancelled(CancellationToken token, Action action);
}
public class ActionRunner : IActionRunner{
public void ExecIfNotCancelled(CancellationToken token, Action action) {
token.ThrowIfCancellationRequested();
action();
}
}
This can be fairly trivially tested with two tests. One to check that action is called if the token isn't cancelled and one to validate that it isn't if it is cancelled. These tests would look like:
[TestMethod]
public void TestActionRunnerExecutesAction() {
bool run = false;
var runner = new ActionRunner();
var token = new CancellationToken();
runner.ExecIfNotCancelled(token, () => run = true);
// Validate action has been executed
Assert.AreEqual(true, run);
}
[TestMethod]
public void TestActionRunnerDoesNotExecuteIfCancelled() {
bool run = false;
var runner = new ActionRunner();
var token = new CancellationToken(true);
try {
runner.ExecIfNotCancelled(token, () => run = true);
Assert.Fail("Exception not thrown");
}
catch (OperationCanceledException) {
// Swallow only the expected exception
}
// Validate action hasn't been executed
Assert.AreEqual(false, run);
}
I would then inject the IActionRunner into the UploadEngine and validate that it was being called correctly. So, your PerformUpload method would change to:
public void PerformUpload(CancellationToken token) {
_actionRunner.ExecIfNotCancelled(token, () => _uploader.Upload(token, "Foo"));
_actionRunner.ExecIfNotCancelled(token, () => _uploader.Upload(token, "Bar"));
}
You can then write a pair of tests to validate PerformUpload. The first checks that if an ActionRunner mock has been setup to execute the supplied action then Upload is called at least once. The second test validates that if the ActionRunner mock has been setup to ignore the action, then Upload isn't called. This essentially ensures that all Upload calls in the method are done via the ActionRunner. These tests look like this:
[TestMethod]
public void TestUploadCallsMadeThroughActionRunner() {
var uploader = new Mock<IUploader>();
var runner = new Mock<IActionRunner>();
var token = new CancellationToken();
int callCount = 0;
uploader.Setup(a => a.Upload(token, It.IsAny<string>())).Callback(() => callCount++);
// Use callback to invoke actions supplied to runner
runner.Setup(x => x.ExecIfNotCancelled(token, It.IsAny<Action>()))
.Callback<CancellationToken, Action>((tok,act)=>act());
var engine = new UploadEngine(uploader.Object, runner.Object);
engine.PerformUpload(token);
Assert.IsTrue(callCount > 0);
}
[TestMethod]
public void TestNoUploadCallsMadeThroughWithoutActionRunner() {
var uploader = new Mock<IUploader>();
var runner = new Mock<IActionRunner>();
var token = new CancellationToken();
int callCount = 0;
uploader.Setup(a => a.Upload(token, It.IsAny<string>())).Callback(() => callCount++);
// NOP callback on runner prevents uploader action being run
runner.Setup(x => x.ExecIfNotCancelled(token, It.IsAny<Action>()))
.Callback<CancellationToken, Action>((tok, act) => { });
var engine = new UploadEngine(uploader.Object, runner.Object);
engine.PerformUpload(token);
Assert.AreEqual(0, callCount);
}
There would obviously be other tests that you might want to write for your UploadEngine but they seem out of scope for the current question...
I am using the DropNet API for connecting to DropBox. I am struggling around async/await concepts though.
I have a method is calling the api GetTokenAsync. The return type is void, and there is a success and failure action to callback.
public async Task<GetTokenResult> GetAuthorizationUrl()
{
var result = new GetTokenResult();
_dropNetClient.GetTokenAsync(
login =>
{
result.Url = _dropNetClient.BuildAuthorizeUrl(_authorizationCallback.ToString());
result.Success = true;
},
exception =>
{
result.Error = exception.ToDiagnosticString();
result.Success = false;
}
);
return result;
}
The problem? I am thinking that changing the return type to just GetTokenResult may return faster than the actions will, therefore my results will never get set. I cannot await the async method as it returns void.
This is the one concept around async/await that I cannot wrap my head around.
You might want to consider using a TaskCompletionSource:
public Task<GetTokenResult> GetAuthorizationUrl()
{
var tcs = new TaskCompletionSource<GetTokenResult>();
_dropNetClient.GetTokenAsync(
login => tcs.SetResult(new GetTokenResult {
Url = _dropNetClient.BuildAuthorizeUrl(
_authorizationCallback.ToString()),
Success = true
},
exception => tcs.SetResult(new GetTokenResult {
Error = exception.ToDiagnosticString(),
Success = true
});
return tcs.Task;
}
The returned task will only complete when the GetTokenAsync operation completes (via one of the callbacks), and you can await it from an async method.
I would personally use SetException instead of SetResult on failure though - so that if you await the returned task, it will throw the appropriate failure on an exception, rather than just setting the value differently. It's more idiomatically .NET-like.
EDIT: You could then change the code to return Task<string> instead:
public Task<string> GetAuthorizationUrl()
{
var tcs = new TaskCompletionSource<string>();
_dropNetClient.GetTokenAsync(
login => tcs.SetResult(_dropNetClient.BuildAuthorizeUrl
_authorizationCallback.ToString()),
exception => tcs.SetException(exception));
return tcs.Task;
}
Now the exception would be propagated within the task itself - if you await a task which faults, the exception is thrown at that point. No need for extra properties, etc - it's much closer to how you'd write the corresponding synchronous code.