Verifying ArgumentException and its message in Nunit , C# - c#

In my test program in Nunit, I want to verify that it's getting the write Argument Exception by verifying the message.
[Test]
public void ArgumentsWorkbookNameException()
{
const string workbookName = "Tester.xls";
var args = new[] { workbookName, "Sheet1", "Source3.csv", "Sheet2", "Source4.csv" };
Assert.Throws(typeof(ArgumentException), delegate { var appargs = new ApplicationArguments(args); }, "Invalid ending parameter of the workbook. Please use .xlsx");
}
After testing this out, this doesn't work when I modified the message in the main program.
int wbLength = args[0].Length;
// Telling the user to type in the correct workbook name file.
if (args[0].Substring(wbLength-5,5)!=".xlsx")
{
throw new ArgumentException(
"Invalid ending parameter of the workbook. Please use .xlsx random random");
}
The unit test still passed, regardless if I changed the message.
How do I do it? Or is there no such things in C#. My colleague said there are options like that in Ruby and RSPEC, but he's not 100% sure on C#.

Use the fluent interface to create assertions:
Assert.That(() => new ApplicationArguments(args),
Throws.TypeOf<ArgumentException>()
.With.Message.EqualTo("Invalid ending parameter of the workbook. Please use .xlsx random random"));

I agree with Jon that "such tests are unnecessarily brittle". However, there are at least two ways to check for exception message:
1: Assert.Throws returns an exception, so you can make an assertion for its message:
var exception = Assert.Throws<ArgumentException>(() => new ApplicationArguments(args));
Assert.AreEqual("Invalid ending parameter of the workbook. Please use .xlsx random random", exception.Message);
2: [HISTORICAL] Before NUnit 3, you could also use ExpectedException attribute. But, take a note that attribute waits for an exception in the whole tested code, not only in code which throws an exception you except. Thus, using this attribute is not recommended.
[Test]
[ExpectedException(typeof(ArgumentException), ExpectedMessage = "Invalid ending parameter of the workbook. Please use .xlsx random random")]
public void ArgumentsWorkbookNameException()
{
const string workbookName = "Tester.xls";
var args = new[] { workbookName, "Sheet1", "Source3.csv", "Sheet2", "Source4.csv" };
new ApplicationArguments(args);
}
You may also use FluentAssertions to do so, e.g.
subject.Invoking(y => y.Foo("Hello"))
.Should().Throw<InvalidOperationException>()
.WithMessage("Hello is not allowed at this moment");

The message parameter in Assert.Throws isn't the expected exception message; it's the error message to include with the assertion failure if the test fails.
I don't believe that NUnit supports testing the exception message out of the box, and I'd argue that such tests are unnecessarily brittle anyway. If you really want to write your own such helper method you can do so, but I personally wouldn't encourage it. (I very rarely specify a test failure message either, unless it's to include some diagnostic information. If a test fails I'm going to look at the test anyway, so the message doesn't add much.)
I would encourage you to use the generic overload instead though, and a lambda expression, for simplicity:
Assert.Throws<ArgumentException>(() => new ApplicationArguments(args));
(If that's your actual code by the way, there are other problems - try passing in new[] { "xyz" } as an argument...)

In .NET Core 3.1 MSTest project, this is how I did it.
[TestMethod]
public async Task SaveItemAsync_NameIsNull_ThrowsException()
{
var item = new Item
{
Name = null
};
var result = await Assert.ThrowsExceptionAsync<ArgumentException>(() => _service.SaveItemAsync(item));
Assert.AreEqual("The item's name must be set.", result.Message);
}

Related

How can I get a method's parameters in plain text in C#?

We use our "own" assert methods which allow to pass a message but do not force it.
Most of our code base use the "without message" one... Which is problematic when you have more than one assert and the code changes between the bug reporting and the bug fixing...
I would like to print, in the assert message, the caller line in plain text. Something like:
int toto = 1;
SomObject obj = null;
Assert(toto == 0);
Assert(obj);
Assert(toto == 0, "some useful info");
Expected output:
Assertion failed. Parameter='toto == 0'
Assertion failed. Parameter='obj'
Assertion failed. 'some useful info'
We found this thread talking about Cecil Mono.Cecil - simple example how to get method body ...
It could be a way to go, but would means we have to rebuild the line based on the ILCode.
Is there a way to get the "plain text code" in an other way?
[EDIT] We are running on C# 8.
You can try CallerArgumentExpressionAttribute (https://learn.microsoft.com/en-us/dotnet/api/system.runtime.compilerservices.callerargumentexpressionattribute?view=net-6.0) to get actual assert expression/condition and show it as a message.
public static void Assert(bool condition,
[CallerArgumentExpression("condition")] string message = null)
{
//code here
}
Usage
// message argument in Assert will be "result==true" (as string)
Assert(result==true);
// message argument in Assert will be "Custom message"
Assert(result==true, "Custom message");

Testing Exception in Nunit [duplicate]

I've got a set of test cases, some of which are expected to throw exceptions. Because of this, I have have set the attributes for these tests to expect exceptions like so:
[ExpectedException("System.NullReferenceException")]
When I run my tests locally all is good. However when I move my tests over to the CI server running TeamCity, all my tests that have expected exceptions fail. This is a known bug.
I am aware that there is also the Assert.Throws<> and Assert.Throws methods that NUnit offers.
My question is how can I make use of these instead of the attribute I'm currently using?
I've had a look around StackOverflow and tried a few things none of which seem to work for me.
Is there a simple 1 line solution to using this?
I'm not sure what you've tried that is giving you trouble, but you can simply pass in a lambda as the first argument to Assert.Throws. Here's one from one of my tests that passes:
Assert.Throws<ArgumentException>(() => pointStore.Store(new[] { firstPoint }));
Okay, that example may have been a little verbose. Suppose I had a test
[Test]
[ExpectedException("System.NullReferenceException")]
public void TestFoo()
{
MyObject o = null;
o.Foo();
}
which would pass normally because o.Foo() would raise a null reference exception.
You then would drop the ExpectedException attribute and wrap your call to o.Foo() in an Assert.Throws.
[Test]
public void TestFoo()
{
MyObject o = null;
Assert.Throws<NullReferenceException>(() => o.Foo());
}
Assert.Throws "attempts to invoke a code snippet, represented as a delegate, in order to verify that it throws a particular exception." The () => DoSomething() syntax represents a lambda, essentially an anonymous method. So in this case, we are telling Assert.Throws to execute the snippet o.Foo().
So no, you don't just add a single line like you do an attribute; you need to explicitly wrap the section of your test that will throw the exception, in a call to Assert.Throws. You don't necessarily have to use a lambda, but that's often the most convenient.
Here's a simple example using both ways.
string test = null;
Assert.Throws( typeof( NullReferenceException ), () => test.Substring( 0, 4 ) );
Assert.Throws<NullReferenceException>( () => test.Substring( 0, 4 ) );
If you don't want to use lambdas.
[Test]
public void Test()
{
Assert.Throws<NullReferenceException>( _TestBody );
}
private void _TestBody()
{
string test = null;
test.Substring( 0, 4 );
}
By default, TeamCity uses NUnit 2.2.10, which doesn't have ExpectedException. Check the TeamCity "NUnit for NAnt" docs to see how to change it to something more modern, including the specific list of releases TeamCity provides.
NUnit has added a new Record.Exception method.
If you prefer to separate Acts and Asserts then
Act:
ex = Record.Exception(()={throw new Exception()}
Assert:
Assert.NotNull(ex);

NSubstitute: Received() does not check string parameter when string comes from resource

I use NSubstitute for my NUnit tests and try to check if a method is called with the right values. Until know everything works fine. I have to localize the texts and so use resource strings for this. Every unit test now fails, that tests for a method to be recieved, where a string parameter contains a new line.
Here is a simplified example:
// Old Version, where the unit test succeed
public void CallWithText(ICallable callable)
{
callable.ShowText("Some text with a new\nline.");
}
// New Version, where the unit test fails
// Text of `Properties.Resources.TextWithNewLine` in the
// Resources.resx is "Some text with a new
// line."
public void CallWithText(ICallable callable)
{
callable.ShowText(Properties.Resources.TextWithNewLine);
}
[Test]
public void CallWithText_WhenCalled_CallsCallable()
{
var caller = new Caller();
var callable = Substitute.For<ICallable>();
caller.CallWithText(callable);
callable.Received(1).ShowText("Some text with a new\nline.");
}
It seems for me, that there is a problem with the new line. Does anybody has a solution, because it is a mess to adapt all the unit tests.
The problem isn't related to NSubstitute but to String itself. The new line symbol isn't just \n. It's environment specific: \r\n on Windows, \n on Unix, \r on early Mac OSes. Please use Shift+Enter to add new lines properly in Resource manager.
You have a couple of ways to use:
Environment.NewLine property which knows the new line symbol for the current environment:
callable.Received(1).ShowText("Some text with a new" + Environment.NewLine + "line.");
Use explicit new line in your expected string:
callable.Received(1).ShowText(#"Some text with a new
line.");
Comparing exact strings in the unit test will add more maintaining work.
In your case new line can be different in different execution environment.
For strings I suggest asserting passed parameter with Contains method. Where you can check for the more important words
[Test]
public void CallWithText_WhenCalled_CallsCallable()
{
var caller = new Caller();
var callable = Substitute.For<ICallable>();
caller.CallWithText(callable);
callable.Received(1).ShowText(Arg.Is<string>(text => text.Contains("Some text")));
}
This indicates the Properties.Resources.TextWithNewLine and "Some text with a new\nline." and different. Without knowing too much about Properties.Resources.TextWithNewLine, I suggest you try change the test to
[Test]
public void CallWithText_WhenCalled_CallsCallable()
{
var caller = new Caller();
var callable = Substitute.For<ICallable>();
caller.CallWithText(callable);
callable.Received(1).ShowText(Arg.Any<string>());
}
Or use the actual string if you really want to assert the content of the string, change to (make sure the resources file is in the test project)
[Test]
public void CallWithText_WhenCalled_CallsCallable()
{
var caller = new Caller();
var callable = Substitute.For<ICallable>();
caller.CallWithText(callable);
callable.Received(1).ShowText(Properties.Resources.TextWithNewLine);
}

How do I check "no exception occurred" in my MSTest unit test?

I'm writing a unit test for this one method which returns "void". I would like to have one case that the test passes when there is no exception thrown. How do I write that in C#?
Assert.IsTrue(????)
(My guess is this is how I should check, but what goes into "???")
I hope my question is clear enough.
Your unit test will fail anyway if an exception is thrown - you don't need to put in a special assert.
This is one of the few scenarios where you will see unit tests with no assertions at all - the test will implicitly fail if an exception is raised.
However, if you really did want to write an assertion for this - perhaps to be able to catch the exception and report "expected no exception but got this...", you can do this:
[Test]
public void TestNoExceptionIsThrownByMethodUnderTest()
{
var myObject = new MyObject();
try
{
myObject.MethodUnderTest();
}
catch (Exception ex)
{
Assert.Fail("Expected no exception, but got: " + ex.Message);
}
}
(the above is an example for NUnit, but the same holds true for MSTest)
In NUnit, you can use:
Assert.DoesNotThrow(<expression>);
to assert that your code does not throw an exception. Although the test would fail if an exception is thrown even if there was no Assert around it, the value of this approach is that you can then distinguish between unmet expectations and bugs in your tests, and you have the option of adding a custom message that will be displayed in your test output. A well-worded test output can help you locate errors in your code that have caused a test to fail.
I think it's valid to add tests to ensure that your code is not throwing exceptions; for example, imagine you are validating input and need to convert an incoming string to a long. There may be occasions when the string is null, and this is acceptable, so you want to ensure that the string conversion does not throw an exception. There will therefore be code to handle this occasion, and if you haven't written a test for it you will be missing coverage around an important piece of logic.
This helper class scratched my itch with MSTest. Maybe it can scratch yours also.
[TestMethod]
public void ScheduleItsIneligibilityJob_HasValid_CronSchedule()
{
// Arrange
var factory = new StdSchedulerFactory();
IScheduler scheduler = factory.GetScheduler();
// Assert
AssertEx.NoExceptionThrown<FormatException>(() =>
// Act
_service.ScheduleJob(scheduler)
);
}
public sealed class AssertEx
{
public static void NoExceptionThrown<T>(Action a) where T:Exception
{
try
{
a();
}
catch (T)
{
Assert.Fail("Expected no {0} to be thrown", typeof(T).Name);
}
}
}
Don't test that something doesn't happen. It's like assuring that code doesn't break. That's sort of implied, we all strive for non-breaking, bug-less code. You want to write tests for that? Why just one method? Don't you want all your methods being tested that they don't throw some exception? Following that road, you'll end up with one extra, dummy, assert-less test for every method in your code base. It brings no value.
Of course, if your requirement is to verify method does catch exceptions, you do test that (or reversing it a bit; test that it does not throw what it is supposed to catch).
However, the general approach/practices remain intact - you don't write tests for some artificial/vague requirements that are out of scope of tested code (and testing that "it works" or "doesn't throw" is usually an example of such - especially in scenario when method's responsibilities are well known).
To put it simple - focus on what your code has to do and test for that.
I like to see an Assert.Whatever at the end of each test, just for consistency... without one, can I really be sure there's not supposed to be one there?
For me, this is as simple as putting Assert.IsTrue(true);
I know I didn't accidentally put that code in there, and thus I should be confident enough at quick a skim through that this was as intended.
[TestMethod]
public void ProjectRejectsGappedVersioningByDefault() {
var files = new List<ScriptFile>();
files.Add(ScriptProjectTestMocks.GetVersion1to2());
files.Add(ScriptProjectTestMocks.GetVersion3to4());
Assert.Throws<ScriptProject.InvalidProjectFormatException>(() => {
var sut = new ScriptProject(files);
});
}
[TestMethod]
public void ProjectAcceptsGappedVersionsExplicitly() {
var files = new List<ScriptFile>();
files.Add(ScriptProjectTestMocks.GetVersion1to2());
files.Add(ScriptProjectTestMocks.GetVersion3to4());
var sut = new ScriptProject(files, true);
Assert.IsTrue(true); // Assert.Pass() would be nicer... build it in if you like
}
My friend Tim told me about ExpectedException. I really like this b/c it is more succinct, less code, and very explicit that you are testing for an exception.
[TestMethod()]
[ExpectedException(typeof(System.Exception))]
public void DivideTest()
{
int numerator = 4;
int denominator = 0;
int actual = numerator / denominator;
}
You can read way more about it here: ExpectedException Attribute Usage.
With Xunit you can use this:
var exception = Record.Exception(() =>
MethodUnderTest());
Assert.Null(exception);
or for async operations
var exception = await Record.ExceptionAsync(async () =>
await MethodUnderTestAsync());
Assert.Null(exception);
Another way which worked for me is to store it in a variable and check output.
var result = service.Run()
Assert.IsFalse(result.Errors.Any())
using Moq;
using Xunit;
[Fact]
public void UnitTest_DoesNotThrow_Exception()
{
var builder = new Mock<ISomething>().Object;
//Act
var exception = Record.Exception(() => builder.SomeMethod());
//Assert
Assert.Null(exception);
}

NUnit expected exceptions

I've got a set of test cases, some of which are expected to throw exceptions. Because of this, I have have set the attributes for these tests to expect exceptions like so:
[ExpectedException("System.NullReferenceException")]
When I run my tests locally all is good. However when I move my tests over to the CI server running TeamCity, all my tests that have expected exceptions fail. This is a known bug.
I am aware that there is also the Assert.Throws<> and Assert.Throws methods that NUnit offers.
My question is how can I make use of these instead of the attribute I'm currently using?
I've had a look around StackOverflow and tried a few things none of which seem to work for me.
Is there a simple 1 line solution to using this?
I'm not sure what you've tried that is giving you trouble, but you can simply pass in a lambda as the first argument to Assert.Throws. Here's one from one of my tests that passes:
Assert.Throws<ArgumentException>(() => pointStore.Store(new[] { firstPoint }));
Okay, that example may have been a little verbose. Suppose I had a test
[Test]
[ExpectedException("System.NullReferenceException")]
public void TestFoo()
{
MyObject o = null;
o.Foo();
}
which would pass normally because o.Foo() would raise a null reference exception.
You then would drop the ExpectedException attribute and wrap your call to o.Foo() in an Assert.Throws.
[Test]
public void TestFoo()
{
MyObject o = null;
Assert.Throws<NullReferenceException>(() => o.Foo());
}
Assert.Throws "attempts to invoke a code snippet, represented as a delegate, in order to verify that it throws a particular exception." The () => DoSomething() syntax represents a lambda, essentially an anonymous method. So in this case, we are telling Assert.Throws to execute the snippet o.Foo().
So no, you don't just add a single line like you do an attribute; you need to explicitly wrap the section of your test that will throw the exception, in a call to Assert.Throws. You don't necessarily have to use a lambda, but that's often the most convenient.
Here's a simple example using both ways.
string test = null;
Assert.Throws( typeof( NullReferenceException ), () => test.Substring( 0, 4 ) );
Assert.Throws<NullReferenceException>( () => test.Substring( 0, 4 ) );
If you don't want to use lambdas.
[Test]
public void Test()
{
Assert.Throws<NullReferenceException>( _TestBody );
}
private void _TestBody()
{
string test = null;
test.Substring( 0, 4 );
}
By default, TeamCity uses NUnit 2.2.10, which doesn't have ExpectedException. Check the TeamCity "NUnit for NAnt" docs to see how to change it to something more modern, including the specific list of releases TeamCity provides.
NUnit has added a new Record.Exception method.
If you prefer to separate Acts and Asserts then
Act:
ex = Record.Exception(()={throw new Exception()}
Assert:
Assert.NotNull(ex);

Categories