We Keep Coding

Implement a Custom Exception that Fails a Unit Test as Inconclusive

Using the test tools built into Visual Studio (in my case VS 2013, though I believe this extends to other versions as well), a 'TestMethod' (any method marked with a [TestMethod] attribute) can result in 1 of 3 statuses after being run:
Passed
Skipped/Inconclusive
Failed
A test is passed if it runs to completion with no uncaught exceptions. A test fails it throws an uncaught exception. A test is inconclusive if it throws a special type of uncaught exception: AssertInconclusiveException.
My question is, is it possible (in, say, a [TestCleanup] method) to treat an custom uncaught exception similarly to the AssertInconclusiveException and set the test status to 'Inconclusive' instead of 'Failed'? Or is the recognition of AssertInconclusiveException as a special exception built into Visual Studio itself? What I'm looking for, in rough pseudocode form, is roughly:
[TestCleanup]
public void TestCleanup()
{
if(this.TestContext.CurrentTestOutcome == UnitTestOutcome.Failed
&& this.TestContext.UncaughtException.GetType() == typeof(MyCustomException))
{
this.TestContext.CurrentTestOutcome = UnitTestOutcome.Inconclusive;
}
}
This would be used as part of an integration test suite to mark a test as inconclusive whenever the setup method for the test throws a special SetupFailureException (usually indicative of broken downstream system that is not the one under test).
I've been able to produce the desired behavior in the interim by defining:
public class SetupFailedException : AssertInconclusiveException
{
}
// example test that demonstrates desired behavior
[TestMethod]
public void TestThatFailsAsInconclusiveByThrowingUncaughtSpecialException()
{
throw new SetupFailedException();
}
But this seems messy since the thrown exception has nothing to do with an assertion and prevents custom exceptions which should fail tests as inconclusive from inheriting from any other abstract class.

The Inconclusive recognition is built into the test framework, by the time you get to cleanup, it's too late to change a fail into an inconclusive.
It feels like you should really be catching the exceptions from within your integration tests anyway, since this shows that you're recognising the method your testing can throw a certain type of exception and that the right thing to do is to ignore the result in that situation.
If you need to have multiple exceptions that can be detected like this, I'd be tempted to have them implement an interface, so that the catch block can just check if any exceptions caught implement that interface.
If you wanted to minimize the impact/visibility on the rest of your test code then you could wrap the call that could fail in an action, in which case your code might look something like this:
The interface:
public interface ISomeNonCriticalException {
}
The Caller code:
public void CallPossiblyInconclusiveMethod(Action something) {
try {
something();
}
catch (Exception ex) {
if (ex is ISomeNonCriticalException) Assert.Inconclusive();
else throw;
}
}
And in your test:
CallPossiblyInconclusiveMethod(()=>SomeMethodThatMightFail());

Using try/catch blocks in unit tests

Using VS's testing framework, I'm currently writing my tests like this:
[TestMethod]
public void TestSomething()
{
try
{
var someTestValue = _someTestClass.SomeTestMethod();
Assert.IsNotNull(someTestValue);
}
catch (Exception e)
{
Assert.Fail(e.Message);
}
}
My logic is that if an exception is thrown in SomeTestMethod(), I'll immediately terminate the test displaying the exception message through Assert.Fail(e.Message).
The "normal way" of doing things would be:
[TestMethod]
public void TestSomething()
{
var someTestValue = _someTestClass.SomeTestMethod();
Assert.IsNotNull(someTestValue);
}
Is my approach correct, or is the "normal way" correct? Am I writing redundant code ?

I'd say this is redundant, yes. If the exception is an unexpected result then the test fails. Whether it fails by throwing an exception or by failing an Assert is treated the same way by the testing framework. Basically, a failed test is a failed test.

The testing framework will already fail the test if the test method throws an exception. You're adding extra work for no added value.

You're writing redundant code. The Test method already traps exceptions and will fail the test method.

If you are expecting some exception, use this attribute:
[TestMethod]
[ExpectedException(typeof(//Here the exception you expect))] <------------
public void TestSomething()
{
//Your test code
}
This means that if the tests throw this exception, will return as passed the tests.
If you don´t go to tests exceptions, then avoid try/catch blocks inside your tests. You will notice that something went wrong because the tests will fail ;)
I hope this helps

See this tutorial; there is an example unit test using the [expectedException()] attribute:
[Tutorial for ExpectedException Unit tests][1]
[1]: http://msdn.microsoft.com/en-us/library/hh694602%28v=vs.110%29.aspx#BKMK_Writing_your_tests

Why My Assert Fails? (Exception thrown)

I'm using testing in my code, and by my understanding, all a test requires to run in a method is this:
private ClassName(Argument)
{
return;
}
So what the test is actually checking is if the "Argument" is the correct output at the end of the method/test.
However in my case the argument passed in has nothing to do with the output. Therefore I am uncertain of how to test it.
NOTE: The code itself is sound, it outputs the correct variables. Also I actually wrote the test after the code (Big booboo, I know. I want to get into the habit of always testing.)
EDIT: I believe the issue is to do with the Test being exited via the LoginView.Close and LoginView.Menu.SetMenuView. (The function ends on this line, and the test cannot complete?)
The Location of the return; also affects the rest of the code, so it needs to be at the end of the code. If I move it up, the rest of the code becomes unreachable, and the program breaks.
I have looked for ways to try and bypass the exceptions, and all I have come up with is to test that the exception is actually being thrown correctly. Is there a way to bypass the exception in the test?
Here is the test:
public void Compare_LoginTest()
{
User_LoginView LoginView = new User_LoginView();
User_Controller.screenName = "ben";
User_Controller.screenPwd = "password";
User_Controller.Compare_Login(LoginView);
int actual = User_Controller.screenAccess;
int expected = 1;
Assert.AreEqual(expected, actual);
}
Here is the method being tested:
public static void Compare_Login(User_LoginView LoginView)
{
// Creates a new oject of User_Model and populates it from the User_Controller.
User_Model AccessModel = new User_Model();
AccessModel.Name = screenName;
AccessModel.Pwd = screenPwd;
// Runs the Login Comparsion in the Database_Facade, and passes in the Model.
Database_Facade Database = new Database_Facade();
Database.GetLoginAccess(AccessModel);
screenAccess = AccessModel.AccessLevel;
Menu_View.accessLevelSet = AccessModel.AccessLevel;
// Compares the returned AccessLevel.
// if it is corect; closes the Login and runs the SetMenuView method,
// if it is incorrect; shows an error.
if (AccessModel.AccessLevel > 0)
{
Console.WriteLine("Access Level " + AccessModel.AccessLevel);
// Exception Thrown in testing.
LoginView.Close();
LoginView.Menu.SetMenuView();
}
else
{
ErrorCodes_Controller LoginError = new ErrorCodes_Controller();
LoginError.WrongLoginError();
}
return;
}

I don't understand what you're asking... The test will pass if the static UserController.screenAccess variable is 1 - and it looks like the Compare_Login method always sets it to 1.
This all looks fine, so if your test is failing, then there's something preventing the code from reaching that point.
For most unit test frameworks (I'm guessing you're just using the built in microsoft one?), exceptions are how unit test methods pass or fail - The Assert.Xyz methods throw an exception if the assertion isn't right, otherwise they don't. Any other exceptions that your code may throw also count as a fail.
The test framework catches any exceptions and flags the test as passed if there aren't any
I would guess that something in the Check_Login function or earlier in the Compare_Login function is throwing an exception. This is the only thing I can see that would cause your unit test to fail. I'm surprised however that you haven't noticed such an exception (try clicking on the failing unit test and seeing what details you can see about the failure)

I want to assert that an exception was thrown within a certain class

I am building a TestProject for my client/server setup. I want to verify that a test fails within a certain block of execution (there is no client to send to so the server's Send() method will throw an exception). Since I do not want to have the tests boot up a client and server and have them communicate (which I have had problems doing on a single machine.)
If the code reaches this line, that means that the program's execution flow could only fail within the responsibilities of another test. Is there an easier way to do this other than doing a substring check on the thrown exception's stacktrace? I feel like this method is not very expandable and would require constant attention if class names change.
Is there a way that doesn't even involve manually checking the exception's stacktrace?

If you are using NUnit
Without using DataAnnotations
[Test]
public void Test_XXXXXXX
{
var yourClass = new YourClass();
Assert.That(()=>yourClass.Method(),
.Throws.Exception
.TypeOf<TypeOfYourException>
.With.Property("Message")
.EqualTo("the message you are expecting goes here")
);
}
Using DataAnnotations
[Test]
[ExpectedException(typeof(ExceptionType), ExpectedMessage="your message goes here!")]
public void Test_XXXXXXX
{
var yourClass = new YourClass();
// Call your method in a way that it will fail
yourClass.YourMethod();
}

Is there anything unique about the exception in the class, other than that it's specific to that class?
If it's identifiable by the message you can test it as the other answer has shown, or like this if you're not using NUnit:
try {
myMethod();
Assert.Fail("Expected exception to be thrown.");
} catch (MyException ex) {
Assert.Equals("My Exception Message", ex.Message, "Exception message was formatted incorrectly.");
} catch (Exception) {
Assert.Fail("An exception was thrown, but of the wrong type.");
}

When you are unit-testing some class, there is two sources of exception:
Class which you are testing can throw an exception
Dependency of a class can throw an exception
In second case you usually either handle exception, or wrap it in more high-level exception and throw it to caller. So, how to test all these cases?
System under test throws exception
E.g. throwing exception in case of wrong argument passed to method (NUnit sample):
StockService service = new StockService();
Assert.Throws<ArgumentNullException>(() => service.Connect(null));
You don't need to check stack trace, because it's a class under test who supposed to throw exception.
Dependency throws exception and we handle it
When your class has dependencies, you should mock dependencies in order to test your class in isolation. So, it's very easy to setup mocks to throw exceptions when your class interacts with them. Consider case when service should run on default settings if configuration file not found (Moq sample):
var configMock = new Mock<IStockServiceConfig>();
configMock.Setup(c => c.Load()).Throws<FileNotFoundException>();
StockService service = new StockService(configMock.Object);
service.Connect("google");
configMock.VerifyAll();
Assert.That(service.Port, Is.EqualTo(80));
This test will fail if you will not try to load config, or if you will not handle FileNotFoundException.
Exception stacktrace does not matters here - we don't care whether our direct dependency thrown exception, or it was some other class inside dependency. Actually we don't know if that class exists - we are interacting only with direct dependency. And we should care only about fact that dependency can throw exception, which we can handle.
Dependency throws exception and we wrap it
And last case - wrapping exception in something more high-level. Consider previous sample, but config is really important and you can't start without configuration file. In this case you wrap FileNotFoundException into something more business-specific, which makes sense at higher levels of your application. E.g. StockServiceInitializationException:
var configMock = new Mock<IStockServiceConfig>();
configMock.Setup(c => c.Load()).Throws<FileNotFoundException>();
StockService service = new StockService(configMock.Object);
Assert.Throws<StockServiceInitializationException>(_ => service.Connect("bing"));
configMock.VerifyAll();
As you can see, we also don't care about stacktrace of exception, which throws our dependency. It also could be some wrapper of more low level exception. Expected behavior of service - throw high-level initialization exception if config not found. We are verifying that behavior here.

ExpectedException not catching exception, but I can catch it with try catch

Any ideas on this one? I'm trying to write a unit test that will delete an item and confirm that item is no longer in a repository by trying to retrieve the item by its ID which should throw a DataAccessException. However, the test keeps failing. I added a try catch block and sure enough I caught the exception I was expecting. I'm using VS Test Tools for unit testing.
[ExpectedException(typeof(DataAccessException))]
private static void NHibernateRepositoryBaseDeleteHelper<T, TKey>(T myItem, TKey myItemId)
{
MyTestRepository<T, TKey> myRepository = new MyTestRepository<T, TKey>();
myRepository.Delete(myItem);
myRepository.CommitChanges();
try
{
myRepository.GetById(myItemId, false);
}
catch (DataAccessException dae)
{
Assert.IsTrue(true);
}
}

I'll add to what Jared said by pointing out that the "ExpectedException" attribute sucks. There's no way to assert that the exception's message is correct (the "message" parameter doesn't do what you might think it does) and you can't check for multiple exceptions in one test.
A better solution is to do something like this:
Link
That class lets you do neat stuff like this:
[TestMethod]
public void TestAFewObviousExceptions()
{
// some setup here
ExceptionAssert.Throws("Category 47 does not exist", () =>
wallet.Categories.GetChildCategoryIds(47));
ExceptionAssert.Throws("Id Flim is not valid", () =>
wallet.Categories.IdFromName("Flim"));
}

You need to add the ExpectedException attribute onto the same method which has the TestMethod attribute. The VS Unit Test Framework will only look for an ExpectedException attribute at the entry point of a particular test.
[TestMethod]
[ExpectedException(typeof(DataAccessException))]
public void ATestMethod() {
...
NHibernateRepositoryBaseDeleteHelper(itemValue, keyValue);
}

You are munging up your delete method first of all. I would have a separate delete method and test method. Yes, you are testing the delete, but really you are testing the behavior that is associated with delete. There are probably additional tests here.
As this is part of what appears to be an autogenerated test, I would not add to the same repository object in that test, but would perform my test(s) in their own test methods and use the delete as part of the set up for the class. That will certainly separate the test from the delete.
Hope this helps.