We Keep Coding

Implement a Custom Exception that Fails a Unit Test as Inconclusive

Using the test tools built into Visual Studio (in my case VS 2013, though I believe this extends to other versions as well), a 'TestMethod' (any method marked with a [TestMethod] attribute) can result in 1 of 3 statuses after being run:
Passed
Skipped/Inconclusive
Failed
A test is passed if it runs to completion with no uncaught exceptions. A test fails it throws an uncaught exception. A test is inconclusive if it throws a special type of uncaught exception: AssertInconclusiveException.
My question is, is it possible (in, say, a [TestCleanup] method) to treat an custom uncaught exception similarly to the AssertInconclusiveException and set the test status to 'Inconclusive' instead of 'Failed'? Or is the recognition of AssertInconclusiveException as a special exception built into Visual Studio itself? What I'm looking for, in rough pseudocode form, is roughly:
[TestCleanup]
public void TestCleanup()
{
if(this.TestContext.CurrentTestOutcome == UnitTestOutcome.Failed
&& this.TestContext.UncaughtException.GetType() == typeof(MyCustomException))
{
this.TestContext.CurrentTestOutcome = UnitTestOutcome.Inconclusive;
}
}
This would be used as part of an integration test suite to mark a test as inconclusive whenever the setup method for the test throws a special SetupFailureException (usually indicative of broken downstream system that is not the one under test).
I've been able to produce the desired behavior in the interim by defining:
public class SetupFailedException : AssertInconclusiveException
{
}
// example test that demonstrates desired behavior
[TestMethod]
public void TestThatFailsAsInconclusiveByThrowingUncaughtSpecialException()
{
throw new SetupFailedException();
}
But this seems messy since the thrown exception has nothing to do with an assertion and prevents custom exceptions which should fail tests as inconclusive from inheriting from any other abstract class.

The Inconclusive recognition is built into the test framework, by the time you get to cleanup, it's too late to change a fail into an inconclusive.
It feels like you should really be catching the exceptions from within your integration tests anyway, since this shows that you're recognising the method your testing can throw a certain type of exception and that the right thing to do is to ignore the result in that situation.
If you need to have multiple exceptions that can be detected like this, I'd be tempted to have them implement an interface, so that the catch block can just check if any exceptions caught implement that interface.
If you wanted to minimize the impact/visibility on the rest of your test code then you could wrap the call that could fail in an action, in which case your code might look something like this:
The interface:
public interface ISomeNonCriticalException {
}
The Caller code:
public void CallPossiblyInconclusiveMethod(Action something) {
try {
something();
}
catch (Exception ex) {
if (ex is ISomeNonCriticalException) Assert.Inconclusive();
else throw;
}
}
And in your test:
CallPossiblyInconclusiveMethod(()=>SomeMethodThatMightFail());

Using try/catch blocks in unit tests

Using VS's testing framework, I'm currently writing my tests like this:
[TestMethod]
public void TestSomething()
{
try
{
var someTestValue = _someTestClass.SomeTestMethod();
Assert.IsNotNull(someTestValue);
}
catch (Exception e)
{
Assert.Fail(e.Message);
}
}
My logic is that if an exception is thrown in SomeTestMethod(), I'll immediately terminate the test displaying the exception message through Assert.Fail(e.Message).
The "normal way" of doing things would be:
[TestMethod]
public void TestSomething()
{
var someTestValue = _someTestClass.SomeTestMethod();
Assert.IsNotNull(someTestValue);
}
Is my approach correct, or is the "normal way" correct? Am I writing redundant code ?

I'd say this is redundant, yes. If the exception is an unexpected result then the test fails. Whether it fails by throwing an exception or by failing an Assert is treated the same way by the testing framework. Basically, a failed test is a failed test.

The testing framework will already fail the test if the test method throws an exception. You're adding extra work for no added value.

You're writing redundant code. The Test method already traps exceptions and will fail the test method.

If you are expecting some exception, use this attribute:
[TestMethod]
[ExpectedException(typeof(//Here the exception you expect))] <------------
public void TestSomething()
{
//Your test code
}
This means that if the tests throw this exception, will return as passed the tests.
If you don´t go to tests exceptions, then avoid try/catch blocks inside your tests. You will notice that something went wrong because the tests will fail ;)
I hope this helps

See this tutorial; there is an example unit test using the [expectedException()] attribute:
[Tutorial for ExpectedException Unit tests][1]
[1]: http://msdn.microsoft.com/en-us/library/hh694602%28v=vs.110%29.aspx#BKMK_Writing_your_tests

Why My Assert Fails? (Exception thrown)

I'm using testing in my code, and by my understanding, all a test requires to run in a method is this:
private ClassName(Argument)
{
return;
}
So what the test is actually checking is if the "Argument" is the correct output at the end of the method/test.
However in my case the argument passed in has nothing to do with the output. Therefore I am uncertain of how to test it.
NOTE: The code itself is sound, it outputs the correct variables. Also I actually wrote the test after the code (Big booboo, I know. I want to get into the habit of always testing.)
EDIT: I believe the issue is to do with the Test being exited via the LoginView.Close and LoginView.Menu.SetMenuView. (The function ends on this line, and the test cannot complete?)
The Location of the return; also affects the rest of the code, so it needs to be at the end of the code. If I move it up, the rest of the code becomes unreachable, and the program breaks.
I have looked for ways to try and bypass the exceptions, and all I have come up with is to test that the exception is actually being thrown correctly. Is there a way to bypass the exception in the test?
Here is the test:
public void Compare_LoginTest()
{
User_LoginView LoginView = new User_LoginView();
User_Controller.screenName = "ben";
User_Controller.screenPwd = "password";
User_Controller.Compare_Login(LoginView);
int actual = User_Controller.screenAccess;
int expected = 1;
Assert.AreEqual(expected, actual);
}
Here is the method being tested:
public static void Compare_Login(User_LoginView LoginView)
{
// Creates a new oject of User_Model and populates it from the User_Controller.
User_Model AccessModel = new User_Model();
AccessModel.Name = screenName;
AccessModel.Pwd = screenPwd;
// Runs the Login Comparsion in the Database_Facade, and passes in the Model.
Database_Facade Database = new Database_Facade();
Database.GetLoginAccess(AccessModel);
screenAccess = AccessModel.AccessLevel;
Menu_View.accessLevelSet = AccessModel.AccessLevel;
// Compares the returned AccessLevel.
// if it is corect; closes the Login and runs the SetMenuView method,
// if it is incorrect; shows an error.
if (AccessModel.AccessLevel > 0)
{
Console.WriteLine("Access Level " + AccessModel.AccessLevel);
// Exception Thrown in testing.
LoginView.Close();
LoginView.Menu.SetMenuView();
}
else
{
ErrorCodes_Controller LoginError = new ErrorCodes_Controller();
LoginError.WrongLoginError();
}
return;
}

I don't understand what you're asking... The test will pass if the static UserController.screenAccess variable is 1 - and it looks like the Compare_Login method always sets it to 1.
This all looks fine, so if your test is failing, then there's something preventing the code from reaching that point.
For most unit test frameworks (I'm guessing you're just using the built in microsoft one?), exceptions are how unit test methods pass or fail - The Assert.Xyz methods throw an exception if the assertion isn't right, otherwise they don't. Any other exceptions that your code may throw also count as a fail.
The test framework catches any exceptions and flags the test as passed if there aren't any
I would guess that something in the Check_Login function or earlier in the Compare_Login function is throwing an exception. This is the only thing I can see that would cause your unit test to fail. I'm surprised however that you haven't noticed such an exception (try clicking on the failing unit test and seeing what details you can see about the failure)

Using Nunit to test constructor

I'm new to Nunit testing and I wanted to test the following constructor:
public class IngredientDAONHibernate : NutritionLibrary.DAO.IngredientDAO
{
private Configuration config;
private ISessionFactory factory;
public IngredientDAONHibernate()
{
try
{
config = new Configuration();
config.AddClass(typeof(NutritionLibrary.Entity.Ingredient));
config.AddClass(typeof(Entity.Nutrient));
config.AddClass(typeof(Entity.NutrientIngredient));
factory = config.BuildSessionFactory();
}
catch (Exception e)
{
// logger.Error("Exception Occured", e);
}
}
The test stub is as follows:
[TestMethod()]
public void IngredientDAONHibernateConstructorTest()
{
IngredientDAONHibernate target = new IngredientDAONHibernate();
}
Could someone help me with some tips as to how I can get started? Thanks!

I would suggest that you don't catch all the exceptions, but, if there are certain you want to catch and ignore then just do those, otherwise it is harder to tell if you have a problem.
I tend not to test the constructor, unless it is doing quite a bit, and it will be obvious if there is a problem when doing the other unit tests, as, if the constructor fails you should see by the exception, and by the fact that all the tests will be failing.
If you want to test this constructor, limit your exceptions and just ensure that there is no exception when you run the test.

Let me turn the question back on to you.. How do you know if the constructor executed as intended ?
Normally constructors are trivial.. but here it seems that you have some third party lib interfacing code that you need some confidence with.
If you only want to test that there are no exceptions raised from within the constructor... then Extract a logger interface. Now in your test pass in a mock logger (a fake can also suffice), which should help you to sense if an exception was logged.
[TestMethod()]
public void IngredientDAONHibernateConstructorTest()
{
_errorLogged = false;
ILogger logger = this; // make test fixture implement the logger interface ; self-shunt
IngredientDAONHibernate target = new IngredientDAONHibernate(logger);
Assert.IsNotNull(target);
Assert.IsFalse(_errorLogged,
String.Format("ERROR! Constructor has thrown {0}", _loggedException) );
}
bool _errorLogged;
Exception _loggedException;
public void Error(string message, Exception e)
{
_errorLogged = true;
_loggedException = e;
}

Like everyone else, I would caution against catching Exception. The documentation should tell you what exceptions can be expected and you could write specific Catch clauses for each of them, if that made sense.
Leaving that aside, your tests should verify that your objects are in the state you expect. Given what we can see, all you can do is test that it's not null. If you can force an exception, you could test that the logger gets the exception message. If there was a public property that gave access to your factory, you would want to test that (not null, perhaps other things) and likewise with the config field. For that, if you have access to the added classes, you could test not null and count==3. If the exact objects are important, you could test that they are there. It's always tricky to decide how much to trust third-party stuff. Generally, I do unless I get evidence that I shouldn't.
When it comes to testing methods, test for each parameter whatever is possible. So, for a string, test null and emptystring plus relevant (valid and invalid) values. If you're expecting alphanumerics, test special chars as well. If data has boundaries test values on and either side of the boundary(ies) plus typical values. E.g. if you're expecting an int between 0 and 10, test -1, 0, 1, 5, 9, 10, 11. It only takes seconds to write but it can save you hours in two years when you're trying to fix a bug.

ExpectedException not catching exception, but I can catch it with try catch

Any ideas on this one? I'm trying to write a unit test that will delete an item and confirm that item is no longer in a repository by trying to retrieve the item by its ID which should throw a DataAccessException. However, the test keeps failing. I added a try catch block and sure enough I caught the exception I was expecting. I'm using VS Test Tools for unit testing.
[ExpectedException(typeof(DataAccessException))]
private static void NHibernateRepositoryBaseDeleteHelper<T, TKey>(T myItem, TKey myItemId)
{
MyTestRepository<T, TKey> myRepository = new MyTestRepository<T, TKey>();
myRepository.Delete(myItem);
myRepository.CommitChanges();
try
{
myRepository.GetById(myItemId, false);
}
catch (DataAccessException dae)
{
Assert.IsTrue(true);
}
}

I'll add to what Jared said by pointing out that the "ExpectedException" attribute sucks. There's no way to assert that the exception's message is correct (the "message" parameter doesn't do what you might think it does) and you can't check for multiple exceptions in one test.
A better solution is to do something like this:
Link
That class lets you do neat stuff like this:
[TestMethod]
public void TestAFewObviousExceptions()
{
// some setup here
ExceptionAssert.Throws("Category 47 does not exist", () =>
wallet.Categories.GetChildCategoryIds(47));
ExceptionAssert.Throws("Id Flim is not valid", () =>
wallet.Categories.IdFromName("Flim"));
}

You need to add the ExpectedException attribute onto the same method which has the TestMethod attribute. The VS Unit Test Framework will only look for an ExpectedException attribute at the entry point of a particular test.
[TestMethod]
[ExpectedException(typeof(DataAccessException))]
public void ATestMethod() {
...
NHibernateRepositoryBaseDeleteHelper(itemValue, keyValue);
}

You are munging up your delete method first of all. I would have a separate delete method and test method. Yes, you are testing the delete, but really you are testing the behavior that is associated with delete. There are probably additional tests here.
As this is part of what appears to be an autogenerated test, I would not add to the same repository object in that test, but would perform my test(s) in their own test methods and use the delete as part of the set up for the class. That will certainly separate the test from the delete.
Hope this helps.