Using Selenium in Visual Studio. Using NUnit to sort my testcases.
I'm writing a testcase that compares two serialnumbers with a if statement like this:
[Test]
public void CompareVariables()
{
if (string.Equals(serialNumberInfo, serialNumberReport))
Console.WriteLine($"{serialNumberInfo} and {serialNumberReport} are a match! Proceed!");
else
Console.WriteLine($"{serialNumberInfo} and {serialNumberReport} don't match! Cancel test!");
//method for stopping test missing!
I want to be able to abort the rest of the testsequence if the serialnumbers don't match.
Is there a "end/stop test" method or something similar I could put in else section?
I think you have a couple of options.
1) simply throw an exception (and fail the test)
Throwing an exception will fail a unit test. There are loads of different types of exceptions but the base is simply Exception. You can check the different types of exceptions available here. Where possible try to pick the exception that most closely represents the error (so bad arguments for example use ArgumentException or some derivative there of).
Your test would then look something like this:
[Test]
public void CompareVariables()
{
if (!string.Equals(serialNumberInfo, serialNumberReport))
throw new Exception($"{serialNumberInfo} and {serialNumberReport} don't match! Cancel test!");
// The rest of your tests (only run if serialNumberInfo and serialNumberReport) are equal.
}
2) Use an assertion (and fail the test)
Unit tests are usually supposed to assert that something desirable happened. If that thing didn't happen then an exception should be thrown (which is often handled for you by some assertion framework).
So you could flip the test to do this:
[Test]
public void CompareVariables()
{
serialNumberInfo.ShouldBe(serialNumberReport);
// The rest of your tests (only run if serialNumberInfo and serialNumberReport) are equal.
}
This is done with Shouldly but there are countless assertion frameworks so pick your favourite. (mstest has one built in but I find it less readable but that is a personal preference).
Note, only use an assertion when you want to explicitly make sure that it should have happened. I.e. This needs to be true for my test to pass, rather than if this happened then abort. That's hard to explain so I hope that makes sense?
Exceptions for when something went wrong, Assertions for when something should have gone right.
3) Leave the test (and pass the test)
If the test exits without an exception being thrown (either manually or via an assertion framework) then the test is considered to be a passing test. Therefor if you wanted to treat this as a pass you could simply return from the test.
[Test]
public void CompareVariables()
{
if (string.Equals(serialNumberInfo, serialNumberReport))
{
Console.WriteLine($"{serialNumberInfo} and {serialNumberReport} are a match! Proceed!");
}
else
{
Console.WriteLine($"{serialNumberInfo} and {serialNumberReport} don't match! Cancel test!");
return;
}
// The rest of your tests
}
This will mark the test as passing, but mean the rest of the operations in the test are not run. I would try not to do this however - unless you really understand why you want this because you could start passing tests without knowing why they passed (i.e. without asserting anything)
I hope that helps
If you want to end the test early without failing it, simply use return.
[Test]
public void MyTest() {
//Do some stuff
if(!shouldContinue) {
return;
}
}
I do this reasonably often given certain conditions may warrant additional assertions, and other conditions may not. Throwing an exception will fail the test. This will not fail it.
Edit: I just noticed that the other responder mentioned this at the end of their answer. So ignore me :)
Related
I use Theory with MemberData like this:
[Theory]
[MemberData(nameof(TestParams))]
public void FijewoShortcutTest(MapMode mapMode)
{
...
and when it works, it is all fine, but when it fails XUnit iterates over all data I pass as parameters. In my case it is fruitless attempt, I would like to stop short -- i.e. when the first set of parameters make the test to fail, stop the rest (because they will fail as well -- again, it is my case, not general rule).
So how to tell XUnit to stop Theory on the first fail?
The point of a Theory is to have multiple independent tests running the same code of different data. If you only actually want one test, just use a Fact and iterate over the data you want to test within the method:
[Fact]
public void FijewoShortcutTest()
{
foreach (MapMode mapMode in TestParams)
{
// Test code here
}
}
That will mean you can't easily run just the test for just one MapMode though. Unless it takes a really long time to execute the tests for some reason, I'd just live with "if something is badly messed up, I get a lot of broken tests".
I have some code that contains a try-catch block. Upon checking my code coverage, I noticed that my catch block isn't covered.
public bool TryDoSomething()
{
try
{
SomePrivateMethod();
return true;
}
catch(Exception e)
{
// some logs
return false;
}
}
I know that, using a stub, I can easily make it throw for me, thus entering the catch block.
But what if the code is self contained like the example above?
Do I have to somehow extract the code just to make it throw? Like this
public bool TryDoSomething()
{
try
{
_someDependencyInjectedEarlier.SomeMethod();
return true;
}
catch(Exception e)
{
// some logs
return false;
}
}
Or the other way around (feels odd):
public bool TryDoSomething()
{
return _injectedHelper.Try<Exception>(
() => SomePrivateMethods(),
e => { /* some logs */ }
);
}
with Try being
public bool Try<T>(Action action, Action<T> onError) where T: Exception
{
try
{
action();
return true;
}
catch (T e)
{
onError(e);
return false;
}
}
Or maybe some IL generated at runtime to make it throw? Which seems very complicated and wrong as well.
What are my options?
Should I be posting this on another sub instead?
You need to use mocking frameworks like Mockito to write unit tests. Here is an example from 2013 using Mockito with PowerMock.
Mocking private method with Mockito
With a mocking framework, you intercept calls to real methods and provide mocked responses without actually executing the called methods. In this case, since you want to trigger an exception, you need to have Mockito intercept the call and throw an exception.
Here is example Mockito code to do that:
doThrow(new Exception()).when(mockedObject).SomePrivateMethod(...);
You may want your original code to be restructured to make mocking that called method easier.
When writing unit tests for your objects, don't think of mocks in terms of "methods" or "try blocks". Think of mocks in terms of two things:
Inputs
Dependencies
The unit test provides those two things, in the form of directly inputting values and mocking dependencies.
So the question isn't:
How can I get this method call to fail?
But rather:
What input or dependency can cause this functionality to fail?
As a contrived example, imagine the method being invoked does this:
return 1 + 1;
There are no inputs, no dependencies. All of the logic is fully enclosed within the object being tested. And that logic can't fail. So there's no need for a try/catch in the first place and the solution is to remove that structure.
Or, suppose the method being invoked does this:
return 1 / this.x;
Where does this.x come from? That's potentially an input. For example, maybe it's supplied to the constructor when creating the object. In this case, if you were to supply 0 to the constructor, you'd trigger an exception. The unit test should supply 0 and validate the result.
Or, suppose the method being invoked does this:
return this._someService.SomeOperation();
What is _someService? Is that another object on which this object depends? Then that's a dependency to be mocked. If it's being internally constructed:
this._someService = new SomeService();
Then now is the time to "invert the dependency" and make it a constructor parameter instead of internally creating it. If it's already a constructor parameter, that's how the unit test would provide a mock.
Overall the point is that the unit test isn't testing implementation (it doesn't care what/where the try/catch structure it, it doesn't care what private methods are invoked, etc.), it's testing functionality. And the object should advertise the inputs/dependencies it needs to perform that functionality.
If there are no inputs or dependencies then the logic shouldn't be able to fail, because such failure potential would literally be hard-coded into the logic itself. You can manually test it during development to harden it against failures. But if there are inputs and/or dependencies, those can provide a situation in which the logic being tested could potentially fail. Those should be mocked for tests.
I have a method that takes a JSON object and puts it through several stages of processing, updating values in the database at each stage. We wanted this method to be fault tolerant, and decided that the desired behaviour would be, if any processing stage failed, log an error to the database and carry on with the next stage of processing, rather than aborting.
I've just made several changes to the behaviour of one of the processing steps. I then ran our unit test suite, expecting several of the tests to fail due to the new behaviour and point me at potential problem areas. Instead, the tests all passed.
After investigating, I realised that the mock data the tests run against didn't include certain key values important for the new behaviour. The tests were in fact throwing exceptions when they ran, but the exceptions were being caught and handled - and, because the tests don't run with a logger enabled, they were completely suppressed. So the new code didn't change the data in a way that would cause the tests to fail, because it was silently erroring instead.
This seems like the sort of problem that unit tests are there to catch, and the fact that they showed no trace means they're not serving their purpose. Is there any way that I can use NUnit to assert that no exception was ever thrown, even if it was handled? Or alternatively, is there a sensible way to refactor that would expose this issue better?
(Working in C#, but the question seems fairly language-agnostic)
First and foremost, in the scenario you describe, the presence or absence of exceptions is secondary. If you wrote the code to produce a desired result while catching and handling exceptions, then that result - whether it's a return value or some other effect - is the most important thing to test.
If the exceptions that you didn't see caused that result to be incorrect, then testing for the correct result will always reveal the problems. If you don't know what the expected result will be and are only interested in whether or not exceptions are getting handled, something is wrong. We can never determine whether or not anything works correctly according to whether or not it throws exceptions.
That aside, here's how to test whether or not your code is catching and logging exceptions that you wouldn't otherwise be able to observe:
If you're injecting a logger that looks something like this:
public interface ILogger
{
void LogError(Exception ex);
void LogMessage(string message);
}
...then a simple approach is to create a test double which stores the exceptions so that you can inspect it and see what was logged.
public class ListLoggerDouble : ILogger
{
public List<Exception> Exceptions = new List<Exception>();
public List<string> Messages = new List<string>();
public void LogError(Exception ex)
{
Exceptions.Add(ex);
}
public void LogMessage(string message)
{
Messages.Add(message);
}
}
After you've executed the method you're testing you can assert that a collection contains the exception(s) or message(s) you expect. If you wish you can also verify that there are none, although it seems like that might be redundant if the result you're testing for is correct.
I wouldn't create a logger that throws an exception and then write a test that checks for a thrown exception. That makes it look like the expected behavior of your code is to throw an exception, which is exactly the opposite of what it does. Tests help us to document expected behaviors. Also, what will you do if you want to verify that you caught and logged two exceptions?
Im running some tests on my code at the moment. My main test method is used to verify some data, but within that check there is a lot of potential for it to fail at any one point.
Right now, I've set up multiple Assert.Fail statements within my method and when the test is failed, the message I type is displayed as expected. However, if my method fails multiple times, it only shows the first error. When I fix that, it is only then I discover the second error.
None of my tests are dependant on any others that I'm running. Ideally what I'd like is the ability to have my failure message to display every failed message in one pass. Is such a thing possible?
As per the comments, here are how I'm setting up a couple of my tests in the method:
private bool ValidateTestOne(EntityModel.MultiIndexEntities context)
{
if (context.SearchDisplayViews.Count() != expectedSdvCount)
{
Assert.Fail(" Search Display View count was different from what was expected");
}
if (sdv.VirtualID != expectedSdVirtualId)
{
Assert.Fail(" Search Display View virtual id was different from what was expected");
}
if (sdv.EntityType != expectedSdvEntityType)
{
Assert.Fail(" Search Display View entity type was different from what was expected");
}
return true;
}
Why not have a string/stringbuilder that holds all the fail messages, check for its length at the end of your code, and pass it into Assert.Fail? Just a suggestion :)
The NUnit test runner (assuming thats what you are using) is designed to break out of the test method as soon as anything fails.
So if you want every failure to show up, you need to break up your test into smaller, single assert ones. In general, you only want to be testing one thing per test anyways.
On a side note, using Assert.Fail like that isn't very semantically correct. Consider using the other built-in methods (like Assert.Equals) and only using Assert.Fail when the other methods are not sufficient.
None of my tests are dependent on any others that I'm running. Ideally
what I'd like is the ability to have my failure message to display
every failed message in one pass. Is such a thing possible?
It is possible only if you split your test into several smaller ones.
If you are afraid code duplication which is usually exists when tests are complex, you can use setup methods. They are usually marked by attributes:
NUnit - SetUp,
MsTest - TestInitialize,
XUnit - constructor.
The following code shows how your test can be rewritten:
public class HowToUseAsserts
{
int expectedSdvCount = 0;
int expectedSdVirtualId = 0;
string expectedSdvEntityType = "";
EntityModelMultiIndexEntities context;
public HowToUseAsserts()
{
context = new EntityModelMultiIndexEntities();
}
[Fact]
public void Search_display_view_count_should_be_the_same_as_expected()
{
context.SearchDisplayViews.Should().HaveCount(expectedSdvCount);
}
[Fact]
public void Search_display_view_virtual_id_should_be_the_same_as_expected()
{
context.VirtualID.Should().Be(expectedSdVirtualId);
}
[Fact]
public void Search_display_view_entity_type_should_be_the_same_as_expected()
{
context.EntityType.Should().Be(expectedSdvEntityType);
}
}
So your test names could provide the same information as you would write as messages:
Right now, I've set up multiple Assert.Fail statements within my
method and when the test is failed, the message I type is displayed as
expected. However, if my method fails multiple times, it only shows
the first error. When I fix that, it is only then I discover the
second error.
This behavior is correct and many testing frameworks follow it.
I'd like to recommend stop using Assert.Fail() because it forces you to write specific messages for every failure. Common asserts provide good enough messages so you can replace you code with the following lines:
// Act
var context = new EntityModelMultiIndexEntities();
// Assert
Assert.Equal(expectedSdvCount, context.SearchDisplayViews.Count());
Assert.Equal(expectedSdVirtualId, context.VirtualID);
Assert.Equal(expectedSdvEntityType, context.EntityType);
But I'd recommend start using should-frameworks like Fluent Assertions which make your code mere readable and provide better output.
// Act
var context = new EntityModelMultiIndexEntities();
// Assert
context.SearchDisplayViews.Should().HaveCount(expectedSdvCount);
context.VirtualID.Should().Be(expectedSdVirtualId);
context.EntityType.Should().Be(expectedSdvEntityType);
What is the best way to unit test a method that doesn't return anything? Specifically in c#.
What I am really trying to test is a method that takes a log file and parses it for specific strings. The strings are then inserted into a database. Nothing that hasn't been done before but being VERY new to TDD I am wondering if it is possible to test this or is it something that doesn't really get tested.
If a method doesn't return anything, it's either one of the following
imperative - You're either asking the object to do something to itself.. e.g change state (without expecting any confirmation.. its assumed that it will be done)
informational - just notifying someone that something happened (without expecting action or response) respectively.
Imperative methods - you can verify if the task was actually performed. Verify if state change actually took place. e.g.
void DeductFromBalance( dAmount )
can be tested by verifying if the balance post this message is indeed less than the initial value by dAmount
Informational methods - are rare as a member of the public interface of the object... hence not normally unit-tested. However if you must, You can verify if the handling to be done on a notification takes place. e.g.
void OnAccountDebit( dAmount ) // emails account holder with info
can be tested by verifying if the email is being sent
Post more details about your actual method and people will be able to answer better.
Update: Your method is doing 2 things. I'd actually split it into two methods that can now be independently tested.
string[] ExamineLogFileForX( string sFileName );
void InsertStringsIntoDatabase( string[] );
String[] can be easily verified by providing the first method with a dummy file and expected strings. The second one is slightly tricky.. you can either use a Mock (google or search stackoverflow on mocking frameworks) to mimic the DB or hit the actual DB and verify if the strings were inserted in the right location. Check this thread for some good books... I'd recomment Pragmatic Unit Testing if you're in a crunch.
In the code it would be used like
InsertStringsIntoDatabase( ExamineLogFileForX( "c:\OMG.log" ) );
Test its side-effects. This includes:
Does it throw any exceptions? (If it should, check that it does. If it shouldn't, try some corner cases which might if you're not careful - null arguments being the most obvious thing.)
Does it play nicely with its parameters? (If they're mutable, does it mutate them when it shouldn't and vice versa?)
Does it have the right effect on the state of the object/type you're calling it on?
Of course, there's a limit to how much you can test. You generally can't test with every possible input, for example. Test pragmatically - enough to give you confidence that your code is designed appropriately and implemented correctly, and enough to act as supplemental documentation for what a caller might expect.
As always: test what the method is supposed to do!
Should it change global state (uuh, code smell!) somewhere?
Should it call into an interface?
Should it throw an exception when called with the wrong parameters?
Should it throw no exception when called with the right parameters?
Should it ...?
Try this:
[TestMethod]
public void TestSomething()
{
try
{
YourMethodCall();
Assert.IsTrue(true);
}
catch {
Assert.IsTrue(false);
}
}
Void return types / Subroutines are old news. I haven't made a Void return type (Unless I was being extremely lazy) in like 8 years (From the time of this answer, so just a bit before this question was asked).
Instead of a method like:
public void SendEmailToCustomer()
Make a method that follows Microsoft's int.TryParse() paradigm:
public bool TrySendEmailToCustomer()
Maybe there isn't any information your method needs to return for usage in the long-run, but returning the state of the method after it performs its job is a huge use to the caller.
Also, bool isn't the only state type. There are a number of times when a previously-made Subroutine could actually return three or more different states (Good, Normal, Bad, etc). In those cases, you'd just use
public StateEnum TrySendEmailToCustomer()
However, while the Try-Paradigm somewhat answers this question on how to test a void return, there are other considerations too. For example, during/after a "TDD" cycle, you would be "Refactoring" and notice you are doing two things with your method... thus breaking the "Single Responsibility Principle." So that should be taken care of first. Second, you might have idenetified a dependency... you're touching "Persistent" Data.
If you are doing the data access stuff in the method-in-question, you need to refactor into an n-tier'd or n-layer'd architecture. But we can assume that when you say "The strings are then inserted into a database", you actually mean you're calling a business logic layer or something. Ya, we'll assume that.
When your object is instantiated, you now understand that your object has dependencies. This is when you need to decide if you are going to do Dependency Injection on the Object, or on the Method. That means your Constructor or the method-in-question needs a new Parameter:
public <Constructor/MethodName> (IBusinessDataEtc otherLayerOrTierObject, string[] stuffToInsert)
Now that you can accept an interface of your business/data tier object, you can mock it out during Unit Tests and have no dependencies or fear of "Accidental" integration testing.
So in your live code, you pass in a REAL IBusinessDataEtc object. But in your Unit Testing, you pass in a MOCK IBusinessDataEtc object. In that Mock, you can include Non-Interface Properties like int XMethodWasCalledCount or something whose state(s) are updated when the interface methods are called.
So your Unit Test will go through your Method(s)-In-Question, perform whatever logic they have, and call one or two, or a selected set of methods in your IBusinessDataEtc object. When you do your Assertions at the end of your Unit Test you have a couple of things to test now.
The State of the "Subroutine" which is now a Try-Paradigm method.
The State of your Mock IBusinessDataEtc object.
For more information on Dependency Injection ideas on the Construction-level... as they pertain to Unit Testing... look into Builder design patterns. It adds one more interface and class for each current interface/class you have, but they are very tiny and provide HUGE functionality increases for better Unit-Testing.
You can even try it this way:
[TestMethod]
public void ReadFiles()
{
try
{
Read();
return; // indicates success
}
catch (Exception ex)
{
Assert.Fail(ex.Message);
}
}
it will have some effect on an object.... query for the result of the effect. If it has no visible effect its not worth unit testing!
Presumably the method does something, and doesn't simply return?
Assuming this is the case, then:
If it modifies the state of it's owner object, then you should test that the state changed correctly.
If it takes in some object as a parameter and modifies that object, then your should test the object is correctly modified.
If it throws exceptions is certain cases, test that those exceptions are correctly thrown.
If its behaviour varies based on the state of its own object, or some other object, preset the state and test the method has the correct Ithrough one of the three test methods above).
If youy let us know what the method does, I could be more specific.
Use Rhino Mocks to set what calls, actions and exceptions might be expected. Assuming you can mock or stub out parts of your method. Hard to know without knowing some specifics here about the method, or even context.
Depends on what it's doing. If it has parameters, pass in mocks that you could ask later on if they have been called with the right set of parameters.
What ever instance you are using to call the void method , You can just use ,Verfiy
For Example:
In My case its _Log is the instance and LogMessage is the method to be tested:
try
{
this._log.Verify(x => x.LogMessage(Logger.WillisLogLevel.Info, Logger.WillisLogger.Usage, "Created the Student with name as"), "Failure");
}
Catch
{
Assert.IsFalse(ex is Moq.MockException);
}
Is the Verify throws an exception due to failure of the method the test would Fail ?