I'm watching a course for NUnit and I came to this exercise to write a test method for below function
public static string GetOutput(int number)
{
if ((number % 3 == 0) && (number % 5 == 0))
return "FizzBuzz";
if (number % 3 == 0)
return "Fizz";
if (number % 5 == 0)
return "Buzz";
return number.ToString();
}
What I did is that I have created a Parameterized test to cover all the cases like below
[Test]
[TestCase(15 , "FizzBuzz")]
[TestCase(3, "Fizz")]
[TestCase(5, "Buzz")]
public void GetOutput_WhenCalled_ReturnString(int number , string word)
{
var result = FizzBuzz.GetOutput(number);
Assert.That(result, Is.EqualTo(word));
}
But the mentor has wrote each case in a seperate method like for example
public void GetOutput_NumberDivisiableNy3And5_ReturnFizzBuzz()
{
var result = FizzBuzz.GetOutput(15);
Assert.That(result, Is.EqualTo("FizzBuzz"));
}
public void GetOutput_NumberDivisiableBy3_ReturnFizz()
{
var result = FizzBuzz.GetOutput(3);
Assert.That(result, Is.EqualTo("FizzBuzz"));
}
My question is why he wrote it like that? does separating it like this has some advantage?
or is it simply a personal preference. I'm very new to unit test and Nunit
It's a personal preference.
An important guideline when testing is that the names you choose for your tests should describe the desired behavior of the method being tested for a given input. Think of the test names as being akin to executable documentation: "under circumstance X, the method should exhibit behavior Y."
Using parameterized tests means that the test name is by its very nature less descriptive, which makes it more difficult to identify at a glance exactly what's failing -- you have to look at the test parameters to determine the failing case, as the test name isn't providing you with that information.
This is apparent even in your implementation: Your test name says it "returns string" "when called". Which is technically correct, but is not descriptive. What string should it return when called with a given value?
The individual test cases, on the other hand, make the desired behavior very clear: A number divisible by 3 and 5 should return "FizzBuzz", and so on.
Also, think of it like this: What happens if you're asked to go back to this method and expand it to print Fuzz when a number is divisible by 7? And Biz when divisible by 9? And so on. Your single test method can do all of that heavy lifting, but the more requirements you have, the more burden is being put on that single test case and the harder it becomes to clearly identify which scenarios are failing and why.
Your approach isn't wrong, but if I were writing these tests, I would write one test case for each scenario: Fizz, Buzz, FizzBuzz, and not divisible by anything.
Each test case can still be parameterized, so you can test a larger number of values that satisfy each test case.
At bottom, it's a personal preference. However, when the method you are testing takes an argument, it seems very natural to use a parameterized test. OTOH, do not use a parameterized test to write a test for two or more entirely different things... in fact, don't write a test for more than one purpose at a time at all!
My guess is that your instructor is trying to start with a fundamental approach and will only introduce parameterized tests in a later lesson. But of course that's only a guess.
Related
I started looking at FsCheck yesterday, and I am trying to write a simple test, that any instance of DiscountAmount will always have negative value. My question is, is it ok to have multiple asserts within one test. For example, here I am saying that amount from which discountAmount has been created plus discount amount should be 0. But I also say that discount amount should be less than 0. Should this be 2 tests or 1?
public class DiscountAmountTests
{
[Property()]
public void value_or_created_discountAmount_should_be_negative()
{
Arb.Register<AmountArbitrary>();
Prop.ForAll<Amount>(
v =>
{
var sut = new DiscountAmount(v);
var expectedResult = 0;
var result = v + sut;
result.Should().Be(expectedResult);
sut.Value.Should().BeLessThan(0);
})
.QuickCheckThrowOnFailure();
}
public class AmountArbitrary
{
public static Arbitrary<Amount> Amounts()
{
return Arb.Generate<decimal>().Where(x => x > 0)
.Select(x => new Amount(x))
.ToArbitrary();
}
}
}
}
I would say this is really up to you. I think there are arguments pro and cons - on the one hand, sometimes setup cost is expensive (be it in terms of programmer work to get the system into a particular state, or really compute resource cost, e.g. you have to do a expensive query to the DB or something) and then in my opinion it's worth making tests more coarsely grained.
The trade-off is that it's typically less clear what the problem is if a coarse grained test fails.
In comparison with unit tests, FsCheck has a bit more setup costs in terms of argument generation, and it is attractive to make FsCheck tests more coarse-grained than unit tests. Also note that FsCheck has some methods like Label, And. Or to combine different properties together while sharing the argument generation, and still allow you to see what part of your test fails, somewhat off-setting one downside.
Hello I am a newbie to TDD style programming in c# and am struggling a lot for getting it right. Could you please let me know if I am doing this in the right way. I have followed a lot of tutorials but haven't succeeded. I get the theory aspect of it but when it comes to putting it practically I always fail.
I have this repository for practising tdd https://github.com/dev-test-tdd/AlgorithmPractice/. I have started writing all the algorithms from scratch to understand tdd. For example , I have this simple method to check if the given string is a palindrome or not.
Here is my test
[Test]
public void IsPalindrome3Test()
{
var sourceString = "civic";
var result = Program.IsPalindrome3(sourceString);
Assert.AreEqual(true, result);
}
and the function
public static bool IsPalindrome3(string source)
{
int min = 0;
int max = source.Length - 1;
while(true)
{
if(min > max)
{
return true;
}
char a = source[min];
char b = source[max];
if(char.ToLower(a)!= char.ToLower(b))
{
return false;
}
min++;
max--;
}
}
Am I right here when writing the test ? Please let me know if the approach taken is right. Any pointers for that matter would be great !!
This isn't really TDD you're talking about. This is just a unit test. TDD refers specifically to the process of writing your tests before your code. You start out with the most trivial case, see the test fail, make it pass in the simplest way possible and then you impose some new assumptions by writing more tests. The point is that as your tests become more specific and cover more edge cases, the code becomes more generic.
There are many ways to do this and people prefer different levels of granularity. One version would be something like:
// Single character is always a palindrome
Assert.True(IsPalindrome("a"));
Which would prompt us to write the simplest possible code to make this pass
bool IsPalindrome(string input)
{
return true;
}
This code isn't "correct" though (although it's correct for all things we are testing for at the moment). We need more tests!
// Two non-equal characters are not a palindrome
Assert.False(IsPalindrome("ab"));
leading to
bool IsPalindrome(string input)
{
return input.Length == 1;
}
And so forth. Stepping through the whole process of implementing the full algorithm takes too long for an SO answer, I just want to show that it's an iterive process with short feedback loops where you constantly impose stronger and stronger assertions about how the code should work, and then you let the algorithm grow. There are plenty of videos on youtube about this, and books and blogs as well. Go check them out!
Last but not least, it's also important that we when our tests are passing make sure to "clean up" the code too. Making the code pass in the simplest way possible often leads to some ugly repetitions and such. When tests are passing we can refactor this while staying confident that our code still holds up to the assertions we made. It's important not to add more functionality when refactoring though, because then that functionality isn't written test first, which is the whole point of the endeavour.
Say I'm trying to test a simple Set class
public IntSet : IEnumerable<int>
{
Add(int i) {...}
//IEnumerable implementation...
}
And suppose I'm trying to test that no duplicate values can exist in the set. My first option is to insert some sample data into the set, and test for duplicates using my knowledge of the data I used, for example:
//OPTION 1
void InsertDuplicateValues_OnlyOneInstancePerValueShouldBeInTheSet()
{
var set = new IntSet();
//3 will be added 3 times
var values = new List<int> {1, 2, 3, 3, 3, 4, 5};
foreach (int i in values)
set.Add(i);
//I know 3 is the only candidate to appear multiple times
int counter = 0;
foreach (int i in set)
if (i == 3) counter++;
Assert.AreEqual(1, counter);
}
My second option is to test for my condition generically:
//OPTION 2
void InsertDuplicateValues_OnlyOneInstancePerValueShouldBeInTheSet()
{
var set = new IntSet();
//The following could even be a list of random numbers with a duplicate
var values = new List<int> { 1, 2, 3, 3, 3, 4, 5};
foreach (int i in values)
set.Add(i);
//I am not using my prior knowledge of the sample data
//the following line would work for any data
CollectionAssert.AreEquivalent(new HashSet<int>(values), set);
}
Of course, in this example, I conveniently have a set implementation to check against, as well as code to compare collections (CollectionAssert). But what if I didn't have either ? This code would be definitely more complicated than that of the previous option! And this is the situation when you are testing your real life custom business logic.
Granted, testing for expected conditions generically covers more cases - but it becomes very similar to implementing the logic again (which is both tedious and useless - you can't use the same code to check itself!). Basically I'm asking whether my tests should look like "insert 1, 2, 3 then check something about 3" or "insert 1, 2, 3 and check for something in general"
EDIT - To help me understand, please state in your answer if you prefer OPTION 1 or OPTION 2 (or neither, or that it depends on the case, etc). Just to clarify, it's pretty clear that in this case (IntSet), option 2 is better in all aspects. However, my question pertains to the cases where you don't have an alternative implementation to check against, so the code in option 2 would be definitely more complicated than option 1.
I usually prefer to test use cases one by one - this works nicely the TDD manner: "code a little, test a little". Of course, after a while my test cases start to contain duplicated code, so I refactor. The actual method of verifying the results does not matter to me as long as it is working for sure, and doesn't get into the way of testing itself. So if there is a "reference implementation" to test against, it is all the better.
An important thing, however, is that the tests should be reproducable and it should be clear what each test method is actually testing. To me, inserting random values into a collection is neither - of course if there is a huge amount of data/use cases involved, every tool or approach is welcome which helps to handle the situation better without lulling me into a false sense of security.
If you have an alternative implementation, then definitely use it.
In some situations, you can avoid reimplementing an alternative implementation, but still test the functionality in general. For instance, in your example, you could first generate a set of unique values, and then randomly duplicate elements before passing it to your implementation. You can test that the output is equivalent to your starting vector, without having to reimplement the sort.
I try to take this approach whenever it's feasible.
Update: Essentially, I'm advocating the OP's "Option #2". With this approach, there's precisely one output vector that will allow the test to pass. With "Option #1", there's an infinite number of acceptable output vectors (it's testing an invariant, but it's not testing for any relationship to the input data).
Basically I'm asking whether my tests
should look like "insert 1, 2, 3 then
check something about 3" or "insert 1,
2, 3 and check for something in
general"
I am not a TDD purist but it seems people are saying that the test should break if the condition that you are trying to test is broken. e.i. if you implement a test which checks a general condition, then your test will break in more than a few cases so it is not optimal.
If I am testing for not being able to add duplicates, then I would only test that. So in this case, I would say I would go with first.
(Update)
OK, now you have updated the code and I need to update my answer.
Which one would I choose? It depends on the implementation of CollectionAssert.AreEquivalent(new HashSet<int>(values), set);. For example, IEnumerable<T> does keep the order while HashSet<T> does not so even this could break the test while it should not. For me first is still superior.
According to xUnit Test Patterns, it's usually more favorable to test the state of the system under test. If you want to test its behavior and the way in which the algorithm operates, you can use Mock Object Testing.
That being said, both of your tests are known as Data Driven Tests. What is usually acceptable is to use as much knowledge as the API provides. Remember, those tests also serve as documentation for your software. Therefore it's critical to keep them as simple as possible - whatever that means for your specific case.
The first step should be demonstrating the correctness of the Add method using an activity diagram/flowchart. The next step would be to formally prove the correctness of the Add method if you have the time. Then testing with a specific sets of data where you expect duplications and non-duplications (i.e. some sets of data have duplications and some sets don't and you are seeing if the data structure performs correctly - it's important to have cases that should succeed (no duplicates) and to check that they were added correctly to the set rather than just testing for failure cases (cases in which duplicates should be found)). And finally checking generically. Even though it is now somewhat deprecated I would suggest constructing data to fully exercise every execution path in the method being tested. At any point you made a code change then begin all over applying regression testing.
I would opt for the algorithmic approach, but preferably without relying on an alternate implementation such as HashSet. You're actually testing for more than just "no duplicates" with the HashSet match. For example, the test will fail if any items didn't make it into the resulting set, and you presumably have other tests that check for this.
A cleaner verification of the "no duplicates" expectation might be something like the following:
Assert.AreEqual(values.Distinct().Count(), set.Count());
NUnit and MbUnit has a RowTest attribute that allows you to sent different set of parameters into a single test.
[RowTest]
[Row(5, 10, 15)]
[Row(3.5, 2.7, 6.2)]
[Row(-5, 6, 1)]
public void AddTest(double firstNumber, double secondNumber, double result)
{
Assert.AreEqual(result, firstNumber + secondNumber);
}
I used to be huge fan of this feature. I used it everywhere. However, lately I'm not sure if it's a very good idea to use RowTest in Unit Tests. Here are more reasons:
A unit test must be very simple. If there's a bug, you don't want to spent a lot of time to figure out what your test tests. When you use multiple rows, each row has different sent set of parameter and tests something different.
Also I'm using TestDriven.NET, that allows me to run my unit tests from my IDE, Visual Studio. With TestDrivent.NET I cannot instruct to run a specific row, it will execute all the rows. Therefore, when I debug I have to comment out all other rows and leave only the one I'm working with.
Here's an example how would write my tests today:
[Test]
public void Add_with_positive_whole_numbers()
{
Assert.AreEqual(5, 10 + 15);
}
[Test]
public void Add_with_one_decimal_number()
{
Assert.AreEqual(6.2, 3.5 + 2.7);
}
[Test]
public void Add_with_negative_number()
{
Assert.AreEqual(1, -5 + 6);
}
Saying that I still occasionally use RowTest attribute but only when I believe that it's not going to slow me down when I need to work on this later.
Do you think it's a good idea to use this feature in a Unit test?
Yes. It's basically executing the same test over and over again with different inputs... saving you the trouble of repeating yourself for each distinct input combination.
Thus upholding the 'once and only once' or DRY principle. So if you need to update this test you just update one test (vs multiple) tests.
Each Row should be a representative input from a distinct set - i.e. this input is different from all others w.r.t. this function's behavior.
The RowTest actually was a much-asked for feature for NUnit - having originated from MBUnit... I think Schlapsi wrote it as a NUnit extension which then got promoted to std distribution status. The NUnit GUI also groups all RowTests under one node in the GUI and shows which input failed/passed.. which is cool.
The minor disadvantage of the 'need to debug' is something I personally can live with.. It's after all commenting out a number of Row attributes temporarily (First of all most of the time I can eyeball the function once I find ScenarioX failed and solve it without needing a step-through) or conversely just copy the test out and pass it fixed (problematic) inputs temporarily
I don't think that this is specific to a language or framework, but I am using xUnit.net and C#.
I have a function that returns a random date in a certain range. I pass in a date, and the returning date is always in range of 1 to 40 years before the given date.
Now I just wonder if there is a good way to unit test this. The best approach seems to be to create a loop and let the function run i.e. 100 times and assert that every of these 100 results are in the desired range, which is my current approach.
I also realize that unless I am able to control my Random generator, there will not be a perfect solution (after all, the result IS random), but I wonder what approaches you take when you have to test functionality that returns a random result in a certain range?
Mock or fake out the random number generator
Do something like this... I didn't compile it so there might be a few syntax errors.
public interface IRandomGenerator
{
double Generate(double max);
}
public class SomethingThatUsesRandom
{
private readonly IRandomGenerator _generator;
private class DefaultRandom : IRandomGenerator
{
public double Generate(double max)
{
return (new Random()).Next(max);
}
}
public SomethingThatUsesRandom(IRandomGenerator generator)
{
_generator = generator;
}
public SomethingThatUsesRandom() : this(new DefaultRandom())
{}
public double MethodThatUsesRandom()
{
return _generator.Generate(40.0);
}
}
In your test, just fake or mock out the IRandomGenerator to return something canned.
In addition to testing that the function returns a date in the desired range, you want to ensure that the result is well-distributed. The test you describe would pass a function that simply returned the date you sent in!
So in addition to calling the function multiple times and testing that the result stays in the desired range, I would also try to assess the distribution, perhaps by putting the results in buckets and checking that the buckets have roughly equal numbers of results after you are done. You may need more than 100 calls to get stable results, but this doesn't sound like an expensive (run-time wise) function, so you can easily run it for a few K iterations.
I've had a problem before with non-uniform "random" functions.. they can be a real pain, it's worth testing for early.
I think there are three different aspects of this problem that you test.
The first one: is my algorithm the right one? That is, given a properly-functioning random-number generator, will it produce dates that are randomly distributed across the range?
The second one: does the algorithm handle edge cases properly? That is, when the random number generator produces the highest or lowest allowable values, does anything break?
The third one: is my implementation of the algorithm working? That is, given a known list of pseudo-random inputs, is it producing the expected list of pseudo-random dates?
The first two things aren't something I'd build into the unit-testing suite. They're something I'd prove out while designing the system. I'd probably do this by writing a test harness that generated a zillion dates and performed a chi-square test, as daniel.rikowski suggested. I'd also make sure this test harness didn't terminate until it handled both of the edge cases (assuming that my range of random numbers is small enough that I can get away with this). And I'd document this, so that anyone coming along and trying to improve the algorithm would know that that's a breaking change.
The last one is something I'd make a unit test for. I need to know that nothing has crept into the code that breaks its implementation of this algorithm. The first sign I'll get when that happens is that the test will fail. Then I'll go back to the code and find out that someone else thought that they were fixing something and broke it instead. If someone did fix the algorithm, it'd be on them to fix this test too.
You don't need to control the system to make the results deterministic. You're on the right approach: decide what is important about the output of the function and test for that. In this case, it is important that the result be in a range of 40 days, and you are testing for that. It's also important that it not always return the same result, so test for that too. If you want to be fancier, you can test that the results pass some kind of randomness test..
Normaly I use exactly your suggested approach: Control the Random generator.
Initialize it for test with a default seed (or replace him by a proxy returning numbers which fit my testcases), so I have deterministic/testable behaviour.
If you want to check the quality of the random numbers (in terms of independance) there are several ways to do this. One good way is the Chi square test.
Sure, using a fixed seed random number generator will work just fine, but even then you're simply trying to test for that which you cannot predict. Which is ok. It's equivalent to having a bunch of fixed tests. However, remember--test what is important, but don't try to test everything. I believe random tests are a way to try to test everything, and it's not efficient (or fast). You could potentially have to run a great many randomized tests before hitting a bug.
What I'm trying to get at here is that you should simply write a test for each bug you find in your system. You test out edge cases to make sure your function is running even in the extreme conditions, but really that's the best you can do without either spending too much time or making the unit tests slow to run, or simply wasting processor cycles.
Depending on how your function creates the random date, you may also want to check for illegal dates: impossible leap years, or the 31st day of a 30-day month.
Methods that do not exhibit a deterministic behavior cannot be properly unit-tested,as the results will differ from one execution to another. One way to get around this is to seed the random number generator with a fixed value for the unit test. You can also extract the randomness of the date generation class (and thus applying the Single Responsibility Principle), and inject known values for the unit-tests.
I would recommend overriding the random function. I am unit testing in PHP so I write this code:
// If we are unit testing, then...
if (defined('UNIT_TESTING') && UNIT_TESTING)
{
// ...make our my_rand() function deterministic to aid testing.
function my_rand($min, $max)
{
return $GLOBALS['random_table'][$min][$max];
}
}
else
{
// ...else make our my_rand() function truly random.
function my_rand($min = 0, $max = PHP_INT_MAX)
{
if ($max === PHP_INT_MAX)
{
$max = getrandmax();
}
return rand($min, $max);
}
}
I then set the random_table as I require it per test.
Testing the true randomness of a random function is a separate test altogether. I would avoid testing the randomness in unit tests and would instead do separate tests and google the true randomness of the random function in the programming language you are using. Non-deterministic tests (if any at all) should be left out of unit tests. Maybe have a separate suite for those tests, that requires human input or much longer running times to minimise the chances of a fail that is really a pass.
I don't think Unit testing is meant for this. You can use Unit testing for functions that return a stochastic value, but use a fixed seed, in which case in a way they are not stochastic, so to speak,
for random seed, I dont think Unit testing is what you want, for example for RNGs what you mean to have is a system test, in which you run the RNG many many times and look at the distribution or moments of it.