I'm creating a TestClass for unit testing an application, and one thing I'd like to do is run a test method, check method has run correctly, store a value in class property within test class base on outcome, and then use that value in a later method.
I've tried doing this and found that as soon as the compiler moves from one method to another, all the properties I have set are wiped clean. I have checked with breakpoints and at the end of the first method the value is in the property, and then at the beginning of the second method that same property is null.
Looked this up and no one else seems to be attempting the same thing, so is it possible to share a value between methods or am I taking the wrong approach?
Thanks in advance.
You are taking the wrong approach.
Unit tests, by definition, should be completely self-contained and deterministic. They should not depend on one another.
You should be able to refactor out the repetitive portion of your first unit test into a helper method which can be invoked by your other unit test. The work will be done twice, but Unit Tests should be really fast, so the overhead should be really minimal.
It's not the compiler - it's the test runner which will (potentially) create a new instance for each test.
Tests should generally be independent - even if you could find a way of getting this to work, I would avoid doing so. Design your way around it as best you can.
This smells like bad practice to me, no matter what testing framework you're using. All automated tests (let alone formal unit tests) should be independent of one another. A static field/property might work, but I would recommend refactoring your tests first.
Related
When writing unit test, Is there a simple way to ensure that nothing unexpected happened ?
Since the list of possible side effect is infinite, adding tons of Assert to ensure that nothing changed at every steps seems vain and it obfuscate the purpose of the test.
I might have missed some framework feature or good practice.
I'm using C#7, .net 4.6, MSTest V1.
edit:
The simpler example would be to test the setter of a viewmodel, 2 things should happen: the value should change and PropertyChanged event should be raised.
These 2 things are easy to check but now I need to make sure that other properties values didn't changed, no other event was raised, the system clipboard was not touched...
You're missing the point of unit tests. They are "proofs". You cannot logically prove a negative assertion, so there's no point in even trying.
The assertions in each unit test should prove that the desired behavior was accomplished. That's all.
If we reduce the question to absurdity, every unit test would require that we assert that the function under test didn't start a thermonuclear war.
Unit tests are not the only kind of tests you'll need to perform. There are functional tests, integration tests, usability tests, etc. Each one has its own focus. For unit tests, the focus is proving the expected behavior of a single function. So if the function is supposed to accomplish 2 things, just assert that each of those 2 things happened, and move on.
One of the options to ensure that nothing 'bad' or unexpected happens is to ensure good practices of using dependency injection and mocking:
[Test]
public void TestSomething()
{
// Arrange
var barMock = RhinoMocks.MockRepository.GenerateStrictMock<IBar>();
var foo = new Foo(barMock);
// Act
foo.DoSomething();
// Assert
...
}
In the example above if Foo accidentally touches Bar, that will result in an exception (the strict mock) and the test fails. Such approach might not be applicable in all test cases, but serves as a good addition to other potential practices.
Some addition to your edit:
In Test Driven Development you are writing only code, which will pass the test and nothing more. Furthermore you want to choose the simplest possible solutoin to accomplish this goal.
That said you will start most likely with a failing unit-test. In your situation you will not get a failing unit test at the beginning.
If you push it to the limits, you will have to check that format C:\ is not called in your application when you want to check every outcome. You might want to have a look at design principles like the KISS-principle (Keep it simple, stupid).
If the scope of "check that nothing else happened" is to ensure the state of the model didn't change, which it appears is the case from the question.
Write a helper function that takes the model before your event and the model after and compares them. Let it return the properties that are changed, then you can assert that only those properties that you intended to update are in the return list. This sort of helper is portable, maintainable, and reusable
Checking model state is a valid application of a unit test.
This is only possible in referentially transparent languages such as Safe Haskell.
I'm writing unit tests for the implementation of an API I wrote myself in my company's application. Still new to this whole thing. When looking for answeres on how to unit test certain things I come across a certain pattern. It goes something like this:
Question:
I have this private method I need to unit test.
Top voted answer:
Don't.
I also came across this article arguing against unit testing private methods as well.
Basically how I'm implementing an API I'm given is I write the code first, then I write unit tests to "break it the worst way possible" (as my superior puts it). Once I notice something broke I fix it in the code. To me this seems like a mash-up of OOD and TDD. Is that a legit approach?
The reason I got so many private methods in the first place is that I'm required to break up larger chunks of code into methods. Since these methods are only supposed to be used within the scope of this API implementation I set them to private. Since the file structure established by my team requires me to write all the code into a single file corresponding to an API I can't separate these private methods into a new class and set them to public.
My superior expects me to test these private methods as well. But I'm beginning to doubt if this is even really necessary if the Asserts on the public methods all run successfully?
From my point of view, if my tests on the public methods return the values I expected, I infer that my private methods also work like I intended.
Or am I missing something?
The core point is: unit tests exist to guarantee that your class under tests behaves as expected.
The behavior of your classes manifests itself via those methods that can be called from "outside" of your classes.
Therefore there is neither need nor sense in trying to directly test private methods.
Of course, it is fair to measure coverage while running unit tests; in order to understand which paths in your code are taken. This information can be used to either enhance test cases (to gain more coverage); or to delete production code (which isn't required).
And to align with your question: you do not use TDD to implement private methods.
You use TDD to create a special form of your "contract" that can be executed automatically. You verify what needs to be done; not how it is actually done in detail. That is especially true since the TDD methodology includes continuous refactoring. You write your tests, you turn them green (by writing production code); and then, at some point, you look into improving the quality of your code. Meaning: you start reworking internal aspects of your class under test. Like: creating more private methods, moving content around; maybe even creating internal-only helper classes and so on. But you keep running your existing tests ... which should still all work; because as said: you write them to check the externally observable behavior (as far as possible).
And beyond that: you should rather looking into "fuzzying" the test data that your unit tests drive into your code instead of worrying about private methods.
What I mean: instead of trying to manually find that test data that makes your production code break, look into concepts like QuickCheck that try to do exactly that automatically.
Final words: and if your management keeps hammering on "test private methods"; then it is your responsibility as engineer to convince them that they are wrong about this. And there is plenty of material out there to back that up.
The way you are splitting your code at the moment is out of necessity. You are delegating some work in a private method, because, well, other public methods need to re-use this, and you don't want to copy-paste that code. Of course, since these methods don't make sense being used as standalone methods, you keep them private.
Good, at least you're true to the DRY (Don't Repeat Yourself) principle.
Now, another way to look it is that you want to separate your private methods from the rest of the code, because you want to have a Separation of Concerns. If you do this, you will see that these private methods, although they can't be used on their own, don't really belong to the class containing your public methods, because they don't solve the same concern : This is the Single Responsibility principle: the S in SOLID.
Instead of having your private method within your class, what you can do is move it to another class (a service as I call them), inject it in the class in which they were before, and call these methods instead of the call to the private ones.
Why should you do this ?
Because it will be so much easier to test: you delegate a big part of the code, that you will not have to test under a big combination of scenarios.
Because you can then inject an alternative implementation (think maintainability: it's easier to replace a brick, than a part of a brick)
Because you can delegate the implementation (and the testing) of this service to someone else (you can have 2 developers in parallel working on a very small area of the code)
Sometimes, it makes even more sense, because these service classes will then be re-used by other completely different classes that will have the same needs, if they really take care of one single concern.
This last point doesn't always happen, but quite often, it does. I found it is easier to re-use existing data services when they are self-documented: properly-named services and properly-named methods. (your co-workers will discover them more easily)
Now, you don't need to test a private method... because it's public.
You may think it's cheating, because you just made it public, but this comes from a very legitimate approach: Separation of Concerns.
Final notes:
I am convinced your superior is right about asking you to test this code. One thing he could have added was to do that separation into different classes. Also, make sure that you inject these classes using Dependency Injection and Inversion of Control containers. don't instantiate them using the new statement, otherwise, you will not be able to assert that the right method was called with the right arguments !
I'm using C# although this question would probably apply to any language.
I have two methods A and B.
A does some things and has 5 unit tests for it.
B is basically the same as A, but does something else too. B calls A in its implementation.
How do I go about testing A and B? Copying and pasting A's tests seems like a lot of unnecessary repetition.
If A belonged to another class, we could just mock that class and verify that A was called, but seeing as A is part of the same class as B, we can't do that here.
What are the best practices in such a case?
I think Oliver has the right idea but I'll take a shot at explaining it, since it seems like there's still some confusion.
You start by writing a set of tests for Method A. Then when you go to write a set of tests for B, you should only be testing functionality in B. Meaning when you write your tests for B you should assume that A works perfectly, because you already tested it. And I wouldn't worry about mocking A in this case.
As a side note, I have a suspicion that A does not need to be public. Is it called from outside the class at all? If not, I would make it private and then write one set of tests, testing A through B.
Or if A is complex enough that you want to test it separately, then it probably warrants being broken out into its own helper class. Then you can mock it if you want as well.
If you could give some example code it would make it easier to help you too.
So simply write only the tests for what you expect what method B should does. If it internally calls another private or public method or maybe achieves the same through internal code clones (or whatever) doesn't matter to you.
Only test one thing per test. If you have all your tests for A and all your tests for B and you make any breaking changes you'll get informed from your test environment.
I recently joined a company where they use TDD, but it's still not clear for me, for example, we have a test:
[TestMethod]
public void ShouldReturnGameIsRunning()
{
var game = new Mock<IGame>();
game.Setup(g => g.IsRunning).Returns(true);
Assert.IsTrue(game.Object.IsRunning);
}
What's the purpose of it? As far as I understand, this is testing nothing! I say that because it's mocking an interface, and saying, return true for IsRunning, it will never return a different value...
I'm probably thinking wrong, as everybody says it's a good practice etc etc... or is this test wrong?
This test is excerising an interface, and verifying that the interface exposes a Boolean getter named IsRunning.
This test was probably written before the interface existed, and probably well before any concrete classes of IGame existed either. I would imagine, if the project is of any maturity, there are other tests that exercise the actual implementions and verify behavior for that level, and probably use mocking in the more traditional isolationist context rather than stubbing theoretical shapes.
The value of these kinds of tests is that it forces one to think about how a shape or object is going to be used. Something along the lines of "I don't know exactly what a game is going to do yet, but I do know that I'm going to want to check if it's running...". So this style of testing is more to drive design decisions, as well as validate behavior.
Not the most valuable test on the planet, but serves its purpose in my opinion.
Edit: I was on the train last night when I was typing this, but I wanted to address Andrew Whitaker comment directly in the answer.
One could argue that the signature in the interface itself is enough of an enforcement of the desired contract. However, this really depends on how you treat your interfaces and ultimately how you validate the system that you are trying to test.
There may be tangible value in explicitly stating this functionality as a test. You are explicitly verifying that this is a desired functionality for an IGame.
Lets take a use case, lets say that as a designer you know that you want to expose a public getter for IsRunning, so you add it to the interface. But lets say that another developer gets this and sees a property, but doesn't see any usages of it anywhere else in the code, one might assume that it is eligible for deletion (or code-rot removal, which is normally a good thing) without harm. The existence of this test, however, explicitly states this properity should exist.
Without this test, a developer could modify the interface, drop the implementation and the project would still compile and run seemingly correctly. And it would be until much later that someone might notice that interface has been changed.
With this test, even if it appears as if no one is using it, your test suite will fail. The self documenting nature of the test tells you that ShouldReturnGameIsRunning(), which at the very least should spawn a conversation if IGame should really expose an IsRunning getter.
The test is invalid. Moq is a framework used to "mock" objects used by the method under test. If you were testing IsRunning, you might use Mock to provide a certain implementation for those methods that IsRunning depends on, for example to cause certain execution paths to execute.
People can tell you all day that testing first is better; you won't realize that it actually is until you start doing it yourself and realize that you have fewer bugs with code you wrote tests for first, which will save you and your co-workers time and probably make the quality of your code higher.
It could be that the test has been written (before) the code which is good practice.
It might equally be that this is a pointless test created by a tool.
Another possibility is that the code was written by somebody who, at the time, didn't understand how to use that particular mocking framework - and wrote one or more simple tests to learn. Kent Beck talked about that sort of "learning test" in his original TDD book.
This test could have had a point in the past. It could have been testing a concrete class. It could have been testing an abstract one -- as it sometimes makes sense to mock an abstract class that you want to test. I can see how a (less informed(like myself)) follower of the TDD way of doing things could have left a small mess like this. Cleanup of dead code and dead test code, does not really fit in the natural workflow of red, green, refactor (since it doesn't break any tests!).
However, history is not justification for a test that does nothing but confuse the reader. So, be a good boyscout and remove it.
In my honest oppinion this is just rubbish, it only proves that moq works as expected
There are some other variations of this question here at SO, but please read the entire question.
By just using fakes, we look at the constructor to see what kind of dependencies that a class have and then create fakes for them accordingly.
Then we write a test for a method by just looking at it's contract (method signature). If we can't figure out how to test the method by doing so, shouldn't we rather try to refactor the method (most likely break it up in smaller pieces) than to look inside it to figure our how we should test it? In other words, it also gives us a quality control by doing so.
Isn't mocks a bad thing since they require us to look inside the method that we are going to test? And therefore skip the whole "look at the signature as a critic".
Update to answer the comment
Say a stub then (just a dummy class providing the requested objects).
A framework like Moq makes sure that Method A gets called with the arguments X and Y. And to be able to setup those checks, one needs to look inside the tested method.
Isn't the important thing (the method contract) forgotten when setting up all those checks, as the focus is shifted from the method signature/contract to look inside the method and create the checks.
Isn't it better to try to test the method by just looking at the contract? After all, when we use the method we'll just look at the contract when using it. So it's quite important the it's contract is easy to follow and understand.
This is a bit of a grey area and I think that there is some overlap. On the whole I would say using mock objects is preferred by me.
I guess some of it depends on how you go about testing code - test or code first?
If you follow a test driven design plan with objects implementing interfaces then you effectively produce a mock object as you go.
Each test treats the tested object / method as a black box.
It focuses you onto writing simpler method code in that you know what answer you want.
But above all else it allows you to have runtime code that uses mock objects for unwritten areas of the code.
On the macro level it also allows for major areas of the code to be switched at runtime to use mock objects e.g. a mock data access layer rather than one with actual database access.
Fakes are just stupid dummy objects. Mocks enable you to verify that the controlflow of the unit is correct (e.g. that it calls the correct functions with the expected arguments). Doing so is very often a good way to test things. An example is that a saveProject()-function probably want's to call something like saveToProject() on the objects to be saved. I consider doing this a lot better than saving the project to a temporary buffer, then loading it to verify that everything was fine (this tests more than it should - it also verifies that the saveToProject() implementation(s) are correct).
As of mocks vs stubs, I usually (not always) find that mocks provide clearer tests and (optionally) more fine-grained control over the expectations. Mocks can be too powerful though, allowing you to test an implementation to the level that changing implementation under test leaving the result unchanged, but the test failing.
By just looking on method/function signature you can test only the output, providing some input (stubs that are only able to feed you with needed data). While this is ok in some cases, sometimes you do need to test what's happening inside that method, you need to test whether it behaves correctly.
string readDoc(name, fileManager) { return fileManager.Read(name).ToString() }
You can directly test returned value here, so stub works just fine.
void saveDoc(doc, fileManager) { fileManager.Save(doc) }
here you would much rather like to test, whether method Save got called with proper arguments (doc). The doc content is not changing, the fileManager is not outputting anything. This is because the method that is tested depends on some other functionality provided by the interface. And, the interface is the contract, so you not only want to test whether your method gives correct results. You also test whether it uses provided contract in correct way.
I see it a little different. Let me explain my view:
I use a mocking framework. When I try to test a class, to ensure it will work as intended, I have to test all the situations may happening. When my class under test uses other classes, I have to ensure in certain test situation that a special exceptions is raised by a used class or a certain return value, and so on... This is hardly to simulate with the real implementations of those classes, so I have to write fakes of them. But I think that in the case I use fakes, tests are not so easy to understand. In my tests I use MOQ-Framework and have the setup for the mocks in my test method. In case I have to analyse my testmethod, I can easy see how the mocks are configured and have not to switch to the coding of the fakes to understand the test.
Hope that helps you finding your answer ...