We Keep Coding

When would I actually be required to unit test private methods?

I'm writing unit tests for the implementation of an API I wrote myself in my company's application. Still new to this whole thing. When looking for answeres on how to unit test certain things I come across a certain pattern. It goes something like this:
Question:
I have this private method I need to unit test.
Top voted answer:
Don't.
I also came across this article arguing against unit testing private methods as well.
Basically how I'm implementing an API I'm given is I write the code first, then I write unit tests to "break it the worst way possible" (as my superior puts it). Once I notice something broke I fix it in the code. To me this seems like a mash-up of OOD and TDD. Is that a legit approach?
The reason I got so many private methods in the first place is that I'm required to break up larger chunks of code into methods. Since these methods are only supposed to be used within the scope of this API implementation I set them to private. Since the file structure established by my team requires me to write all the code into a single file corresponding to an API I can't separate these private methods into a new class and set them to public.
My superior expects me to test these private methods as well. But I'm beginning to doubt if this is even really necessary if the Asserts on the public methods all run successfully?
From my point of view, if my tests on the public methods return the values I expected, I infer that my private methods also work like I intended.
Or am I missing something?

The core point is: unit tests exist to guarantee that your class under tests behaves as expected.
The behavior of your classes manifests itself via those methods that can be called from "outside" of your classes.
Therefore there is neither need nor sense in trying to directly test private methods.
Of course, it is fair to measure coverage while running unit tests; in order to understand which paths in your code are taken. This information can be used to either enhance test cases (to gain more coverage); or to delete production code (which isn't required).
And to align with your question: you do not use TDD to implement private methods.
You use TDD to create a special form of your "contract" that can be executed automatically. You verify what needs to be done; not how it is actually done in detail. That is especially true since the TDD methodology includes continuous refactoring. You write your tests, you turn them green (by writing production code); and then, at some point, you look into improving the quality of your code. Meaning: you start reworking internal aspects of your class under test. Like: creating more private methods, moving content around; maybe even creating internal-only helper classes and so on. But you keep running your existing tests ... which should still all work; because as said: you write them to check the externally observable behavior (as far as possible).
And beyond that: you should rather looking into "fuzzying" the test data that your unit tests drive into your code instead of worrying about private methods.
What I mean: instead of trying to manually find that test data that makes your production code break, look into concepts like QuickCheck that try to do exactly that automatically.
Final words: and if your management keeps hammering on "test private methods"; then it is your responsibility as engineer to convince them that they are wrong about this. And there is plenty of material out there to back that up.

The way you are splitting your code at the moment is out of necessity. You are delegating some work in a private method, because, well, other public methods need to re-use this, and you don't want to copy-paste that code. Of course, since these methods don't make sense being used as standalone methods, you keep them private.
Good, at least you're true to the DRY (Don't Repeat Yourself) principle.
Now, another way to look it is that you want to separate your private methods from the rest of the code, because you want to have a Separation of Concerns. If you do this, you will see that these private methods, although they can't be used on their own, don't really belong to the class containing your public methods, because they don't solve the same concern : This is the Single Responsibility principle: the S in SOLID.
Instead of having your private method within your class, what you can do is move it to another class (a service as I call them), inject it in the class in which they were before, and call these methods instead of the call to the private ones.
Why should you do this ?
Because it will be so much easier to test: you delegate a big part of the code, that you will not have to test under a big combination of scenarios.
Because you can then inject an alternative implementation (think maintainability: it's easier to replace a brick, than a part of a brick)
Because you can delegate the implementation (and the testing) of this service to someone else (you can have 2 developers in parallel working on a very small area of the code)
Sometimes, it makes even more sense, because these service classes will then be re-used by other completely different classes that will have the same needs, if they really take care of one single concern.
This last point doesn't always happen, but quite often, it does. I found it is easier to re-use existing data services when they are self-documented: properly-named services and properly-named methods. (your co-workers will discover them more easily)
Now, you don't need to test a private method... because it's public.
You may think it's cheating, because you just made it public, but this comes from a very legitimate approach: Separation of Concerns.
Final notes:
I am convinced your superior is right about asking you to test this code. One thing he could have added was to do that separation into different classes. Also, make sure that you inject these classes using Dependency Injection and Inversion of Control containers. don't instantiate them using the new statement, otherwise, you will not be able to assert that the right method was called with the right arguments !

Test strategy advice - need to record verified results of running a method and use them for testing purposes

I'm new to testing and I need an advice for the best testing strategy (and its application).
Here's my problem:
I have a program that reads a file and automatically extracts its structure. I want to test the method that makes this "intelligent" extraction. Initially I can use a few files to check if the method is doing the correct extraction. Then I want to use these files and the (correct) extraction results for testing purposes. As the extraction results have been verified they should (and must) be used for further tests.
So, I have something like: for "this particular file" I expect "this result".
The problems:
It's easy to get the input files for the test. I will store them in a specific directory. What about the results? They affect the contents of the object that stores the file structure. In this case I may need to save this object in a file as well. With serialization I'm afraid that with object's structure changes, it will be difficult to reuse the previous saved objects.
With more and more results I may have hundreds of files and results and the test will take a lot of time. I expect that testing time will not be a big issue.
I need testing because my "extraction algorithm" used in the method will change often. I can't cope with all possibilities in order to have a perfect extraction algorithm. So, my solution is to build an initial algorithm that works in a dozen files and each time that I find a failure in the algorithm for a particular file I change the algorithm in order to solve the problem with that file. This change should be tested in order the previous files and results are still valid.
Any suggestion for the testing strategy?

For tests you need somewhere a place where you can inject input test data and some place where you can observe some behavior or output.
On the input side: Are files really the only possibility to inject input test data? If yes, the application does not have a good testable design. Tests with files are hard to maintain. On the output side: The application does not seem to offer a possibility to observe behavior or output. This is points to a design which is not testable.
Even if you find a way to observe behavior our output, there will be only end-to-end tests for all the extraction algorithms. Such end-to-end tests are brittle and a maintenance nightmare. The cause is a not good testable design.
You will be not able to implement a good test strategy without a good testable design. You will need to change the design of your application. On the other hand you may argue that you do not want to change the design without any tests in place. It seems to be a chicken-and-egg problem.
How to get out of such a situation? A combination of test and refactoring strategies might help. On a high-level this might work like this:
Build a few representative end-to-end tests. Therefore even use the
serialization trick. This is just to verify that your program works
like before you start refactoring. They act as migration tests.
Refactor you program. Give it places to inject and observe. Such
places are known as seams.
As a result you will have testable chunks, which you can put into a
test harness.
You refactor and put new seams into the code, to test smaller
chunks, until you get to the point that you have unit tests in
place. Ideally you will have all your algorithms encapsulated into a family of classes which are all unit tested.
Sounds like a hard job? No, in reality it is even harder than it sounds. Refactoring an application to a testable design needs a lot of experience. Luckily there is a guy who wrote a book about this: Michael Feather’s 'Working Effectively with Legacy Code'.
If you really, really want to implement a good test strategy for the existing application, then read that book. If you want to know what you can do better next time, read that book. If you think that unit testing might be the key to avoid not testable design then start learning about unit tests now. There are a lot of resources in the internet and books about unit testing.

If I understand the problem, you need to persist the results of a test for a later inspection or additional testing.
It has often been reluctant to invest too much time in writing test code, but in this case I see no alternative immediately available.
My advice is to decouple as much as possible the parts involved: the algorithm, the persistence layer (serialization/deserialization), its product and the verification code.
It is possible also that later algorithms implementations can share the same interface, e.g.:
interface IMyAlgorithm {
AbstractOutput DoSomething (InputData);
}
class ConcreteOutput : AbstractOutput {
// Output for version XXX of your algorithm
}
class XXXAlgorithm {
ConcreteOutput DoSomething (InputData inputData)
// Version XXX of you alogorithm
}
}
interface IPersistenceManager {
Serialize(AbstractOutput output, string filename);
AbstractOutput Deserialize(string filename)
}
class XXXPersistenceManager : IPersistenceManager {
// Handle persistence for XXX hierarchy
}
class XXXTestFixture {
void BuildObjectWithXXXAlgorithm() {
IMyAlgorithm XXX = new XXXAlgorithm();
// run XXX here
AbstractOutput objXXX = XXX.DoSomething(new InputData());
IPersistenceManager pmXXX = new XXXPersistenceManager();
pmXXX.Serialize(objXXX);
}
void VerifyThatXXXWorkAsExpected() {
IPersistenceManager pmXXX = new XXXPersistenceManager();
AbstractOutput objXXX = pmXXX.Deserialize(path);
// check object here
}
}
So when you need to create a new algorithm, say YYY, you create the corresponding hierarchy. Anyway I don't know details and this is just a draft of pseudocode, placed here only to put emphasis on loosely coupling application components.

You can try to use approvaltests to verify that
a given input file always generates the same in memory-object-graph.
To do this you need code that converts the in-memory-object-graph into a string representation of it (i.e overwriting ToString() or having an xml-serializer)
approvaltests verifies that the generated string is always the same.
if the string representation chages you will get a diff-viewer and you are asked to verify if the changes are still ok or not. If the changes are ok you will take this result for later verifications.

How can I automate testing for my database driven function?

I'm trying to improve the automated testing in my application, but am unsure of the best way to proceed.
My app gathers data from multiple forms, recodes it and stores it in a database. I have created a pretty complex SQL view, which flattens the structure out, so it can be imported into a stats package (SPSS).
My concern is that the view is complex, and I want to automate some tests around it.
Currently I have some functional tests, which create a complete form objects model, and sends it into the application. I then retrieve the view from the database, and use reflection to test that the retrieved view fields match the original data.
The problem is that this is very manual and heavy, my fixtures are lengthy, and it is cumbersome to add new scenarios (i.e. various parts of the model incomplete).
Does anyone have any advice on how I could improve my test strategy? Tips tricks all welcome!
Thanks!

DbFit is perfect for this. DbFit is an extension of FitNesse which maybe you are already using since you spoke of using "fixtures". In any case, DbFit makes it really easy to set up a test where you can seed some data, run the View, compare the expected results, and then it will automatically rollback the data that you just seeded for the test. And it is very easy to update as you add more fields to the View. AND it requires no additional objects in your DB like some other SQL "unit" testing suites.
You can find more info on using DbFit at:
http://benilovj.github.com/dbfit
http://groups.google.com/group/dbfit
And here is a tutorial that I wrote for it that explains the basic options:
http://www.sqlservercentral.com/articles/Testing/64636/

This is a very difficult question to answer. It almost sounds to me like You want to make a single test that tests all in one go.
First, Your app should be constructed, so each functionality is isolated in its own class, thereby making it easy to test AND easy to replace by stubs when testing other things. Dependencies on other functions should be injected (Dependency Injection).
Second, you should use the same technique for external systems like database connections and SPSS file writers. This involves wrapping such functionality so these dependencies also can be injected, and thus replaced by stubs when testing other aspects of your app.
Third, be aware that if tests are hard to write, 99,99% of the time this indicates that your design is not as strong as it could be.
Regards,
Morten

Writing standards for unit testing

I plan to introduce a set of standards for writing unit tests into my team. But what to include?
These two posts (Unit test naming best practices and Best practices for file system dependencies in unit/integration tests) have given me some food for thought already.
Other domains that should be covered in my standards should be how test classes are set up and how to organize them. For example if you have class called OrderLineProcessor there should be a test class called OrderLineProcessorTest. If there's a method called Process() on that class then there should be a test called ProcessTest (maybe more to test different states).
Any other things to include?
Does your company have standards for unit testing?
EDIT: I'm using Visual Studio Team System 2008 and I develop in C#.Net

Have a look at Michael Feathers on what is a unit test (or what makes unit tests bad unit tests)
Have a look at the idea of "Arrange, Act, Assert", i.e. the idea that a test does only three things, in a fixed order:
Arrange any input data and processing classes needed for the test
Perform the action under test
Test the results with one or more asserts. Yes, it can be more than one assert, so long as they all work to test the action that was performed.
Have a Look at Behaviour Driven Development for a way to align test cases with requirements.
Also, my opinion of standard documents today is that you shouldn't write them unless you have to - there are lots of resources available already written. Link to them rather than rehashing their content. Provide a reading list for developers who want to know more.

You should probably take a look at the "Pragmatic Unit Testing" series. This is the C# version but there is another for Java.
With respect to your spec, I would not go overboard. You have a very good start there - the naming conventions are very important. We also require that the directory structure match the original project. Coverage also needs to extend to boundary cases and illegal values (checking for exceptions). This is obvious but your spec is the place to write it down for that argument that you'll inevitably have in the future with the guy who doesn't want to test for someone passing an illegal value. But don't make the spec more than a few pages or no one will use it for a task that is so context-dependent.
Update: I disagree with Mr. Potato Head about only one assert per Unit Test. It sounds quite fine in theory but, in practice, it leads to either loads of mostly redundant tests or people doing tons of work in setup and tear-down that itself should be tested.

I follow the BDD style of TDD. See:
http://blog.daveastels.com/files/BDD_Intro.pdf
http://dannorth.net/introducing-bdd
http://behaviour-driven.org/Introduction
In short this means that
The tests are not thought as "tests", but as specifications of the system's behaviour (hereafter called "specs"). The intention of the specs is not to verify that the system works under every circumstance. Their intention is to specify the behaviour and to drive the design of the system.
The spec method names are written as full English sentences. For example the specs for a ball could include "the ball is round" and "when the ball hits a floor then it bounces".
There is no forced 1:1 relation between the production classes and the spec classes (and generating a test method for every production method would be insane). Instead there is a 1:1 relation between the behaviour of the system and the specs.
Some time ago I wrote TDD tutorial (where you begin writing a Tetris game using the provided tests) which shows this style of writing tests as specs. You can download it from http://www.orfjackal.net/tdd-tutorial/tdd-tutorial_2008-09-04.zip The instructions about how to do TDD/BDD are still missing from that tutorial, but the example code is ready, so you can see how the tests are organized and write code that passes them.
You will notice that in this tutorial the production classes are named such as Board, Block, Piece and Tetrominoe which are centered around the concepts of a Tetris game. But the test classes are centered around the behaviour of the Tetris game: FallingBlocksTest, RotatingPiecesOfBlocksTest, RotatingTetrominoesTest, FallingPiecesTest, MovingAFallingPieceTest, RotatingAFallingPieceTest etc.

Try to use as few assert statements per test method as possible. This makes sure that the purpose of the test is well-defined.
I know this will be controversial, but don't test the compiler - time spent testing Java Bean accessors and mutators is better spent writing other tests.
Try, where possible, to use TDD instead of writing your tests after your code.

I've found that most testing conventions can be enforced through the use of a standard base class for all your tests. Forcing the tester to override methods so that they all have the same name.
I also advocate the Arrange-Act-Assert (AAA) style of testing as you can then generate fairly useful documentation from your tests. It also forces you to consider what behaviour you are expecting due to the naming style.

Another item you can put in your standards is to try and keep your unit test size small. That is the actuall test methods themselves. Unless you are doing a full integration unit test there usually is no need for large unit tests, like say more than 100 lines. I'll give you that much in case you have a lot of setup to get to your one test. However if you do you should maybe refactor it.
People also talk about refactoring there code make sure people realize that unit tests is code too. So refactor, refactor, refactor.
I find the biggest problem in the uses I have seen is that people do not tend to recognize that you want to keep your unit tests light and agile. You don't want a monolithic beast for your tests after all. With that in mind if you have a method you are trying to test you should not test every possible path in one unit test. You should have multiple unit tests to account for every possible path through the method.
Yes if you are doing your unit tests correctly you should on average have more lines of unit test code than your application. While this sounds like a lot of work it will save you alot of time in the end when comes time for the inevitable business requirement change.

Users of full-featured IDE's will find that "some of them" have quite detailed support for creating tests in a specific pattern. Given this class:
public class MyService {
public String method1(){
return "";
}
public void method2(){
}
public void method3HasAlongName(){
}
}
When I press ctrl-shift-T in intellij IDEA I get this test class after answering 1 dialog box:
public class MyServiceTest {
#Test
public void testMethod1() {
// Add your code here
}
#Test
public void testMethod2() {
// Add your code here
}
#Test
public void testMethod3HasAlongName() {
// Add your code here
}
}
So you may want to take a close look at tool support before writing your standards.

I use nearly plain English for my unit test function names. Helps to define what they do exactly:
TEST( TestThatVariableFooDoesNotOverflowWhenCalledRecursively )
{
/* do test */
}
I use C++ but the naming convention can be used anywhere.

Make sure to include what is not an unit tests. See: What not to test when it comes to Unit Testing?
Include a guideline so integration tests are clearly identified and can be run separately from unit tests. This is important, because you can end with a set of "unit" tests that are really slow if the unit tests are mixed with other types of tests.
Check this for more info on it: How can I improve my junit tests ... specially the second update.

If you are using tools from the family of Junit (OCunit, SHunit, ...), names of tests already follow some rules.
For my tests, I use custom doxygen tags in order to gather their documentation in a specific page.

How to make an existing public API testable for external programmers using it?

I have a C# public API that is used by many third-party developers that have written custom applications on top of it. In addition, the API is used widely by internal developers.
This API wasn't written with testability in mind: most class methods aren't virtual and things weren't factored out into interfaces. In addition, there are some helper static methods.
For many reasons I can't change the design significantly without causing breaking changes for applications developed by programmers using my API. However, I'd still like to give internal and external developers using this API the chance to write unit tests and be able to mock the objects in the API.
There are several approaches that come to mind, but none of them seem great:
The traditional approach would be to force developers to create a proxy class that they controlled that would talk to my API. This won't work in practice because there are now hundreds of classes, many of which are effectively strongly typed data transfer objects that would be a pain to reproduce and maintain.
Force all developers using the API that want to unit test it to buy TypeMock. This seems harsh to force people to pay $300+ per developer and potentially require them to learn a different mock object tool than what their used to.
Go through the entire project and make all the methods virtual. This would allow mock-ing of objects using free tools like Moq or Rhino Mocks, but it could potentially open up security risks for classes that were never meant to be derived from. Additionally this could cause breaking changes.
I could create a tool that given an input assembly would output an assembly with the same namespaces, classes, and members, but would make all of the methods virtual and it would make the method body just return the default value for the return type. Then, I could ship this dummy test assembly each time I released an update to the API. Developers could then write tests for the API against the dummy assembly since it had virtual members that are very mock-able. This might work, but it seems a bit tedious to write a custom tool for this and I can't seem to find an existing one that does it well (especially that works well with generics). Furthermore, it has the complication that it requires developers to use two different assemblies that could possibly go out of date.
Similar to #4, I could go through every file and add something like "#ifdef UNITTEST" to every method and body to do the equivalent of what a tool would do. This doesn't require an external tool, but it would pollute the codebase with a lot of ugly "#ifdef"'s.
Is there something else that I haven't considered that would be a good fit? Does a tool like what I mentioned in #4 already exist?
Again, the complicating factor is that this is a rather large API (hundreds of classes and ~10 files) and has existing applications using it which makes it hard to do drastic design changes.
There have been several questions on Stack Overflow that were generic about retrofitting an existing application to make it testable, but none seem to address the concerns I have (specifically in the context of a widely used API with many third-party developers). I'm also aware of "Working Effectively With Legacy Code" and think it has good advice, but I am looking for a specific .net approach given the constraints mentioned above.
UPDATE: I appreciate the answers so far. One that Patrik Hägne brought up is "why not extract interfaces?" This indeed works to a point, but has some problems such as the existing design has many cases where we take expose a concrete class. For example:
public class UserRepository
{
public UserData GetData(string userName)
{
...
}
}
Existing customers that are expecting the concrete class (e.g. "UserData") would break if they were given an "IUserData."
Additionally, as mentioned in the comments there are cases where we take in a class and then expose it for convenience. This could cause problems if we took in an interface and then had to expose it as a concrete class.
The biggest challenge to a significant rewrite or redesign is that there is a huge investment in the current API (thousands of hours of development and probably just as much third party training). So, while I agree that a better SOLID design rewrite or abstraction layer (that eventually could become the new API) that focused on items like the Interface Separation Principle would be a plus from a testability perspective, it'd be a large undertaking that probably can't be cost justified at the present time.
We do have testing for the current API, but it is more complicated integration testing rather than unit-testing.
Additionally, as mentioned by Chad Myers, this is question addresses a similar problem that the .NET framework itself faces in some areas.
I realize that I'm probably looking for a "silver bullet" here that doesn't exist, but all help is appreciated. The important part is protecting the huge time investments by many third party developers as well as the huge existing development to create the current API.
All answers, especially those that consider the business side of the problem, will be carefully reviewed. Thanks!

What you're really asking is, "How do I design my API with SOLID and similar principles in mind so my API plays well with others?" It's not just about testability. If your customers are having problems testing their code with yours, then they're also having problems WRITING/USING their code with yours, so this is a bigger problem than just testability.
Simply extracting interfaces will not solve the problem because it's likely your existing class interfaces (what the concrete classes expose as their methods/properties) aren't design with Interface Segregation Principle in mind, so the extracted interface would have all sorts of problems (some of which you mentioned in comment to a previous answer).
I like to call this the IHttpContext problem. ASP.NET, as you know, is very difficult to test around or with due to the "Magic Singleton Dependency" problem of HttpContext.Current. HttpContext is not mockable without fancy tricks like what TypeMock uses. Simply extracting an interface of HttpContext is not going to help that much because it's SO huge. Eventually, even IHttpContext would become a burden to test with so much so that it's almost not worth doing any more than trying to mock HttpContext itself.
Identifying object responsibilities, slicing up interfaces and interactions appropriately, and designing with Open/Closed Principle in mind is not something you and try to force/cram into an existing API designed without these principles in mind.
I hate to leave you with such a grim answer, so I'll give you one positive suggest: How's about YOU take all the grief on behalf of your customers and make some sort of service/facade layer over top of your old API. This service layer will have to deal with the minutiae and pain of your API, but will present a nice, clean, SOLID-friendly public API that your customers can use with much less friction.
This also has the added benefit of allowing you to slowly replace parts of your API and eventually make it so your new API isn't just a facade, it IS the API (and the old API is phased out).

Another approach would be to create a seperate branch of the API and do option 3 there. Then you just maintain these two versions and deprecate the former. Merging changes from one branch into the other should work automatically most of the time.

As a reply to your edit, interface extraction does indeed work very well here:
public interface IUserRepository
{
IUserData GetData(string userName);
}
public class UserRepository
: IUserRepository
{
// The old method is not touched.
public UserData GetData(string userName)
{
...
}
// Explicitly implement the interface method.
IUserData IUserRepository.GetData(string userName)
{
return this.GetData(userName);
}
}
As I also said in a comment this may not be the way to go in every place. I think you should identify some main points in your API where it's extra important for your customers to be able to fake the interaction and start there. You don't have to make a complete rewrite of the whole API but it can transform gradually.

One approach you don't mention (and the one I'd prefer in most cases) is to extract interfaces for the classes you want the user of the API to be able to fake. Not knowing your API not every single class in it has to have it's interface extracted.

Third party users should not be testing your API. They would want to test their code against your API and so they need to create Mocks of the API etc. but they would be relying on your testing of the API to ensure it works. Or is that what you meant? Do you want to make your API easy to test against?
Start again in that case, and this time think about the testers :)

I agree with Kim. Why not re-write your core API using the best practices you explained, and supply a set of proxy/adapter classes that expose the old interface but talk to your new API?
Old developers will be naturally encouraged to migrate to the new API, but not be forced to immediately do so. New developers will simply use your new API. Announce an EOL for your old API interface if you are concerned about developers staying on the old API.