I work in TDD environment and basically I am facing with a dilemma which I think is very important in TDD environment. As a programmer, you want your methods to be as readable as possible. To achieve that, we tend to partition our methods in multiple private methods as well. While doing that all that code which was moved to the private function looses it's test ability.
Rhino test class cannot see all those private methods and I need to be able to run tests against those methods as well. I do not want them to be public because it does not make sense to keep them public.
Any ideas?
If I qoute a part of your question:
[...] we tend to partition our methods in multiple private methods [...]
This is wrong. If you follow a single responsibility principle and good OOP design, your methods would be much independent and simpler. If you feel like you want to extract a yet another private method to make your public one look shorter, give it a thought first. Maybe, you can refactor it in a separate class?
You do not test private methods, because you test public contracts and not the details of implementations. If you want to have something distantly similar to private methods testing, make them internal and set InternalsVisibleTo attribute.
Another method (pointed by R. Harvey) is to write a wrapper class that wraps you private methods into public ones. This approach has a benefit that you don't need to make your private methods internal. The downside is that for every private method you will have a wrapper public method. So the amount of methods may double.
As suggested by others, one way to test non-public methods is to make them internal and use InternalsVisibleTo attribute. However, I would strongly suggest against that.
Private methods should be covered by unit tests by testing public methods that use them. Of course, as time progresses and you add more functionality to the class under test, it gets more and more complicated to setup your tests. This is a good indicator that the class has too much responsibility and you should split it into multiple smaller classes. You can then make these smaller classes a dependency of original class and mock them in your tests - that will simplify the tests again.
While doing that, you don't have to entirely relinquish private methods - it's a good idea to use them to make your code more readable without using comments.
what are pros and cons of using partial classes for writing NUnit tests?
I am going to start:
pro: private methods can be tested
con: TDD is not really possible anymore
What else?
Con: either you have to test a different build to the one you ship, or you end up with your unit test code (and members) in your shipping code.
Sounds like a really bad idea to me in general.
I usually go for separate production/test projects, with [InternalsVisibleTo] allowing testing of internal methods (which goes against the dogma of some folks, but seems pragmatic to me).
I would argue that being able to test private methods is a 'con', as it can encourage accretion of new code within an existing class. If the private logic is so complex that it needs dedicated tests (beyond what's publicly accessible), then there's a lot of value in pulling out that logic as a separate class with a publicly testable interface. For borderline cases, I cautiously agree with Jon's approach of using internal methods (just be careful how much the internals expose.)
That said, I've occasionally made a class unsealed specifically so that I could write tests that used virtual method calls for sensing of effects, which is loosely related to the partial class approach. This always feels a bit dirty to me and I sometimes wished there were a way to mark a class as "internal unsealed".
Many times I find myself torn between making a method private to prevent someone from calling it in a context that doesn't make sense (or would screw up the internal state of the object involved), or making the method public (or typically internal) in order to expose it to the unit test assembly. I was just wondering what the Stack Overflow community thought of this dilemma?
So I guess the question truly is, is it better to focus on testability or on maintaining proper encapsulation?
Lately I've been leaning towards testability, as most of the code is only going to be leveraged by a small group of developers, but I thought I would see what everyone else thought?
Its NOT ok to change method visibility on methods that the customers or users can see. Doing this is ugly, a hack, exposes methods that any dumb user could try to use and explode your app... its a liability you do not need.
You are using C# yes? Check out the internals visible to attribute class.
You can declare your testable methods as internal, and allow your unit testing assembly access to your internals.
It depends on whether the method is part of a public API or not. If a method does not belong to part of a public API, but is called publicly from other types within the same assembly, use internal, friend your unit test assembly, and unit test it.
However, if the method is not/should not be part of a public API, and it is not called by other types internal to the assembly, DO NOT test it directly. It should be protected or private, and it should only be tested indirectly by unit testing your public API. If you write unit tests for non-public (or what should be non-public) members of your types, you are binding test code to internal implementation details.
Thats a bad kind of coupling, increases the amount of unit tests you need, increases workload both in the short term (more unit tests) as well as in the long term (more test maintenance and modification in response to refactoring internal implementation details). Another problem with testing non-public members is that you test code that may not actually be needed or used. A GREAT way to find dead code is when it is not covered by any of your unit tests when your public API is covered 100%. Removing dead code is a great way to keep your code base lean and mean, and is impossible if you are not careful about what you put into your public API, and what parts of your code you unit test.
EDIT:
As a quick additional note...with a properly designed public API, you can very effectively use a tool like Microsoft PEX to automatically generate full-coverage unit tests that test every execution path of your code. Combined with a few manually written tests that cover critical behavior, anything not covered can be considered dead code and removed, and you can greatly shortcut your unit testing process.
This is a common thought.
It's generally best to test the private methods by testing the public methods that call them (so you don't explicitly test the private methods). However, I understand that there are times when you really do want to test those private methods.
The answers to this question (Java) and this question (.NET) should be helpful.
To answer the question: no, you shouldn't change method visibility for the sake of testing. You generally shouldn't be testing private methods, and when you do, there are better ways to do it.
In general I agree with #jrista. But, as usual, it depends.
When trying to work with legacy code, the key is to get it under test. After that, you can add tests for new features and existing bugs, refactor to improve design, etc. This is risky without tests. Legacy code tends to be rife with dependencies, and is often extremely difficult to get under test.
In Working Effectively with Legacy Code, Michael Feathers suggests multiple techniques for getting code under test. Many of these techniques involve breaking encapsulation or complicating the design, and the author is up front about this. Once tests are in place, the code can be improved safely.
So for legacy code, do what you have to do.
In .NET you should use Accessors for unit testing, even rather than the InternalsVisibleTo attribute. Accessors allow you to get access to any method in the class even if it is private. They even let you test abstract classes using an empty mock derived object (see the "PrivateObject" class).
Basically in your test project you use the accessor class rather than the actual class with the methods you want to test. The accessor class is the same as the "real" class, except everything is public to your test project. Visual studio can generate accessors for you.
NEVER make a type more visible to facilitate unit testing.
IMO is it WRONG to say that you should not unit test private methods. Unit tests are of exceptional value for regression testing and there is no reason why private methods should not be regression tested with granular unit tests.
This question already has answers here:
Unit testing private methods in C#
(17 answers)
Closed 6 years ago.
I am currently involved in developing with C# - Here is some background:
We implement MVP with our client application and we have a cyclomatic rule which states that no method should have a cyclomatic complexity greater than 5.
This leads to a lot of small private methods which are generally responsible for one thing.
My question is about unit testing a class:
Testing the private implementation through the public methods is all fine... I don't have a problem implementing this.
But... what about the following cases:
Example 1. Handle the result of an async data retrival request (The callback method shouldn't be public purely for testing)
Example 2. An event handler which does an operation (such as update a View label's text - silly example I know...)
Example 3. You are using a third party framework which allows you to extend by overriding protected virtual methods (the path from the public methods to these virtual methods are generally treated as black box programming and will have all sorts of dependancies that the framework provides that you don't want to know about)
The examples above don't appear to me to be the result of poor design.
They also do not appear be be candidates for moving to a seperate class for testing in isolation as such methods will lose their context.
Doesn anyone have any thoughts about this?
Cheers,
Jason
EDIT:
I don't think I was clear enough in my original question - I can test private methods using accessors and mock out calls/ methods using TypeMock. That isn't the problem. The problem is testing things which don't need to be public, or can't be public.
I don't want to make code public for the sake of testing as it can introduce security loopholes (only publishing an interface to hide this is not an option because anyone can just cast the object back to its original type and get access to stuff I wouldn't want them to)
Code that gets refactored out to another class for testing is fine - but can lose context. I've always thought it bad practice to have 'helper' classes which can contain a pot of code with no specific context - (thinking SRP here). I really don't think this works for event handlers either.
I am happy to be proven wrong - I just am unsure how to test this functionality! I have always been of the mind that if it can break or be changed - test it.
Cheers, Jason
As Chris has stated, it is standard practice to only unit test public methods. This is because, as a consumer of that object, you are only concerned about what is publically available to you. And, in theory, proper unit tests with edge cases will fully exercise all private method dependencies they have.
That being said, I find there are a few times where writing unit tests directly against private methods can be extremely useful, and most succinct in explaining, through your unit tests, some of the more complex scenarios or edge cases that might be encountered.
If that is the case, you can still invoke private methods using reflection.
MyClass obj = new MyClass();
MethodInfo methodInfo = obj.GetType().GetMethod("MethodName", BindingFlags.Instance | BindingFlags.NonPublic);
object result = methodInfo.Invoke(obj, new object[] { "asdf", 1, 2 });
// assert your expected result against the one above
we have a cyclomatic rule which states
that no method should have a
cyclomatic complexity greater than 5
I like that rule.
The point is that the private methods are implementation details. They are subject to change/refactoring. You want to test the public interface.
If you have private methods with complex logic, consider refactoring them out into a separate class. That can also help keep cyclomatic complexity down. Another option is to make the method internal and use InternalsVisibleTo (mentioned in one of the links in Chris's answer).
The catches tend to come in when you have external dependencies referenced in private methods. In most cases you can use techniques such as Dependency Injection to decouple your classes. For your example with the third-party framework, that might be difficult. I'd try first to refactor the design to separate the third-party dependencies. If that's not possible, consider using Typemock Isolator. I haven't used it, but its key feature is being able to "mock" out private, static, etc. methods.
Classes are black boxes. Test them that way.
EDIT: I'll try to respond to Jason's comment on my answer and the edit to the original question. First, I think SRP pushes towards more classes, not away from them. Yes, Swiss Army helper classes are best avoided. But what about a class designed to handle async operations? Or a data retrieval class? Are these part of the responsibility of the original class, or can they be separated?
For example, say you move this logic to another class (which could be internal). That class implements an Asynchronous Design Pattern that permits the caller to choose if the method is called synchronously or asynchronously. Unit tests or integration tests are written against the synchronous method. The asynchronous calls use a standard pattern and have low complexity; we don't test those (except in acceptance tests). If the async class is internal, use InternalsVisibleTo to test it.
There is really only two cases you need to consider:
the private code is called, directly or indirectly from public code and
the private code is not called from public code.
In the first case, the private code is automatically being tested by the tests which exercise the public code that calls the private code, so there is no need to test the private code. And in the second case, the private code cannot be called at all, therefore it should be deleted, not tested.
Ergo: there is no need to explicitly test the private code.
Note that when you do TDD it is impossible for untested private code to even exist. Because when you do TDD, the only way that private code can be appear, is by an Extract {Method|Class|...} Refactoring from public code. And Refactorings are, by definition, behavior-preserving and therefore test-coverage-preserving. And the only way that public code can appear is as the result of a failing test. If public code can only appear as already tested code as the result of a failing test, and private code can only appear as the result of being extracted from public code via a behavior-preserving refactoring, it follows that untested private code can never appear.
In all of my unit testing, I've never bothered testing private functions. I typically just tested public functions. This goes along with the Black Box Testing methodology.
You are correct that you really can't test the private functions unless you expose the private class.
If your "seperate class for testing" is in the same assembly, you can choose to use internal instead of private. This exposes the internal methods to your code, but they methods will not be accessible to code not in your assembly.
EDIT: searching SO for this topic I came across this question. The most voted answer is similar to my response.
A few points from a TDD guy who has been banging around in C#:
1) If you program to interfaces then any method of a class that is not in the interface is effectively private. You might find this a better way to promote testability and a better way to use interfaces as well. Test these as public members.
2) Those small helper methods may more properly belong to some other class. Look for feature envy. What may not be reasonable as a private member of the original class (in which you found it) may be a reasonable public method of the class it envies. Test these in the new class as public members.
3) If you examine a number of small private methods, you might find that they have cohesion. They may represent a smaller class of interest separate from the original class. If so, that class can have all public methods, but be either held as a private member of the original class or perhaps created and destroyed in functions. Test these in the new class as public members.
4) You can derive a "Testable" class from the original, in which it is a trivial task to create a new public method that does nothing but call the old, private method. The testable class is part of the test framework, and not part of the production code, so it is cool for it to have special access. Test it in the test framework as if it were public.
All of these make it pretty trivial to have tests on the methods that are currently private helper methods, without messing up the way intellisense works.
There are some great answers here, and I basically agree with the repeated advice to sprout new classes. For your Example 3, however, there's a sneaky, simple technique:
Example 3. You are using a third party
framework which allows you to extend
by overriding protected virtual
methods (the path from the public
methods to these virtual methods are
generally treated as black box
programming and will have all sorts of
dependencies that the framework
provides that you don't want to know
about)
Let's say MyClass extends FrameworkClass. Have MyTestableClass extend MyClass, and then provide public methods in MyTestableClass that expose the protected methods of MyClass that you need. Not a great practice - it's kind of an enabler for bad design - but useful on occasion, and very simple.
Would accessor files work? http://msdn.microsoft.com/en-us/library/bb514191.aspx I've never directly worked with them, but I know a coworker used them to test private methods on some Windows Forms.
Several people have responded that private methods shouldn't be tested directly, or they should be moved to another class. While I think this is good, sometimes its just not worth it. While I agree with this in principle, I've found that this is one of those rules that cna be broken to save time without negative repercussions. If the function is small/simple the overhead of creating another class and test class is overkill. I will make these private methods public, but then not add them to the interface. This way consumers of the class (who should be getting the interface only through my IoC library) won't accidentally use them, but they're available for testing.
Now in the case of callbacks, this is a great example where making a private property public can make tests a lot easier to write and maintain. For instance, if class A passes a callback to class B, I'll make that callback a public property of class A. One test for class A use a stub implementation for B that records the callback passed in. The test then verify the the callback is passed in to B under appropriate conditions. A separate test for class A can then call the callback directly, verifying it has the appropriate side effects.
I think this approach works great for verifying async behaviors, I've been doing it in some javascript tests and some lua tests. The benefit is I have two small simple tests (one that verifies the callback is setup, one that verifies it behaves as expected). If you try to keep the callback private then the test verifying the callback behavior has a lot more setup to do, and that setup will overlap with behavior that should be in other tests. Bad coupling.
I know, its not pretty, but I think it works well.
I will admit that when recently writing units tests for C# I discovered that many of the tricks I knew for Java did not really apply (in my case it was testing internal classes).
For example 1, if you can fake/mock the data retrieval handler you can get access to the callback through the fake. (Most other languages I know that use callbacks also tend not to make them private).
For example 2 I would look into firing the event to test the handler.
Example 3 is an example of the Template Pattern which does exist in other languages. I have seen two ways to do this:
Test the entire class anyway (or at least relevant pieces of it). This particularly works in cases where the abstract base class comes with its own tests, or the overall class is not too complex. In Java I might do this if I were writing an extension of AbstractList, for example. This may also be the case if the template pattern was generated by refactoring.
Extend the class again with extra hooks that allow calling the protected methods directly.
Don't test private code, or you'll be sorry later when it's time to refactor. Then, you'll do like Joel and blog about how TDD is too much work because you constantly have to refactor your tests with your code.
There are techniques (mocks, stub) to do proper black box testing. Look them up.
This is a question that comes up pretty early when introducing testing. The best technique to solving this problem is to black-box test (as mentioned above) and follow the single responsibility principle. If each of your classes only have only one reason to change, they should be pretty easy to test their behavior without getting at their private methods.
SRP - wikipedia / pdf
This also leads to more robust and adaptable code as the single responsibility principle is really just saying that your class should have high cohesion.
In C# you can use the attribute in AssemblyInfo.cs:
[assembly: InternalsVisibleTo("Worker.Tests")]
Simply mark your private methods with internal, and the test project will still see the method. Simple! You get to keep encapsulation AND have testing, without all the TDD nonsense.
We are building a framework that will be used by other developers and for now we have been using a lot of TDD practices. We have interfaces everywhere and have well-written unit tests that mock the interfaces.
However, we are now reaching the point where some of the properties/methods of the input classes need to be internal, and not visible to our framework users (for example object Id). The problem then is that we can't put those fields/methods on the interface as the interface does not describe accessibility.
We could:
Still use interfaces and upcast in the first line of the method, but that seems to defeat the purpose of interfaces.
Use classes as input parameters - breaking the TDD rule that everything should be interfaces
Provide another layer which does some translation between public interfaces and internal interfaces
Is there an existing pattern/approach to deal with this? What do the TDD people say should be done?
First, there is no general TDD rule that says everything should be an interface. This is coming from a specific style that is not practiced by every TDDer. See http://martinfowler.com/articles/mocksArentStubs.html
Second, you are experiencing the dichotomy of public vs. published. Our team "solved" this problem by introducing a #Published annotation that shows up in the API documentation. Eclipse uses naming conventions, as far as I know. I don't know of a really good solution to the problem, unfortunately.
You need to be able to replicate those internal methods in your mock up objects. And call them in the same way the real object would call them. Then you focus your unit test on the public method that relies on that private method you need to test. If these internal methods are calling other objects or doing a lot of work, you may need to refactor your design.
Good luck.
Sounds like you want your class to have a dependency injection. Search stackoverflow too. Then you can set this Id by your choice of either within constructor or through a setter.
[1l