I'm running unit tests in a class library project with NSpec framework, AutofacContrib.NSubstitute v3.3.2.0, NSubstitute v1.7.0.0 (the latest as of now is 1.8.2).
The Class Under Test instance is built with AutoSubstitute, in order to automock all its needed dependencies.
AutoSubstitute autoSubstitute = new AutoSubstitute();
MainPanelViewModel viewModel = autoSubstitute.Resolve<MainPanelViewModel>();
If working properly, my Class Under Test at some point will invoke one of it's base class methods with some specific input parameter (the base class is out of my control):
// ...
base.ActivateItem(nextScreen);
// ...
So, for test expectation, I need to check (spy) that the instance invokes the base method:
viewModel.Received().ActivateItem(Arg.Any<SomeSpecificScreenType>());
Here's the problem: when I try to do this, at runtime NSubstitute complains that I can only ran Received() against an object created with Substitute.For<>(). I also checked quickly AutofacContrib.NSubstitute source code, but I could not find a way to obtain the instance with automocking and at the same time wrap it somehow in a spy object or something like that.
I also thought that maybe Substitute.ForPartsOf<>() could be helpful, but that method does not seem to be found in NSubstitute v1.7.0.
For sake of completeness, here's NSubstitute full error:
NSubstitute extension methods like .Received() can only be called on objects created using Substitute.For() and related methods.
For completeness, I did some experimenting with NSubstitute's partial substitutions with ForPartsOf.
The way ForPartsOf works is essentially to define a new class that inherits from the class you're using as a template. This restricts what it is that the mocking framework can intercept to methods that are either defined as abstract or virtual. This is the same limitation you would have for modifying the behaviour of a class if you were to inherit from it with your own class.
Taking this information, lets look at your problem. You want to intercept this call:
base.ActivateItem(nextScreen);
So, because of the limitations above, for you to be able to intercept a call to ActivateItem, the method has to be marked as virtual in the base class. If it's not, there's nothing you can do without changing the application structure.
If the method is marked as virtual, then you can intercept it with NSubstitute but you can only do it if the NSubstituted implementation is called. This works through normal method dispatch, because the highest level implementation of a virtual method is called (the one provided by NSubstitute) when you invoke it. However, it doesn't work when you're calling the method via the base reference.
So, whilst you could intercept this:
ActivateItem(nextScreen)
You simply can't intercept this:
base.ActivateItem(nextScreen);
The fact that you're using base.ActivateItem in your code suggests that your class under test has its own implementation of the method that you don't want to call, so with your current tools you can't achieve what you were trying to do. Which is why it's a good thing that you found a workaround.
You're in the same situation with most other mocking frameworks, including Moq. The exception is TypeMock, which uses a totally different way to intercept method calls which means that it can do things that other frameworks simply can't.
So, the actual issue has not been really solved: it's just that the issue itself disappeared.
In order to check for correct behaviour, I recalled that I could also resort to an ActiveItem public property from base class, so with that I stopped using Receive() and went back to simple value comparison.
Still, for future reference, I did not find a way to spy the Class Under Test with these libraries. I know that spying the class under test should be avoided, but as with many things, sometimes you need to do that.
HTH
Related
We are working on three layer architecture (UI - Library1 - Library2 -> Hardware) developed in C# , but I wanted to test the code which I have implemented in library1, but the problem we do not have access to hardware (various reasons) but we can access library2.
My question is can I return a fake value from library2 like if the functionality is implemented to test the hardware connection and when ever we call that library2 function return true or any related default return parameter that is accessed from library1?
I am using VS2010.
There are a lot of examples you can find by doing a google search on: C# Mocking. I tend to write my own mock classes and implement an ObjectFactory that I can use to specifically instantiate mock classes in tests, but otherwise default to the real class.
For example: If I wanted to use System.Net.WebRequest I would first create my common interface - IWebRequest that exposes .Timeout, .UploadString, and any others I might use in my application.
Then I would create my implementation class using that interface. Inside the implementation of say, .UploadString, I would create an instance of the real System.Net.WebRequest class and call its .UploadString.
In the mock class, implement it using the IWebRequest interface, just like the real wrapper class, but when you implement the .UploadString method, instead just have it return a hardcoded response. You can put these mock classes in their own "Mocks" library that is only included in your test projects.
To get all this to run you will need your ObjectFactory which is included in both your regular code projects and your unit test projects. You will have static functions with return types of your interfaces (IWebRequest) that check if a classes has been manually specified. If it hasn't it returns an instance of the default "real" implementation.
In your test you would access the ObjectFactory and manually set that you want to use a specific mock class and when your real code runs it instantiates that instead of your real class.
Keep in mind, like I said, there are a lot of examples out there, and even full implementations of mocking frameworks that you can use. Just take a look on google.
Although there are many different frameworks for providing this function, I would recommend just sticking with the built-in VS.net framework as described here. http://msdn.microsoft.com/en-us/library/hh549175.aspx
Is there a standard method for unit testing code where dependencies do not implement interfaces? For instance, the System.Net.Http namespace only exposes concrete classes. If I'm trying to unit test a class which relies upon one of the concrete classes in System.Net.Http, should I merely construct an instance of, say, HttpRequestMessage, set it's properties and then supply this newly constructed object to the system under test? Would it make sense to subclass HttpRequestMessage and have it implement a custom interface which could then be mocked/stubbed?
The recommended practice is to wrap this object in a class of your creation, which itself implements an interface. You would then use this wrapper class in your code, and you can then supply mocked versions of this wrapper in place of the real class. You would not subclass it using this method, but rather contain it and use delegation (not to be confused with delegates!) to forward each method.
For instance, you might create a class HttpRequestMessageWrapper that inherits from IHttpRequestMessage (that you define, including all the public properties of HttpRequestMessage.. although you could probably get by with only implementing the properties you use).
Alternatively, you can use a testing framework that supports shims, which is a framework that essentially does this wrapper for you and replaces calls to the objects with shim versions. Microsoft Fakes (introduced in VS 2012 MS Test) does this.
Shims are typically used to replace common framework calls, such as DateTime.Now so you can supply a specific value during tests.
I would suggest you have a look at: AutoFixture http://autofixture.codeplex.com/ It helps you construct the object in the way you want. In your example, HttpRequestMessage, you can customize the fixture: fixture.Customize<HttpRequestMessage>(c => {});
There are many examples out there using AutoFixture in Unit Testing. Or you can post the code you want to test here and i can try to help.
Another good tool for testing legacy code is Typemock
I'd like to intercept and inject custom code when calling 3rd party code in C#. I am using an external library (AutoIt) for GUI automation. The AutoIt dll is provided without source code.
All actions done with this framework are performed from a single class (AutoItClass) providing access to all the methods. I'd like to be able to inject custom code when calling methods on this class, is this possible? For example:
Log some information from within the called method.
Perform any other action from within the method (wait for X seconds).
This would be possible very simply by inheriting from this class and overriding all its methods (which is a must since this is a COM object), but this is not the preferred way. Any comments will be helpful!
I wouldn't use inheritance - you can use composition here. Create your own class which has the same methods - or in fact only the ones you're interested in - and delegate through that. That way you can be sure you won't "miss" any methods accidentally, because anything you don't implement won't be callable through the rest of your codebase... so long as you make sure the rest of your code doesn't refer to the original library class, of course.
You can investigate PostSharp, which is a commercial product that can inject IL into compiled assemblies to perform aspect oriented programming. You can define different kind of behaviour that should happen before and after a method gets executed, for example, which seems to be what you want. This way, as PostSharp handles this in a post-compilation step, you don't need to create any inherited classes from the classes that you want to intercept.
Otherwise if you want a more "pure" solution I would follow Jon's advice about creating a new class that wraps the functionality of the one that you want to intercept. (see Decorator pattern) .
This question already has answers here:
Unit testing private methods in C#
(17 answers)
Closed 6 years ago.
I am currently involved in developing with C# - Here is some background:
We implement MVP with our client application and we have a cyclomatic rule which states that no method should have a cyclomatic complexity greater than 5.
This leads to a lot of small private methods which are generally responsible for one thing.
My question is about unit testing a class:
Testing the private implementation through the public methods is all fine... I don't have a problem implementing this.
But... what about the following cases:
Example 1. Handle the result of an async data retrival request (The callback method shouldn't be public purely for testing)
Example 2. An event handler which does an operation (such as update a View label's text - silly example I know...)
Example 3. You are using a third party framework which allows you to extend by overriding protected virtual methods (the path from the public methods to these virtual methods are generally treated as black box programming and will have all sorts of dependancies that the framework provides that you don't want to know about)
The examples above don't appear to me to be the result of poor design.
They also do not appear be be candidates for moving to a seperate class for testing in isolation as such methods will lose their context.
Doesn anyone have any thoughts about this?
Cheers,
Jason
EDIT:
I don't think I was clear enough in my original question - I can test private methods using accessors and mock out calls/ methods using TypeMock. That isn't the problem. The problem is testing things which don't need to be public, or can't be public.
I don't want to make code public for the sake of testing as it can introduce security loopholes (only publishing an interface to hide this is not an option because anyone can just cast the object back to its original type and get access to stuff I wouldn't want them to)
Code that gets refactored out to another class for testing is fine - but can lose context. I've always thought it bad practice to have 'helper' classes which can contain a pot of code with no specific context - (thinking SRP here). I really don't think this works for event handlers either.
I am happy to be proven wrong - I just am unsure how to test this functionality! I have always been of the mind that if it can break or be changed - test it.
Cheers, Jason
As Chris has stated, it is standard practice to only unit test public methods. This is because, as a consumer of that object, you are only concerned about what is publically available to you. And, in theory, proper unit tests with edge cases will fully exercise all private method dependencies they have.
That being said, I find there are a few times where writing unit tests directly against private methods can be extremely useful, and most succinct in explaining, through your unit tests, some of the more complex scenarios or edge cases that might be encountered.
If that is the case, you can still invoke private methods using reflection.
MyClass obj = new MyClass();
MethodInfo methodInfo = obj.GetType().GetMethod("MethodName", BindingFlags.Instance | BindingFlags.NonPublic);
object result = methodInfo.Invoke(obj, new object[] { "asdf", 1, 2 });
// assert your expected result against the one above
we have a cyclomatic rule which states
that no method should have a
cyclomatic complexity greater than 5
I like that rule.
The point is that the private methods are implementation details. They are subject to change/refactoring. You want to test the public interface.
If you have private methods with complex logic, consider refactoring them out into a separate class. That can also help keep cyclomatic complexity down. Another option is to make the method internal and use InternalsVisibleTo (mentioned in one of the links in Chris's answer).
The catches tend to come in when you have external dependencies referenced in private methods. In most cases you can use techniques such as Dependency Injection to decouple your classes. For your example with the third-party framework, that might be difficult. I'd try first to refactor the design to separate the third-party dependencies. If that's not possible, consider using Typemock Isolator. I haven't used it, but its key feature is being able to "mock" out private, static, etc. methods.
Classes are black boxes. Test them that way.
EDIT: I'll try to respond to Jason's comment on my answer and the edit to the original question. First, I think SRP pushes towards more classes, not away from them. Yes, Swiss Army helper classes are best avoided. But what about a class designed to handle async operations? Or a data retrieval class? Are these part of the responsibility of the original class, or can they be separated?
For example, say you move this logic to another class (which could be internal). That class implements an Asynchronous Design Pattern that permits the caller to choose if the method is called synchronously or asynchronously. Unit tests or integration tests are written against the synchronous method. The asynchronous calls use a standard pattern and have low complexity; we don't test those (except in acceptance tests). If the async class is internal, use InternalsVisibleTo to test it.
There is really only two cases you need to consider:
the private code is called, directly or indirectly from public code and
the private code is not called from public code.
In the first case, the private code is automatically being tested by the tests which exercise the public code that calls the private code, so there is no need to test the private code. And in the second case, the private code cannot be called at all, therefore it should be deleted, not tested.
Ergo: there is no need to explicitly test the private code.
Note that when you do TDD it is impossible for untested private code to even exist. Because when you do TDD, the only way that private code can be appear, is by an Extract {Method|Class|...} Refactoring from public code. And Refactorings are, by definition, behavior-preserving and therefore test-coverage-preserving. And the only way that public code can appear is as the result of a failing test. If public code can only appear as already tested code as the result of a failing test, and private code can only appear as the result of being extracted from public code via a behavior-preserving refactoring, it follows that untested private code can never appear.
In all of my unit testing, I've never bothered testing private functions. I typically just tested public functions. This goes along with the Black Box Testing methodology.
You are correct that you really can't test the private functions unless you expose the private class.
If your "seperate class for testing" is in the same assembly, you can choose to use internal instead of private. This exposes the internal methods to your code, but they methods will not be accessible to code not in your assembly.
EDIT: searching SO for this topic I came across this question. The most voted answer is similar to my response.
A few points from a TDD guy who has been banging around in C#:
1) If you program to interfaces then any method of a class that is not in the interface is effectively private. You might find this a better way to promote testability and a better way to use interfaces as well. Test these as public members.
2) Those small helper methods may more properly belong to some other class. Look for feature envy. What may not be reasonable as a private member of the original class (in which you found it) may be a reasonable public method of the class it envies. Test these in the new class as public members.
3) If you examine a number of small private methods, you might find that they have cohesion. They may represent a smaller class of interest separate from the original class. If so, that class can have all public methods, but be either held as a private member of the original class or perhaps created and destroyed in functions. Test these in the new class as public members.
4) You can derive a "Testable" class from the original, in which it is a trivial task to create a new public method that does nothing but call the old, private method. The testable class is part of the test framework, and not part of the production code, so it is cool for it to have special access. Test it in the test framework as if it were public.
All of these make it pretty trivial to have tests on the methods that are currently private helper methods, without messing up the way intellisense works.
There are some great answers here, and I basically agree with the repeated advice to sprout new classes. For your Example 3, however, there's a sneaky, simple technique:
Example 3. You are using a third party
framework which allows you to extend
by overriding protected virtual
methods (the path from the public
methods to these virtual methods are
generally treated as black box
programming and will have all sorts of
dependencies that the framework
provides that you don't want to know
about)
Let's say MyClass extends FrameworkClass. Have MyTestableClass extend MyClass, and then provide public methods in MyTestableClass that expose the protected methods of MyClass that you need. Not a great practice - it's kind of an enabler for bad design - but useful on occasion, and very simple.
Would accessor files work? http://msdn.microsoft.com/en-us/library/bb514191.aspx I've never directly worked with them, but I know a coworker used them to test private methods on some Windows Forms.
Several people have responded that private methods shouldn't be tested directly, or they should be moved to another class. While I think this is good, sometimes its just not worth it. While I agree with this in principle, I've found that this is one of those rules that cna be broken to save time without negative repercussions. If the function is small/simple the overhead of creating another class and test class is overkill. I will make these private methods public, but then not add them to the interface. This way consumers of the class (who should be getting the interface only through my IoC library) won't accidentally use them, but they're available for testing.
Now in the case of callbacks, this is a great example where making a private property public can make tests a lot easier to write and maintain. For instance, if class A passes a callback to class B, I'll make that callback a public property of class A. One test for class A use a stub implementation for B that records the callback passed in. The test then verify the the callback is passed in to B under appropriate conditions. A separate test for class A can then call the callback directly, verifying it has the appropriate side effects.
I think this approach works great for verifying async behaviors, I've been doing it in some javascript tests and some lua tests. The benefit is I have two small simple tests (one that verifies the callback is setup, one that verifies it behaves as expected). If you try to keep the callback private then the test verifying the callback behavior has a lot more setup to do, and that setup will overlap with behavior that should be in other tests. Bad coupling.
I know, its not pretty, but I think it works well.
I will admit that when recently writing units tests for C# I discovered that many of the tricks I knew for Java did not really apply (in my case it was testing internal classes).
For example 1, if you can fake/mock the data retrieval handler you can get access to the callback through the fake. (Most other languages I know that use callbacks also tend not to make them private).
For example 2 I would look into firing the event to test the handler.
Example 3 is an example of the Template Pattern which does exist in other languages. I have seen two ways to do this:
Test the entire class anyway (or at least relevant pieces of it). This particularly works in cases where the abstract base class comes with its own tests, or the overall class is not too complex. In Java I might do this if I were writing an extension of AbstractList, for example. This may also be the case if the template pattern was generated by refactoring.
Extend the class again with extra hooks that allow calling the protected methods directly.
Don't test private code, or you'll be sorry later when it's time to refactor. Then, you'll do like Joel and blog about how TDD is too much work because you constantly have to refactor your tests with your code.
There are techniques (mocks, stub) to do proper black box testing. Look them up.
This is a question that comes up pretty early when introducing testing. The best technique to solving this problem is to black-box test (as mentioned above) and follow the single responsibility principle. If each of your classes only have only one reason to change, they should be pretty easy to test their behavior without getting at their private methods.
SRP - wikipedia / pdf
This also leads to more robust and adaptable code as the single responsibility principle is really just saying that your class should have high cohesion.
In C# you can use the attribute in AssemblyInfo.cs:
[assembly: InternalsVisibleTo("Worker.Tests")]
Simply mark your private methods with internal, and the test project will still see the method. Simple! You get to keep encapsulation AND have testing, without all the TDD nonsense.
I'm working with a 3rd party c# class that has lots of great methods and properties - but as time has gone by I need to extend that class with methods and properties of my own. If it was my code I would just use that class as my base class and add my own properties and method on top - but this class has an internal constructor. (In my opinion it was short sited to make the constructor internal in the first place - why limit the ability to subclass?)
The only thing I could think of was to create method / properties on my class that simply called into theirs - but it's acres of code and, well, it just doesn't "feel" right.
Is there any way to use this class a base class?
You ask: "Why limit the ability to subclass?"
Because designing for inheritance is tricky, particularly if you're designing for other developers to inherit from your class. As Josh Bloch says in Effective Java, you should design for inheritance or prohibit it. In my view, unless you have a good reason to design for inheritance, you shouldn't do so speculatively.
Does the class implement an interface which you could also implement (possibly by proxying most calls back to an instance of the original)? There's often no really elegant answer here - and the best solution will depend on the exact situation, including what you're trying to add to the class.
If you're not adding any more state - just convenience methods, effectively - then extension methods may work well for you. But they don't change what data an object is capable of storing, so if you need to add your own specialised data, that won't work.
Sounds like a perfect application for extension methods:
MSDN extension method docs
"Extension methods enable you to "add" methods to existing types without creating a new derived type, recompiling, or otherwise modifying the original type. Extension methods are a special kind of static method, but they are called as if they were instance methods on the extended type. For client code written in C# and Visual Basic, there is no apparent difference between calling an extension method and the methods that are actually defined in a type."
If the class has an internal constructor, and there are no public constructors, then that suggests that the designers did not intend for it to be subclassed. In that case, you can use encapsulation, or you can use extension methods.
Only if your class lives in the same assembly as the class you want to inherit from. An internal constructor limits the concrete implementations of the abstract class to the assembly defining the class. A class containing an internal constructor cannot be instantiated outside of the assembly.
Resharper has a nice feature to create delegating members.
Here is a sample of what you can do with it. It takes a couple of seconds.
I will not discuss whether you can build your own Facade around that 3rd party class. Previous authors are right, the library could be designed in the way that will not allow this. Suppose they have some coupled classes that have singletons that should be initialized in specific order or something like this - there may be a lot of design mistakes (or features) that 3rd party developers never care about, because they do not suppose that you will use their library in that way.
But OK, lets suppose that building a facade is not an impossible task, and you have in fact only one problem - there are too many methods you have to write wrappers around, and it is not good to do this manually.
I see 3 solutions to address exactly that problem
1) I suppose that new "dynamic" types of .NET 4.0 will allow you to workaround that problem without having to write "acres of code"
You should incapsulate an instance of 3rd party class into your class as a privare member with dynamic keyword
Your class should be derived from Dynamic or implement IDynamicObject interface. You will have to implement GetMember/SetMember functions that will forward all calls to the encapsulated instance of 3rd party class
Well, c# 4.0 is a future, Let's see on other solutions:
2) Do not write code manually if you have significant number of public methods (say more then 100). I would write a little console app that uses reflection and finds all public members and then automatically generates code to call encapsulated instance. For example
public type MethodName(params)
{
this.anInstanceOf3rdPartyClass.MethodName(params);
}
3) You can do the same as 2, but with the help of existing reflection tools, for example RedGate .NET Reflector. It will help you to list all classes and methods signatures. Then, paste all this in Word and a simple VB macro will let you generate the same code as you could do in 2.
Remark: As soon as you are not copying the code, but only copying method signatures, that are publicly available, I don't think you will violate the license agreement, but anyway it worth to re-check