I have some test-code that is initializing some members which should be done just once at the beginning. This is why I used the constructor for that:
[TestFixture]
public class MyTestClass
{
private readonly IUnitTestGeometryProvider m_GeometryProvider;
public MyTestClass()
{
// do some heavy init-op
}
private IEnumerable<TestCaseData> TestCases()
{
yield return new TestCaseData(this.m_GeometryProvider.GetPolyline())
.Throws(typeof(ArgumentException));
}
[TestCaseSource("TestCases")]
public double Check_GetReducedArea_For_Invalid_Arguments(IGeometry theGeom)
{
return theGeom.GetReducedArea();
}
}
I know of the convention to use the FixtureSetup-attribute for initializing tests, e.g. from this question on SO. However I noticed that the method TestCases is executed before the method marked with that attribute so I run into an NullReferenceException when evaluating the different testcases as the m_GeometryProvider is null at this time.
So I debugged my code and set a breakpoint into the constructor. I noticed, that it is executed twice before any test has even been run. I assumed every testcase having its own instance of the MyTestClass, but as I have three different testcases and the constructor running twice this doesn't explain it.
As the initialization is heavy I´d like to execute it just once. Is there a way to guarantee this? I´d like to avoid a static member as it often attracts colleagues to heavily use other static members just because there already is one. Furthermore I´d consider the test-init to be specific for one instance of my MyTestClass instead of the class itself - assuming there´s just one however.
I'm using NUnit 2.5.10.
Basically NUnit reserves the right to construct your fixture object whenever it needs to, as often as it needs to. Therefore, you should avoid doing heavy initialization in your constructor, especially if some initialization is only needed when the tests are being run.
Generally, initialization for executing the tests should be done in the TestFixtureSetUp method, which is run once and only once each time the fixture is executed. This is more complicated, however, when you are generating test cases using a TestCaseSource attribute.
Because TestCases must be executed in order to create your tests in the first place, eons before they are ever run, an instance of the object must be created to do that. Then when the tests are run, another instance is created for the purpose of running them. There's not enough info in your question to figure out why there are two "extra" calls to your constructor, but it's either due to some other aspect of your code or simply a bug in the somewhat old version of NUnit you are using. You would have to step through those constructor calls and examine the stack trace to see what is calling them.
Is your m_GeometryProvider member used anywhere else besides the TestCases method? If not, you can simplify things by making it a temporary field created in that method. You can then eliminate an extra constructor call by making the TestCases method static. You said you didn't like that, but it's what we recommend. In fact, in NUnit 3.0 and later it is required to be static.
If you have other initialization that's needed to run tests besides the creation of the geometry provider, that should go in a TestFixtureSetUp method.
Related
I am quite new at C# and also rhino mocks. I searched and found similar topics with my question but couldnt find a proper solution.
I am trying to understand if the private method is called or not in my unit test. I am using rhino mock, read many files about it, some of them just say that change the access specifier of the method from private to public, but I can not change the source code. I tried to link source file to my test project but it doesnt change.
public void calculateItems()
{
var result = new Result(fileName, ip, localPath, remotePath);
calculateItems(result, nameOfString);
}
private void calculateItems(Result result, string nameOfString )
As you see from the code above, I have two methods have exactly same name, calculateItems, but public one has no parameter, private one has two parameters. I am trying to understand when I called public one in my unittest, is private method called?
private CalculateClass sut;
private Result result;
[SetUp]
public void Setup()
{
result = MockRepository.GenerateStub<Result>();
sut = new CalculateClass();
}
[TearDown]
public void TearDown()
{
}
[Test]
public void test()
{
sut.Stub(stub => stub.calculateItems(Arg<Result>.Is.Anything, Arg<string>.Is.Anything));
sut.calculateItems();
sut.AssertWasCalled(stub => stub.calculateItems(Arg<Result>.Is.Anything, Arg<string>.Is.Anything));
}
In my unittest, I am taking such an error which says "No overload method for calculateItems take two arguments". Is there a way to test it without any changing in source code?
You're testing the wrong thing. Private methods are private. They are of no concern to consuming code, and unit tests are consuming code like any other.
In your tests you test and validate the outward facing functionality of the component. Its inner implementation details aren't relevant to the tests. All the tests care about is whether the invoked operation produces the expected results.
So the question you must ask yourself is... What are the expected results when invoking this operation?:
calculateItems()
It doesn't return anything, so what does it do? What state does it modify in some way? That is what your test needs to observe, not the implementation details but the observable result. (And if the operation has no observable result, then there's no difference between "passed" or "failed" so there's nothing to test.)
We can't see the details of your code, but it's possible that the observable result is coupled to another component entirely. If that's the case then that other component is a dependency for this operation and the goal of the unit test is to mock that dependency so the operation can be tested independently of the dependency. The component may then need to be modified so that a dependency is provided rather than internally controlled. (This is referred to as the Dependency Inversion Principle.)
Also of note...
but I can not change the source code
That's a separate problem entirely. If you truly can't change the source code, then the value of these tests is drastically reduced and possibly eliminated entirely. If a test fails, what can you do about it? Nothing. Because you can't change the code. So what are you testing?
Keep in mind that it's not only possible but unfortunately very common for programmers to write code which can't be meaningfully unit tested. If this code was provided to you by someone else and you are forbidden to change it for some non-technical reason, then it will be the responsibility of that someone else to correct the code. "Correcting" may include "making it possible to meaningfully unit test". (Or, honestly, they should be unit testing it. Not you.)
If your public method calls your private one then the same thing will happen in your tests. Tests are nothing more than code that can be run and debugged and you can try that so see what happens.
Private methods can't be tested directly but they can be tested via their public callers which is what you are doing, so it's all good. Whether it's a good idea to have a setup like this well, that's a different story entirely but I am not going into that now.
Now, let's discuss what you are actually testing.
Unit tests should not have deep knowledge of the code they test. The reason is that you should have inputs and outputs and you shouldn't care what happens in between.
If you refactor the code and eliminate the private method then your test would break, even if your inputs and outputs to your public method remain the same. That's not a good position to be in, this is what we call brittle tests.
So add your functional tests around the public method, verify that you get hat you expect and don't worry whether it calls your private method or not.
When you say you need to know whether your private methods are called, this can have two different interpretations:
You want to ensure that the private method is called within one particular test, making it a success criterion for that very test.
You want to know if the private method is called at all, by any of your test cases. You might be interested in this because you want to be sure if the private method is covered by your test suite, or as you said, just to form an understanding of what is actually going on in your code.
Regarding the second interpretation: If you want to understand what is going on in the code, a good approach is to use a debugger and just step through the code to see what function is called. As I am not a C# expert here, I can not recommend any specific debugging tool, but finding some recommendations about this on the web should not be difficult. This approach would fulfill your requirements not to require changes to the source code
Another possibility, in particular if you are interested in whether your private function is covered by the tests, is to use a test coverage tool for C#. The coverage tool would show you whether or not the private method was called or not. Again, this would not require to make any changes to the source code.
Regarding the first interpretation of your question: If you want to test that some privat function is called as part of your test's success criterion, you preferrably do this with tests that use the public API. Then, in these tests, you should be able to judge if the private function is called because of the effect that the private function has on the test result.
And, in contrast to other opinions, you should test the implementation. The primary goal of unit-testing is to find the bugs in the code. Different implementations have different bugs. This is why people also use coverage tools, to see if they have covered the code of their implementation. And, coverage is not enough, you also need to check boundary cases of expressions etc. Certainly, having maintainable tests and tests that do not break unnecessarily in case of refactorings are good goals (why testing through the public API is typically a good approach - but not always), but they are secondary goals compared to the goal to find all bugs.
I've got a codebase that makes use of static variables in a number of cases where that makes sense, for example flagging that something's already run once since launch, etc etc.
Of course, this can lead to issues with unit testing whereupon suddenly order matters and the outcome of a test on a method of such a class may depend on whether other code has been hit before, etc. My understand of TestTools.UnitTesting is that whenever I run a set of unit tests, any within the same project run within the same process, thus any static state is maintained from test to test, whereas a Unit Test project boundary also implies a process boundary and thus, if I run 3 tests from project A then a fourth from project B, state is maintained from 1>2>3 (in whatever order they run) but then 4 is virgin and any static state is default.
So now my questions are two:
1) is my assessment correct that a unit test projects have a 1:1 relationship with processes when tests are run in a group (run all or run selected), or is there more nuance there that I'm missing?
2) Regardless, if I have a test that definitely needs fresh, default static state for the custom objects it uses and tests, do I have a more elegant option for creating it than giving it its own test project?
Statics are not actually per process, but per application domain, represented by the AppDomain class. A single process can have several AppDomains. AppDomains have their own statics, can provide sandboxing to partially trusted code, and can be unloaded allowing newer versions of the same assembly to be hot swapped without restarting the application.
Your test runner is likely creating a new AppDomain per test assembly so each assembly gets its own static variables. You can create an AppDomain to do the same on the fly. This is not typically great for pure unit tests, but I've had to work with "rude" libraries that do all kinds of static initialization and caching that cannot be cleaned out or reset. In those sorts of integration scenarios it is very useful.
You can use this helper to run a simple delegate:
public static class AppDomainHelper
{
public static void Run(Action action)
{
var domain = AppDomain.CreateDomain("test domain");
try
{
domain.DoCallBack(new CrossAppDomainDelegate(action));
}
finally
{
AppDomain.Unload(domain);
}
}
}
One caution is that the delegate action passed to Run cannot have any captured variables (as in from a lambda). That doesn't work because the compiler will generate a hidden class that is not serializable and so it cannot pass through an AppDomain boundary.
Your assessment is correct as far as I know -- the assembly is loaded at the start of the test process, and any static state is maintained throughout the tests.
You should always start with a "fresh" state. Unit tests should be able to be run in any order, with no dependencies whatsoever. The reason is because your tests need to be reliable -- a test should only ever fail for one reason: The code it's testing changed. If you have tests that depend on other tests, then you can easily end up with one test failing and "breaking the chain" such that a dozen other tests fail.
You can use the TestInitialize attribute to define a method that will run before every test that will reset the state to the baseline.
Another way to enable this is to wrap your static state into a singleton, then put a "back door" into the singleton so that you can inject an instance of the singleton class, allowing you to configure the state of the application as part of arranging your test.
If you're not looking to test global state, but rather what you do with values you get from global state, you can work around it.
Consider this simple class definition that uses some static property.
public class Foo
{
public int Bar(int baz)
{
return baz + GlobalState.StaticValue;
}
}
You can refactor it like this.
public class Foo
{
public virtual int GetGlobalStaticValue
{
return GlobalState.StaticValue;
}
public virtual int Bar(int baz)
{
return baz + this.GetGlobalStaticValue();
}
}
I added virtuals to the method definitions because that's particular to Rhino Mocks, but you get the idea - while running live, your class will pull global state as it does now, but it gives you the hooks to mock out the values that will be returned in your test scenario.
Let's say I write a private method which has a string as an argument and returns a string.
private string SomeMethod(string a)
{ ... }
After writing it, I want to make sure this method works correctly and input a sample string to see what the return value is. Now I test it the following way. I either:
comment out everything I have in main() function and call just the method SomeMethod() or
I call the method where it should be called in the code, set the breakpoint at that place and debug to see if the return value was correct.
I don't like either method. Is there any better way to simply check and test the method while developing code? I can't (shouldn't) write a unit test for this method as it's private.
If you just want to test it once after creating it, I would recommend to keep another instance of your Development Environment (e.g. Visual Studio) open, with a simple console application where you can just copy-paste the function and test it there.
If you want it regularly tested to ensure it doesn't go wrong with later changes, then I fear Unit testing is exactly what you need. You might use precompiler flags to disable the private scope for testing purposes:
#define TESTING
(...)
#if TESTING
public void foo()
#else
private void foo()
#endif
This is the academia approach for all first year classes at the University of Waterloo, I imagine a similar approach is taken in the real world (my first co-op is 2014 so I don't know just yet). I find it very useful but it takes some time.
If you create a class (henceforth mentioned as a module) then you need you create an associative driver module for each class. This driver file will consist of all your tests: testing edge cases and expected input to ensure your function(s) from your module behave as expected. Each driver module will have a main method and for your testing you will need to change the scope of your function from private to public (or call them all in the constructor).
Keep in mind we are programming in C and driver modules are much easier to make without changing scope (the occasional static for modular scope).
Put the logic in a public method, which you unit test while your are developing. When you are satisfied you have got the logic correct, you can copy and paste the logic in to your private method.
What is the best way to unit test a method that doesn't return anything? Specifically in c#.
What I am really trying to test is a method that takes a log file and parses it for specific strings. The strings are then inserted into a database. Nothing that hasn't been done before but being VERY new to TDD I am wondering if it is possible to test this or is it something that doesn't really get tested.
If a method doesn't return anything, it's either one of the following
imperative - You're either asking the object to do something to itself.. e.g change state (without expecting any confirmation.. its assumed that it will be done)
informational - just notifying someone that something happened (without expecting action or response) respectively.
Imperative methods - you can verify if the task was actually performed. Verify if state change actually took place. e.g.
void DeductFromBalance( dAmount )
can be tested by verifying if the balance post this message is indeed less than the initial value by dAmount
Informational methods - are rare as a member of the public interface of the object... hence not normally unit-tested. However if you must, You can verify if the handling to be done on a notification takes place. e.g.
void OnAccountDebit( dAmount ) // emails account holder with info
can be tested by verifying if the email is being sent
Post more details about your actual method and people will be able to answer better.
Update: Your method is doing 2 things. I'd actually split it into two methods that can now be independently tested.
string[] ExamineLogFileForX( string sFileName );
void InsertStringsIntoDatabase( string[] );
String[] can be easily verified by providing the first method with a dummy file and expected strings. The second one is slightly tricky.. you can either use a Mock (google or search stackoverflow on mocking frameworks) to mimic the DB or hit the actual DB and verify if the strings were inserted in the right location. Check this thread for some good books... I'd recomment Pragmatic Unit Testing if you're in a crunch.
In the code it would be used like
InsertStringsIntoDatabase( ExamineLogFileForX( "c:\OMG.log" ) );
Test its side-effects. This includes:
Does it throw any exceptions? (If it should, check that it does. If it shouldn't, try some corner cases which might if you're not careful - null arguments being the most obvious thing.)
Does it play nicely with its parameters? (If they're mutable, does it mutate them when it shouldn't and vice versa?)
Does it have the right effect on the state of the object/type you're calling it on?
Of course, there's a limit to how much you can test. You generally can't test with every possible input, for example. Test pragmatically - enough to give you confidence that your code is designed appropriately and implemented correctly, and enough to act as supplemental documentation for what a caller might expect.
As always: test what the method is supposed to do!
Should it change global state (uuh, code smell!) somewhere?
Should it call into an interface?
Should it throw an exception when called with the wrong parameters?
Should it throw no exception when called with the right parameters?
Should it ...?
Try this:
[TestMethod]
public void TestSomething()
{
try
{
YourMethodCall();
Assert.IsTrue(true);
}
catch {
Assert.IsTrue(false);
}
}
Void return types / Subroutines are old news. I haven't made a Void return type (Unless I was being extremely lazy) in like 8 years (From the time of this answer, so just a bit before this question was asked).
Instead of a method like:
public void SendEmailToCustomer()
Make a method that follows Microsoft's int.TryParse() paradigm:
public bool TrySendEmailToCustomer()
Maybe there isn't any information your method needs to return for usage in the long-run, but returning the state of the method after it performs its job is a huge use to the caller.
Also, bool isn't the only state type. There are a number of times when a previously-made Subroutine could actually return three or more different states (Good, Normal, Bad, etc). In those cases, you'd just use
public StateEnum TrySendEmailToCustomer()
However, while the Try-Paradigm somewhat answers this question on how to test a void return, there are other considerations too. For example, during/after a "TDD" cycle, you would be "Refactoring" and notice you are doing two things with your method... thus breaking the "Single Responsibility Principle." So that should be taken care of first. Second, you might have idenetified a dependency... you're touching "Persistent" Data.
If you are doing the data access stuff in the method-in-question, you need to refactor into an n-tier'd or n-layer'd architecture. But we can assume that when you say "The strings are then inserted into a database", you actually mean you're calling a business logic layer or something. Ya, we'll assume that.
When your object is instantiated, you now understand that your object has dependencies. This is when you need to decide if you are going to do Dependency Injection on the Object, or on the Method. That means your Constructor or the method-in-question needs a new Parameter:
public <Constructor/MethodName> (IBusinessDataEtc otherLayerOrTierObject, string[] stuffToInsert)
Now that you can accept an interface of your business/data tier object, you can mock it out during Unit Tests and have no dependencies or fear of "Accidental" integration testing.
So in your live code, you pass in a REAL IBusinessDataEtc object. But in your Unit Testing, you pass in a MOCK IBusinessDataEtc object. In that Mock, you can include Non-Interface Properties like int XMethodWasCalledCount or something whose state(s) are updated when the interface methods are called.
So your Unit Test will go through your Method(s)-In-Question, perform whatever logic they have, and call one or two, or a selected set of methods in your IBusinessDataEtc object. When you do your Assertions at the end of your Unit Test you have a couple of things to test now.
The State of the "Subroutine" which is now a Try-Paradigm method.
The State of your Mock IBusinessDataEtc object.
For more information on Dependency Injection ideas on the Construction-level... as they pertain to Unit Testing... look into Builder design patterns. It adds one more interface and class for each current interface/class you have, but they are very tiny and provide HUGE functionality increases for better Unit-Testing.
You can even try it this way:
[TestMethod]
public void ReadFiles()
{
try
{
Read();
return; // indicates success
}
catch (Exception ex)
{
Assert.Fail(ex.Message);
}
}
it will have some effect on an object.... query for the result of the effect. If it has no visible effect its not worth unit testing!
Presumably the method does something, and doesn't simply return?
Assuming this is the case, then:
If it modifies the state of it's owner object, then you should test that the state changed correctly.
If it takes in some object as a parameter and modifies that object, then your should test the object is correctly modified.
If it throws exceptions is certain cases, test that those exceptions are correctly thrown.
If its behaviour varies based on the state of its own object, or some other object, preset the state and test the method has the correct Ithrough one of the three test methods above).
If youy let us know what the method does, I could be more specific.
Use Rhino Mocks to set what calls, actions and exceptions might be expected. Assuming you can mock or stub out parts of your method. Hard to know without knowing some specifics here about the method, or even context.
Depends on what it's doing. If it has parameters, pass in mocks that you could ask later on if they have been called with the right set of parameters.
What ever instance you are using to call the void method , You can just use ,Verfiy
For Example:
In My case its _Log is the instance and LogMessage is the method to be tested:
try
{
this._log.Verify(x => x.LogMessage(Logger.WillisLogLevel.Info, Logger.WillisLogger.Usage, "Created the Student with name as"), "Failure");
}
Catch
{
Assert.IsFalse(ex is Moq.MockException);
}
Is the Verify throws an exception due to failure of the method the test would Fail ?
I'm using MSTEST inside Visual Studio 2008. How can I have each unit test method in a certain test class act as if it were the first test to run so that all global state is reset before running each test? I do not want to explicitly clean up the world using TestInitialize, ClassInitialize, AssemblyInitialize, etc. For example:
[TestClass]
public class MyClassTests
{
[TestMethod]
public void Test1()
{
// The "Instance" property creates a new instance of "SomeSingleton"
// if it hasn't been created before.
var i1 = SomeSingleton.Instance;
...
}
[TestMethod]
public void Test2()
{
// When I select "Test1" and "Test2" to run, I'd like Test2
// to have a new AppDomain feel so that the static variable inside
// of "SomeSingleton" is reset (it was previously set in Test1) on
// the call to ".Instance"
var i2 = SomeSingleton.Instance;
// some code
}
Although a similar question appeared on this topic, it only clarified that tests do not run in parallel. I realize that tests run serially, but there doesn't seem to be a way to explicitly force a new AppDomain for each method (or something equivalent to clear all state).
Ideally, I'd like to specify this behavior for only a small subset of my unit tests so that I don't have to pay the penalty of a new AppDomain creation for tests that don't care about global state (the vast majority of my tests).
In the end, I wrote a helper that used AppDomain.CreateDomain and then used reflection to call the unit test under a different AppDomain. It provides the isolation I needed.
This post on MSDN's forums shows how to handle the situation if you only have a few statics that need to be reset. It does mention some options (e.g. using Reflection and PrivateType ).
I continue to welcome any further ideas, especially if I'm missing something obvious about MSTEST.
Add a helper in your tests that uses reflection to delete the singleton instance (you can add a reset method to the singleton as well, but I would be concerned about its use). Something like:
public static class SingletonHelper {
public static void CleanDALFactory()
{
typeof(DalFactory)
.GetField("_instance",BindingFlags.Static | BindingFlags.NonPublic)
.SetValue(null, null);
}
}
Call this in your TestInitialize method. [ I know this is "cleaning up the world", but you only have to write the method once in a helper per singleton, its very trivial and gives you explicit control ]
I think you are looking for the TestIntialize attribute and the TestCleanUp attribute. Here is an MSDN blog showing the execution orderlink text
We had a similar issue arise with our MSTests. We handled it by calling a function at the beginning and end of the specific tests that needed it.
We are storing a test expiration date in our app configuration. Three tests needed this date to fall into a specific range to determine the appropriate values. The way our application is set up, the configuration values would only be reset if there was not a value assigned in session. So, we created two new private static functions - one to explicitly set the configuration value to a specified date and one to clear that date from session after the test runs. In our three tests, we called these two functions. When the next test runs, the application sees an empty value for the date and refetches it from the configuration file.
I'm not sure if that's helpful, but that was how we worked around our similar issue.