We Keep Coding

How to verify number of calls method of 'this' service

I'm using NUnit framework with moq for testing. I've got a problem with veryfing how many times private method of this class has been called. To do so with mock object it's enough to call Verify() with parameters of type Times, but my method is part of this class. I was trying to mock current service (SUT), but it probably isn't the best idea and it doesn't work properly.
SUT:
public object Post(Operations.Campaign.Merge request)
{
List<CampaignIdWithNumberOfAds> campaignList = new List<CampaignIdWithNumberOfAds>();
for (int i = 0; i < request.CampaignIdsToMerge.Count; i++)
{
if (this.CampaignRepository.Exist(request.CampaignIdsToMerge[i]))
{
campaignList.Add(new CampaignIdWithNumberOfAds()
{
CampaignId = request.CampaignIdsToMerge[i],
NumberOfAdvertisement = this.CampaignRepository.GetNumberOfAdvertisementsInCampaign(request.CampaignIdsToMerge[i])
});
}
}
if (campaignList.Count > 1)
{
campaignList = campaignList.OrderByDescending(p => (p == null) ? -1 : p.NumberOfAdvertisement).ToList();
List<CampaignIdWithNumberOfAds> campaignsToMerge = campaignList.Skip(1).ToList();
CampaignIdWithNumberOfAds chosenCampaign = campaignList.FirstOrDefault<CampaignIdWithNumberOfAds>();
uint chosenCampaignId = chosenCampaign.CampaignId;
foreach (var campaignToMerge in campaignsToMerge)
{
this.MergeCampaigns(chosenCampaignId, campaignToMerge.CampaignId);
}
}
return true;
}
Test:
[Test]
public void MergeCampaignsPost_ValidMergeCampaignsRequest_ExecuteMergeCampaignsMethodAppropriateNumberOfTimes()
{
// Arrange
var mockCampaignService = new Mock<Toucan.Api.Services.CampaignService>();
var request = Mother.GetValidMergeCampaignsRequest_WithDifferentNumbersOfAdvertisement();
mockCampaignService.Setup(x => x.MergeCampaigns(It.IsAny<uint>(), It.IsAny<uint>()));
// Act
var response = this.Service.Post(request);
// Assert
mockCampaignService.Verify(x => x.MergeCampaigns(It.IsAny<uint>(), It.IsAny<uint>()), Times.Exactly(request.CampaignIdsToMerge.Count - 1));
}

I am afraid that I won't give you a solution here, although I would rather suggest you some sort of guidance. There are many different strategies to unit testing and different people would suggest different solutions. Basically in my opinion you could change the way you are testing your code (you might agree or disagree with those, but please take them into consideration).
Unit test should be independent from the implementation
Easy as it sounds, it is very hard to keep to this approach. Private methods are your implementation of solving the problem. The typical pitfall for a developer writing a unit test for his own code is the fact that you know how your code works and mirror it in unit test. What if the implementation changes, but the public method will still fulfill the requested contract? Hardly ever you want to directly your unit test with a private method. This is related to the following...
Test should check the output result of the method
Which basically means do not check how many times something is executed if you don't have to. I am not sure what is your MergeCampaigns method doing but it would be better if you check the result of the operation instead of how many times it is executed.
Don't overdo your unit tests - keep it maintainable
Try to test each functional scenario you can imagine with as simple and as independent test as possible. Don't go too deep checking if something is called. Otherwise, you will get a 100% coverage at start, but you will curse each time changing a thing in your service, because this will make half of your test fail (assuming that the service is still doing its job, but in different way than designed at the beginning). So you will spend time rewriting unit tests that actually give you no gain in terms of creating a bulletproof solution.
It is very easy to start writing unit tests and keep the coverage green, it starts to get very tricky if you want to write good unit tests. There are many valuable resources to help with that. Good luck!

When is it OK to group similar unit tests?

I'm writing unit tests for a simple IsBoolean(x) function to test if a value is boolean. There's 16 different values I want to test.
Will I be burnt in hell, or mocked ruthlessly by the .NET programming community (which would be worse?), if I don't break them up into individual unit tests, and run them together as follows:
[TestMethod]
public void IsBoolean_VariousValues_ReturnsCorrectly()
{
//These should all be considered Boolean values
Assert.IsTrue(General.IsBoolean(true));
Assert.IsTrue(General.IsBoolean(false));
Assert.IsTrue(General.IsBoolean("true"));
Assert.IsTrue(General.IsBoolean("false"));
Assert.IsTrue(General.IsBoolean("tRuE"));
Assert.IsTrue(General.IsBoolean("fAlSe"));
Assert.IsTrue(General.IsBoolean(1));
Assert.IsTrue(General.IsBoolean(0));
Assert.IsTrue(General.IsBoolean(-1));
//These should all be considered NOT boolean values
Assert.IsFalse(General.IsBoolean(null));
Assert.IsFalse(General.IsBoolean(""));
Assert.IsFalse(General.IsBoolean("asdf"));
Assert.IsFalse(General.IsBoolean(DateTime.MaxValue));
Assert.IsFalse(General.IsBoolean(2));
Assert.IsFalse(General.IsBoolean(-2));
Assert.IsFalse(General.IsBoolean(int.MaxValue));
}
I ask this because "best practice" I keep reading about would demand I do the following:
[TestMethod]
public void IsBoolean_TrueValue_ReturnsTrue()
{
//Arrange
var value = true;
//Act
var returnValue = General.IsBoolean(value);
//Assert
Assert.IsTrue(returnValue);
}
[TestMethod]
public void IsBoolean_FalseValue_ReturnsTrue()
{
//Arrange
var value = false;
//Act
var returnValue = General.IsBoolean(value);
//Assert
Assert.IsTrue(returnValue);
}
//Fell asleep at this point
For the 50+ functions and 500+ values I'll be testing against this seems like a total waste of time.... but it's best practice!!!!!
-Brendan

I would not worry about it. This sort of thing isn't the point. JB Rainsberger talked about this briefly in his talk Integration Tests are a Scam. He said something like, "If you have never forced yourself to use one assert per test, I recommend you try it for a month. It will give you a new perspective on test, and teach you when it matters to have one assert per test, and when it doesn't". IMO, this falls into the doesn't matter category.
Incidentally, if you use nunit, you can use the TestCaseAttribute, which is a little nicer:
[TestCase(true)]
[TestCase("tRuE")]
[TestCase(false)]
public void IsBoolean_ValidBoolRepresentations_ReturnsTrue(object candidate)
{
Assert.That(BooleanService.IsBoolean(candidate), Is.True);
}
[TestCase("-3.14")]
[TestCase("something else")]
[TestCase(7)]
public void IsBoolean_InvalidBoolRepresentations_ReturnsFalse(object candidate)
{
Assert.That(BooleanService.IsBoolean(candidate), Is.False);
}
EDIT: wrote the tests in a slightly different way, that I think communicates intent a little better.

Although I agree it's best practice to separate the values in order to more easily identify the error. I think one still has to use their own common sense and follow such rules as guidelines and not as an absolute. You want to minimize assertion counts in a unit test, but what's generally most important is to insure a single concept per test.
In your specific case, given the simplicity of the function, I think that the one unit test you provided is fine. It's easy to read, simple, and clear. It also tests the function thoroughly and if ever it were to break somewhere down the line, you would be able to quickly identify the source and debug it.
As an extra note, in order to maintain good unit tests, you'll want to always keep them up to date and treat them with the same care as you do the actual production code. That's in many ways the greatest challenge. Probably the best reason to do Test Driven Development is how it actually allows you to program faster in the long run because you stop worrying about breaking the code that exists.

It's best practice to split each of the values you want to test into separate unit tests. Each unit test should be named specifically to the value you're passing and the expected result. If you were changing code and broke just one of your tests, then that test alone would fail and the other 15 would pass. This buys you the ability to instantly know what you broke without then having to debug the one unit test and find out which of the Asserts failed.
Hope this helps.

I can't comment on "Best Practice" because there is no such thing.
I agree with what Ayende Rahien says in his blog:
At the end, it boils down to the fact that I don’t consider tests to
be, by themselves, a value to the product. Their only value is their
binary ability to tell me whatever the product is okay or not.
Spending a lot of extra time on the tests distract from creating real
value, shippable software.
If you put them all in one test and this test fails "somewhere", then what do you do? Either your test framework will tell you exactly which line it failed on, or, failing that, you step through it with a debugger. The extra effort required because it's all in one function is negligible.
The extra value of knowing exactly which subset of tests failed in this particular instance is small, and overshadowed by the ponderous amount of code you had to write and maintain.

Think for a minute the reasons for breaking them up into individual tests. It's to isolate different functionality and to accurately identify all the things that went wrong when a test breaks. It looks like you might be testing two things: Boolean and Not Boolean, so consider two tests if your code follows two different paths. The bigger point, though, is that if none of the tests break, there are no errors to pinpoint.
If you keep running them, and later have one of these tests fail, that would be the time to refactor them into individual tests, and leave them that way.

How to keep my unit tests DRY when mocking doesn't work?

Edit:
It seems that by trying to provide some solutions to my own problem I blurred the whole problem. So I'm modifying the question little bit.
Suppose I have this class:
public class ProtocolMessage : IMessage
{
public IHeader GetProtocolHeader(string name)
{
// Do some logic here including returning null
// and throw exception in some cases
return header;
}
public string GetProtocolHeaderValue(string name)
{
IHeader header = GetProtocolHeader(name);
// Do some logic here including returning null
// and throw exception in some cases
return value;
}
}
It is actually not important what's going on in these methods. The important is that I have multiple unit tests to cover GetProtocolHeader method covering all situations (returning correct header, null or exception) and now I'm writing unit tests for GetProtocolHeaderValue.
If GetProtocolHeaderValue would be dependent on external dependency I would be able to mock it and inject it (I'm using Moq + NUnit). Then my unit test would just test expectation that external dependency was called and returned expected value. The external dependency would be tested by its own unit test and I would be done but how to correctly proceed in this example where method is not external dependency?
Clarification of the problem:
I believe my test suite for GetProtocolHeaderValue must test situation where GetProtocolHeader returns header, null or exception. So the main question is: Should I write tests where GetProtocolHeader will be really executed (some tests will be duplicated because they will test same code as tests for GetProtocolHeader itself) or should I use mocking approach described by #adrift and #Eric Nicholson where I will not run real GetProtoclHeader but just configure mock to return header, null or exception when this method is called?

In the call to GetProtocolHeaderValue, do you actually need to know whether or not it called GetProtocolHeader?
Surely it is enough to know that it is getting the correct value from the correct header. How it actually got it is irrelevant to the unit test.
You are testing units of functionality, the unit of functionality of GetProtocolHeaderValue is whether it returns the expected value, given a header name.
It is true that you may wish to guard against inappropriate caching or cross-contamination or fetching the value from a different header, but I don't think that testing that it has called GetProtocolHeader is the best way to do this. You can infer that it somehow fetched the right header from the fact that it returned the expected value for the header.
As long as you craft your tests and test data in such a way as to ensure that duplicate headers don't mask errors, then all should be well.
EDIT for updated question:
If GetProtocolHeader works quickly, reliably and is idempotent, then I still believe that there is no need to mock it. A shortfall in any of those three aspects is (IMO) the principal reason for mocking.
If (as I suspect from the question title), the reason you wish to mock it is that the preamble required to set up an appropriate state to return a real value is too verbose, and you'd rather not repeat it across the two tests, why not do it in the setup phase?
One of the roles performed by good unit tests is documentation.
If someone wishes to know how to use your class, they can examine the tests, and possibly copy and alter the test code to fit their purpose. This becomes difficult if the real idiom of usage has been obscured by the creation and injection of mocks.
Mocks can obscure potential bugs.
Let's say that GetProtocolHeader throws an exception if name is empty. You create a mock accordingly, and ensure that GetProtocolHeaderValue handles that exception appropriately. Later, you decide that GetProtocolHeader should return null for an empty name. If you forget to update your mock, GetProtocolHeaderValue("") will now behave differently in real life vs. the test suite.
Mocking might present an advantage if the mock is less verbose than the setup, but give the above points due consideration first.
Though you give three different GetProtocolHeader responses (header, null or exception) that GetProtocolHeaderValue needs to test, I imagine that the first one is likely to be "a range of headers". (e.g. What does it do with a header that is present, but empty? How does it treat leading and trailing whitespace? What about non-ASCII chars? Numbers?). If the setup for all of these is exceptionally verbose, it might be better to mock.

I often use a partial mock (in Rhino) or the equivalent (like CallsBaseMethod in FakeItEasy) to mock the actual class I'm testing. Then you can make GetProtocolHeader virtual and mock your calls to it. You could argue that it's violating the single responsibility principal, but that's still clearly very cohesive code.
Alternatively you could make a method like
internal static string GetProtocolHeaderValue(string name, IHeader header )
and test that processing independently. The public GetProtocolHeaderValue method wouldn't have any/many tests.
Edit: In this particular case, I'd also consider adding GetValue() as an extension method to IHeader. That would be very easy to read, and you could still do the null checking.

I'm probably missing something, but given the code listed it seems to me that you don't need to worry about whether its called or not.
Two possibilities exist:
That the GetProtocolHeader() method needs to be public in which case you write the set of tests that tell you whether it works as expected or not.
That its an implementation detail and doesn't need to be public except in so far as you want to be able to test it directly but in that case all you really care about is the set of tests that tell you whether GetProtocolHeaderValue() works as required.
In either case you are testing the exposed functionality and at the end of the day that's all that matters. If it were a dependency then yes you might be worrying about whether it was called but if its not the surely its an implemenation detail and not relevant?

With Moq, you can use CallBase to do the equivalent of a partial mock in Rhino Mocks.
In your example, change GetProtocolHeader to virtual, then create a mock of ProtocolMessage, setting CallBase = true. You can then setup the GetProtocolHeader method as you wish, and have your base class functionality of GetProtocolHeaderValue called.
See the Customizing Mock Behavior section of the moq quickstart for more details.

Why not simply change GetProtocolHeaderValue(string name) so that it calls 2 methods, the second one accepting a IHeader? That way, you can test all the // do some logic part in a separate test, via the RetrieveHeaderValue method, without having to worry about Mocks. Something like:
public string GetProtocolHeaderValue(string name)
{
IHeader header = GetProtocolHeader(name);
return RetrieveHeaderValue(IHeader header);
}
now you can test both parts of GetProtocolHeaderValue fairly easily. Now you still have the same problem testing that method, but the amount of logic in it has been reduced to a minimum.
Following the same line of thinking, these methods could be extracted in a IHeaderParser interface, and the GetProtocol methods would take in a IHeaderParser, which would be trivial to test/mock.
public string GetProtocolHeaderValue(string name, IHeaderParser parser)
{
IHeader header = parser.GetProtocolHeader(name);
return parser.HeaderValue(IHeader header);
}

Try the simplest thing that might work.
If the real GetProtocolHeader() implementation is quick and easy to control (e.g. to simulate header, null and exception cases), just use it.
If not (i.e. either the real implementation is time-consuming or you can easily simulate the 3 cases), then look at redesigning such that the constraints are eased.
I refrain from using Mocks unless absolutely required (e.g. file/network/external dependency), but as you may know this is just a personal choice not a rule. Ensure that the choice is worth the extra cognitive overhead (drop in readability) of the test.

It's all a matter of oppinion, pure tdd-ists will say no mocks, mockers will mock it all.
In my honest oppinion there is something wrong with the code you wrote, the GetProtocolHeader seems important enough not to be discarded as an implementation detail, as you defined it public.
The problem here lies within the second method GetProtocolHeaderValue that does not have the possibility to use an existing instance of IHeader
I would suggest a GetValue(string name) on IHeader interface

Test does not fail at first run

I have the following test:
[Test]
public void VerifyThat_WhenProvidingAServiceOrderWithALinkedAccountGetSerivceProcessWithStatusReasonOfEndOfEntitlementToUpdateStatusAndStopReasonForAccountGetServiceProcessesAndServiceOrders_TheProcessIsUpdatedWithAStatusReasonOfEndOfEntitlement()
{
IFixture fixture = new Fixture()
.Customize(new AutoMoqCustomization());
Mock<ICrmService> crmService = new Mock<ICrmService>();
fixture.Inject(crmService);
var followupHandler = fixture.CreateAnonymous<FollowupForEndOfEntitlementHandler>();
var accountGetService = fixture.Build<EndOfEntitlementAccountGetService>()
.With(handler => handler.ServiceOrders, new HashedSet<EndOfEntitlementServiceOrder>
{
{
fixture.Build<EndOfEntitlementServiceOrder>()
.With(order => order.AccountGetServiceProcess, fixture.Build<EndOfEntitlementAccountGetServiceProcess>()
.With(process => process.StatusReason, fixture.Build<StatusReason>()
.With(statusReason=> statusReason.Id == MashlatReasonStatus.Worthiness)
.CreateAnonymous())
.CreateAnonymous())
.CreateAnonymous()
}
})
.CreateAnonymous();
followupHandler.UpdateStatusAndStopReasonForAccountGetServiceProcessesAndServiceOrders(accountGetService);
crmService.Verify(svc => svc.Update(It.IsAny<DynamicEntity>()), Times.Never());
}
My problem is that it will never fail on the first run, like TDD specifies that it should.
What it should test is that whenever there is a certain value to a status for a process of a service order, perform no updates.
Is this test checking what it should?

I'm struggling a bit to understand the question here...
Is your problem that this test passes on the first try?
If yes, that means one of two things
your test has an error
you have already met this spec/requirement
Since the first has been ruled out, Green it is. Off you go to the next one on the list..
Somewhere down the line I assume, you will implement more functionality that results in the expected method being called. i.e. when the status value is different, perform an update.
The fix for that test must ensure that both tests pass.
If not, give me more information to help me understand.

Following TDD methodology, we only write new tests for functionality that doesn't exist. If a test passes on the first run, it is important to understand why.
One of my favorite things about TDD is its subtle ability to challenge our assumptions, and knock our egos flat. The practice of "Calling your Shots" is not only a great way to work through tests, but it's also a lot of fun. I love when a test fails when I expect it to pass - many great learning opportunities come from this; Time after time, evidence of working software trumps developer ego.
When a test passes when I think it shouldn't, the next step is to make it fail.
For example, your test, which expects that something doesn't happen, is guaranteed to pass if the implementation is commented out. Tamper with the logic that you think you are implementing by commenting it out or by altering the conditions of the implementation and verify if you get the same results.
If after doing this, and you're confident that the functionality is correct, write another test that proves the opposite. Will Update get called with different state or inputs?
With both sets in place, you should be able to comment out that feature and have the ability to know in advance which test will be impacted. (8-ball, corner pocket)
I would also suggest that you add another assertion to the above test to ensure that the subject and functionality under test is actually being invoked.

change the Times.Never() to Times.AtLeastOnce() and you got a good start for tdd.
Try to find nothing in nothing, well that's a good test ,but not they way to start tdd, first go with the simple specification, the naive operation the user could do (from your view point of course).
As you done some work, keep it for later, when it fails.

Mock Verify/VerifyAll before or after Assertion

I have been used to following code pattern while writing my test
public void TestMethod_Condition_Output()
{
//Arrange----------------
Mock<x> temp = new Mock<x>();
temp.setup.......
//Act--------------------
classinstance.TestMethod()
//Assert------------------
temp.VerifyAll();
Assert.AreNotEqual(.....)
}
I have been used to do the VerifyAll() before performing Assertions. But lately on some online examples, I have seen people doing Assertion first and then VerifyAll, if any. I do feel that my way is the correct way unless I am missing something.
Could you please alert me if I am missing anything.

In my opinion, the verify should come after the asserts. I want the asserts close to the invocation of the method under test as they are documenting what the method does. The verifications of the mock invocations are detailing how the class uses it's dependencies. This is less important to tie directly to the method itself.
In a sense the mocking of the dependencies becomes a wrapper around the actual test itself. This makes the test more understandable (to me, anyway, YMMV). My tests then follow this pattern:
Arrange
Mock
Set up expectations for dependencies
Set up expected results
Create class under test
Act
Invoke method under test
Assert
Assert actual results match expected results
Verify that expectations were met
I don't know that I would be pedantic about it, but this is the order that makes the most sense to me.

In a AAA style testing I do not use VerifyAll but rather than verify methods were called explicitly as part of the unit of test. Within the Arrange area I only setup methods that need to return a value.
using Rhino as an example...
//Arrange
mockedInterface.Stub(x => x.SomeMethod1()).Returns(2);
...
//Assert
mockedInterface.AssertWasCalled(x => x.SomeMethod1());
mockedInterface.AssertWasCalled(x => x.SomeMethod2());
Assert.AreEqual(...); // stanmdard NUnit asserttions
I do not need to setup the expected call to SomeMethod2() if it does not return anything.
With Loose mocks there is no real need to call VerifyAll as calls to other methods would not fail the test (unless a return is needed then it is required in the Arrange section).
The amount of assertions should be kept to a minimum (create more tests if it gets too large) and the order of them should not really matter either.