I'm trying to better understand how to use unit tests. I want to test Send and Receive in Xsockets with Nunit.
Right now when I run the test nothing is happening
I just want to create the listener and then call invoke, then wait and update the result value.
I want something like when you test an event
Unit testing that an event is raised in C#
Xsockets C#
[Test]
public void Xsockets_Change_Test()
{
var mocks = new MockRepository();
var metaData = mocks.CreateMock<MetaData>();
var result = 0;
controller.On<MetaData>("Change",
(message) =>
{
result++;
});
controller.Invoke("Change", metaData);
Assert.That(result, Is.EqualTo(1));
}
IListener On<T>(string target, Action<T> action);
void Invoke(string target, object data);
You should show in your code sample exactly what controller is and where it comes from, but looking at the XSockets link you provide I think I know what you are expecting.
The problem you are having is a common one when testing with threads, sockets, or any scenario where there is a continuity break in code execution. The solution is to either remove the discontinuity or account for it.
The problem lies in the fact that you don't know exactly when the event will be processed. In most cases (every case?) your assertion will be executed before the event handler and the test will fail.
In effect here you are testing both the XSocket functionality and your code. I have found that most times you do not want to test the XSocket library (at least not here - this is a unit test for your code). In this case you want to remove the XSocket code from the test. To do this you can mock (or stub) the controller to invoke your event code directly.
The other possibility is to account for the discontinuity by waiting for the result value to change for some (probably short) period of time and failing the test only if the change does not happen in the allotted time. I wouldn't usually recommend this approach, but it is justified occasionally if there is the desire to test the code with the socket functionality in place.
Related
I use Theory with MemberData like this:
[Theory]
[MemberData(nameof(TestParams))]
public void FijewoShortcutTest(MapMode mapMode)
{
...
and when it works, it is all fine, but when it fails XUnit iterates over all data I pass as parameters. In my case it is fruitless attempt, I would like to stop short -- i.e. when the first set of parameters make the test to fail, stop the rest (because they will fail as well -- again, it is my case, not general rule).
So how to tell XUnit to stop Theory on the first fail?
The point of a Theory is to have multiple independent tests running the same code of different data. If you only actually want one test, just use a Fact and iterate over the data you want to test within the method:
[Fact]
public void FijewoShortcutTest()
{
foreach (MapMode mapMode in TestParams)
{
// Test code here
}
}
That will mean you can't easily run just the test for just one MapMode though. Unless it takes a really long time to execute the tests for some reason, I'd just live with "if something is badly messed up, I get a lot of broken tests".
I am quite new at C# and also rhino mocks. I searched and found similar topics with my question but couldnt find a proper solution.
I am trying to understand if the private method is called or not in my unit test. I am using rhino mock, read many files about it, some of them just say that change the access specifier of the method from private to public, but I can not change the source code. I tried to link source file to my test project but it doesnt change.
public void calculateItems()
{
var result = new Result(fileName, ip, localPath, remotePath);
calculateItems(result, nameOfString);
}
private void calculateItems(Result result, string nameOfString )
As you see from the code above, I have two methods have exactly same name, calculateItems, but public one has no parameter, private one has two parameters. I am trying to understand when I called public one in my unittest, is private method called?
private CalculateClass sut;
private Result result;
[SetUp]
public void Setup()
{
result = MockRepository.GenerateStub<Result>();
sut = new CalculateClass();
}
[TearDown]
public void TearDown()
{
}
[Test]
public void test()
{
sut.Stub(stub => stub.calculateItems(Arg<Result>.Is.Anything, Arg<string>.Is.Anything));
sut.calculateItems();
sut.AssertWasCalled(stub => stub.calculateItems(Arg<Result>.Is.Anything, Arg<string>.Is.Anything));
}
In my unittest, I am taking such an error which says "No overload method for calculateItems take two arguments". Is there a way to test it without any changing in source code?
You're testing the wrong thing. Private methods are private. They are of no concern to consuming code, and unit tests are consuming code like any other.
In your tests you test and validate the outward facing functionality of the component. Its inner implementation details aren't relevant to the tests. All the tests care about is whether the invoked operation produces the expected results.
So the question you must ask yourself is... What are the expected results when invoking this operation?:
calculateItems()
It doesn't return anything, so what does it do? What state does it modify in some way? That is what your test needs to observe, not the implementation details but the observable result. (And if the operation has no observable result, then there's no difference between "passed" or "failed" so there's nothing to test.)
We can't see the details of your code, but it's possible that the observable result is coupled to another component entirely. If that's the case then that other component is a dependency for this operation and the goal of the unit test is to mock that dependency so the operation can be tested independently of the dependency. The component may then need to be modified so that a dependency is provided rather than internally controlled. (This is referred to as the Dependency Inversion Principle.)
Also of note...
but I can not change the source code
That's a separate problem entirely. If you truly can't change the source code, then the value of these tests is drastically reduced and possibly eliminated entirely. If a test fails, what can you do about it? Nothing. Because you can't change the code. So what are you testing?
Keep in mind that it's not only possible but unfortunately very common for programmers to write code which can't be meaningfully unit tested. If this code was provided to you by someone else and you are forbidden to change it for some non-technical reason, then it will be the responsibility of that someone else to correct the code. "Correcting" may include "making it possible to meaningfully unit test". (Or, honestly, they should be unit testing it. Not you.)
If your public method calls your private one then the same thing will happen in your tests. Tests are nothing more than code that can be run and debugged and you can try that so see what happens.
Private methods can't be tested directly but they can be tested via their public callers which is what you are doing, so it's all good. Whether it's a good idea to have a setup like this well, that's a different story entirely but I am not going into that now.
Now, let's discuss what you are actually testing.
Unit tests should not have deep knowledge of the code they test. The reason is that you should have inputs and outputs and you shouldn't care what happens in between.
If you refactor the code and eliminate the private method then your test would break, even if your inputs and outputs to your public method remain the same. That's not a good position to be in, this is what we call brittle tests.
So add your functional tests around the public method, verify that you get hat you expect and don't worry whether it calls your private method or not.
When you say you need to know whether your private methods are called, this can have two different interpretations:
You want to ensure that the private method is called within one particular test, making it a success criterion for that very test.
You want to know if the private method is called at all, by any of your test cases. You might be interested in this because you want to be sure if the private method is covered by your test suite, or as you said, just to form an understanding of what is actually going on in your code.
Regarding the second interpretation: If you want to understand what is going on in the code, a good approach is to use a debugger and just step through the code to see what function is called. As I am not a C# expert here, I can not recommend any specific debugging tool, but finding some recommendations about this on the web should not be difficult. This approach would fulfill your requirements not to require changes to the source code
Another possibility, in particular if you are interested in whether your private function is covered by the tests, is to use a test coverage tool for C#. The coverage tool would show you whether or not the private method was called or not. Again, this would not require to make any changes to the source code.
Regarding the first interpretation of your question: If you want to test that some privat function is called as part of your test's success criterion, you preferrably do this with tests that use the public API. Then, in these tests, you should be able to judge if the private function is called because of the effect that the private function has on the test result.
And, in contrast to other opinions, you should test the implementation. The primary goal of unit-testing is to find the bugs in the code. Different implementations have different bugs. This is why people also use coverage tools, to see if they have covered the code of their implementation. And, coverage is not enough, you also need to check boundary cases of expressions etc. Certainly, having maintainable tests and tests that do not break unnecessarily in case of refactorings are good goals (why testing through the public API is typically a good approach - but not always), but they are secondary goals compared to the goal to find all bugs.
I have a class which is basically a pipeline. It processes messages and then deletes them in batches. In order to do this the ProcessMessage() method doesn't directly delete messages; it adds them to a private Observable<IMessage>(). I then have another public method which watches that observable and deletes the messages en masse.
That results in code similar to:
public void CreateDeletionObservable(int interval = 30, int messageCount = 10)
{
this.processedMessages.Buffer(TimeSpan.FromSeconds(interval), messageCount).Subscribe(observer =>
{
client.Value.DeleteMessages(observer.ToList());
});
}
The problem is that my unit test doesn't have a value for processedMessages. I can't provide a moq'd value as it's private. I don't need to test what values are in processedMessages; I just need for them to exist in order to test that method's behavior. Specifically I need to test that my observable will continue running if an exception is thrown (that logic isn't in the code yet). As I see it I have a few options:
1) Refactor my class to use a single monster observable chain with a single entry point and a few exits (success, error, retry, etc.). This would avoid the use of private properties to pass collections around between public methods. However, that chain would be extremely difficult to parse much less unit test. I don't believe that making my code less readable and testable is a viable option.
2) Modify my CreateDeletionObservable method to accept a test list of Messages:
public void CreateDeletionObservable(int interval = 30, int messageCount = 10, IObservable<IMessage> processedMessages = null)
That would allow me to supply stubbed data for the method to use, but it's a horrible code smell. A variation on this is to inject that Observable at the constructor level, but that's no better. Possibly worse.
3) Make processedMessages public.
4) Don't test this functionality.
I don't like any of these options, but I'm leaning towards 2; injecting a list for testing purposes. Is there an option I'm missing here?
Your senses serve you well. I think in this case you can revert to guidance I find useful of "Test your boundaries" (Udi Dahan, but cant find the reference).
It seems that you can input message (via an Observable Sequence) and that as a side effect you will eventually delete these messages from the Client. So it seems that your test should read something like
"Given an EventProcessor, When 10 Messages are Processed, Then the Events are deleted from the client"
"Given an EventProcessor, When 5 Messages are Processed in 30s, Then the Events are deleted from the client"
So instead of testing this small part of the pipe that somehow knows about this.processedMessages (where did that instance come from?), test the chain. But this doesn't mean you need to create a massive unusable chain. Just create enough of the chain to make it testable.
Providing more of the code base would also help, e.g. where does this.processedMessages & client.Value come from? This is probably key and at a guess applying a more functional approach might help?
There seems to be a ton of advice for this sorta thing in the context of a GUI application. I think my particular scenario is different enough to warrent me asking. To sum up my question how do you test events?
Now for the long winded explanation. I've worked with Point of Service hardware for a little while now. This means that I had to write a few OPOS Service Objects. After a few years of doing that I managed to write my first COM visible C# service object and put it into production. I tried my best to unit test the entire project but found it rather difficult, but was able to come up with good unit tests for most all of the Service Object's interface implementation. The hardest part was the Event's part. Granted this scenario was the biggest and most common one that I faced, but I've come across similar scenarios in other applications of mine where testing for an event just seemed awkward. So to set the scene the Common Control object (CO) has at most 5 events that a person can subscribe too. When the CO calls the method Open OPOS finds the Service Object (SO) and creates an instance of it, then calls it's OpenService method. The third parameter of the SO is a reference to the CO. Now I don't have to define the entire CO, I only have to define the call back method for those 5 events. An example of the msr's definition is this
[ComImport, InterfaceType(ComInterfaceType.InterfaceIsDual), Guid("CCB91121-B81E-11D2-AB74-0040054C3719")]
internal interface MsrControlObject
{
[DispId(1)]
void SOData(int status);
[DispId(2)]
void SODirectIO(int eventNumber, ref int pData, ref string pString);
[DispId(3)]
void SOError(int resultCode, int resultCodeExtended, int errorLocus, ref int pErrorResponse);
[DispId(4)]
void SOOutputCompleteDummy(int outputId);
[DispId(5)]
void SOStatusUpdate(int data);
}
and my OpenService method would have this line of code
public class FakeMsrServiceObject : IUposBase
{
MsrControlObject _controlObject;
public int OpenService(string deviceClass, string deviceName, object dispatchObject)
{
_controlObject = (MsrControlObject)dispatchObject;
}
//example of how to fire an event
private void FireDataEvent(int status)
{
_controlObject.SODataEvent(status);
}
}
So I thought to myself for better testing, lets make a ControlObjectDispatcher. It will allow me to enqueue events, then fire them to the CO when conditions are correct. This is where I'm at. Now I know sorta how to test drive the implementation of it. But it just feels wrong. Lets take the DataEvent as an example. 2 conditions have to be met for a DataEvent to be fired. First the boolean property DataEventEnabled must be true, and the other boolean property FreezeEvents must be false. Also all events are strictly FIFO. So.. a Queue is perfect. And since I've written this before I know what the implementation will be. But writing a test for it that instills confidence to a new person to the project is difficult. Consider this pseudo code
[Test]
public void WhenMultipleEventsAreQueuedTheyAreFiredSequentiallyWhenConditionsAreCorrect()
{
_dispatcher.EnqueueDataEvent(new DataEvent(42));
_dispatcher.EnqueueStatusUpdateEvent(new StatusUpdateEvent(1));
Sleep(1000);
_spy.AssertNoEventsHaveFired();
_spy.AssertEventsCount(2);
_serviceObject.SetNumericProperty(PIDX_DataEventEnabled, 1);
_spy.AssertDataEventFired();
_spy.AssertStatusUpdateEventFired();
_serviceObject.GetnumericProperty(PIDX_DataEventEnabled).Should().BeEqualTo(0, "because firing a DataEvent sets DataEventEnabled to false");
}
Everyone reading this hear could wonder (without knowing the implementation) How do i know that say after 1 minute that this event fires? How do I know that that crazy Robert Snyder person didn't use a for loop and forget to exit the FireDataEvent routine after the iterations were all up? You really don't. Granted you could test for a minute.. but that defeats the purpose of a unit test.
So to sum up again... How does a person write a test for events? Events can fire whenever.. and they can sometimes take longer to process and fire then expected. I've seen in my integration tests for the first implementation of this where if I didn't sleep for say 50ms before asserting that an event was called then the test would fail with something like. expected data event to have fired, but was never fired
Are their any test frameworks built for events? Are their any common coding practices that cover this?
It’s a bit unclear if you’re looking to do unit testing or integration testing for your events, since you talk about both. However, given the tags on the question, I’m going to assume your primary interest is from a unit testing perspective. From a unit testing perspective it doesn’t make much difference if you are testing an event, or a normal method. The goal of the unit is to test individual chunks of functionality in isolation, so whilst having a sleep in an integration test might make sense (although I’d still try to avoid it and use some other kind of synchronisation where possible), in a unit test I’d take it as a flag that the functionality being tested hasn’t been isolated sufficiently.
So, for me, there’s two slices of event driven testing. Firstly you want to test that any events your class fires are fired when the appropriate conditions are met. Secondly you want to test that any handlers for the events perform the expected actions.
Testing the handlers behave as expected should be similar to any other test that you would right. You setup the handler to the expected state, set your expectations, call into it as if you were the event generator and then verify any relevant behaviour.
Testing that events are fired is essentially the same, setup the state that you would expect to fire an event, and then verify that an appropriately populated event is fired and any other state change takes place.
So, looking at your pseudo code, I would say that you have at least two tests:
// Setup (used for both tests)
// Test may be missing setup for dispatcher state == data event disabled.
_dispatcher.EnqueueDataEvent(new DataEvent(42));
_dispatcher.EnqueueStatusUpdateEvent(new StatusUpdateEvent(1));
// Sleep(1000); // This shouldn’t be in a unit test.
// Test 1 – VerifyThatDataAndStatusEventsDoNotFireWhenDataEventDisabled
_spy.AssertNoEventsHaveFired();
_spy.AssertEventsCount(2);
// Test 2 – VerifyThatEnablingDataEventFiresPendingDataEvents
_serviceObject.SetNumericProperty(PIDX_DataEventEnabled, 1);
_spy.AssertDataEventFired();
_spy.AssertStatusUpdateEventFired();
// Test 3? – This could be part of Test 2, or it could be a different test to VerifyThatDataEventsAreDisabledOnceADataEventHasBeenTriggered.
_serviceObject.GetnumericProperty(PIDX_DataEventEnabled).Should().BeEqualTo(0, "because firing a DataEvent sets DataEventEnabled to false");
Looking at the test code, there isn’t anything to suggest that you’ve implemented the actual serviceObject using any kind of for loop, or that a minute of testing would have any impact on the behaviour of the serviceObject. If you weren’t thinking about it from the perspective of event programming would you really be considering if calling into SetNumericProperty would result in you using ‘a for loop and forgeting to exit the FireDataEvent routine after the iterations were all up?’ It seems like if that’s the case then either SetNumericProperty wouldn’t return or you have a non-linear implementation and you’re possibly testing the wrong thing in the wrong place. Without seeing your event generation code it’s hard to advise on that though…
Events can fire whenever… and they can sometimes take longer to
process and fire then expected
Whilst this may be true when your application is running, it shouldn’t be true when you’re doing unit testing. Your unit tests should trigger events for defined conditions and test that those events have been triggered and been generated correctly. To achieve this you need to aim for unit isolation and you may have to accept that some thin elements need to be integration tested, rather than unit tested in order to achieve this isolation. So if you were dealing with events triggered from outside your app you may end up with something like this:
public interface IInternalEventProcessor {
void SOData(int status);
void SODirectIO(int eventNumber, int pData, string pString);
};
public class ExternalEventProcessor {
IInternalEventProcessor _internalProcessor;
public ExternalEventProcessor(IInternalEventProcessor internalProcessor, /*Ideally pass in interface to external system to allow unit testing*/) {
_internalProcessor = internalProcessor;
// Register event subscriptions with external system
}
public void SOData(int status) {
_internalProcessor.SOData(status);
}
void SODirectIO(int eventNumber, ref int pData, ref string pString) {
_internalProcessor.SODirectIO(eventNumber, pData, pString);
}
}
The purpose of the ExternalEventProcessor is to decouple the dependency on the external system so that unit testing of the event handling in the InternalEventProcessor is easier. Ideally, you would still be able to unit test the ExternalEventProcessor registers for events correctly and passes through by supplying mocks of the external system and the internal implementation, however if you can’t then because the class is cut down the bare minimum having integration testing only for this class might be a realistic option.
I have the following test:
[Test]
public void VerifyThat_WhenProvidingAServiceOrderWithALinkedAccountGetSerivceProcessWithStatusReasonOfEndOfEntitlementToUpdateStatusAndStopReasonForAccountGetServiceProcessesAndServiceOrders_TheProcessIsUpdatedWithAStatusReasonOfEndOfEntitlement()
{
IFixture fixture = new Fixture()
.Customize(new AutoMoqCustomization());
Mock<ICrmService> crmService = new Mock<ICrmService>();
fixture.Inject(crmService);
var followupHandler = fixture.CreateAnonymous<FollowupForEndOfEntitlementHandler>();
var accountGetService = fixture.Build<EndOfEntitlementAccountGetService>()
.With(handler => handler.ServiceOrders, new HashedSet<EndOfEntitlementServiceOrder>
{
{
fixture.Build<EndOfEntitlementServiceOrder>()
.With(order => order.AccountGetServiceProcess, fixture.Build<EndOfEntitlementAccountGetServiceProcess>()
.With(process => process.StatusReason, fixture.Build<StatusReason>()
.With(statusReason=> statusReason.Id == MashlatReasonStatus.Worthiness)
.CreateAnonymous())
.CreateAnonymous())
.CreateAnonymous()
}
})
.CreateAnonymous();
followupHandler.UpdateStatusAndStopReasonForAccountGetServiceProcessesAndServiceOrders(accountGetService);
crmService.Verify(svc => svc.Update(It.IsAny<DynamicEntity>()), Times.Never());
}
My problem is that it will never fail on the first run, like TDD specifies that it should.
What it should test is that whenever there is a certain value to a status for a process of a service order, perform no updates.
Is this test checking what it should?
I'm struggling a bit to understand the question here...
Is your problem that this test passes on the first try?
If yes, that means one of two things
your test has an error
you have already met this spec/requirement
Since the first has been ruled out, Green it is. Off you go to the next one on the list..
Somewhere down the line I assume, you will implement more functionality that results in the expected method being called. i.e. when the status value is different, perform an update.
The fix for that test must ensure that both tests pass.
If not, give me more information to help me understand.
Following TDD methodology, we only write new tests for functionality that doesn't exist. If a test passes on the first run, it is important to understand why.
One of my favorite things about TDD is its subtle ability to challenge our assumptions, and knock our egos flat. The practice of "Calling your Shots" is not only a great way to work through tests, but it's also a lot of fun. I love when a test fails when I expect it to pass - many great learning opportunities come from this; Time after time, evidence of working software trumps developer ego.
When a test passes when I think it shouldn't, the next step is to make it fail.
For example, your test, which expects that something doesn't happen, is guaranteed to pass if the implementation is commented out. Tamper with the logic that you think you are implementing by commenting it out or by altering the conditions of the implementation and verify if you get the same results.
If after doing this, and you're confident that the functionality is correct, write another test that proves the opposite. Will Update get called with different state or inputs?
With both sets in place, you should be able to comment out that feature and have the ability to know in advance which test will be impacted. (8-ball, corner pocket)
I would also suggest that you add another assertion to the above test to ensure that the subject and functionality under test is actually being invoked.
change the Times.Never() to Times.AtLeastOnce() and you got a good start for tdd.
Try to find nothing in nothing, well that's a good test ,but not they way to start tdd, first go with the simple specification, the naive operation the user could do (from your view point of course).
As you done some work, keep it for later, when it fails.