We Keep Coding

How do I get NSubstitute mocks to fail when the arguments don't match the given pattern?

I'm responsible for the testing of a legacy software developed in C and C# that my team is maintaining. The original team used NSubstitute 3.1 to create test doubles for delegates in order to perform unit test of the APIs for the C# sections. Here's one such test double, where irrelevant details have been omitted:
private static byte[] MockSelectByAidWithoutData(ushort retVal)
{
var expectedIn= "FFFEFDFCFB".HexToBytes();
var expectedOut= "010203040506070809".HexToBytes();
var fake = Substitute.For<SomeDelegate>();
fake(Arg.Is<byte[]>(x => expectedIn.SequenceEqual(x.Take(expectedIn.Length))),
Arg.Is(0x00),
Arg.Is(expectedIn.Length),
Arg.Any<int>(),
Arg.Any<int>(),
out int outputLength)
.Returns(x =>
{
expectedOut.CopyTo((Array)x[0], 0);
x[5] = expectedOut.Length;
return retVal;
}
);
Mediator.GetInstance().Delegate = fake;
return expectedOut;
}
Now, if the fake delegate is invoked with arguments that match what is specified in the fake() call, it returns the retVal value and everybody is happy. However, if some value won't match, it returns zero. Since zero is a valid but incorrect value, the execution continues and I get an error that is not the root cause of the issue I am testing (i.e. bad output when the problem is actually bad input)
I am looking for a way to either:
specify a "catch all" behaviour for the values that won't match the expectations, or
get an exception if the arguments don't match the expectation
so that the test case would fail immediately upon reception of the wrong input with a meaningful message and without triggering further behaviour that would just pollute the outcome of the test.
Thanks in advance,
DeK
P.S. I can probably switch safely to a more recent version of NSubstitute if that's really necessary.

specify a "catch all" behaviour for the values that won't match the expectations
I think I've found a way you can do this. If you first stub the "catch all" / failure case for all arguments, you can then stub more specific calls. NSubstitute will try to match the most recent specifications provided, falling back to earlier stubbed values.
Here is a sample.
Note it is using Configure from NSubstitute.Extensions namespace introduced in NSubstitute 4.x. This isn't strictly necessary because NSubstitute will automatically assume you are configuring a call if you are using argument matchers, but it is a good pattern to use when configuring overlapping calls like this.
using NSubstitute;
using NSubstitute.Extensions; // required for Configure()
public class Thing {
public string Id { get; set; }
}
public interface ISample {
int Example(Thing a, string b);
}
public class UnexpectedCallException : Exception { }
[Fact]
public void ExampleOfStubOneCallButFailOthers() {
var sub = Substitute.For<ISample>();
// Catch all case:
sub.Example(null, null).ReturnsForAnyArgs(x => throw new UnexpectedCallException());
// Specific case. We use Configure from NSubstitute.Extensions to
// be able to stub this without getting an UnexpectedCallException.
// Not strictly necessary here as we're using argument matchers so NSub
// already knows we're configuring a call, but it's a good habit to get into.
// See: https://nsubstitute.github.io/help/configure/
sub.Configure()
.Example(Arg.Is<Thing>(x => x.Id == "abc"), Arg.Any<string>())
.Returns(x => 42);
// Example of non-matching call:
Assert.Throws<UnexpectedCallException>(() =>
sub.Example(new Thing { Id = "def" }, "hi")
);
// Example of matching call:
Assert.Equal(42, sub.Example(new Thing { Id = "abc" }, "hello"));
}
You could extend this to include information about arguments that do not match, but that will be a bit of custom work. If you look at some of NSubstitute's argument formatting code that might be re-usable to help with this.
Update to include delegate example
I just ran this with a delegate instead and it also passes:
public delegate int SomeDelegate(Thing a, string b);
[Fact]
public void ExampleOfStubOneDelegateCallButFailOthers() {
var sub = Substitute.For<SomeDelegate>();
sub(null, null).ReturnsForAnyArgs(x => throw new UnexpectedCallException());
sub.Configure()
.Invoke(Arg.Is<Thing>(x => x.Id == "abc"), Arg.Any<string>())
.Returns(x => 42);
Assert.Throws<UnexpectedCallException>(() => sub(new Thing { Id = "def" }, "hi"));
Assert.Equal(42, sub(new Thing { Id = "abc" }, "hello"));
}

Create multiple instances of the same type with different behaviour in Moq (with Autofac)

I'm trying to write a unit test over an class that has 4 instances of the same object (object is an interface to hardware, the class is a manager of a configuration of hardware). I've used Autofac, so I'm using the Autofac.Extras.Moq library.
I need to have multiple instances of mocks, with different behaviour (basically I'm trying to test behaviour when a device fails by setting a property. I'm trying hard but I can see how to configure a mock to do what I want, but when I change the behaviour, it's changing all instances of that mock, not just the mocks.
using(mock = AutoMock.GetLoose())
{
var goodHW = mock.Create<IHW>();
((Mock<IHW>)goodHW).Setup(x => x.OK).Returns(true);
var badHW = mock.Create<IHW>();
((Mock<IHW>)badHW).Setup(x => x.OK).Returns(false);
mock.Mock<IHWManager>().SetupGet(x => x.HW1).Returns(goodHW);
mock.Mock<IHWManager>().SetupGet(x => x.HW2).Returns(badHW);
Assert.AreNotEqual(goodHW, badHW) //FAILS!!!
}
As the two mocks are actually the same object, the goodHW instance returns false. I can use a sequence, but that ties the test logic to the implementation logic significantly (order of calls etc.)
Is what I'm asking possible?

You could try creating two mock objects (call GetLoose()twice) and set them up differently; then use one to create goodHW and the other to create badHW
Something like this:
using(mockGood = AutoMock.GetLoose())
using(mockBad = AutoMock.GetLoose())
{
var goodHW = mockGood.Create<IHW>();
((Mock<IHW>)goodHW).Setup(x => x.OK).Returns(true);
var badHW = mockBad.Create<IHW>();
((Mock<IHW>)badHW).Setup(x => x.OK).Returns(false);
mockGood.Mock<IHWManager>().SetupGet(x => x.HW1).Returns(goodHW);
mockBad.Mock<IHWManager>().SetupGet(x => x.HW2).Returns(badHW);
Assert.AreNotEqual(goodHW, badHW) // SUCCESS??
}

Fluentvalidation - tie validation to specific process best practice

what would be the best practice for tying validation rules to specific process using http://fluentvalidation.codeplex.com/
Currently I'm using "Rule Sets" feature to group rules to different processes:
public class ObjAValidation: AbstractValidator<A>
{
public ObjAValidation()
{
RuleSet("ProcessA", () =>
{
RuleFor(x => ...);
RuleFor(x => ...);
});
RuleSet("ProcessB", () =>
{
RuleFor(x => ...);
RuleFor(x => ...);
});
}
}
And then validate using:
var a = new A(){...};
IValidator<A> validator = new ObjAValidation();
var result = validator.Validate(a, ruleSet: "ProcessA");
I have two problems with this approach:
I don't like to use strings as process names. I would like to use a
more strongly typed approach. For example to be able to use marker
interfaces or attributes.
In my Unit tests I can't setup the Validate method of IValidator
because you can't use optional arguments with Moq.
Mock<IValidator<A>> _mockValidator = new Mock<IValidator<A>>();
_mockValidator.Setup(x => x.Validate(new A(), ruleSet: "ProcessA"));
Second line generates a run time error: An expression tree may not contain a named argument specification. And if you want to pass the ruleSet argument without a named argument to Validate method you have to provide a "IValidatorSelector selector" object. But this interface is not documented.

What prevents you to create a helper class where you can use variables, data structures, anything you like to prevent the usage of hard-coded string parameters? Also, don't forget the possibility of using enums.
What prevents you from creating a class which implements IValidator where you can also implement custom functionality needed by you?

I have encountered the same issue and found out the reason why you can not mock this method via Moq etc validator.Validate(a, ruleSet: "ProcessA"); is because it is an extension method and Moq can't mock static methods.
Someone raised the same issue here in the FluentValidation issues: https://github.com/JeremySkinner/FluentValidation/issues/191
The simple solution is not to use the extension method rather use the instant method IValidator.validate(context). All you need to do is to build the context. Check out the source code from here: https://github.com/JeremySkinner/FluentValidation/blob/master/src/FluentValidation/DefaultValidatorExtensions.cs#L819
if(ruleSet != null) {
var ruleSetNames = ruleSet.Split(',', ';').Select(x => x.Trim());
selector = ValidatorOptions.ValidatorSelectors.RulesetValidatorSelectorFactory(ruleSetNames.ToArray());
}
var context = new ValidationContext<T>(instance, new PropertyChain(), selector);
return validator.Validate(context);

Using Moq to verify calls are made in the correct order

I need to test the following method:
CreateOutput(IWriter writer)
{
writer.Write(type);
writer.Write(id);
writer.Write(sender);
// many more Write()s...
}
I've created a Moq'd IWriter and I want to ensure that the Write() methods are called in the right order.
I have the following test code:
var mockWriter = new Mock<IWriter>(MockBehavior.Strict);
var sequence = new MockSequence();
mockWriter.InSequence(sequence).Setup(x => x.Write(expectedType));
mockWriter.InSequence(sequence).Setup(x => x.Write(expectedId));
mockWriter.InSequence(sequence).Setup(x => x.Write(expectedSender));
However, the second call to Write() in CreateOutput() (to write the id value) throws a MockException with the message "IWriter.Write() invocation failed with mock behavior Strict. All invocations on the mock must have a corresponding setup.".
I'm also finding it hard to find any definitive, up-to-date documentation/examples of Moq sequences.
Am I doing something wrong, or can I not set up a sequence using the same method?
If not, is there an alternative I can use (preferably using Moq/NUnit)?

There is bug when using MockSequence on same mock. It definitely will be fixed in later releases of Moq library (you can also fix it manually by changing Moq.MethodCall.Matches implementation).
If you want to use Moq only, then you can verify method call order via callbacks:
int callOrder = 0;
writerMock.Setup(x => x.Write(expectedType)).Callback(() => Assert.That(callOrder++, Is.EqualTo(0)));
writerMock.Setup(x => x.Write(expectedId)).Callback(() => Assert.That(callOrder++, Is.EqualTo(1)));
writerMock.Setup(x => x.Write(expectedSender)).Callback(() => Assert.That(callOrder++, Is.EqualTo(2)));

I've managed to get the behaviour I want, but it requires downloading a 3rd-party library from http://dpwhelan.com/blog/software-development/moq-sequences/
The sequence can then be tested using the following:
var mockWriter = new Mock<IWriter>(MockBehavior.Strict);
using (Sequence.Create())
{
mockWriter.Setup(x => x.Write(expectedType)).InSequence();
mockWriter.Setup(x => x.Write(expectedId)).InSequence();
mockWriter.Setup(x => x.Write(expectedSender)).InSequence();
}
I've added this as an answer partly to help document this solution, but I'm still interested in whether something similar could be achieved using Moq 4.0 alone.
I'm not sure if Moq is still in development, but fixing the problem with the MockSequence, or including the moq-sequences extension in Moq would be good to see.

I wrote an extension method that will assert based on order of invocation.
public static class MockExtensions
{
public static void ExpectsInOrder<T>(this Mock<T> mock, params Expression<Action<T>>[] expressions) where T : class
{
// All closures have the same instance of sharedCallCount
var sharedCallCount = 0;
for (var i = 0; i < expressions.Length; i++)
{
// Each closure has it's own instance of expectedCallCount
var expectedCallCount = i;
mock.Setup(expressions[i]).Callback(
() =>
{
Assert.AreEqual(expectedCallCount, sharedCallCount);
sharedCallCount++;
});
}
}
}
It works by taking advantage of the way that closures work with respect to scoped variables. Since there is only one declaration for sharedCallCount, all of the closures will have a reference to the same variable. With expectedCallCount, a new instance is instantiated each iteration of the loop (as opposed to simply using i in the closure). This way, each closure has a copy of i scoped only to itself to compare with the sharedCallCount when the expressions are invoked.
Here's a small unit test for the extension. Note that this method is called in your setup section, not your assertion section.
[TestFixture]
public class MockExtensionsTest
{
[TestCase]
{
// Setup
var mock = new Mock<IAmAnInterface>();
mock.ExpectsInOrder(
x => x.MyMethod("1"),
x => x.MyMethod("2"));
// Fake the object being called in order
mock.Object.MyMethod("1");
mock.Object.MyMethod("2");
}
[TestCase]
{
// Setup
var mock = new Mock<IAmAnInterface>();
mock.ExpectsInOrder(
x => x.MyMethod("1"),
x => x.MyMethod("2"));
// Fake the object being called out of order
Assert.Throws<AssertionException>(() => mock.Object.MyMethod("2"));
}
}
public interface IAmAnInterface
{
void MyMethod(string param);
}

The simplest solution would be using a Queue:
var expectedParameters = new Queue<string>(new[]{expectedType,expectedId,expectedSender});
mockWriter.Setup(x => x.Write(expectedType))
.Callback((string s) => Assert.AreEqual(expectedParameters.Dequeue(), s));

Recently, I put together two features for Moq: VerifyInSequence() and VerifyNotInSequence(). They work even with Loose Mocks. However, these are only available in a moq repository fork:
https://github.com/grzesiek-galezowski/moq4
and await more comments and testing before deciding on whether they can be included in official moq releaase. However, nothing prevents you from downloading the source as ZIP, building it into a dll and giving it a try. Using these features, the sequence verification you need could be written as such:
var mockWriter = new Mock<IWriter>() { CallSequence = new LooseSequence() };
//perform the necessary calls
mockWriter.VerifyInSequence(x => x.Write(expectedType));
mockWriter.VerifyInSequence(x => x.Write(expectedId));
mockWriter.VerifyInSequence(x => x.Write(expectedSender));
(note that you can use two other sequences, depending on your needs. Loose sequence will allow any calls between the ones you want to verify. StrictSequence will not allow this and StrictAnytimeSequence is like StrictSequence (no method calls between verified calls), but allows the sequence to be preceeded by any number of arbitrary calls.
If you decide to give this experimental feature a try, please comment with your thoughts on:
https://github.com/Moq/moq4/issues/21
Thanks!

I've just had a similar scenario, and inspired by the accepted answer, I've used the following approach:
//arrange
var someServiceToTest = new SomeService();
var expectedCallOrder = new List<string>
{
"WriteA",
"WriteB",
"WriteC"
};
var actualCallOrder = new List<string>();
var mockWriter = new Mock<IWriter>();
mockWriter.Setup(x => x.Write("A")).Callback(() => { actualCallOrder.Add("WriteA"); });
mockWriter.Setup(x => x.Write("B")).Callback(() => { actualCallOrder.Add("WriteB"); });
mockWriter.Setup(x => x.Write("C")).Callback(() => { actualCallOrder.Add("WriteC"); });
//act
someServiceToTest.CreateOutput(_mockWriter.Object);
//assert
Assert.AreEqual(expectedCallOrder, actualCallOrder);

Moq has a little-known feature called Capture.In, which can capture arguments passed to a method. With it, you can verify call order like this:
var calls = new List<string>();
var mockWriter = new Mock<IWriter>();
mockWriter.Setup(x => x.Write(Capture.In(calls)));
CollectionAssert.AreEqual(calls, expectedCalls);
If you have overloads with different types, you can run the same setup for overloads too.

My scenario was methods without parameters:
public interface IWriter
{
void WriteA ();
void WriteB ();
void WriteC ();
}
So I used Invocations property on the Mock to compare what was called:
var writer = new Mock<IWriter> ();
new SUT (writer.Object).Run ();
Assert.Equal (
writer.Invocations.Select (invocation => invocation.Method.Name),
new[]
{
nameof (IWriter.WriteB),
nameof (IWriter.WriteA),
nameof (IWriter.WriteC),
});
You could also append the invocation.Arguments to check method calls with parameters.
Also the failure message is more clear than just expected 1 but was 5:
expected
["WriteB", "WriteA", "WriteC"]
but was
["WriteA", "WriteB"]

I suspect that expectedId is not what you expect.
However i'd probably just write my own implementation of IWriter to verify in this case ... probably a lot easier (and easier to change later).
Sorry for no Moq advice directly. I love it, but haven't done this in it.
do you maybe need to add .Verify() at the end of each setup? (That really is a guess though i'm afraid).

I am late to this party but I wanted to share a solution that worked for me since it seems as though all of the referenced solutions did not work with verifying the same method call (with the same arguments) multiple times in order. In addition the referenced bug, Moq Issue #478 was closed without a solution.
The solution presented utilizes the MockObject.Invocations list to determine order and sameness.
public static void VerifyInvocations<T>(this Mock<T> mock, params Expression<Action<T>>[] expressions) where T : class
{
Assert.AreEqual(mock.Invocations.Count, expressions.Length,
$"Number of invocations did not match expected expressions! Actual invocations: {Environment.NewLine}" +
$"{string.Join(Environment.NewLine, mock.Invocations.Select(i => i.Method.Name))}");
for (int c = 0; c < mock.Invocations.Count; c++)
{
IInvocation expected = mock.Invocations[c];
MethodCallExpression actual = expressions[c].Body as MethodCallExpression;
// Verify that the same methods were invoked
Assert.AreEqual(expected.Method, actual.Method, $"Did not invoke the expected method at call {c + 1}!");
// Verify that the method was invoked with the correct arguments
CollectionAssert.AreEqual(expected.Arguments.ToList(),
actual.Arguments
.Select(arg =>
{
// Expressions treat the Argument property as an Expression, do this to invoke the getter and get the actual value.
UnaryExpression objectMember = Expression.Convert(arg, typeof(object));
Expression<Func<object>> getterLambda = Expression.Lambda<Func<object>>(objectMember);
Func<object> objectValueGetter = getterLambda.Compile();
return objectValueGetter();
})
.ToList(),
$"Did not invoke step {c + 1} method '{expected.Method.Name}' with the correct arguments! ");
}
}

Need help to understand Moq better

I've been looking at the Moq documentation and the comments are too short for me to understand each of things it can do.
The first thing I don't get is It.IsAny<string>(). //example using string
Is there an advantage of using this over just putting some value in? I know people say to use this if you don't care about the value, but if you don't care about the value can't you just do "a" or something? This just seems like more typing.
Secondly, when would be an example be of when you would not care about the value? I thought Moq needs the value to match up stuff.
I don't get what It.Is<> is for at all or how to use it. I don't understand the example and what it is trying to show.
Next, I don't get when to use Times (and its AtMost methods and similar). Why would you limit the number of times something is set up? I have some AppConfig value that I need to use twice. Why would I want to limit it to, say, once? This would just make the test fail. Is this to stop other people from adding another one to your code or something?
I don't get how to use mock.SetupAllProperties();
What does it set up the properties with?
I don't also get why there are so many different ways to set up a property and what their differences are. The documentation has:
SetupGet(of property)
SetupGet<TProperty>
I noticed that a lot of the stuff in Moq shows () and <> - what's the difference between them and what would they look like in use?
I also don't get why they have SetupGet. Would you not use SetupSet to set a property?
SetupSet has five different ways to use it in the documentation. Plus another one called SetupProperty. So I don't understand why there are so many.
On a side note, I am wondering if variables used in lambdas are independent of other lambdas. E.g.:
mock.setup(m => m.Test);
stop.setup(m => m.Test);
Would this be ok or would there be some conflict between the variable m?
Finally, I was watching this video and I am wondering if it shows Visual Studio. His Intellisense looks different. A lightbulb pops up for him (I am happy mine does not, as it brings back painful memories of netbeans), and there are lines going from one opening brace to the closing brace and etc.

It.IsAny / It.Is
These can be useful when you're passing a new reference type within the code under test. For instance, if you had a method along the lines of:
public void CreatePerson(string name, int age) {
Person person = new Person(name, age);
_personRepository.Add(person);
}
You might want to check the add method has been called on the repository,
[Test]
public void Create_Person_Calls_Add_On_Repository () {
Mock<IPersonRepository> mockRepository = new Mock<IPersonRepository>();
PersonManager manager = new PersonManager(mockRepository.Object);
manager.CreatePerson("Bob", 12);
mockRepository.Verify(p => p.Add(It.IsAny<Person>()));
}
If you wanted to make this test more explicit you can use It.Is by supplying a predicate the person object must match,
[Test]
public void Create_Person_Calls_Add_On_Repository () {
Mock<IPersonRepository> mockRepository = new Mock<IPersonRepository>();
PersonManager manager = new PersonManager(mockRepository.Object);
manager.CreatePerson("Bob", 12);
mockRepository.Verify(pr => pr.Add(It.Is<Person>(p => p.Age == 12)));
}
This way the test will through an exception if the person object that was used to call the add method didn't have the age property set to 12.
Times
If you had a method along the lines of:-
public void PayPensionContribution(Person person) {
if (person.Age > 65 || person.Age < 18) return;
//Do some complex logic
_pensionService.Pay(500M);
}
One of the things that you might want to test is that the pay method does not get called when a person aged over 65 is passed into the method
[Test]
public void Someone_over_65_does_not_pay_a_pension_contribution() {
Mock<IPensionService> mockPensionService = new Mock<IPensionService>();
Person p = new Person("test", 66);
PensionCalculator calc = new PensionCalculator(mockPensionService.Object);
calc.PayPensionContribution(p);
mockPensionService.Verify(ps => ps.Pay(It.IsAny<decimal>()), Times.Never());
}
Similarly, it's possible to imagine situations where you're iterating over a collection and calling a method for each item in the collection and you'd like to make sure that it's been called a certain amount of times, other times you simply don't care.
SetupGet / SetupSet
What you need to be aware of with these guys is that they reflect how your code is interacting with the mock rather than how you're setting up the mock
public static void SetAuditProperties(IAuditable auditable) {
auditable.ModifiedBy = Thread.CurrentPrincipal.Identity.Name;
}
In this case, the code is setting the ModifiedBy property of the IAuditable instance while it's getting the Name property of the current instance of IPrincipal,
[Test]
public void Accesses_Name_Of_Current_Principal_When_Setting_ModifiedBy() {
Mock<IPrincipal> mockPrincipal = new Mock<IPrincipal>();
Mock<IAuditable> mockAuditable = new Mock<IAuditable>();
mockPrincipal.SetupGet(p => p.Identity.Name).Returns("test");
Thread.CurrentPrincipal = mockPrincipal.Object;
AuditManager.SetAuditProperties(mockAuditable.Object);
mockPrincipal.VerifyGet(p => p.Identity.Name);
mockAuditable.VerifySet(a => a.ModifiedBy = "test");
}
In this case, we're setting up the name property on the mock of IPrincipal so it returns "test" when the getter is called on the Name property of Identity we're not setting the property itself.
SetupProperty / SetupAllProperties
Looking at the test above if it was changed to read
[Test]
public void Accesses_Name_Of_Current_Principal_When_Setting_ModifiedBy() {
Mock<IPrincipal> mockPrincipal = new Mock<IPrincipal>();
Mock<IAuditable> mockAuditable = new Mock<IAuditable>();
mockPrincipal.SetupGet(p => p.Identity.Name).Returns("test");
var auditable = mockAuditable.Object;
Thread.CurrentPrincipal = mockPrincipal.Object;
AuditManager.SetAuditProperties(auditable);
Assert.AreEqual("test", auditable.ModifiedBy);
}
The test would fail. This is because the proxy created by Moq doesn't actually do anything in the set method of a property unless you tell it to. In effect, the mock object looks a bit like this
public class AuditableMock : IAuditable {
public string ModifiedBy { get { return null; } set { } }
}
To get the test to pass you have to tell Moq to set up the property to have the standard property behavior. You can do this by calling SetupProperty and the mock will look more like
public class AuditableMock : IAuditable {
public string ModifiedBy { get; set; }
}
and the test above would pass as the value "test" would now get stored against the mock. When mocking complex objects you might want to do this for all properties, hence the SetupAllProperties shortcut
Finally, the lightbulb in the IDE is the ReSharper plugin.

If you don't care about the exact value of a property, it's far better to use .IsAny because you are being explicit about the fact that the exact value is not important. If you hardcode it as "abc", then it is not clear if your code you are testing depends on starting with "a" or ending with "c" or being 3 chars long, etc. etc.