We Keep Coding

Did 9.1.3 FluentValidation update ruin validators mocks?

I updated to 9.1.3 version of the package and now my validator mocks do not work.
Somehow code runs no matter whether .Validate() returns true or false.
Here is the code for the validator mock:
validatorMock
.Setup(x => x.Validate(It.IsAny<IValidationContext>()).IsValid)
.Returns(false);
Assert.Throws<ValidationException>(() => command.Execute(request), "Position field validation error");
repositoryMock.Verify(repository => repository.EditPosition(It.IsAny<DbPosition>()), Times.Never);
And here is the test failing:
Message:
Position field validation error
Expected: <FluentValidation.ValidationException>
But was: null
Validator.cs:
public class SampleValidator : AbstractValidator<Position>
{
public SampleValidator()
{
RuleFor(position => position.Name)
.NotEmpty()
.MaximumLength(80)
.WithMessage("Position name is too long");
RuleFor(position => position.Description)
.NotEmpty()
.MaximumLength(350)
.WithMessage("Position description is too long");
}
}
Dependency Injection:
services.AddTransient<IValidator<Position>, SampleValidator>();
Usage:
public class SampleCommand : ISampleCommand
{
private readonly IValidator<Position> validator;
private readonly ISampleRepository repository;
private readonly IMapper<Position, DbPosition> mapper;
public SampleCommand(
[FromServices] IValidator<Position> validator,
[FromServices] ISampleRepository repository,
[FromServices] IMapper<Position, DbPosition> mapper)
{
this.validator = validator;
this.repository = repository;
this.mapper = mapper;
}
public bool Execute(Position request)
{
validator.ValidateAndThrow(request);
var position = mapper.Map(request);
return repository.EditPosition(position);
}
}
Validator Mocks in test:
private Mock<IValidator<EditPositionRequest>> validatorMock;
...
validatorMock = new Mock<IValidator<Position>>();
UPDATE
Before the update, all tests were running perfectly. Now they're ruined and I have to install the previous version.

Expanding on my comment:
Yes, 9.1 changed how throwing validation exceptions is handled.
Some context:
Validator classes return a ValidationResult with an IsValid boolean property. The ValidateAndThrow extension method checks this property, and throws an exception if IsValid is false. If you mocked the validator, you could still use the "real" ValidateAndThrow extension method on your mock to throw an exception if your mock returned an invalid validation result.
In FluentValidation 9.1, the logic for throwing the exception was moved out of the extension method and into the validator class itself, in the RaiseValidationException. This was done so that the logic for throwing the exception could be customized (by overriding this method) which couldn't be done before when it was an extension method.
// This is the ValidateAndThrow method definition versions older than 9.1
public static void ValidateAndThrow<T>(this IValidator<T> validator, T instance) {
var result = validator.Validate(instance);
if (!result.IsValid) {
throw new ValidationException(result.Errors);
}
}
// This is the ValidateAndThrowMethod in 9.1 and newer
public static void ValidateAndThrow<T>(this IValidator<T> validator, T instance) {
validator.Validate(instance, options => {
options.ThrowOnFailures();
});
}
For runtime use, this doesn't make a difference - the exception is still thrown (unless you've overridden the method to prevent this).
However, this had the side effect that if you were relying on the exception being thrown by the extension method rather than the validator, this will have an undesirable result. This is really only the case when mocking the validator. Now when you create a mock, the exception won't be thrown because the mock doesn't behave as a real validator.
My recommendation with FluentValidation has always been "don't mock validators", instead treat them as black boxes and supply real validator instances with valid/invalid input for testing purposes - this leads to much less brittle tests in the long run. However, I'm also aware that it may not be possible to rewrite your tests in this way if you already have lots of them.
As a workaround, you can mock the overload of Validate that takes a ValidationContext and check the context for the ThrowOnFailures property, and have your mock throw the exception if this is set to true.
However, be aware that if you do this you could run into a situation where your mock behaves one way and the real validator behaves differently (if its RaiseValidationException message has been overridden).
As this is a breaking change, shouldn't it have been made in a major version? Ideally yes, this was my bad as I didn't foresee this particular use case.
Edit: Here's an example of creating a mock that checks the ThrowOnFailures property. The example uses the Moq library, but the same concept will apply to other mocking libraries too.
private static Mock<IValidator<T>> CreateFailingMockValidator<T>() {
var mockValidator = new Mock<IValidator<T>>();
var failureResult = new ValidationResult(new List<ValidationFailure>() {
new ValidationFailure("Foo", "Bar")
});
// Setup the Validate/ValidateAsync overloads that take an instance.
// These will never throw exceptions.
mockValidator.Setup(p => p.Validate(It.IsAny<T>()))
.Returns(failureResult).Verifiable();
mockValidator.Setup(p => p.ValidateAsync(It.IsAny<T>(), It.IsAny<CancellationToken>()))
.ReturnsAsync(failureResult);
// Setup the Validate/ValidateAsync overloads that take a context.
// This is the method called by ValidateAndThrow, so will potentially support throwing the exception.
// Setup method invocations for with an exception and without.
mockValidator.Setup(p => p.Validate(It.Is<ValidationContext<T>>(context => context.ThrowOnFailures)))
.Throws(new ValidationException(failureResult.Errors));
mockValidator.Setup(p => p.ValidateAsync(It.Is<ValidationContext<T>>(context => context.ThrowOnFailures), It.IsAny<CancellationToken>()))
.Throws(new ValidationException(failureResult.Errors));
// If ThrowOnFailures is false, return the result.
mockValidator.Setup(p => p.Validate(It.Is<ValidationContext<T>>(context => !context.ThrowOnFailures)))
.Returns(failureResult).Verifiable();
mockValidator.Setup(p => p.ValidateAsync(It.Is<ValidationContext<T>>(context => !context.ThrowOnFailures), It.IsAny<CancellationToken>()))
.ReturnsAsync(failureResult);
return mockValidator;
}

Mocking a Collection ClusterState

I am trying to mock the cluster state of a mongo collection so it gives back a ClusterState.Connected. Below you will see one of my unit tests. Currently this is giving back an error, is there any way to Mock the Collection.Database.Client.Cluster.Description.State?
[Fact]
public void HealthCheck_SucceededDatabase_Connection()
{
//Arrange
var myRepository = new Mock<IRepository<RepoEntityObject>>();
var healthCheck = new HealthCheck(myRepository.Object);
//Setup
myRepository.Setup(mcr => mcr.Collection.Database.Client.Cluster.Description.State).Returns(ClusterState.Connected);
//Act
var result = healthCheck.ExecuteHealthchecks();
//Results
result[0].CheckType.ShouldBe("Service is alive");
result[0].Message.ShouldBe("");
result[0].Passed.ShouldBe(true);
result[1].CheckType.ShouldBe("MongoDB");
result[1].Message.ShouldBe("Service is alive");
result[1].Passed.ShouldBe(True);
}
Error Stack Trace:
System.NotSupportedException occurred HResult=0x80131515
Message=Invalid setup on a non-virtual (overridable in VB) member: mcr
=> mcr.Collection.Database.Client.Cluster.Description.State Source= StackTrace: at
Moq.Mock.ThrowIfSetupExpressionInvolvesUnsupportedMember(Expression
setup, MethodInfo method) at
Moq.Mock.<>c__DisplayClass62_0`2.b__0() at
Test.Unit.HealthCheckTests.HealthCheck_SucceededDatabase_Connection()
in
C:\HealthCheckTests.cs:line
50
Edit One Possible solution:
Making the Description.State attribute virtual is not something I am trying to implement since this method is coming from the MongoDb C# driver that I am using and I am not looking into overriding it.

This is the key phrase Invalid setup on a non-virtual member. Make the property virtual. otherwise if unable to because you are not in control of said property then encapsulate the desired property and expose it as virtual
public interface IRepository<T> {
ClusterState State { get; }
//...other members removed for brevity
}
Now you can mock the member you control
//Arrange
var myRepository = new Mock<IRepository<RepoEntityObject>>();
var healthCheck = new HealthCheck(myRepository.Object);
//Setup
myRepository.Setup(mcr => mcr.State).Returns(ClusterState.Connected);
//...code removed for brevity
Now with that said, you have now realized that your repository is leaking implementation concerns that are difficult to mock in isolation. Consider reviewing your design choices regarding your level of abstractions.

How do I provide ILogger<T> in my unit tests of .NET Core code?

Given a class with a constructor signature of
public Foo(ILogger<Foo> logger) {
// ...
}
that I want to test, I need some way to provide an ILogger<Foo> in the test. It's been asked before, but the only answer then was to set up a full-blown service registry, configure logging and resolve the logger from there. This seems very overkill to me.
Is there a simple way to provide an implementation of ILogger<T> for testing purposes?
Note: it doesn't have to actually log anything - just not blow up when the subject under test tries to log something.

Starting from dotnet core 2.0 there's a generic NullLogger<T> class available:
var foo = new Foo(NullLogger<Foo>.Instance);
https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.logging.abstractions.nulllogger-1?view=aspnetcore-2.1 (docs)
https://github.com/aspnet/Logging/blob/master/src/Microsoft.Extensions.Logging.Abstractions/NullLoggerOfT.cs (source)
Or if you need it as part of your services:
services.AddSingleton<ILoggerFactory, NullLoggerFactory>();
https://learn.microsoft.com/en-us/dotnet/api/microsoft.extensions.logging.abstractions.nullloggerfactory?view=aspnetcore-2.1 (docs)

You can create an instance of ILogger<Foo> using NullLoggerFactory as the factory.
Consider the following controller:
public abstract class Foo: Controller
{
public Foo(ILogger<Foo> logger) {
Logger = logger;
}
public ILogger Logger { get; private set; }
}
A sample unit test could be:
[TestMethod]
public void FooConstructorUnitTest()
{
// Arrange
ILogger<FooController> logger = new Logger<FooController>(new NullLoggerFactory());
// Act
FooController target = new FooController(logger);
// Assert
Assert.AreSame(logger, target.Logger);
}

If you use generic logger (ILogger<>) in your classes those instances are generated from IServiceProvider you should register generic NullLogger<> on service provider as below. Not important what you use generic type T in ILogger<>
services.AddSingleton(typeof(ILogger<>), typeof(NullLogger<>));

You have two options:
Create empty implementation of ILogger<Foo> by hand and pass an instance of it to ctor.
Create same empty implementation on the fly using some mocking framework like Moq, NSubstitute, etc.

You could inject ILoggerFactory instead and then create the logger
public Foo(ILoggerFactory loggerFactory) {
logger = loggerFactory.CreateLogger<Foo>();
// ...
}
At startup you need to add the NullLoggerFactory service of course:
services.AddSingleton<ILoggerFactory, NullLoggerFactory>()

From the docs for ILogger<T> (emphasis mine):
A generic interface for logging where the category name is derived from the specified TCategoryName type name. Generally used to enable activation of a named ILogger from dependency injection.
So one option would be to change the implementation of the Foo method to take a plain ILogger and use the NullLogger implementation.

You should use the Null Object Pattern. This has two advantages for you: 1) you can get your tests up and running quickly and they won't "blow up", and 2) anyone will be able to use your class without supplying a logger. Just use NullLogger.Instance, or NullLoggerFactory.Instance.
However, you should use a mocking framework to verify that log calls get made. Here is some sample code with Moq.
[TestMethod]
public void TestLogError()
{
var recordId = new Guid("0b88ae00-7889-414a-aa26-18f206470001");
_logTest.ProcessWithException(recordId);
_loggerMock.Verify
(
l => l.Log
(
//Check the severity level
LogLevel.Error,
//This may or may not be relevant to your scenario
It.IsAny<EventId>(),
//This is the magical Moq code that exposes internal log processing from the extension methods
It.Is<It.IsAnyType>((state, t) =>
//This confirms that the correct log message was sent to the logger. {OriginalFormat} should match the value passed to the logger
//Note: messages should be retrieved from a service that will probably store the strings in a resource file
CheckValue(state, LogTest.ErrorMessage, "{OriginalFormat}") &&
//This confirms that an argument with a key of "recordId" was sent with the correct value
//In Application Insights, this will turn up in Custom Dimensions
CheckValue(state, recordId, nameof(recordId))
),
//Confirm the exception type
It.IsAny<NotImplementedException>(),
//Accept any valid Func here. The Func is specified by the extension methods
(Func<It.IsAnyType, Exception, string>)It.IsAny<object>()),
//Make sure the message was logged the correct number of times
Times.Exactly(1)
);
}
private static bool CheckValue(object state, object expectedValue, string key)
{
var keyValuePairList = (IReadOnlyList<KeyValuePair<string, object>>)state;
var actualValue = keyValuePairList.First(kvp => string.Compare(kvp.Key, key, StringComparison.Ordinal) == 0).Value;
return expectedValue.Equals(actualValue);
}
For more context, see this article.

If you need to verify the calls in addition to just provide the instance, it gets somewhat complicated. The reason is that most calls does not actually belong to the ILogger interface itself.
I have written a more detailed answer here.
Here is a small overview.
Example of a method that I have made to work with NSubstitute:
public static class LoggerTestingExtensions
{
public static void LogError(this ILogger logger, string message)
{
logger.Log(
LogLevel.Error,
0,
Arg.Is<FormattedLogValues>(v => v.ToString() == message),
Arg.Any<Exception>(),
Arg.Any<Func<object, Exception, string>>());
}
}
And this is how it can be used:
_logger.Received(1).LogError("Something bad happened");

You should try this for mocking ILogger:
mock.Setup(m => m.Log<object>(It.IsAny<LogLevel>(),It.IsAny<EventId>(),It.IsAny<object>(),It.IsAny<Exception>(),It.IsAny<Func<object, Exception,string>>()))
.Callback<LogLevel, EventId, object, Exception, Func<object, Exception, string>>((logLevel, eventId, obj, exception, func) =>
{
string msg = func.Invoke(obj, exception);
Console.WriteLine(msg);
});

This worked for me:
private FooController _fooController;
private Mock<ILogger<FooController>> _logger;
[TestInitialize]
public void Setup()
{
_logger = new Mock<ILogger<FooController>>();
_fooController = new FooController(_logger.Object);
}

How to add an exception test for the below Http call that returns a json

[HttpPost]
[Route("TnC")]
public IHttpActionResult TnC(CustomViewModel myViewModel)
{
try
{
return Json(_Internal.TnC(myViewModel, LoggedInUser));
}
catch (BusinessException exception)
{
return Json(BuildErrorModelBase(exception));
}
}
Where the _Internal is a Service with a guaranteed 99.99% up-time and not formalized fault contract interfaces defined.
Exceptions which are handled in my application level(business layer level) as a BusinessException - root class
Where BusinessException is defined as follows
public class BusinessException : Exception
{
BusinessException()...
BusinessExceptionFoo()...
BusinessExceptionBar()...
//...
}
And the current test Method is
To do : Add Exception test
[TestMethod]
[ExpectedException(typeof(BusinessException),
"Not a valid Business Case")]
public void TnCTest()
{
var bookingService = myContainer.Resolve<mySvc>();
var controller = new LinkBookingController(mySvc, myContainer);
var expectedResult = controller.TnC(new myViewModel
{
...params
});
var actualResult = GetData<Result<myViewModel>>(expectedResult);
Assert.AreEqual(expectedResult, actualResult);
}
expectedResult==actualResult does not test the exception block of the code.
How do I construct a request that makes the service throw the exception other than manually removing the ethernet cable to get this specific type of server error.
The best I could come up with was
#if DEBUG && UnitTestExceptions
throw new BusinessException();
#endif
But there is gotta be a better option.

There are a few things of concern with the method under test.
It is mixing cross-cutting concerns in the action that should be refactored out into an ExceptionHandler. Chances are that piece of code is repeated many times in that controller and others like it (DRY).
public class WebApiExceptionHandler : ExceptionHandler {
public override void Handle(ExceptionHandlerContext context) {
var innerException = context.ExceptionContext.Exception;
// Ignore HTTP errors
if (innerException.GetType().IsAssignableFrom(typeof(System.Web.HttpException))) {
return;
}
if(innerException is BusinessException) {
context.Result = BuildErrorResult(exception);
return;
}
//...other handler code
}
IHttpActionResult BuildErrorResult(BusinessException exception) {
//... your logic here
}
}
The following extension method could be used to add the handler to the HttpConfiguration during startup which also assumes that the application is taking advantage of dependency inversion services.
public static HttpConfiguration ReplaceExceptionHandler(this HttpConfiguration config) {
var errorHandler = config.Services.GetExceptionHandler();
if (!(errorHandler is WebApiExceptionHandler)) {
var service = config.Services.GetService(typeof(WebApiExceptionHandler));
config.Services.Replace(typeof(IExceptionHandler), service);
}
return config;
}
Now that cross-cutting concerns have been dealt with the action becomes a lot simpler and easier to test. This is a simplified example of an ApiController
public class LinkBookingController : ApiController {
private IBookingService bookingService;
public LinkBookingController(IBookingService service) {
bookingService = service;
}
[HttpPost]
[Route("TnC")]
public IHttpActionResult TnC(CustomViewModel myViewModel) {
return Json(bookingService.TnC(myViewModel, User));
}
}
where IBookingService is defined as
public interface IBookingService {
BookingModel TnC(CustomViewModel viewModel, IPrincipal user);
}
Using a mocking framework like Moq an exception can be made to be thrown as needed.
[TestMethod]
[ExpectedException(typeof(BusinessException), "Not a valid Business Case")]
public void TnC_Should_Throw_BusinessException() {
//Arrange
var bookingService = new Mock<IBookingService>();
var controller = new LinkBookingController(bookingService.Object);
var viewModel = new myViewModel
{
//...params
};
bookingService.Setup(_ => _.TnC(viewModel, It.IsAny<IPrincipal>())).Throws<BusinessException>()
//Act
var expectedResult = controller.TnC(viewModel);
//Assert
//...the ExpectedException attribute should assert if it was thrown
}
To test how that exception can be handled do a unit test on the exception handler and not the controller as that is not the responsibility of the controller.
Try to keep controllers lean and focused on its UI concerns.

As Nkosi mentioned in his comment, what you need to do is add an interface to whatever type _Internal is so that your controller now depends on an interface as a contract, rather than the specific implementation.
Next, create a second constructor for your controller that accepts a IInternalService (whatever its called) and assigns that to _Internal. Your parameterless constructor can still assign whatever instance you're using now.
Now that you have this configuration (often called "poor mans dependency injection"), your unit test can create an instance of your controller passing in a different implementation of your service that throws an exception. You can do this by creating a new class, or you can do it dynamically using a library like Moq.
Hope that makes sense.

need to create convention for ApiControllers

I have a set of working imperative code in test and I'm trying to boil it down to an essential test convention.
My test looks like the following:
[Theory, BasicConventions]
public void GetVersionOnSiteVersionControllerReturnsASiteVersion(IFixture fixture)
{
fixture.OmitAutoProperties = true;
SiteVersion expected = fixture.Create<SiteVersion>();
SiteVersion actual = null;
var sut = fixture.Create<SiteVersionController>();
var response = sut
.GetSiteVersion()
.ExecuteAsync(new CancellationToken())
.Result
.TryGetContentValue<SiteVersion>(out actual);
actual.AsSource().OfLikeness<SiteVersion>().ShouldEqual(expected);
}
I also have a customization that allows this to work, namely by setting the HttpConfiguration and HttpRequestMessage to default non-null values.
public class ApiControllerCustomization : ICustomization
{
public void Customize(IFixture fixture)
{
var origin = fixture.OmitAutoProperties;
fixture.OmitAutoProperties = true;
var sut = fixture.Create<SiteVersionController>();
sut.Configuration = fixture.Create<HttpConfiguration>();
sut.Request = fixture.Create<HttpRequestMessage>();
fixture.Inject<SiteVersionController>(sut);
fixture.OmitAutoProperties = origin;
}
}
First, this looks ugly, but if I use Build<>().omit().with(config).with(request), it shuts off the automoq customization which it needs to build those instances.
Second, this only works for a SiteVersionController. I'd much rather generalize this for all my ApiControllers (maybe that's a bad idea, but I won't know until I try).
Essentially my convention would be as follows:
for all ApiControllers, create them without auto properties but do set the http configuration and request message properties to default non-null values

ApiControllers are quite difficult to wire up, because you'll need to assign certain properties to them in order for everything to work. At the very least, you'll need the Request property to be assigned, because otherwise, the Controller can't invoke Request.CreateResponse. Thus, switching off auto-properties for ApiController instances isn't a good strategy. Instead, you can configure AutoFixture to wire up HttpRequestMessage instances correctly.
Web API 1
With ASP.NET Web API 1, I usually use a Customization like this:
public class WebApiCustomization : ICustomization
{
public void Customize(IFixture fixture)
{
fixture.Customize<HttpRequestMessage>(c => c
.Do(x =>
x.Properties.Add(
HttpPropertyKeys.HttpConfigurationKey,
new HttpConfiguration())));
}
}
Since auto-properties are on by default, this is going to assign an appropriate instance of HttpRequestMessage to ApiController.Request. Together with an Auto-mocking Customization, Fixture can now create instances of all your ApiController classes.
Web API 2
With ASP.NET Web API 2 (5.0.0.0), things are a little more complicated, but with a bit of trial and error, I got this Customization to pass 808 tests:
public class WebApiCustomization : ICustomization
{
public void Customize(IFixture fixture)
{
fixture.Customize<HttpConfiguration>(c => c
.OmitAutoProperties());
fixture.Customize<HttpRequestMessage>(c => c
.Do(x =>
x.Properties.Add(
HttpPropertyKeys.HttpConfigurationKey,
fixture.Create<HttpConfiguration>())));
fixture.Customize<HttpRequestContext>(c => c
.Without(x => x.ClientCertificate));
}
}
Conventions
If you package that into an [AutoData] attribute, you should be able to refactor your test to:
[Theory, BasicConventions]
public void GetVersionOnSiteVersionControllerReturnsASiteVersion(
SiteVersionController sut,
SiteVersion expected)
{
SiteVersion actual = null;
var response = sut
.GetSiteVersion()
.ExecuteAsync(new CancellationToken())
.Result
.TryGetContentValue<SiteVersion>(out actual);
actual.AsSource().OfLikeness<SiteVersion>().ShouldEqual(expected);
}