We Keep Coding

XUnit: Generalized tests

I usually create a test class for each method of the SUT to test. Currently, I have the problem that I write a lot of tests that are code duplication.
Checking if the method throws if the object was disposed or async disposed
Checking if the method throws if any argument is null
There are around 6-7 tests each time to check this. In every class for every method of any SUT. Currently, I have 150+ tests to test the same thing and they keep getting more. I just want to have a base class containing all these tests and I just want to define the test data to use.
Something like this: (does not work)
public class TestBase<TSut> where TSut : IDisposable {
protected TSut Sut { get; set; }
protected Delegate MethodToTest { get; set; }
protected IEnumerable<object?[]> DefaultParameters { get; set; } // Should be valid data
protected IEnumerable<object?[]> NullCheckingParameters { get; set; } // Every set should contain valid data except for one null value
[Theory]
[MemberData(nameof(DefaultParameters))]
public void TestMethod_ShouldThrowException_WhenObjectWasDisposed(object?[] data) {
this.Sut.Dispose();
Assert.Throws<ObjectDisposedException>(
() => this.MethodToTest.Method.Invoke(this.Sut, data)
);
}
[Theory]
[MemberData(nameof(NullCheckingParameters))]
public void TestMethod_ShouldThrowException_WhenParameterWasNull(object?[] data) {
this.Sut.Dispose();
Assert.Throws<ArgumentNullException>(
() => this.MethodToTest.Method.Invoke(this.Sut, data)
);
}
}
public class MethodTests : TestBase<MySut> {
public MethodTests() {
this.Sut = new MySut();
this.MethodToTest = this.Sut.MethodToTest;
this.DefaultParameters = new[] {
new object?[] {"Valid1", "Valid2", "Valid3"}
};
this.NullCheckingParameters = new[] {
new object?[] {null, "Valid2", "Valid3"},
new object?[] {"Valid1", null, "Valid3"},
new object?[] {"Valid1", "Valid2", null}
};
}
}
The problem is that the MemberData has to be a static member. So is there a way to generalize these tests?

In Xunit, the InlineData, MemberData and ClassData are all derived from the abstract class DataAttribute which is basically just this:
{
//
// Summary:
// Abstract attribute which represents a data source for a data theory. Data source
// providers derive from this attribute and implement GetData to return the data
// for the theory. Caution: the property is completely enumerated by .ToList() before
// any test is run. Hence it should return independent object sets.
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true, Inherited = true)]
[DataDiscoverer("Xunit.Sdk.DataDiscoverer", "xunit.core")]
public abstract class DataAttribute : Attribute
{
protected DataAttribute();
//
// Summary:
// Marks all test cases generated by this data source as skipped.
public virtual string Skip { get; set; }
//
// Summary:
// Returns the data to be used to test the theory.
//
// Parameters:
// testMethod:
// The method that is being tested
//
// Returns:
// One or more sets of theory data. Each invocation of the test method is represented
// by a single object array.
public abstract IEnumerable<object[]> GetData(MethodInfo testMethod);
}
}
You can derive your own classes from this to return any sets of data you want.
For your first point, this class returns all the public methods in a class under test. It returns them as string's because XUnit won't enumerate generic object types in the test explorer window. The class provides an easy way to get the method info back from the string with method getMethodFromName
using System;
using System.Collections.Generic;
using System.Reflection;
namespace SO74727807_xunit_generalized_tests
{
public class GenericPublicMethodsDataAttribute : Xunit.Sdk.DataAttribute
{
private Type objectType;
private MethodInfo[] methodInfos;
private Dictionary<string, MethodInfo> index = new Dictionary<string, MethodInfo>();
public GenericPublicMethodsDataAttribute(Type objectType_)
{
objectType = objectType_ ?? throw new ArgumentNullException($"Parameter {nameof(objectType_)} is null.");
methodInfos = objectType.GetMethods();
foreach (MethodInfo methodInfo in methodInfos)
{
string key = methodInfo.ToString();
index.Add(key, methodInfo);
}
}
public override IEnumerable<object[]> GetData(MethodInfo testMethod)
{
foreach (string key in index.Keys)
{
string[] methodKey = new string[] { key };
yield return methodKey;
}
}
public MethodInfo getMethodFromName(string methodName)
{
if (index.TryGetValue(methodName, out MethodInfo info))
{
return info;
}
throw new ArgumentException($"No method info found for method name \"{methodName}\"");
}
}
}
For your second point, for example, this class would return sets of valid data with one null value on each iteration, like this:
using System;
using System.Collections.Generic;
using System.Reflection;
namespace SO74727807_xunit_generalized_tests
{
public class GenericNullParameterDataAttribute : Xunit.Sdk.DataAttribute
{
private object[] validData;
public GenericNullParameterDataAttribute(params object[] validData_)
{
validData = new object[validData_.Length];
Array.Copy(validData_, validData, validData.Length);
}
public override IEnumerable<object[]> GetData(MethodInfo testMethod)
{
ParameterInfo[] parameters = testMethod.GetParameters();
// TODO: check that the object types in the valid data match the test method parameter types.
for (int i = 0; i < validData.Length; i++)
{
// Skip value types, they're not nullable
if (!parameters[i].ParameterType.IsValueType)
{
object[] methodData = new object[validData.Length];
Array.Copy(validData, methodData, validData.Length);
methodData[i] = null;
yield return methodData;
}
}
}
}
}
The method skips over value types since they are not nullable.
I discovered that it is important to make a new object array for each yield instead of trying to reuse the same one-- that confused the test discoverer.
To demonstrate, I made a small "class under test"
using System;
namespace SO74727807_xunit_generalized_tests
{
class ClassUnderTest : IDisposable
{
private bool disposedValue;
public void BadMethod(string param1, string param2, string param3)
{
// Doesn't throw on call in disposed object
// Doesn't throw on call with null parameters
// Doesn't do anything
}
public void GoodMethod(string param1, string param2, int param3)
{
// Throws if disposed
if (disposedValue) throw new ObjectDisposedException(this.ToString());
// Throws on null arguments
if (param1 == null)
{
throw new ArgumentNullException(nameof(param1));
}
if (param2 == null)
{
throw new ArgumentNullException(nameof(param2));
}
// int is a non-nullable type.
}
protected virtual void Dispose(bool disposing)
{
if (!disposedValue)
{
disposedValue = true;
}
else
{
throw new ObjectDisposedException(this.ToString());
}
}
public void Dispose()
{
Dispose(disposing: true);
GC.SuppressFinalize(this);
}
}
}
Then you can write your generic unit tests like this:
using System;
using System.Reflection;
using Xunit;
namespace SO74727807_xunit_generalized_tests
{
public class GenericUnitTests
{
[Theory]
[GenericNullParameterData("Valid1", "Valid2", 3)]
public void GoodMethodThrowsOnNullArg(string param1, string param2, int param3)
{
ClassUnderTest sut = new ClassUnderTest();
Assert.Throws<ArgumentNullException>(() => sut.GoodMethod(param1, param2, param3));
}
[Theory]
[GenericNullParameterData("Valid1", "Valid2", "Valid3")]
public void BadMethodThrowsOnNullArg(string param1, string param2, string param3)
{
ClassUnderTest sut = new ClassUnderTest();
Assert.Throws<ArgumentNullException>(() => sut.BadMethod(param1, param2, param3));
}
[Theory]
[GenericPublicMethodsData(typeof(ClassUnderTest))]
public void PublicMethodsThrowOnDisposedObject(string methodName)
{
// Get a reference to the data provider attribute object
MethodBase method = MethodBase.GetCurrentMethod();
GenericPublicMethodsDataAttribute attr = (GenericPublicMethodsDataAttribute)method.GetCustomAttributes(typeof(GenericPublicMethodsDataAttribute), true)[0];
// Now we can get the method info
MethodInfo methodInfo = attr.getMethodFromName(methodName);
// Make default parameters
ParameterInfo[] parameterInfos = methodInfo.GetParameters();
object[] args = new object[parameterInfos.Length];
for (int i = 0; i < parameterInfos.Length; i++)
{
args[i] = getDefault(parameterInfos[i].ParameterType);
}
// Now make the object under test and dispose it
ClassUnderTest cut = new ClassUnderTest();
cut.Dispose();
// Methods in disposed objects should throw
// Note that the ObjectDisposedException will be an inner exception,
// the actual type thrown is System.Reflection.TargetInvocationException
Assert.ThrowsAny<Exception>(() => methodInfo.Invoke(cut, args));
}
private static object getDefault(Type type)
{
if (type.IsValueType)
{
return Activator.CreateInstance(type);
}
return null;
}
[Fact]
public void testGenericPublicMethodsDataAttribute()
{
var data = new GenericPublicMethodsDataAttribute(typeof(ClassUnderTest));
Assert.NotNull(data);
}
}
}
The tests are shown in the Test Explorer like this:
We can see that the good method passes them and the bad method fails.
The inherited methods like ToString() and GetHashCode() also fail--they do not throw on disposed objects. You could modify the GenericPublicMethodsDataAttribute class to skip the inherited methods by checking the DeclaringType property of the method info.

NUnit use different objects as Test Cases

I am trying to write an Update Test for different products (which are classes in this case) and different time steps :
public class UpdateTest
{
private static Product ProductLastYear;
private static Product ProductTwoYearsAgo;
public UpdateTest(Product product)
{
var previousReleasedProducts = new Products();
ProductLastYear = previousReleasedProducts.GetProduct(Years.GetLastYear());
ProductTwoYearsAgo = previousReleasedProducts.GetProduct(Years.GetTwoYearsAgo());
}
Each product needs to be installed, and afterwards it is checked if the installation was successful (this is basically a pre-step before the Update). Right now, I am using two Tests for this:
[Test, Category("UpdateLastYear")), Order((int) NunitTestOrderEnum.Order.First)]
public void InstallPreviousReleasedProduct()
{
using (var controller = new BootstrapperController(ProductLastYear))
{
controller.Install();
}
var successfulInstallation = InstallationValidator.ValidateInstall(ProductLastYear);
Assert.That(successfulInstallation, Is.True);
}
[Test, Category("UpdateTwoYearsAgo"), Order((int) NunitTestOrderEnum.Order.First)]
public void InstallTwoYearsAgoProduct()
{
using (var controller = new BootstrapperController(ProductTwoYearsAgo))
{
controller.Install();
}
var successfulInstallation = InstallationValidator.ValidateInstall(ProductTwoYearsAgo);
Assert.That(successfulInstallation, Is.True);
}
Now, both tests have some code redundancy, which I would like to avoid. I was thinking about using TestCases for this, something like :
[TestCase(ProductLastYear), Category("UpdateLastYear"), Order((int) NunitTestOrderEnum.Order.First)]
[TestCase(ProductTwoYearsAgo), Category("UpdateTwoYearsAgo"), Order((int) NunitTestOrderEnum.Order.First)]
public void InstallPreProduct(Product product)
{
using (var controller = new BootstrapperController(product))
{
controller.Install();
}
var successfulInstallation = InstallationValidator.ValidateInstall(product);
Assert.That(successfulInstallation, Is.True);
}
Is something like this possible? I tried different Syntax for that approach, but it does not seem to work that easily.

You can only use compile-time constants within attributes. However your static fields are no constants.
You can use the TestCaseSource-attribute:
[TestCaseSource(nameof(ProvideTestcases))]
public void InstallPreProduct(Product product)
{
using (var controller = new BootstrapperController(product))
{
controller.Install();
}
var successfulInstallation = InstallationValidator.ValidateInstall(product);
Assert.That(successfulInstallation, Is.True);
}
public static IEnumerable<TestCaseData> ProvideTestcases()
{
yield return new TestCaseData(ProductLastYear).SetCategory("UpdateLastYear");
yield return new TestCaseData(ProductTwoYearsAgo).SetCategory("UpdateTwoYearsAgo");
}
However that assumes your static fields are already initialized, which isn't the case. So you need a static constructor for your testclass.
[TestFixture]
public UpdateTest
{
public static ProductLastYear;
public static ProductTwoYearsAgo;
static
{
ProductLastYear = ...;
ProductTwoYearsAgo = ...;
}
}
This is pretty much boilerplate for so little duplication. So it's a tradeoff if or of it not this is worth the afford.
Another opportunity is to introduce some static constant, like an enum that reflects the property to be used:
[TestCase(MyEnum.LastYear), Category("UpdateLastYear")]
[TestCase(MyEnum.TwoYearsAgo), Category("UpdateTwoYearsAgo")]
public void InstallPreProduct(MyEnum product)
{
var Product = product == MyEnum.LastYear ?
ProductLastYear :
ProductTwoYearsAgo ;
using (var controller = new BootstrapperController(product))
{
controller.Install();
}
var successfulInstallation = InstallationValidator.ValidateInstall(product);
Assert.That(successfulInstallation, Is.True);
}
An enum is a constant expression, so you can easily use it within the test-attributes.

How can i get the parameter values of an anonymous method passed as a func?

I am calling methods on a remote system. The remote system implements an interface that both systems have a copy of (via shared nuget repository). At the moment i am sending the requests like this:
var oldRequest = new FooRequest("GetEmployeeById", new object[] { "myPartner", 42, DateTime.Now.AddDays(-1) });
Here is the interface:
public class FooResponse<T> { }
public interface IFooController
{
FooResponse<string> GetEmployeeById(string partnerName, int employeeId, DateTime? ifModifiedSince);
}
As you can image, sometimes programmers passes arguments in the wrong order to the array in the constructor, and things start to fail. To resolve this I have created the following code to have intellisense support when creating the FooRequest:
public static FooRequest Create<T>(Func<FooResponse<T>> func)
{
return new FooRequest(null, null); // Here goes some magic reflection stuff instead of null.
}
It is now possible to create a FooRequest like this:
public static IFooController iFooController => (IFooController)new object();
public static FooRequest CreateRequest<T>(Func<FooResponse<T>> func)
{
return FooRequest.Create(func);
}
var newRequest = CreateRequest(() => iFooController.GetEmployeeById("myPartner", 42, DateTime.Now.AddDays(-1)));
My question then is: How will i be able to get the name of the method and the value of the parameters in the FooRequest.Create-method?
I have exhausted both my reflection and google-skills trying to find the values, but no luck so far.
Complete compiling code can be found here if someone wants to give it a shot: http://ideone.com/ovWseI

Here is a sketch of how you can do this with expressions:
public class Test {
public static IFooController iFooController => (IFooController) new object();
public static FooRequest CreateRequest<T>(Expression<Func<FooResponse<T>>> func) {
return FooRequest.Create(func);
}
public static void Main() {
var newRequest = CreateRequest(() => iFooController.GetEmployeeById("myPartner", 42, DateTime.Now.AddDays(-1)));
Console.ReadKey();
}
}
public class FooRequest {
public static FooRequest Create<T>(Expression<Func<FooResponse<T>>> func) {
var call = (MethodCallExpression) func.Body;
var arguments = new List<object>();
foreach (var arg in call.Arguments) {
var constant = arg as ConstantExpression;
if (constant != null) {
arguments.Add(constant.Value);
}
else {
var evaled = Expression.Lambda(arg).Compile().DynamicInvoke();
arguments.Add(evaled);
}
}
return new FooRequest(call.Method.Name, arguments.ToArray());
}
public FooRequest(string function, object[] data = null) {
//SendRequestToServiceBus(function, data);
Console.Write($"Function name: {function}");
}
}
public class FooResponse<T> {
}
public interface IFooController {
FooResponse<string> GetEmployeeById(string partnerName, int employeeId, DateTime? ifModifiedSince);
}

Testing using RHinomock

I have a class to test which is tricky to test using Rhinomock unlike normal classes bacause its constructor is injected with a dependency which is not a single interface but an array of Interface objects. Please help me set up all stuff to write a test using rhinomock.
namespace ClinicalAdvantage.Domain.UserAppSettings
{
using System;
using System.Collections.Generic;
using System.Linq;
using Newtonsoft.Json.Linq;
public class Agg : IAgg
{
private readonly ISource[] sources;
public Agg(ISource[] sources)
{
this.sources = sources;
}
public JObject GetAll()
{
var obj = new JObject();
foreach (var source in this.sources)
{
var token = source.GetCurr();
if (token != null)
{
obj.Add(new JProperty(source.Name, token));
}
}
return obj;
}
}
ISource is an interface which has 2 implementations. GetALL() iterates thro each implementated class object and calls the GetCurr method in each of the object and aggregates the result. I have to stub GetCurr method to return a standard Jtoken. I am unable to create a mock of this class Agg or a stub of ISource.
public interface ISource
{
string Name { get; }
bool Enabled { get; }
JToken GetCurr();
}
}

Something like this might work:
[TestClass]
public class AggTest
{
private ISource Isource;
private Agg agg;
[TestInitialize]
public void SetUp()
{
Isource = MockRepository.GenerateMock<ISource>();
agg = new Agg(new [Isource]);
}
[TestMethod]
public void GetAll()
{
Isource.Stub(x => x.GetCurr()).
Return(new JToken());
var jObject = agg.GetAll();
Assert.IsNotNull(jObject);
// Do your assertion that all JProperty objects are in the jObject
// I don't know the syntax
}
}

Moq Verify with anonymous type parameter

I have the following test, with supporting classes, but I can't figure out how to verify the call on the dependency.
[TestFixture]
public class AnonymousGenericTypeParameterTests
{
[Test]
public void Test()
{
// Arrange
var dependency = new Mock<IDependency>();
var towns = new List<Town>
{
new Town { Name = "Lifford", County = "Donegal", Country="Ireland", Population = 1658 },
new Town { Name = "Ballyshannon", County = "Donegal", Country="Ireland", Population = 2504 },
new Town { Name = "Buxton", County = "Derbyshire", Country="United Kingdom", Population = 13599 },
};
var sut = new MyClass(dependency.Object);
// Act
sut.DoSomething(towns);
// Assert
// The following line needs to be fixed.
dependency.Verify(d => d.Execute(It.IsAny<IEnumerable<object>>(), It.IsAny<Func<object, decimal?>>()));
}
}
public interface IDependency
{
void Execute<T>(IEnumerable<T> collection, Func<T, decimal?> rateSelector);
}
public class MyClass
{
private readonly IDependency dependency;
public MyClass(IDependency dependency)
{
this.dependency = dependency;
}
public void DoSomething(IEnumerable<Town> towns)
{
var counties = towns.GroupBy(t => new {t.Country,t.County});
foreach (var county in counties)
{
dependency.Execute(county, c => c.Population);
}
}
}
public class Town
{
public string Name { get; set; }
public string County { get; set; }
public int Population { get; set; }
public string Country { get; set; }
}
According to Moq's test output, the performed invocations are:
Dependency.Execute(System.Linq.Lookup`2+Grouping[<>f__AnonymousType0`2[System.String,System.String],UniFocus.Staffscope4.Tests.Town], System.Func`2[UniFocus.Staffscope4.Tests.Town,System.Nullable`1[System.Decimal]])
I see plenty of questions regarding anonymous parameters in Moq (such as this and this and this), but can't find anything relating to using an anonymous type as the actual type parameter.
What can be put in the Verify line so that it actually verifies the call inside?
Note: My example IDependency doesn't return a value (it's already complex enough, I think), but there will be bonus kudos for answers that implictly or explicitly address Setup() as well as Verify().
Update
Jesse's solution only passes the test because I made a bad choice when crafting my example. I should have realised that any IGrouping<out TKey, out TElement> is also an IEnumerable<TElement>. Is there a more universal solution?
Update 2
I feel like my original example was possibly too elaborate and didn't represent well the actual title of my question. Is there any solution that works for this more straightforward and to-the-point example?
using Moq;
using NUnit.Framework;
namespace Tests
{
[TestFixture]
public class SimpleAnonymousGenericTypeParameterTests
{
[Test]
public void Test()
{
// Arrange
var dependency = new Mock<IDependency>();
var sut = new MyClass(dependency.Object);
// Act
sut.DoSomething("Donegal", "Lifford");
// Assert
// This verify works for both calls to Execute()
dependency.Verify(d => d.Execute(It.IsAny<object>()), Times.Exactly(2));
// This verify should specifically refer to only the first call to Execute()
dependency.Verify(d => d.Execute(It.IsAny</*HowToRepresentAnonymousTypeHere*/object>()), Times.Once);
}
public interface IDependency
{
void Execute<T>(T thing);
}
public class MyClass
{
private readonly IDependency dependency;
public MyClass(IDependency dependency)
{
this.dependency = dependency;
}
public void DoSomething(string county, string town)
{
dependency.Execute(new { county, town });
object someUnknownObject = "";
dependency.Execute(someUnknownObject);
}
}
}
}

The accepted answer doesn't work for me, I believe it's because the tests and the object in question are in a different assembly so Moq doesn't know how to reconcile the types and does not match them.
Instead, I created the following helper methods that can verify that the anonymous type provided has the correct fields and values:
public static class AnonHelpers
{
public static object MatchAnonymousType(object expected)
{
return Match.Create(Matcher(expected));
}
private static Predicate<object> Matcher(object expected)
{
return actual =>
{
var expectedProp = expected.GetType().GetProperties().ToDictionary(x => x.Name, x => x.GetValue(expected));
var actualProp = actual.GetType().GetProperties().ToDictionary(x => x.Name, x => x.GetValue(actual));
foreach (var prop in expectedProp)
{
if (!actualProp.ContainsKey(prop.Key))
return false;
if (!prop.Value.Equals(actualProp[prop.Key]))
return false;
}
return true;
};
}
}
They can be used like so:
var anon = new { SomeKey = "some value", SomeOtherKey = 123 };
myMock.Setup(x => x.MyMethod(personIDs, AnonHelpers.MatchAnonymousType(anon))).Verifiable();
This will create a matcher that will use reflection to match the anonymous type based on it's keys and values and then you can use normal verification to see when it's been called.

Since the types are known in the context of the test, you could provide the specific type arguments to the Verify call. The following change got the test to pass:
dependency.Verify(d =>
d.Execute(It.IsAny<IEnumerable<Town>>(), It.IsAny<Func<Town, decimal?>>()));
The same should also work for setups.
With regards to the example in Update 2, the following passes, but it requires knowledge of the inner workings of the DoSomething() method and as far as I know it's the only way to make it work:
var anonymousType = new {county = "Donegal", town = "Lifford"};
dependency.Verify(d => d.Execute(anonymousType), Times.Once);