We Keep Coding

XUnit: Generalized tests

I usually create a test class for each method of the SUT to test. Currently, I have the problem that I write a lot of tests that are code duplication.
Checking if the method throws if the object was disposed or async disposed
Checking if the method throws if any argument is null
There are around 6-7 tests each time to check this. In every class for every method of any SUT. Currently, I have 150+ tests to test the same thing and they keep getting more. I just want to have a base class containing all these tests and I just want to define the test data to use.
Something like this: (does not work)
public class TestBase<TSut> where TSut : IDisposable {
protected TSut Sut { get; set; }
protected Delegate MethodToTest { get; set; }
protected IEnumerable<object?[]> DefaultParameters { get; set; } // Should be valid data
protected IEnumerable<object?[]> NullCheckingParameters { get; set; } // Every set should contain valid data except for one null value
[Theory]
[MemberData(nameof(DefaultParameters))]
public void TestMethod_ShouldThrowException_WhenObjectWasDisposed(object?[] data) {
this.Sut.Dispose();
Assert.Throws<ObjectDisposedException>(
() => this.MethodToTest.Method.Invoke(this.Sut, data)
);
}
[Theory]
[MemberData(nameof(NullCheckingParameters))]
public void TestMethod_ShouldThrowException_WhenParameterWasNull(object?[] data) {
this.Sut.Dispose();
Assert.Throws<ArgumentNullException>(
() => this.MethodToTest.Method.Invoke(this.Sut, data)
);
}
}
public class MethodTests : TestBase<MySut> {
public MethodTests() {
this.Sut = new MySut();
this.MethodToTest = this.Sut.MethodToTest;
this.DefaultParameters = new[] {
new object?[] {"Valid1", "Valid2", "Valid3"}
};
this.NullCheckingParameters = new[] {
new object?[] {null, "Valid2", "Valid3"},
new object?[] {"Valid1", null, "Valid3"},
new object?[] {"Valid1", "Valid2", null}
};
}
}
The problem is that the MemberData has to be a static member. So is there a way to generalize these tests?

In Xunit, the InlineData, MemberData and ClassData are all derived from the abstract class DataAttribute which is basically just this:
{
//
// Summary:
// Abstract attribute which represents a data source for a data theory. Data source
// providers derive from this attribute and implement GetData to return the data
// for the theory. Caution: the property is completely enumerated by .ToList() before
// any test is run. Hence it should return independent object sets.
[AttributeUsage(AttributeTargets.Method, AllowMultiple = true, Inherited = true)]
[DataDiscoverer("Xunit.Sdk.DataDiscoverer", "xunit.core")]
public abstract class DataAttribute : Attribute
{
protected DataAttribute();
//
// Summary:
// Marks all test cases generated by this data source as skipped.
public virtual string Skip { get; set; }
//
// Summary:
// Returns the data to be used to test the theory.
//
// Parameters:
// testMethod:
// The method that is being tested
//
// Returns:
// One or more sets of theory data. Each invocation of the test method is represented
// by a single object array.
public abstract IEnumerable<object[]> GetData(MethodInfo testMethod);
}
}
You can derive your own classes from this to return any sets of data you want.
For your first point, this class returns all the public methods in a class under test. It returns them as string's because XUnit won't enumerate generic object types in the test explorer window. The class provides an easy way to get the method info back from the string with method getMethodFromName
using System;
using System.Collections.Generic;
using System.Reflection;
namespace SO74727807_xunit_generalized_tests
{
public class GenericPublicMethodsDataAttribute : Xunit.Sdk.DataAttribute
{
private Type objectType;
private MethodInfo[] methodInfos;
private Dictionary<string, MethodInfo> index = new Dictionary<string, MethodInfo>();
public GenericPublicMethodsDataAttribute(Type objectType_)
{
objectType = objectType_ ?? throw new ArgumentNullException($"Parameter {nameof(objectType_)} is null.");
methodInfos = objectType.GetMethods();
foreach (MethodInfo methodInfo in methodInfos)
{
string key = methodInfo.ToString();
index.Add(key, methodInfo);
}
}
public override IEnumerable<object[]> GetData(MethodInfo testMethod)
{
foreach (string key in index.Keys)
{
string[] methodKey = new string[] { key };
yield return methodKey;
}
}
public MethodInfo getMethodFromName(string methodName)
{
if (index.TryGetValue(methodName, out MethodInfo info))
{
return info;
}
throw new ArgumentException($"No method info found for method name \"{methodName}\"");
}
}
}
For your second point, for example, this class would return sets of valid data with one null value on each iteration, like this:
using System;
using System.Collections.Generic;
using System.Reflection;
namespace SO74727807_xunit_generalized_tests
{
public class GenericNullParameterDataAttribute : Xunit.Sdk.DataAttribute
{
private object[] validData;
public GenericNullParameterDataAttribute(params object[] validData_)
{
validData = new object[validData_.Length];
Array.Copy(validData_, validData, validData.Length);
}
public override IEnumerable<object[]> GetData(MethodInfo testMethod)
{
ParameterInfo[] parameters = testMethod.GetParameters();
// TODO: check that the object types in the valid data match the test method parameter types.
for (int i = 0; i < validData.Length; i++)
{
// Skip value types, they're not nullable
if (!parameters[i].ParameterType.IsValueType)
{
object[] methodData = new object[validData.Length];
Array.Copy(validData, methodData, validData.Length);
methodData[i] = null;
yield return methodData;
}
}
}
}
}
The method skips over value types since they are not nullable.
I discovered that it is important to make a new object array for each yield instead of trying to reuse the same one-- that confused the test discoverer.
To demonstrate, I made a small "class under test"
using System;
namespace SO74727807_xunit_generalized_tests
{
class ClassUnderTest : IDisposable
{
private bool disposedValue;
public void BadMethod(string param1, string param2, string param3)
{
// Doesn't throw on call in disposed object
// Doesn't throw on call with null parameters
// Doesn't do anything
}
public void GoodMethod(string param1, string param2, int param3)
{
// Throws if disposed
if (disposedValue) throw new ObjectDisposedException(this.ToString());
// Throws on null arguments
if (param1 == null)
{
throw new ArgumentNullException(nameof(param1));
}
if (param2 == null)
{
throw new ArgumentNullException(nameof(param2));
}
// int is a non-nullable type.
}
protected virtual void Dispose(bool disposing)
{
if (!disposedValue)
{
disposedValue = true;
}
else
{
throw new ObjectDisposedException(this.ToString());
}
}
public void Dispose()
{
Dispose(disposing: true);
GC.SuppressFinalize(this);
}
}
}
Then you can write your generic unit tests like this:
using System;
using System.Reflection;
using Xunit;
namespace SO74727807_xunit_generalized_tests
{
public class GenericUnitTests
{
[Theory]
[GenericNullParameterData("Valid1", "Valid2", 3)]
public void GoodMethodThrowsOnNullArg(string param1, string param2, int param3)
{
ClassUnderTest sut = new ClassUnderTest();
Assert.Throws<ArgumentNullException>(() => sut.GoodMethod(param1, param2, param3));
}
[Theory]
[GenericNullParameterData("Valid1", "Valid2", "Valid3")]
public void BadMethodThrowsOnNullArg(string param1, string param2, string param3)
{
ClassUnderTest sut = new ClassUnderTest();
Assert.Throws<ArgumentNullException>(() => sut.BadMethod(param1, param2, param3));
}
[Theory]
[GenericPublicMethodsData(typeof(ClassUnderTest))]
public void PublicMethodsThrowOnDisposedObject(string methodName)
{
// Get a reference to the data provider attribute object
MethodBase method = MethodBase.GetCurrentMethod();
GenericPublicMethodsDataAttribute attr = (GenericPublicMethodsDataAttribute)method.GetCustomAttributes(typeof(GenericPublicMethodsDataAttribute), true)[0];
// Now we can get the method info
MethodInfo methodInfo = attr.getMethodFromName(methodName);
// Make default parameters
ParameterInfo[] parameterInfos = methodInfo.GetParameters();
object[] args = new object[parameterInfos.Length];
for (int i = 0; i < parameterInfos.Length; i++)
{
args[i] = getDefault(parameterInfos[i].ParameterType);
}
// Now make the object under test and dispose it
ClassUnderTest cut = new ClassUnderTest();
cut.Dispose();
// Methods in disposed objects should throw
// Note that the ObjectDisposedException will be an inner exception,
// the actual type thrown is System.Reflection.TargetInvocationException
Assert.ThrowsAny<Exception>(() => methodInfo.Invoke(cut, args));
}
private static object getDefault(Type type)
{
if (type.IsValueType)
{
return Activator.CreateInstance(type);
}
return null;
}
[Fact]
public void testGenericPublicMethodsDataAttribute()
{
var data = new GenericPublicMethodsDataAttribute(typeof(ClassUnderTest));
Assert.NotNull(data);
}
}
}
The tests are shown in the Test Explorer like this:
We can see that the good method passes them and the bad method fails.
The inherited methods like ToString() and GetHashCode() also fail--they do not throw on disposed objects. You could modify the GenericPublicMethodsDataAttribute class to skip the inherited methods by checking the DeclaringType property of the method info.

Moq parent method being called in child class

I have a base class with a protected method that's being called in a public method in the child class I want to test. I'm failing to find a way to moq the base protected method for easier testing in child class.
public class MyBaseClass
{
protected virtual bool MyMethod(int number)
{
return number == 1;
}
}
public class MyChildClass : MyBaseClass
{
public bool DoSomething(int number)
{
return MyMethod(number);
}
}
[TestFixture]
public class MyChildClassTests
{
[Test]
public void Expected_Returns_False_WhenPassed_1()
{
var myChildClass = new MyChildClass();
// How do I mock MyMethod used in myBaseClass here?
// var mock = new Mock<MyBaseClass>();
// mock.Protected().Setup<bool>("MyMethod", ItExpr.IsAny<int>()).Returns(false);
// The above mock is correct, but it's in a different instance object than myBaseClass
var result = myChildClass.DoSomething();
Assert.AreEqual(false, result);
}
}
I can't change the classes to have a better architecture and I must do the best I can to implement unit test for DoSomething(), what I did so far is mock and prepare all the data that method uses, but since it's in another class I'd love my MyChildClassTests to not do all that and just limit to test DoSomething().
I've read about partial mocking and a whole lot of other questions and answers and I can't get it to work right.
I appreciate any suggestions!
Edit: Forgot to put public in all the classes, in my real world case, they are public.

class MyChildClassTests
{
[Test]
public void Expected_Returns_False_WhenPassed_1()
{
var myChildClass = new FakeChildClass();
var result = myChildClass.DoSomething(1);
Assert.AreEqual(false, result);
}
}
public class FakeChildClass: MyChildClass
{
protected override bool MyMethod(int number)
{
return number == 1;
}
}

First of all, ensure your classes are public.
Moq will complain about not being able to proxy into them if they're not.
public class MyBaseClass
{
public virtual bool MyMethod(int number)
{
return number == 1;
}
}
public class MyChildClass : MyBaseClass
{
public bool DoSomething(int number)
{
return MyMethod(number);
}
}
Next, make your base class method public. You won't be able to set it up unless you do.
After that, create a mock of the child object and mock the parent method.
var mockChild = new Mock<MyChildClass>(){CallBase = true};
mockChild.Setup(x => x.MyMethod(It.IsAny<int>())).Returns(false);
Pull your result.... result will return false even though the actual implementation would have returned true with 1 as a parameter.
var result = mockChild.Object.DoSomething(1);
When calling the DoSomething method, you'll actually enter the real implementation of that (put a breakpoint on if you don't believe me!) - but the mocked version of MyMethod will kick in.

Thanks all for your replies, gathering all I was able to get the actual answer to my use case:
Without changing MyBaseClass and MyChildClass:
public class MyBaseClass
{
protected virtual bool MyMethod(int number)
{
return number == 1;
}
}
public class MyChildClass : MyBaseClass
{
public bool DoSomething(int number)
{
return MyMethod(number);
}
}
I was able to mock the protected method and save me a LOT of work and duplicate code (that was in MyBaseClassTests already)
[TestFixture]
public class MyChildClassTests
{
[Test]
public void Expected_Returns_False_WhenPassed_1()
{
var expected = false;
var myChildClass = new Mock<MyChildClass> {CallBase = true};
myChildClass.Protected().Setup<bool>("MyMethod", 1).Returns(expected);
var result = myChildClass.Object.DoSomething(1);
Assert.AreEqual(expected, result);
}
[Test]
public void Expected_Returns_True_WhenPassed_1()
{
var expected = true;
var myChildClass = new Mock<MyChildClass> {CallBase = true};
myChildClass.Protected().Setup<bool>("MyMethod", 1).Returns(expected);
var result = myChildClass.Object.DoSomething(1);
Assert.AreEqual(expected, result);
}
}
Thanks everyone for your help! :)

Automatically calling an init function whenever an object is used for the 1st time

I have an object that only initializes itself with barebones data when constructed (fast), and loads itself for real (slow) when first accessed. The idea is that I'm creating a lot of these barebones objects at startup and hash them into a map, then fully load each object whenever it is individually accessed for the first time. The problem is that I cannot guarantee how clients will interact with this object, there are multiple public methods that might be invoked.
Is there a good pattern to support this kind of situation? The obvious (and my current) solution is to track state with an internal bool, check against that bool in every function that might be invoked, and load that way. But that requires code duplication of that behavior across all public functions, and is vulnerable to errors.
I can imagine a single point-of-entry method that then dishes out behaviors based on a client request type etc., but before I go consider going down that road I want to see if there's a commonly accepted approach/pattern that I might not be aware of. I'm doing this in C#, but any insight is appreciated.

If I understood what you want to achieve, you are looking for the Proxy Design Pattern, more specifically, a virtual Proxy.
Refer to http://www.dofactory.com/net/proxy-design-pattern
A small example would be something like:
public abstract class IObjectProvider
{
public abstract IObjectProvider Object{get;}
public abstract void doStuff();
}
public class RealObject : IObjectProvider
{
public RealObject()
{
//Do very complicated and time taking stuff;
}
public override IObjectProvider Object
{
get { return this; }
}
public override void doStuff()
{
//do this stuff that these objects normally do
}
}
public class ObjectProxy : IObjectProvider
{
private IObjectProvider objectInstance = null;
public override IObjectProvider Object
{
get
{
if (objectInstance == null)
objectInstance = new RealObject();
return objectInstance;
}
}
public override void doStuff()
{
if(objectInstance!=null)
objectInstance.doStuff();
}
}
public class SkeletonClass
{
public IObjectProvider Proxy1 = new ObjectProxy();
public IObjectProvider Proxy2 = new ObjectProxy();
}
static void Main(String[] args)
{
//Objects Not Loaded
SkeletonClass skeleton = new SkeletonClass();
//Proxy1 loads object1 on demand
skeleton.Proxy1.Object.doStuff();
//Proxy2 not loaded object2 until someone needs it
}

Here's an example of dynamic proxy approach.
using System;
using System.Diagnostics;
using Castle.DynamicProxy; //Remember to include a reference, too. It's nugettable package is Castle.Core
namespace ConsoleApp
{
public class ActualClass
{
//Have static instances of two below for performance
private static ProxyGenerator pg = new ProxyGenerator();
private static ActualClassInterceptor interceptor = new ActualClassInterceptor();
//This is how we get ActualClass items that are wrapped in the Dynamic Proxy
public static ActualClass getActualClassInstance()
{
ActualClass instance = new ActualClass();
return pg.CreateClassProxyWithTarget<ActualClass>(instance, interceptor);
}
//Tracking whether init has been called
private bool initialized = false;
//Will be used as evidence of true initialization, i.e. no longer null
private int? someValue = null;
public void Initialize()
{
if (!initialized)
{
//do some initialization here.
someValue = -1; //Will only get set to non-null if we've run this line.
initialized = true;
}
}
//Any methods you want to intercept need to be virtual!
public virtual int replaceValue(int value)
{
//below will blow up, if someValue has not been set to -1 via Initialize();
int oldValue = someValue.Value;
someValue = value;
return oldValue;
}
//block off constructor from public to enforce use of getActualClassInstance
protected ActualClass() { }
}
public class ActualClassInterceptor : ActualClass, IInterceptor
{
public void Intercept(IInvocation invocation)
{
//Call initialize before proceeding to call the intercepted method
//Worth noting that this is the only place we actually call Initialize()
((ActualClass)invocation.InvocationTarget).Initialize();
invocation.Proceed();
}
}
class Program
{
static void Main(string[] args)
{
ActualClass instance1 = ActualClass.getActualClassInstance();
ActualClass instance2 = ActualClass.getActualClassInstance();
int x1 = instance1.replaceValue(41);
int x2 = instance2.replaceValue(42);
int y1 = instance1.replaceValue(82);
Debug.Assert(y1 == 41);
int y2 = instance2.replaceValue(84);
Debug.Assert(y2 == 42);
var read = Console.ReadKey();
}
}
}

Using Moq to verify a method call on class in a collection

I'm not sure if I'm using Moq the right way, so if anyone could help, I'd be grateful.
I want to test the call of Clone() method on object in a collection. The test looks like this:
[Test]
public void CloneTest()
{
var mdFake = new Mock<MachineDecision>();
var clonable = mdFake.As<ICloneable>();
clonable.Setup(x => x.Clone()).Verifiable();
var decision = new Decision()
{
MachineDecisions = new List<MachineDecision> { mdFake.Object }
};
var newDecision = (Decision) decision.Clone();
clonable.Verify(x => x.Clone());
}
The test fails: Moq.MockException :
Expected invocation on the mock at least once, but was never performed: x => x.Clone() but I believe it should actually pass.
Used classes look as follows:
public class Decision : Entity<Guid>, ICloneable
{
public Decision()
{
Id = Guid.NewGuid();
MachineDecisions = new List<MachineDecision>();
}
public List<MachineDecision> MachineDecisions { get; set; }
public object Clone()
{
var obj = new Decision();
if (this.MachineDecisions != null)
{
obj.MachineDecisions = MachineDecisions.Select(item => (MachineDecision) item.Clone()).ToList();
}
return obj;
}
}
public class MachineDecision : Entity<Guid>, ICloneable
{
//...
}

There are two options available.
First, you can make an implementation of method Clone() virtual and your test will be 'Green'
public class MachineDecision : Entity<Guid>, ICloneable
{
public virtual object Clone()
{
throw new NotImplementedException();
}
}
Second, you can invoke Clone() method from ICloneable interface: (MachineDecision)(item as ICloneable).Clone(); and your test will be 'Green' also.
public class Decision : Entity<Guid>, ICloneable
{
public Decision()
{
Id = Guid.NewGuid();
MachineDecisions = new List<MachineDecision>();
}
public List<MachineDecision> MachineDecisions { get; set; }
public object Clone()
{
var obj = new Decision();
if (this.MachineDecisions != null)
{
obj.MachineDecisions = MachineDecisions.Select(item =>
{
return (MachineDecision)(item as ICloneable).Clone();
}).ToList();
}
return obj;
}
}
I realise that now it is not the best code but it is up to you how to refactor it further.

I'd do it like this:
[Test]
public void CloneTest()
{
// create the mock
var mdFake = new Mock<MachineDecision>();
var decision = new Decision
{
// setup (pass it to my collection)
MachineDecisions = new List<MachineDecision> { mdFake.Object }
};
// call the method being tested (you need to make Clone() virtual)
decision.Clone();
// check for the side effects -> It was called once !
mdFake.Verify(x => x.Clone(), Times.Once());
}
I hope this helps you.
EDIT - I'm sorry, as it was pointed in the comments - I forgot to mention, that what I'm suggesting requires you to make Clone() (in MachineDecision) - virtual, which might not be ideal in your case.

Try this:
...
clonable.Expect(x => x.Clone()).Verifiable().Returns(null);
...
clonable.Verify();

Unit Testing of a static factory method containing logic

before I begin with my question I want to point out that I am aware that there are tons of similar questions on stack overflow. Unfortunately none of these questions helped me finding a good solution in my concrete scenario.
The Problem:
I want to write a unit test for a static factory method which contains logic. I am looking for a way to unit test this method even if it is static. If that is not possible maybe someone can point out a better design for my class under test. I also considered using IoC but couldn't see the advantage considering unit-testing.
The Code:
public class Db
{
private XmlMapping mapping;
public static Db<T> Create()
{
var mapping = XmlMapping.Create(typeOf(T).Name);
return new Db(mapping);
}
private Db(XmlMapping mapping)
{
this.mapping = mapping;
}
}
public class XmlMapping //class under test
{
public static XmlMapping Create(string filename) //method under test
{
try
{
ValidateFilename(filename);
//deserialize xml to object of type XmlMapping
var result = Deserialize(filename);
if (result.IsInValid())
throw Exception()
return result;
}
catch (Exception)
{
throw new DbException();
}
}
}
The method Create which I want to unit test is within the class XmlMapping. This method serializes a xml file and generates an object of type XmlMapping. I tried to write a stub for the serialization part. But didn't want to call my Database Factory with a Mapping class in the constructor (constructor injection).
Edit:
My database factory is generic. The generic type is used to figure out which xml file should be louded i.e.: typeOf(T) = Customer --> XmlMapping-File = Customer.xml
The Solution (Thx to Jeff!):
public class XmlMapping : IMapping //class under test
{
internal static Func<Type, IMapping> DeserializeHandler { get; set; }
static XmlMapping()
{
DeserializeHandler = DeserializeMappingFor;
}
public static IMapping Create(Type type)
{
try
{
var mapping = DeserializeHandler(type);
if (!mapping.IsValid())
throw new InvalidMappingException();
return mapping;
}
catch (Exception ex)
{
throw new DataException("Failed to load mapping configuration from xml file.", ex);
}
}
internal XmlMapping(IMapping mapping)
{
this.Query = mapping.Query;
this.Table = mapping.Table;
this.Entity = mapping.Entity;
this.PropertyFieldCollection = mapping.PropertyFieldCollection;
}
private XmlMapping() { }
}
[TestClass]
public class MappingTests //testing class
{
[TestMethod]
public void Create_ValidDeserialization_ReturnsObjectInstance()
{
XmlMapping.DeserializeHandler = MakeFakeHandlerFor(MakeMappingStub());
var result = XmlMapping.Create(typeof(ActivityDto));
Assert.IsInstanceOfType(result, typeof(XmlMapping));
}
}

I would use a fake action handler to assist in verifying the content of the call to deserialize. Let's add a Func delegate property and default that to your serialize method. Your XmlMapping class and test would like something like:
public class XmlMapping //class under test
{
static XmlMapping()
{
// Default the handler to the normal call to Deserialize
DeserializeHandler = Deserialize;
}
public static XmlMapping Create(string filename) //method under test
{
//deserialize xml to object of type XmlMapping
//preudocode:
var result = DeserializeHandler(string.Format("{0}.xml",filename));
//...
return result;
}
// Abstract indirection function to allow you to swap out Deserialize implementations
internal static Func<string, XmlMapping> DeserializeHandler { get; set; }
private static XmlMapping Deserialize(string fileName)
{
return new XmlMapping();
}
}
public class CreateTests {
public void CallingDeserializeProperly()
{
// Arrange
var called = false;
Func<string, XmlMapping> fakeHandler = (string f) =>
{
called = true; // do your test of the input and put your result here
return new XmlMapping();
};
// Act
XmlMapping.DeserializeHandler = fakeHandler;
var m = XmlMapping.Create("test");
// Assert
Assert.IsTrue(called);
}
}