We Keep Coding

ApprovalTests with TestCase (Nunit) use the same approval file

I have a problem that using ApprovalTests.Net and NUnit all the tests that are run use the same approval file which name originates from Test method name.
Is it possible to specify somehow for each test case separate approval file?
[Test]
[TestCase("test",TestName="Test1")]
[TestCase("test2",TestName="Test2")]
Testing_Method(string param)
{
//somestuff
ObjectApprover.VerifyJson(someObject); // fails for second case
// because it uses the same
// approval file as the first case,
// which is not the same
}

You need to use
ApprovalResults.ForScenario("extra info")
Like such:
[Test]
[TestCase("test",TestName="Test1")]
[TestCase("test2",TestName="Test2")]
Testing_Method(string param)
{
//somestuff
using (ApprovalTests.Namers.ApprovalResults.ForScenario(param))
{
ObjectApprover.VerifyJson(someObject);
}
}
This will result in 2 approval files:
TestClassName.Testing_Method.ForScenario.Test1.approved.json
TestClassName.Testing_Method.ForScenario.Test2.approved.json

Instead of [TestCase] attribute, you could use [TestCaseSource] attribute.
https://ignas.me/tech/nunit-testcasesource-example/ - TestCaseSource example with explanation.

Unit testing a void method that writes to a file and doesn't mutate any class members

I need to unit test a protected method since it has some core logic(I can't change this part as it is not in my control). It doesn't change any members so can't write assertions based on that. It doesn't return anything except reading from a registry and write to a log file. How can I write a unit test with Moq and VisualStudio NUnit testing for this scenario:
protected override void OnPluginStarting()
{
try
{
using (var key = registryAccessor.LocalMachine.OpenSubKey(ProdInstallRegKey))
{
foreach (string subkeyName in key.GetSubKeyNames())
{
using (var subkey = key.OpenSubKey(subkeyName))
{
string displayName = subkey.GetValue("DisplayName") == null ? string.Empty : subkey.GetValue("DisplayName").ToString();
if (!string.IsNullOrEmpty(displayName) && installedProducts.ContainsKey(displayName))
{
Log.Info(installedProducts[displayName] + " version is " + subkey.GetValue("DisplayVersion"));
}
}
}
}
}
catch (Exception ex)
{
Log.Error("An error occured while getting the version numbers of DDP components. "+ex.Message);
}
}
RegistryAccessor is a wrapper for the Registry and I have used it so that I can mock it from my unit tests. But how can I pass the mock registry object to the method when it does not take any params? The only things I have figured out is that I can use PrivateObject to create private and protected instances. Could someone please help?

You should have someway of injecting registryAccessor and logger, that way you can control what the code reads from registry and confirm the output by checking the call to the logger.
Otherwise you can try some integration tests (probably more difficult in your case) or something like TypeMock

You can't.
There are commercial tools like TypeMock that use the Profiler API to achieve things like that. This will allow to intercept sealed classes, static methods etc.

Incorrect TestOutcome status in Teardown when created from StaticTestFactory

Using MbUnit, I create tests using several StaticTestFactory methods, each having corresponding test setup and teardown methods. A requirement is to log test results to an external system, especially failed ones.
However, I am unable to get the correct test outcome status using TestContext.CurrentContext.Outcome.Status. Using below code, you will see that the test fails, but the Outcome.status is always returned as 'Passed' from FactoryAssignedTearDownMethod, even when both Gallio Icarus and Echo show the test as failed.
Looking for any workaround or fix to get the correct outcome in this scenario.
public class FactoryTest
{
[StaticTestFactory]
public static IEnumerable<Test> CreateStaticTests()
{
var testcase = new TestCase("simpletest" , () =>
{
Assert.Fail("staticfactory created test failed.");
});
testcase.TearDown = FactoryAssignedTearDownMethod;
yield return testcase;
}
public static void FactoryAssignedTearDownMethod()
{
//outcome value is always 'Passed', even when test fails
TestLog.WriteLine("Test Outcome Status from factory assigned method: " + TestContext.CurrentContext.Outcome.Status);
}
}

I worked around this by writing a Gallio TestRunnerExtension. By handling the TestStepFinished event, I can get the proper test result for all tests created with the StaticTestFactory.

Using category expressions in Nunit console runner

I was reading through this link on category expressions when using /include or /exclude statement. I want to be able to include only run test to be run out of two tests available or run all tests but using the /include:A+B or /exclude:A. However, for some reason, it displays the wrong number of tests to be run and/or not run. Why is that?
Can anyone provide me with an example on how to category expressions (by manipulating source code) and add how to run the command in the console?
Essentially what I did was:
using System;
using NUnit;
using NUnit_Application;
using NUnit.Framework;
namespace NUnit_Application.Test
{
[TestFixture]
[Category("MathS")]
public class TestClass
{
[TestCase]
[Category("MathA")]
public void AddTest()
{
MathsHelper helper = new MathsHelper();
int result = helper.Add(20, 10);
Assert.AreEqual(40, result);
}
[TestCase]
[Category("MathB")]
public void SubtractTest()
{
MathsHelper helper = new MathsHelper();
int result = helper.Subtract(20, 10);
Assert.AreEqual(10, result);
}
}
}
And my command line statement was
nunit-console /framework:net-4.0 /run:NUnit_Application.Test.TestClass.AddTest C:~\NUnit_Application\NUnit_Application\NUnit_Application.Test\bin\Debug\NUnit_Application.Test.dll /include:"MathA"
The thing is, the console is familiar with what the commands means and it says it included Math A category. However, it shows that zero tests have ran and zero tests have not run.
I'm running NUnit 2.6.2, the console runner.

Here is command I used initially:
nunit-console /framework:net-4.0 /run:NUnit_Application.Test.TestClass.AddTest C:~\NUnit_Application\NUnit_Application\NUnit_Application.Test\bin\Debug\NUnit_Application.Test.dll /include:"MathA"
I noticed if I just call TestClass and not the individual test case, it works:
nunit-console /framework:net-4.0 /run:NUnit_Application.Test.TestClass C:~\NUnit_Application\NUnit_Application\NUnit_Application.Test\bin\Debug\NUnit_Application.Test.dll /include:"MathA"

I think it's because you have the whole class with the attribute :
[Category("MathS")]
So it skips over it.

VS MSTest runner locks System.Data.SQLite.dll when running in memory tests

I'm using Fluent NHibernate to run in-memory database tests (MS Test) using SQLite 1.0.66.0:
[TestClass]
public abstract class InMemoryDatabaseTest
{
private NHibernate.Cfg.Configuration configuration;
private ISessionFactory sessionFactory;
[TestInitialize]
public void Initialize()
{
// All "CreateConfiguration" does is load FNh mappings.
this.configuration = new NhConfigurationBuilder()
.CreateConfiguration()
.Database(() => SQLiteConfiguration.Standard.InMemory())
.BuildConfiguration();
this.sessionFactory = this.configuration.BuildSessionFactory();
}
[TestCleanup]
public void Cleanup()
{
new SchemaExport(this.configuration).Drop(false, true);
sessionFactory.Dispose();
}
protected ISession CreateSession()
{
var session = this.sessionFactory.OpenSession();
// Re-create the database every time a new session is created.
new SchemaExport(this.configuration)
.Execute(script: false, export: true, justDrop: false, connection: session.Connection, exportOutput: null);
session.BeginTransaction();
return session;
}
}
And then using this as an example:
[TestClass]
public class MessagesControllerTests : InMemoryDatabaseTest
{
[TestMethod]
public void SQLite_should_have_all_handles_released()
{
using (var session = this.CreateSession())
{
// Don't even need to do anything.
}
}
}
After running this test, I try to Clean the whole solution. The results are as follows:
When running this test (CTRL + R, CTRL + T), the clean is able to succeed as expected.
When debugging this test in (CTRL + R, T), the clean fails with the error: C:\Windows\Microsoft.NET\Framework\v4.0.30319\Microsoft.Common.targets(3607,9): warning MSB3061: Unable to delete file "PathToProject\bin\Debug\System.Data.SQLite.DLL". Access to the path 'PathToProject\bin\Debug\System.Data.SQLite.DLL' is denied.
My first thought was ok, delete the DLL. When I try to, I'm prompted that QTAgent32.exe is currently using the DLL. I used Process Explorer to verify this. For some reason the ms test runner is keeping a handle on the DLL. I've tried modifying the Cleanup metehod with some suggestions from another question, but it still didn't work:
[TestCleanup]
public void Cleanup()
{
new SchemaExport(this.configuration).Drop(false, true);
sessionFactory.Close();
sessionFactory.Dispose();
SQLiteConnection.ClearAllPools();
GC.Collect();
}
I've been able to reproduce this on 3 different machines. Any know method to solve this issue would be greatly appreciated.
Update: I've cleaned up some linguistic confusion. The actual solution configuration can be in Debug/Relase. However, running the tests versus debugging the tests causes the difference in error messages.

I was continually having a similar problem (SQLite.dll being locked by the Visual Studio test runner, although in Debug mode mostly and using MbUnit tests) and found that using TestDriven.net to run my tests solved the problem for me. I never looked any further into the MSTest runner after this, sorry.