We Keep Coding

Why does NUnit ignore the tests in my C# source?

I'm trying to use the NUnit framework to write some tests. The test source is:
namespace UnitTests
{
// obsolete: [NUnit.Framework.TestFixture]
public class NameSorterTests
{
public NameSorterTests () {
System.Console.WriteLine("Creating test object");
}
[NUnit.Framework.SetUp]
public void GetReady()
{
System.Console.WriteLine("starting test");
}
[NUnit.Framework.TearDown]
public void Clean()
{
System.Console.WriteLine("closing test");
}
[NUnit.Framework.Test]
public void monotonicSequence ()
{
System.Console.WriteLine("running test");
NUnit.Framework.Assert.IsTrue (true);
}
}
}
and when I try to run the tests, from the command line:
$ mcs -r:nunit.framework -target:library UnitTests.cs && nunit-console UnitTests.dll -run:blobby
Note: nunit-console shipped with Mono is deprecated, please use the NUnit
NuGet package or some other form of acquiring NUnit.
NUnit version 2.4.8
Copyright (C) 2002-2007 Charlie Poole.
Copyright (C) 2002-2004 James W. Newkirk, Michael C. Two, Alexei A. Vorontsov.
Copyright (C) 2000-2002 Philip Craig.
All Rights Reserved.
Runtime Environment -
OS Version: Unix 4.14.12.6
CLR Version: 4.0.30319.42000 ( 5.16.0.179 (tarball Fri Nov 30 09:54:19 AEST 2018) )
Selected test: blobby
Tests run: 0, Failures: 0, Not run: 0, Time: 0.032 seconds
Note the apparently spurious -run:blobby argument; if I omit that, the Selected test: blobby line is omitted, but the rest of the output is the same. It doesn't seem to care that I specified a non-existent test. What is it doing, and how can I make it run my tests?
BTW, I don't care about the deprecation warning, so long as it just works for now; the exercise is just for a job application.

For NUnit 2.4, the [TestFixture] atttribute on the test class is required.

Are you sure [TestFixture] is obsolete in the NUnit you are using? What happens when you uncomment the [TestFixture] attribute on NameSorterTests?

How to filter NUnit tests by category using "dotnet test"

I have a project that has a
[TestFixture, Category("Oracle")]
and a
[TestFixture, Category("OracleOdbc")]
with a couple of tests which I would like to execute separately using dotnet test.
Here's what I tried after some Googling:
dotnet test MyProject.csproj --where "cat==Oracle" but this switch does not exists anymore.
dotnet test MyProject.csproj --filter Category="Oracle" yields 0 applicable tests: No test is available in ....
Then, I've stumbled over this article and although it describes MSTest (and NUnit has the CategoryAttribute and not a TestCategoryAttribute), I've tried
dotnet test MyProject.csproj --filter TestCategory="Oracle"
Bingo. This time all "Oracle" tests were executed. But now comes the confusing part. If I run dotnet test MyProject.csproj --filter TestCategory="OracleOdbc", all tests are being executed, including "Oracle" and "OracleOdbc". This makes me wonder if TestCategroy is the proper way to go for NUnit or if this is a bug.
I'm using .NET Command Line Tools (2.1.2) and the project references are:
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.5.0" />
<PackageReference Include="NUnit" Version="3.8.1" />
<PackageReference Include="NUnit3TestAdapter" Version="3.9.0" />
<PackageReference Include="TeamCity.VSTest.TestAdapter" Version="1.0.7" />
BTW, I don't know if it matters but my test project is multi-targeting netcoreapp2.0 and net462.

This might not be very helpful, but it seems to be working for me correctly. I created the projects using the dotnet-cli.
First I installed the NUnit3 test adapter instructions from here. This only needs to be run once on each machine so you don't need to do it again if you have already run it.
dotnet new -i NUnit3.DotNetNew.Template
Then I created my solution, created my test project and added the test project to the solution.
dotnet new sln -n Solution
dotnet new nunit -n TestProject -o tests\TestProject
dotnet sln add tests\TestProject\TestProject.csproj
Then I updated UnitTest1.cs to include two test fixtures, one with the category Oracle and one with the category OracleOdbc.
using NUnit.Framework;
namespace Tests
{
[TestFixture]
[Category("Oracle")]
public class OracleTests
{
[Test]
public void OracleTest()
{
Assert.Fail();
}
}
[TestFixture]
[Category("OracleOdbc")]
public class OracleOdbcTests
{
[Test]
public void OracleOdbcTest()
{
Assert.Fail();
}
}
}
Then I can specify which category I choose to run.
dotnet test tests/TestProject/TestProject.csproj --filter TestCategory="Oracle"
or
dotnet test tests/TestProject/TestProject.csproj --filter TestCategory="OracleOdbc"
both run only one test and the message shows it is the correct test that fails.
Using DotNet-Cli version 2.1.4 and NUnit3TestAdapter version 3.9.0

In Nunit Framework, Category Attribute can be on the level of the Method.
Example:
public class NUnitTest
{
[Test]
[Category("CategoryA")]
public void TestMethod1()
{
}
[Test]
[Category("CategoryB")]
public void TestMethod2()
{
}
}
and command is:
dotnet test --filter TestCategory=CategoryA #Runs tests which are annotated with [Category("CategoryA")].
Also, there are many options on the level of method and others
For more details: read

There are now two options for filtering tests by category using dotnet test. You can use either dotnet.exe's built-in test filtering syntax, or the NUnit filtering syntax.
First add the NUnit3TestAdapter to your project using NuGet:
install-package nunit3testadapter -proj YourProject
You can then filter tests either like this:
dotnet.exe test .\ --test-adapter-path:. --filter TestCategory=Foo
or like this:
dotnet.exe test .\ --test-adapter-path:. -- NUnit.Where="cat=Foo"
This blog post goes into further detail.

You can also place a filter inside your test if you like enums like me:
[Test]
public void Test_Foo()
{
// filter test
switch (GlobalDefinitions.Category)
{
// OK Test
case Category.Oracle:
case Category.SQL:
break;
// Do NOT test
case Category.MongoDb:
Assert.Inconclusive();
// Error
default:
Assert.Fail("Not implemented case");
}
// perform test...
}
Have the variable GlobalDefinitions.Category obtain the value from a resource file or whatever works best for you.
Edit make the same code shorter with Flags
Create your categories
[Flags] // <-------------------- Important to shorten code
public enum Category: int
{
None = 0,
Oracle = 1 << 0,
SQL = 1 << 1,
MongoDb = 1 << 2,
// future categories
ALL = -1
}
// create your filter method
public static void Filter(Category category)
{
if(GlobalDefinitions.Category.HasFlag(category) == false)
Assert.Inconclusive(); // do not perform test
// else perform test
}
// then create your test as:
[Test]
public void Test_Foo()
{
Filter(Category.SQL | Category.MongoDb); // place filter (in this test we are testing for SQL and MongoDb
// perform your test ...
}

How to programmatically run unit tests with Gallio and MBUnit?

I'm trying to programmatically check my unit tests are passing as part of my deployment process. The application uses MBunit and Gallio for it's unit testing framework.
Here's my code:
var setup = new Gallio.Runtime.RuntimeSetup();
setup.AddPluginDirectory(#"C:\Program Files\Gallio\bin");
using (TextWriter tw = new StreamWriter(logFilename))
{
var logger = new Gallio.Runtime.Logging.TextLogger(tw);
RuntimeBootstrap.Initialize(setup, logger);
TestLauncher launcher = new TestLauncher();
launcher.AddFilePattern(dllToRunFilename);
TestLauncherResult result = launcher.Run();
}
Here's the test which is contained in the DLL I'm loading (I've validated this works with the Icarus test runner):
public class Tests
{
[Test]
public void Pass()
{
Assert.IsTrue(true);
}
[Test]
public void Fail()
{
Assert.Fail();
}
}
When I run the application I get the following values in results
Which is incorrect as there are indeed tests to run! The log file has the following in it
Disabled plugin 'Gallio.VisualStudio.Shell90': The plugin enable
condition was not satisfied. Please note that this is the intended
behavior for plugins that must be hosted inside third party
applications in order to work. Enable condition:
'${process:DEVENV.EXE_V9.0} or ${process:VSTESTHOST.EXE_V9.0} or
${process:MSTEST.EXE_V9.0} or ${framework:NET35}'. Disabled plugin
'Gallio.VisualStudio.Tip90': The plugin depends on another disabled
plugin: 'Gallio.VisualStudio.Shell90'.
How do I resolve this issue and find the results to the tests?

This works for me, note I used this GallioBundle nuget to get gallio and mbunit, so perhaps there is some difference to what you have installed.
The log messages regarding plugins are expected, those plugins wont work if you are self-hosting the Gallio runtime.
using System;
using System.IO;
using Gallio.Runner;
using Gallio.Runtime;
using Gallio.Runtime.Logging;
using MbUnit.Framework;
public static class Program
{
public static void Main()
{
using (TextWriter tw = new StreamWriter("RunTests.log"))
{
var logger = new TextLogger(tw);
RuntimeBootstrap.Initialize(new RuntimeSetup(), logger);
TestLauncher launcher = new TestLauncher();
launcher.AddFilePattern("RunTests.exe");
TestLauncherResult result = launcher.Run();
Console.WriteLine(result.ResultSummary);
}
}
}
public class Tests
{
[Test]
public void Pass()
{
Assert.IsTrue(true);
}
[Test]
public void Fail()
{
Assert.Fail();
}
}
Tested like this:
› notepad RunTests.cs
› nuget.exe install -excludeversion GallioBundle
Installing 'GallioBundle 3.4.14.0'.
Successfully installed 'GallioBundle 3.4.14.0'.
› cd .\GallioBundle\bin
› csc ..\..\RunTests.cs /r:Gallio.dll /r:MbUnit.dll
Microsoft (R) Visual C# Compiler version 12.0.21005.1
for C# 5
Copyright (C) Microsoft Corporation. All rights reserved.
› .\RunTests.exe
2 run, 1 passed, 1 failed, 0 inconclusive, 0 skipped

These are instruction for running MBUnit tests in Visual Studio 2012 and above using a neat NUnit trick.
Firstly, install the NUnit Test Adapter extension (yes, NUnit)
Tools > Extension and Updates > Online > search for NUnit > install
NUnit Test Adapter.
You may need to restart the Visual Studio IDE.
Then, you simply need to add a new NUnit test attribute to your test methods. See example code here (notice the using statements at the top) ...
//C# example
using MbUnit.Framework;
using NuTest = NUnit.Framework.TestAttribute;
namespace MyTests
{
[TestFixture]
public class UnitTest1
{
[Test, NuTest]
public void myTest()
{
//this will pass
}
}
}
You can run and debug the test in visual studio as NUnit and Gallio Icarus GUI Test Runner will run them as MBUnit (enabling parallel runs for example). You will need to stop Gallio from running the NUnit tests by deleting the NUnit folder in the gallio install location i.e. C:\Program Files\Gallio\bin\NUnit

Running Functional Tests

When i try to run my tests from Visual Studio 2012 Ultimate i get this output
------ Discover test started ------
========== Discover test finished: 0 found (0:00:05.8242806) ==========
Here is the code:
[Then(#"the submitter company list is in alphabetical order")]
public void ThenTheSubmitterCompanyListIsInAlphabeticalOrder()
{
List<string> submitterCompanyList = _currentFilingPage.SubmitterCompanyList;
submitterCompanyList.Should().BeInAscendingOrder();
}
I have created a .bat file and from there i can run my tests. Please assits me with a way to run my tests from visual studio. (Extra Information: I can't see my tests on Test Explorer)

All the Tests must have the TestMethodAttribute so MSTest/Visual Studio can find them.
see: MSDN Anatomy of a Unit Test
So edit your code and add the TestMethod to your method so it can be found as a Test. Also your method must be added inside a TestClass:
[TestClass]
public class TestClass
{
[TestMethod]
public void ThenTheSubmitterCompanyListIsInAlphabeticalOrder()
{
}
}

sonar with gallio and opencover, code coverage: 0%

I'm using sonar to check my c# project. I would like to measure code coverage that's why i installed gallio and opencover. When I run soner-runner everything works fine, my unit test is performed, ... but the code coverage is 0% on the sonar web UI.
do you know the reason why the code coverage is 0%?
My solution, project and classes:
(S) SonarTestSolution
(P) ClassLibrary1
(C) Class1.cs
(P) ClassLibrary1NUnitTest
(C) Class1NUnitTest.cs
content of Class1.cs:
public class Class1 {
public String getTestString() { return "abc";}
}
content of Class1NUnitTest.cs:
[TestFixture]
public class Class1NUnitTest {
[Test]
public void createServiceFromFactoryTest() {
Class1 c = new Class1();
Assert.That(c.getTestString(), Is.EqualTo("abc"));
}}
sonar-project.properties:
sonar.projectKey=cs_sonar_test
sonar.projectVersion=1.0-SNAPSHOT
sonar.projectName=C# Sonar Test Project
sonar.sourceEncoding=UTF-8
sources=.
sonar.language=cs
sonar.donet.visualstudio.testProjectPattern=*NUnitTest*
If you want I can include the log og sonar-runner, and the generated coverage-report.xml and gallio-report.xml files

solved - the project was build another machine, this was the problem. Building the solution in the same folder solves the proble.