I'm writing a project using Win7 x64. Some part of my tests requires using SQLServer CE which only represents support for x86. I'm using Visual Studio 2010 Express and I gotta change platform target for my projects manually editing *.cproj files to run, for example, schema export test (NHibernate). Howcome I run a part of my tests depending on a platform target.
Thanks!
I don't know if there's a built-in mechanism in NUnit to handle this scenario, but at the very least you can use preprocessor directives.
For instance, create a "Debug x86" solution configuration, targeting x86. Then define the DEBUG_X86 conditional compilation symbol (in the properties of the project). Finally, surround your unit test with preprocessor directives:
#if DEBUG_X86
[Test]
public void Test()
{
// This test will only run when compiled with Debug x86
}
#endif
Edit: Actually, you don't even have to create a new solution configuration, as it's possible to define the conditional symbols depending on the platform (https://stackoverflow.com/a/1313450/869621). So define a WIN32 compilation symbol, and surround your test with it:
#if WIN32
[Test]
public void Test()
{
// This test will only run when compiled for x86
}
#endif
Related
I'm trying to setup AppVeyor for my project (here) and I can't seem to find a way to only have it run tests from the .NET Core Unit test project.
This is the link for the AppVeyor project: ci.appveyor.com/project/Sergio0694/neuralnetwork-net
I also have a .NET Framework Unit test project that executes some GPU-based tests, so they just fail when run with AppVeyor as they're missing both a CUDA GPU and the required cuDNN files.
This is my configuration:
version: 1.0.{build}
image: Visual Studio 2017
before_build:
- cmd: dotnet restore
build:
verbosity: minimal
test:
assemblies:
only:
- NeuralNetwork.NET.Unit.dll
categories:
except:
- NetworkTest
That NetworkTest category that's skipped is just a group of tests that are very CPU intensive and require some time to run, so I'm just skipping them now to make the builds finish sooner.
The project builds fine, but AppVeyor keeps running the tests in the other NeuralNetwork.NET.Cuda.Unit.dll assembly as well, which fail as expected and cause the whole build to be marked as failing.
So my questions are:
Am I doing something wrong here? Why isn't the assembly constraint respected?
Is there another way to have AppVeyor only run the tests from NeuralNetwork.NET.Unit.dll?
EDIT: I've tried to set the "all assemblies excluding" option and all of these combinations to specify the .NET Framework Unit test project to skip:
NeuralNetwork.NET.Cuda.Unit.dll
**\*.NeuralNetwork.NET.Cuda.Unit.dll
**\NeuralNetwork.NET.Cuda.Unit.dll
The tests keep being executed (failing as expected), am I missing something here?
EDIT #2: This is the workaround I'm using for now, I've manually excluded all the test categories in the .NET Framework project and so far this seems to be working (even if it's not so clean to see):
version: 1.0.{build}
image: Visual Studio 2017
configuration: Release
before_build:
- cmd: dotnet restore
build:
verbosity: minimal
test:
categories:
except:
- CuDnnInceptionLayerTest
- CuDnnLayersTest
- GpuExtensionsTest
- SerializationTest
.NET Core tests detection works differently than detection of classic .NET Framework tests. When searching .NET Core tests, AppVeyor look for .csproj files with certain properties, not for assemblies. Assembly filter does not work for .NET Core tests (Categories filter works though).
Sorry for confusion. Created this issue to implement behavior similar to assembly name filtering, but based on .csproj file names for .NET Core tests auto-detection.
For now workaround is to do a "black list", e.g. list all .NET Framework assemblies you do not want to run under assemblies/except. You can use wildcard syntax to make it more compact as described here.
I have a NUnit test project that I'm trying to run on my TFS 2012 Build server.
I'm getting the following error:
Exception System.InvalidOperationException, Exception thrown
executing tests
No test is available in C:\Builds\1\Projects\NUNIT
Test\bin\NUnitTestProject.dll C:\Builds\1\Projects\NUNIT
Test\bin\NUnitTestProject.dll. Make sure that installed test
discoverers & executors, platform & framework version settings are
appropriate and try again.
I Have the NUnit.VisualStudio.TestAdapter 1.1.0.0 installed from NuGet and referenced in the project.
I also have set the Version control path to custom assemblies set on the controller, which contains (they basically are a copy of the NuGet package):
nunit.core.dll
nunit.core.interfaces.dll
nunit.util.dll
NUnit.VisualStudio.TestAdapter.dll
For test purposes, I'm only building the NUnit test project.
My test class is very simple and is the following:
using NUnit.Framework;
namespace NUnitTestProject
{
[TestFixture]
public class MySuperTests
{
[Test]
public void MyFirstUnitTestMethod_ReturnsTrue()
{
Assert.IsTrue(true);
}
}
}
Is there something I haven't done correctly?
Edit 1:
Using TFS 2012 Update 4 on my build server
Edit 2:
The problem I believe lies in the 1.1.0.X version.
Updating to 1.1.0.7 I did not have the exception anymore, however, I did have the tests not found problem.
There are some issues going on about that on github https://github.com/nunit/nunit-vs-adapter/
Edit 3:
The problem has been fixed in 1.1.0.8 of the NUnit test adapter!
Do you have your test runner set correctly in the build defiintion?
Image shows TFS2013 but 2012 has a different option for 2010 compatability which may be selected. The Visual Studio Test Runner should support multiple test frameworks, whereas the 2010 compatability one only supports MSTest
When running unit tests in my project, a few are decorated with a HostType("VS IDE") attributes to indicate that the test must take place within Visual Studio to succeed. This works perfectly for any test that does not interact with my VSIX package.
From my looking around it seems that the VS IDE host loads the default standard configuration for Visual Studio instead of the 12.0Exp experimental hive in which my package is registered.
I can make this work by setting up testsettings that configure the host and it's parameters, but then all my tests run in that adapter. I've tried using the generated XML as a parameter to the host type but it doesn't seem to work.
<VSSDKTestHostRunConfig name="VS IDE" HiveKind="DevEnv" HiveName="12.0Exp" xmlns="http://microsoft.com/schemas/VisualStudio/SDK/Tools/IdeHostAdapter/2006/06" />
[TestMethod]
[HostType("VS IDE", #"<VSSDKTestHostRunConfig name=""VS IDE"" HiveKind=""DevEnv"" HiveName=""12.0Exp"" xmlns=""http://microsoft.com/schemas/VisualStudio/SDK/Tools/IdeHostAdapter/2006/06"" />")]
public void PackageLoadTest()
Is there any way to use attributes or code to configure the VS IDE host adapter for my unit tests dependent on that environment?
Add something like:
[TestProperty("VsHiveName", "12.0Exp")]
or
[Microsoft.VisualStudio.TestTools.UnitTesting.TestProperty(VsIdeTestHostContants.TestPropertyName.RegistryHiveName, "12.0Exp")]
We're using .NET 4.5 and have had no problem with open generic exports in MEF up until a month ago when it suddenly stopped working. 70 tests on our CI server turned red and we traced it down to missing parts in the container.
I found it strange, so I wrote this test code:
var catalog = new TypeCatalog(typeof(Impersonator<>), typeof(Cache<>));
var parts = catalog.Parts;
But it looks like none of the open generic types will register in the catalog. Parts is a TypeCatalog with Types = <Empty>.
Usually we go about this by declarative exports using attributes as in this SO question, but none of the alternatives seems to be working anymore.
Any idea will be most appreciated.
Any idea what triggered it to start failing? Did you get a .NET update in that timeframe?
Out of curiosity if you write a simple console app targeting .NET 4.5 does this issue repro?
The reason I ask is I believe there was an update released for .NET 4.5 to make sure open generics didn't break existing .NET 4.0 apps and so if for some reason your tests are running in .NET 4.0 mode then then the open generic support will be disabled which could cause what you are seeing.
I resolved this in my web app by putting the entry < httpRuntime targetFramework="4.5" /> in my web.config. For non-web apps, make sure the app.config has the entry
< supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />.
see http://blogs.msdn.com/b/webdev/archive/2012/11/19/all-about-httpruntime-targetframework.aspx for more info.
I know this is an old issue, but I have some additional information that may help others.
I ran into the same issue. My unit tests were working then suddenly stopped. We narrowed it down to a pair of Windows updates: KB2840642v2 and KB2840642. After uninstalling these, MEF started working properly again in unit tests. I didn't see anything in the release notes for those updates that should affect things, but you know how that goes.
However, I am now running Windows 8 unfortunately, and the issue has popped up again. I do not have those updates installed. I think they were incorporated natively into the operating system.
When you say “on our CI server”, that does not tell us much about how you’re running the tests. However, I seem to get an experience like yours using vstest.console.exe.
I found a situation where MEF’s automatic open type closing feature, new in .net-4.5, will appear to break when running tests with vstest.console.exe. Basically, vstest.console.exe might decide to run in .net-4.0 compatibility mode as if /Framework:Framework40 had been passed to it. .net-4.5 assemblies will load and run perfectly fine even when loaded by a .net-4.0 runtime, but MEF detects that you’re running in .net-4.0 mode and disables its support for automatically closing open generic types. You can get MEF to behave properly again by forcing vstest.console.exe to run in .net-4.5 mode by passing the /Framework:Framework45 switch.
(This tested on a machine with .net-4.6.1 installed, though, not sure if that changes things yet more).
Minimal Repro
When I make a simple test assembly targeting the .net-4.5 framework, vstest.console.exe correctly guesses that it should run the tests with .net-4.5. However, when I make an assembly that integrates into a more complex build environment, vstest.console.exe suddenly starts running in .net-4.0 mode rather than .net-4.5 mode. So I started with my complex build environment and started paring things down until the issue disappeared.
To get vstest.console.exe to guess the wrong framework, you need two assemblies. In one, define a custom assembly Attribute. In another, apply that custom assembly attribute and define your unit tests.
First assembly:
using System;
[AttributeUsage(AttributeTargets.Assembly)]
public class BreakVSTestFrameworkDetectionAttribute : Attribute
{
}
Second assembly referencing the prior one:
using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.ComponentModel.Composition;
using System.ComponentModel.Composition.Hosting;
[assembly: BreakVSTestFrameworkDetection]
[InheritedExport]
public interface IGenericService<T>
{
void Print(T thing);
}
public class SomeGenericService<T> : IGenericService<T>
{
public void Print(T thing) => Console.WriteLine($"{typeof(T)}:{thing}");
}
[TestClass]
public class UnitTest1
{
[TestMethod]
public void Run()
{
using (var catalogue = new ApplicationCatalog())
using (var container = new CompositionContainer(catalogue))
{
container.GetExportedValue<IGenericService<string>>().Print("asdf"); // System.String:asdf
container.GetExportedValue<IGenericService<int>>().Print(123); // System.Int32:123
}
}
static void Main(string[] args) => new UnitTest1().Run();
}
With these two assemblies and the defined test, I get the following outcomes in different scenarios:
Running the tests using Visual Studio 2015 Community Editions’s GUI works perfectly fine.
Running the code directly as an executable works fine.
C:\Users\ohnob\OneDrive\Documents\Visual Studio 2015\Projects\MefGenericsUnitTests>bin\Debug\MefGenericsUnitTests.exe
System.String:asdf
System.Int32:123
Running the tests with something like mstest (except I can’t tell why one would or wouldn’t use this instead of vstest) magically works (at least in this case) (output trimmed).
C:\Users\ohnob\OneDrive\Documents\Visual Studio 2015\Projects\MefGenericsUnitTests>"\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" /testcontainer:bin\Debug\MefGenericsUnitTests.exe
Microsoft (R) Test Execution Command Line Tool Version 14.0.23107.0
Copyright (c) Microsoft Corporation. All rights reserved.
Loading bin\Debug\MefGenericsUnitTests.exe...
Starting execution...
Results Top Level Tests
------- ---------------
Passed UnitTest1.Run
1/1 test(s) Passed
Running with straight up vstest.console.exe fails (output trimmed):
C:\Users\ohnob\OneDrive\Documents\Visual Studio 2015\Projects\MefGenericsUnitTests>"\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" bin\Debug\MefGenericsUnitTests.exe
Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
Failed Run
Error Message:
Test method UnitTest1.Run threw exception:
System.ComponentModel.Composition.ImportCardinalityMismatchException: No exports were found that match the constraint:
ContractName IGenericService(System.String)
RequiredTypeIdentity IGenericService(System.String)
But vstest.console.exe /Framework:Framework45 succeeds:
C:\Users\ohnob\OneDrive\Documents\Visual Studio 2015\Projects\MefGenericsUnitTests>"\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" /Framework:Framework45 bin\Debug\MefGenericsUnitTests.exe
Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
Passed Run
Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 3.7855 Seconds
I think you can not use open generic and should provide a concrete running class like:
var catalog = new TypeCatalog(typeof(Impersonator<type1>), typeof(Impersonator<type2>), typeof(Cache<type3>), typeof(Cache<type4>));
var parts = catalog.Parts;
I tried using interfaces like IImpersonator and ICache and does not work also:
interface IImpersonator { }
class Impersonator<T> : IImpersonator
...
var catalog = new TypeCatalog(typeof(IImpersonator), typeof(ICache));
var parts = catalog.Parts;
When I run the following test in Gallio's Icarus it passes, but when I step into it using TestDriven.NET (Test With->Debugger), it fails because the parameters are not set according to the Row attributes.
I was expecting that the method would be called once for each Row attribute applied.
What am I doing wrong? If nothing, then what do I need to do to debug these tests when they break? Should I avoid parametrized tests if they are not debuggable?
[TestFixture]
public class TestDrivenIgnoresMbUnitAttributesWhenDebugging
{
[Test]
[Row(1)]
[Row(2)]
public void SomeFunc(int x)
{
Assert.AreNotEqual(default(int), x);
}
}
Hmm... did you install TestDriven.Net BEFORE installing Gallio?
If not, then the Gallio extensions for TestDriven.Net will not be installed. In that case, TestDriven.Net might run the test in "ad-hoc" mode with default values for its parameters.
It should be pretty to tell whether this is the case. If the Gallio extensions for TestDriven.Net are installed then you'll see a "Gallio" banner message in the Output Window during test execution. If not you may see something else like "ad-hoc".
To fix the problem, reinstall Gallio. Alternately you can use the add/remove features part of the Gallio installer then ensure that the TestDriven.Net components are selected for installation (under "Test Runners").