We Keep Coding

TFS 2012 Build Server NUnit Exception

I have a NUnit test project that I'm trying to run on my TFS 2012 Build server.
I'm getting the following error:
Exception System.InvalidOperationException, Exception thrown
executing tests
No test is available in C:\Builds\1\Projects\NUNIT
Test\bin\NUnitTestProject.dll C:\Builds\1\Projects\NUNIT
Test\bin\NUnitTestProject.dll. Make sure that installed test
discoverers & executors, platform & framework version settings are
appropriate and try again.
I Have the NUnit.VisualStudio.TestAdapter 1.1.0.0 installed from NuGet and referenced in the project.
I also have set the Version control path to custom assemblies set on the controller, which contains (they basically are a copy of the NuGet package):
nunit.core.dll
nunit.core.interfaces.dll
nunit.util.dll
NUnit.VisualStudio.TestAdapter.dll
For test purposes, I'm only building the NUnit test project.
My test class is very simple and is the following:
using NUnit.Framework;
namespace NUnitTestProject
{
[TestFixture]
public class MySuperTests
{
[Test]
public void MyFirstUnitTestMethod_ReturnsTrue()
{
Assert.IsTrue(true);
}
}
}
Is there something I haven't done correctly?
Edit 1:
Using TFS 2012 Update 4 on my build server
Edit 2:
The problem I believe lies in the 1.1.0.X version.
Updating to 1.1.0.7 I did not have the exception anymore, however, I did have the tests not found problem.
There are some issues going on about that on github https://github.com/nunit/nunit-vs-adapter/
Edit 3:
The problem has been fixed in 1.1.0.8 of the NUnit test adapter!

Do you have your test runner set correctly in the build defiintion?
Image shows TFS2013 but 2012 has a different option for 2010 compatability which may be selected. The Visual Studio Test Runner should support multiple test frameworks, whereas the 2010 compatability one only supports MSTest

Open Generic types won't register in the catalog

We're using .NET 4.5 and have had no problem with open generic exports in MEF up until a month ago when it suddenly stopped working. 70 tests on our CI server turned red and we traced it down to missing parts in the container.
I found it strange, so I wrote this test code:
var catalog = new TypeCatalog(typeof(Impersonator<>), typeof(Cache<>));
var parts = catalog.Parts;
But it looks like none of the open generic types will register in the catalog. Parts is a TypeCatalog with Types = <Empty>.
Usually we go about this by declarative exports using attributes as in this SO question, but none of the alternatives seems to be working anymore.
Any idea will be most appreciated.

Any idea what triggered it to start failing? Did you get a .NET update in that timeframe?
Out of curiosity if you write a simple console app targeting .NET 4.5 does this issue repro?
The reason I ask is I believe there was an update released for .NET 4.5 to make sure open generics didn't break existing .NET 4.0 apps and so if for some reason your tests are running in .NET 4.0 mode then then the open generic support will be disabled which could cause what you are seeing.

I resolved this in my web app by putting the entry < httpRuntime targetFramework="4.5" /> in my web.config. For non-web apps, make sure the app.config has the entry
< supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5" />.
see http://blogs.msdn.com/b/webdev/archive/2012/11/19/all-about-httpruntime-targetframework.aspx for more info.

I know this is an old issue, but I have some additional information that may help others.
I ran into the same issue. My unit tests were working then suddenly stopped. We narrowed it down to a pair of Windows updates: KB2840642v2 and KB2840642. After uninstalling these, MEF started working properly again in unit tests. I didn't see anything in the release notes for those updates that should affect things, but you know how that goes.
However, I am now running Windows 8 unfortunately, and the issue has popped up again. I do not have those updates installed. I think they were incorporated natively into the operating system.

When you say “on our CI server”, that does not tell us much about how you’re running the tests. However, I seem to get an experience like yours using vstest.console.exe.
I found a situation where MEF’s automatic open type closing feature, new in .net-4.5, will appear to break when running tests with vstest.console.exe. Basically, vstest.console.exe might decide to run in .net-4.0 compatibility mode as if /Framework:Framework40 had been passed to it. .net-4.5 assemblies will load and run perfectly fine even when loaded by a .net-4.0 runtime, but MEF detects that you’re running in .net-4.0 mode and disables its support for automatically closing open generic types. You can get MEF to behave properly again by forcing vstest.console.exe to run in .net-4.5 mode by passing the /Framework:Framework45 switch.
(This tested on a machine with .net-4.6.1 installed, though, not sure if that changes things yet more).
Minimal Repro
When I make a simple test assembly targeting the .net-4.5 framework, vstest.console.exe correctly guesses that it should run the tests with .net-4.5. However, when I make an assembly that integrates into a more complex build environment, vstest.console.exe suddenly starts running in .net-4.0 mode rather than .net-4.5 mode. So I started with my complex build environment and started paring things down until the issue disappeared.
To get vstest.console.exe to guess the wrong framework, you need two assemblies. In one, define a custom assembly Attribute. In another, apply that custom assembly attribute and define your unit tests.
First assembly:
using System;
[AttributeUsage(AttributeTargets.Assembly)]
public class BreakVSTestFrameworkDetectionAttribute : Attribute
{
}
Second assembly referencing the prior one:
using System;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.ComponentModel.Composition;
using System.ComponentModel.Composition.Hosting;
[assembly: BreakVSTestFrameworkDetection]
[InheritedExport]
public interface IGenericService<T>
{
void Print(T thing);
}
public class SomeGenericService<T> : IGenericService<T>
{
public void Print(T thing) => Console.WriteLine($"{typeof(T)}:{thing}");
}
[TestClass]
public class UnitTest1
{
[TestMethod]
public void Run()
{
using (var catalogue = new ApplicationCatalog())
using (var container = new CompositionContainer(catalogue))
{
container.GetExportedValue<IGenericService<string>>().Print("asdf"); // System.String:asdf
container.GetExportedValue<IGenericService<int>>().Print(123); // System.Int32:123
}
}
static void Main(string[] args) => new UnitTest1().Run();
}
With these two assemblies and the defined test, I get the following outcomes in different scenarios:
Running the tests using Visual Studio 2015 Community Editions’s GUI works perfectly fine.
Running the code directly as an executable works fine.
C:\Users\ohnob\OneDrive\Documents\Visual Studio 2015\Projects\MefGenericsUnitTests>bin\Debug\MefGenericsUnitTests.exe
System.String:asdf
System.Int32:123
Running the tests with something like mstest (except I can’t tell why one would or wouldn’t use this instead of vstest) magically works (at least in this case) (output trimmed).
C:\Users\ohnob\OneDrive\Documents\Visual Studio 2015\Projects\MefGenericsUnitTests>"\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" /testcontainer:bin\Debug\MefGenericsUnitTests.exe
Microsoft (R) Test Execution Command Line Tool Version 14.0.23107.0
Copyright (c) Microsoft Corporation. All rights reserved.
Loading bin\Debug\MefGenericsUnitTests.exe...
Starting execution...
Results Top Level Tests
------- ---------------
Passed UnitTest1.Run
1/1 test(s) Passed
Running with straight up vstest.console.exe fails (output trimmed):
C:\Users\ohnob\OneDrive\Documents\Visual Studio 2015\Projects\MefGenericsUnitTests>"\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" bin\Debug\MefGenericsUnitTests.exe
Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
Failed Run
Error Message:
Test method UnitTest1.Run threw exception:
System.ComponentModel.Composition.ImportCardinalityMismatchException: No exports were found that match the constraint:
ContractName IGenericService(System.String)
RequiredTypeIdentity IGenericService(System.String)
But vstest.console.exe /Framework:Framework45 succeeds:
C:\Users\ohnob\OneDrive\Documents\Visual Studio 2015\Projects\MefGenericsUnitTests>"\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" /Framework:Framework45 bin\Debug\MefGenericsUnitTests.exe
Microsoft (R) Test Execution Command Line Tool Version 14.0.24720.0
Copyright (c) Microsoft Corporation. All rights reserved.
Starting test execution, please wait...
Passed Run
Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.
Test Run Successful.
Test execution time: 3.7855 Seconds

I think you can not use open generic and should provide a concrete running class like:
var catalog = new TypeCatalog(typeof(Impersonator<type1>), typeof(Impersonator<type2>), typeof(Cache<type3>), typeof(Cache<type4>));
var parts = catalog.Parts;
I tried using interfaces like IImpersonator and ICache and does not work also:
interface IImpersonator { }
class Impersonator<T> : IImpersonator
...
var catalog = new TypeCatalog(typeof(IImpersonator), typeof(ICache));
var parts = catalog.Parts;

UnitTestIsolationException: Throws Exception while running Shims test in Release/Debug mode

I am trying to run this unit test using Microsoft Shims, but it throws me exception in Shims.Context.Create(); method.
Environment: VS 2012, Win2K8 R2
namespace MyShimsUnitTest
{
[TestClass]
public class MyUnitTest
{
[TestMethod]
public void GetCurrentYear()
{
using (Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create())
{
// Some Logic...
}
}
}
}
Detailed Exception:
Result Message:
Test method MyShimsUnitTest.MyUnitTest.GetCurrentYear threw exception:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: UnitTestIsolation instrumentation failed to initialize. Please restart Visual Studio and rerun this test
Result StackTrace:
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext()
at Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create()
at MyShimsUnitTest.MyUnitTest.GetCurrentYear()

Shims require runtime instrumentation performed by the IntelliTrace profiler. The test runner is responsible for setting up the environment variables required for CLR to load the profiler as well as providing the list of types the profiler must instrument for Shims. The UnitTestIsolationException is thrown when the ShimRuntime is unable to locate and attach to the IntelliTrace profiler, which it expects to be already loaded by the CLR.
As Jin-Wook mentioned earlier, this problem occurs when the test is executed by a runner that does not perform the required profiler initialization. Test Explorer and the vstest.console.exe are two such runners that ship with Visual Studio 2012. At this time, the Visual Studio test runners do not perform the required profiler instrumentation when running tests in "legacy" mode, which happens when you have a .TESTSETTINGS file selected for your run or a .RUNSETTINGS file that forces legacy mode.
You may be able to use third-party test runners that support profiler instrumentation required by Shims.

I had the same issue. The solution to my problem was to uncheck the selected .testsettings file from the menu: TEST/Test Settings and here the item(s) above the Select Test Settings File.

It could be caused by not using the test explorer of vs 2012. To use the shim, you should run tests only using the test explorer.
You can use other test framework such as Nunit or Xunit with the shim if installing appropriate test runner for vs 2012. It can be downloaded from the vs extension manager.

I ran into this issue too. Thankfully the other answers here helped me fix my issue:
I'm using Resharper and when using the context menu I noticed that the runner is using MSTest. Even when finding the test in test explorer and selecting debug I received the same exception.
I then went into Resharpers's options and under Tools -> Unit Testing -> MsTest I unchecked "Enable MSTest support". This unfortunately disables the option to right click on your test and hit run/debug, but it did allow ShimsContext.Create() to behave correctly when selecting debug from the Test Explorer view!

Go to your TestProject Properties -> Under Debug section Check the "ENABLE NATIVE CODE DEBUGGING" checkbox.
This is should do.

We saw this error reported by Bamboo, our build server. It was invoking an MSbuild 4.0 task. The unit test work fine on the dev's local PCs. I deleted this bamboo task and created a new task that invokes Visual Studio 2012's vstest.console. The tests now pass but Bamboo is not able to count the number of tests. This is a Bamboo problem not mine.

Unit Testing - Extending the Visual Studio Unit Test Type - Not working

We're asked to move from NUnit to MSTest and now have to convert all the existing tests to the new platform. Most of it converted fine but we have an issue with parameterised tests. We found the following web site showing how to write an extension and downloaded the code for it.
http://blogs.msdn.com/b/vstsqualitytools/archive/2009/09/04/extending-the-visual-studio-unit-test-type-part-2.aspx
However, although it compiles fine, the RowTestClass test all fail to run with the following error message:
"Failed to initialize the unit test extension 'urn:RunAsTestClassAttribute': A unit test extension is not registered for the following attribute: Microsoft.VisualStudio.Test.Sample.UnitTestTypeExtension.RunAsSample.RunAsTestClassAttribute."
We're using VS2010 (10.0.40219.1 SP1Rel) (Microsoft .NET Framework Version 4.0.30319 SP1Rel)
I wonder if it has something to do with the following libraries:
Microsoft.VisualStudio.QualityTools.Common.dll
Runtime Version: v2.0.50727
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\ReferenceAssemblies\v2.0\Microsoft.VisualStudio.QualityTools.Common.dll
Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
Runtime Version: v4.0.30319
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PublicAssemblies\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
Microsoft.VisualStudio.QualityTools.Vsip.dll
v4.0.30319
C:\Windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.VisualStudio.QualityTools.Vsip\v4.0_10.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.QualityTools.Vsip.dll
Any help to get this working greatly appreciated.

Have you registered the new types (in the Windows Registry)? There's a section on how to do it in the tutorial - have a look at Row Test Test Type Extension: Registering your test type extension.
If you're having trouble, try downloading the complete code + registry keys from the MSDN code gallery.

You may want to look into using SpecFlow to make your tests more parameterized. It is traditionally a BDD tool, but it will allow you to make re-usable steps that take parameters.
For example you can create a step saying
"Given the user 'admin' exists."
and on the c# side you get a method like this:
[Given(#"the user '(.*)' exists"]
public void GivenTheUserExists(string name)
{
//create the user
}
Going forward you can reuse the step in any "feature file" without having to rewrite the implementation of that step.

Just refer to the blog post Extending Unit Tests in Visual Studio. And see what you might have missed. I tried the same, and it works well for me.

What are the Steps needed to use NUnit?

I Developed a small Application in C#. I want to test my application with NUnit.I am a new to NUnit.I Installed NUnit but don't Know How to use it what are the basic steps needed for it or please provide me a good reference link about using NUnit.

Check out the NUnit quick start:
Let’s start with a simple example.
Suppose we are writing a bank
application and we have a basic domain
class – Account. Account supports
operations to deposit, withdraw, and
transfer funds.

I recommend you to have an own project for your tests (like Project.Tests).
Place the following basic files somewhere in folder of your project structure (e.g. lib\nunit\nunit):
nunit.core.dll
nunit.core.interfaces.dll
nunit.framework.dll
nunit.util.dll
nunit-console.exe
nunit-console.exe.config
nunit-console-runner.dll
nunit-console-x86.exe
nunit-console-x86.exe.config
Then you need to reference the NUnit.Framework assembly in your Project.Tests project.
For example, a simple test would look like this:
using NUnit.Framework;
namespace Project.Tests
{
[TestFixture]
public class MyTestClass
{
[Test]
public void MyTestMethod()
{
var a = "a";
var b = "a";
Assert.AreEqual(a, b);
}
}
}
You can run this test then for example with the NUnit-console or directly in VisualStudio (e.g. with the help of ReSharper) or through a MSBuild task with the help of MSBuild Community Tasks.

If you don't use resharper I recommend you to use this plugin - http://www.testdriven.net/ .