TFS 2012 Build Server NUnit Exception - c#

I have a NUnit test project that I'm trying to run on my TFS 2012 Build server.
I'm getting the following error:
Exception System.InvalidOperationException, Exception thrown
executing tests
No test is available in C:\Builds\1\Projects\NUNIT
Test\bin\NUnitTestProject.dll C:\Builds\1\Projects\NUNIT
Test\bin\NUnitTestProject.dll. Make sure that installed test
discoverers & executors, platform & framework version settings are
appropriate and try again.
I Have the NUnit.VisualStudio.TestAdapter 1.1.0.0 installed from NuGet and referenced in the project.
I also have set the Version control path to custom assemblies set on the controller, which contains (they basically are a copy of the NuGet package):
nunit.core.dll
nunit.core.interfaces.dll
nunit.util.dll
NUnit.VisualStudio.TestAdapter.dll
For test purposes, I'm only building the NUnit test project.
My test class is very simple and is the following:
using NUnit.Framework;
namespace NUnitTestProject
{
[TestFixture]
public class MySuperTests
{
[Test]
public void MyFirstUnitTestMethod_ReturnsTrue()
{
Assert.IsTrue(true);
}
}
}
Is there something I haven't done correctly?
Edit 1:
Using TFS 2012 Update 4 on my build server
Edit 2:
The problem I believe lies in the 1.1.0.X version.
Updating to 1.1.0.7 I did not have the exception anymore, however, I did have the tests not found problem.
There are some issues going on about that on github https://github.com/nunit/nunit-vs-adapter/
Edit 3:
The problem has been fixed in 1.1.0.8 of the NUnit test adapter!

Do you have your test runner set correctly in the build defiintion?
Image shows TFS2013 but 2012 has a different option for 2010 compatability which may be selected. The Visual Studio Test Runner should support multiple test frameworks, whereas the 2010 compatability one only supports MSTest

Related

MSTest: CS0117 'Assert' does not contain a definition for 'ThrowsException'

I'm writing some unit tests with MSTest, using C#, MSVS 2015 and .Net 4.6.1.
This line:
Assert.ThrowsException<ArgumentOutOfRangeException>( () =>
select.AllSelectedOptions[0]
);
Fails with this compile error:
CS0117 'Assert' does not contain a definition for 'ThrowsException'
My namespace is Microsoft.VisualStudio.TestTools.UnitTesting (the default when you create a unit test project in MSVS).
According to the documentation, Assert.ThrowsException(Action) should exist. But I don't see it in Intellisense ... and I'm getting the compile error.
I've tried a couple of different versions of MSVS (MSVS 2015 and MSVS 2019) and a couple of different versions of MSTest.
Q: Any ideas what might be wrong?
As Clint said below, I need to install MSTest v2 from NuGet in order to use Assert.ThrowsException<T>() in MSVS 2015.
But after doing this, MSVS isn't finding any of my tests anymore:
MSVS > Test > Run All (or "Test > Debug > All Tests"):
------ Discover test started ------
========== Discover test finished: 0 found (0:00:01.127375) ==========
Any suggestions?
You need to use [MSTest V2] to be able to Assert.ThrowsException
Starting with VS2017, the in-box Unit Test Project templates use only MSTest V2.
Now that you're on VS2015 you can install this package MSTest.Test from Nuget but make sure to remove the old test references like Microsoft.VisualStudio.QualityTools.UnitTestFramework before upgrading to this package
Add > New Test project > Select MSTest project type
After this you should be able to use Assert.ThrowsException<ArgumentOutOfRangeException>(
To discover and execute tests also ensure to install MSTest.TestAdapter.
Further Reading

Visual Studio Unit Test runs fine, MSTEST does not

So I have this really simple test:
[TestMethod]
public void CheckAllParametersExist()
{
try
{
new ParametersService().Get();
Assert.IsTrue(true);
}
catch(Exception ex)
{
Assert.Fail(ex.Message);
}
}
ParametersService().Get() runs through a parameters class finding all the properties and tries to populate them with values from a database. Occasionally when we release the website we might forget to publish some values, so I wanted a unit test in Bamboo to identify any parameters we may have missed.
To the crux of my problem: In Visual Studio 2017 the unit test passes fine. With MSTest is fails with:
Assert.Fail failed. The type initializer for 'System.Data.Entity.Internal.AppConfig' threw an exception.
Looking this particular error up it looks like I've got the wrong or mixed versions of EntityFramework. Having tried to fix this myself I have removed Entity framework from the Tests project nuget and app.config, then under the solution I consolidated the EntityFramework nuget and included Tests project again, still the same error. App.config shows version 6.0.0.0, nuget has installed 6.2.0
I am stuck, if anyone can suggest any solutions or identify any reasons why I might be seeing this problem I would be greatful.
Fyi: I am running MSTest with /testcontainer:tests.dll in the Tests project bin output debug folder.
So it turns out the problem was with the App.Config. The problem is that we have some sections of the configuration loaded from external files. In this case connection strings. In VS it loads the config file for connections strings just fine. In MSTest it copies the DLLs to another folder but doesn't include the folders/config files.
Also MSTest has now been retired, as of VS2013 we should be using VsTest.console, however Bamboo hasn't caught up with that yet.

How to make a .dll and run it with NUnit?

I'am totally new to Windows Store App programming,
so i'am also new to Visual Studio Express.
My goal is to test a simpple class method.
As mentioned here the Express version do not have any built-in Unit testing.
In this thread, Rafal provides a solution.
I exactly did it like described, so my external tools look like this:
When i execute it (Tools --> "Execute NUnit"), Nunit starts and the Gui of NUnit appears. But suddenly this exception occurs:
And in Exception Details:
System.IO.FileNotFoundException...
at NUnit.Util.ProjectService.WrapAssembly(String assemblyPath)
at NUnit.Util.ProjectService.ConvertFrom(String path)
at NUnit.Util.ProjectService.LoadProject(String path)
at NUnit.Util.TestLoader.LoadProject(String filePath, String configName)
My project folder has this structure:
The test classes are in "WebTest.Shared".
I think i need a .dll to run in NUnit as mentioned by Jon here.
So, how can I make a dll out of my project to run it with NUnit?
Can anyone guide me through this problem? (Please step by step)
EDIT:
After i worked in ChrisM idea, the exception stll arises without "${BinDir}${TargetName}.dll/run" block (the exception details are the same as before):
EDIT No. 2:
I have set those values:
Title: Execute NUnit
Command: D:\Path\To\NUnit\bin\nunit.exe
Arguments: $(BinDir)$(TargetDir)$(TargetExt)/run
Initial directory: $(BinDir)
EDIT No. 3:
After closing and reopening VS Express
i got this new Exception:
And in NUnit Exception Details:
System.ApplicationException: Unable to find test in assembly
System.ApplicationException...
EDIT No. 4
Here is my test class (StringUtilitiesTest.cs):
using System;
using System.Collections.Generic;
using System.Text;
using WebappTest.Shared.Utilities;
using NUnit.Framework;
namespace WebappTest.UnitTest
{
[TestFixture]
public class StringUtilitiesTest
{
[Test]
public void TransferFunds()
{
Assert.AreEqual("Hello", StringUtilites.getString("Hello"));
}
}
}
In external Tools:
Have you tried replacing the curly braces {} in the argument box with normal ones ()?
Visual Studio 2017 Express (the final express version) includes the test explorer. Add the NUnit3TestAdapter NuGet to your project, and the test explorer should discover your tests.

UnitTestIsolationException: Throws Exception while running Shims test in Release/Debug mode

I am trying to run this unit test using Microsoft Shims, but it throws me exception in Shims.Context.Create(); method.
Environment: VS 2012, Win2K8 R2
namespace MyShimsUnitTest
{
[TestClass]
public class MyUnitTest
{
[TestMethod]
public void GetCurrentYear()
{
using (Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create())
{
// Some Logic...
}
}
}
}
Detailed Exception:
Result Message:
Test method MyShimsUnitTest.MyUnitTest.GetCurrentYear threw exception:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: UnitTestIsolation instrumentation failed to initialize. Please restart Visual Studio and rerun this test
Result StackTrace:
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext()
at Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create()
at MyShimsUnitTest.MyUnitTest.GetCurrentYear()
Shims require runtime instrumentation performed by the IntelliTrace profiler. The test runner is responsible for setting up the environment variables required for CLR to load the profiler as well as providing the list of types the profiler must instrument for Shims. The UnitTestIsolationException is thrown when the ShimRuntime is unable to locate and attach to the IntelliTrace profiler, which it expects to be already loaded by the CLR.
As Jin-Wook mentioned earlier, this problem occurs when the test is executed by a runner that does not perform the required profiler initialization. Test Explorer and the vstest.console.exe are two such runners that ship with Visual Studio 2012. At this time, the Visual Studio test runners do not perform the required profiler instrumentation when running tests in "legacy" mode, which happens when you have a .TESTSETTINGS file selected for your run or a .RUNSETTINGS file that forces legacy mode.
You may be able to use third-party test runners that support profiler instrumentation required by Shims.
I had the same issue. The solution to my problem was to uncheck the selected .testsettings file from the menu: TEST/Test Settings and here the item(s) above the Select Test Settings File.
It could be caused by not using the test explorer of vs 2012. To use the shim, you should run tests only using the test explorer.
You can use other test framework such as Nunit or Xunit with the shim if installing appropriate test runner for vs 2012. It can be downloaded from the vs extension manager.
I ran into this issue too. Thankfully the other answers here helped me fix my issue:
I'm using Resharper and when using the context menu I noticed that the runner is using MSTest. Even when finding the test in test explorer and selecting debug I received the same exception.
I then went into Resharpers's options and under Tools -> Unit Testing -> MsTest I unchecked "Enable MSTest support". This unfortunately disables the option to right click on your test and hit run/debug, but it did allow ShimsContext.Create() to behave correctly when selecting debug from the Test Explorer view!
Go to your TestProject Properties -> Under Debug section Check the "ENABLE NATIVE CODE DEBUGGING" checkbox.
This is should do.
We saw this error reported by Bamboo, our build server. It was invoking an MSbuild 4.0 task. The unit test work fine on the dev's local PCs. I deleted this bamboo task and created a new task that invokes Visual Studio 2012's vstest.console. The tests now pass but Bamboo is not able to count the number of tests. This is a Bamboo problem not mine.

Unit Testing - Extending the Visual Studio Unit Test Type - Not working

We're asked to move from NUnit to MSTest and now have to convert all the existing tests to the new platform. Most of it converted fine but we have an issue with parameterised tests. We found the following web site showing how to write an extension and downloaded the code for it.
http://blogs.msdn.com/b/vstsqualitytools/archive/2009/09/04/extending-the-visual-studio-unit-test-type-part-2.aspx
However, although it compiles fine, the RowTestClass test all fail to run with the following error message:
"Failed to initialize the unit test extension 'urn:RunAsTestClassAttribute': A unit test extension is not registered for the following attribute: Microsoft.VisualStudio.Test.Sample.UnitTestTypeExtension.RunAsSample.RunAsTestClassAttribute."
We're using VS2010 (10.0.40219.1 SP1Rel) (Microsoft .NET Framework Version 4.0.30319 SP1Rel)
I wonder if it has something to do with the following libraries:
Microsoft.VisualStudio.QualityTools.Common.dll
Runtime Version: v2.0.50727
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\ReferenceAssemblies\v2.0\Microsoft.VisualStudio.QualityTools.Common.dll
Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
Runtime Version: v4.0.30319
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PublicAssemblies\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
Microsoft.VisualStudio.QualityTools.Vsip.dll
v4.0.30319
C:\Windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.VisualStudio.QualityTools.Vsip\v4.0_10.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.QualityTools.Vsip.dll
Any help to get this working greatly appreciated.
Have you registered the new types (in the Windows Registry)? There's a section on how to do it in the tutorial - have a look at Row Test Test Type Extension: Registering your test type extension.
If you're having trouble, try downloading the complete code + registry keys from the MSDN code gallery.
You may want to look into using SpecFlow to make your tests more parameterized. It is traditionally a BDD tool, but it will allow you to make re-usable steps that take parameters.
For example you can create a step saying
"Given the user 'admin' exists."
and on the c# side you get a method like this:
[Given(#"the user '(.*)' exists"]
public void GivenTheUserExists(string name)
{
//create the user
}
Going forward you can reuse the step in any "feature file" without having to rewrite the implementation of that step.
Just refer to the blog post Extending Unit Tests in Visual Studio. And see what you might have missed. I tried the same, and it works well for me.

Categories