Is it at all possible to execute tests marked with the NUnit [Explicit] attribute using dotnet test?
I have some tests which I would like to be ran as a separate build on Bamboo. They take a while to run so didn't want them to be included in the standard test run in VS so marked them with the [Explicit] attribute. I also marked them with [Category("Nightly")] so on Bamboo I hoped to have a step of:
dotnet test --filter "TestCategory=Nightly"
but as expected they still get ignored because of the Explicit property.
I tried explicitly naming, like this:
dotnet test --filter "ClassName=MyTests.Tests.TestClassName"
but still no luck, I've tried all of the possible arguments to dotnet test https://learn.microsoft.com/en-us/dotnet/core/testing/selective-unit-tests?pivots=mstest to refer to the tests specifically but they still get ignored.
Has anyone managed this?
Rich
As per this Github issue this is actually a known bug with NUnit3TestAdapter where tests marked as [Explicit] would not execute regardless of how they were referenced in the arguments to dotnet test. This has been addressed in the latest pre-release version of the nuget package.
I'm writing some unit tests with MSTest, using C#, MSVS 2015 and .Net 4.6.1.
This line:
Assert.ThrowsException<ArgumentOutOfRangeException>( () =>
select.AllSelectedOptions[0]
);
Fails with this compile error:
CS0117 'Assert' does not contain a definition for 'ThrowsException'
My namespace is Microsoft.VisualStudio.TestTools.UnitTesting (the default when you create a unit test project in MSVS).
According to the documentation, Assert.ThrowsException(Action) should exist. But I don't see it in Intellisense ... and I'm getting the compile error.
I've tried a couple of different versions of MSVS (MSVS 2015 and MSVS 2019) and a couple of different versions of MSTest.
Q: Any ideas what might be wrong?
As Clint said below, I need to install MSTest v2 from NuGet in order to use Assert.ThrowsException<T>() in MSVS 2015.
But after doing this, MSVS isn't finding any of my tests anymore:
MSVS > Test > Run All (or "Test > Debug > All Tests"):
------ Discover test started ------
========== Discover test finished: 0 found (0:00:01.127375) ==========
Any suggestions?
You need to use [MSTest V2] to be able to Assert.ThrowsException
Starting with VS2017, the in-box Unit Test Project templates use only MSTest V2.
Now that you're on VS2015 you can install this package MSTest.Test from Nuget but make sure to remove the old test references like Microsoft.VisualStudio.QualityTools.UnitTestFramework before upgrading to this package
Add > New Test project > Select MSTest project type
After this you should be able to use Assert.ThrowsException<ArgumentOutOfRangeException>(
To discover and execute tests also ensure to install MSTest.TestAdapter.
Further Reading
I am using the latest VS2017 version 15.6.4, NUnit3TestAdapter 3.10.0 and Nunit version 3.9.0.0.
When I try to run a unit test in Test Explorer the test are grayed out, when I right click and and run selected tests I see the following error: No test is available
Here is how my test class looks
[TestFixture]
public partial class ListViewBOTest
{
[Test]
public void TestSearch_DateTime()
{
Assert.AreEqual(1,0);
}
}
Text from output:
[3/26/2018 10:53:55 AM Informational] ------ Run test started ------
[3/26/2018 10:53:55 AM Informational] NUnit Adapter 3.10.0.21: Test execution started
[3/26/2018 10:53:55 AM Informational] Running all tests in C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll
[3/26/2018 10:53:56 AM Informational] NUnit failed to load C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll
[3/26/2018 10:53:56 AM Informational] NUnit Adapter 3.10.0.21: Test execution complete
[3/26/2018 10:53:56 AM Warning] No test is available in C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[3/26/2018 10:53:56 AM Informational] ========== Run test finished: 0 run (0:00:01.0435303) ==========
Make sure you have installed the NUnit3 Test Adapter from here
https://marketplace.visualstudio.com/items?itemName=NUnitDevelopers.NUnit3TestAdapter
Can you check the following steps and see if it works?
In the Visual Studio menu, go to Test > Test Settings > Default Processor Architecture and make a note if X86 is selected or X64
Now go to the Build section in the Properties window of the project where the tests are written. Make sure the Platform target drop-down is selected to either Any CPU or at least it matches the architecture from the above step 1.
Now if you build the solution and try running those tests, you should see that they are now running.
I had the same problem as Amete Blessed and commenting out other Test methods made Test Explorer work and run my test
I found that my Build Events were wrong. Invalid copy command blew all my tests and half a day:
Copy C:\repo\Architecture\*.json $(ProjectPath)/Y
Copy C:\repo\Architecture\*.json $(TargetPath) /Y
instead of
Copy C:\repo\Architecture\*.json $(ProjectDir)/Y
Copy C:\repo\Architecture\*.json $(TargetDir) /Y
The messages in the Test Output window ("NUnit failed to load [assembly]", "No test is available...", etc.) can hide the underlying issue that's causing the runner to not load the test assembly. This includes hiding failures to load dependencies of the test assembly or the item under test.
If there's a test assembly that's showing up in the Test Explorer window, but the tests refuse to run, it's worth temporarily enabling fusion logging to see if any assembly binding errors occur when trying to run the tests.
After installing NUnit through nuget tests have appeared into Test Explorer
but when I ran them I got "No test is available".
Installing NUnit Test Adapter fixed the issue.
I had similar problem when using Xamarin.Forms.
The solution was to install NUnit.XForms from NuGet and add
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0" />
to csproj file of the project where the tests are.
What worked for me was to delete the ComponentModelCache folder located at %localappdata%\Microsoft\VisualStudio\<version>
You may have been a casualty of this problem that was (theoretically) resolved in 15.6.3. According to this answer, try deleting your %temp%\VisualStudioTestExplorerExtensions folder. That has resolved the issue for some other users.
Rather than using the Test Explorer, can you right click on the solution and Run Unit Tests from there?
Had the same problem. In my case I found that the NUnit test adapter will not be used by Test Explorer if your test project contains a reference to MSTest. Typically the Microsoft.VisualStudio.TestPlatform.TestFramework.dll but also check your .csproj file for "MSTest" and your packages.config file.
I discovered this by first enabling diagnostic logging for Visual Studio tests. This is found under "Tools" -> "Options" -> "Tests" -> "Logging Level".
In my log I found this entry:
[22/11/2018 10:36:42 Diagnostic] Project C:\Git\myProject\src\myProject.Tests\myProject.Tests.csproj references test adapter: MSTest.TestAdapter, version 1.1.18
This is caused due to memory problem.
Clean object after test execution
Sample Code
private TestController testController;
[OneTimeSetUp]
public void TestSetup()
{
testController= new TestController();
}
[OneTimeTearDown]
public void TestCleanup()
{
testController= null;
}
Change test execution to 64 bits in settings
Test -> Test Settings -> Default Processors Architecture -> x64.
I have also encountered the same issue.
Steps to resolve this issue -
1. Install/Reinstall NUnit3TestAdapter package
2. Delete Debug folder from the Bin
3. Clean --> Build the project
Note - Also ensure all the packages are installed properly.
Now try to execute the tests.
I had the same issue, but it was fixed after installing "NUnit 3 TestAdapter version 3.16.1" via NuGet to my project.
So I have this really simple test:
[TestMethod]
public void CheckAllParametersExist()
{
try
{
new ParametersService().Get();
Assert.IsTrue(true);
}
catch(Exception ex)
{
Assert.Fail(ex.Message);
}
}
ParametersService().Get() runs through a parameters class finding all the properties and tries to populate them with values from a database. Occasionally when we release the website we might forget to publish some values, so I wanted a unit test in Bamboo to identify any parameters we may have missed.
To the crux of my problem: In Visual Studio 2017 the unit test passes fine. With MSTest is fails with:
Assert.Fail failed. The type initializer for 'System.Data.Entity.Internal.AppConfig' threw an exception.
Looking this particular error up it looks like I've got the wrong or mixed versions of EntityFramework. Having tried to fix this myself I have removed Entity framework from the Tests project nuget and app.config, then under the solution I consolidated the EntityFramework nuget and included Tests project again, still the same error. App.config shows version 6.0.0.0, nuget has installed 6.2.0
I am stuck, if anyone can suggest any solutions or identify any reasons why I might be seeing this problem I would be greatful.
Fyi: I am running MSTest with /testcontainer:tests.dll in the Tests project bin output debug folder.
So it turns out the problem was with the App.Config. The problem is that we have some sections of the configuration loaded from external files. In this case connection strings. In VS it loads the config file for connections strings just fine. In MSTest it copies the DLLs to another folder but doesn't include the folders/config files.
Also MSTest has now been retired, as of VS2013 we should be using VsTest.console, however Bamboo hasn't caught up with that yet.
I downloaded a c# .net project from git onto ubuntu and installed .net libraries from http://www.mono-project.com/download/#download-lin and also donet core from https://www.microsoft.com/net/core#linuxubuntu.
when I run donet restore, I am getting the following error
/home/ubuntu/DotNetWallet/src/DotNetWallet/DotNetWallet.xproj(8,3): error MSB4019: The imported project "/usr/share/dotnet/sdk/1.0.3/Microsoft/VisualStudio/v14.0/DotNet/Microsoft.DotNet.Props" was not found. Confirm that the path in the declaration is correct, and that the file exists on disk.
I guess the problem is with the DotNetWallet.xproj which generally is compatible with visual studio but how can I run it on ubuntu?
So, I
used 'dotnet migrate' ,and then used 'donet restore', it now works fine, but when I run 'donet build', I am getting the following error
FakeData.cs(1,20): error CS0234: The type or namespace name 'KeyManagement' does not exist in the namespace 'DotNetWallet' (are you missing an assembly reference?) [/home/ubuntu/DotNetWallet/src/DotNetWallet/DotNetWallet.csproj]
And, here is the git repo link https://github.com/nopara73/DotNetWallet
Many step are required in order to run this project.
First delete global.json file and then run dotnet migrate and remove the backup directory.
(Optional) In some case you will need to clean the solution. If DotNetWallet.xproj and project.json are still there, remove it and run the twxo following command.
dotnet sln remove src/DotNetWallet/DotNetWallet.xsproj
dotnet sln add src/DotNetWallet/DotNetWallet.csproj
Then, remove the first line (using DotNetWallet.KeyManagement;) from FakeData.cs file. This is an unused using statment. there are no impact to remove it.
Now, you can run dotnet restore, dotnet build then dotnet test.