We Keep Coding

VSTest-testAssemblies agent is not running my Category search filter for NUnit framework in Azure Pipelines

I am using in the Test Filter criteria TestCategory=CategoryA
In my Tests I have the below saved and think it should be the only test run but I get success and no tests ran. I have tried with dotnet test and used the same in the arguments and gain it fails with other issues. I can get dot net test to run without adding any arguments for VS Test agent it seems to not want to run even when I do not add the test filter criteria.
[Test]
[Category("CategoryA")]
public async Task ActivateDevice()
2022-02-18T20:14:54.5643568Z ##[section]Starting: VsTest - testAssemblies
2022-02-18T20:14:54.5771277Z ==============================================================================
2022-02-18T20:14:54.5771649Z Task : Visual Studio Test
2022-02-18T20:14:54.5772497Z Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2022-02-18T20:14:54.5773342Z Version : 2.198.0
2022-02-18T20:14:54.5773587Z Author : Microsoft Corporation
2022-02-18T20:14:54.5773921Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/test/vstest
2022-02-18T20:14:54.5774333Z ==============================================================================
2022-02-18T20:14:55.5713320Z SystemVssConnection exists true
2022-02-18T20:14:55.5732257Z SystemVssConnection exists true
2022-02-18T20:14:55.5991193Z Running tests using vstest.console.exe runner.
2022-02-18T20:14:55.5991725Z ======================================================
2022-02-18T20:14:55.5997078Z Test selector : Test assemblies
2022-02-18T20:14:55.5998382Z Test filter criteria : null
2022-02-18T20:14:55.5999337Z Search folder : D:\a\r1\a
2022-02-18T20:14:55.6102675Z Action when minimum tests threshold not met : donothing
2022-02-18T20:14:55.6103140Z Minimum tests expected to be run: 0
2022-02-18T20:14:55.6104476Z VisualStudio version selected for test execution : latest
2022-02-18T20:14:55.6107744Z Attempting to find vstest.console from a visual studio installation with version [17.0,18.0).
2022-02-18T20:14:56.0357719Z Attempting to find vstest.console from a visual studio build tools installation with version [17.0,18.0).
2022-02-18T20:14:56.0642271Z Attempting to find vstest.console from a visual studio installation with version [16.0,17.0).
2022-02-18T20:14:56.0884398Z Attempting to find vstest.console from a visual studio build tools installation with version [16.0,17.0).
2022-02-18T20:14:56.1142098Z Attempting to find vstest.console from a visual studio installation with version [15.0,16.0).
2022-02-18T20:14:56.3498931Z Run in parallel : false
2022-02-18T20:14:56.3500994Z Run in isolation : false
2022-02-18T20:14:56.3502074Z Path to custom adapters : null
2022-02-18T20:14:56.3504871Z Other console options : null
2022-02-18T20:14:56.3505911Z Code coverage enabled : false
2022-02-18T20:14:56.3510216Z Diagnostics enabled : false
2022-02-18T20:14:56.3519771Z SystemVssConnection exists true
2022-02-18T20:14:56.3697972Z Run the tests locally using vstest.console.exe
2022-02-18T20:14:56.3698460Z ========================================================
2022-02-18T20:14:56.5771570Z Source filter: D:\a\r1\a/_Test.AppiumTest/AppiumTesting/AppiumTest
2022-02-18T20:14:56.5973322Z ##[warning]No test sources found matching the given filter

1. Check whether the following latest packages are installed or not:
Microsoft.NET.Test.Sdk
MSTest.TestAdapter
MSTest.TestFramework
NUnit3TestAdapter
2. As answered by Marina Liu , you can configure MSTest or the "Visual Studio Test" task in VSO to fail if there are warnings:
a. Add a PowerShell task after Visual Studio Test task.
b. Then get Visual Studio Test task build information by Timeline:
c. Search for Visual Studio Test task information by task name or task display name.
d. Fail the build result if VS test has the warning you specified.
References: How to Fix this C# issue No test matches the given testcase filter `FullyQualifiedName = , How can I configure MSTest to fail the unit test run on any warnings? and Run NUnit tests in Azure DevOps pipeline

MSTest: CS0117 'Assert' does not contain a definition for 'ThrowsException'

I'm writing some unit tests with MSTest, using C#, MSVS 2015 and .Net 4.6.1.
This line:
Assert.ThrowsException<ArgumentOutOfRangeException>( () =>
select.AllSelectedOptions[0]
);
Fails with this compile error:
CS0117 'Assert' does not contain a definition for 'ThrowsException'
My namespace is Microsoft.VisualStudio.TestTools.UnitTesting (the default when you create a unit test project in MSVS).
According to the documentation, Assert.ThrowsException(Action) should exist. But I don't see it in Intellisense ... and I'm getting the compile error.
I've tried a couple of different versions of MSVS (MSVS 2015 and MSVS 2019) and a couple of different versions of MSTest.
Q: Any ideas what might be wrong?
As Clint said below, I need to install MSTest v2 from NuGet in order to use Assert.ThrowsException<T>() in MSVS 2015.
But after doing this, MSVS isn't finding any of my tests anymore:
MSVS > Test > Run All (or "Test > Debug > All Tests"):
------ Discover test started ------
========== Discover test finished: 0 found (0:00:01.127375) ==========
Any suggestions?

You need to use [MSTest V2] to be able to Assert.ThrowsException
Starting with VS2017, the in-box Unit Test Project templates use only MSTest V2.
Now that you're on VS2015 you can install this package MSTest.Test from Nuget but make sure to remove the old test references like Microsoft.VisualStudio.QualityTools.UnitTestFramework before upgrading to this package
Add > New Test project > Select MSTest project type
After this you should be able to use Assert.ThrowsException<ArgumentOutOfRangeException>(
To discover and execute tests also ensure to install MSTest.TestAdapter.
Further Reading

VS2017 and NUnit 3.9 No test is available

I am using the latest VS2017 version 15.6.4, NUnit3TestAdapter 3.10.0 and Nunit version 3.9.0.0.
When I try to run a unit test in Test Explorer the test are grayed out, when I right click and and run selected tests I see the following error: No test is available
Here is how my test class looks
[TestFixture]
public partial class ListViewBOTest
{
[Test]
public void TestSearch_DateTime()
{
Assert.AreEqual(1,0);
}
}
Text from output:
[3/26/2018 10:53:55 AM Informational] ------ Run test started ------
[3/26/2018 10:53:55 AM Informational] NUnit Adapter 3.10.0.21: Test execution started
[3/26/2018 10:53:55 AM Informational] Running all tests in C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll
[3/26/2018 10:53:56 AM Informational] NUnit failed to load C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll
[3/26/2018 10:53:56 AM Informational] NUnit Adapter 3.10.0.21: Test execution complete
[3/26/2018 10:53:56 AM Warning] No test is available in C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[3/26/2018 10:53:56 AM Informational] ========== Run test finished: 0 run (0:00:01.0435303) ==========

Make sure you have installed the NUnit3 Test Adapter from here
https://marketplace.visualstudio.com/items?itemName=NUnitDevelopers.NUnit3TestAdapter

Can you check the following steps and see if it works?
In the Visual Studio menu, go to Test > Test Settings > Default Processor Architecture and make a note if X86 is selected or X64
Now go to the Build section in the Properties window of the project where the tests are written. Make sure the Platform target drop-down is selected to either Any CPU or at least it matches the architecture from the above step 1.
Now if you build the solution and try running those tests, you should see that they are now running.

I had the same problem as Amete Blessed and commenting out other Test methods made Test Explorer work and run my test

I found that my Build Events were wrong. Invalid copy command blew all my tests and half a day:
Copy C:\repo\Architecture\*.json $(ProjectPath)/Y
Copy C:\repo\Architecture\*.json $(TargetPath) /Y
instead of
Copy C:\repo\Architecture\*.json $(ProjectDir)/Y
Copy C:\repo\Architecture\*.json $(TargetDir) /Y

The messages in the Test Output window ("NUnit failed to load [assembly]", "No test is available...", etc.) can hide the underlying issue that's causing the runner to not load the test assembly. This includes hiding failures to load dependencies of the test assembly or the item under test.
If there's a test assembly that's showing up in the Test Explorer window, but the tests refuse to run, it's worth temporarily enabling fusion logging to see if any assembly binding errors occur when trying to run the tests.

After installing NUnit through nuget tests have appeared into Test Explorer
but when I ran them I got "No test is available".
Installing NUnit Test Adapter fixed the issue.

I had similar problem when using Xamarin.Forms.
The solution was to install NUnit.XForms from NuGet and add
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0" />
to csproj file of the project where the tests are.

What worked for me was to delete the ComponentModelCache folder located at %localappdata%\Microsoft\VisualStudio\<version>

You may have been a casualty of this problem that was (theoretically) resolved in 15.6.3. According to this answer, try deleting your %temp%\VisualStudioTestExplorerExtensions folder. That has resolved the issue for some other users.

Rather than using the Test Explorer, can you right click on the solution and Run Unit Tests from there?

Had the same problem. In my case I found that the NUnit test adapter will not be used by Test Explorer if your test project contains a reference to MSTest. Typically the Microsoft.VisualStudio.TestPlatform.TestFramework.dll but also check your .csproj file for "MSTest" and your packages.config file.
I discovered this by first enabling diagnostic logging for Visual Studio tests. This is found under "Tools" -> "Options" -> "Tests" -> "Logging Level".
In my log I found this entry:
[22/11/2018 10:36:42 Diagnostic] Project C:\Git\myProject\src\myProject.Tests\myProject.Tests.csproj references test adapter: MSTest.TestAdapter, version 1.1.18

This is caused due to memory problem.
Clean object after test execution
Sample Code
private TestController testController;
[OneTimeSetUp]
public void TestSetup()
{
testController= new TestController();
}
[OneTimeTearDown]
public void TestCleanup()
{
testController= null;
}
Change test execution to 64 bits in settings
Test -> Test Settings -> Default Processors Architecture -> x64.

I have also encountered the same issue.
Steps to resolve this issue -
1. Install/Reinstall NUnit3TestAdapter package
2. Delete Debug folder from the Bin
3. Clean --> Build the project
Note - Also ensure all the packages are installed properly.
Now try to execute the tests.

I had the same issue, but it was fixed after installing "NUnit 3 TestAdapter version 3.16.1" via NuGet to my project.

dotnet test - exit code 0 when test project doesn't compile

I'm using TeamCity to build my .NET Core project and dotnet test with xunit to run tests.
My build is configured to fail if any tests fail, and this works fine if a test fails, but when the test project doesn't compile the exit code ends up being zero.
I believe this is because I run dotnet test in the folder that contains the test-directories like this: for /f %%%a in ('dir /b /s project.json') do dotnet test %%%a.
Looking at the log I can see the individual jobs that do not compile return with exit code 1, but the build step itself returns with exit code 0.
How can I make the exit code from the failed compilation attempts propagate down to the build step?

You may create a separate step in TC to just compile target test project, and then run tests without compilation (using no-build option) in next step (if all previous steps were successful).
--no-build
Does not build the test project prior to running it.

Using VSTest to run unit test cases instead of MSTest

I have an x64 platform C# solution(VS2012) on a TFS2010 server. I have attached a unit test project (also x64) to this solution and created a build definition. When I queue the build, it succeeds but the unit test cases will not be executed. This is because MSTest is a 32 bit application. So, I decided to customize the default build process template (DefaultTemplate.xaml) to invoke VSTest(VSTest.console.exe) instead of MSTest. This is quite complex and I am unable to add a build activity to the toolbox for VSTest.
Has anyone done this kind of customization? I have also considered other approaches like configuring .runsettings file. Do we have a VSTest adapter interface that can be added in the .runsettings file ?

Executing unit tests through VSTest and publishing the test results through MSTest gave me a successful outcome. Given below is the Powershell script:
# Get the UnitTest Binaries
$files = Get-ChildItem $TestAssembliesDir\*est*.dll
# VSTest.console.exe path
$VSTestPath = 'C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe'
# MSTest path
$MSTestpath = "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\mstest.exe"
# Loop through the test assemblies and add them to the VSTestFiles
$VSTestFiles = ''
foreach($file in $files)
{
$VSTestFiles += "`"$file`""
$VSTestFiles += " "
}
# Run the UnitTests using VSTest
&$VSTestPath $vstestplatform "/Framework:Framework45" "/InIsolation" $VSTestFiles "/logger:trx"
# Get TRX files
$TrxFilesList = Get-ChildItem $TestResDir\*.trx
$TrxFiles = ''
$loop = 1
foreach($file in $TrxFilesList)
{
$TrxFiles = "$file"
# copy the trx file into a space-less named trx
$newTrxFileName = "$BuildUri" + "_" + "$loop" + ".trx"
copy-item $TrxFiles -destination $TestResDir\$newTrxFileName
$loop = $loop + 1
$newTrxFile += "$TestResDir\$newTrxFileName"
$newTrxFile += " "
}
# specify MSTest arguments
$mspubl = "/publish:"+$TeamProjColUri
$msteampr = "/teamproject:" + $TeamProj
$mspublbuild = "/publishbuild:" +$BuildUri
$mspubresfile = "/publishresultsfile:" +"`"$newTrxFile`""
#Publish test results through MSTest
&$MSTestpath $mstestplatform $flavor $mspubl $msteampr $mspublbuild $mspubresfile

I too have the exact same need for using VSTest.Console.exe instead of MSTest.exe for a TFS2010 build process that compiles a VS2012/.NET 4.5 x64 application, while waiting for the upgrade to TFS2012 to commence.
The approach I have taken was to edit the build script XAML, deleted the existing workflow for unit tests and replaced it with a customised workflow that builds up the VSTest.Console.exe parameters and then executes VSTest.Console.exe via InvokeProcess. I then ensured that in the Finally block that regardless of test result that we publish the test results and code coverage to TFS using MSTest.exe from a VS2012 installation on the build server.
Unfortunately I cannot post the XAML in the answer as it exceeds character length, but I do have a text file consisting of the snippet to be replaced in DefaultTemplate.xaml and what to replace it with. The file can be found here. Please note that although this approach works it is a hack.
Another alternative would be to use NUnit instead of MSTest or VSTest.Console as this support 64-bit binaries. This article explains how to integrate NUnit in a TFS2010 build script, and has links to tools and resources required to make this happen. The only issues with NUnit are code coverage (need yet another tool plus work out how to publish these results to TFS) and MSTest-style integration tests using attributes such as DeploymentItem and properties such as TestContext, which is why where I work we opted with the VSTest.Console.exe approach.
And from what I have read TFS2012 offers easy integration to VSTest.Console.exe from build scripts, so if you do ever upgrade to TFS2012 the VSTest.Console.exe hack that I have documented may not be required.

This does not directly answer you question, but it might help. I did a similar thing for TeamCity. I used command-line to call vstest.console.exe and created a .runsettings file.
I used this Microsoft template for the runsettings file. Note however that on my machine, the path mentioned in the comment in Line 5 is relative to the .runsettings location, not the .sln.
If you use /logger:trx option of vstest.console.exe, it will generate output in the same format as MSTest (good for result visualization).