I am using in the Test Filter criteria TestCategory=CategoryA
In my Tests I have the below saved and think it should be the only test run but I get success and no tests ran. I have tried with dotnet test and used the same in the arguments and gain it fails with other issues. I can get dot net test to run without adding any arguments for VS Test agent it seems to not want to run even when I do not add the test filter criteria.
[Test]
[Category("CategoryA")]
public async Task ActivateDevice()
2022-02-18T20:14:54.5643568Z ##[section]Starting: VsTest - testAssemblies
2022-02-18T20:14:54.5771277Z ==============================================================================
2022-02-18T20:14:54.5771649Z Task : Visual Studio Test
2022-02-18T20:14:54.5772497Z Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2022-02-18T20:14:54.5773342Z Version : 2.198.0
2022-02-18T20:14:54.5773587Z Author : Microsoft Corporation
2022-02-18T20:14:54.5773921Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/test/vstest
2022-02-18T20:14:54.5774333Z ==============================================================================
2022-02-18T20:14:55.5713320Z SystemVssConnection exists true
2022-02-18T20:14:55.5732257Z SystemVssConnection exists true
2022-02-18T20:14:55.5991193Z Running tests using vstest.console.exe runner.
2022-02-18T20:14:55.5991725Z ======================================================
2022-02-18T20:14:55.5997078Z Test selector : Test assemblies
2022-02-18T20:14:55.5998382Z Test filter criteria : null
2022-02-18T20:14:55.5999337Z Search folder : D:\a\r1\a
2022-02-18T20:14:55.6102675Z Action when minimum tests threshold not met : donothing
2022-02-18T20:14:55.6103140Z Minimum tests expected to be run: 0
2022-02-18T20:14:55.6104476Z VisualStudio version selected for test execution : latest
2022-02-18T20:14:55.6107744Z Attempting to find vstest.console from a visual studio installation with version [17.0,18.0).
2022-02-18T20:14:56.0357719Z Attempting to find vstest.console from a visual studio build tools installation with version [17.0,18.0).
2022-02-18T20:14:56.0642271Z Attempting to find vstest.console from a visual studio installation with version [16.0,17.0).
2022-02-18T20:14:56.0884398Z Attempting to find vstest.console from a visual studio build tools installation with version [16.0,17.0).
2022-02-18T20:14:56.1142098Z Attempting to find vstest.console from a visual studio installation with version [15.0,16.0).
2022-02-18T20:14:56.3498931Z Run in parallel : false
2022-02-18T20:14:56.3500994Z Run in isolation : false
2022-02-18T20:14:56.3502074Z Path to custom adapters : null
2022-02-18T20:14:56.3504871Z Other console options : null
2022-02-18T20:14:56.3505911Z Code coverage enabled : false
2022-02-18T20:14:56.3510216Z Diagnostics enabled : false
2022-02-18T20:14:56.3519771Z SystemVssConnection exists true
2022-02-18T20:14:56.3697972Z Run the tests locally using vstest.console.exe
2022-02-18T20:14:56.3698460Z ========================================================
2022-02-18T20:14:56.5771570Z Source filter: D:\a\r1\a/_Test.AppiumTest/AppiumTesting/AppiumTest
2022-02-18T20:14:56.5973322Z ##[warning]No test sources found matching the given filter
1. Check whether the following latest packages are installed or not:
Microsoft.NET.Test.Sdk
MSTest.TestAdapter
MSTest.TestFramework
NUnit3TestAdapter
2. As answered by Marina Liu , you can configure MSTest or the "Visual Studio Test" task in VSO to fail if there are warnings:
a. Add a PowerShell task after Visual Studio Test task.
b. Then get Visual Studio Test task build information by Timeline:
c. Search for Visual Studio Test task information by task name or task display name.
d. Fail the build result if VS test has the warning you specified.
References: How to Fix this C# issue No test matches the given testcase filter `FullyQualifiedName = , How can I configure MSTest to fail the unit test run on any warnings? and Run NUnit tests in Azure DevOps pipeline
I'm writing some unit tests with MSTest, using C#, MSVS 2015 and .Net 4.6.1.
This line:
Assert.ThrowsException<ArgumentOutOfRangeException>( () =>
select.AllSelectedOptions[0]
);
Fails with this compile error:
CS0117 'Assert' does not contain a definition for 'ThrowsException'
My namespace is Microsoft.VisualStudio.TestTools.UnitTesting (the default when you create a unit test project in MSVS).
According to the documentation, Assert.ThrowsException(Action) should exist. But I don't see it in Intellisense ... and I'm getting the compile error.
I've tried a couple of different versions of MSVS (MSVS 2015 and MSVS 2019) and a couple of different versions of MSTest.
Q: Any ideas what might be wrong?
As Clint said below, I need to install MSTest v2 from NuGet in order to use Assert.ThrowsException<T>() in MSVS 2015.
But after doing this, MSVS isn't finding any of my tests anymore:
MSVS > Test > Run All (or "Test > Debug > All Tests"):
------ Discover test started ------
========== Discover test finished: 0 found (0:00:01.127375) ==========
Any suggestions?
You need to use [MSTest V2] to be able to Assert.ThrowsException
Starting with VS2017, the in-box Unit Test Project templates use only MSTest V2.
Now that you're on VS2015 you can install this package MSTest.Test from Nuget but make sure to remove the old test references like Microsoft.VisualStudio.QualityTools.UnitTestFramework before upgrading to this package
Add > New Test project > Select MSTest project type
After this you should be able to use Assert.ThrowsException<ArgumentOutOfRangeException>(
To discover and execute tests also ensure to install MSTest.TestAdapter.
Further Reading
I am using the latest VS2017 version 15.6.4, NUnit3TestAdapter 3.10.0 and Nunit version 3.9.0.0.
When I try to run a unit test in Test Explorer the test are grayed out, when I right click and and run selected tests I see the following error: No test is available
Here is how my test class looks
[TestFixture]
public partial class ListViewBOTest
{
[Test]
public void TestSearch_DateTime()
{
Assert.AreEqual(1,0);
}
}
Text from output:
[3/26/2018 10:53:55 AM Informational] ------ Run test started ------
[3/26/2018 10:53:55 AM Informational] NUnit Adapter 3.10.0.21: Test execution started
[3/26/2018 10:53:55 AM Informational] Running all tests in C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll
[3/26/2018 10:53:56 AM Informational] NUnit failed to load C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll
[3/26/2018 10:53:56 AM Informational] NUnit Adapter 3.10.0.21: Test execution complete
[3/26/2018 10:53:56 AM Warning] No test is available in C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[3/26/2018 10:53:56 AM Informational] ========== Run test finished: 0 run (0:00:01.0435303) ==========
Make sure you have installed the NUnit3 Test Adapter from here
https://marketplace.visualstudio.com/items?itemName=NUnitDevelopers.NUnit3TestAdapter
Can you check the following steps and see if it works?
In the Visual Studio menu, go to Test > Test Settings > Default Processor Architecture and make a note if X86 is selected or X64
Now go to the Build section in the Properties window of the project where the tests are written. Make sure the Platform target drop-down is selected to either Any CPU or at least it matches the architecture from the above step 1.
Now if you build the solution and try running those tests, you should see that they are now running.
I had the same problem as Amete Blessed and commenting out other Test methods made Test Explorer work and run my test
I found that my Build Events were wrong. Invalid copy command blew all my tests and half a day:
Copy C:\repo\Architecture\*.json $(ProjectPath)/Y
Copy C:\repo\Architecture\*.json $(TargetPath) /Y
instead of
Copy C:\repo\Architecture\*.json $(ProjectDir)/Y
Copy C:\repo\Architecture\*.json $(TargetDir) /Y
The messages in the Test Output window ("NUnit failed to load [assembly]", "No test is available...", etc.) can hide the underlying issue that's causing the runner to not load the test assembly. This includes hiding failures to load dependencies of the test assembly or the item under test.
If there's a test assembly that's showing up in the Test Explorer window, but the tests refuse to run, it's worth temporarily enabling fusion logging to see if any assembly binding errors occur when trying to run the tests.
After installing NUnit through nuget tests have appeared into Test Explorer
but when I ran them I got "No test is available".
Installing NUnit Test Adapter fixed the issue.
I had similar problem when using Xamarin.Forms.
The solution was to install NUnit.XForms from NuGet and add
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0" />
to csproj file of the project where the tests are.
What worked for me was to delete the ComponentModelCache folder located at %localappdata%\Microsoft\VisualStudio\<version>
You may have been a casualty of this problem that was (theoretically) resolved in 15.6.3. According to this answer, try deleting your %temp%\VisualStudioTestExplorerExtensions folder. That has resolved the issue for some other users.
Rather than using the Test Explorer, can you right click on the solution and Run Unit Tests from there?
Had the same problem. In my case I found that the NUnit test adapter will not be used by Test Explorer if your test project contains a reference to MSTest. Typically the Microsoft.VisualStudio.TestPlatform.TestFramework.dll but also check your .csproj file for "MSTest" and your packages.config file.
I discovered this by first enabling diagnostic logging for Visual Studio tests. This is found under "Tools" -> "Options" -> "Tests" -> "Logging Level".
In my log I found this entry:
[22/11/2018 10:36:42 Diagnostic] Project C:\Git\myProject\src\myProject.Tests\myProject.Tests.csproj references test adapter: MSTest.TestAdapter, version 1.1.18
This is caused due to memory problem.
Clean object after test execution
Sample Code
private TestController testController;
[OneTimeSetUp]
public void TestSetup()
{
testController= new TestController();
}
[OneTimeTearDown]
public void TestCleanup()
{
testController= null;
}
Change test execution to 64 bits in settings
Test -> Test Settings -> Default Processors Architecture -> x64.
I have also encountered the same issue.
Steps to resolve this issue -
1. Install/Reinstall NUnit3TestAdapter package
2. Delete Debug folder from the Bin
3. Clean --> Build the project
Note - Also ensure all the packages are installed properly.
Now try to execute the tests.
I had the same issue, but it was fixed after installing "NUnit 3 TestAdapter version 3.16.1" via NuGet to my project.
I'm using TeamCity to build my .NET Core project and dotnet test with xunit to run tests.
My build is configured to fail if any tests fail, and this works fine if a test fails, but when the test project doesn't compile the exit code ends up being zero.
I believe this is because I run dotnet test in the folder that contains the test-directories like this: for /f %%%a in ('dir /b /s project.json') do dotnet test %%%a.
Looking at the log I can see the individual jobs that do not compile return with exit code 1, but the build step itself returns with exit code 0.
How can I make the exit code from the failed compilation attempts propagate down to the build step?
You may create a separate step in TC to just compile target test project, and then run tests without compilation (using no-build option) in next step (if all previous steps were successful).
--no-build
Does not build the test project prior to running it.
I'm re-using reports in my sonar config:
sonar.gallio.mode=reuseReport
sonar.gallio.reports.path=gallio-report.xml
sonar.gallio.coverage.reports.path=results.xml
I've previously run Gallio and OpenCover and can confirm that both completed successfully and that Sonar is able to retrieve the files (I've checked the log produced by the -X flag thoroughly). When I view the project in the sonar dashboard, I see code coverage, but not test results:
Kindly ignore the low code coverage percentage, I'm running a small subset of tests while I figure this out.
I should see something that reflects the results I saw when I ran Gallio:
14 run, 13 passed, 1 failed (1 error), 0 inconclusive, 0 skipped
I'm happy to include the gallio-report.xml if that's helpful, but it's 103kb so clearly it contains plenty of data, and I think this is more likely to be a configuration issue.
I'm running OpenCover 4.0.1118 and Gallio 3.2.750 (tests are written with NUnit).
Any thoughts why I don't see any test results?
Chances are that you don't have the test sources in your .NET solution, so when SonarQube tries to import the test execution results, it can't find to which files they should be attached.
In the .NET sample solution, you can see that there is a test project (Example.Core.Tests) which contains the sources of the test classes.