I am trying to run this unit test using Microsoft Shims, but it throws me exception in Shims.Context.Create(); method.
Environment: VS 2012, Win2K8 R2
namespace MyShimsUnitTest
{
[TestClass]
public class MyUnitTest
{
[TestMethod]
public void GetCurrentYear()
{
using (Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create())
{
// Some Logic...
}
}
}
}
Detailed Exception:
Result Message:
Test method MyShimsUnitTest.MyUnitTest.GetCurrentYear threw exception:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: UnitTestIsolation instrumentation failed to initialize. Please restart Visual Studio and rerun this test
Result StackTrace:
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext()
at Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create()
at MyShimsUnitTest.MyUnitTest.GetCurrentYear()
Shims require runtime instrumentation performed by the IntelliTrace profiler. The test runner is responsible for setting up the environment variables required for CLR to load the profiler as well as providing the list of types the profiler must instrument for Shims. The UnitTestIsolationException is thrown when the ShimRuntime is unable to locate and attach to the IntelliTrace profiler, which it expects to be already loaded by the CLR.
As Jin-Wook mentioned earlier, this problem occurs when the test is executed by a runner that does not perform the required profiler initialization. Test Explorer and the vstest.console.exe are two such runners that ship with Visual Studio 2012. At this time, the Visual Studio test runners do not perform the required profiler instrumentation when running tests in "legacy" mode, which happens when you have a .TESTSETTINGS file selected for your run or a .RUNSETTINGS file that forces legacy mode.
You may be able to use third-party test runners that support profiler instrumentation required by Shims.
I had the same issue. The solution to my problem was to uncheck the selected .testsettings file from the menu: TEST/Test Settings and here the item(s) above the Select Test Settings File.
It could be caused by not using the test explorer of vs 2012. To use the shim, you should run tests only using the test explorer.
You can use other test framework such as Nunit or Xunit with the shim if installing appropriate test runner for vs 2012. It can be downloaded from the vs extension manager.
I ran into this issue too. Thankfully the other answers here helped me fix my issue:
I'm using Resharper and when using the context menu I noticed that the runner is using MSTest. Even when finding the test in test explorer and selecting debug I received the same exception.
I then went into Resharpers's options and under Tools -> Unit Testing -> MsTest I unchecked "Enable MSTest support". This unfortunately disables the option to right click on your test and hit run/debug, but it did allow ShimsContext.Create() to behave correctly when selecting debug from the Test Explorer view!
Go to your TestProject Properties -> Under Debug section Check the "ENABLE NATIVE CODE DEBUGGING" checkbox.
This is should do.
We saw this error reported by Bamboo, our build server. It was invoking an MSbuild 4.0 task. The unit test work fine on the dev's local PCs. I deleted this bamboo task and created a new task that invokes Visual Studio 2012's vstest.console. The tests now pass but Bamboo is not able to count the number of tests. This is a Bamboo problem not mine.
Related
I am using in the Test Filter criteria TestCategory=CategoryA
In my Tests I have the below saved and think it should be the only test run but I get success and no tests ran. I have tried with dotnet test and used the same in the arguments and gain it fails with other issues. I can get dot net test to run without adding any arguments for VS Test agent it seems to not want to run even when I do not add the test filter criteria.
[Test]
[Category("CategoryA")]
public async Task ActivateDevice()
2022-02-18T20:14:54.5643568Z ##[section]Starting: VsTest - testAssemblies
2022-02-18T20:14:54.5771277Z ==============================================================================
2022-02-18T20:14:54.5771649Z Task : Visual Studio Test
2022-02-18T20:14:54.5772497Z Description : Run unit and functional tests (Selenium, Appium, Coded UI test, etc.) using the Visual Studio Test (VsTest) runner. Test frameworks that have a Visual Studio test adapter such as MsTest, xUnit, NUnit, Chutzpah (for JavaScript tests using QUnit, Mocha and Jasmine), etc. can be run. Tests can be distributed on multiple agents using this task (version 2).
2022-02-18T20:14:54.5773342Z Version : 2.198.0
2022-02-18T20:14:54.5773587Z Author : Microsoft Corporation
2022-02-18T20:14:54.5773921Z Help : https://learn.microsoft.com/azure/devops/pipelines/tasks/test/vstest
2022-02-18T20:14:54.5774333Z ==============================================================================
2022-02-18T20:14:55.5713320Z SystemVssConnection exists true
2022-02-18T20:14:55.5732257Z SystemVssConnection exists true
2022-02-18T20:14:55.5991193Z Running tests using vstest.console.exe runner.
2022-02-18T20:14:55.5991725Z ======================================================
2022-02-18T20:14:55.5997078Z Test selector : Test assemblies
2022-02-18T20:14:55.5998382Z Test filter criteria : null
2022-02-18T20:14:55.5999337Z Search folder : D:\a\r1\a
2022-02-18T20:14:55.6102675Z Action when minimum tests threshold not met : donothing
2022-02-18T20:14:55.6103140Z Minimum tests expected to be run: 0
2022-02-18T20:14:55.6104476Z VisualStudio version selected for test execution : latest
2022-02-18T20:14:55.6107744Z Attempting to find vstest.console from a visual studio installation with version [17.0,18.0).
2022-02-18T20:14:56.0357719Z Attempting to find vstest.console from a visual studio build tools installation with version [17.0,18.0).
2022-02-18T20:14:56.0642271Z Attempting to find vstest.console from a visual studio installation with version [16.0,17.0).
2022-02-18T20:14:56.0884398Z Attempting to find vstest.console from a visual studio build tools installation with version [16.0,17.0).
2022-02-18T20:14:56.1142098Z Attempting to find vstest.console from a visual studio installation with version [15.0,16.0).
2022-02-18T20:14:56.3498931Z Run in parallel : false
2022-02-18T20:14:56.3500994Z Run in isolation : false
2022-02-18T20:14:56.3502074Z Path to custom adapters : null
2022-02-18T20:14:56.3504871Z Other console options : null
2022-02-18T20:14:56.3505911Z Code coverage enabled : false
2022-02-18T20:14:56.3510216Z Diagnostics enabled : false
2022-02-18T20:14:56.3519771Z SystemVssConnection exists true
2022-02-18T20:14:56.3697972Z Run the tests locally using vstest.console.exe
2022-02-18T20:14:56.3698460Z ========================================================
2022-02-18T20:14:56.5771570Z Source filter: D:\a\r1\a/_Test.AppiumTest/AppiumTesting/AppiumTest
2022-02-18T20:14:56.5973322Z ##[warning]No test sources found matching the given filter
1. Check whether the following latest packages are installed or not:
Microsoft.NET.Test.Sdk
MSTest.TestAdapter
MSTest.TestFramework
NUnit3TestAdapter
2. As answered by Marina Liu , you can configure MSTest or the "Visual Studio Test" task in VSO to fail if there are warnings:
a. Add a PowerShell task after Visual Studio Test task.
b. Then get Visual Studio Test task build information by Timeline:
c. Search for Visual Studio Test task information by task name or task display name.
d. Fail the build result if VS test has the warning you specified.
References: How to Fix this C# issue No test matches the given testcase filter `FullyQualifiedName = , How can I configure MSTest to fail the unit test run on any warnings? and Run NUnit tests in Azure DevOps pipeline
I am using the latest VS2017 version 15.6.4, NUnit3TestAdapter 3.10.0 and Nunit version 3.9.0.0.
When I try to run a unit test in Test Explorer the test are grayed out, when I right click and and run selected tests I see the following error: No test is available
Here is how my test class looks
[TestFixture]
public partial class ListViewBOTest
{
[Test]
public void TestSearch_DateTime()
{
Assert.AreEqual(1,0);
}
}
Text from output:
[3/26/2018 10:53:55 AM Informational] ------ Run test started ------
[3/26/2018 10:53:55 AM Informational] NUnit Adapter 3.10.0.21: Test execution started
[3/26/2018 10:53:55 AM Informational] Running all tests in C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll
[3/26/2018 10:53:56 AM Informational] NUnit failed to load C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll
[3/26/2018 10:53:56 AM Informational] NUnit Adapter 3.10.0.21: Test execution complete
[3/26/2018 10:53:56 AM Warning] No test is available in C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[3/26/2018 10:53:56 AM Informational] ========== Run test finished: 0 run (0:00:01.0435303) ==========
Make sure you have installed the NUnit3 Test Adapter from here
https://marketplace.visualstudio.com/items?itemName=NUnitDevelopers.NUnit3TestAdapter
Can you check the following steps and see if it works?
In the Visual Studio menu, go to Test > Test Settings > Default Processor Architecture and make a note if X86 is selected or X64
Now go to the Build section in the Properties window of the project where the tests are written. Make sure the Platform target drop-down is selected to either Any CPU or at least it matches the architecture from the above step 1.
Now if you build the solution and try running those tests, you should see that they are now running.
I had the same problem as Amete Blessed and commenting out other Test methods made Test Explorer work and run my test
I found that my Build Events were wrong. Invalid copy command blew all my tests and half a day:
Copy C:\repo\Architecture\*.json $(ProjectPath)/Y
Copy C:\repo\Architecture\*.json $(TargetPath) /Y
instead of
Copy C:\repo\Architecture\*.json $(ProjectDir)/Y
Copy C:\repo\Architecture\*.json $(TargetDir) /Y
The messages in the Test Output window ("NUnit failed to load [assembly]", "No test is available...", etc.) can hide the underlying issue that's causing the runner to not load the test assembly. This includes hiding failures to load dependencies of the test assembly or the item under test.
If there's a test assembly that's showing up in the Test Explorer window, but the tests refuse to run, it's worth temporarily enabling fusion logging to see if any assembly binding errors occur when trying to run the tests.
After installing NUnit through nuget tests have appeared into Test Explorer
but when I ran them I got "No test is available".
Installing NUnit Test Adapter fixed the issue.
I had similar problem when using Xamarin.Forms.
The solution was to install NUnit.XForms from NuGet and add
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0" />
to csproj file of the project where the tests are.
What worked for me was to delete the ComponentModelCache folder located at %localappdata%\Microsoft\VisualStudio\<version>
You may have been a casualty of this problem that was (theoretically) resolved in 15.6.3. According to this answer, try deleting your %temp%\VisualStudioTestExplorerExtensions folder. That has resolved the issue for some other users.
Rather than using the Test Explorer, can you right click on the solution and Run Unit Tests from there?
Had the same problem. In my case I found that the NUnit test adapter will not be used by Test Explorer if your test project contains a reference to MSTest. Typically the Microsoft.VisualStudio.TestPlatform.TestFramework.dll but also check your .csproj file for "MSTest" and your packages.config file.
I discovered this by first enabling diagnostic logging for Visual Studio tests. This is found under "Tools" -> "Options" -> "Tests" -> "Logging Level".
In my log I found this entry:
[22/11/2018 10:36:42 Diagnostic] Project C:\Git\myProject\src\myProject.Tests\myProject.Tests.csproj references test adapter: MSTest.TestAdapter, version 1.1.18
This is caused due to memory problem.
Clean object after test execution
Sample Code
private TestController testController;
[OneTimeSetUp]
public void TestSetup()
{
testController= new TestController();
}
[OneTimeTearDown]
public void TestCleanup()
{
testController= null;
}
Change test execution to 64 bits in settings
Test -> Test Settings -> Default Processors Architecture -> x64.
I have also encountered the same issue.
Steps to resolve this issue -
1. Install/Reinstall NUnit3TestAdapter package
2. Delete Debug folder from the Bin
3. Clean --> Build the project
Note - Also ensure all the packages are installed properly.
Now try to execute the tests.
I had the same issue, but it was fixed after installing "NUnit 3 TestAdapter version 3.16.1" via NuGet to my project.
I've been running Selenium tests in Visual Studio using C# and everything works fine locally. I have an automated build process in TFS that runs unit tests in my solution. That task (Test Assemblies) is finding the new Selenium tests but failing to run them. However, when I created a separate build definition that deployed a test agent (successfully) and then attempts to run functional tests, I get this message:
2017-11-03T18:49:43.1345753Z ##[warning]DistributedTests: Test Run Discovery Aborted . Test run id : 1600
2017-11-03T18:49:43.1345753Z ##[warning]DistributedTests: UnExpected error occured during test execution. Try again.
2017-11-03T18:49:43.1345753Z ##[warning]DistributedTests: Error : No tests were discovered from the specified test sources
I have searched the DTALog, and found that the test sources are being found successfully, just no actual tests within them. Any ideas what I am doing wrong? I have removed the 'Owner' decoration from the tests.
Please try below things to narrow down the issue:
Make sure the appropriate test adapter getting deployed along with
the test assemblies as Daniel mentioned.
If not deployed, you would need to copy the appropriate adapter from your local vs machine (\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions) to your test agent box (\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions\Microsoft.VisualStudio.TestPlatform.Extensions)
Try run directly from vstest.console.exe on test agent machine with
specifing test adapter path using /testadapterpath flag and see if
vstest.console works.
eg:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "F:\Test****TestAutomation.dll" /TestAdapterPath:F:\Test*\
Check if you have included the dependencies with the test dlls.
eg: manually add reference of Microsoft.VisualStudio.QualityTools.UnitTestFramework in the test project.
If the isse persist, please share the logs for furthre troubleshoot.
I have a NUnit test project that I'm trying to run on my TFS 2012 Build server.
I'm getting the following error:
Exception System.InvalidOperationException, Exception thrown
executing tests
No test is available in C:\Builds\1\Projects\NUNIT
Test\bin\NUnitTestProject.dll C:\Builds\1\Projects\NUNIT
Test\bin\NUnitTestProject.dll. Make sure that installed test
discoverers & executors, platform & framework version settings are
appropriate and try again.
I Have the NUnit.VisualStudio.TestAdapter 1.1.0.0 installed from NuGet and referenced in the project.
I also have set the Version control path to custom assemblies set on the controller, which contains (they basically are a copy of the NuGet package):
nunit.core.dll
nunit.core.interfaces.dll
nunit.util.dll
NUnit.VisualStudio.TestAdapter.dll
For test purposes, I'm only building the NUnit test project.
My test class is very simple and is the following:
using NUnit.Framework;
namespace NUnitTestProject
{
[TestFixture]
public class MySuperTests
{
[Test]
public void MyFirstUnitTestMethod_ReturnsTrue()
{
Assert.IsTrue(true);
}
}
}
Is there something I haven't done correctly?
Edit 1:
Using TFS 2012 Update 4 on my build server
Edit 2:
The problem I believe lies in the 1.1.0.X version.
Updating to 1.1.0.7 I did not have the exception anymore, however, I did have the tests not found problem.
There are some issues going on about that on github https://github.com/nunit/nunit-vs-adapter/
Edit 3:
The problem has been fixed in 1.1.0.8 of the NUnit test adapter!
Do you have your test runner set correctly in the build defiintion?
Image shows TFS2013 but 2012 has a different option for 2010 compatability which may be selected. The Visual Studio Test Runner should support multiple test frameworks, whereas the 2010 compatability one only supports MSTest
We're asked to move from NUnit to MSTest and now have to convert all the existing tests to the new platform. Most of it converted fine but we have an issue with parameterised tests. We found the following web site showing how to write an extension and downloaded the code for it.
http://blogs.msdn.com/b/vstsqualitytools/archive/2009/09/04/extending-the-visual-studio-unit-test-type-part-2.aspx
However, although it compiles fine, the RowTestClass test all fail to run with the following error message:
"Failed to initialize the unit test extension 'urn:RunAsTestClassAttribute': A unit test extension is not registered for the following attribute: Microsoft.VisualStudio.Test.Sample.UnitTestTypeExtension.RunAsSample.RunAsTestClassAttribute."
We're using VS2010 (10.0.40219.1 SP1Rel) (Microsoft .NET Framework Version 4.0.30319 SP1Rel)
I wonder if it has something to do with the following libraries:
Microsoft.VisualStudio.QualityTools.Common.dll
Runtime Version: v2.0.50727
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\ReferenceAssemblies\v2.0\Microsoft.VisualStudio.QualityTools.Common.dll
Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
Runtime Version: v4.0.30319
C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\PublicAssemblies\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll
Microsoft.VisualStudio.QualityTools.Vsip.dll
v4.0.30319
C:\Windows\Microsoft.Net\assembly\GAC_MSIL\Microsoft.VisualStudio.QualityTools.Vsip\v4.0_10.0.0.0__b03f5f7f11d50a3a\Microsoft.VisualStudio.QualityTools.Vsip.dll
Any help to get this working greatly appreciated.
Have you registered the new types (in the Windows Registry)? There's a section on how to do it in the tutorial - have a look at Row Test Test Type Extension: Registering your test type extension.
If you're having trouble, try downloading the complete code + registry keys from the MSDN code gallery.
You may want to look into using SpecFlow to make your tests more parameterized. It is traditionally a BDD tool, but it will allow you to make re-usable steps that take parameters.
For example you can create a step saying
"Given the user 'admin' exists."
and on the c# side you get a method like this:
[Given(#"the user '(.*)' exists"]
public void GivenTheUserExists(string name)
{
//create the user
}
Going forward you can reuse the step in any "feature file" without having to rewrite the implementation of that step.
Just refer to the blog post Extending Unit Tests in Visual Studio. And see what you might have missed. I tried the same, and it works well for me.