I have a series of unit tests that connect to an Azure Storage emulator. In Setup my code checks if there is something listening on the emulator's port, and if not sets a flag StorageNotAvailable.
In each of my tests I have some code...
if ( StorageNotAvailable )
Assert.Inconclusive( "Storage emulator is not available" )
// where storage emulator is available, continue as normal
As expected, when the test returns void this reports correctly in the Test Explorer as "Inconclusive".
When the test is exercising some async methods, and the [TestMethod] signature returns Task then the test is reported in the TestExplorer as "Failed" instead of "Inconclusive".
How can I get an async method to report as Inconclusive?
EDIT
Some additional detail may be in order. Here are some sample tests I rigged up to demonstrate the problem I am seeing.
[TestMethod]
public void MyTestMethod()
{
Assert.Inconclusive( "I am inconclusive" );
}
[TestMethod]
public async Task MyTestMethodAsync()
{
Assert.Inconclusive( "I am an error" );
}
Some environment details may be in order as well:
Windows 10 x64 1703 Build 15063.608
Visual Studio Enterprise 2017 15.3.5
.NET 4.7.02046
VS Extensions that might be relevant
Microsoft Visual Studio Test Platform
MSTest V2 Create Unit Test Extension
MSTest V2 IntelliTest Extension
MSTest V2 Templates
Project references that might be relevant
Microsoft.VisualStudio.TestPlatform.TestFramework v14.0.0.0
Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions v14.0.0.0
Project nuGet packages that might be relevant
MSTest.TestAdapter v1.1.18
MSTest.TestFramework v1.1.18
Project build target is .NET Framework v4.7
Assert.Inconclusive raises a special kind of exception, which will cause the task to catch that exception. Since the Task library and async don't know about, we can't blame them for complaining. The Task framework will wrap the exception in an AggregateException, which I suspect is getting reported. This was a nice assumption, but it turned out that the code looking for AssetInconclusiveException was comparing the raised instance against MstestV1's implementation and not MsTestV2.
But I suppose this should be considered a bug in the MsTest v2 runner, which should inspect all Tasks that failed and look at the exception that caused their failure.
The behaviour is a known behaviour at the moment and I've just submitted a PR to fix this. Pull Request Merged, now just wait for the next Nuget build to trigger.
This fix was merged and released in the latest 1.2.0 package.
This is caused by a confirmed bug in MsTest.Framework code -- issue #249 on GitHub is tracking the problem and eventual solution:
Async tests which use Assert.Inconclusive report as Error
https://github.com/Microsoft/testfx/issues/249
My situation is similar to user2768132's question (VS2010 pro not able to start Moles host), however, I have a couple differences. I also attempted the suggestion by SouthShoreAK but it didn't resolve my problem. Pardon the similar post.
I'm new to this project at work as well as Moles so I might be missing something simple/obvious. We are also using Moles framework to write unit test cases but are not able to either debug or run a unit test that involves Moling a public static class.
System - Win-7 Professional SP1
.NET - .NET v4.0.30319 SP1 Rel
VS - VS2010 Professional v10.0.40219.1
Moles - v0.94.51023.0
The solution builds successfully. The 6 simple unit tests (i.e. not Moling static classes) pass but the one unit test that requires Moles to deal with a static class aborts.
Error for the aborted test:
Error 3/26/2014 2:26:06 PM System.InvalidOperationException: Could not start Moles host. Please review the Test Run Errors for more information.
at Microsoft.Moles.VsHost.Agent.HostTestAdapterDriver.EnsureHostAdapter()
at Microsoft.Moles.VsHost.Agent.HostTestAdapterDriver.Microsoft.VisualStudio.TestTools.Execution.IBaseAdapter.Run(ITestElement testElement, ITestContext testContext)
at Microsoft.Moles.VsHost.Agent.MolesAgentAdapter.Run(ITestElement testElement, ITestContext testContext)
My fuslogvw was empty and did not identify any errors during assembly loading.
I found the same blog as user2768132 that mentioned removing the .exe.config file from privateassemblies folder under VS2010 IDE folder. I did that and it didn't fix my problem either.
I believe the tests are 32 bit. With that said, I attempted SouthShoreAK's suggestion of editing the test settings to 64 bit and adding the bitness line to the bottom of the AssemblyInfo.cs file. Unfortunately, that didn't solve my problem either. The simple tests that originally passed would fail and the unit test requiring Moles would error saying that Moles cannot be loaded because the key 'Moles' cannot be found. I undid these changes and I'm back to my starting point.
Anyone have any ideas/suggestions?
Thanks,
Steve
Answer was to make sure the "Platform target" parameter was set to x86 in the Build tab of the test project's Properties.
I am trying to run this unit test using Microsoft Shims, but it throws me exception in Shims.Context.Create(); method.
Environment: VS 2012, Win2K8 R2
namespace MyShimsUnitTest
{
[TestClass]
public class MyUnitTest
{
[TestMethod]
public void GetCurrentYear()
{
using (Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create())
{
// Some Logic...
}
}
}
}
Detailed Exception:
Result Message:
Test method MyShimsUnitTest.MyUnitTest.GetCurrentYear threw exception:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: UnitTestIsolation instrumentation failed to initialize. Please restart Visual Studio and rerun this test
Result StackTrace:
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext()
at Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create()
at MyShimsUnitTest.MyUnitTest.GetCurrentYear()
Shims require runtime instrumentation performed by the IntelliTrace profiler. The test runner is responsible for setting up the environment variables required for CLR to load the profiler as well as providing the list of types the profiler must instrument for Shims. The UnitTestIsolationException is thrown when the ShimRuntime is unable to locate and attach to the IntelliTrace profiler, which it expects to be already loaded by the CLR.
As Jin-Wook mentioned earlier, this problem occurs when the test is executed by a runner that does not perform the required profiler initialization. Test Explorer and the vstest.console.exe are two such runners that ship with Visual Studio 2012. At this time, the Visual Studio test runners do not perform the required profiler instrumentation when running tests in "legacy" mode, which happens when you have a .TESTSETTINGS file selected for your run or a .RUNSETTINGS file that forces legacy mode.
You may be able to use third-party test runners that support profiler instrumentation required by Shims.
I had the same issue. The solution to my problem was to uncheck the selected .testsettings file from the menu: TEST/Test Settings and here the item(s) above the Select Test Settings File.
It could be caused by not using the test explorer of vs 2012. To use the shim, you should run tests only using the test explorer.
You can use other test framework such as Nunit or Xunit with the shim if installing appropriate test runner for vs 2012. It can be downloaded from the vs extension manager.
I ran into this issue too. Thankfully the other answers here helped me fix my issue:
I'm using Resharper and when using the context menu I noticed that the runner is using MSTest. Even when finding the test in test explorer and selecting debug I received the same exception.
I then went into Resharpers's options and under Tools -> Unit Testing -> MsTest I unchecked "Enable MSTest support". This unfortunately disables the option to right click on your test and hit run/debug, but it did allow ShimsContext.Create() to behave correctly when selecting debug from the Test Explorer view!
Go to your TestProject Properties -> Under Debug section Check the "ENABLE NATIVE CODE DEBUGGING" checkbox.
This is should do.
We saw this error reported by Bamboo, our build server. It was invoking an MSbuild 4.0 task. The unit test work fine on the dev's local PCs. I deleted this bamboo task and created a new task that invokes Visual Studio 2012's vstest.console. The tests now pass but Bamboo is not able to count the number of tests. This is a Bamboo problem not mine.
I have a c# solution with the following structure:
mySolution
myProject
myProject.MSTests
References
Microsoft.VisualStudio.QualityTools.UnitTestFramework
sutMSTests.cs
sutMSTests.cs:
[TestClass()]
public class sutMSTests
{
[TestMethod]
public void MyTest0()
{
Microsoft.VisualStudio.TestTools.UnitTesting.Assert.AreEqual(4, 2 + 2);
}
}
When I try to run the tests via Test, Run, All Tests In Solution, I get the following on the VS2008 status line:
No tests are run because no tests are loaded or the selected tests are disabled.
Test, Windows, Test View shows no tests.
Note: I created the tests manually (works for xUnit.net) instead of using Microsoft's wizards.
I've compared my hand created MSTest setup to the setup another test that I generated using the wizard and they appear to be sufficiently similar.
Question: What are the most likely causes of the error message above?
Edit 2010-02-25: More information:
I right clicked the Solution Items folder, and choose Add, New Project, type Test Projects,Test Documents::Visual Studio Test Project template.
The new project's default do nothing test "TestMethod1" was detected and passed.
However, my test did not show up ... so I copied and pasted my test method into the default test test project "TestProject1".
My test was detected in "TestProject" BUT not in its original location.
I closely compared the files, organization, and settings of "TestProject1" with my hand created test project.
At this point, I am guessing that some setting gets made by the Visual Studio Test Project template that is not easily detectable.
imo, it should be just as easy to create a test project by hand as it is to create one with the Visual Studio Test Project template.
please note: I'm not saying that I'm against using the Visual Studio Test Project template; for me, I like to understand what's behind the curtain since this makes me imho a much better programmer.
Another one for the googlers - this one turned out to be my problem, and it's embarrassingly boneheaded of me. Make sure that your test project is set to build in whatever solution configuration you're using. If the test assembly isn't being built, VS won't be able to find any tests in the non-existent assembly, and you'll bang your head against the wall for a while :-)
Possibly a bit late, but this question googles up well, I thought I'd throw some crumbs in for future googlers.
Bryan Cook suggests checking the ProjectTypeGuids in his blog post about Manually creating a MS Test Project. Apparently the magic GUIDs you need are {3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC} for c# and {3AC096D0-A1C2-E12C-1390-A8335801FDAB};{F184B08F-C81C-45F6-A57F-5ABD9991F28F} for VB. See his blog post for more details.
In case the blog post ever goes away you need to add the following element in the main property group in the csproj file:
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
Another idea for the Googlers out there. My problem was trying to get ignored tests running again. Same MS error message occurs if you remove the Ignore label. Does not automatically re-enable the test. This article takes you through the last step. http://richallen.blogspot.com/2008/05/ms-test-re-enabling-ignored-tests.html
The fix is simple, even though it shouldn't be needed, if Visual Studio worked as it should.
To sum up what others have contributed, particularly in this article, here's what ultimately worked for me:
Use the Configuration Manager to make sure your test project is selected to build in whatever configuration and platform you're using (ex: configuration=Debug and platform=x86)
Make sure your method belongs to a [TestClass] and that it's both marked [TestMethod], and NOT using the attribute [Ignore]
Use Test View to find your test.
Open your Properties window (F4), and make sure your test is enabled
The original poster did do this, but I arrived here after not having done this:
Be sure that [TestClass] is declared at the top, public in scope:
namespace XYZ.API.Repository.Tests
{
[TestClass()]
public class ClientTests
{
I was receiving the same message and it turned out to be that I had my unit test project on a network drive. Once I moved it local it ran fine. Just something to try if you get this error.
John
I've just manually done this:
Created a new C# class library project with the following code:
namespace SO_Answer
{
public class Class1
{
public void Test()
{
var k = "Hello";
}
}
}
Saved the project and then went to 'File->Add->New Project' and chose 'Test Project'. After VS created the unit test project, I added a reference to the class library project I created earlier.
In my test I have this code:
namespace Unit_Test
{
/// <summary>
/// Summary description for UnitTest1
/// </summary>
[TestClass]
public class UnitTest1
{
/// <summary>
///Gets or sets the test context which provides
///information about and functionality for the current test run.
///</summary>
public TestContext TestContext { get; set; }
#region Additional test attributes
// You can use the following additional attributes as you write your tests:
// Use ClassInitialize to run code before running the first test in the class
// [ClassInitialize()]
// public static void MyClassInitialize(TestContext testContext) { }
// Use ClassCleanup to run code after all tests in a class have run
// [ClassCleanup()]
// public static void MyClassCleanup() { }
// Use TestInitialize to run code before running each test
// [TestInitialize()]
// public void MyTestInitialize() { }
// Use TestCleanup to run code after each test has run
// [TestCleanup()]
// public void MyTestCleanup() { }
#endregion
/// <summary>
/// The test method 1.
/// </summary>
[TestMethod]
public void TestMethod1()
{
var f = new Class1();
}
}
}
The only code I added was the a using statement and the var f = new Class1(); statement. Looking at the MSTest runner, I can see TestMethod1 appear.
I can't think of a reason why your unit tests are not being picked up. The only time I've had this is because I was using the MSTest runner to try and view NUnit tests by mistake. Try starting from scratch.
This could be another reason. Check whether the solution is running on 64bit. If so change it to x86.
This must be a bug and is an absolute pain especially as you have to reenable every single test method individually. However a bit iof lateral thinking produced a better solution - rename the test class and rebuild. Then rename it back. Seems to work.
Ooops - no it doesn't. Renaming the class works but when it's renamed back it reverts to the original settings.
The trick is to close down Visual Studio and delete the .vsmdi (visual studio test meta data) file. This will be regenrated.
When you run into this issue, in Visual Studio, you have to create a Test Project.
1. Select Test in Tool bar and choose "New Test". Create your project and at this point create your test method. It should work after this point.
For posterity: I just found that marking tests as static made them silently fail to appear in the test list. Apparently that isn't allowed.
None of the other answers worked for me. I kept getting the following message in the output window:
------ Discover test started ------
========== Discover test finished: 2 found (0:00:00.1310428) ==========
No tests found to run.
In my case, the problem only occurred after I created a new configuration called 0-Local. I had to add <DebugSymbols>true</DebugSymbols to the relevant section of my csproj file, so it looks like this:
<PropertyGroup Condition="'$(Configuration)|$(Platform)' == '0-Local|AnyCPU'">
<DebugSymbols>true</DebugSymbols>
<OutputPath>bin\0-Local\</OutputPath>
</PropertyGroup>
Do you have a VSMDI file in your solution? I believe this file is required (NOT VERIFIED).
this is typical problem i have faced too. but the easiest solution I followed as my own is...just to build the project once and rebuild it again. so that you can resolve it.
Had this same issue but reading over the previous answers, everything looked good.
In my case, I had just run the test suite made a small change, built the solution and tried to run the test. No go. I tried building a couple more times and looking for problems other people had tried. Still no go.
I hit enter in one of my test methods to add a new and hit F6 to build the solution and clicked run Unit Tests.
Bingo! Everything ran smoothly.
I was making use of a public TestContext TestContext method to write to the test output and changed the scope to private. This made every test not discoverable. Changing it back to public helped.
Another one for googlers using NUnit, especially those who have migrated from MS Unit test to NUnit. Please remove the project type Guids that are identifying the project as MS Test project from the project file.
<ProjectTypeGuids>{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}</ProjectTypeGuids>
If your code is CLI (managed c++) and your test class inherits from an abstract base class, make sure that your test class implements the base's pure virtual method.
if you didn't implement it, you may see the "no tests found to run" message.
When I run the following test in Gallio's Icarus it passes, but when I step into it using TestDriven.NET (Test With->Debugger), it fails because the parameters are not set according to the Row attributes.
I was expecting that the method would be called once for each Row attribute applied.
What am I doing wrong? If nothing, then what do I need to do to debug these tests when they break? Should I avoid parametrized tests if they are not debuggable?
[TestFixture]
public class TestDrivenIgnoresMbUnitAttributesWhenDebugging
{
[Test]
[Row(1)]
[Row(2)]
public void SomeFunc(int x)
{
Assert.AreNotEqual(default(int), x);
}
}
Hmm... did you install TestDriven.Net BEFORE installing Gallio?
If not, then the Gallio extensions for TestDriven.Net will not be installed. In that case, TestDriven.Net might run the test in "ad-hoc" mode with default values for its parameters.
It should be pretty to tell whether this is the case. If the Gallio extensions for TestDriven.Net are installed then you'll see a "Gallio" banner message in the Output Window during test execution. If not you may see something else like "ad-hoc".
To fix the problem, reinstall Gallio. Alternately you can use the add/remove features part of the Gallio installer then ensure that the TestDriven.Net components are selected for installation (under "Test Runners").