I'm using a simple proof-of-concept Fakes nUnit test:
[Test]
public void TestFakes()
{
using (var ctx = ShimsContext.Create())
{
System.Fakes.ShimDateTime.NowGet = () => { return new DateTime(2000, 1, 1); };
Assert.That(DateTime.Now.Year, Is.EqualTo(2000));
}
}
This test runs in the Visual Studio Test Explorer, but doesn't run in:
nUnit GUI
nUnit console
The JetBrains test runner (dotCover OR Resharper)
TestDriven.net test runner
In each of these, I receive the following error:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException : Failed to resolve profiler path from COR_PROFILER_PATH and COR_PROFILER environment variables
When I reflect into that assembly, it seems like it's looking for Intellitrace, a VS Ultimate-only feature - I only have Premium installed.
Any suggestions on how to work around this (we use the nUnit runner on our build servers, so this is a blocker to using Fakes)
I don't think you will be able to execute MS Fakes based tests using anything other then MS Test framework.
I believe that the way MS Fakes works causes problems for test runners such as NUnit. Precisely why this is the case, I don't know, since other mocking frameworks such as TypeMock work fine in NUnit, etc. But there is something specific to MS Fakes which make it harder (if not impossible) to run with anything other than MS Test. That's my theory anyway.
Unless the authors of NUnit, xUnit, etc add support for MS Fakes (or there is a crafty workaround), I think you will have to stick with MS Test.
EDIT:
It looks like the latest version of NCrunch v2.5 does work with MS Fakes. I've tried the beta during it's development, and can confirm that MS Fake tests were executed without fail using NCrunch.
Fakes works only with Visual Studio Test Runner(AKA VStest.Console.exe). Even previous Microsoft MSTest runner doesn't support it.
I believe your answer is to use one of following commands:
VSTest.Console.exe UnitTests.dll /UseVsixExtensions
VSTest.Console.exe UnitTests.dll /TestAdapterPath:%LocalAppData%\Microsoft\VisualStudio\12.0\Extensions\<nunit test adapter installation folder>
These commands will run fine on dev machines. In case you want to run unit tests on build server, copy nunit test adapter folder to build server and mention that path.
Related
I've been running Selenium tests in Visual Studio using C# and everything works fine locally. I have an automated build process in TFS that runs unit tests in my solution. That task (Test Assemblies) is finding the new Selenium tests but failing to run them. However, when I created a separate build definition that deployed a test agent (successfully) and then attempts to run functional tests, I get this message:
2017-11-03T18:49:43.1345753Z ##[warning]DistributedTests: Test Run Discovery Aborted . Test run id : 1600
2017-11-03T18:49:43.1345753Z ##[warning]DistributedTests: UnExpected error occured during test execution. Try again.
2017-11-03T18:49:43.1345753Z ##[warning]DistributedTests: Error : No tests were discovered from the specified test sources
I have searched the DTALog, and found that the test sources are being found successfully, just no actual tests within them. Any ideas what I am doing wrong? I have removed the 'Owner' decoration from the tests.
Please try below things to narrow down the issue:
Make sure the appropriate test adapter getting deployed along with
the test assemblies as Daniel mentioned.
If not deployed, you would need to copy the appropriate adapter from your local vs machine (\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions) to your test agent box (\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions\Microsoft.VisualStudio.TestPlatform.Extensions)
Try run directly from vstest.console.exe on test agent machine with
specifing test adapter path using /testadapterpath flag and see if
vstest.console works.
eg:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "F:\Test****TestAutomation.dll" /TestAdapterPath:F:\Test*\
Check if you have included the dependencies with the test dlls.
eg: manually add reference of Microsoft.VisualStudio.QualityTools.UnitTestFramework in the test project.
If the isse persist, please share the logs for furthre troubleshoot.
I have a NUnit test project that I'm trying to run on my TFS 2012 Build server.
I'm getting the following error:
Exception System.InvalidOperationException, Exception thrown
executing tests
No test is available in C:\Builds\1\Projects\NUNIT
Test\bin\NUnitTestProject.dll C:\Builds\1\Projects\NUNIT
Test\bin\NUnitTestProject.dll. Make sure that installed test
discoverers & executors, platform & framework version settings are
appropriate and try again.
I Have the NUnit.VisualStudio.TestAdapter 1.1.0.0 installed from NuGet and referenced in the project.
I also have set the Version control path to custom assemblies set on the controller, which contains (they basically are a copy of the NuGet package):
nunit.core.dll
nunit.core.interfaces.dll
nunit.util.dll
NUnit.VisualStudio.TestAdapter.dll
For test purposes, I'm only building the NUnit test project.
My test class is very simple and is the following:
using NUnit.Framework;
namespace NUnitTestProject
{
[TestFixture]
public class MySuperTests
{
[Test]
public void MyFirstUnitTestMethod_ReturnsTrue()
{
Assert.IsTrue(true);
}
}
}
Is there something I haven't done correctly?
Edit 1:
Using TFS 2012 Update 4 on my build server
Edit 2:
The problem I believe lies in the 1.1.0.X version.
Updating to 1.1.0.7 I did not have the exception anymore, however, I did have the tests not found problem.
There are some issues going on about that on github https://github.com/nunit/nunit-vs-adapter/
Edit 3:
The problem has been fixed in 1.1.0.8 of the NUnit test adapter!
Do you have your test runner set correctly in the build defiintion?
Image shows TFS2013 but 2012 has a different option for 2010 compatability which may be selected. The Visual Studio Test Runner should support multiple test frameworks, whereas the 2010 compatability one only supports MSTest
I am trying to run this unit test using Microsoft Shims, but it throws me exception in Shims.Context.Create(); method.
Environment: VS 2012, Win2K8 R2
namespace MyShimsUnitTest
{
[TestClass]
public class MyUnitTest
{
[TestMethod]
public void GetCurrentYear()
{
using (Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create())
{
// Some Logic...
}
}
}
}
Detailed Exception:
Result Message:
Test method MyShimsUnitTest.MyUnitTest.GetCurrentYear threw exception:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: UnitTestIsolation instrumentation failed to initialize. Please restart Visual Studio and rerun this test
Result StackTrace:
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext()
at Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create()
at MyShimsUnitTest.MyUnitTest.GetCurrentYear()
Shims require runtime instrumentation performed by the IntelliTrace profiler. The test runner is responsible for setting up the environment variables required for CLR to load the profiler as well as providing the list of types the profiler must instrument for Shims. The UnitTestIsolationException is thrown when the ShimRuntime is unable to locate and attach to the IntelliTrace profiler, which it expects to be already loaded by the CLR.
As Jin-Wook mentioned earlier, this problem occurs when the test is executed by a runner that does not perform the required profiler initialization. Test Explorer and the vstest.console.exe are two such runners that ship with Visual Studio 2012. At this time, the Visual Studio test runners do not perform the required profiler instrumentation when running tests in "legacy" mode, which happens when you have a .TESTSETTINGS file selected for your run or a .RUNSETTINGS file that forces legacy mode.
You may be able to use third-party test runners that support profiler instrumentation required by Shims.
I had the same issue. The solution to my problem was to uncheck the selected .testsettings file from the menu: TEST/Test Settings and here the item(s) above the Select Test Settings File.
It could be caused by not using the test explorer of vs 2012. To use the shim, you should run tests only using the test explorer.
You can use other test framework such as Nunit or Xunit with the shim if installing appropriate test runner for vs 2012. It can be downloaded from the vs extension manager.
I ran into this issue too. Thankfully the other answers here helped me fix my issue:
I'm using Resharper and when using the context menu I noticed that the runner is using MSTest. Even when finding the test in test explorer and selecting debug I received the same exception.
I then went into Resharpers's options and under Tools -> Unit Testing -> MsTest I unchecked "Enable MSTest support". This unfortunately disables the option to right click on your test and hit run/debug, but it did allow ShimsContext.Create() to behave correctly when selecting debug from the Test Explorer view!
Go to your TestProject Properties -> Under Debug section Check the "ENABLE NATIVE CODE DEBUGGING" checkbox.
This is should do.
We saw this error reported by Bamboo, our build server. It was invoking an MSbuild 4.0 task. The unit test work fine on the dev's local PCs. I deleted this bamboo task and created a new task that invokes Visual Studio 2012's vstest.console. The tests now pass but Bamboo is not able to count the number of tests. This is a Bamboo problem not mine.
I found this neat way to use NUnit in Powershell. http://elegantcode.com/2009/10/25/integration-test-brought-to-you-by-powershell-nunit-with-a-little-specification-syntax-for-flavoring/
and we are using it many of our tests.
However I want to run these tests in TeamCity.
I want similar behavior when we use a NUnit runner for running C# tests in TeamCity ie the build fails when the execution of tests fail. Has anyone of you achieved this? I suspect the Powershell runner will just execute it as a simple script, without any indication whether the test passes or fails.
Take a look at http://confluence.jetbrains.net/display/TCD7/Build+Script+Interaction+with+TeamCity and http://confluence.jetbrains.net/display/TCD7/Build+Failure+Conditions
There is an issue in Powershell runner support http://youtrack.jetbrains.com/issue/TW-21554
I'm not familiar with the approach you're referencing for executing NUnit tests via Powershell in TeamCity. But, we are successfully using PSake for Powershell build scripts, including executing NUnit tests and failing the build appropriately. The same issue exists with PSake and TeamCity with exit codes, but you can get around it by specifying in TeamCity in the Script Source for the Build Step using -Command for Script execution:
import-module .\tools\psake\psake.psm1
$psake.use_exit_on_error = $true
invoke-psake build.ps1
remove-module psake
You can also integrate the Test results into the TeamCity using the Build Feature option in TeamCity Build Steps.
When I run the following test in Gallio's Icarus it passes, but when I step into it using TestDriven.NET (Test With->Debugger), it fails because the parameters are not set according to the Row attributes.
I was expecting that the method would be called once for each Row attribute applied.
What am I doing wrong? If nothing, then what do I need to do to debug these tests when they break? Should I avoid parametrized tests if they are not debuggable?
[TestFixture]
public class TestDrivenIgnoresMbUnitAttributesWhenDebugging
{
[Test]
[Row(1)]
[Row(2)]
public void SomeFunc(int x)
{
Assert.AreNotEqual(default(int), x);
}
}
Hmm... did you install TestDriven.Net BEFORE installing Gallio?
If not, then the Gallio extensions for TestDriven.Net will not be installed. In that case, TestDriven.Net might run the test in "ad-hoc" mode with default values for its parameters.
It should be pretty to tell whether this is the case. If the Gallio extensions for TestDriven.Net are installed then you'll see a "Gallio" banner message in the Output Window during test execution. If not you may see something else like "ad-hoc".
To fix the problem, reinstall Gallio. Alternately you can use the add/remove features part of the Gallio installer then ensure that the TestDriven.Net components are selected for installation (under "Test Runners").