Generating TestResult with nunit-console failing - c#

According to the SpecFlow website, I can generate the TestResult by executing the following statement:
nunit3-console.exe --labels=All --out=TestResult.txt "--result=TestResult.xml;format=nunit2" bin\Debug\BookShop.AcceptanceTests.dll
I've adapted this statement to the following:
nunit3-console.exe --labels=All --out=C:\temp\TestResult.txt "--result=C:\temp\TestResult.xml;format=nunit2" C:\Projects\DataService.IntegrationTests\bin\Debug\DataService.IntegrationTests.dll
Unfortunately, I get the following errors:
Errors, Failures and Warnings
1) Invalid : C:\Projects\DataService.IntegrationTests\bin\Debug\DataService.IntegrationTests.dll
No suitable tests found in 'C:\Projects\DataService.IntegrationTests\bin\Debug.DataService.IntegrationTests.dll'.
Either assembly contains no tests or proper test driver has not been found.
The .feature file are available in this assembly...
What could be the problem here?
An assumption of mine is, that we're using Specflow.MsTest...
Is there a way to generate the TestResult.xml by using MsTest?
Thanks in advance

NUnit3-console can only run NUnit tests. (Well technically, it can run any tests for which a driver is provided, but it amounts to the same thing.)
If you want to create NUnit output, then use both the NUnit framework and the NUnit console runner. MsTest doesn't do NUnit output.

In app.condig I don't have any unitTestProvider
<specFlow>
<stepAssemblies>
<stepAssembly assembly="otherProject" />
</stepAssemblies>
<runtime detectAmbiguousMatches="true" stopAtFirstError="false" missingOrPendingStepsOutcome="Inconclusive" />
<trace traceSuccessfulSteps="true" traceTimings="false" minTracedDuration="0:0:0.1" />
<!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
</specFlow>

Related

xUnit does not produce results file with lots of tests

Issue
When I run xUnit tests on a project containing a lot of tests, it does not produce the xml result file. If I reduce the number of tests in the same project to a smaller set of tests it does produce the result file. Why is this happening?
Details
My project has about 3200 xUnit tests running in a Jenkins build. I am using the following command to run the tests.
dotnet test --logger:"xunit;LogFilePath=/results/integrationtest_results.xml"
However, this is not creating the integrationtest_results.xml file. I have confirmed the tests did run even though it did not produce the result file.
When I use the following command to filter down the tests to less than 1000 tests it DOES create the integrationtest_results.xml result file.
dotnet test --logger:"xunit;LogFilePath=/results/integrationtest_results.xml" --filter:"someAttribute=someValue"
In the same Jenkins build, I have another project that runs 500 unit tests with the following command.
dotnet test --logger:"xunit;LogFilePath=/results/unittest_results.xml"
This has always produced the expected unittest_results.xml.
In the cases where it does not produce the result file, I do not see any logs stating an error occurred relating to the xunit results file.
Environment
C# dotnet core 6.0
Xunit v2.4.1
Test logger: XunitXml.TestLogger v3.0.70

VS2017 and NUnit 3.9 No test is available

I am using the latest VS2017 version 15.6.4, NUnit3TestAdapter 3.10.0 and Nunit version 3.9.0.0.
When I try to run a unit test in Test Explorer the test are grayed out, when I right click and and run selected tests I see the following error: No test is available
Here is how my test class looks
[TestFixture]
public partial class ListViewBOTest
{
[Test]
public void TestSearch_DateTime()
{
Assert.AreEqual(1,0);
}
}
Text from output:
[3/26/2018 10:53:55 AM Informational] ------ Run test started ------
[3/26/2018 10:53:55 AM Informational] NUnit Adapter 3.10.0.21: Test execution started
[3/26/2018 10:53:55 AM Informational] Running all tests in C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll
[3/26/2018 10:53:56 AM Informational] NUnit failed to load C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll
[3/26/2018 10:53:56 AM Informational] NUnit Adapter 3.10.0.21: Test execution complete
[3/26/2018 10:53:56 AM Warning] No test is available in C:\Projects\MVPPlant\DEV\CMMSdg.Plant\CMMSdg.Plant\Sln.2010\CMMSdg.Plant.BusinessObjects.Test\bin\Debug\CMMSdg.Plant.BusinessObjects.Test.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
[3/26/2018 10:53:56 AM Informational] ========== Run test finished: 0 run (0:00:01.0435303) ==========
Make sure you have installed the NUnit3 Test Adapter from here
https://marketplace.visualstudio.com/items?itemName=NUnitDevelopers.NUnit3TestAdapter
Can you check the following steps and see if it works?
In the Visual Studio menu, go to Test > Test Settings > Default Processor Architecture and make a note if X86 is selected or X64
Now go to the Build section in the Properties window of the project where the tests are written. Make sure the Platform target drop-down is selected to either Any CPU or at least it matches the architecture from the above step 1.
Now if you build the solution and try running those tests, you should see that they are now running.
I had the same problem as Amete Blessed and commenting out other Test methods made Test Explorer work and run my test
I found that my Build Events were wrong. Invalid copy command blew all my tests and half a day:
Copy C:\repo\Architecture\*.json $(ProjectPath)/Y
Copy C:\repo\Architecture\*.json $(TargetPath) /Y
instead of
Copy C:\repo\Architecture\*.json $(ProjectDir)/Y
Copy C:\repo\Architecture\*.json $(TargetDir) /Y
The messages in the Test Output window ("NUnit failed to load [assembly]", "No test is available...", etc.) can hide the underlying issue that's causing the runner to not load the test assembly. This includes hiding failures to load dependencies of the test assembly or the item under test.
If there's a test assembly that's showing up in the Test Explorer window, but the tests refuse to run, it's worth temporarily enabling fusion logging to see if any assembly binding errors occur when trying to run the tests.
After installing NUnit through nuget tests have appeared into Test Explorer
but when I ran them I got "No test is available".
Installing NUnit Test Adapter fixed the issue.
I had similar problem when using Xamarin.Forms.
The solution was to install NUnit.XForms from NuGet and add
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0" />
to csproj file of the project where the tests are.
What worked for me was to delete the ComponentModelCache folder located at %localappdata%\Microsoft\VisualStudio\<version>
You may have been a casualty of this problem that was (theoretically) resolved in 15.6.3. According to this answer, try deleting your %temp%\VisualStudioTestExplorerExtensions folder. That has resolved the issue for some other users.
Rather than using the Test Explorer, can you right click on the solution and Run Unit Tests from there?
Had the same problem. In my case I found that the NUnit test adapter will not be used by Test Explorer if your test project contains a reference to MSTest. Typically the Microsoft.VisualStudio.TestPlatform.TestFramework.dll but also check your .csproj file for "MSTest" and your packages.config file.
I discovered this by first enabling diagnostic logging for Visual Studio tests. This is found under "Tools" -> "Options" -> "Tests" -> "Logging Level".
In my log I found this entry:
[22/11/2018 10:36:42 Diagnostic] Project C:\Git\myProject\src\myProject.Tests\myProject.Tests.csproj references test adapter: MSTest.TestAdapter, version 1.1.18
This is caused due to memory problem.
Clean object after test execution
Sample Code
private TestController testController;
[OneTimeSetUp]
public void TestSetup()
{
testController= new TestController();
}
[OneTimeTearDown]
public void TestCleanup()
{
testController= null;
}
Change test execution to 64 bits in settings
Test -> Test Settings -> Default Processors Architecture -> x64.
I have also encountered the same issue.
Steps to resolve this issue -
1. Install/Reinstall NUnit3TestAdapter package
2. Delete Debug folder from the Bin
3. Clean --> Build the project
Note - Also ensure all the packages are installed properly.
Now try to execute the tests.
I had the same issue, but it was fixed after installing "NUnit 3 TestAdapter version 3.16.1" via NuGet to my project.

NUnit with Specflow: SetUpAttribute attribute not allowed in a SetUpFixture

I upgraded my Nunit to 3.0 from 2.6 and now all of my Unit Tests are failing with the following error:
OneTimeSetUp: SetUpAttribute attribute not allowed in a SetUpFixture
I read through the NUnit documentation and found that there are some "breaking changes" with the new release and I can no longer use a "SetUpAttribute" inside a "SetUpFixture".
I will have to replace this with "OneTimeSetUpAttribute" and I would but my MAIN problem is that my Nunit test fixtures are generated by Specflow.
I have specflow 2.1 and this is generating test fixtures with default "OneTimeSetUpAttribute" regardless of Nunit version.
Did anyone else come across this problem and have a solution for this?
Thanks in advance.

Running automated Powershell-NUnit tests on TeamCity

I found this neat way to use NUnit in Powershell. http://elegantcode.com/2009/10/25/integration-test-brought-to-you-by-powershell-nunit-with-a-little-specification-syntax-for-flavoring/
and we are using it many of our tests.
However I want to run these tests in TeamCity.
I want similar behavior when we use a NUnit runner for running C# tests in TeamCity ie the build fails when the execution of tests fail. Has anyone of you achieved this? I suspect the Powershell runner will just execute it as a simple script, without any indication whether the test passes or fails.
Take a look at http://confluence.jetbrains.net/display/TCD7/Build+Script+Interaction+with+TeamCity and http://confluence.jetbrains.net/display/TCD7/Build+Failure+Conditions
There is an issue in Powershell runner support http://youtrack.jetbrains.com/issue/TW-21554
I'm not familiar with the approach you're referencing for executing NUnit tests via Powershell in TeamCity. But, we are successfully using PSake for Powershell build scripts, including executing NUnit tests and failing the build appropriately. The same issue exists with PSake and TeamCity with exit codes, but you can get around it by specifying in TeamCity in the Script Source for the Build Step using -Command for Script execution:
import-module .\tools\psake\psake.psm1
$psake.use_exit_on_error = $true
invoke-psake build.ps1
remove-module psake
You can also integrate the Test results into the TeamCity using the Build Feature option in TeamCity Build Steps.

Determining currently executing test case by Nunit

I am using Cruisecontrol for building code and executing ~200 test cases.
Sometimes the build gets hung due to some of the test cases and I am not able to determine which test case it is.
Is it possible to print the name of the currently executing test case without modifying the current test case code?
If yes, how?
I'm assuming you are using NUnit to do your tests. If so, instead of using the <nunit> block in the CruiseControl config, use the <exec> task. In the <buildArgs> element, include the /labels command line argument. This will print the info to the server log.
Instead of using:
<nunit>
<path>C:\Program Files (x86)\NUnit 2.5.10\bin\net-2.0\nunit-console.exe</path>
<assemblies>
<assembly>C:\Projects\Personal\MyTestApp\MyTestApp.Tests\bin\Debug\MyTestApp.Tests.dll</assembly>
</assemblies>
</nunit>
Use:
<exec>
<executable>C:\Program Files (x86)\NUnit 2.5.10\bin\net-2.0\nunit-console.exe</executable>
<buildArgs>/labels C:\Projects\Personal\MyTestApp\MyTestApp.Tests\bin\Debug\MyTestApp.Tests.dll</buildArgs>
</exec>
I know it isn't ideal, but it will print each test as it runs to the log. You can then use the merge task to merge the xml file output by nunit into your build log.
Try doing this using the ccnet console application first, so you can see the output in real time. It should help you see what you are looking for.
It might also be good to submit a patch to CruiseControl to add the following line to the "project\core\tasks\NUnitArgument.cs" file:
line: 53 argsBuilder.AddArgument("/labels");
Or you could just add that line, build CruiseControl and use your own version.
TestContext.CurrentContext.Test.Name will help in identifying the currently executing test case. Putting
Console.WriteLine("Currently Executing Test Case : " + TestContext.CurrentContext.Test.Name);
in method with SetUp attribute will print the UT currently executing.

Categories