We Keep Coding

How to import nUnit/OpenCover results to SonarQube

We are running SonarQube Community Edition Version 7.9.1 (build 27448) (the currently latest docker image)
I have some nUnit tests but I'm struggling to get the code coverage data imported to SonarQube.
I have successfully used SonarScanner.MSBuild (sonar-scanner-msbuild-4.6.2.2108-net46) to import the code analysis and number of nUnit tests to SonarQube using the following commands:
SonarScanner.MSBuild.exe begin /k:"ProjectName" /d:sonar.host.url="..." /d:sonar.login="..." /d:sonar.cs.nunit.reportsPaths="NUnitResults.xml"
MSBuild.exe ....sln /t:Rebuild
nunit3-console.exe --result=NUnitResults.xml path\to\tests.dll
SonarScanner.MSBuild.exe end /d:sonar.login="..."
Now I want to use OpenCover to get the code coverage info to SonarQube so I'm using the following commands
SonarScanner.MSBuild.exe begin /k:"ProjectName" /d:sonar.host.url="..." /d:sonar.login="..." /d:sonar.cs.nunit.reportsPaths="NUnitResults.xml" /d:sonar.cs.opencover.reportsPaths="opencover.xml"
MSBuild.exe ....sln /t:Rebuild
nunit3-console.exe --result=NUnitResults.xml path\to\tests.dll
OpenCover.Console.exe -output:opencover.xml -register:user -target:"nunit3-console.exe" -targetargs:"path\to\tests.dll --result=NUnitResults.xml"
SonarScanner.MSBuild.exe end /d:sonar.login="..."
[I think I'm probably unnecessarily running the nUnit tests twice, but I'll sort that out once I've resolved the following issue]
The problem is I'm not getting any code coverage details on SonarQube.
Checking the SonarScanner log I can see the following output from OpenCover:
Executing: nunit3-console.exe
NUnit Console Runner 3.10.0 (.NET 2.0)
Copyright (c) 2019 Charlie Poole, Rob Prouse
06 September 2019 09:24:53
Runtime Environment
OS Version: Microsoft Windows NT 10.0.14393.0
CLR Version: 4.0.30319.42000
Test Files
path\to\UnitTests.dll
Run Settings
DisposeRunners: True
WorkDirectory: ...
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETFramework,Version=v4.6.1
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False
NumberOfTestWorkers: 8
Test Run Summary
Overall result: Passed
Test Count: 35, Passed: 35, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
Start time: 2019-09-06 08:24:54Z
End time: 2019-09-06 08:24:57Z
Duration: 3.111 seconds
Results (nunit3) saved as NUnitResults.xml
Committing...
Visited Classes 12 of 12 (100)
Visited Methods 41 of 41 (100)
Visited Points 240 of 240 (100)
Visited Branches 41 of 41 (100)
==== Alternative Results (includes all methods including those without corresponding source) ====
Alternative Visited Classes 12 of 12 (100)
Alternative Visited Methods 41 of 53 (77.36)
Now checking the dir I can see I have NUnitResults.xml and opencover.xml created and populated. (I don't really know how to read OpenCover results but it has data for my unit tests)
The SonarScanner.MSBuild.exe end log has this tidbit:
INFO: Sensor C# Tests Coverage Report Import [csharp]
INFO: Parsing the OpenCover report path.\opencover.xml
INFO: Adding this code coverage report to the cache for later reuse: path.\opencover.xml
WARN: The Code Coverage report doesn't contain any coverage data for the included files. For troubleshooting hints, please refer to https://docs.sonarqube.org/x/CoBh
INFO: Sensor C# Tests Coverage Report Import [csharp] (done) | time=14ms
INFO: Sensor C# Unit Test Results Import [csharp]
INFO: Parsing the NUnit Test Results file path.\NUnitResults.xml
INFO: Sensor C# Unit Test Results Import [csharp] (done) | time=34ms
Now the classes that the unit tests cover are included (I can see them in SonarQube marked as uncovered) so I guess either OpenCover isn't collecting the correct info or something is going wrong matching the covered files with the file in the solution.
n.b. Checking the project that should be covered by these tests, under Project > Build > Advanced the Debug Info is set to full for Debug configuration, which some resources seem to suggest is required.
Any idea what I'm missing/where I'm going wrong?
UPDATE
Checking the bottom of opencode.xml it has the following:
<Module skippedDueTo="MissingPdb" hash="DD-83-09-69-9F-A7-11-FF-F5-BC-43-7C-87-B2-54-99-0D-A5-D5-61">
<ModulePath>C:\Windows\Microsoft.Net\assembly\GAC_MSIL\MyCompany.Common.Utility\v4.0_1.0.0.0__bf95fa7f15863c9f\MyCompany.Common.Utility.dll</ModulePath>
<ModuleTime>2019-09-05T16:07:28.6111533Z</ModuleTime>
<ModuleName>MyCompany.Common.Utility</ModuleName>
<Classes />
</Module>
The original authors of this solution seemed to like shoving everything in the GAC, so I'm guessing OpenCover is looking in the GAC rather than locally for the dll, and the GAC doesn't hold pdbs.
I tried setting the OpenCover -targetdir setting as suggested in the manual:
The path to the target directory; if the target argument already contains a path then this argument can be used to provide an alternate path where PDB files may be found.
I also tried to use -searchdirs (Alternative locations to look for PDBs.)
but neither of them resolved it.
Can I tell OpenCover to use the bin\Debug pdbs?

https://github.com/OpenCover/opencover/wiki/Usage#notes-on-spaces-in-arguments
Using \" in targetargs to represent quotes to nunit3-console
SonarScanner.MSBuild.exe begin /k:"ProjectName" /d:sonar.host.url="..." /d:sonar.login="..." /d:sonar.cs.nunit.reportsPaths="NUnitResults.xml" /d:sonar.cs.opencover.reportsPaths="opencover.xml"
MSBuild.exe ....sln /t:Rebuild
OpenCover.Console.exe -output:opencover.xml -register:user -target:"nunit3-console.exe" -targetargs:"\"path\to\tests.dll\" --result=NUnitResults.xml"
SonarScanner.MSBuild.exe end /d:sonar.login="..."

Step1 : SonarScanner.MSBuild.exe begin /k:"Project-Name" /d:sonar.cs.opencover.reportsPaths=%cd%\TestCoverResult.xml /d:sonar.exclusions="**/*.css,**/*.js,**/*.cshtml" /d:sonar.verbose="true" /d:sonar.host.url="https://host-url.com/" /d:sonar.login="Login-Token"
Step2 : "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\MSBuild.exe" /t:Rebuild
Step3 : ".\packages\OpenCover.4.7.1221\tools\OpenCover.Console.exe" "-target:.\packages\NUnit.ConsoleRunner.3.12.0\tools\nunit3-console.exe" "-targetargs:.\Test-Project\bin\Debug\Test-Project.dll" "-output:.\TestCoverResult.xml" -register:user
Step4 : SonarScanner.MSBuild.exe end
Explanation:
1 : reportsPath: specifies the path of the code-coverage result file.
2 : ".\packages\OpenCover.4.7.1221\tools\OpenCover.Console.exe" : it is system path of the "OpenCover.Console.exe"
3 : ".\packages\NUnit.ConsoleRunner.3.12.0\tools\nunit3-console.exe" : it is system path of the "nunit3-console.exe"

Unable to see code coverage in SonarQube

I am unable to Collect the SonarQube coverage values. Can any one suggest me where am I missing anything here. I could see the ***.Tests.dll file is available.
Test Files
C:\AirSearchBnd\src\Fortius.Air.Search.Tests\bin\Debug\WorldVentures.Fortius.Search.Tests.dll
Errors and Failures
1) Invalid : C:\AirSearchBnd\src\Fortius.Air.Search.Tests\bin\Debug\WorldVentures.Fortius.Search.Tests.dll
File not found: C:\AirSearchBnd\src\Fortius.Air.Search.Tests\bin\Debug\WorldVentures.Fortius.Search.Tests.dll
Test Run Summary
Overall result: Failed
Test Count: 0, Passed: 0, Failed: 0, Inconclusive: 0, Skipped: 0
Start time: 2017-01-27 03:35:50Z
End time: 2017-01-27 03:35:50Z
Duration: 0.025 seconds
Results (nunit3) saved as TestResult.xml
Committing...
No results, this could be for a number of reasons. The most common reasons are:
1) missing PDBs for the assemblies that match the filter please review the
output file and refer to the Usage guide (Usage.rtf) about filters.
2) the profiler may not be registered correctly, please refer to the Usage
guide and the -register switch.
C:\AirSearchBnd>exit 0
Windows Batch command
SonarQube Scanner Begin Annalysis
SonarQube code coverage screen

Try executing the OpenCover command after the MSBuild command, but before End Analysis, as described here:
Code Coverage Results

WorldVentures.Fortius.Search.Tests.dll doesn't seem to exist when OpenCover is run, judging by the OpenCover output. In your Jenkins job, the OpenCover batch script is run before the MSBuild step. If the unit tests are built by that MSBuild step, it needs to be executed before the OpenCover batch script step.

VSTest unit test results are not showing in SonarQube

I have an issue getting SonarQube to pickup the unittest results.
The setup is as following:
TFS2015 Update 3
New SonarQube Build Tasks (released a few days ago)
SonarQube 6.2
C# Plugin 5.5.1.522
From the verboose log it appears that the trx files are identified, but I can't find anything about those beeing processed, even with verbose logging.
2016-12-16T13:26:54.5935752Z 14:26:54.516 Looking for TRX files in: E:\TFS_Build\Agent01\Work\49\TestResults
2016-12-16T13:26:54.5945826Z 14:26:54.518 Located a test results file: E:\TFS_Build\Agent01\Work\49\TestResults\tap_sysadmin_TFS-BUILD1 2016-12-16 14_26_32.trx
The trx file does contain test results.
Any idea how to get this to work?

dotnet test - exit code 0 when test project doesn't compile

I'm using TeamCity to build my .NET Core project and dotnet test with xunit to run tests.
My build is configured to fail if any tests fail, and this works fine if a test fails, but when the test project doesn't compile the exit code ends up being zero.
I believe this is because I run dotnet test in the folder that contains the test-directories like this: for /f %%%a in ('dir /b /s project.json') do dotnet test %%%a.
Looking at the log I can see the individual jobs that do not compile return with exit code 1, but the build step itself returns with exit code 0.
How can I make the exit code from the failed compilation attempts propagate down to the build step?

You may create a separate step in TC to just compile target test project, and then run tests without compilation (using no-build option) in next step (if all previous steps were successful).
--no-build
Does not build the test project prior to running it.

Getting code coverage in Sonar but no test results (.Net)

I'm re-using reports in my sonar config:
sonar.gallio.mode=reuseReport
sonar.gallio.reports.path=gallio-report.xml
sonar.gallio.coverage.reports.path=results.xml
I've previously run Gallio and OpenCover and can confirm that both completed successfully and that Sonar is able to retrieve the files (I've checked the log produced by the -X flag thoroughly). When I view the project in the sonar dashboard, I see code coverage, but not test results:
Kindly ignore the low code coverage percentage, I'm running a small subset of tests while I figure this out.
I should see something that reflects the results I saw when I ran Gallio:
14 run, 13 passed, 1 failed (1 error), 0 inconclusive, 0 skipped
I'm happy to include the gallio-report.xml if that's helpful, but it's 103kb so clearly it contains plenty of data, and I think this is more likely to be a configuration issue.
I'm running OpenCover 4.0.1118 and Gallio 3.2.750 (tests are written with NUnit).
Any thoughts why I don't see any test results?

Chances are that you don't have the test sources in your .NET solution, so when SonarQube tries to import the test execution results, it can't find to which files they should be attached.
In the .NET sample solution, you can see that there is a test project (Example.Core.Tests) which contains the sources of the test classes.