I am unable to Collect the SonarQube coverage values. Can any one suggest me where am I missing anything here. I could see the ***.Tests.dll file is available.
Test Files
C:\AirSearchBnd\src\Fortius.Air.Search.Tests\bin\Debug\WorldVentures.Fortius.Search.Tests.dll
Errors and Failures
1) Invalid : C:\AirSearchBnd\src\Fortius.Air.Search.Tests\bin\Debug\WorldVentures.Fortius.Search.Tests.dll
File not found: C:\AirSearchBnd\src\Fortius.Air.Search.Tests\bin\Debug\WorldVentures.Fortius.Search.Tests.dll
Test Run Summary
Overall result: Failed
Test Count: 0, Passed: 0, Failed: 0, Inconclusive: 0, Skipped: 0
Start time: 2017-01-27 03:35:50Z
End time: 2017-01-27 03:35:50Z
Duration: 0.025 seconds
Results (nunit3) saved as TestResult.xml
Committing...
No results, this could be for a number of reasons. The most common reasons are:
1) missing PDBs for the assemblies that match the filter please review the
output file and refer to the Usage guide (Usage.rtf) about filters.
2) the profiler may not be registered correctly, please refer to the Usage
guide and the -register switch.
C:\AirSearchBnd>exit 0
Windows Batch command
SonarQube Scanner Begin Annalysis
SonarQube code coverage screen
Try executing the OpenCover command after the MSBuild command, but before End Analysis, as described here:
Code Coverage Results
WorldVentures.Fortius.Search.Tests.dll doesn't seem to exist when OpenCover is run, judging by the OpenCover output. In your Jenkins job, the OpenCover batch script is run before the MSBuild step. If the unit tests are built by that MSBuild step, it needs to be executed before the OpenCover batch script step.
Related
We are running SonarQube Community Edition Version 7.9.1 (build 27448) (the currently latest docker image)
I have some nUnit tests but I'm struggling to get the code coverage data imported to SonarQube.
I have successfully used SonarScanner.MSBuild (sonar-scanner-msbuild-4.6.2.2108-net46) to import the code analysis and number of nUnit tests to SonarQube using the following commands:
SonarScanner.MSBuild.exe begin /k:"ProjectName" /d:sonar.host.url="..." /d:sonar.login="..." /d:sonar.cs.nunit.reportsPaths="NUnitResults.xml"
MSBuild.exe ....sln /t:Rebuild
nunit3-console.exe --result=NUnitResults.xml path\to\tests.dll
SonarScanner.MSBuild.exe end /d:sonar.login="..."
Now I want to use OpenCover to get the code coverage info to SonarQube so I'm using the following commands
SonarScanner.MSBuild.exe begin /k:"ProjectName" /d:sonar.host.url="..." /d:sonar.login="..." /d:sonar.cs.nunit.reportsPaths="NUnitResults.xml" /d:sonar.cs.opencover.reportsPaths="opencover.xml"
MSBuild.exe ....sln /t:Rebuild
nunit3-console.exe --result=NUnitResults.xml path\to\tests.dll
OpenCover.Console.exe -output:opencover.xml -register:user -target:"nunit3-console.exe" -targetargs:"path\to\tests.dll --result=NUnitResults.xml"
SonarScanner.MSBuild.exe end /d:sonar.login="..."
[I think I'm probably unnecessarily running the nUnit tests twice, but I'll sort that out once I've resolved the following issue]
The problem is I'm not getting any code coverage details on SonarQube.
Checking the SonarScanner log I can see the following output from OpenCover:
Executing: nunit3-console.exe
NUnit Console Runner 3.10.0 (.NET 2.0)
Copyright (c) 2019 Charlie Poole, Rob Prouse
06 September 2019 09:24:53
Runtime Environment
OS Version: Microsoft Windows NT 10.0.14393.0
CLR Version: 4.0.30319.42000
Test Files
path\to\UnitTests.dll
Run Settings
DisposeRunners: True
WorkDirectory: ...
ImageRuntimeVersion: 4.0.30319
ImageTargetFrameworkName: .NETFramework,Version=v4.6.1
ImageRequiresX86: False
ImageRequiresDefaultAppDomainAssemblyResolver: False
NumberOfTestWorkers: 8
Test Run Summary
Overall result: Passed
Test Count: 35, Passed: 35, Failed: 0, Warnings: 0, Inconclusive: 0, Skipped: 0
Start time: 2019-09-06 08:24:54Z
End time: 2019-09-06 08:24:57Z
Duration: 3.111 seconds
Results (nunit3) saved as NUnitResults.xml
Committing...
Visited Classes 12 of 12 (100)
Visited Methods 41 of 41 (100)
Visited Points 240 of 240 (100)
Visited Branches 41 of 41 (100)
==== Alternative Results (includes all methods including those without corresponding source) ====
Alternative Visited Classes 12 of 12 (100)
Alternative Visited Methods 41 of 53 (77.36)
Now checking the dir I can see I have NUnitResults.xml and opencover.xml created and populated. (I don't really know how to read OpenCover results but it has data for my unit tests)
The SonarScanner.MSBuild.exe end log has this tidbit:
INFO: Sensor C# Tests Coverage Report Import [csharp]
INFO: Parsing the OpenCover report path.\opencover.xml
INFO: Adding this code coverage report to the cache for later reuse: path.\opencover.xml
WARN: The Code Coverage report doesn't contain any coverage data for the included files. For troubleshooting hints, please refer to https://docs.sonarqube.org/x/CoBh
INFO: Sensor C# Tests Coverage Report Import [csharp] (done) | time=14ms
INFO: Sensor C# Unit Test Results Import [csharp]
INFO: Parsing the NUnit Test Results file path.\NUnitResults.xml
INFO: Sensor C# Unit Test Results Import [csharp] (done) | time=34ms
Now the classes that the unit tests cover are included (I can see them in SonarQube marked as uncovered) so I guess either OpenCover isn't collecting the correct info or something is going wrong matching the covered files with the file in the solution.
n.b. Checking the project that should be covered by these tests, under Project > Build > Advanced the Debug Info is set to full for Debug configuration, which some resources seem to suggest is required.
Any idea what I'm missing/where I'm going wrong?
UPDATE
Checking the bottom of opencode.xml it has the following:
<Module skippedDueTo="MissingPdb" hash="DD-83-09-69-9F-A7-11-FF-F5-BC-43-7C-87-B2-54-99-0D-A5-D5-61">
<ModulePath>C:\Windows\Microsoft.Net\assembly\GAC_MSIL\MyCompany.Common.Utility\v4.0_1.0.0.0__bf95fa7f15863c9f\MyCompany.Common.Utility.dll</ModulePath>
<ModuleTime>2019-09-05T16:07:28.6111533Z</ModuleTime>
<ModuleName>MyCompany.Common.Utility</ModuleName>
<Classes />
</Module>
The original authors of this solution seemed to like shoving everything in the GAC, so I'm guessing OpenCover is looking in the GAC rather than locally for the dll, and the GAC doesn't hold pdbs.
I tried setting the OpenCover -targetdir setting as suggested in the manual:
The path to the target directory; if the target argument already contains a path then this argument can be used to provide an alternate path where PDB files may be found.
I also tried to use -searchdirs (Alternative locations to look for PDBs.)
but neither of them resolved it.
Can I tell OpenCover to use the bin\Debug pdbs?
https://github.com/OpenCover/opencover/wiki/Usage#notes-on-spaces-in-arguments
Using \" in targetargs to represent quotes to nunit3-console
SonarScanner.MSBuild.exe begin /k:"ProjectName" /d:sonar.host.url="..." /d:sonar.login="..." /d:sonar.cs.nunit.reportsPaths="NUnitResults.xml" /d:sonar.cs.opencover.reportsPaths="opencover.xml"
MSBuild.exe ....sln /t:Rebuild
OpenCover.Console.exe -output:opencover.xml -register:user -target:"nunit3-console.exe" -targetargs:"\"path\to\tests.dll\" --result=NUnitResults.xml"
SonarScanner.MSBuild.exe end /d:sonar.login="..."
Step1 : SonarScanner.MSBuild.exe begin /k:"Project-Name" /d:sonar.cs.opencover.reportsPaths=%cd%\TestCoverResult.xml /d:sonar.exclusions="**/*.css,**/*.js,**/*.cshtml" /d:sonar.verbose="true" /d:sonar.host.url="https://host-url.com/" /d:sonar.login="Login-Token"
Step2 : "C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Current\Bin\MSBuild.exe" /t:Rebuild
Step3 : ".\packages\OpenCover.4.7.1221\tools\OpenCover.Console.exe" "-target:.\packages\NUnit.ConsoleRunner.3.12.0\tools\nunit3-console.exe" "-targetargs:.\Test-Project\bin\Debug\Test-Project.dll" "-output:.\TestCoverResult.xml" -register:user
Step4 : SonarScanner.MSBuild.exe end
Explanation:
1 : reportsPath: specifies the path of the code-coverage result file.
2 : ".\packages\OpenCover.4.7.1221\tools\OpenCover.Console.exe" : it is system path of the "OpenCover.Console.exe"
3 : ".\packages\NUnit.ConsoleRunner.3.12.0\tools\nunit3-console.exe" : it is system path of the "nunit3-console.exe"
I am having an issue with SonarQube 6.7 where it is not displaying any C# code issues or bugs, during the analysis I can see C# is being analysed and the build is successful but when I check the SonarQube Dashboard it is only showing Typescript and XML.
I am using Jenkins to build the project using Mono as all our building is done on Ubuntu 16.04.
Setup as follows:
Jenkins 2.89.2 on Ubuntu 16.04
SonarQube Version 6.7 (build 33306) on Ubuntu 16.04 (Separate server)
Sonar C# 6.8.1
SonarQube Scanner for MSBuild 4.0.1
Mono 5.10.0 Stable
Using a Pipeline In Jenkins I am doing the following:
node {
sh 'mono ~/tools/hudson.plugins.sonar.MsBuildSQRunnerInstallation/MSBuild/SonarQube.Scanner.MSBuild.exe begin /key:12345678FAKEKEY9l02998745'
sh 'dotnet restore WSP1.sln'
sh 'dotnet msbuild WSP1.sln'
sh 'mono ~/tools/hudson.plugins.sonar.MsBuildSQRunnerInstallation/MSBuild/SonarQube.Scanner.MSBuild.exe end' } }
The above pipeline script works without any issues.
When I run this pipeline I can see that the C# code is being Analysed, this is just a snippet of code being analysed:
Services/Project1/Services/Project1EventsShould.cs(130,21): warning S1186: Add a nested comment explaining why this method is empty, throw a 'NotSupportedException' or complete the implementation. [/var/lib/jenkins/workspace/CSharpAnalysis/WSP1/test/Unit/Unit.csproj]
Services/Project1/Services/Project1EventsShould.cs(137,21): warning S1186: Add a nested comment explaining why this method is empty, throw a 'NotSupportedException' or complete the implementation. [/var/lib/jenkins/workspace/CSharpAnalysis/WSP1/test/Unit/Unit.csproj]
Services/Project1/Models/ZenEventShould.cs(24,17): warning S1481: Remove this unused 'date' local variable. [/var/lib/jenkins/workspace/CSharpAnalysis/WSP1/test/Unit/Unit.csproj]
Services/Project1/Models/CollaborationUserShould.cs(16,17): warning S1481: Remove this unused 'sut' local variable. [/var/lib/jenkins/workspace/CSharpAnalysis/WSP1/test/Unit/Unit.csproj]
At the end I get a SUCCESS, everything looks good but when I check SonarQube I can see that the uploaded code is being analysed for display in SonarQube, this usually takes a minute or so but when this is completed I can see that only TypeScript and XML is displayed.
In amongst the output I get some warnings, info and errors but nothing looks sinister, at the end of the output I get the success message
INFO: 68 files had no CPD blocks
INFO: Calculating CPD for 281 files
INFO: CPD calculation finished
INFO: Analysis report generated in 1771ms, dir size=5 MB
INFO: Analysis reports compressed in 3908ms, zip size=2 MB
INFO: Analysis report uploaded in 5866ms
INFO: ANALYSIS SUCCESSFUL, you can browse http://sonar.serverdomain.com:9000/dashboard/index/12345678FAKEKEY9l02998745
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://sonar.serverdomain.com:9000/api/ce/task?id=AKsdHc7FAKEIDVOeCzBYv
INFO: Task total time: 1:32.107 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 1:42.735s
INFO: Final Memory: 54M/397M
INFO: ------------------------------------------------------------------------
The SonarQube Scanner has finished
14:59:57.577 Creating a summary markdown file...
14:59:57.59 Analysis results: http://sonar.serverdomain.com:9000/dashboard/index/12345678FAKEKEY9l02998745
14:59:57.591 Post-processing succeeded.
Finished: SUCCESS
I have the full output handy, I just need to sanitize this to remove anything that may be sensitive, if you would like a look just let me know and I will post this.
I am pretty new to SonarQube so the issue I am having could be something silly but I have been ripping my hair out trying to figure this out.
EDIT : I was digging through the output again and noticed 2 warning in the following:
INFO: Quality profile for cs: Sonar way
INFO: Quality profile for ts: Sonar way
INFO: Quality profile for xml: Sonar way
INFO: Sensor C# Properties [csharp]
WARNING: WARN: Property missing: 'sonar.cs.analyzer.projectOutPaths'. No protobuf files will be loaded for this project.
WARNING: WARN: No roslyn issues report not found for this project.
INFO: Sensor C# Properties [csharp] (done) | time=1ms
I just ran into the same issue on Sonarqube 6.7.4 (build 38452).
Solved by two steps:
After updating the built-in SonarC# plugin to version 7.2 (build 5463) (Uninstall-Restart-Install-Restart), and re-scanning, code issues (vulnerabilities, code smells, etc) showed up.
Introduced package coverlet.msbuild into test projects to generate coverage data, configured Project - Administration - C# - OpenCover Unit Tests Reports Paths to use path/to/coverage.opencover.xml. After re-scanning, test coverage data showed up.
Here are all the commands executed after above configurations:
dotnet path/to/SonarScanner.MSBuild.dll begin /k:"project-key"
dotnet build
dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=opencover
dotnet path/to/SonarScanner.MSBuild.dll end
We have a C# solution, and a CI solution with jenkins.
Every night, jenkins clears it workspace, get the latest version, build it in debug, run the tests through OpenCover.
We have the following builds steps(I'm not mentionning the steps before):
SonarQube Begin
First things we do in jenkins is the "SonarQube Scanner for MSBuild - Begin Analysis", we specifiy the project key/name and the additional arguments:
/d:sonar.cs.nunit.reportsPaths="%CD%\TestResult.xml"
/d:sonar.cs.opencover.reportsPaths="%CD%\opencover.xml"
Build
We use the MSBUILD14 to build or solution in debug
Execute tests through OpenCover
We use openCover to check the code coverage and execute the tests:
"C:\Program Files (x86)\OpenCover\OpenCover.Console.exe" -filter:"+[*]* -[*.Test]*" -output:"%CD%\opencover.xml" -threshold:1 -register:User -target:"C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" -targetargs:"Solution\MySolution.sln --config=Debug --result=%CD%\TestResult.xml;format=nunit2"
exit 0
SonarQube end Analysis
This should publish the results of the SonarQube Results
PostBuild: Publish NUnit test result report
I've an additional steps that should publish the TestResult.xml with the following argument:
%CD%\TestResult.xml
Once the whole job is done, I've in sonarqube, the static analysis result, but I DON'T have the Unit tests results and the code coverage results.
I've checked the logs, the tests have been executed:
Test Run Summary
Overall result: Failed
Test Count: 4331, Passed: 3819, Failed: 189, Warnings: 0, Inconclusive: 0, Skipped: 323
Failed Tests - Failures: 39, Errors: 147, Invalid: 3
Skipped Tests - Ignored: 323, Explicit: 0, Other: 0
Start time: 2017-02-07 10:03:42Z
End time: 2017-02-07 11:51:56Z
Duration: 6494.182 seconds
I only have one weird exception:
An System.IO.DirectoryNotFoundException occured: Could not find a part of the path 'c:\Projects\CommonServiceLocator\main\Microsoft.Practices.ServiceLocation\ActivationException.cs'.
An System.IO.DirectoryNotFoundException occured: Could not find a part of the path 'c:\Projects\CommonServiceLocator\main\Microsoft.Practices.ServiceLocation\ServiceLocator.cs'.
An System.IO.DirectoryNotFoundException occured: Could not find a part of the path 'c:\Projects\CommonServiceLocator\main\Microsoft.Practices.ServiceLocation\ServiceLocatorImplBase.cs'.
An System.IO.DirectoryNotFoundException occured: Could not find a part of the path 'c:\Projects\CommonServiceLocator\main\Microsoft.Practices.ServiceLocation\Properties\Resources.Designer.cs'.
Which happens at the end of openCover step. The 2 XML files are still generated, and I also have this exception locally.
Any idea why I don't have the code coverage/unit tests results in sonarQube?
Thank you!
I have an issue getting SonarQube to pickup the unittest results.
The setup is as following:
TFS2015 Update 3
New SonarQube Build Tasks (released a few days ago)
SonarQube 6.2
C# Plugin 5.5.1.522
From the verboose log it appears that the trx files are identified, but I can't find anything about those beeing processed, even with verbose logging.
2016-12-16T13:26:54.5935752Z 14:26:54.516 Looking for TRX files in: E:\TFS_Build\Agent01\Work\49\TestResults
2016-12-16T13:26:54.5945826Z 14:26:54.518 Located a test results file: E:\TFS_Build\Agent01\Work\49\TestResults\tap_sysadmin_TFS-BUILD1 2016-12-16 14_26_32.trx
The trx file does contain test results.
Any idea how to get this to work?
I'm re-using reports in my sonar config:
sonar.gallio.mode=reuseReport
sonar.gallio.reports.path=gallio-report.xml
sonar.gallio.coverage.reports.path=results.xml
I've previously run Gallio and OpenCover and can confirm that both completed successfully and that Sonar is able to retrieve the files (I've checked the log produced by the -X flag thoroughly). When I view the project in the sonar dashboard, I see code coverage, but not test results:
Kindly ignore the low code coverage percentage, I'm running a small subset of tests while I figure this out.
I should see something that reflects the results I saw when I ran Gallio:
14 run, 13 passed, 1 failed (1 error), 0 inconclusive, 0 skipped
I'm happy to include the gallio-report.xml if that's helpful, but it's 103kb so clearly it contains plenty of data, and I think this is more likely to be a configuration issue.
I'm running OpenCover 4.0.1118 and Gallio 3.2.750 (tests are written with NUnit).
Any thoughts why I don't see any test results?
Chances are that you don't have the test sources in your .NET solution, so when SonarQube tries to import the test execution results, it can't find to which files they should be attached.
In the .NET sample solution, you can see that there is a test project (Example.Core.Tests) which contains the sources of the test classes.