We Keep Coding

Consuming files in UnitTests running in Azure DevOps Pipeline

I am running a collection of unit tests in an Azure DevOps Pipeline. The unit testing framework in use is MSTest.
Some unit tests load files that are inside the repo. Therefore the Property Copy to Output Directory is set to Copy Always.
Down below you can see the property being set inside the csproj file.
<Project Sdk="Microsoft.NET.Sdk">
<!-- shortened csproj file -->
<ItemGroup>
<None Update="TestFiles\sample.xml">
<CopyToOutputDirectory>Always</CopyToOutputDirectory>
</None>
</ItemGroup>
</Project>
Test Method
Inside the test, the file gets loaded as seen in the next code block.
var path = Path.Join("TestFiles", "sample.xml");
var fileStream = File.OpenRead(path);
Local Run
I set the verbosity to detailed in order to see if the file gets copied or not.
Running dotnet test .\Project\Project.csproj --logger trx --verbosity detailed locally, everything works as expected with the test results.
Same applies for running the tests in Visual Studio.
Pipeline
I execute the tests inside an Azure DevOps Pipeline with the DotNetCoreCLI task.
trigger:
- none
pool:
vmImage: ubuntu-latest
steps:
- task: DotNetCoreCLI#2
displayName: Run Unit Tests
inputs:
command: test
projects: Project/Project.csproj
arguments: --verbosity detailed
I extracted the generated command from the pipeline logs: /usr/bin/dotnet test /home/vsts/work/1/s/Project/Project.csproj --logger trx --results-directory /home/vsts/work/_temp --verbosity detailed. I don't see any significant differences between my local command compared to the one being generated by the pipeline.
MSBuild
Indeed inspecting the local and Azure DevOps Pipeline logs, the file gets copied as expected.
Copying file from "/home/vsts/work/1/s/Project/TestFiles/sample.xml" to "/home/vsts/work/1/s/Project/bin/Debug/net6.0/TestFiles/sample.xml".
Test Error
The path matches the one from the MSBuild log /home/vsts/work/1/s/Project/bin/Debug/net6.0/TestFiles/sample.xml. Then again, I get a FileNotFoundException:
System.IO.FileNotFoundException: Could not find file '/home/vsts/work/1/s/Project/bin/Debug/net6.0/TestFiles/sample.xml'.
Pipeline debugging
I tried to list all files inside the output directory adding the continueOnError key to the test task.
# ...
tasks:
# ...
- script: ls -la /home/vsts/work/1/s/Project/bin/Debug/net6.0
- script: ls -la /home/vsts/work/1/s/Project/bin/Debug/net6.0/TestFiles
The first script action already fails with the following error message:
ls: cannot access '/home/vsts/work/1/s/Project/bin/Debug/net6.0': No such file or directory
At this point I'm just lost and don't know what to do about this seemingly simple problem... Any suggestions how I get the files containing test data to the unit test? Where is my mistake?
Update 1
I tried to switch to xUnit. But it gave me the same FileNotFoundException as before. I don't think it's a problem with the test itself but with the Azure DevOps Pipeline environment.

After some further debugging of the pipeline, I found a mistake I caused myself.
When debugging the pipeline I used the - script: ls -la /home/vsts/work/1/s/Project/... task. The path I specified was the one from this post (which I abstracted from the real thing cuz its from work yk) NOT the actual one of my test project.
Listing a non existing directory does not work for obvious reasons...
Comparing the paths from the MSBuild log message and the actual test run showed that they were almost identical. The actual file had a lowercase extension and the file name inside the test project was uppercase.
Running the pipeline in the ubuntu-latest environment means that we are working with a case sensitive file system (unlike on my local machine which is a Windows 11 machine).
tl;dr - Everything was correct, I just got to match the actual file name and file path inside the consuming test 100% correctly or run it in a windows environment.

SonarQube - C# Code not being displayed on Dashboard

I am having an issue with SonarQube 6.7 where it is not displaying any C# code issues or bugs, during the analysis I can see C# is being analysed and the build is successful but when I check the SonarQube Dashboard it is only showing Typescript and XML.
I am using Jenkins to build the project using Mono as all our building is done on Ubuntu 16.04.
Setup as follows:
Jenkins 2.89.2 on Ubuntu 16.04
SonarQube Version 6.7 (build 33306) on Ubuntu 16.04 (Separate server)
Sonar C# 6.8.1
SonarQube Scanner for MSBuild 4.0.1
Mono 5.10.0 Stable
Using a Pipeline In Jenkins I am doing the following:
node {
sh 'mono ~/tools/hudson.plugins.sonar.MsBuildSQRunnerInstallation/MSBuild/SonarQube.Scanner.MSBuild.exe begin /key:12345678FAKEKEY9l02998745'
sh 'dotnet restore WSP1.sln'
sh 'dotnet msbuild WSP1.sln'
sh 'mono ~/tools/hudson.plugins.sonar.MsBuildSQRunnerInstallation/MSBuild/SonarQube.Scanner.MSBuild.exe end' } }
The above pipeline script works without any issues.
When I run this pipeline I can see that the C# code is being Analysed, this is just a snippet of code being analysed:
Services/Project1/Services/Project1EventsShould.cs(130,21): warning S1186: Add a nested comment explaining why this method is empty, throw a 'NotSupportedException' or complete the implementation. [/var/lib/jenkins/workspace/CSharpAnalysis/WSP1/test/Unit/Unit.csproj]
Services/Project1/Services/Project1EventsShould.cs(137,21): warning S1186: Add a nested comment explaining why this method is empty, throw a 'NotSupportedException' or complete the implementation. [/var/lib/jenkins/workspace/CSharpAnalysis/WSP1/test/Unit/Unit.csproj]
Services/Project1/Models/ZenEventShould.cs(24,17): warning S1481: Remove this unused 'date' local variable. [/var/lib/jenkins/workspace/CSharpAnalysis/WSP1/test/Unit/Unit.csproj]
Services/Project1/Models/CollaborationUserShould.cs(16,17): warning S1481: Remove this unused 'sut' local variable. [/var/lib/jenkins/workspace/CSharpAnalysis/WSP1/test/Unit/Unit.csproj]
At the end I get a SUCCESS, everything looks good but when I check SonarQube I can see that the uploaded code is being analysed for display in SonarQube, this usually takes a minute or so but when this is completed I can see that only TypeScript and XML is displayed.
In amongst the output I get some warnings, info and errors but nothing looks sinister, at the end of the output I get the success message
INFO: 68 files had no CPD blocks
INFO: Calculating CPD for 281 files
INFO: CPD calculation finished
INFO: Analysis report generated in 1771ms, dir size=5 MB
INFO: Analysis reports compressed in 3908ms, zip size=2 MB
INFO: Analysis report uploaded in 5866ms
INFO: ANALYSIS SUCCESSFUL, you can browse http://sonar.serverdomain.com:9000/dashboard/index/12345678FAKEKEY9l02998745
INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
INFO: More about the report processing at http://sonar.serverdomain.com:9000/api/ce/task?id=AKsdHc7FAKEIDVOeCzBYv
INFO: Task total time: 1:32.107 s
INFO: ------------------------------------------------------------------------
INFO: EXECUTION SUCCESS
INFO: ------------------------------------------------------------------------
INFO: Total time: 1:42.735s
INFO: Final Memory: 54M/397M
INFO: ------------------------------------------------------------------------
The SonarQube Scanner has finished
14:59:57.577 Creating a summary markdown file...
14:59:57.59 Analysis results: http://sonar.serverdomain.com:9000/dashboard/index/12345678FAKEKEY9l02998745
14:59:57.591 Post-processing succeeded.
Finished: SUCCESS
I have the full output handy, I just need to sanitize this to remove anything that may be sensitive, if you would like a look just let me know and I will post this.
I am pretty new to SonarQube so the issue I am having could be something silly but I have been ripping my hair out trying to figure this out.
EDIT : I was digging through the output again and noticed 2 warning in the following:
INFO: Quality profile for cs: Sonar way
INFO: Quality profile for ts: Sonar way
INFO: Quality profile for xml: Sonar way
INFO: Sensor C# Properties [csharp]
WARNING: WARN: Property missing: 'sonar.cs.analyzer.projectOutPaths'. No protobuf files will be loaded for this project.
WARNING: WARN: No roslyn issues report not found for this project.
INFO: Sensor C# Properties [csharp] (done) | time=1ms

I just ran into the same issue on Sonarqube 6.7.4 (build 38452).
Solved by two steps:
After updating the built-in SonarC# plugin to version 7.2 (build 5463) (Uninstall-Restart-Install-Restart), and re-scanning, code issues (vulnerabilities, code smells, etc) showed up.
Introduced package coverlet.msbuild into test projects to generate coverage data, configured Project - Administration - C# - OpenCover Unit Tests Reports Paths to use path/to/coverage.opencover.xml. After re-scanning, test coverage data showed up.
Here are all the commands executed after above configurations:
dotnet path/to/SonarScanner.MSBuild.dll begin /k:"project-key"
dotnet build
dotnet test /p:CollectCoverage=true /p:CoverletOutputFormat=opencover
dotnet path/to/SonarScanner.MSBuild.dll end

Mstest Pass/Fail report in sonarqube

I have a solution with mstest unit testcases and using Opencover for testcoverage.
TestCoverage results are coming fine in Sonarqube but it is not displaying the result from generated TestResult.trx file in Sonar. It only displays the unit test cases count but not testcases names, which one is pass and which one is fail.
I have followed the same steps mentioned in the link below:
https://docs.sonarqube.org/pages/viewpage.action?pageId=6389772#UnitTestExecutionResultsImport(C#,VB.NET)-MSTest
TRX file is also generated correctly but no data in sonar.
Please let me know if this possible or any alternative.
Below are the command I am using:
>MSBuild.SonarQube.Runner.exe begin /k:rent /n:rent /v:1.0.0 /d:sonar.host.url=http://127.0.0.1:9000/ /d:sonar.exclusions=**/Files/**/* /d:sonar.cs.opencover.reportsPaths=E:\MSTest\rent\opencover.xml /d:sonar.cs.vstest.reportsPaths=E:\MSTest\rent\TestResults.trx
>MSBuild.exe /p:Configuration=debug /t:Publish /p:TargetProfile=Local E:\rent\rent.sln
>OpenCover.Console.exe -output:"E:\MSTest\rent\opencover.xml" -register:user -target:"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" -targetargs:"/testcontainer:E:\rent\rent.Tests\bin\debug\rent.Tests.dll /resultsfile:E:\MSTest\rent\TestResults.trx" -excludebyattribute:*.ExcludeFromCodeCoverage*
>MSBuild.SonarQube.Runner.exe end

SonarQube project not Code Coverage as blank when integrated with Atlassian Bamboo

I have an Atlassian Bamboo plan referring to my C# project. The tasks configured are as follows:
a) MSBuild task to build my C# solution file.
b) NUnit Runner and NUnit Parser tasks to run the Unit tests.
c) OpenCover task to execute the NUnit Unit tests and generate the metrics like Lines covered, Branch coverage etc.
d) SonarQube Runner task to generate the code coverage metrics on SonarQube project.
The metrics are successfully generated at step c) and results are written out to .xml file as output of OpenCover command. But after step d), SonarQube runner executes successfully and shows the results on Sonar project in terms of Number of unit tests, tests passed, tests skipped, tests failed. But the Code Coverage column shows as "-" blank.
Please note that my C# solution as multiple projects and I have the sonar-project.properties file at the root (Solution) level.
I use the following Sonar-project.properties file:
sonar.projectKey=<KeyName> (Key name of C# project in SonarQube)
sonar.projectName=<ProjectName> (Project name in SonarQube)
sonar.projectVersion=1.0
sonar.sources=.
sonar.language=cs
sonar.cs.nunit.reportsPaths=ABC.xml(Output of Nunit task)
sonar.cs.opencover.reportsPath=XYZ.xml(Output of OpenCover task)
Please guide me in this regard, any help will be greatly appreciated.
Thanks

Very late answer but it might help someone in the future:
You misspelled reportsPaths in your line
sonar.cs.opencover.reportsPath=XYZ.xml

Getting code coverage in Sonar but no test results (.Net)

I'm re-using reports in my sonar config:
sonar.gallio.mode=reuseReport
sonar.gallio.reports.path=gallio-report.xml
sonar.gallio.coverage.reports.path=results.xml
I've previously run Gallio and OpenCover and can confirm that both completed successfully and that Sonar is able to retrieve the files (I've checked the log produced by the -X flag thoroughly). When I view the project in the sonar dashboard, I see code coverage, but not test results:
Kindly ignore the low code coverage percentage, I'm running a small subset of tests while I figure this out.
I should see something that reflects the results I saw when I ran Gallio:
14 run, 13 passed, 1 failed (1 error), 0 inconclusive, 0 skipped
I'm happy to include the gallio-report.xml if that's helpful, but it's 103kb so clearly it contains plenty of data, and I think this is more likely to be a configuration issue.
I'm running OpenCover 4.0.1118 and Gallio 3.2.750 (tests are written with NUnit).
Any thoughts why I don't see any test results?

Chances are that you don't have the test sources in your .NET solution, so when SonarQube tries to import the test execution results, it can't find to which files they should be attached.
In the .NET sample solution, you can see that there is a test project (Example.Core.Tests) which contains the sources of the test classes.