We Keep Coding

SonarCloud - Cannot setup coverage analysis for Azure DevOps pipeline

I am currently implementing an azure DevOps pipeline for my .Net microservices.
I researched and read the sonar cloud doc, but nothing helped yet. I did generate the code coverage report using cobertura and that is working correctly in azure devops but I found out that you cannot link that type of format to sonarcloud so I was trying somehow to generate also the opencover and use it for exporting to sonarcloud, this is what I have done so far:
trigger:
- dev
- main
pool:
vmImage: ubuntu-latest
jobs:
- job: check_code_quality
steps:
- task: SonarCloudPrepare#1
displayName: 'Setup Sonar Cloud'
inputs:
SonarCloud: 'SonarCloud'
organization: '***'
scannerMode: 'MSBuild'
projectKey: '***'
projectName: '****'
extraProperties: |
sonar.exclusions=**/obj/**,**/*.dll
sonar.cs.vstest.reportsPaths=$(Agent.TempDirectory)/*.trx
sonar.cs.opencover.reportsPaths=$(Agent.TempDirectory)/**/coverage.opencover.xml
- task: DotNetCoreCLI#2
displayName: 'Build main code'
inputs:
command: build
projects: '**/*.csproj'
arguments: '--configuration Release'
- task: DotNetCoreCLI#2
displayName: 'Run unit tests'
continueOnError: true
inputs:
command: 'test'
projects: '**/*[Tt]est*/*.csproj'
testRunTitle: 'Backend Unit Testing'
arguments: '--configuration Release --collect:"XPlat Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura,opencover'
publishTestResults: true
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage report'
inputs:
codeCoverageTool: 'Cobertura'
summaryFileLocation: '$(Agent.TempDirectory)/**/coverage.cobertura.xml'
- task: SonarCloudAnalyze#1
displayName: 'Run Sonar Analysis'
- task: SonarCloudPublish#1
displayName: 'Publish Sonar Results'
inputs:
pollingTimeoutSec: '300'
I checked the logs and I saw this:
Attachments:
/home/vsts/work/_temp/111bef07-6e19-43fe-a689-7597bc24dda3/coverage.cobertura.xml
/home/vsts/work/_temp/111bef07-6e19-43fe-a689-7597bc24dda3/coverage.opencover.xml
Also I have an issue that sonar cloud is only analyzing my test projects not the main code

I suppose that you could re-modify your dotnet test task with argument below.
arguments: '--logger trx --collect:"XPlat Code Coverage" --collect:"Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura --output $(build.artifactstagingdirectory)'
Full yaml for dotnet part as below.
steps:
- task: DotNetCoreCLI#2
displayName: 'dotnet test'
inputs:
command: 'test'
projects: 'xxx.csproj'
arguments: '--logger trx --collect:"XPlat Code Coverage" --collect:"Code Coverage" -- DataCollectionRunSettings.DataCollectors.DataCollector.Configuration.Format=cobertura --output $(build.artifactstagingdirectory)'
publishTestResults: false
- task: PublishBuildArtifacts#1
inputs:
PathtoPublish: '$(Build.Repository.LocalPath)\{reponame}\TestResults'
ArtifactName: 'drop'
publishLocation: 'Container'
- task: PublishTestResults#2
inputs:
testResultsFormat: 'VSTest'
testResultsFiles: '**/*.trx'
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage from $(Agent.TempDirectory)\**\coverage.cobertura.xml'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(Build.Repository.LocalPath)\{reponame}\TestResults\**\coverage.cobertura.xml'
reportDirectory: '$(System.DefaultWorkingDirectory)\reports' ###please keep this setting as mine
For the sonarqube part, the sonarqube task would search for the .trx file in your agent folder.
My path is below, and you could check your local agent folder in the debug log and set the path in sonarqube task.
==================================================
Updated on 1/12
i suppose that you could find all the test result files with coverage..xml, .trx, and .coverage, you could check in pipeline artifacts drop-down list for the existence of the files and then go to your local folder again,.

I have written my Unit Test Specflow. Iam not able to implement code coverage using Azure pipeline . Can someone help me?

In my web api project I have written unit test in Specflow (MStest).
FYI: all my test cases pass.
This is the build pipeline:
- task: DotNetCoreCLI#2
displayName: 'Install ReportGenerator'
inputs:
command: custom
custom: tool
arguments: 'install --global dotnet-reportgenerator-globaltool
- task: DotNetCoreCLI#2
displayName: Test
inputs:
command: test
projects: '**/project/webApi/**/*Tests.csproj'
publishTestResults: false
arguments: '--configuration $(buildConfiguration) /p:Collect="XPlat Code Coverage" /p:CollectCoverage=true /p:CoverletOutputFormat=cobertura /p:CoverletOutput=$(Build.SourcesDirectory)/TestResults/Coverage/ "/p:MergeWith=../TestResults/coverlet.json"'
- task: PublishCodeCoverageResults#1
displayName: CodeCoverage
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(Build.SourcesDirectory)/**/Coverage/*.xml'
- task: reportgenerator#4
inputs:
reports: '$(Build.SourcesDirectory)/**/*.cobertura.xml'
targetdir: '$(Build.SourcesDirectory)/CoverageResults'
The xml file which has to be generated is not generated.
When tried to publish the code using Coberta
Error: the report file pattern 'D:\a\1\s/**/*.cobertura.xml' is invalid. No matching files found.
No report files specified
Here's a screenshot for illustration:

You don't need reportgenerator with the newest dotnet cli.
And pleae try with this configuration:
# You just added coverlet.collector to use 'XPlat Code Coverage'
- task: DotNetCoreCLI#2
displayName: Test
inputs:
command: test
projects: '**/*Tests/*.csproj'
arguments: '--configuration $(buildConfiguration) --collect:"XPlat Code Coverage"'
- task: PublishCodeCoverageResults#1
displayName: 'Publish code coverage'
inputs:
codeCoverageTool: Cobertura
summaryFileLocation: '$(Agent.TempDirectory)/**/coverage.cobertura.xml'

How to have both xUnit and NUnit in a solution built in Azure DevOps Pipelines?

I have a C# solution where there are several unit test projects that use xUnit and an "end to end" tests project that uses NUnit. (This is intentional and fine so I'd appreciate not trying to convince me to ditch one for the other, thanks)
I use an Azure DevOps Pipeline to build, test and package my solution. This is my test step (from azure-pipelines.yml) at the moment:
- task: DotNetCoreCLI#2
displayName: 'Run tests in solution'
inputs:
command: 'test'
arguments: '--configuration $(buildConfiguration) --collect "Code coverage" --filter Category!=Integration'
publishTestResults: true
When I run this my pipeline fails with this error message:
An exception occurred while invoking executor 'executor://nunit3testexecutor/': Unexpected Word 'Category' at position 29 in selection expression.
I'm pretty sure this happens because NUnit is getting picked up and it does not understand the Category filter term. (NUnit expects the term TestCategory instead)
I tried getting the pipeline to not pick up the NUnit project (called "EndToEnd")this way:
--filter FullyQualifiedName!~EndToEnd&Category!=Integration'
But this does not work and I get the same error message.
How can I get Azure DevOps Pipelines to only run the tests in my xUnit projects in this step and not fail because of the presence of the NUnit project?

You may use projects to pick projects. I tested this against this solution and it works:
variables:
buildConfiguration: 'Release'
rootDirectory: '$(Build.SourcesDirectory)/stackoverflow/69-nunit-and-xunit'
steps:
- task: DotNetCoreCLI#2
displayName: Restore nuget packages
inputs:
command: restore
projects: '**/*.csproj'
workingDirectory: $(rootDirectory)
- task: DotNetCoreCLI#2
displayName: Test xUnit
inputs:
command: test
projects: '$(rootDirectory)/**/*XUnit.csproj'
arguments: '--configuration $(buildConfiguration)'
- task: DotNetCoreCLI#2
displayName: Test NUnit
inputs:
command: test
projects: '$(rootDirectory)/**/*NUnit.csproj'
arguments: '--configuration $(buildConfiguration)'

Azure build pipeline NuGet Restore error NETSDK1045

I created an Azure CI pipeline through Visual Studio 2019. The project target framework is .net core 3.1. However, during the NuGet Restore it fails.
I have .Net Core 2.2.2017 and 3.1.101 installed on my computer and my Visual Studio 2019 community version is 16.4. My environment variables paths are in place.
I also tried creating an app service manually that uses .net core 3.1 LTS stack but I still get the same error.
##[error]The nuget command failed with exit code(1) and error(C:\Program Files\dotnet\sdk\2.2.110\Sdks\Microsoft.NET.Sdk\targets\Microsoft.NET.TargetFrameworkInference.targets(137,5):
error NETSDK1045: The current .NET SDK does not support targeting .NET Core 3.1. Either target .NET Core 2.2 or lower, or use a version of the .NET SDK that supports .NET Core 3.1.
This is the YAML file generated by Visual Studio
pool:
name: Hosted VS2017
demands:
- msbuild
- visualstudio
- vstest
steps:
- task: NuGetToolInstaller#1
displayName: 'Use NuGet 5.0.0'
inputs:
versionSpec: 5.0.0
- task: NuGetCommand#2
displayName: 'NuGet restore'
inputs:
restoreSolution: '$(Parameters.solution)'
- task: VSBuild#1
displayName: 'Build solution'
inputs:
solution: '$(Parameters.solution)'
msbuildArgs: '/p:DeployOnBuild=true /p:WebPublishMethod=Package /p:PackageAsSingleFile=true /p:SkipInvalidConfigurations=true /p:PackageLocation="$(build.artifactstagingdirectory)\\"'
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
- task: VSTest#2
displayName: 'Test Assemblies'
inputs:
testAssemblyVer2: |
**\$(BuildConfiguration)\*test*.dll
!**\obj\**
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
- task: PublishSymbols#1
displayName: 'Publish symbols path'
inputs:
SearchPattern: '**\bin\**\*.pdb'
continueOnError: true
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
ArtifactName: '$(Parameters.ArtifactName)'

I believe, you do not need to install NuGetToolInstaller and vsbuild, you should firstly install DotNetCoreCLI after that you can use it. Here is manual how your yaml file should looks like.

Publishing by the IDE has no pain because it use the tools you have installed on your machine, in the pipeline you need to specify the toolin you gonna use and that is the pain.
For those who are not using the new YAML based pipeline but instead use classic editor on devops you might need to add dot net core add task and change from nuget restore to dotnet restore task

When I create a pipeline from the Azure Devops UI, my YAML looks like the below. It uses DotNetCoreCLI instead of VSBuild. Notice that it uses the Restore command instead of anything having to do with Nuget.
pool:
name: Azure Pipelines
#Your build pipeline references an undefined variable named ‘Parameters.RestoreBuildProjects’. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab. See https://go.microsoft.com/fwlink/?linkid=865972
#Your build pipeline references an undefined variable named ‘Parameters.RestoreBuildProjects’. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab. See https://go.microsoft.com/fwlink/?linkid=865972
#Your build pipeline references the ‘BuildConfiguration’ variable, which you’ve selected to be settable at queue time. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab, and then select the option to make it settable at queue time. See https://go.microsoft.com/fwlink/?linkid=865971
#Your build pipeline references the ‘BuildConfiguration’ variable, which you’ve selected to be settable at queue time. Create or edit the build pipeline for this YAML file, define the variable on the Variables tab, and then select the option to make it settable at queue time. See https://go.microsoft.com/fwlink/?linkid=865971
steps:
- task: DotNetCoreCLI#2
displayName: Restore
inputs:
command: restore
projects: '$(Parameters.RestoreBuildProjects)'
- task: DotNetCoreCLI#2
displayName: Build
inputs:
projects: '$(Parameters.RestoreBuildProjects)'
arguments: '--configuration $(BuildConfiguration)'
- task: DotNetCoreCLI#2
displayName: Publish
inputs:
command: publish
publishWebProjects: True
arguments: '--configuration $(BuildConfiguration) --output $(build.artifactstagingdirectory)'
zipAfterPublish: True
- task: PublishBuildArtifacts#1
displayName: 'Publish Artifact'
inputs:
PathtoPublish: '$(build.artifactstagingdirectory)'
condition: succeededOrFailed()

Azure Pipelines multi stage problems for basic steps

I come from GitLab and its .gitlab-ci.yml and I am experimenting with Azure DevOps multi stage pipelines but I am quite confused about how it works and what's the best strategy even after reading several documentation articles at https://learn.microsoft.com/en-us/azure/devops/pipelines/?view=azure-devops
Please allow me to ask several related questions for the basic scenario I'm trying, which is compile, run unit tests, pack a nuget Package for the whole solution (it could contain multiple projects/nuGet packages) and publish the package to a nuGet feed (if the branch is master, a release version, otherwise a pre-release version).
This is the repository I'm grabbing the code from: https://github.com/sasw-diego/sasw-test-support
It would generate only a nuGet package but I've got other multiproject solutions which should generate many nuGet packages
This is my azure-pipelines.yml so far:
trigger:
- master
- feature/*
pool:
vmImage: ubuntu-latest
variables:
NUGET_FOLDER_NAME: nupkgs
NUGET_REPOSITORY: https://whatever
PRERELEASE_SUFFIX: $(Build.BuildId)
PIPELINE_ARTIFACT_NAME: $(Build.BuildNumber)
stages:
- stage:
displayName: 'Build'
jobs:
- job: 'Build'
steps:
- task: NuGetAuthenticate#0
displayName: 'Authenticate in NuGet feed'
- script: dotnet restore --no-cache --force
displayName: 'Restore dependencies'
- script: dotnet build --configuration Release --no-restore
displayName: 'Build for Release'
- script: ls $(System.DefaultWorkingDirectory)
displayName: 'List content'
- publish: $(System.DefaultWorkingDirectory)
artifact: $(PIPELINE_ARTIFACT_NAME)
- stage:
displayName: 'Automated Tests'
condition: succeeded()
jobs:
- job:
displayName: 'Unit Tests'
steps:
- download: current
artifact: $(PIPELINE_ARTIFACT_NAME)
- script: ls -a
displayName: 'View'
- script: ls ./test
displayName: 'View test'
- script: ls ./test/Sasw.TestSupport.UnitTests
displayName: 'View folder'
- script: dotnet vstest test/*UnitTests/bin/Release/**/*UnitTests.dll
displayName: 'Run unit tests'
- stage:
displayName: 'NuGet Package'
condition: succeeded()
jobs:
- job:
displayName: 'Pack Preview Version'
condition: ne(variables['Build.SourceBranch'], 'refs/heads/master')
steps:
- script: dotnet pack *.sln --configuration Release --output $(NUGET_FOLDER_NAME)
displayName: 'Pack'
- job:
displayName: 'Pack Stable Version'
condition: eq(variables['Build.SourceBranch'], 'refs/heads/master')
steps:
- script: dotnet pack *.sln --configuration Release --output $(NUGET_FOLDER_NAME) --version-suffix $(PRERELEASE_SUFFIX) --include-source --include-symbols -p:SymbolPackageFormat=snupkg
displayName: 'Pack'
Which is the "best" strategy for a multistage? I see Azure DevOps pipelines have the concept of Stage > Jobs > Task but they all look similar to me. So I decide to divide the process in stages such as Build - AutomatedTests - NuGet Package - Publish
As you can see, it's a sequential process where each stage needs something from the previous one. The automated tests needs the built code (dll), the nuGet package needs also access to the built code, the publish needs access to the nupkg generated, etc. I don't know whether it's ok to follow this strategy or it's better to have a single stage with multiple jobs, or even a single job with multiple tasks. As I said I don't fully understand the benefit of having so many concepts and how do they fit in with my needs.
Is Azure multistage pipelines supposed to replace the old Build and Release as separate concepts? It makes sense to me to have the CI/CD in one place with a multistage approach and scriptable to make it versioned in a source control repository. But I still see the Release concept as a separate one currently on Azure DevOps. So probably I should use the azure pipelines yml up until the package "step" and then with Release go grab this nupkg and publish it onto some feed. Not sure what the benefit would be though.
I am having problems making the output of a stage as the input of the next one, and probably it's because I don't fully get it. In the yml above the build stage succeeds but the Automated Testing stage fails on its Run Unit Tests job with an error No test source files were specified. I verified this happens because there is no generated folder bin at all. It's seems strange since I am copying everything from the previous stage (and the previous stage generated bin folders with dll), but then the next stage, despite being able to download everything, it cannot find the tests.
This is the log for the stage that fails: https://gist.github.com/sasw-diego/df66eccf71bbfc044a4d72be96268c9a
It'd be very helpful if somebody spots what am I missing to be able to understand this process. Any link to clarify all these concepts would be much appreciated. Ta
PS: This is a similar generic CI/CD I had in GitLab to upload 1 or many nuGets to a feed:
https://gist.github.com/sasw-diego/bf46258cb1ad0aa5241e8d1866b53f48
UPDATE:
Thanks for the answer. I successfully created a CI/CD yml with multi-stage pipelines that restores, builds, executes tests, runs a container (e.g: an eventStore host) to run integration tests against it, and releases the nuGet in artifacts. So mission accomplished! I've separated it into different stages and jobs to probe some points
trigger:
- master
- feature/*
pool:
vmImage: ubuntu-18.04
variables:
- group: sasw-common-variables
- name: NUGET_FOLDER_NAME
value: nupkgs
- name: PIPELINE_ARTIFACT_NAME
value: $(Build.BuildNumber)
- name: PATH_PIPELINE_ARTIFACT_NAME
value: $(Pipeline.Workspace)/$(PIPELINE_ARTIFACT_NAME)
- name: NUGET_API_KEY
value: $(nuget-api-key)
- name: NUGET_FEED
value: $(nuget-feed)
- name: PRERELEASE_SUFFIX
value: $(nuget-prerelease-suffix)
resources:
containers:
- container: eventstore
image: eventstore/eventstore:release-5.0.2
ports:
- 1113:1113
env:
EVENTSTORE_INT_TCP_PORT: 1113
EVENTSTORE_EXT_TCP_PORT: 1113
EVENTSTORE_INT_HTTP_PORT: 2113
EVENTSTORE_EXT_HTTP_PORT: 2113
EVENTSTORE_EXT_HTTP_PREFIXES: http://*:2113/
stages:
- stage:
displayName: 'Build'
jobs:
- job: 'Build'
displayName: 'Build & Create nuGet Package'
services:
eventstore: eventstore
steps:
- task: NuGetAuthenticate#0
displayName: 'Authenticate in NuGet feed'
- script: dotnet restore --no-cache --force
displayName: 'Restore dependencies'
- script: dotnet build --configuration Release --no-restore
displayName: 'Build with Release Configuration'
- script: dotnet vstest test/*UnitTests/bin/Release/**/*UnitTests.dll
displayName: 'Run unit tests'
- script: dotnet vstest test/*IntegrationTests/bin/Release/**/*IntegrationTests.dll
displayName: 'Run integration tests'
- script: dotnet pack *.sln --configuration Release --output $(NUGET_FOLDER_NAME)
condition: and(succeeded(), eq(variables['Build.SourceBranch'], 'refs/heads/master'))
displayName: 'Create release nuGet'
- script: dotnet pack *.sln --configuration Release --output $(NUGET_FOLDER_NAME) --version-suffix $(PRERELEASE_SUFFIX) --include-source --include-symbols -p:SymbolPackageFormat=snupkg
condition: and(succeeded(), ne(variables['Build.SourceBranch'], 'refs/heads/master'))
displayName: 'Create pre-release nuGet'
- publish: $(System.DefaultWorkingDirectory)/$(NUGET_FOLDER_NAME)
artifact: $(PIPELINE_ARTIFACT_NAME)
displayName: 'Publish pipeline artifact'
- stage:
displayName: 'Release'
condition: succeeded()
jobs:
- job: 'Publish'
displayName: 'Publish nuGet Package'
steps:
- download: current
artifact: $(PIPELINE_ARTIFACT_NAME)
displayName: 'Download pipeline artifact'
- script: ls $(PATH_PIPELINE_ARTIFACT_NAME)
displayName: 'Display contents of downloaded articacts path'
- task: NuGetAuthenticate#0
displayName: 'Authenticate in NuGet feed'
- task: UseDotNet#2
displayName: 'Use latest .NET Core sdk 3.x'
inputs:
packageType: sdk
version: 3.x
includePreviewVersions: true
installationPath: $(Agent.ToolsDirectory)/dotnet
- script: dotnet nuget push $(PATH_PIPELINE_ARTIFACT_NAME)/**/*.nupkg --source $(NUGET_FEED) --api-key $(NUGET_API_KEY) --skip-duplicate
displayName: 'Uploads nuGet packages'

your whole build definition should be a single "stage". Those Build - AutomatedTests - NuGet Package - Publish are not stages, those are logical parts of the build.
probably sometime in the future that will\might happen
that is because you are using stages when you are not supposed to. each stage runs on a different agent. you should have a single stage and all the tasks should run inside that stage.