I have a solution with mstest unit testcases and using Opencover for testcoverage.
TestCoverage results are coming fine in Sonarqube but it is not displaying the result from generated TestResult.trx file in Sonar. It only displays the unit test cases count but not testcases names, which one is pass and which one is fail.
I have followed the same steps mentioned in the link below:
https://docs.sonarqube.org/pages/viewpage.action?pageId=6389772#UnitTestExecutionResultsImport(C#,VB.NET)-MSTest
TRX file is also generated correctly but no data in sonar.
Please let me know if this possible or any alternative.
Below are the command I am using:
>MSBuild.SonarQube.Runner.exe begin /k:rent /n:rent /v:1.0.0 /d:sonar.host.url=http://127.0.0.1:9000/ /d:sonar.exclusions=**/Files/**/* /d:sonar.cs.opencover.reportsPaths=E:\MSTest\rent\opencover.xml /d:sonar.cs.vstest.reportsPaths=E:\MSTest\rent\TestResults.trx
>MSBuild.exe /p:Configuration=debug /t:Publish /p:TargetProfile=Local E:\rent\rent.sln
>OpenCover.Console.exe -output:"E:\MSTest\rent\opencover.xml" -register:user -target:"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" -targetargs:"/testcontainer:E:\rent\rent.Tests\bin\debug\rent.Tests.dll /resultsfile:E:\MSTest\rent\TestResults.trx" -excludebyattribute:*.ExcludeFromCodeCoverage*
>MSBuild.SonarQube.Runner.exe end
Related
I have a Windows service written in .net framework 4.6. I'm trying to run Sonar analysis for this service. My requirement is to generate both code coverage result and Unit test case report either by using MStest.exe or vstest.console.exe. I have written test cases using MStest for my service.
Using MSTest, I have written the below command:
MSTest /testcontainer:.\SolutionTests\bin\Release\SolutionTests.dll /resultsfile:"C:\SonarQube\Solution.trx"
Using vstest.console.exe, I have written the below command:
vstest.console.exe SolutionTests\bin\Release\SolutionTests.dll /Enablecodecoverage /Logger:trx;LogFileName="C:\SonarQube\Solution.trx"
In both the cases only Unit test report is generated (.trx file) as I have set the filename explicitly in the command.
Is there any way I can generate .coverage file as well, with in the same command by adding other parameters. I read in few articles which said MSTest command generates both the reports (result.trx and data.coverage), but no where it is written the exact command how to do it. I ran the above command, it did not generate data.coverage file for me.
This can be achieved using a package called Coverlet. Basically you can define the output format of the tests and generate the .trx file for Bamboo. The command will look something like:
executable test --logger "trx;LogFileName=testResults.trx" /p:CollectCoverage=true /p:CoverletOutputFormat=preferredformat.
Check out the documentation for how to specifically integrate with VSTest.
My taskmasters have tasked me with attempting to get output from Visual Studio Unit Tests for C#. Evidently, our build environment can make use of output files in that format. I hear rumors it is possible, but my Googling comes up empty.
Does anyone know how to accomplish this feat?
You can run your tests with VSTest or MSTest from the command line to create a .trx file with your test results. This is the standard format used for Visual Studio Unit Test output.
As mentioned in other answer, MSTest.exe will generate trx files that xUnit is able to parse.
If you have one script that builds all your projects, you might want to execute all tests and aggregate all test results into one trx file for xUnit.
Good practice is to name all your test projects to end with '.Test'. All these projects compile into dll's and they will all end with '.Test.dll'.
Then you can update your build script to pick up all test projects, by searching your repo for Test.dll files in a script like this:
#SET _config=Release
#call "%VS110COMNTOOLS%vsvars32"
#setlocal enabledelayedexpansion enableextensions
#set list=
#for /R ".." %%x in (obj) do #(
#set CTD=%%x
#pushd !CTD!
#for %%y in (%_config%\*Test.dll %_config%\*Tests.dll) do #set list=!list! /testcontainer:%%x\..\bin\%%y
#popd
) 2>nul
#set list=%list:~1%
#del results.trx 2>nul
mstest %list% /resultsfile:results.trx /detail:stdout
#IF NOT %ERRORLEVEL%==0 (GOTO lbl_error)
:lbl_success
#ECHO Successfully ran tests.
#GOTO lbl_end
:lbl_error
#ECHO Failed to run tests.
#EXIT /b 1
I am not able to see code coverage value to one of my projects.I execute below commands to fetch the same.
For NUnit
"C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "C:\Example.Tests.dll" --where "cat == Unit" --result:Example.nunit-result.xml
For OpenCover
C:\Users\Administrator\AppData\Local\Apps\OpenCover\OpenCover.Console.exe "-target:C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "-targetargs:C:\Example.Tests.dll --where=cat=Unit" -register:XXX
I have included --where=cat=Unit in Open-cover batch command and I see all my test cases ran successfully but I could not see the code-coverage. What more is missing from my end.
Sonarqube result of my project screenshot.
Finally after changing location of results.xml I could see the code coverage.
I have an issue getting SonarQube to pickup the unittest results.
The setup is as following:
TFS2015 Update 3
New SonarQube Build Tasks (released a few days ago)
SonarQube 6.2
C# Plugin 5.5.1.522
From the verboose log it appears that the trx files are identified, but I can't find anything about those beeing processed, even with verbose logging.
2016-12-16T13:26:54.5935752Z 14:26:54.516 Looking for TRX files in: E:\TFS_Build\Agent01\Work\49\TestResults
2016-12-16T13:26:54.5945826Z 14:26:54.518 Located a test results file: E:\TFS_Build\Agent01\Work\49\TestResults\tap_sysadmin_TFS-BUILD1 2016-12-16 14_26_32.trx
The trx file does contain test results.
Any idea how to get this to work?
I'm currently changing things to our Sonar setup since Gallio isn't supported anymore by C# Ecosystem 3. I already successfully imported the unit test coverage by using OpenCover with following command.
"C:\Program Files (x86)\OpenCover\OpenCover.Console.exe" -register -target:"c:\Program Files (x86)\NUnit 2.6.3\bin\nunit-console-x86.exe" -targetargs:"d:\Sonar\MyTest.dll /noshadow /framework=net-4.0" -output:"d:\Deploy\Sonar\Coverage.xml" "-filter:+[*]* -[*]Test"
With this command I'm only getting the Unit Test Coverage but I would also like to see the number of failed and successful tests. I think I can achieve this with the /result option like
"C:\Program Files (x86)\OpenCover\OpenCover.Console.exe" -register -target:"c:\Program Files (x86)\NUnit 2.6.3\bin\nunit-console-x86.exe" -targetargs:"d:\Deploy\Sonar\MyTest.dll /noshadow /framework=net-4.0 /result=tests.xml"
This command returns an xml with information about the tests that ran but is there any way to import this xml into SonarQube? Or isn't this supported?
Currently I'm importing the Coverage.xml file with the following command:
sonar.cs.opencover.reportsPaths=D:/Deploy/Sonar/Coverage.xml
Is there a similar property to import the tests.xml file with the test results?
OpenCover is now officially supported by SonarQube, please see the SonarQube documentation. You can pass the location of the XML report OpenCover generated with the command line parameter
/d:sonar.cs.opencover.reportsPaths="%path%\%report_file%"
where %path% is the folder your report file is generated in and %report_file% is the file name of your OpenCover XML report.
From the documentation at http://docs.codehaus.org/display/SONAR/C%23+Plugin, it looks like you can import unit test execution reports of MSTest format only.
Thanks to the following site http://www.codewrecks.com/blog/index.php/2009/07/19/integrate-nunit-test-into-a-tfs-build/ I was able to convert the output of OpenCover test results to a .trx format using an XSLT transformation. After the transformation I just used the default import functionality
sonar.cs.vstest.reportsPaths=MyTestRestultFile.trx