I am not able to see code coverage value to one of my projects.I execute below commands to fetch the same.
For NUnit
"C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "C:\Example.Tests.dll" --where "cat == Unit" --result:Example.nunit-result.xml
For OpenCover
C:\Users\Administrator\AppData\Local\Apps\OpenCover\OpenCover.Console.exe "-target:C:\Program Files (x86)\NUnit.org\nunit-console\nunit3-console.exe" "-targetargs:C:\Example.Tests.dll --where=cat=Unit" -register:XXX
I have included --where=cat=Unit in Open-cover batch command and I see all my test cases ran successfully but I could not see the code-coverage. What more is missing from my end.
Sonarqube result of my project screenshot.
Finally after changing location of results.xml I could see the code coverage.
Related
console app with a batch file which will hit my Automation testing application and runs selected test cases. I have test case with code coverage and it runs from my visual studio . and now on top of it I have to create a console app which will keep some time interval and hit my VS test case and execute it.Any links will be helpful.
I expect the test case pass and fails status
Here's the batch file I use to run my Selenium tests with multiple runsettings files.
#ECHO OFF
IF NOT EXIST Results MKDIR Results
SETLOCAL
SET PATH=%PATH%;"C:\Program Files (x86)\Microsoft Visual Studio\2019\Professional\Common7\IDE\Extensions\TestPlatform"
SET VSTEST=vstest.console.exe
SET TESTS=.\bin\Debug\something.something.Tests.dll
IF "%~1"=="" (
FOR %%J IN (*.runsettings) DO CALL :func %%J
) ELSE (
CALL :func "%~1"
)
GOTO :EOF
:func
ECHO.
ECHO %~1 **********
ECHO.
"%VSTEST%" %TESTS% /Settings:"%~1" /Logger:trx /ResultsDirectory:Results
GOTO :EOF
You could use NUNit's console runner to run tests from the command line, or on a build server. I've had great success with running my tests this way.
First, you need to install the NUnit.ConsoleRunner Nuget package onto your project.
Then, navigate to the NUnit.ConsoleRunner directory under your packages folder that exists in the project directory.
Open NUnit.ConsoleRunner > tools folder to get into the same directory as the .exe itself.
Then, you can run:
nunit3-console {Path to your project's .dll} --testlist={Path to .txt testlist}
With valid parameters, it looks something like this:
nunit3-console C:\Users\christine.harbour\Repository\AutomationTestSuite\AutomationTestSuite.dll --testlist=C:\Users\christine.harbour\Repository\AutomationTestSuite\MyTestList.txt
Your testlist should contain namespaces of the test cases you wish to run, separated by a line break. For example:
AutomationTestSuite.Tests.MyTestClass_1.MyTest
AutomationTestSuite.Tests.MyTestClass_2.MyOtherTest
After you run the tests, the results will be saved in the NUnit.ConsoleRunner > tools directory. The results are in XML format and can be programatically parsed to push your test results to another tool.
There are plenty of arguments you can pass into the ConsoleRunner, including build configuration and framework version, all which are specified on NUnit's documentation.
NUnit console runner also integrates with Cake, which is a build scripting tool for C# projects. So, you could hypothetically clean / build your project, restore missing package references, and run your tests, all from the console.
More info on NUnit console runner can be found here: https://github.com/nunit/docs/wiki/Console-Command-Line
My taskmasters have tasked me with attempting to get output from Visual Studio Unit Tests for C#. Evidently, our build environment can make use of output files in that format. I hear rumors it is possible, but my Googling comes up empty.
Does anyone know how to accomplish this feat?
You can run your tests with VSTest or MSTest from the command line to create a .trx file with your test results. This is the standard format used for Visual Studio Unit Test output.
As mentioned in other answer, MSTest.exe will generate trx files that xUnit is able to parse.
If you have one script that builds all your projects, you might want to execute all tests and aggregate all test results into one trx file for xUnit.
Good practice is to name all your test projects to end with '.Test'. All these projects compile into dll's and they will all end with '.Test.dll'.
Then you can update your build script to pick up all test projects, by searching your repo for Test.dll files in a script like this:
#SET _config=Release
#call "%VS110COMNTOOLS%vsvars32"
#setlocal enabledelayedexpansion enableextensions
#set list=
#for /R ".." %%x in (obj) do #(
#set CTD=%%x
#pushd !CTD!
#for %%y in (%_config%\*Test.dll %_config%\*Tests.dll) do #set list=!list! /testcontainer:%%x\..\bin\%%y
#popd
) 2>nul
#set list=%list:~1%
#del results.trx 2>nul
mstest %list% /resultsfile:results.trx /detail:stdout
#IF NOT %ERRORLEVEL%==0 (GOTO lbl_error)
:lbl_success
#ECHO Successfully ran tests.
#GOTO lbl_end
:lbl_error
#ECHO Failed to run tests.
#EXIT /b 1
I have a solution with mstest unit testcases and using Opencover for testcoverage.
TestCoverage results are coming fine in Sonarqube but it is not displaying the result from generated TestResult.trx file in Sonar. It only displays the unit test cases count but not testcases names, which one is pass and which one is fail.
I have followed the same steps mentioned in the link below:
https://docs.sonarqube.org/pages/viewpage.action?pageId=6389772#UnitTestExecutionResultsImport(C#,VB.NET)-MSTest
TRX file is also generated correctly but no data in sonar.
Please let me know if this possible or any alternative.
Below are the command I am using:
>MSBuild.SonarQube.Runner.exe begin /k:rent /n:rent /v:1.0.0 /d:sonar.host.url=http://127.0.0.1:9000/ /d:sonar.exclusions=**/Files/**/* /d:sonar.cs.opencover.reportsPaths=E:\MSTest\rent\opencover.xml /d:sonar.cs.vstest.reportsPaths=E:\MSTest\rent\TestResults.trx
>MSBuild.exe /p:Configuration=debug /t:Publish /p:TargetProfile=Local E:\rent\rent.sln
>OpenCover.Console.exe -output:"E:\MSTest\rent\opencover.xml" -register:user -target:"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\MSTest.exe" -targetargs:"/testcontainer:E:\rent\rent.Tests\bin\debug\rent.Tests.dll /resultsfile:E:\MSTest\rent\TestResults.trx" -excludebyattribute:*.ExcludeFromCodeCoverage*
>MSBuild.SonarQube.Runner.exe end
I have an Atlassian Bamboo plan referring to my C# project. The tasks configured are as follows:
a) MSBuild task to build my C# solution file.
b) NUnit Runner and NUnit Parser tasks to run the Unit tests.
c) OpenCover task to execute the NUnit Unit tests and generate the metrics like Lines covered, Branch coverage etc.
d) SonarQube Runner task to generate the code coverage metrics on SonarQube project.
The metrics are successfully generated at step c) and results are written out to .xml file as output of OpenCover command. But after step d), SonarQube runner executes successfully and shows the results on Sonar project in terms of Number of unit tests, tests passed, tests skipped, tests failed. But the Code Coverage column shows as "-" blank.
Please note that my C# solution as multiple projects and I have the sonar-project.properties file at the root (Solution) level.
I use the following Sonar-project.properties file:
sonar.projectKey=<KeyName> (Key name of C# project in SonarQube)
sonar.projectName=<ProjectName> (Project name in SonarQube)
sonar.projectVersion=1.0
sonar.sources=.
sonar.language=cs
sonar.cs.nunit.reportsPaths=ABC.xml(Output of Nunit task)
sonar.cs.opencover.reportsPath=XYZ.xml(Output of OpenCover task)
Please guide me in this regard, any help will be greatly appreciated.
Thanks
Very late answer but it might help someone in the future:
You misspelled reportsPaths in your line
sonar.cs.opencover.reportsPath=XYZ.xml
I'm currently changing things to our Sonar setup since Gallio isn't supported anymore by C# Ecosystem 3. I already successfully imported the unit test coverage by using OpenCover with following command.
"C:\Program Files (x86)\OpenCover\OpenCover.Console.exe" -register -target:"c:\Program Files (x86)\NUnit 2.6.3\bin\nunit-console-x86.exe" -targetargs:"d:\Sonar\MyTest.dll /noshadow /framework=net-4.0" -output:"d:\Deploy\Sonar\Coverage.xml" "-filter:+[*]* -[*]Test"
With this command I'm only getting the Unit Test Coverage but I would also like to see the number of failed and successful tests. I think I can achieve this with the /result option like
"C:\Program Files (x86)\OpenCover\OpenCover.Console.exe" -register -target:"c:\Program Files (x86)\NUnit 2.6.3\bin\nunit-console-x86.exe" -targetargs:"d:\Deploy\Sonar\MyTest.dll /noshadow /framework=net-4.0 /result=tests.xml"
This command returns an xml with information about the tests that ran but is there any way to import this xml into SonarQube? Or isn't this supported?
Currently I'm importing the Coverage.xml file with the following command:
sonar.cs.opencover.reportsPaths=D:/Deploy/Sonar/Coverage.xml
Is there a similar property to import the tests.xml file with the test results?
OpenCover is now officially supported by SonarQube, please see the SonarQube documentation. You can pass the location of the XML report OpenCover generated with the command line parameter
/d:sonar.cs.opencover.reportsPaths="%path%\%report_file%"
where %path% is the folder your report file is generated in and %report_file% is the file name of your OpenCover XML report.
From the documentation at http://docs.codehaus.org/display/SONAR/C%23+Plugin, it looks like you can import unit test execution reports of MSTest format only.
Thanks to the following site http://www.codewrecks.com/blog/index.php/2009/07/19/integrate-nunit-test-into-a-tfs-build/ I was able to convert the output of OpenCover test results to a .trx format using an XSLT transformation. After the transformation I just used the default import functionality
sonar.cs.vstest.reportsPaths=MyTestRestultFile.trx