In our Jenkins pipeline, we use SonarQube to report on our code coverage. After running all of our unit/integration tests to produce the .coverage file, we need to analyze this file to create the ".coverage.coveragexml" which is ultimately what is used by SonarQube to interpret the code coverage. We do this by using the CodeCoverage.exe:
"C:\Program Files (x86)\Microsoft Visual Studio\2022\TestAgent\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe" analyze /output:"{somePath}\{someName}.coverage.coveragexml" "{somePath}\{someName}.coverage"
This command appears to be working, but when you run dir /s *.coveragexml (within the directory), it displays something like:
Directory of C:\jenkins\path\to\TestResults\coverageFile
03/22/2022 04:59 PM 64 ContainerAdministrator_DC420D3FA0BA_2022-03-22.16_46_43.coverage.coveragexml
1 File(s) 64 bytes
64 bytes is practically nothing - and I believe this is the reason why our SonarQube metrics show we have 0 coverage now.
I added the same dir command, only this time to check for the .coverage file(s), and those come back with only 10 bytes in them - making me think that these files are essentially empty. I saw this post that seems to be a similar issue. The accepted answer said to change the platform type from x86 to x64, but that did not work in my case.
The vstest.console command for running our tests is:
vstest.console /Parallel /EnableCodeCoverage /Logger:trx /Platform:x86 ".\somePath\Test.dll"
This issue originally started back when we made a change to our Jenkinsfile for it to use Visual Studio 2022 instead of 2019 (the base image was updated) in the command that started the CodeCoverage executable.
What could be causing the coverage files to be nearly/completely empty and how can I fix it?
It seems the base image we use must have a non-enterprise edition of the Code Coverage tools (which is a requirement). We tested our SonarQube projects commands locally using an enterprise edition of the tools (I have Visual Studio 2022 Enterprise installed on my machine), and the coverage files produced contain the correct data. However, when we used a Visual Studio Professional install, the files are empty just like our Jenkins pipeline.
As stated, this started happening when the base image was updated - in particular it was around November 8th 2021. It seems the base docker image we were using (mcr.microsoft.com/dotnet/framework/sdk:4.8-20220210-windowsservercore-ltsc2019) has the latest 2022 tools, but it must not be an enterprise edition - hence the empty files.
We switched our pipeline over to using dotCover instead to perform the analysis, which works as expected and our SonarQube coverage is back to normal.
Related
TL;DR: Clean Windows build server, MS Build Tools 2019 and TFS 2013 installed.
Build Logs says Exception Message: File not found: MSBuild.exe (type FileNotFoundException)
How Do I setup MSBUILD toolpath in a XAML file on TFS 2013, configured to use MSBuild 2019 with a Visual Studio 2019 Enterprise installed on my dev box.
Please provide FULL XAML file that WORKS with TFS2013 (update 3) with an explicit custom msbuild path.
A Bit more info:
We have a multi server setup for our TFS build routines running version 2013.
One controller and a few agent servers here and there, nothing fancy.
Recently I added yet another for GUI Testing - I need the agent service to be in Interactive mode.
Steps so far:
Installed TFS 2013 including two agents - linking to the main controller.
I install Build Tools version 2019
I added tags to control the build destribution.
Activated a build and the rest is history.
I'm four (frustrating) days in on this - I've read every Stack Overflow TFS set toolpath entry I can find, and a huge bunch of odd forums, yet none seems to scratch my itch here.
I even added both paths you see further down, to the Windows environment PATH variable. Im not sure that does anything anymore.
See; Everybody talks about this entry in the XAML that says toolpath (or similar) - but; I dont see it. We use a minor changed version of a TfvcTemplate.12.Xaml.
Many also talk about this GUI tool for editing XALM files, where I get to see all the different options available (I presume) - can someone help me with this too? Its certainly not working out of the box with a 2019 Enterprise????
So; I installed this Extension Process Template Editor, but of course nothing happens when I click a XAML file, just more XML :)
I now have a few added paths on the new server, where I see instances of msbuild.exe installed (MS/.NET get your act together, please)
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin
C:\Program Files (x86)\MSBuild\12.0\Bin
Please try not to write this off as a duplicate at first glance. I know its similar to other questions - however; Devil is in the details.
Set toolpath to MSBuild 2019 in a TFS 2013 using (XALM) build process templates
Just as you know, the Old Visual Studio versions installed msbuild into C:\Program Files (x86)\MSBuild\<version>\bin and apparently the RunMSBuild activity used the ToolVersion + the ToolArchitecture to calculate this path.
VS 2019 build tool instead installs it C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin and the RunMSBuild can't calculate the proper path anymore. You can not use the old DefaultTemplate12.xaml to integrate with VS2019 build tool.
To make this work, you could try to modify the TFSBuildServiceHost.exe.config and you must have a version of VS 2019 build tool or VS2019 installed on the build server in order for this to work.
More detail step please follow Jonesy2488's answer in this link:How to get VS 2017 working with TFS 2017 XAML Builds.
Besides, we could also try to use MSBuild task instead of Visual Studio Build task, then specify the location of MSBuild 16.0:
Hope this helps.
I made it work by changing ToolVersion for mtba:RunMSBuild in TfvcTemplate.12.xaml to the required version and adding MSBuildToolsPath to Windows Registry.
Registry settings for 15.0 and 16.0
Windows Registry Editor Version 5.00
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSBuild\ToolsVersions]
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSBuild\ToolsVersions\15.0]
"MSBuildToolsPath"="C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\BuildTools\\MSBuild\\15.0\\Bin\\amd64"
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\MSBuild\ToolsVersions\16.0]
"MSBuildToolsPath"="C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\MSBuild\\Current\\Bin\\amd64"
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\MSBuild\ToolsVersions]
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\MSBuild\ToolsVersions\15.0]
"MSBuildToolsPath"="C:\\Program Files (x86)\\Microsoft Visual Studio\\2017\\BuildTools\\MSBuild\\15.0\\Bin"
[HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\MSBuild\ToolsVersions\16.0]
"MSBuildToolsPath"="C:\\Program Files (x86)\\Microsoft Visual Studio\\2019\\BuildTools\\MSBuild\\Current\\Bin"
I'm working in C# on Visual Studio and my builds run on TFS.
This week I enabled code coverage via my tfs build (Let's assume that I have a .sln Names MySln and there is Tests.MySln there too).
The code coverage via tfs stood on 6% and it's weird because when I ran it with Visual Studio Code Coverage in Test Section I got 70% (which is more correctly).
After I downloaded the code coverage results from tfs I noticed that it somehow ran with more dlls that we're using via Nuget.
I tried some changes in .runSettings file, but when I added include only MySln.dll, I got 0 code coverage and I don't want to create runSettings that say exclude all the other dll I saw that ran (I have 15 different slns and I don't want to create a list that might change between projects). I even tried to just enable code coverage without changes of runsettings.
Does someone knows which 'settings' does Visual Studio Test Code Coverage deliver? which config?.
Or have another idea how to create .runSettings file?
Update
Op finally re-write the runSettings file fixed it.
This sounds like a configuration setting issue in Visual Studio or .runsettings file not related to TFS side.
Suggest you try to remote to the build agent and manually run the Code Coverage through the Visual Studio in remote agent directly. Check whether you get the same result if run it through TFS build server.
I have a build on TeamCity that packages a web project ready for later deployment, currently using MSBuild 2013. When I pushed some code recently I got build errors (due to the fact that I was using some C# 6 features) so I went to change the build configuration to use MSBuild 2015 instead and got this error:
[Error] C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0\Web\Transform\Microsoft.Web.Publishing.AspNetCompileMerge.targets(132, 5): Can't find the valid AspnetMergePath
This error has been mentioned in other questions before (such as here: VS 2012 Publish: Can't find the valid AspnetMergePath) but unfortunately none of the fixes mentioned in those questions have worked for me.
Things I have already tried or were already the case:
Web and WebApplication directories from local machine (at C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\v14.0) have been copied up to build server (they've always been there)
Added
<TargetFrameworkSDKToolsDirectory>C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\</TargetFrameworkSDKToolsDirectory> to
Microsoft.Web.Publishing.AspNetCompileMerge.targets file
Added /p:AspnetMergePath="C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\" to MSBuild command
I'm at a bit of a loss with this one: pretty much the only thing I haven't tried is installing Visual Studio on the build server, but I'd really like to avoid that if I can (because I think it's ridiculous that you should have to install a full IDE on a CI server!).
Some further info:
everything continues to work just fine if you switch the build back to MSBuild 2013 (although, obviously, we don't want to get stuck in the past)
we have a number of other pure compilation builds that are running against MSBuild 2015 with no errors
Both the second and the third solution originally proposed on VS 2012 Publish: Can't find the valid AspnetMergePath actually do work, but the right version of the SDK tools has to be provided.
The original article (VS 2012 Publish: Can't find the valid AspnetMergePath) was written in 2013 and covers the Visual Studio 2012. The aspnet_merge.exe that supports C# 6.0 and .NET 4.6 is not the one that is found under the path "C:\Program Files (x86)\Microsoft SDKs\Windows\v8.1A\bin\NETFX 4.5.1 Tools\".
In order to get the third solution work, use the following path instead "C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6 Tools\".
In oder words, adding /p:AspnetMergePath="C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.6 Tools\" to MSBuild command solves the problem.
My team had exactly the same issue as the one you are describing and adding this parameter to the MSBuild step immediately fixed it. (I assume of course that you have the NETFX 4.6 version of the tools on your build machine.)
As a side note, since you said that you use Team City - instead of adding the above parameter directly as a command line parameter, you can use the Team City recommended and configure the corresponding build parameter.
The accepted answer is just symptom treatment and doesn't address the underlying issue. Please run MSBuild with diagnostic logging. If you look at the Microsoft.Web.Publishing.AspNetCompileMerge.targets you will see that it attempts to concatenate TargetFrameworkSDKToolsDirectory with AspnetMergeName. If you search in your MSBuild log, you will find that TargetFrameworkSDKToolsDirectory is empty. TargetFrameworkSDKToolsDirectory is build from TargetFrameworkSDKDirectory which is also empty.
That is what you need to fix. In my case:
Install the Windows 7.1 SDK (using this workaround on Windows Server 2016).
In VS2017 the path is determined based on /configuration/msbuildToolsets/toolset/property[#name="FrameworkSDKRoot] which is in my case Registry:HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SDKs\NETFXSDK\4.6.1#InstallationFolder. In my case this node was only available in the WOW6432Node, it needs to be available in the 64-bit registry too. You need to copy the node to the equivalent location in the 64-bit registry.
I am using vstest.console.exe to run a suite of integration tests. I am following the instructions here [http://blogs.msdn.com/b/visualstudioalm/archive/2012/12/06/publishing-test-results-through-command-line-test-runner.aspx][1]
Here is my command line (with added newlines for readability):
"E:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"
/Tests:UserIsAbleToLogIn
"E:\IntegrationTesting\Ads.Slms.IntegrationTesting.Web -Ver_20150206.1\Ads.Slms.IntegrationTesting.Web.Smartfill.dll"
/logger:TfsPublisher;Collection=http://xxxxx:8080/tfs/DefaultCollection;TeamProject=SLMS;BuildName="Ads.Slms.IntegrationTesting.Web -Ver_20150206.1"
However, despite the tool running fine, and reporting that the results have been published, I don't see any results appearing in TFS. I am using TFS2013. I have Visual Studio 2013 Professional installed on the server where my tests are being run.
Is there something I'm missing? Why would vstest report "Publish completed successfully", if I can't see the results in TFS?
Late to the party, but for those who experience the problem in the future, here's what worked for me.
Use the Platform and Flavor in the /Logger argument. Those will tell TFS which built project to associate your test with. By default Platform is 'Any CPU' and Flavor is 'Debug' - in all likelihood your build definition doesn't use that configuration.
Is it possible to obtain Code coverage numbers from VS 2012 Trial Version tool?
When I run my test cases (manually) after instrumentation, the .coverage file says my .pdb's may not be present or may be incorrect.
Is there something else i need to do?
For Instrumentation i do the Following:
use vsinstr with coverage flag and instrument the .dll and .exe
copy all .dll, .exe and .pdb file
I execute the test cases.
I am using Visual Studio ultimate 2012 Trial version downloaded from MSDN site
Try this :
tools ->options ->debugging->symbols->tick the check mark against microsoft symbol server
and click ok .