I am using vstest.console.exe to run a suite of integration tests. I am following the instructions here [http://blogs.msdn.com/b/visualstudioalm/archive/2012/12/06/publishing-test-results-through-command-line-test-runner.aspx][1]
Here is my command line (with added newlines for readability):
"E:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"
/Tests:UserIsAbleToLogIn
"E:\IntegrationTesting\Ads.Slms.IntegrationTesting.Web -Ver_20150206.1\Ads.Slms.IntegrationTesting.Web.Smartfill.dll"
/logger:TfsPublisher;Collection=http://xxxxx:8080/tfs/DefaultCollection;TeamProject=SLMS;BuildName="Ads.Slms.IntegrationTesting.Web -Ver_20150206.1"
However, despite the tool running fine, and reporting that the results have been published, I don't see any results appearing in TFS. I am using TFS2013. I have Visual Studio 2013 Professional installed on the server where my tests are being run.
Is there something I'm missing? Why would vstest report "Publish completed successfully", if I can't see the results in TFS?
Late to the party, but for those who experience the problem in the future, here's what worked for me.
Use the Platform and Flavor in the /Logger argument. Those will tell TFS which built project to associate your test with. By default Platform is 'Any CPU' and Flavor is 'Debug' - in all likelihood your build definition doesn't use that configuration.
Related
I'm trying to establish continuous integration with TeamCity and stuck at running unit tests step with Visual Studio Tests runner.
Loading [02:42:07]C:\TeamCity\buildAgent\work\f7a05248f49c57a1\LodCore\UserPresentationTests\bin\Release\UserPresentationTests.dll...
[02:42:07][Step 3/3] Starting execution...
[02:42:07][Step 3/3] No tests to execute.
All tests are discoverable, when I build test assemblies from Visual Studio.
I've downloaded UserPresentationTests.dll from build agent to my computer and tried to execute same command with my local MSTest.exe, and it failed with the same issue.
I've decompiled it with ILSpy and found no difference with dll built from Visual Studio.
I've tried different build configurations on MSBuild, but it is not working.
My TeamCity version: Professional 2017.2 (build 50574)
If you are using MSTest v2, then you should use VSTest instead of MSTest:
Can you check that there aren't any differences between the environments in terms of installed libraries, permissions, etc.
We have a couple pointers to troubleshoot the differences between a working local build and a failing TeamCity one, you might want to follow those steps: https://confluence.jetbrains.com/display/TCD10/Common+Problems#CommonProblems-BuildfailsorbehavesdifferentlyinTeamCitybutnotlocally
With this in mind, while the step 3 is the one pointing at the failure, I'd also check the build log of step 2, as it might have skipped some of the build parts for some reason.
Also it would be helpful if you could also add to the OP the TeamCity version you are using and the type of runner you are using in step 3.
In addition of build configuration your solution you should create new configuration with Runner type MSTest.
Example here
I'm working on a suite of application that interact with many websites and api's. I have created unit tests that test this code.
As it's beyond my control, I want to know as soon as sites / api's change and the code stops working so I can fix and would like to run the tests periodically against the assemblies running. How can I run the tests outside of VS?
I'd like the option of testing on Linux mono too.
You can use MsTest from the command line:
Example:
"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\mstest" /testcontainer:C:\MyTestProject\bin\debug\MyTestProject.dll /resultsfile:C:\TestResults\results.trx
(Full list of command line switches on the link above)
This is set to Visual Studio 2015, for 2017 just make sure you have the right version number, I believe 15.0
MSTest has been deprecated, you'll want to invoke VSTest from the command line
i have SonarQube 5.1 and i need to use FxCop rules but when i add the Parameter the sonar.cs.fxcop.assembly in TeamCity SonarQube runner it return errors
"The property "sonar.cs.fxcop.assembly" must be set and the project must have been built to execute FxCop rules....etc..."
So we try to manually added the single assembly in Deprecated TAB in SonarQube sites(localhost:9000) and it works but we have multiple assembly that need to analyze so our question now is how to anaylze multiple assemly in sonarqube 5.1 is there someone know how to make this happen?
Please use the MSBuild SonarQube Runner to analyze your C# project as documented on http://redirect.sonarsource.com/plugins/csharp.html
The MSBuild SonarQube Runner has been developed jointly with Microsoft, and fully integrates with MSBuild, solving many of the integration issues we were previously facing with the C# plugin. For example, FxCop now gets executed during the build by MSBuild, with all the right parameters.
You shouldn't have to manually set the sonar.cs.fxcop.assembly property, nor any of the other properties listed under the Deprecated tab.
You should run the commands from a "Developer Command Prompt for Visual Studio 2013 or higher" (http://docs.sonarqube.org/display/SONAR/Analyzing+with+SonarQube+Scanner+for+MSBuild) in order for the variables to be set.
I have just done a fresh install of
- SonarQube Scanner for MSBuild 4.2.0.1214
- Visual Studio Community 2015
and I am getting "The property "sonar.cs.fxcop.assembly" must be set and the project either running from windows cmd line or running from a developper command prompt
the sonarQube server is Version 5.6.2
I'm putting together a new stack of unit tests to be run together as a CI job. I'm using vstest.console.exe instead of mstest.exe mainly for its ability to run tests from several frameworks, but right now the focus is a few xUnit dlls. The jobs are run as part of a Jenkins pipeline.
I have tested everything successfully on a couple of dev boxes, but annoyingly test discovery is not working on any of the CI build boxes so far. This is after the addition of the 0.99.8 xUnit test adapter vsix (also tested with 0.99.7). The xUnit dlls are being built against 4.5 with the 2.0.0.2378 beta nuget version of xUnit.
I have reproduced the symptoms with the simplest possible dll, with a single public test method, working fine on my own box, not working on any build boxes. The deployment enviroment is pretty straight forward, with a VS2012 install and the xUnit test adapter, on Windows 2012.
I have enabled the TpTrace logging via the vstest exe config files and everything looks fine. I guess I'm looking for a way to troubleshoot the issue further (maybe tracing for the xUnit discovery process) or a workaround the problem. I'd prefer to retain the use of vstest console for the simplicity of running several frameworks.
I have written up this issue via the xUnit codeplex site as well.
I've checked out this SO post but none of the suggested solutions make sense here.
It takes me quite some time to find the solution of how to use vstest.console.exe and xunit for VS2013, so I think it worth the time to explain here how I did for everyone...
The first step is to follow what is explained here to install the pre-release xunit.runner.visualstudio nuget package in the xunit projects that need it to be able to run the xunit test from Visual Studio.
Then, when you run your vstest.console.exe command, you MUST use the parameter /TestAdapterPath.
Your command line should look like something (the path toward the xunit adapter is here relative so, you could put it in absolute or adapt depending on the active directory):
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" c:\path\to\your\assembly.to.test.dll /TestAdapterPath:".\packages\xunit.runner.visualstudio.0.99.9-build1021\build\_common\"
edit: Because the adapter dlls are copied to the output folder, we could simplify the command line giving the path "." to the /TestAdapterPath option:
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" c:\path\to\your\assembly.to.test.dll /TestAdapterPath:"."
For information, it works also with NUnit, nuget package NUnitTestAdapter and the command:
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" c:\path\to\your\assembly.to.test.dll /TestAdapterPath:"."
OK so problem solved, but after some frustrating troubleshooting that I will cover in case its useful for someone. The problem was that xunit.execution.dll was not available in the same folder as the dlls containing tests. This is required for xunit discovery. I only got here by:
Setting HKCU\Software\Outercurve Foundation\xUnit.net\Visual Studio Test Plugin\MessageDisplay = Diagnostic (this should be possible via the runsettings file but is not being picked up, and not possible via the VS tools options xunit page because its failing to open)
vstest now spits out 'Skipping xunitTests.dll (no reference to xUnit.net)'
This message actually means that xunit.dll and xunit.execution.dll weren't found in the folder
Problem solved by making sure that dll gets copied into that folder on the build boxes.
I have just updated from VS 2010 to VS 2012.
I have installed specFlow 1.9 and NuGet package.
I have also installed specflow package through Manage NuGet packages.
Also configure my app.config to Ms test. Added all the required dll's and genereted the feature file as well as my .cs files.
But still im not able to run or Debug my the specFlow Tests.
While the same settinngs worked on VS 2010. And pressing F5 button also does not work.
Is there any Step that I am missing here or any setting that needs to be done prior to running the tests.
I had the same problem. Found this post on the SpecFlow google group. Setting ReSharper as test runner tool made it work for me. I'm running SpecFlow 1.9.3.1 on Visual Studio 2013.
https://groups.google.com/forum/#!msg/specflow/sb38tqgDaq4/pAeprtPu_PcJ