I'm working on a suite of application that interact with many websites and api's. I have created unit tests that test this code.
As it's beyond my control, I want to know as soon as sites / api's change and the code stops working so I can fix and would like to run the tests periodically against the assemblies running. How can I run the tests outside of VS?
I'd like the option of testing on Linux mono too.
You can use MsTest from the command line:
Example:
"C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\mstest" /testcontainer:C:\MyTestProject\bin\debug\MyTestProject.dll /resultsfile:C:\TestResults\results.trx
(Full list of command line switches on the link above)
This is set to Visual Studio 2015, for 2017 just make sure you have the right version number, I believe 15.0
MSTest has been deprecated, you'll want to invoke VSTest from the command line
Related
Neither the integrated test environment nor Resharper won't run any unit tests.
When starting Visual Studio the Test Output Pane shows 10+ lines stating:
An exception was thrown while initializing part "Microsoft.VisualStudio.TestWindow.Controller.TestPlatformProvider".
This is what I've tried so far:
Clear folder %LocalAppData%\Microsoft\VisualStudio\14.0\ComponentModelCache
Create a simple test project with one class library, one class and one test library containing one unit test for that class
Uninstallation and new installation of Visual Studio 2015
All actions didn't fix the problem. Even the simple test won't run.
Running tests from command line using MSTest does work.
Ensure that your anti-virus programs are not quarantining Visual Studio components.
I had this problem when our corporate IT department changed our Sophos Protection Quarantine policy. VS2015 Unit testing stopped working for everyone, with a similar error.
I'm putting together a new stack of unit tests to be run together as a CI job. I'm using vstest.console.exe instead of mstest.exe mainly for its ability to run tests from several frameworks, but right now the focus is a few xUnit dlls. The jobs are run as part of a Jenkins pipeline.
I have tested everything successfully on a couple of dev boxes, but annoyingly test discovery is not working on any of the CI build boxes so far. This is after the addition of the 0.99.8 xUnit test adapter vsix (also tested with 0.99.7). The xUnit dlls are being built against 4.5 with the 2.0.0.2378 beta nuget version of xUnit.
I have reproduced the symptoms with the simplest possible dll, with a single public test method, working fine on my own box, not working on any build boxes. The deployment enviroment is pretty straight forward, with a VS2012 install and the xUnit test adapter, on Windows 2012.
I have enabled the TpTrace logging via the vstest exe config files and everything looks fine. I guess I'm looking for a way to troubleshoot the issue further (maybe tracing for the xUnit discovery process) or a workaround the problem. I'd prefer to retain the use of vstest console for the simplicity of running several frameworks.
I have written up this issue via the xUnit codeplex site as well.
I've checked out this SO post but none of the suggested solutions make sense here.
It takes me quite some time to find the solution of how to use vstest.console.exe and xunit for VS2013, so I think it worth the time to explain here how I did for everyone...
The first step is to follow what is explained here to install the pre-release xunit.runner.visualstudio nuget package in the xunit projects that need it to be able to run the xunit test from Visual Studio.
Then, when you run your vstest.console.exe command, you MUST use the parameter /TestAdapterPath.
Your command line should look like something (the path toward the xunit adapter is here relative so, you could put it in absolute or adapt depending on the active directory):
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" c:\path\to\your\assembly.to.test.dll /TestAdapterPath:".\packages\xunit.runner.visualstudio.0.99.9-build1021\build\_common\"
edit: Because the adapter dlls are copied to the output folder, we could simplify the command line giving the path "." to the /TestAdapterPath option:
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" c:\path\to\your\assembly.to.test.dll /TestAdapterPath:"."
For information, it works also with NUnit, nuget package NUnitTestAdapter and the command:
"C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe" c:\path\to\your\assembly.to.test.dll /TestAdapterPath:"."
OK so problem solved, but after some frustrating troubleshooting that I will cover in case its useful for someone. The problem was that xunit.execution.dll was not available in the same folder as the dlls containing tests. This is required for xunit discovery. I only got here by:
Setting HKCU\Software\Outercurve Foundation\xUnit.net\Visual Studio Test Plugin\MessageDisplay = Diagnostic (this should be possible via the runsettings file but is not being picked up, and not possible via the VS tools options xunit page because its failing to open)
vstest now spits out 'Skipping xunitTests.dll (no reference to xUnit.net)'
This message actually means that xunit.dll and xunit.execution.dll weren't found in the folder
Problem solved by making sure that dll gets copied into that folder on the build boxes.
I migrated from Visual Studio 2010 to Visual Studio 2013 Express. In VS2010, the MSTest.exe used to run the tests with NAnt. As in VS2013 no longer has the MSTeste.exe, I'm using VSTest.Console.exe.
The problem is that in a test project, the VSTest.Console.exe reports that test run successfully, but does not terminate or continue the process. It seems that is locked.
In another test project works perfectly. Have rode by NAnt inside VS2013 Express and MS-Dos and got the same error. Because one ends and the other not?
VSTest not working:
vstest.console.exe C:\Daniel\DriveD\dotnet\IntegradorWS\Integrador.Senior.Tests\bin\x86\Release\Integrador.Senior.Tests.dll /Logger:trx
Image for MS-Dos:
VSTest working:
vstest.console.exe C:\Daniel\DriveD\dotnet\IntegradorWS\Integrador.Modelo.Tests\bin\x86\Release\Integrador.Modelo.Tests.dll /Logger:trx
Image for MS-Dos:
Are these Tests also Testing a GUI? Is there a MessageBox or another controll that requires a user interaktion? If yes, you should abstract the Controll, i.e. the MessageBox, into one MessageBox-Adapter thas is easy to be Mocked into your corresponding test.
How can I include Specflow tests into the Build process template for nightly build.
I use VS 2012 and our nightly build run on Server 2008,
And I have some PowerShell script for it, but I don`t know how use it.
What you are describing sounds incredibly vague. Does your powershell script run the build and test parts of your build?
Specflow will be run automatically as part of your nunit or mstest test runs. If you already have these and you are presenting the assemblies to the runner, it just happens.
personally I would suggest the easiest way is to download a copy of TeamCity, which if I remember correctly will be a free licence for just a single project. Just add your source code repo details, point a Visual studio step at your solution, a unit test step at your test dlls and a trigger for whenever you checkin and will all happen automatically.
I'm new to NUnit and I just download it today and I can't get it working with visual studio. (I'm new to Visual Studio as well)
I'm following TekPub's Mastering C# 4.0 tutorial.
Screenshot: Something is missing within the red circle area. There should be some "green thing" on the side that let you double click on it and run the NUnit test.
http://dl.dropbox.com/u/20422001/NunitMissing.jpg
As you can see on the above screenshot, the NUnit seems not working....
Did I miss something? Please advice. Thanks
EDIT
This is how it SHOULD look like. I took a screenshot from the video tutorial:
http://dl.dropbox.com/u/20422001/nunit.jpg
and here is the project files:
http://dl.dropbox.com/u/20422001/MasteringCSharp.rar
As you can see, I don't have "Unit Test" Tab and "Unit Test Session"...
You aren't doing anything wrong. Out of the box, NUnit doesn't integrate directly with Visual Studio. The screenshot from the video appears to be using Resharper's test runner.
Typically testing with NUnit requires you to compile your test project and then load it into the NUnit GUI (nunit.exe) where the tests can be run from there. When using the NUnit GUI, the project will reload anytime the assembly is recompiled. (I sometimes prefer having this in a separate process as it doesn't tie up Visual Studio's UI, but that's a matter of preference.)
The added advantage of using a third party test runner like Resharper, TestDriven.net, Galileo, etc is that it allows you to run or debug the tests directly from within the IDE. You can accomplish the same thing using NUnit by configuring Visual Studio to attach to NUnit and debug your tests from the external process. This post shows you how.
I haven't seen that particular video, but it sounds like you need an NUnit test runner for Visual Studio.
Check out the following:
Visual NUnit 2010 - free!
TestDriven.net
ReSharper
That my friend is JetBrains' Resharper - it's a paid add-in to Visual Studio that allows you to run tests from within the IDE. Highly recommended if you're working for a prolonged period in VS.
Of course since you're just starting out, you could use the NUnit GUI to run your tests. You can also set it to run tests after each build.
Tools > Settings > Test Loader > Assembly Reload. Check "Re-Run last tests run". Apply changes