Is there any way of running tests automatically after each build?
I know this is possible in 2013 Premium/Ultimate (link) but sure there are ways
to do this in the Community edition?
I'm using Nunit + adapter so it would be nice to have tests run through
the adapter instead of using post-build command-line events..?
(which I can figure out myself tbh)
IF I need to write a vsix, no problem, I'm capable of that too. Just need
a pointer in the right direction? If I should need to write one, I would
monitor files changed before build and only run the needed tests.
Related
I have a project, composed of a 3 C#/.NET Core class libraries, targeting .NET Standard 2.0. In addition, I have a XUnit test project, where I use FluentAssertions.
When working on the project, I use Visual Studio 2017 Enterprise, with Resharper Ultimate installed.
When running the full suite of tests, my SQL Server 2016 Express database is hit around 8100 times.
If I leave my profiler running while working in Visual Studio, I can see that the full suite of tests is executed, whenever I change code and do a build.
It's a bit of a nuisance, since it takes a good 1-1½ minute for the tests to finish, and often, I will only want to run a specific subset of the tests, when making incremental changes.
My problem: I can't seem to stop the tests running from running on build.
What puzzles me the most is, that I can't cancel the tests from Test explorer or Resharpers Test window, as they appear not to be running at all.
Also, the Test pane of the output window does not show anything except "....Discovering unittests" without progressing any further.
My impressions is, that from Visual Studios perspective, the tests are not running at all.
Getting the primary suspects out of the way:
The "Run tests after build" button in test explorer is not toggled
on.
Live unit testing appears to be disabled (The only option from the
menu is "Start")
I've spent hours browsing the net trying to figure out what is going on, with no luck. I'm hoping someone here can help me find out why the tests are running, and how to stop them.
This is my first ever StackOverflow question. I hope my questions meets the requirements for asking. If not, please let me know what to do better.
First I would advise you update VS. There was an issue in test discovery taking long time that was fixed in update 15.7 (See here)
If that doesn't do the trick I advise you try disabling this option in Tools > Options > Test:
As this might be attempting to analyse all assemblies in build output folder to discover tests.
I am trying to find some free way to run NUnit tests from VS2013. I tried the NUnit test adapter (both VSIX and NuGet package, but it doesn't seem to work (there the tests appear in the Test Explorer but are all grayed-out and can't be run). Is there something I need to do to get around this error/is there another extension out there that does this?
I am using NUnit 2.6.3.
Are you trying to run these tests for a Windows 8 application? It appears to be incompatible - or at least was when I last tried - I posted a similar thread as I was running into the same problem, and the posted solution did not work for me as I had a mix of W8 and older class libraries: Windows 8 App in Visual Studio 2012 - NUnit Test Adapter sees unit tests but does not run them
How can I include Specflow tests into the Build process template for nightly build.
I use VS 2012 and our nightly build run on Server 2008,
And I have some PowerShell script for it, but I don`t know how use it.
What you are describing sounds incredibly vague. Does your powershell script run the build and test parts of your build?
Specflow will be run automatically as part of your nunit or mstest test runs. If you already have these and you are presenting the assemblies to the runner, it just happens.
personally I would suggest the easiest way is to download a copy of TeamCity, which if I remember correctly will be a free licence for just a single project. Just add your source code repo details, point a Visual studio step at your solution, a unit test step at your test dlls and a trigger for whenever you checkin and will all happen automatically.
I have looked around SO and the web for quite some time now and I am struggling to find any information for how to setup my C# project for unit tests.
I started on the NUnit main site http://www.nunit.org, the download includes a GUI to run your unit tests, sounds mint. Only when I try and point the GUI to my .exe, I get an error immediately saying that NUnit is not designed to be run with this framework. That got me thinking that maybe NUnit doesn't work with .net 4. Apparently it does, plenty of people online seem to state that they are running NUnit unit tests with .net 4 .... of course they don't seem to go on to say how.
Clicking around google I found what appears to be one of the core members of the NUnit development is a dude called Charlie Poole. I found he had a launchpad here: https://launchpad.net/nunit-xtp. The version numbers are newer than on the main NUnit site. When I downloaded the project however, it is just the codebase and solution for the NUnit framework with no GUI. There is a command line runner included but I really need something visual.
We do have jetbrains teamcity as a BMS so I am now going down the path of maybe trying to integrate NUnit into the build pipeline somehow. If anyone can shed any light on how they got NUnit working with an express edition of c# that would be wonderfully helpful.
Edit : I have the teamcity BMS running the tests now, so that part is no longer a problem. However, waiting for the build each time just to find a NullException error or something doesn't really cut it. I have to get the GUI runner working. I found this question NUnit isn't running Visual Studio 2010 code but even that isn't fixing it for me. The error has now updated to read version v4.0.30319, so the config changes have taken effect. Same old error though :(
Right I sussed it out. there is a very similar question here:
NUnit isn't running Visual Studio 2010 code
Only what it doesn't mention is that there are multiple config files for NUnit. I just assumed it would suffice to edit nunit.exe.config. This did indeed update the settings as I saw it in the error message. However, the nunit gui runner must make use of one of the other multiple .config files. I simply went through them all and added the two required settings.
Make sure that you run the correct .exe. One is called "nunit.exe" and it seems like the correct one to use. The other one is called "nunit-x86.exe" and if your app is 32bit then you have to run this one. If "nunit.exe" was names "nunit64.exe" I think I would have noticed that much sooner.
You must also make sure that you check the version number of the CLR matches the setting you are pasting into those config files. Use this link to work out your current CLR version number. How do I find out what CLR i am using? I am Microsoft Visual C# 2010 Express and my CLR version number is v4.0.30319
The unit tests should be in a seperate assembly (of type class library). If you point NUnit to that assembly, it will run. Same applies to the command line runner (which is normaly used in automated build processes).
The default NUnit distro includes a GUI runner called 'nunit.exe'. If using it, I set my debug target on my test assembly to load the GUI runner with the appropriate config.
I can do so using MbUnit framework, but I cannot force everyone to install it (please do not ask); everybody already has mstest.exe as well as the library.
http://www.gallio.org/oldwiki/MbUnit/TestExecutionUsingSelfExecutableTestAssemblies.html
We do not even run unit tests yet, so the easiest way to cram it in is to reuse an existing project. If it gets harder than that, then we will likely not have a test coverage for a while.
Looking forward to gaining some faith in MS Test.
There is no way to run MSTest without Visual Studio. It is designed to run using the IDE or using a Build server that had Visual Studio installed.