I have a project, composed of a 3 C#/.NET Core class libraries, targeting .NET Standard 2.0. In addition, I have a XUnit test project, where I use FluentAssertions.
When working on the project, I use Visual Studio 2017 Enterprise, with Resharper Ultimate installed.
When running the full suite of tests, my SQL Server 2016 Express database is hit around 8100 times.
If I leave my profiler running while working in Visual Studio, I can see that the full suite of tests is executed, whenever I change code and do a build.
It's a bit of a nuisance, since it takes a good 1-1½ minute for the tests to finish, and often, I will only want to run a specific subset of the tests, when making incremental changes.
My problem: I can't seem to stop the tests running from running on build.
What puzzles me the most is, that I can't cancel the tests from Test explorer or Resharpers Test window, as they appear not to be running at all.
Also, the Test pane of the output window does not show anything except "....Discovering unittests" without progressing any further.
My impressions is, that from Visual Studios perspective, the tests are not running at all.
Getting the primary suspects out of the way:
The "Run tests after build" button in test explorer is not toggled
on.
Live unit testing appears to be disabled (The only option from the
menu is "Start")
I've spent hours browsing the net trying to figure out what is going on, with no luck. I'm hoping someone here can help me find out why the tests are running, and how to stop them.
This is my first ever StackOverflow question. I hope my questions meets the requirements for asking. If not, please let me know what to do better.
First I would advise you update VS. There was an issue in test discovery taking long time that was fixed in update 15.7 (See here)
If that doesn't do the trick I advise you try disabling this option in Tools > Options > Test:
As this might be attempting to analyse all assemblies in build output folder to discover tests.
Related
We've been running unit tests in parallel for several years (both with and without code coverage), following the instructions here:
How to run unit tests (MSTest) in parallel?
and here:
Running unit tests in parallel
However, recent updates to Visual Studio 2017, (versions 15.5.0 or 15.5.1 and later) seem to have broken this, and I get the following error now whenever I try to use a custom test settings file and also do a code coverage analysis:
[12/29/2017 1:35:10 PM Error] Microsoft.VisualStudio.TestPlatform.ObjectModel.SettingsException: Data collectors configured via run settings are not supported with embedded test settings. Please see https://aka.ms/vstest-configure-datacollector for more information. Run settings:
I have updated, rebooted, and verified this issue on another system.
When reverting to default settings (no custom test settings file) and using the 'Run Tests in Parallel' button introduced in VS 2015 update 2, it seems to simply ignore the button setting and run the tests serially (whether you do code coverage or not).
I looked into the link provided in the error (https://github.com/Microsoft/vstest-docs/blob/master/docs/analyze.md), but it did little to help clarify the issue, other than possibly indicate that yes, a change was made.
If I turn off the custom settings file, the tests run just fine, but they take about 7x as long to run, which is very frustrating.
How does one get parallel unit test execution working in VS 2017 15.5+?
Is there any way of running tests automatically after each build?
I know this is possible in 2013 Premium/Ultimate (link) but sure there are ways
to do this in the Community edition?
I'm using Nunit + adapter so it would be nice to have tests run through
the adapter instead of using post-build command-line events..?
(which I can figure out myself tbh)
IF I need to write a vsix, no problem, I'm capable of that too. Just need
a pointer in the right direction? If I should need to write one, I would
monitor files changed before build and only run the needed tests.
This question is very similar to this one, but I'm experiencing it in Visual Studio 2010 SP1. I am running my tests locally (there is no remote test controller).
I recently started working on a client's .NET 4.0 project. It has a suite of Visual Studio Unit Testing Framework unit tests. When I launch the unit tests, IIS Express automatically starts. I can run the unit tests dozens of times without any issues to see whether they pass or fail. However, when I attempt to debug the unit tests (by right-clicking on a test in Test View and clicking Debug Selection), it almost always fails with the following error message:
Error 3/19/2014 2:42:44 PM Failed to queue test run 'ThoughtProcess#HOSTNAME 2014-03-19-14:42:40': Process with an Id of [12345] is not running.
I say "almost" because every 20 or so clean starts of VS, it will successfully attach the debugger. But I haven't been able to consistently reproduce this behavior.
I tried the suggestions from the Visual Studio 2012 post, but those didn't work in my situation. Has anyone else encountered this issue?
In the spirit of this xkcd comic, I did find a solution to this issue by installing ReSharper and using its test runner instead of the built-in Visual Studio test runner.
The ReSharper runner consistently and repeatably lets me run and debug unit tests without any issues. I'm still not sure that I understand the underlying cause of why the VS runner can't debug, but hopefully this will save someone else future frustration.
We are working with Visual Studio 2008 in a solution with 128 projects, almost half of them are unit test projects which make a total number of more than 5000 unit tests. For that reason we usually leave all unit test projects unloaded except those ones we are working with. Nevertheless, we often suffer from having Visual Studio 2008 blocked whenever we try to launch a single unit test or load/unload a different unit test project. In Task Manager we see that devenv.exe process takes constantly 25% of CPU and through Process Explorer we have seen that devenv.exe is in a loop of CreateFile, QueryBasicInformationFile and CloseFile for each project file loaded in the solution, iterating through all project files over and over again.
We have tried to set in the registry the key HKLM\Software\Microsoft\VisualStudio\9.0\EnterpriseTools\QualityTools\EnableCMI to 0 as it is stated in some forums where we have searched for a solution or workaround, but with no luck.
As a workaround, we are working with solutions that contain only a small subset of projects, but we don´t see it as a definite solution as we don´t consider very 'developer-friendly' to be switching between different solutions and different instances of VS2008 all the time.
Anyone who had experienced this same behavior could give us any clue on how to prevent Visual Studio 2008 from being unresponsive for so much time when dealing with unit tests? It makes developing unit tests almost impossible.
Many thanks in advance.
Jose Antonio
Consider upgrading Visual Studio. 2008 is ancient and it had serious performance issues in large solutions. 2012 was better, but IIRC a lot of work was done in 2013 to make it workable with a lot of projects.
128 projects in one solution in 2008 times is - ah - challenging it. Not to say totally over the top by factors.
With 2013 it should actually work
There is not a lot you can do here in fact - the problems with a significant number of projects in older versions of Visual Studio are simply documented and were fixed a long time ago - in newer versions.
I have looked around SO and the web for quite some time now and I am struggling to find any information for how to setup my C# project for unit tests.
I started on the NUnit main site http://www.nunit.org, the download includes a GUI to run your unit tests, sounds mint. Only when I try and point the GUI to my .exe, I get an error immediately saying that NUnit is not designed to be run with this framework. That got me thinking that maybe NUnit doesn't work with .net 4. Apparently it does, plenty of people online seem to state that they are running NUnit unit tests with .net 4 .... of course they don't seem to go on to say how.
Clicking around google I found what appears to be one of the core members of the NUnit development is a dude called Charlie Poole. I found he had a launchpad here: https://launchpad.net/nunit-xtp. The version numbers are newer than on the main NUnit site. When I downloaded the project however, it is just the codebase and solution for the NUnit framework with no GUI. There is a command line runner included but I really need something visual.
We do have jetbrains teamcity as a BMS so I am now going down the path of maybe trying to integrate NUnit into the build pipeline somehow. If anyone can shed any light on how they got NUnit working with an express edition of c# that would be wonderfully helpful.
Edit : I have the teamcity BMS running the tests now, so that part is no longer a problem. However, waiting for the build each time just to find a NullException error or something doesn't really cut it. I have to get the GUI runner working. I found this question NUnit isn't running Visual Studio 2010 code but even that isn't fixing it for me. The error has now updated to read version v4.0.30319, so the config changes have taken effect. Same old error though :(
Right I sussed it out. there is a very similar question here:
NUnit isn't running Visual Studio 2010 code
Only what it doesn't mention is that there are multiple config files for NUnit. I just assumed it would suffice to edit nunit.exe.config. This did indeed update the settings as I saw it in the error message. However, the nunit gui runner must make use of one of the other multiple .config files. I simply went through them all and added the two required settings.
Make sure that you run the correct .exe. One is called "nunit.exe" and it seems like the correct one to use. The other one is called "nunit-x86.exe" and if your app is 32bit then you have to run this one. If "nunit.exe" was names "nunit64.exe" I think I would have noticed that much sooner.
You must also make sure that you check the version number of the CLR matches the setting you are pasting into those config files. Use this link to work out your current CLR version number. How do I find out what CLR i am using? I am Microsoft Visual C# 2010 Express and my CLR version number is v4.0.30319
The unit tests should be in a seperate assembly (of type class library). If you point NUnit to that assembly, it will run. Same applies to the command line runner (which is normaly used in automated build processes).
The default NUnit distro includes a GUI runner called 'nunit.exe'. If using it, I set my debug target on my test assembly to load the GUI runner with the appropriate config.