I encounter a problem running my tests via NUnit. I dont know why but each test runs twice. The point is that on another laptop it normally runs only one time.
Have anyone the same problem and know how to deal with it?
I had the same problem. In my case, I had both the Visual Studio extension and the NuGet package of the NUnit Test Runner installed. It seems they were both executing the tests, resulting in tests being run twice.
Uninstalling the Visual Studio extension solved the issue.
I had a similar problem where running the test with the Visual Studio NUnit3 runner, the test would run only once but when running in the command line with NUnit3-Command.exe, the test would run twice. I spent way too long chasing this down. I noticed that the command line runner was reporting 2 tests were run and came to the conclusion it was an issue in the dll. This may sound crazy but I copied all my test code into a new class, deleted the old, compiled and the test now runs just once in both scenarios.
In my case, it was having a base class and a derived class both having the TestFixture Attribute. All tests from the derived class would run twice.
Fixed it by only putting attribute on base class.
The problem typically arises when you load a test container (.dll) multiple times.
This can happen if you load the test containers from a root floder that has both bin and obj folders inside, which is a typical setup of Visual Studio projects.
Try to limit your test container loading from the bin folders.
The running of test can also be duplicated is when sequential tests accidentally have the same test parameters.
[Test, Sequential]
public void Integr_MController_DeleteGet_ReturnsViewAndModel(
[Values("Do#london.com", "Do#london.com")] string firstEmail,
[Values("Zyg#london.com", "Zyg#london.com")] HTTPstring updatedEmail) {test code}
Will run the same test twice and will show up in TestExplorer as the same test.
In case you put the *.dll twice, the test get double executed.
Seen on NUnit.3.10.1, nunit3-console 3.9.0
Example:
nunit3-console myTestDll.dll --test=MyNameSpace.Test myTestDll.dll** --result=myResult.xml
Related
I'm struggling to get my unit tests to work. I've wrestled with this issue for several hours now, and I have no explanation as to why things don't work. I had a fairly major refactor on my codebase and have since gone through and fixed all the unit tests. The test project builds, it outputs a new unit test dll. However, when I go to run the tests in the test explorer, I get this message:
[2/27/2019 5:08:05 PM Warning] [MSTest][Discovery][C:\pathtotest.dll] Failed to discover tests from assembly C:\pathtotest.dll. Reason:Could not load file or assembly 'System.Runtime, Version=4.1.2.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
[2/27/2019 5:08:05 PM Warning] No test matches the given testcase filter FullyQualifiedName=<namespace.namespace...testmethod> in C:\pathtotest.dll
Here is what I know:
I recently updated Visual Studio (within the past two weeks, don't remember exactly when I did it).
All packages in the test project have been updated and are running the latest versions of things.
I have another unit test project that is .Net Core 2.1, this is .Net Framework 4.7.2. The other project works.
Some suggestions other posts give are to make sure your test architecture is correct and to delete a folder in %TEMP% (don't recall the exact name, except is was something about VisualStudioExtensions). The folder they suggest to delete was not found in %TEMP% and I tried running my tests on both architectures with the same result.
So the next step was to take a sanity test and make sure the built test dll exists. It does.
At this point I'm about ready to just start a new test project and copy paste over all the tests one by one and see if maybe one is throwing a silent error. I can't find any useful information with my own Google-fu skills and I'm hoping someone has some useful insight or tricks to try.
From the comment above:
Using this C:\pathtotest.dll
Check also if it 32bit or 64bit runtime.
Most cases are using 32bit dlls.
Hope it helps
I had a similar issue with a brand new test project in an existing solution - all of my other test projects compiled and tested correctly, but the new one repeatedly came up with the error:
No test matches the given testcase filter...
The answer came from this post VSTest: A testsettings file or a runsettings with a ForcedLegacyMode set to true is not supported with the MSTest V2 Adapter. No test is available which suggested switching off the testsettings file, which somehow had been selected on the new project.
As soon as I deselected the Test -> Test Settings -> c:...\Repos...testsettings option in Visual Studio the tests were runnable.
In my case, the issue ended up being a mix of two different test engines. I inadvertently decorated my test methods with [TestMethod] (MSTest) when I was using NUnit. Once I changed my test methods to just [Test] and used the proper test runner, I was finally in business.
I had a similar issue. In my case, the problem was the return type.
The test method needed to use async but the return type was void instead of Task. Simple and small think that might eat a lot of time.
I am running C# specflow tests from visual studio. When I run I am seeing tests are getting executed parallel. How can I make them to run one after another ? I am attaching snapshot which shows in cyan highlighted. I am looking a way to run them one after another. I am using Specflow.AssistDynamic and Techtalk.Specflow
I am able to do this by adding a property to AssemblyInfo.cs
[assembly: CollectionBehavior(DisableTestParallelization = true)]
SpecFlow generates unit test code. so you need to check which xUnit framework you're using in that project.
See this link: https://github.com/techtalk/SpecFlow/wiki/Parallel-Execution
If you are using NUnit, then remove/comment this line from Assembly.cs:
[assembly: Parallelizable(ParallelScope.Fixtures)]
Also, as I remember, in a Visual Studio, in Test explorer, present button - Run tests parallely. Unclick it.
In my project, to do this, I removed this line, and also called test scenarios like: A_A, A_B, in alphabet. In this case they are going one after one.
I am running Visual Studio 2017 & the Test Runner cannot discover the Unit Tests unless Visual Studio is running as an Administrator. Running as Administrator is next to impossible at my new workplace.
As such...
Q: How can get the Test Explorer to discover Unit Tests without being an administrator?
UPDATE:
I have confirmed the following...
POSITIVE: The Unit Test Project is set to build
POSITIVE: The Test Project is a genuine Unit Test Project (w/ magic guids)
NEGATIVE: The Test View is not an available option in VS2017 Community Edition
A summary of various fixes for this no tests found issue.
You have 2 versions of the Microsoft.TestAdapter installed (reference Microsoft.VisualStudio.TestPlatform.TestFramework and Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions). Check NuGet Package Manager, version 1.1.11 doesn't work, you need to use 1.1.18 (at time of writing for it to work).
Also check your Bin folder &/or your Build Agents references folder for duplicate versions.
Make sure that your test project is set to build. If the test assembly isn't being built, VS won't be able to find any tests:
Magic GUIDs are needed in the Test Project:
C#:
{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}
VB:
{3AC096D0-A1C2-E12C-1390-A8335801FDAB};{F184B08F-C81C-45F6-A57F-5ABD9991F28F}
Trying to get ignored tests running again. Same error message occurs if you remove the Ignore label. Does not automatically re-enable the test. This article takes you through the last step. http://richallen.blogspot.com/2008/05/ms-test-re-enabling-ignored-tests.html
Ie
Open the Test List Editor, selected the ignored test, right click and select "Enable" from the context menu.
Make sure your method belongs to a class with the [TestClass] Attribute and the method it's marked with the [TestMethod] Attribute.
Use Test View to find your test.
Open your Properties window (F4), and make sure your test is enabled
Some people have accidentially added a Class Library project instead of Test Project. Running Tests in a Class Library doesn't work, it has to be a Test Project, probably because of the Magic GUIDs.
Check whether the solution is running on 64bit. If so change it to x86.
Marking tests as static makes them fail to appear in the test list.
Close down Visual Studio and delete the .vsmdi (visual studio test meta data) file. This will be regenerated.
I'm working with Unit Test using VS2013 Professional. In particular I'm using the NUnit framework (NUnit TestAdapter for VS2013). My problem is that when I run my tests then VS starts building all the projects inside the solution. Currently the Unit Test project does not reference any solution project.
If I simply code a single test method like:
[Test]
public void SimpleTestMethod(){
Assert.That("a", Is.EqualTo("a"));
}
and the Unit Test project is in a Solution with N project, when I run my test then VS will build all N-1 project... In my case this behavior is boring because it takes too much time (the solution contains many projects) and some projects contain errors.
Is there a way to run my SimpleTestMethod() without complete solution building?
Break your test project to multiple projects that only reference a subset of the solution's projects.
This is also good test housekeeping - have a separate unit test project for each solution project instead of one huge project with dependencies to anything else. There are several advantages to this:
Tests run faster
It's a lot easier to isolate test cases, especially configuration settings
You can version projects and their test cases together
A good naming practice is to name your test projects the same as their target projects with a .Tests suffix. You can also create a solution folder (not a real folder) called "Tests" and move the test projects in it.
As for the why: Test runners use the Unit Test assembly and its dependencies to run their tests. If any of the assembly's dependencies change, the assembly and the dependencies have to be rebuilt. Visual Studio doesn't know what the external tool will call so it has to build all changed assemblies and their dependents.
If the build fails, there are no valid assemblies for the test runner to use so VS has to rebuild the entire solution before the runner can work. In this case, the obvious solution is to fix the error.
There are some stopgap options you can use until you can fix the error:
Temporarily remove the broken project from the build configuration
Split the solution so that you have a solution that can be built and tested
I struggled with this for a very long time as well. I actually hated the automatic build process, even when everything was successful.
I started to run tests through the command line. No build process is necessary. You can write your own .bat files and keep logs of test results. There are plenty of command line parameters that can be added to customize for what you are looking for.
https://msdn.microsoft.com/en-us/library/jj155796.aspx
Is it possible to let Resharper (or NUnit?) know that I want each test to look for an App.config under it's own project, even when running all tests in a solution together?
Background:
I'm using NUnit and the test-runner that ships with Resharper, and I've got several test-projects in the same solution. Some of my tests depend on config-files located under their respective projects.
When I run a test-project by itself, it will use it's internal App.config, and everything works fine. When I try to run all the tests in the solution, or use the shortcut to run all tests in current test session however, no config-file will be selected, and any test depending on a config will fail by default.
For this reason, I typically end up running all tests in the solution once first, then right clicking the nodes in the test-runner for each of the config-dependent projects, and running them separately afterwards.
Solved:
Apparently assemblies containing NUnit-tests can be run in separate processes or domains using command line options.
For the testrunner under Resharper, this setting can be found under Resharper > Options > Unit Testing.
There is an option "Use separate AppDomain for each assempbly with tests". Checking that solved my problem.