We Keep Coding

Is it possible to exclude a Test Project when running 'Run All' in Test Explorer

I have some Specflow(3.9.40)-based UI test using Selenium Webdriver and NUnit(3.13.2) Framework.
I would like these tests to not run when I hit 'Run All' in the Test Explorer.
In my StepDefinition class I have tried adding the Explicit Attribute:
[TestFixture, Explicit("Only run when called explicitly")]
[Binding]
public class LoginStepDefinitions
{ ...
, but for some reason it does not seem to work - The UI Tests still run.
I imagine there may be something I can add to the [BeforeScenario] in my HookInitialization class, but I am stumped:
[BeforeScenario]
public void FirstBeforeScenario()
{
var seleniumDriver = new SeleniumDriver(_scenarioContext);
_scenarioContext.Set(seleniumDriver, "SeleniumDriver");
}
Is there a way for me to tell Visual Studio Test Explorer to ignore the UI Tests or its folder ?

The basic problem is that NUnit's Explicit setting is unique among the frameworks supported by Test Explorer, which knows nothing of this concept of "explicitness."
If Test Explorer calls the NUnit adapter telling it to run all, then NUnit will honor the Explicit setting. However, in some situations, it simply passes a list of all the tests to NUnit. In that case, all tests appear (to NUnit) to have been run explicitly.
Which way Test Explorer calls NUnit is dependent on the version you are running and it's only possible to tell how it calls by debugging or by observing the outcome. To add to the confusion, some versions of the NUnit3 adapter use heuristics to try to guess whether an explicit test was called explicitly. This doesn't solve your problem but should make it easier to understand what is going on.
The best workaround, if you are using TestExplorer, is to avoid Explicit and to set up categories, which control the running of the tests. Fortunately, TestExplorer and NUnit have pretty much the same understanding of categories.

You can filter tests first in Test Explorer, then the "Run All" button is scoped to only those tests in the Test Explorer panel. If you desire to run all tests except those in a certain project, enter this into the test filter search box in the Test Explorer panel:
-project:NameOfTheTestProject
If your test project has space characters in the name:
-project:"Name of the test project"
Note the - character before the word project. This is crucial. This character tells Visual Studio Test Explorer to invert your filter criteria — it is used to exclude tests in a certain project.
If you have these UI tests intermingled with other kinds of tests, obviously filtering out an entire project is not desirable. Consider placing the UI tests in a specific folder and filter on the FullName instead. For example, if your SpecFlow tests exist in a folder called "Specs", then omit those tests using:
-FullName:Specs
Again, include the - character (with no spaces) before the FUllName criteria to exclude tests in that namespace. This works with SpecFlow because the namespaces for C# classes generated from .feature files follow the folder structure of your test project, just like adding a .cs file does.
This technique should work with any unit test provider, not just NUnit.

Feature file execution with NUnit

I am trying to run a feature file with NUnit Console. I tried googling it and checked NUnit3 help also. But I am unable to find any help.
I want to run either single feature file or any scenario in a feature file which has tag assigned. I am using specflow with specrun. I tried NUnit console command for where "test == path of feature file" but it is not executing test. However I am able to execute all test cases by giving project dll file path. But I just want to execute a single feature file or single scenario in a feature file. Please let me know how can I do this so that I will be able to generate NUnit testresult.xml file.
Thanks.

if you are using SpecRun then you can use the command line of SpecRun to run the tests.
If you really want to use NUnit then you first have to make sure that the project containing the feature files has been compiled. Once you have a test dll this will contain the NUnit tests just like any other and the test categories will be set based on the tags, so you can execute them all by telling NUNit to run tests in the test dll, or you can run tests which have a Tag by telling Nunit to run tests which are in a category matching the Tag.
Running just a feature will be more tricky as there is nothing which groups the scenarios by feature in the tests I don't think, though I may be wrong.

instead of test==featurefile use name==FeatureName

Nunit runs each test twice

I encounter a problem running my tests via NUnit. I dont know why but each test runs twice. The point is that on another laptop it normally runs only one time.
Have anyone the same problem and know how to deal with it?

I had the same problem. In my case, I had both the Visual Studio extension and the NuGet package of the NUnit Test Runner installed. It seems they were both executing the tests, resulting in tests being run twice.
Uninstalling the Visual Studio extension solved the issue.

I had a similar problem where running the test with the Visual Studio NUnit3 runner, the test would run only once but when running in the command line with NUnit3-Command.exe, the test would run twice. I spent way too long chasing this down. I noticed that the command line runner was reporting 2 tests were run and came to the conclusion it was an issue in the dll. This may sound crazy but I copied all my test code into a new class, deleted the old, compiled and the test now runs just once in both scenarios.

In my case, it was having a base class and a derived class both having the TestFixture Attribute. All tests from the derived class would run twice.
Fixed it by only putting attribute on base class.

The problem typically arises when you load a test container (.dll) multiple times.
This can happen if you load the test containers from a root floder that has both bin and obj folders inside, which is a typical setup of Visual Studio projects.
Try to limit your test container loading from the bin folders.

The running of test can also be duplicated is when sequential tests accidentally have the same test parameters.
[Test, Sequential]
public void Integr_MController_DeleteGet_ReturnsViewAndModel(
[Values("Do#london.com", "Do#london.com")] string firstEmail,
[Values("Zyg#london.com", "Zyg#london.com")] HTTPstring updatedEmail) {test code}
Will run the same test twice and will show up in TestExplorer as the same test.

In case you put the *.dll twice, the test get double executed.
Seen on NUnit.3.10.1, nunit3-console 3.9.0
Example:
nunit3-console myTestDll.dll --test=MyNameSpace.Test myTestDll.dll** --result=myResult.xml

How to make an NUnit TestProject use its own config when run?

Is it possible to let Resharper (or NUnit?) know that I want each test to look for an App.config under it's own project, even when running all tests in a solution together?
Background:
I'm using NUnit and the test-runner that ships with Resharper, and I've got several test-projects in the same solution. Some of my tests depend on config-files located under their respective projects.
When I run a test-project by itself, it will use it's internal App.config, and everything works fine. When I try to run all the tests in the solution, or use the shortcut to run all tests in current test session however, no config-file will be selected, and any test depending on a config will fail by default.
For this reason, I typically end up running all tests in the solution once first, then right clicking the nodes in the test-runner for each of the config-dependent projects, and running them separately afterwards.

Solved:
Apparently assemblies containing NUnit-tests can be run in separate processes or domains using command line options.
For the testrunner under Resharper, this setting can be found under Resharper > Options > Unit Testing.
There is an option "Use separate AppDomain for each assempbly with tests". Checking that solved my problem.

How do you run SpecFlow scenarios from the command line using MSTest?

I've got Visual Studio 2010, and we have two VS solutions we work with. The first is the web application, and the second is strictly for SpecFlow tests. Having two instances of Visual Studio running at the same time just to run SpecFlow features is eating all the available RAM causing things to slow down.
I've done some searching on Google and here on StackOverflow, plus perused the MS documentation on the MSTest command line tool, but I haven't found the answer. The full SpecFlow test suite takes ~45 minutes to complete, and I really only need to run a few scenarios.
I was wondering if there is a way to run individual SpecFlow features, and even individual scenarios, from the command line using MSTest?

Behind the scene specflow tests are just regular mstest unit tests. So you should be able to run them the same way using something like:
To run a specific scenario:
mstest /testcontainer:tests.dll /test:GivenMyScenarioWhenIDoSomeStuff
To run a several specific scenario you can use the /test flag multiple times:
mstest /testcontainer:tests.dll /test:GivenMyScenarioWhenIDoSomeStuff /test:GivenMyScenarioWhenIDoSomemthingElse
To run a feature
mstest /testcontainer:tests.dll /test:MyFeatureName
If you add tags on your scenarios using #MyTag for example, you could also use the option
/category:MyTag to filter down the scenarios to run.
Please have a look to the generated code behind of your feature files to get a clue of how things actually work, if you are familliar with mstest it should be pretty straightforward.

Now that SpecFlow 3.0 has been released we can use SpecFlow with .NET Core. The CLI tool for .NET Core is dotnet and tests are run like this if you use MSTest (vstest):
dotnet test
If the tests are in a specific project you can specify the project like this
dotnet test TestProject
where TestProject is the name of the project. You can skip the project name if you want to, but specifying it will make dotnet look in only that project. To list all the tests in the project you can use the -t flag:
dotnet test TestProject -t
To run only specific tests you can use the --filter flag:
dotnet test TestProject --filter ShouldBeSuccess_1
where ShouldBeSuccess_1 is the name of the test. The argument after --filter is an expression, and not necessary the name of the test If you had a test called ShouldBeSuccess_12 it would also run. You can see the rules for --filter here.
To only run the tests in a specific category you can use TestCategory:
dotnet test TestProject --filter TestCategory=ci
where ci is the category name. To add a test to a category you use tags.
To create the results file you have to use the --logger flag:
dotnet test TestProject --logger trx
Here it's used to create a trx results file.

There is a nuget package named "Specrun.Specflow" download that. And it will change your app.config and set unitTestProvider name="SpecRun", so you can remove unitTestProvider name="MSTest" or "NUnit", now on saving App.config changes, visual studio prompts you to regenerate your feature files, click on Yes and now build a solution, What you will see is your test files have been regenerated.
Now in your Command Prompt go to C:\Users\\Documents\Visual Studio 2015\Projects\ and type runtests.cmd , it should trigger all your Feature files directly.

With MSTest v2, you can´t use mstest. You can use vstest.console.exe instead.
Example:
vstest.console.exe "Automation.SpecFlow\bin\Release\Automation.SpecFlow.dll"
https://learn.microsoft.com/en-us/visualstudio/test/vstest-console-options?view=vs-2019
If you want to run all scenarios in a single feature file, then add the /trait flag:
vstest.console.exe "Automation.SpecFlow\bin\Release\Automation.SpecFlow.dll" /trait:"My Feature"
And this runs all scenarios in the feature files that begin with:
Feature: My Feature
In order to ...
As a ...
I want to ...
Scenario: 1
...
Scenario: 2
...

I tried the tags technique but it didn't work, I'm using an older version of SpecFlow.
So, I went to the .feature.cs file related to a feature file and searched for TestMethodAttribute()
[Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute()]
I added the TestCategory attribute on top of this, just like the following:
[Microsoft.VisualStudio.TestTools.UnitTesting.TestCategory("MyCat")]
Build and compile and the command worked like a charm with
/Category:MyCat
I hope someone will find the answer useful.