We Keep Coding

NUnit unit tests order or separate runner in TeamCity

Question:
Can I use separate unit test runner for subgroup of unit tests in my build? All of those that would need separate runner process are contained within single .dll
Or at least is it possible to specify order of unit tests?
Background:
I have some unit tests that are testing integration with native components that makes the process memory dirty and so in production code I recycle my process after using them. (it's python integration for .net and some packages are not designed for python engine unload and reload).
However the unit tests are only isolated by app domains - so they still remain in same process and can colide.

You can use [TestCategory] NUnit attribute to create different test group. After grouping, you could run only specific group from TeamCity server. You could also divide it into different steps.
But also as a variant, you could use [OneTimeSetUp] and [OneTimeTearDown] attributes.
Useful links:
https://msdn.microsoft.com/en-us/library/dd286683.aspx - description for TestCategory attribute.
http://nunit.org/docs/2.5/consoleCommandLine.html - how you can run your test categories from nunit-console.
https://confluence.jetbrains.com/display/TCD9/Getting+Started+with+NUnit#GettingStartedwithNUnit-Case1.CommandLine - how you could use nunit-console inside team city.
Second approach:
https://github.com/nunit/docs/wiki/OneTimeSetUp-Attribute
https://github.com/nunit/docs/wiki/OneTimeTearDown-Attribute

Turns out TeamCity supports separation of test assemblies by individual test runner processes - option called 'Run process per assembly' in NUnit build step configuration:
More details here: https://confluence.jetbrains.com/display/TCD10/NUnit (search 'Run process per assembly')

How are unit tests compiled in c# test projects?

I've gone through tutorials on writing unit tests with a "Test" project for my .net 4.5 app but I don't understand how they are running.
When I run unit tests that test specific methods of classes in my projects are the entire projects that the methods are recompiled for each test?
I'm confused about how the tests interact with the methods they are testing. Are the tests themselves compiled? I guess they would have to be because it's c#. Is there a separate binary for the tests?

You actually can't run unit tests independently. Typically they will be executed by a "test runner," e.g. the built-in MSTest test running or an NUnit test runner adapter. The process works like this:
You write the test, marking test classes with a specific attribute, e.g. [TestClass] or [TestFixture].
The test classes are compiled into a DLL.
You start your test runner using menu items in Visual Studio (or it is started as part of the automated build process).
The test runner loads the unit test DLL.
The test runner loads all of the types in the DLL and uses reflection to locate the types (classes) that have the magic attibute that indicates that it contains tests.
The test runner instantiates the test class/fixture.
The test runner iterates over the methods in the DLL that have a [Test] or [TestMethod] attribute.
The test runner invokes each method using Invoke().
The test runner displays the results of its test in the test runner UI, or outputs it to a report.

Feature file execution with NUnit

I am trying to run a feature file with NUnit Console. I tried googling it and checked NUnit3 help also. But I am unable to find any help.
I want to run either single feature file or any scenario in a feature file which has tag assigned. I am using specflow with specrun. I tried NUnit console command for where "test == path of feature file" but it is not executing test. However I am able to execute all test cases by giving project dll file path. But I just want to execute a single feature file or single scenario in a feature file. Please let me know how can I do this so that I will be able to generate NUnit testresult.xml file.
Thanks.

if you are using SpecRun then you can use the command line of SpecRun to run the tests.
If you really want to use NUnit then you first have to make sure that the project containing the feature files has been compiled. Once you have a test dll this will contain the NUnit tests just like any other and the test categories will be set based on the tags, so you can execute them all by telling NUNit to run tests in the test dll, or you can run tests which have a Tag by telling Nunit to run tests which are in a category matching the Tag.
Running just a feature will be more tricky as there is nothing which groups the scenarios by feature in the tests I don't think, though I may be wrong.

instead of test==featurefile use name==FeatureName

How can I split same-project unit tests into separate assemblies?

I have a few test projects, one of which has many unit tests, all very related to doing certain calculations. Running all unit tests takes a long time. So, I'm looking for a way to easily split tests into nightly unit tests and all other unit tests. How can I conveniently specify which test fixtures and test methods should be nightly? In other words, is there a way to split tests from within the same project into separate assemblies?
What I've thought of and tried so far:
Just separate the nightly tests into their own project manually and only run that new project on a nightly basis. This would mean moving files around in the project anytime I wanted to make a test fixture run nightly or regularly, i.e. not desirable.
Come up with a build script that separates these tests at compile time into separate projects. Then I'd end up with separate assemblies. This seems too complicated, but maybe it's the only option.
I would love to be able to use a class attribute to specify which tests will run nightly, e.g.:
[TestClass, Nightly]
public class MyTestClass { }
Any ideas?

You can have as many unit test projects for the same solution as you need.
If you want to go with separate project (i.e. your build system only setup to run test from separate project) I'd have new project sharing tests from old one - create new project and add files from old one as "link" (there is small option on "Open" button of add file dialog to do so).
Alternatively you can add attributes to tests (i.e. TestCategoryAttribute) and make your test runner respect those.

How do you run SpecFlow scenarios from the command line using MSTest?

I've got Visual Studio 2010, and we have two VS solutions we work with. The first is the web application, and the second is strictly for SpecFlow tests. Having two instances of Visual Studio running at the same time just to run SpecFlow features is eating all the available RAM causing things to slow down.
I've done some searching on Google and here on StackOverflow, plus perused the MS documentation on the MSTest command line tool, but I haven't found the answer. The full SpecFlow test suite takes ~45 minutes to complete, and I really only need to run a few scenarios.
I was wondering if there is a way to run individual SpecFlow features, and even individual scenarios, from the command line using MSTest?

Behind the scene specflow tests are just regular mstest unit tests. So you should be able to run them the same way using something like:
To run a specific scenario:
mstest /testcontainer:tests.dll /test:GivenMyScenarioWhenIDoSomeStuff
To run a several specific scenario you can use the /test flag multiple times:
mstest /testcontainer:tests.dll /test:GivenMyScenarioWhenIDoSomeStuff /test:GivenMyScenarioWhenIDoSomemthingElse
To run a feature
mstest /testcontainer:tests.dll /test:MyFeatureName
If you add tags on your scenarios using #MyTag for example, you could also use the option
/category:MyTag to filter down the scenarios to run.
Please have a look to the generated code behind of your feature files to get a clue of how things actually work, if you are familliar with mstest it should be pretty straightforward.

Now that SpecFlow 3.0 has been released we can use SpecFlow with .NET Core. The CLI tool for .NET Core is dotnet and tests are run like this if you use MSTest (vstest):
dotnet test
If the tests are in a specific project you can specify the project like this
dotnet test TestProject
where TestProject is the name of the project. You can skip the project name if you want to, but specifying it will make dotnet look in only that project. To list all the tests in the project you can use the -t flag:
dotnet test TestProject -t
To run only specific tests you can use the --filter flag:
dotnet test TestProject --filter ShouldBeSuccess_1
where ShouldBeSuccess_1 is the name of the test. The argument after --filter is an expression, and not necessary the name of the test If you had a test called ShouldBeSuccess_12 it would also run. You can see the rules for --filter here.
To only run the tests in a specific category you can use TestCategory:
dotnet test TestProject --filter TestCategory=ci
where ci is the category name. To add a test to a category you use tags.
To create the results file you have to use the --logger flag:
dotnet test TestProject --logger trx
Here it's used to create a trx results file.

There is a nuget package named "Specrun.Specflow" download that. And it will change your app.config and set unitTestProvider name="SpecRun", so you can remove unitTestProvider name="MSTest" or "NUnit", now on saving App.config changes, visual studio prompts you to regenerate your feature files, click on Yes and now build a solution, What you will see is your test files have been regenerated.
Now in your Command Prompt go to C:\Users\\Documents\Visual Studio 2015\Projects\ and type runtests.cmd , it should trigger all your Feature files directly.

With MSTest v2, you can´t use mstest. You can use vstest.console.exe instead.
Example:
vstest.console.exe "Automation.SpecFlow\bin\Release\Automation.SpecFlow.dll"
https://learn.microsoft.com/en-us/visualstudio/test/vstest-console-options?view=vs-2019
If you want to run all scenarios in a single feature file, then add the /trait flag:
vstest.console.exe "Automation.SpecFlow\bin\Release\Automation.SpecFlow.dll" /trait:"My Feature"
And this runs all scenarios in the feature files that begin with:
Feature: My Feature
In order to ...
As a ...
I want to ...
Scenario: 1
...
Scenario: 2
...

I tried the tags technique but it didn't work, I'm using an older version of SpecFlow.
So, I went to the .feature.cs file related to a feature file and searched for TestMethodAttribute()
[Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute()]
I added the TestCategory attribute on top of this, just like the following:
[Microsoft.VisualStudio.TestTools.UnitTesting.TestCategory("MyCat")]
Build and compile and the command worked like a charm with
/Category:MyCat
I hope someone will find the answer useful.