We Keep Coding

Feature file execution with NUnit

I am trying to run a feature file with NUnit Console. I tried googling it and checked NUnit3 help also. But I am unable to find any help.
I want to run either single feature file or any scenario in a feature file which has tag assigned. I am using specflow with specrun. I tried NUnit console command for where "test == path of feature file" but it is not executing test. However I am able to execute all test cases by giving project dll file path. But I just want to execute a single feature file or single scenario in a feature file. Please let me know how can I do this so that I will be able to generate NUnit testresult.xml file.
Thanks.

if you are using SpecRun then you can use the command line of SpecRun to run the tests.
If you really want to use NUnit then you first have to make sure that the project containing the feature files has been compiled. Once you have a test dll this will contain the NUnit tests just like any other and the test categories will be set based on the tags, so you can execute them all by telling NUNit to run tests in the test dll, or you can run tests which have a Tag by telling Nunit to run tests which are in a category matching the Tag.
Running just a feature will be more tricky as there is nothing which groups the scenarios by feature in the tests I don't think, though I may be wrong.

instead of test==featurefile use name==FeatureName

How to run a specific NUnit test from code using NUnit 3 (C#)

I am trying to run tests a second time programmatically, that exist in the same project.
Because if a SetUp of tests was failed then tests and SetUp will run again.
I use Selenium Webdriver to run tests and a lot of then has SetUp pre-actions. For example: a pre-action is to buy a ticket and then a set of tests are run to check if all is OK.
Often my SetUp pre-action fails (error occurred) and then no test are run. I would like to give them a second change, that is, to run them again.
A great solution that might work for me is this SO question, but it requires NUnit 3.
There is no NUnit.Core package in NUnit 3 so the solution shows a lot of errors. I'm also not sure that I have to use it if my tests methods are placed in the same project (so maybe I don't need to load an assembly).
Any ideas of how to run a test method from code if this method is placed in the same project?

The missing methods and assemblies are part of the internal implementation of NUnit 2.6.4 and were never intended for public use. The solution you point to is only "great" while it works, but because it uses internal classes and methods, there are no guarantees.
For NUnit 3.0, you are in luck because there is now a public API for use in running tests. By using a public / published API, you gain a guarantee that we will keep it working in the future.
The documentation for the API is now located at Test-Engine-API. Here is a simple example of its use...
ITestEngine engine = TestEngineActivator.CreateInstance();
TestPackage package = new TestPackage("path/to/assembly");
ITestRunner runner = engine.GetRunner(package);
XmlNode result = runner.Run(this, TestFilter.Empty);
You should reference the nunit.engine.api assembly (not nunit.engine) in order to use this. Your class making these calls is assumed to implement ITestEventListener.

How do you run SpecFlow scenarios from the command line using MSTest?

I've got Visual Studio 2010, and we have two VS solutions we work with. The first is the web application, and the second is strictly for SpecFlow tests. Having two instances of Visual Studio running at the same time just to run SpecFlow features is eating all the available RAM causing things to slow down.
I've done some searching on Google and here on StackOverflow, plus perused the MS documentation on the MSTest command line tool, but I haven't found the answer. The full SpecFlow test suite takes ~45 minutes to complete, and I really only need to run a few scenarios.
I was wondering if there is a way to run individual SpecFlow features, and even individual scenarios, from the command line using MSTest?

Behind the scene specflow tests are just regular mstest unit tests. So you should be able to run them the same way using something like:
To run a specific scenario:
mstest /testcontainer:tests.dll /test:GivenMyScenarioWhenIDoSomeStuff
To run a several specific scenario you can use the /test flag multiple times:
mstest /testcontainer:tests.dll /test:GivenMyScenarioWhenIDoSomeStuff /test:GivenMyScenarioWhenIDoSomemthingElse
To run a feature
mstest /testcontainer:tests.dll /test:MyFeatureName
If you add tags on your scenarios using #MyTag for example, you could also use the option
/category:MyTag to filter down the scenarios to run.
Please have a look to the generated code behind of your feature files to get a clue of how things actually work, if you are familliar with mstest it should be pretty straightforward.

Now that SpecFlow 3.0 has been released we can use SpecFlow with .NET Core. The CLI tool for .NET Core is dotnet and tests are run like this if you use MSTest (vstest):
dotnet test
If the tests are in a specific project you can specify the project like this
dotnet test TestProject
where TestProject is the name of the project. You can skip the project name if you want to, but specifying it will make dotnet look in only that project. To list all the tests in the project you can use the -t flag:
dotnet test TestProject -t
To run only specific tests you can use the --filter flag:
dotnet test TestProject --filter ShouldBeSuccess_1
where ShouldBeSuccess_1 is the name of the test. The argument after --filter is an expression, and not necessary the name of the test If you had a test called ShouldBeSuccess_12 it would also run. You can see the rules for --filter here.
To only run the tests in a specific category you can use TestCategory:
dotnet test TestProject --filter TestCategory=ci
where ci is the category name. To add a test to a category you use tags.
To create the results file you have to use the --logger flag:
dotnet test TestProject --logger trx
Here it's used to create a trx results file.

There is a nuget package named "Specrun.Specflow" download that. And it will change your app.config and set unitTestProvider name="SpecRun", so you can remove unitTestProvider name="MSTest" or "NUnit", now on saving App.config changes, visual studio prompts you to regenerate your feature files, click on Yes and now build a solution, What you will see is your test files have been regenerated.
Now in your Command Prompt go to C:\Users\\Documents\Visual Studio 2015\Projects\ and type runtests.cmd , it should trigger all your Feature files directly.

With MSTest v2, you can´t use mstest. You can use vstest.console.exe instead.
Example:
vstest.console.exe "Automation.SpecFlow\bin\Release\Automation.SpecFlow.dll"
https://learn.microsoft.com/en-us/visualstudio/test/vstest-console-options?view=vs-2019
If you want to run all scenarios in a single feature file, then add the /trait flag:
vstest.console.exe "Automation.SpecFlow\bin\Release\Automation.SpecFlow.dll" /trait:"My Feature"
And this runs all scenarios in the feature files that begin with:
Feature: My Feature
In order to ...
As a ...
I want to ...
Scenario: 1
...
Scenario: 2
...

I tried the tags technique but it didn't work, I'm using an older version of SpecFlow.
So, I went to the .feature.cs file related to a feature file and searched for TestMethodAttribute()
[Microsoft.VisualStudio.TestTools.UnitTesting.TestMethodAttribute()]
I added the TestCategory attribute on top of this, just like the following:
[Microsoft.VisualStudio.TestTools.UnitTesting.TestCategory("MyCat")]
Build and compile and the command worked like a charm with
/Category:MyCat
I hope someone will find the answer useful.

Can't load DLL while executing tests with MS-Test

In my program, I use SevenZipSharp to generate zip files. SevenZipSharp is a managed DLL which loads another DLL, 7z.dll. I am manually setting SevenZipSharp's path to 7z.dll using SevenZipCompressor.SetLibraryPath.
When I execute my program in Debug mode, this all works fine, and it generates the zip file as nice as you please. However, when I execute my unit tests with mstest, SevenZipSharp always gives me the following error:
Test method threw exception:
SevenZip.SevenZipLibraryException: Can not load 7-zip library or
internal COM error! Message: failed to load library..
I suspect that MSTest might be doing something that is preventing SevenZipSharp from being able to load 7z.dll, like running in a security-tight sandbox (or something. I'm new to C# and MSTest...)
Does anyone have an idea about what might be happening?
Thank you!

Though the question posits a questionable scenario, the general problem of MSTest not loading required DLLs seems to be a common one, deserving of a less dismissive answer.
By default MSTest will copy the assemblies it believes are required by the test container to the Out folder of the default results folder, which changes for each run.
MSTest does not always automatically infer the necessary assemblies correctly; if there is no explicit direct reference to an assembly it won't be copied. Also, native DLLs are typically not detected.
I am not aware of a direct option to set the MSTest search path. You can determine the search path using procmon.exe as suggested above (it is basically the standard Windows DLL search).
Unintuitively, the default search path does not include the launch directory and I think this is a cause of confusion. When the tests are running the current directory is the test results "Out" directory, not the MSTest launch directory.
However, it is possible to control MSTest search behaviour (and the copying behaviour) with a test settings file. You can easily create and edit these through Visual Studio (see the Test menu) and then specify the created settings file on the MSTest command line. You can use different settings files for Visual Studio and MSTest.
By this means you can control exactly what DLLs are copied to your test directory.
See Create Test Settings to Run Automated Tests from Visual Studio to get more information on this.
Of course, DLL load failures may be due to missing dependencies, and the DLL mentioned in the error message may itself be present. You can use the dependency viewer or procmon to pick up unexpected dependencies in DLLs.

Consider using Process Monitor (aka procmon.exe) from the excellent SysInternals tools to monitor your test harness (MSTest). It will show you where the executable is looking for 7z.dll.

Visual Studio 2017 (and possibly 2015) provides two new ways to indicate that a native dll or other file is required by your tests without the need for a test settings file:
1: Add a link to the dll to your test project and tell VS to copy it to the output directory. Right-click the project in Solution Explorer and choose Add > Existing Item. Browse to 7z.dll, click the down arrow next to the Add button, and choose Add as Link. Then select the new 7z.dll item in Solution Explorer, alt-Enter to bring up Properties, and set "Copy to Output Directory" to "Copy if newer" (or "Copy always").
2: Attach a DeploymentItemAttribute to your test class. This attribute's constructor takes a single string argument, which is the path to the file you want to make available to tests in the class. It is relative to the test project's output directory.

Are you sure you want your unit tests to involve external libraries? Ideally you should have mechanisms to replace external stuff with e.g. mock objects, because testing external libraries like that actually turns your test into an integration test.
For popular libraries such as SevenZipSharp you can assume that it is properly tested, and you can have manual integration tests to verify that it performs correctly in your program.
I would consider getting rid of that dependency through dependency injection, mocking frameworks, etc, and let your unit tests solely test your own code.
Investigate the factory method or abstract factory design patterns for tips on how to easily replace such dependencies.
A good start would be to create your own ICustomZipInterface, and use the wrapper pattern to encapsulate your zip logic for production code. In your unit tests, replace that wrapper class with a dummy implementation. The dummy implementation might for example record how you access your zip component and you could use that information to validate your code, rather than checking if a zip file is actually created.

How to use NUnit with SnippetCompiler?

I try to use NUnit with SnippetCompiler http://www.sliver.com/dotnet/SnippetCompiler/
I have added references to nunit.framework.dll in snippetcompiler (Menu Tools, References) and compiled nunit sample http://www.nunit.org/index.php?p=quickStart&r=2.5.2 to bank.dll
but when I open bank.dll in NUnit GUI it fails saying it cannot load NUnit assembly or one of its dependencies.
Is it possible to fix this ?

I couldn't even get v2.0.8.3 of SnippetCompiler to include the reference. It let me do it, but it wouldn't compile.
In any case writing unit tests isn't the purpose of SnippetCompiler. It's designed to do quick spikes - try something and see if it works. In other words, it's throwaway code.
In addition, the version for .NET 3.5 (the one I'm using) is an alpha release; the developer does not seem to be maintaining this. (Not to put down the author - this was an awesome tool that saved me lots of time!)
For writing spikes against modern versions of .NET, I've switched to LINQPad. Change Edit/Preferences/Query to C# Program and it's very similar to SnippetCompiler. The basic version is free; for a small fee, the registered version provides IntelliSense.
Even if you're writing learning tests with NUnit, you'll want to preserve those tests. Use Visual Studio (or another IDE) and create a separate Class Library project for your tests.

Maybe this is the solution
http://weblogs.asp.net/rosherove/archive/2008/02/21/ad-hoc-unit-tests-with-snippet-compiler.aspx
Ad hoc Unit Tests with Snippet
Compiler
If you're a fan of snippet compiler
(if you're not you should seriously
check it out)Travis Illig published a
little template for writing Typemock
Isolator test inside this handy little
tool.
the reasons you'd need a specialized
template in the first place to write
these kinds of tests in snippet
compiler:
1) Typemock Isolator uses the .NET
profiling APIs to work its magic, so
the .net process running your tests
need to have a couple of environment
variables enabled to work
2) His code template actually creates
and runs a new process that triggers
nunit-console.exe with the path of the
current code you just wrote in snippet
compiler allowing you to effecively
write and run unit tests in snippet
compiler!
3) the nunit-console process will
already have the env. vars as
mentioned in the first item set to it.
Travis' template will work for
anything nunit can run, not just
typemock isolator tests, which is
pretty cool.