Why is my Specflow [AfterTestRun] hook called twice - c#

[AfterTestRun]
This hook for me is being called twice.
My C# code is correct and at the end of each Scenario I am saving my results to a Concurrent Bag.
Then I use the [AfterTestRun] hook to call the Concurrent Bag and save the data to a database. I see duplicated data, so I assume it’s being called twice.
Additional Info:
I am using SpecRun to run my tests in parallel with the following profile
Execution stopAfterFailures="1" retryCount="0" testThreadCount="3" testSchedulingMode="Sequential"
Packages Installed
SpecFlow Version 2.0.0
SpecRun.SpecFlow 1.3.0
SpecRun.Runner 1.3.0
I am using SpecRun.SpecFlow to run my tests.
Also, how will this hook behave if one has multiple scenarios within each feature? Currently I have 1.
Thanks

Steps are global in specflow, inheritance to get step reuse is unnecessary. In fact if you do inherit step classes the the steps they contain end up being duplicated, and you see the issue you have here. See this answer for additional details.
the simple solution is to place the [BeforeScenario] methods into their own class, and do not have your step classes inherit this. If you need to share state between your steps and your before/after scenarios then use one of the state sharing techniques outlined here

Related

Is there a way to skip or satisfy a scenario based on a boolean?

I have a situation where feature-toggling is implemented in an application and I want to have automated Selenium tests that are in place for both states of the application. (Both states are possible in a production state.) The tests that are executed would be based on the feature-toggling configuration that is present in the web application.
In a normal situation, using "Scenario Outline" will result in a generated test that can be executed via Test Explorer, and the CI process for an application can be configured to execute all of the tests that are part of the test assembly after it is built. In my situation, an application state could cause one test to pass and another test to fail (think lack of UI controls because of a feature state). I looked into hooks and I see that there is a BeforeScenario tag, but I don't think this gets me what I'm wanting to achieve so much as it allows me to do some setup before a scenario executes.
I usually include a step that sets the scenario to "pending".
Scenario: Some new feature
Given pending story 123
Given condition
When action that includes new feature
Then something
The step definition for Given pending story 123 would call:
Assert.Inconclusive($"Pending story {storyNumber} (https://dev.azure.com/company/project/workitems/edit/{storyNumber})");
This marks the test inconclusive when using MS Test. We use Jenkins for continuous integration, and our builds continue passing. It's also handy because I can provide a link to the story or feature in something like Azure DevOps, GitHub, Jira or their ilk.
This is not exactly a feature toggle, but it is easy to remove or comment out the Given pending line of a scenario in your topic branch to enable the test.

AfterFeature hook not working properly in specFlow

I have 2 feature files with 2 step definition classes.Each class has methods with [AfterFeature] hook.While executing feature2, the method defined under [AfterFeature] hook in step definition file for feature1 gets invoked.
The After Feature Hook in spec-flow is designed to be run after each feature.
https://specflow.org/documentation/Hooks/
Shows the details of the hooks that can be used and they are designed to all be run before or after.
looking over their Definition you could Try adding a class level Scope Attribute ( Listed Here Binding Documentation) to the class containing the AfterFeature step this may limit it but alas this is not the intended purpose. So i wouldn't overly expect this to work.
sorry its not the best answer in the world

Multistep composition using MEF in C#

I have a rather complex application which is initialized in multiple steps or phases. Some components are created during construction, some when user context is available, some when frond end gets available. I want to use MEF to create an easy extensible initialization process.
My question now... is is possible to have a MEF compose in multiple steps? Some imports can be satisfied, but some others only after e.g. user context is available within second composition.
If I understand you correctly you want to compose in an initial step and want to use the results from this initial step in the following ones. If this is what you mean by
...is it possible to have a MEF compose in multiple steps? ...
you can look into this thread stackoverflow.com and continue with the MSDN for CompositionBatch.
Otherwise if your object tree can be initialized anytime, you can just call Container.GetExportedValue() with the type you need in your process.

Run SpecFlow scenario from code

I am trying to run a SpecFlow scenario from code instead of through Test Explorer or the command line. Has someone managed to do this?
From a scenario, I can extract the method name and test method with recursion, but I cannot run this scenario method. It seems to need a proper initialize and teardown, but I could't manage to do this.
My first thought was to use the TechTalk.SpecFlow.TestRunner class, but it doesn't seem to have a scenario selection method.
EDIT on why I want to do this:
We want to run specific scenarios from TFS. It is very cumbersome to connect TestMethods to WorkItems in TFS, because:
You can only assign one testmethod to one workitem
For each workitem you have to search the method name, with in itself is a hassle, because the list is very long with lots of specflow scenarios.
When your specflow scenario gets a different name (which happens a lot), TFS cannot find the correct method anymore
Specflow Scenario Outlines get practically unusable, while they are a very powerful feature.
I want to create a mechanism where each automated workitem gets the same method assigned. This method extracts the workitem id and search and executes the scenario(s) with this workitem tagged.
I had a similar problem since my tests have some dependencies between Scenarios (shame on me, but it saves tons of copy-paste lines per Feature file). In most cases I would stick to isolated Scenarios of course.
I used reflection
Find all Types with a DescriptionAttribute (aka Features)
Find their MethodInfos with a TestAttribute and DescriptionAttribute (aka Scenarios)
Store them to a Dictionary
Call them by "Title of the Feature/Title of the Scenario" with Activator.CreateInstance and Invoke
You have to set the (private) field "testRunner" according to your needs of course.

Run unit tests in different appdomain with NUnit

I seem to be having an issue, the application we're using uses a Ninject kernel, and contains a particular configuration that's gathered with contents of the WCF call (username, token...).
Based on that a particular configuration the user is assigned rights on the app, this is shielded through a particular instance in the Ninject kernel.
We cannot just recompose the Ninject kernel, what we'd like to do is run a couple of Nunit tests, but run each of them in a separate app domain (recreating the kernel each time with different settings).
I've only found ways to run whole test projects in different app domains but not test per test.
Is there a way to solve this?
Unfortunately the Ninject configuration is not on our side, we have to 'live' with it this way.
I needed to do the the exact same thing, so I created a library which basically takes the current test and re-executes it in a new AppDomain. It's a nuget package called NUnit.ApplicationDomain and is open source.
Example Code:
[Test, RunInApplicationDomain]
public void Method()
{
Console.WriteLine("I'm in a different AppDomain")
}
I don't think there is a way to solve it without re-writing parts of the NUnit code. It has been a while since I've been inside the NUnit code, but I am pretty sure the main app domain loading part has not changed.
NUnit typically uses two app domains. The default one that is created when NUnit is run and a separate one to load the test assemblies and the assemblies they reference. It's main reason in doing this is to allow the unloading of the test assemblies. You can't unload a dll, but you can unload an appdomain.
You might be able to run NUnit once per test and pass the test on the command line, but that's ugly and I'm not sure it will help.
It might also be possible to leverage Action Attributes in NUnit 2.6, but you are going to do a lot of work in there to do it.
You might also be able to create a new app domain in your setup method and call into it in each test. Awkward but possible.
I'm sorry I don't have a more complete answer.
I'm not entirely sure about your question. However it seems like you need some kind of a custom implementation. Did you consider custom test attributes? Then may be configure each attribute to run in a different App Domain? I'm just spinning up some ideas, but there may be better ways of doing this.

Categories