We Keep Coding

Problem manually calling dotnet test on playwright tests generated by specflow

I'm working on a C# project that's using specflow with playwright to handle most of our testing, and I'm trying to set up a way to generate playwright trace files only for tests that have failed. Right now I'm printing the name of each test that fails to a .txt file, then using a shell script to manually call dotnet test on each test in that file at the end of our test run with playwright trace generation enabled. This works fine for tests that have a simple name, such as "FillInAllFieldsOnPage", but if a test that is generated from a scenario outline using examples fails it gives me an error. I believe this is due to the way that dotnet test is parsing the name of the test, as specflow generates the name in the format
CheckSortingAndDataInHoverMenu("C, User",null)
where "C, User" is one of the examples given to the Scenario Outline. I've tried adding a line that reformats the string to be more in line with what I've seen recommended elsewhere, so that the end result looks like this:
CheckSortingAndDataInHoverMenu(\"C%2C User\"%2Cnull)
but I still get the following error:
Running dotnet test on "CheckSortingAndDataInHoverMenu(\"C%2C User\"%2Cnull)"
MSBUILD : error MSB1009: Project file does not exist.
Switch: User"%2Cnull)
I've verified that the .csproj file does exist, and other tests are able to be run from that command so I know it's being called from the correct directory. It looks like dotnet test thinks that "User"%2Cnull)" is a name for a .csproj file that I'm passing it as an arg, and I don't know how to get it to properly process the full test name.

The solution I found to this was to simply remove the part of the test name in parenthesis, which caused the dotnet test command to run every test with a matching name. I then added a random guid to the name of each test trace that was generated in order to prevent them from overwriting eachother.

How to pass parameters to the dotnet test command while using NUnit or XUnit

I'm developing some end-to-end tests using C# with .NET Core, Selenium and NUnit.
Now i want to write a login testcase. My tests are started from console simply by using the dotnet test command.
I simply want to pass username and password to this command and get them in my tests. I can not use NUnit-Console since it doesn't support .NET Core at the moment.
Whats the suggested way to solve this problem? I would prefer to not store the settings in a file but to directly input them into the console.

If you want to avoid a runsettings file, you can use this workaround. One of the recommended ways of passing parameters, is through environment variables. So in your C# nunit (or xunit) file, you can do something like:
// in mytest.cs
var user = Environment.GetEnvironmentVariable("TestUser");
var password = Environment.GetEnvironmentVariable("TestPassword");
var url = Environment.GetEnvironmentVariable("TestUrl");
If you do not want to definitively set your environment variables, remember you can always set them temporarily for just your session process. One way of doing this, is by creating a simple cmd file
#launchtests.cmd
SETLOCAL
SET TestUser='pete001'
SET TestPassword='secret'
SET TestUrl='http://testserver.local/login'
DOTNET TEST mytest.csproj
And now the fun part. You can parameterize every aspect of this. So you can change it to:
#run wity launchtests.cmd pete001 secret 'http://testserver.local/login'
SETLOCAL
SET TestUser=%1
SET TestPassword=%2
SET TestUrl=%3
DOTNET TEST mytest.csproj
Or if you want to launch the test from an Azure DevOps (fka VSTS or TFS) pipeline, you can simply use the $(...) notation to inline variables, even if they're marked secret and/or come from Azure KeyVault.
#In Azure DevOps, variables not marked as secret are already added to the environment
SET TestPassword=$(TestPassword)
dotnet test $(Build.SourcesDirectory)\MyCompany.MyProduct.UITests\MyTest.csproj --configuration $(BuildConfiguration) --collect "Code Coverage" --logger trx --results-directory $(Agent.TempDirectory)

Unfortunately, the only way to pass settings from dotnet test into NUnit is to use a .runsettings file. There's no way for NUnit to create custom command line arguments for the dotnet test tool - although we'd love there to be!
Take a look at the sample .runsettings file here. The specific bit you'll need:
<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
<!-- Parameters used by tests at runtime -->
<TestRunParameters>
<Parameter name="webAppUrl" value="http://localhost" />
<Parameter name="webAppUserName" value="Admin" />
<Parameter name="webAppPassword" value="Password" />
</TestRunParameters>
</RunSettings>
You should just be able to then pass this file into dotnet test with the -s flag.
dotnet test myProj.csproj -s mySettings.runsettings

This documentation suggests that it should now be possible to pass in arguments on the command line, instead of within a runsettings file.
https://github.com/Microsoft/vstest-docs/blob/master/docs/RunSettingsArguments.md
dotnet test -- MSTest.MapInconclusiveToFailed=True MSTest.DeploymentEnabled=False
Note the space after -- .
Edit 1
What worked for me was a combination of adding a runsettings file, and then overriding the param I wanted to, using this syntax:
dotnet test -- TestRunParameters.Parameter(name=\"myParam\", value=\"value\")

Generating TestResult with nunit-console failing

According to the SpecFlow website, I can generate the TestResult by executing the following statement:
nunit3-console.exe --labels=All --out=TestResult.txt "--result=TestResult.xml;format=nunit2" bin\Debug\BookShop.AcceptanceTests.dll
I've adapted this statement to the following:
nunit3-console.exe --labels=All --out=C:\temp\TestResult.txt "--result=C:\temp\TestResult.xml;format=nunit2" C:\Projects\DataService.IntegrationTests\bin\Debug\DataService.IntegrationTests.dll
Unfortunately, I get the following errors:
Errors, Failures and Warnings
1) Invalid : C:\Projects\DataService.IntegrationTests\bin\Debug\DataService.IntegrationTests.dll
No suitable tests found in 'C:\Projects\DataService.IntegrationTests\bin\Debug.DataService.IntegrationTests.dll'.
Either assembly contains no tests or proper test driver has not been found.
The .feature file are available in this assembly...
What could be the problem here?
An assumption of mine is, that we're using Specflow.MsTest...
Is there a way to generate the TestResult.xml by using MsTest?
Thanks in advance

NUnit3-console can only run NUnit tests. (Well technically, it can run any tests for which a driver is provided, but it amounts to the same thing.)
If you want to create NUnit output, then use both the NUnit framework and the NUnit console runner. MsTest doesn't do NUnit output.

In app.condig I don't have any unitTestProvider
<specFlow>
<stepAssemblies>
<stepAssembly assembly="otherProject" />
</stepAssemblies>
<runtime detectAmbiguousMatches="true" stopAtFirstError="false" missingOrPendingStepsOutcome="Inconclusive" />
<trace traceSuccessfulSteps="true" traceTimings="false" minTracedDuration="0:0:0.1" />
<!-- For additional details on SpecFlow configuration options see http://go.specflow.org/doc-config -->
</specFlow>

How can I export the results of a C# unit test?

Is there a way to export the result of a unit test?
When I run NUnit, can I get formal output saying what methods have been tested and out of those, which one failed and which passed?

You can specify the test results to be outputted to a file:
Redirecting Text Output
Output created by the test, which is normally shown on the console, may be redirected to a file. The following command redirects standard output to the file TestResult.txt:
nunit-console nunit.tests.dll /out:TestResult.txt
The following command redirects standard error output to the StdErr.txt file.
nunit-console nunit.tests.dll /err:StdErr.txt
Note:This option only redirects output produced by the tests, together with selected NUnit output that is interspersed with the test output. It does not redirect all console output. If you want to redirect all output to a file, you should use command line redirection as supported by the shell you are using. This option exists for the purpose of separating test output from other output, such as the NUnit summary report.
From the NUnit documentation.
I just tested against one of my test assemblies by completing the following steps:
Downloaded and installed latest version of NUnit
Added 'C:\Program Files (x86)\NUnit 2.6.4\bin' to System Path
Navigated to where my assembly was in the command prompt
Ran nunit-console MyTests.dll /out:TestResult.txt
The results were then written to TestResult.txt in the same directory.

Get Executing Path of C# application when using nunit with Jenkins

I use Jenkins, for building, unit testing, and publishing a windows application in c#.
While executing test cases using nunit-console.exe, some of them needs Application.Executing Path for completing the test case.
And it returns, nunit-console.exe path as result. Hence these test cases getting failed. ( What i need here is C# application executing path)
How can i solve this issue.

Environment.CurrentDirectory
returns the current working directory.