We Keep Coding

Problem manually calling dotnet test on playwright tests generated by specflow

I'm working on a C# project that's using specflow with playwright to handle most of our testing, and I'm trying to set up a way to generate playwright trace files only for tests that have failed. Right now I'm printing the name of each test that fails to a .txt file, then using a shell script to manually call dotnet test on each test in that file at the end of our test run with playwright trace generation enabled. This works fine for tests that have a simple name, such as "FillInAllFieldsOnPage", but if a test that is generated from a scenario outline using examples fails it gives me an error. I believe this is due to the way that dotnet test is parsing the name of the test, as specflow generates the name in the format
CheckSortingAndDataInHoverMenu("C, User",null)
where "C, User" is one of the examples given to the Scenario Outline. I've tried adding a line that reformats the string to be more in line with what I've seen recommended elsewhere, so that the end result looks like this:
CheckSortingAndDataInHoverMenu(\"C%2C User\"%2Cnull)
but I still get the following error:
Running dotnet test on "CheckSortingAndDataInHoverMenu(\"C%2C User\"%2Cnull)"
MSBUILD : error MSB1009: Project file does not exist.
Switch: User"%2Cnull)
I've verified that the .csproj file does exist, and other tests are able to be run from that command so I know it's being called from the correct directory. It looks like dotnet test thinks that "User"%2Cnull)" is a name for a .csproj file that I'm passing it as an arg, and I don't know how to get it to properly process the full test name.

The solution I found to this was to simply remove the part of the test name in parenthesis, which caused the dotnet test command to run every test with a matching name. I then added a random guid to the name of each test trace that was generated in order to prevent them from overwriting eachother.

TFS Functional Testing is not finding my tests

I've been running Selenium tests in Visual Studio using C# and everything works fine locally. I have an automated build process in TFS that runs unit tests in my solution. That task (Test Assemblies) is finding the new Selenium tests but failing to run them. However, when I created a separate build definition that deployed a test agent (successfully) and then attempts to run functional tests, I get this message:
2017-11-03T18:49:43.1345753Z ##[warning]DistributedTests: Test Run Discovery Aborted . Test run id : 1600
2017-11-03T18:49:43.1345753Z ##[warning]DistributedTests: UnExpected error occured during test execution. Try again.
2017-11-03T18:49:43.1345753Z ##[warning]DistributedTests: Error : No tests were discovered from the specified test sources
I have searched the DTALog, and found that the test sources are being found successfully, just no actual tests within them. Any ideas what I am doing wrong? I have removed the 'Owner' decoration from the tests.

Please try below things to narrow down the issue:
Make sure the appropriate test adapter getting deployed along with
the test assemblies as Daniel mentioned.
If not deployed, you would need to copy the appropriate adapter from your local vs machine (\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions) to your test agent box (\Common7\IDE\CommonExtensions\Microsoft\TestWindow\Extensions\Microsoft.VisualStudio.TestPlatform.Extensions)
Try run directly from vstest.console.exe on test agent machine with
specifing test adapter path using /testadapterpath flag and see if
vstest.console works.
eg:
C:\Program Files (x86)\Microsoft Visual Studio\2017\Community\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe "F:\Test****TestAutomation.dll" /TestAdapterPath:F:\Test*\
Check if you have included the dependencies with the test dlls.
eg: manually add reference of Microsoft.VisualStudio.QualityTools.UnitTestFramework in the test project.
If the isse persist, please share the logs for furthre troubleshoot.

build cannot be found under team project

Our coded ui tests take several hours to complete, if I launch run functional tests task from the build definition it looks like it is consuming build minutes and using build pipelines so wrote a small powershell script (and a console application in C#). the plan is to launch either ps1 or console app from the build definition and exit. the ps1 or exe takes a build name as a parameter and launches the vstestconsole with that name as one of the parameters. when I test this locally on azure VM test machine it works fine (updates the results) but when I launch the same through build definition with build name as the parameter I get build cannot be found under team project error:
Log:
[command]C:\Users\automation\Desktop\ps1\cmd\ExecuteVSconsole.exe UI_Automation_NoWait_20170621.9
Error: Build "UI_Automation_NoWait_20170621.9" cannot be found under team project "XXX"
arg list :C:\Uiautomationbinaries\UI.dll /logger:Tfspublisher;Collection=https://xxx.visualstudio.com/;BuildName=UI_Automation_NoWait_20170621.9;TeamProject="xxx"
The build task I am using is : Powershell on target machine.
The build number is unique (name of the def + day+rev)
I have tried some of the solutions I have found online.

Since running locally successful, suggest you RDP to the remote machine (the target machine of the task)with your build service account. Then manually run the powershell script to see whether it works.
Also pay attention to the permission of your build service account.
Besides, since you are using azure VM , also follow this tutorial To set up WinRM for Microsoft Azure Virtual Machines for PowerShell on Target Machines task.

while calling mstest from powershell, extra directory is being added in the relative paths in c#

I will try to make it clear
I have a MSTest Project called IntegrationTests
I have a PowerShell script inside IntegrationTests folder. This script runs the test using MSTest command line arguments.
The tests are being successfully called but after all the tests run there is a call to one method in my test method that creates report. When its trying to load the report.xslt file it is adding extra folder "TestResults"
Unexpected
................
System.IO.DirectoryNotFoundException: Could not find a part of the path 'C:\Users\dev\Desktop\Test Runner\IntegrationTests\TestResults\Report Generator\Reports\report.xslt
Expected
................
C:\Users\dev\Desktop\Test Runner\IntegrationTests\Report Generator\Reports\report.xslt
why is this "TestResults" extra folder is added?
Just to make things clear this project is 100% working when i run this from visual studio.
If you would like to know how the relative path is being constructed in c# here is the code
var directory = Path.GetDirectoryName(Assembly.GetExecutingAssembly().CodeBase);
var xslt = new XslCompiledTransform();
xslt.Load(Path.Combine(directory, "..\\..\\Report Generator\\Reports\\report.xslt"));
I just want to know how can i get rid of "TestResults" so that my test can run normally. Any helps really appreciated.

Running automated Powershell-NUnit tests on TeamCity

I found this neat way to use NUnit in Powershell. http://elegantcode.com/2009/10/25/integration-test-brought-to-you-by-powershell-nunit-with-a-little-specification-syntax-for-flavoring/
and we are using it many of our tests.
However I want to run these tests in TeamCity.
I want similar behavior when we use a NUnit runner for running C# tests in TeamCity ie the build fails when the execution of tests fail. Has anyone of you achieved this? I suspect the Powershell runner will just execute it as a simple script, without any indication whether the test passes or fails.

Take a look at http://confluence.jetbrains.net/display/TCD7/Build+Script+Interaction+with+TeamCity and http://confluence.jetbrains.net/display/TCD7/Build+Failure+Conditions
There is an issue in Powershell runner support http://youtrack.jetbrains.com/issue/TW-21554

I'm not familiar with the approach you're referencing for executing NUnit tests via Powershell in TeamCity. But, we are successfully using PSake for Powershell build scripts, including executing NUnit tests and failing the build appropriately. The same issue exists with PSake and TeamCity with exit codes, but you can get around it by specifying in TeamCity in the Script Source for the Build Step using -Command for Script execution:
import-module .\tools\psake\psake.psm1
$psake.use_exit_on_error = $true
invoke-psake build.ps1
remove-module psake
You can also integrate the Test results into the TeamCity using the Build Feature option in TeamCity Build Steps.