We Keep Coding

C# specflow Selenium tests working fine in local (CLI and VS 2017 NUnit Runner). But failing in team city with error Could not find element

I am working on building Smoke Tests (3 tests) for our company's web portal, which is built using C#, N-Unit, Selenium and Specflow.
These smoke tests are working fine in local in Visual Studio N-Unit Runner and Command Line interface (CLI). But, when we run the same tests in TeamCity CI or in CLI from TeamCity Agent server, one or two tests failing with error, such as below.
"OpenQA.Selenium.NoSuchElementException : Could not find element by: By.CssSelector: #u_r_v_search_btn"
Appropriate web element locators are in place in the test suite and looks good when we run in local. But throwing error when we run the tests in TeamCity CI pipeline.
Below command, which is working fine in local. Same command failing in TeamCity Agent Server when we run it from CLI; With error such as "could not find element".
----> nunit3-console.exe ..\..\..\Tests\Bin\Debug\Tests.dll --noheader --where cat==smoke
Team City CI commmand statement from build log:
C:\buildAgent\tools\NUnit.Console.3.9.0\nunit3-console.exe C:\buildAgent\temp\buildTmp\eahe9vWdDNwsMCnwYM0DJIHv4X7ycBFR.nunit --result=C:\buildAgent\temp\buildTmp\eahe9vWdDNwsMCnwYM0DJIHv4X7ycBFR.nunit.xml --noheader --where cat==smoke
Can someone please provide insights to resolve this issue?

Visual Studio Unit Test runs fine, MSTEST does not

So I have this really simple test:
[TestMethod]
public void CheckAllParametersExist()
{
try
{
new ParametersService().Get();
Assert.IsTrue(true);
}
catch(Exception ex)
{
Assert.Fail(ex.Message);
}
}
ParametersService().Get() runs through a parameters class finding all the properties and tries to populate them with values from a database. Occasionally when we release the website we might forget to publish some values, so I wanted a unit test in Bamboo to identify any parameters we may have missed.
To the crux of my problem: In Visual Studio 2017 the unit test passes fine. With MSTest is fails with:
Assert.Fail failed. The type initializer for 'System.Data.Entity.Internal.AppConfig' threw an exception.
Looking this particular error up it looks like I've got the wrong or mixed versions of EntityFramework. Having tried to fix this myself I have removed Entity framework from the Tests project nuget and app.config, then under the solution I consolidated the EntityFramework nuget and included Tests project again, still the same error. App.config shows version 6.0.0.0, nuget has installed 6.2.0
I am stuck, if anyone can suggest any solutions or identify any reasons why I might be seeing this problem I would be greatful.
Fyi: I am running MSTest with /testcontainer:tests.dll in the Tests project bin output debug folder.

So it turns out the problem was with the App.Config. The problem is that we have some sections of the configuration loaded from external files. In this case connection strings. In VS it loads the config file for connections strings just fine. In MSTest it copies the DLLs to another folder but doesn't include the folders/config files.
Also MSTest has now been retired, as of VS2013 we should be using VsTest.console, however Bamboo hasn't caught up with that yet.

How do I run Typescript tests for Jasmine\Karma through a TFS Build Process

I have a web application using Angular 1.5 with bower/npm/gulp coded in Typescript to do our build. Our back end is a c# .net WebApi2. Both are built and deployed on TFS2015. My c# nUnit tests are easy to integrate as part of the build process. The Typescript jasmine unit tests however are more difficult to integrate. How do I get my Typescript jasmine unit tests to run as part of the TFS build and if they fail, fail the build? We have them running through a Jasmine Spec runner and also Karma but not integrated.
I have read the many posts on StackOverflow integrating Javascript unit tests and each avenue took me through an overly complex solution that didn't work. These include Powershell scripts, Chutzpah amoungst others.

Rather than try to recreate the Specrunner via Chutzpah on the build server, which I found difficult to configure and get working. The aim was to get karma to output the running tests in the 'trx' test format that TFS recognises and then publish them to the build. Please note I am using PhantomJs to run my tests through Karma but won't cover that here as it is well covered elsewhere.
1) install the karma-trx-reporter plugin via npm into your web project (or similar plugin)
2) Configure the Karma.config to include the trx reporter
reporters: ['dots', 'trx'],
trxReporter: { outputFile: 'test-results.trx' },
// notify karma of the available plugins
plugins: [
'karma-jasmine',
'karma-phantomjs-launcher',
'karma-trx-reporter',
],
3) Create a Gulp (or grunt) task to run the karma tests if you don't already have one. Run the task locally and check it creates the 'test-results.trx' specified above. (It doesn't matter where the file is created on the build server):
gulp.task('test', function () {
return gulp.src(['tests/*.js']).pipe(karma({
configFile: __dirname + '/Testing/karma.config.js',
singleRun: true
}));
});
4) Add a Gulp (or Grunt) TFS build task to run the karma tests created in the previous step and output the trx file.
5) Add a Gulp (or Grunt) TFS build task to Publish the test results and merge them into the build. Note that the "Test Result Files" path is a wild card **/*.trx to find any trx files in the build path (i.e. finds our previously created file). "Merge Test results" is checked to Merge both our Jasmine test run and our c# test run into the same session. "Continue on error" is unticked to ensure any jasmine test failures break the build.
You will notice two sets of tests that have been run and included as part of the build!

UnitTestIsolationException: Throws Exception while running Shims test in Release/Debug mode

I am trying to run this unit test using Microsoft Shims, but it throws me exception in Shims.Context.Create(); method.
Environment: VS 2012, Win2K8 R2
namespace MyShimsUnitTest
{
[TestClass]
public class MyUnitTest
{
[TestMethod]
public void GetCurrentYear()
{
using (Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create())
{
// Some Logic...
}
}
}
}
Detailed Exception:
Result Message:
Test method MyShimsUnitTest.MyUnitTest.GetCurrentYear threw exception:
Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: UnitTestIsolation instrumentation failed to initialize. Please restart Visual Studio and rerun this test
Result StackTrace:
at Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
at Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext()
at Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create()
at MyShimsUnitTest.MyUnitTest.GetCurrentYear()

Shims require runtime instrumentation performed by the IntelliTrace profiler. The test runner is responsible for setting up the environment variables required for CLR to load the profiler as well as providing the list of types the profiler must instrument for Shims. The UnitTestIsolationException is thrown when the ShimRuntime is unable to locate and attach to the IntelliTrace profiler, which it expects to be already loaded by the CLR.
As Jin-Wook mentioned earlier, this problem occurs when the test is executed by a runner that does not perform the required profiler initialization. Test Explorer and the vstest.console.exe are two such runners that ship with Visual Studio 2012. At this time, the Visual Studio test runners do not perform the required profiler instrumentation when running tests in "legacy" mode, which happens when you have a .TESTSETTINGS file selected for your run or a .RUNSETTINGS file that forces legacy mode.
You may be able to use third-party test runners that support profiler instrumentation required by Shims.

I had the same issue. The solution to my problem was to uncheck the selected .testsettings file from the menu: TEST/Test Settings and here the item(s) above the Select Test Settings File.

It could be caused by not using the test explorer of vs 2012. To use the shim, you should run tests only using the test explorer.
You can use other test framework such as Nunit or Xunit with the shim if installing appropriate test runner for vs 2012. It can be downloaded from the vs extension manager.

I ran into this issue too. Thankfully the other answers here helped me fix my issue:
I'm using Resharper and when using the context menu I noticed that the runner is using MSTest. Even when finding the test in test explorer and selecting debug I received the same exception.
I then went into Resharpers's options and under Tools -> Unit Testing -> MsTest I unchecked "Enable MSTest support". This unfortunately disables the option to right click on your test and hit run/debug, but it did allow ShimsContext.Create() to behave correctly when selecting debug from the Test Explorer view!

Go to your TestProject Properties -> Under Debug section Check the "ENABLE NATIVE CODE DEBUGGING" checkbox.
This is should do.

We saw this error reported by Bamboo, our build server. It was invoking an MSbuild 4.0 task. The unit test work fine on the dev's local PCs. I deleted this bamboo task and created a new task that invokes Visual Studio 2012's vstest.console. The tests now pass but Bamboo is not able to count the number of tests. This is a Bamboo problem not mine.

Running automated Powershell-NUnit tests on TeamCity

I found this neat way to use NUnit in Powershell. http://elegantcode.com/2009/10/25/integration-test-brought-to-you-by-powershell-nunit-with-a-little-specification-syntax-for-flavoring/
and we are using it many of our tests.
However I want to run these tests in TeamCity.
I want similar behavior when we use a NUnit runner for running C# tests in TeamCity ie the build fails when the execution of tests fail. Has anyone of you achieved this? I suspect the Powershell runner will just execute it as a simple script, without any indication whether the test passes or fails.

Take a look at http://confluence.jetbrains.net/display/TCD7/Build+Script+Interaction+with+TeamCity and http://confluence.jetbrains.net/display/TCD7/Build+Failure+Conditions
There is an issue in Powershell runner support http://youtrack.jetbrains.com/issue/TW-21554

I'm not familiar with the approach you're referencing for executing NUnit tests via Powershell in TeamCity. But, we are successfully using PSake for Powershell build scripts, including executing NUnit tests and failing the build appropriately. The same issue exists with PSake and TeamCity with exit codes, but you can get around it by specifying in TeamCity in the Script Source for the Build Step using -Command for Script execution:
import-module .\tools\psake\psake.psm1
$psake.use_exit_on_error = $true
invoke-psake build.ps1
remove-module psake
You can also integrate the Test results into the TeamCity using the Build Feature option in TeamCity Build Steps.