I'm currently using Selenium Webdriver with C#. I've successfully executed my test in remote webdriver as well using selenium GRID.
I just configured 5 instances of FF, Chrome and IE in my Grid settings and when I ran my test project on chrome browser, I noticed that just only one instance of chrome is picked. Is this the expected behavior? I was initially in an assumption that the number of tests in a single project will be distributed across multiple browser instances based on the maxinstance and maxsessions. But not sure why it is using just one browser instance for the whole project. Please do let me know if I need to do anything to use more than one browser instance to share/run the test.
Unfortunately the standard NUnit runner doesn't support parallelization out of the box.
There are a few alternative unit testing frameworks that you might want to look into that do support parallelized runs like MbUnit or PnUnit.
One workaround is to split up your test. Some common ways are by DLL, namespace, test name, or category. You could then run your NUnit test in parallel using a MSBuild script.
The final command would look something like this c:\proj> msbuild /m:8 RunTests.xml
Check out the answers to this question for more details: How can I run NUnit tests in parallel?
Related
I am working on an existing test automation framework that uses the following for UI tests on a windows application:-
C# for creating UI cases
winAppDriver to interact with UI objects
nUnit for validation
specflow for BDD
I don't have much idea about microsoft projects. Here is a simple structure of the project:-
Application
Source
Modules
features
Tests
Each test folder has a "app.config" file in it that supplies the config for the test to work like DB username, pwd, services urls etc.
We execute our test case from "Test Explorer" pane in VS Enterprise which is not the best way if the test cases need to be run remotely in jenkins. As I said I don't have much frame of reference when it comes to Microsoft apps. So, here are some questions to which I have been looking for answers; there seems to be no definite consensus online. Just curious to know how others are maintaining their projects. Here goes:-
is there a jenkins friendly way of running these test cases? using a cmd line or a runner file perhaps?
if I find a way to accomplish #1 how do I inject app.config properties at runtime?
how does one execute these cases on remote machine? Mine is a desktop windows app. What would a high-level strategy look like? I assume I will have to get a remote machine and install the app on that machine?
any pointers, resources to read about would be helpful. Just looking for a nudge in the right direction.
Since you are using nUnit, it has a default approach of running tests from console see here
It looks like you need to pass parameters from the test runner into the runtime. You can use this approach
Execute on remote machine strategy depends on your current infrastructure. E. g. if you are using Gitlab, you should set up the GitLab CI runner into your machine and set up the GitLab pipeline.
looking for a nudge in the right direction
See CI\CD test run best practices
Ran into a problem recently with VS/MSTest and wondering if anyone has had a similar issue.
I'm currently working on a UI framework in c# using MSTest and over the past few weeks I've been implementing selenium grid/Jenkins for CI.
Selenium grid facilitates parallel test running but the running is done by MSTest. in the past few weeks parallel test running with VS has stopped working.
No changes have been made to how the tests are structured or created but when multiple driver instances are created Driver A works until Driver B is created then Driver B runs while Driver A stops executing steps.
I'm beginning to think it's a bug with WebDriver or MSTest itself as it's weird for something that was working to stop working without any code changes. Wondering if anyone else has had similar problems.
It had been working for me for months and I had found some issues by running in parallel so it's strange that it has suddenly stopped working with no changes. It was last working around 2 months ago.
I tried changing every static method/variable in the solution to not being static as I've read statics can cause problems but it didn't help.
setup is
Unit Test solution using Seleno (Webdriver Nuget Package)
Tests written as unit tests using in MSTest. Page Object approach.
Driver:Chromedriver 2.33
Chrome version: Version 62.0.3202.75 (Official Build) (64-bit)
Parallel test running enabled using a .Testsettings file in the solution
same thing happening with latest version of IE/IEDriverServer which makes me think it may be WebDriver that is an issue.
Any help or thoughts would be much appreciated.
Thanks,
G
I'm having exactly the same issue. In my case, it's the first time I implement Selenium grid and try to run tests in parallel. They run sequentially.
I've found this:
http://colinsalmcorner.com/post/parallel-testing-in-a-selenium-grid-with-vsts
Apparently the only way to solve it is putting tests in separate .csproj files, or maybe running multiple mstest processes with appropriate list of tests as arguments on each one (with some utility you can create). Please keep me posted.
The issue here ended up being the javascript executor had been made static to add a feature one of the in house teams wanted. Every time a new instance was opened the executor was assigned a new value and inputs happened only in the console of this window.
Implemented feature in a way that didn't require executor to be static and all worked well
I'm just getting started with Selenium, using VS2012 and C#. I'm not sure if using a separate testing framework such as NUnit or HtmlUnit is necessary. I tried out the google search example that is available online in C# without using NUnit and it worked fine.
So my question at this point is, why would i need to use NUnit with Selenium?
You don't NEED to use NUnit (or any other unit testing framework) with Selenium if you don't want to. However, there may be instances where you might want to use NUnit (or others) to leverage the benefits of other things. For example:
If you have existing unit tests it keeps everything in one place. (If that's the way you want to organise things)
If you already use NUnit (or your preferred unit testing framework) for unit tests then you can re-use the same test runner (e.g. NUnit console, NUnit GUI, ReSharper, etc.) you use for NUnit meaning you can run all your tests (NUnit and Selenium) with one button press/keyboard shortcut.
If you use continuous integration it can run your selenium tests through the existing NUnit (or which ever you prefer) test runner which means you don't have to configure your continuous integration server for the selenium tests separately.
The above assumes that you have unit tests already. If you don't already have unit tests, or you are only interested in the Selenium tests (For example, we have a development team and a tester, we write unit tests, the tester writes Selenium tests and they are run independently of each other) then there is no need to add that extra layer.
Unit tests frameworks and selenium test for different things. Unit tests typically look at a single unit of code at a time (although in practice, I find it often spills out into adjacent units especially if they are small and deterministic). Selenium looks at a web page (or series of pages) as a single test. Selenium's tests need a system to have many of its components integrated together to run the test. It is therefore testing at a higher level as it is checking many things at once. (e.g. that the system can cope with requests, that the responses arrive back, that the responses contain the expected data, that pressing buttons on the page do the correct things, go to the correct pages, etc.)
Ultimately, just because you can do something doesn't mean you should. Running Selenium tests through a unit testing framework is a convenience if you have to handle both. It may work for you, it may not.
I wrote unit tests using NUnit. Once all the tests are done, I want to email the Test results to all my team. Is there a way to do it?
Usually this is done using an automated build tool like Cruise Control. It checks your code out of version control, builds the app, runs all the tests, packages the app, and sends it to the first deployment server. Team members can view the complete results of the build and test cycle using a browser to check the build server dashboard.
I'd prefer that to getting e-mailed test results. E-mail would soon become an annoyance.
Since you are using TFS, you can make use of NUnit for Team Build which would make your NUnit test results visible in the build log and incorporated into the data warehouse for reporting.
There is also NUnitForVS which can publish results within TFS.
Going one of the above routes will allow you to leverage the current CI environment and is a much better approach to emailing the results as they surface.
Is there any option available to run c#/.NET tests directly using selenium server like following option for html based tests?
( java -jar selenium-server.jar C# based tests/suites)
-htmlSuite : Run a
single HTML Selenese (Selenium Core) suite and then exit
immediately, using the specified browser (e.g. "*firefox") on the
specified URL (e.g. "http://www.google.com"). You need to specify
the absolute path to the HTML test suite as well as the path to the
HTML results file we'll generate.
Thanks!
To run your C# tests you will need to have the Selenium server running with
java -jar selenium-server.jar
and then you will need to compile the tests and use a .NET testing framework like NUnit, MBUnit or MSTest.
Well, the C# tests aren't java, so I'd say no. Guess you'd have to create a c# class, or something to run the c# API for selenium, or use nUnit. Maybe I'm misunderstanding the question though?
Selenium-RC acts as a web server. It accepts commands (in "selenese") in two fashions - via a HTML file, or via HTTP requests. (It can also accept them on the command line if you start it with the -interactive flag).
The various client languages are APIs that wrap the HTTP protocol. If you are using C#, then you need to write a C# program (possibly using NUnit, or some other .NET-baesd unit testing tool).
If you want a single-command option that will launch the Selenium-RC server and run your tests, you'll need to get your testing tool to execute the java -jar selenium-server.jar command on startup, and then send a shutdown command to it at the end.