So that we may perform front-to-back web UI testing, we are using Selenium and ChromeDriver to automate page loads/interaction as part of our testing pack.
This is behaving as expected during developer testing (on a developer's local machine), but we are struggling to perform these checks as part of our continuous integration build.
Our server plant is *NIX based, and all of our CI infrastructure runs on these machines. So that we may test Chrome under Windows (our delivery mechanism), we have configured a Selenium Grid. When the CI tests run, they access the grid, in order to locate a Windows node to run the tests on.
We have had a Windows desktop provisioned solely for the purpose of running these tests. This contains our standard enterprise build of Windows 7. This machine will be periodically rebooted in-line with the IT department's update policy.
In an effort to ensure the Selenium server is always running, we have added the Selenium Server (running in "node" mode) as a Windows service. The selenium Server is configured to start-up ChromeDriver to invoke the simulated user-interaction.
However, when running the tests from CI they fail due to timeout. Our working theory is, the system user that is running the service cannot create interactive windows. A web search has raised reference to the "Session 0" problem, but with little to no constructive advice on how to move forward.
Starting the Selenium Server process manually from an interactive session is not a viable solution, as this is leading to brittle tests - which are failing due to an infrastructure problem, rather than a genuine test regression.
How can we have an instance of Selenium Server started via a Windows Service whenever the system reboots, that is capable of launching Chrome instances?
It could be easily done with NSSM.
Installation of services looks like these:
nssm install seleniumhub java -jar C:\selenium\selenium-server-standalone-2.45.0.jar -role hub -hubConfig C:\selenium\hub.json
nssm install seleniumnode java -jar C:\selenium\selenium-server-standalone-2.45.0.jar -role node -nodeConfig C:\selenium\node.json
It provides easily way to remove service if needed:
nssm remove seleniumnode confirm
Add destination to nssm to your PATH variable and run from console as admin
UPDATE April 2021
NSSM is not supported for more than 3 years. So please consider other options like winsw or any other. WinSW does the same job as NSSM and allows to keep run configuration in xml.
You cannot run Selenium Grid as a windows service ever since Windows Vista. Microsoft calls it "Session 0 Isolation". You could do it in Windows 2000 or XP but since the time that Vista came out, Microsoft no longer will let Grid interact with the desktop (or any other UI programs for that matter). Regardless of the fact that you still see that "interact with desktop" checkbox, it is a red herring. So, you MUST run Selenium Grid in the foreground on that server in order for it to get access to the session. If it is running Windows Server, you could in theory have multiple sessions and leave Grid running in the foreground on one of the non-zero user sessions.
Right now you can't help it - it used to work fine in session 0 but for the past few days after chrome update only works for interactive sessions.
Related bugs:
https://code.google.com/p/selenium/issues/detail?id=8029
https://code.google.com/p/chromium/issues/detail?id=422218
My preferred solution to this problem (and my default choice for running Selenium Grid as a service) is to use a simple tool called AlwaysUp. It has a free 30 day trial to try it out.
What to do:
Download AlwaysUp
Configure AlwaysUp to start the Selenium Grid node on startup
Configure AlwaysUp to run the Selenium node as a specific user (not the default System user)
This way the the node will run as a service, survive machine restarts and work with the latest version Chrome.
If the user account you use to login to the machine is different from the user account you specify to run the node as a service then you will not see the browsers pop up on the desktop as they are running in a different user session. The end result is that it is almost identical to running as a normal service but gets round the Session 0 issue.
Yeah, you should use NSSM. Important is, that you add your windows account in the "Log on" tab, or any other valid account. If you run your node with the "Local System account" option, you will get the session 0 problem. With a normal user session, the nodes run smoothly invisible in the background :)
we don't use selenium GRID, we were disappointed with its stability. We use a "Jenkins Grid", that is jenkins slaves nodes on various servers.
The slaves are services with the interact with desktop flag. They run as services with NSSM, and the SERVICE_INTERACTIVE_PROCESS flag. Making sure that NoInteractiveProcess is set to 1 (cf https://learn.microsoft.com/en-us/windows/desktop/services/interactive-services).
We don't have the fancy features of the grid (that is balancing according the browser types slots). Instead, we have Jenkins balancing the test jobs using a slave node or another.
Initially we did not use the interact with desktop flag, having browsers to run without "real" display, but the behavior was not very stable (especially with resize commands).
Hope this helps.
As I explained on this thread, I found that using a small paid tool FireDaemon Pro saved me a lot of time from trying to configure NSSM and other free tools.
It works well in the background, and restarts Selenium along with the server, which was my main requirement for running Selenium Standalone Server as a Windows Service.
This free tool would probably do it:
http://yajsw.sourceforge.net/
For that to work, you need a wrapper.conf file and a script to run the YAJSW wrapper. I takes time to read the documentation, but it is a free solution.
I wrote an example shared here, that installs JBoss7 as a Windows service.
Of course, you can simplify my example by a lot.
I am trying to clean up the process around our UI Automation testing.
Right now we have a console application that we run on a server that listens for RPC calls using Apache Thrift, depending on the test being run this application will either launch Selenium with the browser the user has specified (Chrome, IE, Firefox) or it will launch our WPF desktop application.
My goal is to allow this console application to be run as a windows service for a couple puposes:
A user is no longer needed to be logged into the server.
A third party app is no longer needed to prevent the screen saver from activating.
Will effectively allow the tests to be run in a farm so that multiple can be run in parallel
So my question is provided a server with no active user sessions and this service running, is it possible for the service to spin up both Selenium with a web browser and the WPF application and be able to drive them still.
If it isn't possible, what about programatically logging into the server?
The only way you can do this is by using PhantomJS ( or some other headless browser ). Windows services do NOT have access to running browsers on a desktop (although they used to back in the Windows XP and Windows 2000 Server days). Since then Microsoft has locked down those permissions even though you see a 'interact with desktop' checkbox on the service, I don't think it will work.
Also, Selenium Grid servers MUST run in the foreground on a remote server (for the same reasons). The only exception is, using PhantomJS on the Grid, your grid servers can run as a service to start headless browsers. Of course, running Grid node as a service and grid hub as a service is not a trivial thing to configure (using YAJSW) but its doable. Your milage may vary if you go that route as I have not actually tried it. The idea is that you could write batch DOS scripts to start/register YAJSW to run grid as a service.
I'm trying to automate a third party windows application. I'm using UIAutomation which is part of .net 3.0 framework.
It works fine when run locally but if I deploy it in a windows server 2003 and make it run scheduled without logging in it does not work. I've done a fair amount of googling and I understand that for security reasons windows may not permit this. Is there a way to achieve this ? Its very similar to doing automated UI testing. How do people do large number of automated UI testing (without opening the server in remote and watching the test cases execute)
My solution is a windows application. (Ideally I would want it to be a web application which automates a manual task on a windows software installed on the server but that seems to be even more of a challenge)
Desperate ! Will provide more information on request.
I am trying to do some unit testing in for a Web API project. I am going simulate the web API hosting environment. It seems like that I could use In memory host (HttpServer) or self host (HttpSelfHostServer).
Just wondering what are the difference and which technology is good for what and is there any limitation for those options.
You should use in memory host for end-to-end tests and then test the network connectivity of your environment separately.
For a number of reasons:
In memory host, as the name suggests, runs entirely in memory so will be much faster
Self host needs to be run with elevated privileges, so your tests will need to be executed within the context of an "admin" identity. This is far from desired. It is especially troublesome if you want to execute tests from i.e. build scripts or from PowerShell, since, as a result, these processes would also have to be started with elevated privileges. Moreover, this will have to happen on any of the servers you test on.
In self host you end up testing the given operating system’s networking stack which really is something that shouldn't be tested – since it might differ across different environments (development, staging, QA, production and so on). For example - a given port might not be available. As a result you might get dragged into unnecessary debugging efforts across different machines to even get the tests running.
Finally, testing using self-hosting, still doesn't guarantee that the service will run correctly when web-hosted and vice versa - so you might as well just test in memory
I added some simple WatiN tests to our app today to check that a cookie value is stored correctly.
The tests pass locally on all machines in the team. However, when CruiseControl runs the tests on our Build server these new tests fail on the line containing
browser.GetCookie(url, cookieName)
The error given in the CruiseControl log is the old chestnut of:
Object reference not set to an instance of an object.
I have logged on to the Build server with Remote Desktop, using the same user account that CruiseControl runs under, and run MbUnit manually, and the tests pass. So it can't be a problem with the permissions on the Build server to access cookies.
I have looked through all the WatiN documentation for help, but come up empty. I've restarted the CruiseControl service. I've tried everything I can think of and I'm now completely at a loss now as to what could be different in the way Cruise Control runs these tests.
Does anybody know what could be causing this and/or how to resolve it?
Try running CruiseControl as a application instead of as a service. WatiN tends to hang when run from a service (since windows services are not attached to a UI session, and WatiN requires one for handling dialog boxes in IE).
But besides that ... recall that IE cookies are stored in your user profile. Profiles are not loaded when services run -- they are daemon processes that run quietly in the background and don't actually run in the context of a logon session. I suspect that's the cause of your exception.