We've created a Selenium test project that starts the (ASP.NET) web application and runs a couple of tests using the ChromeDriver. Locally this all runs fine (in headless and non-headless mode).
But on the build server (using an Azure DevOps agent) this fails without ever starting the tests. It looks like it fails when starting the ChromeDriver: the driver starts, but then it's immediately followed by 403 errors. It never gets to the part where it actually loads a webpage.
Any ideas where to look?
Answering my own question to document possible solutions.
After some rigorous investigation (which included using the source code to get to the bottom of things) we found out that the proxy server somehow got in the way. It turned out that the ChromeDriver tries to communicate over a local port (e.g. http://localhost:12345), which was redirected through the proxy server. This failed with a 403 error.
This gave us a lead on possible solutions. First we tried to use the .proxybypass file to exclude localhost addresses. This didn't work -- it turns out that this proxy bypass only works for https requests. And the ChromeDriver control commands are sent over http :-(
We then made sure that no proxy was used in our test code. We did this with the following lines:
var options = new ChromeOptions();
options.AddArgument("--no-sandbox");
options.AddArgument("headless");
options.AddArgument("ignore-certificate-errors");
options.Proxy = new Proxy()
{
Kind = ProxyKind.Direct
};
var driver = new ChromeDriver(options);
In addition to these settings (note that some arguments were added to solve other issues and might not apply to your own situation), we also disabled the proxy for other requests:
WebRequest.DefaultWebProxy = null;
HttpClient.DefaultProxy = new WebProxy()
{
BypassProxyOnLocal = true,
};
This allowed our tests to finally run on the build server without the 403 errors.
One last remark (which might be obvious) is to always run your tests in non-headless mode if you encounter any issues. This allowed us to see the "invalid certificate error" which would otherwise be hidden.
Currently, I'm trying to run my Selenium tests on Safari using Selenium Grid and RemoteWebDriver. This is my setup:
Mac OS Sierra 10.12.6 as a machine for running tests.
Selenium server 3.5.3.
Safari 11.
C# Selenium WebDriver and Selenium Support (latest version).
I'm using port forwarding on my host OS (Windows 10) to forward requests to Mac, running on my Virtual Machine. On my Mac I have Selenium Grid hub, which I run like this:
java -jar selenium-server-standalone-3.5.3.jar -role hub -port 4723
Also, there is a node:
java -jar selenium-server-standalone-3.5.3.jar -role node -hub http://10.0.2.15:4723/grid/register
In code, I start my driver like this:
SafariOprions options = new SafariOptions();
IWebDriver driver = new RemoteWebDriver(new Uri(hubURL), options.ToCapabilities());
My tests are running fine with current setup. But when it comes to clicking a link with attribute target='_blank' I'm starting to face some troubles. For other drivers, which I run locally, I can switch tab without any trouble: I'm getting driver.WindowHandles before I click a link, then I click a link and again retrieve Window Handles to compare with previous handles. After that I use driver.SwitchTo().Window(newHandle) and everything is ok.
But when it comes to RemoteWebDriver (or SafariDriver from Apple, I cannot say more precisely) I'm always getting only one Window Handle, even if the new tab is opened and i can see it.
I'm trying to avoid switching tab with "Command + T" as one of solutions suggested, because my tests are meant to be run on all browsers (Chrome, Firefox, Opera, Edge, Safari) and this wont work.
UPDATE: I've tried running Chrome and other browsers in Selenium Grid via RemoteWebDriver and I can say that this is not an issue of RemoteWebDriver. Next, I've installed Visual Studio for Mac and rewrite several things to run my tests without Selenium Grid, just using this code:
//if memory serves, just like this
SafariOptions options = new SafariOptions();
SafariDriver driver = new SafariDriver(options);
But, unfortunately, this didn't help. Driver navigated me to the page, clicked the link and opened a new tab, but without any switch. When I checked for driver.WindowHandles I've only got one, although there was two visible tabs. Neither driver.SwitchTo().ActiveElement nor driver.SwitchTo().Frame(hardcoded_frame_name) doesn't seem to work. Pretty long waits (for 60 seconds after opening the link and another one after that) aren't working too. Now I think that this is really a bug and I will try to report this to Apple as soon as I can.
But for now, maybe someone has a fancy workaround for this?
As we discussed in the comments, it seems to be a timing issue. So we will induce
WebDriverWait to sync up with the trailing Browser instance. I am providing a code block as a solution through Selenium-Java, consider implementing it in C# and update me if it works for you.
driver.get("http://www.google.com");
System.out.println("Page Title is : "+driver.getTitle());
String parent_window = driver.getWindowHandle();
((JavascriptExecutor) driver).executeScript("window.open('http://facebook.com/');");
WebDriverWait wait = new WebDriverWait(driver,3);
wait.until(ExpectedConditions.numberOfWindowsToBe(2));
Set<String> allWindows_1 = driver.getWindowHandles();
ArrayList<String> tabs = new ArrayList<>(allWindows_1);
driver.switchTo().window(tabs.get(1));
wait.until(ExpectedConditions.titleContains("Facebook"));
System.out.println("First Child Handle : "+driver.getTitle());
I could help you with the Java version:
After the actions done, do this below.
//Store the parent window
String parentWindow = driver.getWindowHandle();
//Open a new Windows(Mailtrap)
String a = "window.open('https://mailtrap.io/signin','_blank');";
((JavascriptExecutor)driver).executeScript(a);
//This Thread.sleep is useful with Safari. Do not remove.
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
//Take control over new browser
for(String handle: driver.getWindowHandles()){
driver.switchTo().window(handle);
}
I am trying to open url with Selenium and Google chrome, however i always end up with chromedriver.has stopped working.
ChromeDriver driver = new ChromeDriver(#"Path\To\The\Driver");
driver.Navigate().GoToUrl("https://www.google.com/");
i tried to sleep between initializing and going to url, however it does nothing.
As stated above, chromedriver version 2.25 will work. The problem with version 2.25 is that it crashes when running. You can see here the update history.
https://sites.google.com/a/chromium.org/chromedriver/download .
I would suggest you use a later version of the driver.
Here is the link to the drivers that was given to me by visual studio. http://chromedriver.storage.googleapis.com
Choose a chrome driver version 2.37 for windows selenium version 3.11.1.
Any news on this topic? I have the same issue. The execution stops at driver.Navigate().GoToUrl(desiredUri); where the login window is shown. It waits for me to enter the username and password and manually it can be done but the execution has stopped there and it does not go to my next line var alert = driver.SwitchTo().Alert(); from where I want to add the username and password.
I've been using Selenium for a number of months, which we're using to automate some of our internal testing processes. The scripts have been passing fine. I've recently upgraded to C# 2.40.0 webdriver using FF 27.01 and our scripts are now failing in random places with the following error.
[Portal.SmokeTest.SmokeRunTest.Booking] TearDown method failed. OpenQA.Selenium.WebDriverException : The HTTP request to the remote WebDriver server for URL htt(p)://localhost:7055/hub/session/56e99e88-ba17-4d12-bef1-c6a6367ccc2f/element timed out after 60 seconds.
----> System.Net.WebException : The operation has timed out
TearDown : OpenQA.Selenium.WebDriverException : The HTTP request to the remote WebDriver server for URL htt(p)://localhost:7055/hub/session/56e99e88-ba17-4d12-bef1-c6a6367ccc2f/window timed out after 60 seconds.
----> System.Net.WebException : The operation has timed out
[09:01:20]
[Portal.SmokeTest.SmokeRunTest.Booking] TearDown method failed. OpenQA.Selenium.WebDriverException : The HTTP request to the remote WebDriver server for URL htt(p)://localhost:7055/hub/session/56e99e88-ba17-4d12-bef1-c6a6367ccc2f/element timed out after 60 seconds.
----> System.Net.WebException : The operation has timed out
TearDown : OpenQA.Selenium.WebDriverException : The HTTP request to the remote WebDriver server for URL htt(p)://localhost:7055/hub/session/56e99e88-ba17-4d12-bef1-c6a6367ccc2f/window timed out after 60 seconds.
----> System.Net.WebException : The operation has timed out
at OpenQA.Selenium.Support.UI.DefaultWait`1.PropagateExceptionIfNotIgnored(Exception e)
at OpenQA.Selenium.Support.UI.DefaultWait`1.Until[TResult](Func`2 condition)
at Portal.Test.Helpers.Process_Bookings.OpenBookings.SelectBooking(String bookingnumber)
at Portal.SmokeTest.SmokeRunTest.Booking() in d:\TeamCityAgent\work\dac1dcea7f2e80df\SmokeTests\SmokeRunTest.cs:line 68
--WebException
at System.Net.HttpWebRequest.GetResponse()
at OpenQA.Selenium.Remote.HttpCommandExecutor.CreateResponse(WebRequest request)
--TearDown
at OpenQA.Selenium.Remote.HttpCommandExecutor.CreateResponse(WebRequest request)
at OpenQA.Selenium.Remote.HttpCommandExecutor.Execute(Command commandToExecute)
at OpenQA.Selenium.Firefox.Internal.ExtensionConnection.Execute(Command commandToExecute)
at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
at OpenQA.Selenium.Remote.RemoteWebDriver.Close()
at Portal.Test.Helpers.Setup.CloseWebdriver()
at Portal.SmokeTest.SmokeRunTest.TearDown() in d:\TeamCityAgent\work\dac1dcea7f2e80df\SmokeTests\SmokeRunTest.cs:line 162
--WebException
at System.Net.HttpWebRequest.GetResponse()
at OpenQA.Selenium.Remote.HttpCommandExecutor.CreateResponse(WebRequest request)
The latest error I've managed to track down to one single line of code:
_setup.driver.FindElement(By.XPath("//button[#class='buttonSmall lockBookingButton']")).Click();
The annoying thing is, trying to fix the problem is proving difficult, as if I run the test on my local machine, in debug it passes. Additionally, if I run it via the NUNIT runner on the build machine I'm running the test off, it also passes. It only seems to fail as part of our automated build running process when using Teamcity. Like I said, this has been running fine for months previously, and the only thing that has changed is the selenium webdriver kit.
I have experienced this problem before, whilst in debug, and when a Click() line of code was called, Firefox appeared to lock up, and only stopping the test would allow Firefox to continue. There are a number of suggestions on here including modifying the webdriver source? I'd like to not go down that route if possible if anyone else can offer any suggestions.
I had a similar issue using the Chrome driver (v2.23) / running the tests thru TeamCity. I was able to fix the issue by adding the "no-sandbox" flag to the Chrome options:
var options = new ChromeOptions();
options.AddArgument("no-sandbox");
I'm not sure if there is a similar option for the FF driver. From what I understand the issue has something to do with TeamCity running Selenium under the SYSTEM account.
new FirefoxDriver(new FirefoxBinary(),new FirefoxProfile(),TimeSpan.FromSeconds(180));
Launch your browser using the above lines of code. It worked for me.
I first encountered this issue months ago (also on the click() command), and it has been an issue for me ever since. It seems to be some sort of problem with the .NET Selenium bindings. This blog post by the guy that works on the IE driver is helpful in explaining what's happening:
http://jimevansmusic.blogspot.com/2012/11/net-bindings-whaddaymean-no-response.html
Unfortunately, there doesn't seem to be a real solution to this problem. Whenever this issue has been raised to the Selenium developers (see here), this is a typical response:
We need a reproducible scenario, that must include a sample page or a link to a public site's page where the issue can be reproduced.
If you are able to submit a consistently reproducible test case, that could be very helpful in putting this bug to rest for good.
That said, perhaps you can try this workaround in the meantime. If the HTML button that you are trying to click() has an onclick attribute which contains Javascript, consider using a JavascriptExecutor to execute that code directly, rather than calling the click() command. I found that executing the onclick Javascript directly allows some of my tests to pass.
Had same issue with Firefox. I switched over to Chrome with options and all has been fine since.
ChromeOptions options = new ChromeOptions();
options.AddArgument("no-sandbox");
ChromeDriver driver = new ChromeDriver(ChromeDriverService.CreateDefaultService(), options, TimeSpan.FromMinutes(3));
driver.Manage().Timeouts().PageLoad.Add(System.TimeSpan.FromSeconds(30));
In my case, my button's type is submit not button and I change the Click to Sumbit then every work good. Something like below,
from driver.FindElement(By.Id("btnLogin")).Click();
to driver.FindElement(By.Id("btnLogin")).Submit();
BTW, I have been tried all the answer in this post but not work for me.
Got similar issue. Try to set more time in driver's constructor - add eg.
var timespan = TimeSpan.FromMinutes(3);
var driver = new FirefoxDriver(binary, profile, timeSpan);
In my case, it's because I deleted the chrome update folder. After chrome reinstall, it's working fine.
In my case none of the answers above solved my problem completely.
I ended up using the (no-sandbox) mode, the connection with extended timeout period (driver = new RemoteWebDriver(new Uri("http://localhost:4444/wd/hub"), capability, TimeSpan.FromMinutes(3));) and the page load timeout (driver.Manage().Timeouts().PageLoad.Add(System.TimeSpan.FromSeconds(30));) so now my code looks like this:
public IWebDriver GetRemoteChromeDriver(string downloadPath)
{
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.AddArguments(
"start-maximized",
"enable-automation",
"--headless",
"--no-sandbox", //this is the relevant other arguments came from solving other issues
"--disable-infobars",
"--disable-dev-shm-usage",
"--disable-browser-side-navigation",
"--disable-gpu",
"--ignore-certificate-errors");
capability = chromeOptions.ToCapabilities();
SetRemoteWebDriver();
SetImplicitlyWait();
Thread.Sleep(TimeSpan.FromSeconds(2));
return driver;
}
private void SetImplicitlyWait()
{
driver.Manage().Timeouts().PageLoad.Add(TimeSpan.FromSeconds(30));
}
private void SetRemoteWebDriver()
{
driver = new RemoteWebDriver(new Uri("http://localhost:4444/wd/hub"), capability, TimeSpan.FromMinutes(3));
}
But as I mentioned none of the above method solved my problem, I was continuously get the error, and multiple chromedriver.exe and chrome.exe processses were active (~10 of the chromedriver and ~50 of chrome).
So somewhere I read that after disposing the driver I should wait a few seconds before starting the next test, so I added the following line to dispose method:
driver?.Quit();
driver?.Dispose();
Thread.Sleep(3000);
With this sleep modification I have no longer get the timeout error and there is no unnecessarily opened chromedriver.exe and chrome.exe processses.
I hope I helped someone who struggles with this issue for that long as I did.
I think this problem occurs when you try to access your web driver object after
1) a window has closed and you haven't yet switched to the parent
2) you switched to a window that wasn't quite ready and has been updated since you switched
waiting for the windowhandles.count to be what you're expecting doesn't take into account the page content nor does document.ready. I'm still searching for a solution to this problem
In my case I found this error happening in our teams build server. The tests worked on our local dev machines.
The problem was that the target website was not configured correctly on the build server, so it couldn't open the browser correctly.
We were using the chrome driver but I'm not sure that makes a difference.
The problem is that the evaluation of Click() times out on your build env.. you might want to dig into what happens on Click().
Also, try adding Retrys for the Click() because occssionally the evaluations take longer time depending on network speeds, etc
In my case the issue was with SendKeys() and Remote Desktop. Posting the workaround I have so far:
I had a Selenium test which would fail when run as part of a Jenkins job on a node hosted in vSphere and administered through RDP. After some troubleshooting it turned out it succeeds if Remote Desktop is connected and focused but fails with the exception if Remote Desktop is disconnected or even minimized.
As a workaround, I logged through vSphere Console instead of RDP and then even after closing vSphere the test didn't fail anymore. This is a workaround but I would have to be careful never to login through RDP and always to administer only through vSphere Console.
We had the same problem. In our case, the browser was blocked by a login popup (Windows authentication), so not returning after 60 seconds. Adding correct access rights to the Windows account Chrome was running under solved the problem.
I had the same exception when trying to run a headless ChromeDriver with a scheduled task on a windows server (unattended). What solved it for me is to run the task as the user "Administrators" (notice the S at the end). What I also did (I don't know if its relevant) is selected the "Any Connection" from the task "Conditions" tab.
I was getting this issue. I'd neglected to update the Selenium.WebDriver.ChromeDriver nuget package to match the version of Chrome I was using. Once I did that the issue was resolved.
changing the Selenium.WebDriver.ChromeDriver from 2.40.0 to 2.27.0 is ok for me
The new FirefoxDriver(binary, profile, timeSpan) has been obsolete.
You can now use new FirefoxDriver(FirefoxDriverService.CreateDefaultService(), FirefoxOptions options, TimeSpan commandTimeout) instead.
There is also a new FirefoxDriver(string geckoDriverDirectory, FirefoxOptions options, TimeSpan commandTimeout) and it does works. But it's undocumented, and you need to manually specify geckoDriverDirectory even though it's already in Path.
Arrrgh! Faced this on macOS today and the issue was as simple as - the pop-up window suggesting to install the new Appium version was showing on the remote CI build server.
Just VNC'ing to it and clicking "Install later" fixed it.
I see this issue as well when running my tests so have been searching for the 'why' and the 'how to fix'.
My initial thought (after looking at the screenshot failures and the timeout message) was that the website performance must have gotten worse and that the website is taking longer to do some stuff which is causing the issue.
I've found a lot of solutions (which I plan to try out) HOWEVER, it seems like Most of the solution sort of indicate that the website performance (reaction time) got slower. If you have to change wait for 30 seconds to wait for 3 minutes, it is true that this will allow the test to pass ... but doesn't this mean we are having to wait around 3 minutes for the website to do something?
Question: Can 90% of the time this error message be chalked up to degradation of the website performance? I read some sites mentioned above and it sounds like the issue could also be with the chromedriver causing delays (maybe nothing to do with application issues). Is anyone brining this issue up to their application development team or are you just changing the wait times and not reporting?
I posted in selenium slack channel on how the timeout that we pass for Chromedriver constructor works and got this answer(C#).
"The timeout that you can set in the driver constructor is the 'command timeout' which is essentially the read timeout for the http client. The http client knows nothing about what the driver is doing.
If the page never loads and the command timeout is less than the page load timeout, you'll get an error saying "HTTP request to the remote WebDriver server timed out".
If the command timeout is greater than the page load timeout, the driver will respond to the client with a timeout error, and Selenium will report the message received by the driver."
For ChromeDriver the below worked for me:
string chromeDriverDirectory = "C:\\temp\\2.37";
var options = new ChromeOptions();
options.AddArgument("-no-sandbox");
driver = new ChromeDriver(chromeDriverDirectory, options,
TimeSpan.FromMinutes(2));
Selenium version 3.11, ChromeDriver 2.37
I'm trying to use Selenium WebDriver to automatically login in to a site with a user-name and password. I've done my research and I don't believe this feature is supported by WebDriver, so I need to find another way. The site I'm trying to automate logging into is located here.
When prompted to login a popup window comes up that doesn't seem to be part of the browser. I'm using Firefox and Chrome. It seems Windows API may be required? I already tried passing the credentials in the URL but that didn't work. Also tried sendkeys, but received a Windows exception that the application was not accepting Windows messages. I also tried switching the current handle using driver.windowhandles but the popup doesn't seem to be a new handle.
Does anybody have any ideas? I'm kinda stuck. The preliminary code to get to the popup window is:
IWebDriver driver = new FirefoxDriver();
driver.Navigate().GoToUrl("http://www.portal.adp.com");
string currentWindow = driver.CurrentWindowHandle;
IWebElement userLogin = driver.FindElement(By.Id("employee"));
userLogin.Click();
The popup you are seeing is prompted by web server and is a authentication prompt. Selenium doesn't support this operation.
One of the way to handle this limitation is to pass user and password in the url like like below:
http://user:password#example.com
More info available here : http://aleetesting.blogspot.in/2011/10/selenium-webdriver-tips.html
I wanted my answer out there because I think I've solved it. This answer does not require passing the credentials through the URL (for those of you that are unable to like me). It also does not require any custom Firefox Profiles or extensions to be installed or included with the solution or installed onto the browser eliminating cross-machine compatibility issues.
The issue with me was that the authentication could not be completed via passing the credentials through the URL because the login was behind a proxy.
So, I turned to windows automation toolkits and found AutoIT. Using AutoIT and Selenium, you can login automatically by sending the username and password to the windows dialog that appears. Here's how (note the steps below are for c#:
1 - Download AutoIT from http://www.autoitscript.com/site/autoit/downloads/
2 - Add the autoit .dll to your project references.
Right click on references, select Add Reference. Next click the browse button and browse to the dll location (most default installations it will be c:\Program Files (x86)\AutoIt3\AutoItX\AutoItX3.dll), and add to project.
3 - use AutoIT and Selenium like this (assuming your web driver is already initialized):
//Initialize AutoIT
var AutoIT = new AutoItX3();
//Set Selenium page load timeout to 2 seconds so it doesn't wait forever
Driver.Manage().Timeouts().SetPageLoadTimeout(TimeSpan.FromSeconds(2));
//Ingore the error
try
{
Driver.Url = url;
}
catch
{
return;
}
//Wait for the authentication window to appear, then send username and password
AutoIT.WinWait("Authentication Required");
AutoIT.WinActivate("Authentication Required");
AutoIT.Send("username");
AutoIT.Send("{TAB}");
AutoIt.Send("password");
AutoIT.Send("{ENTER}");
//Return Selenium page timeout to infinity again
Driver.Manage().Timeouts().SetPageLoadTimeout(TimeSpan.FromSeconds(-1));
Anyway, that's it, and it works like a charm :)
Also note that there are some special characters that need to be escaped in AutoIT using the sequence "{x}". For example, if your password is "!tRocks", you'd need to pass it into AutoIT as "{!}tRocks".
Happy automating.
FirefoxProfile profile = new FirefoxProfile();
profile.SetPreference("network.http.phishy-userpass-length", 255);
profile.SetPreference("network.automatic-ntlm-auth.trusted-uris", hostname);
Driver = new FirefoxDriver(profile);
hostname is your URL (example.com) then try to
Driver.Navigate().GoToUrl(http://user:password#example.com);
I just got done working on a prototype project that is supposed to handle exactly this kind of situation.
It utilizes BrowserMob, a popular open source proxy, to perform the authentication.
SeleniumBasicAuthWrapper Hope it helps! It is still a work in progress, but hopefully we'll get any kinks or defects ironed out in the near future.