Selenium ChromeDriver - Run in background but not headless - c#

I am looking for a way to run selenium tests with a chrome driver, in the background. With the background I mean as in, not the foreground focussed window. I can actually do this, but as soon as actions like opening a new tab or switching between windows (so basically switching window handles) happen in the chrome driver, the browser window gets pushed to the foreground.
So my question is, how can I prevent this from happening without running the test headless?
Any suggestions are appreciated, open for discussion.
EDIT
As a somewhat temporary solution I came up with the following.
Using the Windows 10 Virtual Desktops feature, I run the test and thus the chrome browser window in a seperate virual desktop.
I then switch back to my main virtual desktop to continue with other tasks.
This prevents the chrome browser window from being forced to the foreground.
Note that this still makes a flashing chrome icon appear in the taskbar when any of the actions described above appear.
Still looking for a more solid solution, so any suggestions are still appreaciated.
To hopefully open new perspectives and discussion points, and ultimately a solution, I will provide some more detailed info of what my code is doing.
I have 5 chrome webdrivers, and each of these webdrivers contains 6 tabs (WindowHandles).
The idea is that a certain process has to be repeated continuously on each tab. So we loop over each webdriver, and within that webdriver over each tab and set this tab as the webdrivers current WindowHandle. This makes the chrome window visibly switch to the assigned tab.
After that switch has taken place, so basic selenium automation is performed on the content of the tab, after wich we repeat the whole process.
The actual issue seems to take place when a chrome webdriver switches to a new tab (WindowHandle), at this moment the chrome window containing the tab is pushed to the foreground and steals focus. Note that this does not always takes place, often it can switch tabs without any issues. So it is unclear wether there is another factor which would cause the window to steal focus.
EDIT 2
After doing the following:
I overloaded the selenium method which is used to switch between tabs (WindowHandles), and called SetWindoPos each time. Unfortunately this did not solve this issue either. I will try to look deeper into what might be causing this and will report back. – S. Van den Wyngaert
I went out for a few hours while running the tests, and came back to see that surprisingly I was still on my main Virtual Desktop (win10 feature). This means that the issue didn't occur during the time I was gone. I started working again, opened a chrome window and noticed that shortly after I did this, focus was stolen by one of the chrome driver's windows again.
What I conclude from this is that the issue only occurs when another chrome window (not opened by a chrome driver from code) is opened.
I will keep investigating and will report back with updates.
After investigating this behavior for a few more hours I noticed that when another chrome window is open, as long as this has focus, the issue doesn't take place. So to quickly summarize this:
The issue doesn't take place when:
No other chrome windows (not selenium driven) are opened or minimised
Another chrome window (not selenium driven) is opened and has focus
Another application running fullscreen mode has focus
The issue does take place when:
Another chrome window (not selenium driven) is opened and has no focus
Note that when I talk about another chrome window I specificly mean a chrome window that is not driven by selenium, so a regular chrome window opened by the user.

Easiest way would be to run a local selenium grid. Start your node(s) as a windows service. This way the test will run in the background, without being headless.

Another good option to scale your solution, and if your machine has the capacity to run docker, is to use
zalenium
It's a docker based, auto scaling, selenium-grid solution that works pretty quick out of the box.
You can watch your tests live via the management pages, watch a recording after the fact, pause/debug with live interaction via VNC. Also something to be said for not having the worry about changing browser versions.
Last time i used it, it had the odd bug and throws an end of stream error every now and then - but that was a good year ago.

Related

Selenium Webdriver click and send keys etc dont work on Google Chrome Version 78.0.3904.87

Google chrome automatically updated to Version 78.0.3904.87 yesterday and i now seem to have a problem with Selenium Webdriver.
When i run my tests in Visual Studio, the click events seem to occur but the expected result on screen does not happen e.g. clicking a link passes through the code (link.Click) successfully with no failure but the link does not get clicked on screen.
When i run the same test in Debug mode and Step Into each section of code it works fine, its as if selenium is running too fast for the browser (i have an implicit wait built in but its irrelevant, the element is there its just not clicking properly). I'm also finding things like sendKeys does not populate the field though it passes through the code successfully, again fine when stepping into it in Debug mode.
I've tried updating my Chromedriver to the version for V78 but this has made no difference
Similar problem here but my issue is not restricted to pdfs only
Chromedriver 78 cant find element in pdf
My team is running into the same issue. The issue seems to be the time between our WebDriverWaits and our Clicks -- the waits seem to be passing through instantaneously in cases they should not and then any Click element call (or action) hovers over what we want to click on but doesn't actually click.
I finally got to the bottom of this with our setup and it turned out to be the --enable-automation flag that seemed to be causing page flickering on page load.
After adding excludeSwitches: ['enable-automation'] to my config it has resolved the issue.

Selenium how to close Internet Explorer has stopped working

I am screen scraping a legacy website that uses activex controls. I am using selenium to accomplish this.
This website is buggy and whenever you signout of it, it crashes IE and you get the "Internet Explorer has stopped working" window. This happens no matter what, website needs to be recoded.
Is there a way in InternetExplorerDriver.Quit() to detect if this window is up and close it? The annoyance is that while the dialog is doing "searching for a solution..." it doesn't release selenium, so my code hangs up waiting.
Right now I am attempting to kill the process through the windows api, but that requires waiting for selenium to finish closing.
The solution Richard posted in the comments worked. I just told windows to stop checking for solutions.

.NET 4 WebBrowser hangs

I am using .NET 4 and trying to use the desktop authentication for the StackApps site via the web-browser control (WPF and/or WinForms) to develop a NNTP Bridge for accessing StackOverflow (https://stackapps.com/questions/4215/stackapp-nntp-bridge-for-accessing-stackexchange-forums-like-stackoverflow).
It seems that the login cannot be done, because the web browser hangs up, after the page from "StackExchange Login" is displayed.
I use the following URL:
https://stackexchange.com/oauth/dialog?client_id=1736&scope=no_expiry&redirect_uri=https://stackexchange.com/oauth/login_success
It works in the normal IE browser, but not in a WinForms or WPF window... Does anyone know what the problem is?
It is simple to repoduce:
Create a WinForms-Project
Add the "WebBrowser" control to the dialog
Double-Click on the Form1
Add the following code
private void Form1_Load(object sender, EventArgs e)
{
webBrowser1.Navigate("https://stackexchange.com/oauth/dialog?client_id=1736&scope=no_expiry&redirect_uri=https://stackexchange.com/oauth/login_success");
}
Start the application
Login by pressing the "login with Stack Exchange" account symbol
A new page gets loaded; it is displayed correctly, but you cannot enter your login name; the window hangs...
The same happens, if I use WPF-App and the WPF-WebBrowser-Control... it seems that it is stuck in an endless-loop in JavaScript...
Any hint on how to solve ths problem?
Or is it possible to debug the JavaScript in the WebBrowser-Control???
Fixing WebBrowser Control
I also had the issue of the WebBrowser control handing when trying to login.
Although requesting a token in IE (11) works, I found that IE itself also hangs when I put it in IE 7 emulation mode. This suggested to me that my previous attempt to make the WebBrowser control use a newer version had failed.
I found this article, Web Browser Control Specifying the IE Version, which suggests that for 32 bit applications in 64 bit mode, you need to set a different registry value.
So now, I've added two values in the registry:
HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_BROWSER_EMULATION
and
HKEY_CURRENT_USER\Software\Wow6432Node\Microsoft\Internet Explorer\MAIN\FeatureControl\FEATURE_BROWSER_EMULATION
In both of them, I added a DWORD value named 'MyExecutable.exe' (where MyExecutable is the actual name of my executable). The value for each of them is 9000 which will work for IE9 and above. Watch out when using RegEdit to test this, it will default to hexadecimal instead of decimal. Also, make sure it is a DWORD value, not any other type.
This seems to do the trick. I can now run the application, go through the login process, and eventually I am redirected to the url specified by me, which I can then capture using the OnNavigate event of the webbrowser control.
Remaining issue
It doesn't really work perfectly. The first time I was redirected to some OpenID page as well, but at least the form didn't hang. With subsequent attempts, apparently the login (which succeeded before) is remembered and I get the message "Navigation to the webpage was cancelled" with a link to refresh the page. When I click that link, I am immediately redirected to the redirect_url I specified when requesting the login form. At least that part works, and I get an actual access_token and an expiry time, so for now I'm happy.
Update: After some testing, it turned out that the previous login was remembered. That causes the request uri to direct to the redirect_uri immediately. I used the OnBeforeNavigate event to detect this, but it isn't fired in this case. I now linked the NavigateComplete2 event, and that one is triggered in this scenario.
Fix for .NET?
I think for .NET the solution should be the same: add the executable (and MyExecutabl.vshost.exe as well, for debugging purposes in Visual Studio) to the first key. If it's a 32 bit executable running on 62 bit Windows, you might also need to add it to the second key, although I'm sure if that rule applies to .NET as well. I don't do C# on a daily basis, and I'm trying to get it to work in Delphi first, but if I find time to test this in C# I will post the update here.
In the end, it would be nice if the actual issue would be solved, and the JavaScript would work in IE7 mode as well, but at least this seems to be a proper work-around.
Unfortunately, I'm not a JavaScript developer, so I only could try to explain you how to debug a JavaScript that executes in the WebBrowser control.
This approach is for the Microsoft Visual Studio, I don't know if Delphi can provide similar functionality.
Enable Script Debugging (both Internet Explorer and Other) in Internet Explorer settings.
Disable Friendly HTTP messages in Internet Explorer settings.
Enable Display a notification about every script error in Internet Explorer settings.
From Visual Studio, start your WebBrowser hosting application without debugging (i. e., Ctrl + F5).
In Visual Studio, go to Debug → Attach to Process… and select your application in the list.
Hit the Select… button to the right of Attach to: field and choose Script code.
Hit the Attach button. Visual Studio starts the script debugger.
In your application, navigate to the deadlocking page by pressing the Login with Stack Exchange account symbol.
Go to Visual Studio and press the Pause button on the debugging toolbar.
Now you can look into your script code and investigate the code itself, the call stack, the variable values and so on. You can set breakpoints too. Perhaps you can then find the place where the script hangs. As I said before, I'm not a web developer and cannot help you with this…
Update:
I guess I can propose you a working solution.
My investigation shows that the WebBrowser hangs when it renders the content in the IE7 mode (what is the default mode even if you have IE10 in your system). You should force it to switch into IE9 mode. In IE9 mode, the page renders well and does not cause the script to stuck in an endless loop.
You can switch your WebBrowser to the IE9 mode using one of the following methods:
Define the global browser emulation mode for your application in Windows registry.
For 32 bit OS, go to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Internet Explorer\MAIN\FeatureControl\FEATURE_BROWSER_EMULATION.
For 64 bit OS, use HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\Internet Explorer\MAIN\FeatureControl\FEATURE_BROWSER_EMULATION.
In this node, create a new DWORD parameter called YourApplicationExeName.exe with a value 9000 (0x2710). You can create another entries for your *.vshost.exe executables, if you want this to work in Visual Studio debug mode.
Manipulate the source for the WebBrowser to switch it in the IE9 mode.
This will be more complicated. You need to alter the <head> tag of the html document adding a new <meta> tag preferably as a first element: <meta http-equiv="X-UA-Compatible" content="IE=9"/>.
This will cause the WebBrowser to switch its mode on document rendering. Since you can't change the compatibility mode after the document is rendered, you could use a proxy as a source for your WebBrowser, so this proxy will add the header.
I`ve successfully tested this with the 1st approach, and the second one should work too.
Hope that helps!
Project + Properties, Debug tab, tick the "Enable native code debugging" option. Ensure that you've got the Microsoft Symbol server enabled (Tools + Options, Debugging, Symbols), I know you do :)
You can now use Debug + Break All and see what's going on inside the browser. Do so repeatedly to get the lay of the land. You'll see that Javascript is executing (jscript8.dll) and constantly forcing the layout engine to recalculate layout, never getting the job done.
Browsers in general are vulnerable to Javascript that's stuck in an endless loop. A regular browser tends to put up a dialog after several dozen seconds to allow the user to abort the JS, but WebBrowser doesn't have that feature. Nor does it have the dev tools so debugging the JS isn't going to be a great joy either. This is going to be difficult to get fixed.
You might consider using the OAuth 2.0 api instead. Notes on usage are on this web page. Exactly how to integrate that with WebBrowser is a bit murky to me, I don't have a key to test this. Find help for this at the Stackapps site. You are probably not the first SE api user that ran into this problem.

Disable Libraries window popup when explorer restarted

I have written an application that changes some of the registry settings and then restarts explorer to save them. Everytime explorer is restarted an annoying "Libraries" popup comes up.
I want to disable it somehow though C# code, (probably by tweaking some other registry keys). However, I do NOT want to disable libraries feature, just a popup that comes up when explorer is restarted.
All articles on the web show how to disable libraries completely.
The most likely reason for this to happen is that the Libraries window was open the last time Explorer was shut down naturally, i.e., the last time the user logged off. If you close the window and log off, then log in again, your software will probably work correctly.
Caveat: killing and restarting the explorer process is inherently risky. Resolving this particular issue does not mean that other similar issues won't show up in future.
When restarting explorer I just had to add some randow parameter as it seems to default to Libraries window. so calling "explorer.exe xxx" would cause explorer to ignore default parameters.

How do i close all open windows from my C# program?

Hi i have a windows application where i show a webbrowser, i have a button that removes the browser (its a preview) and goes to another "view" in my application (another tab). My problem is that my users are getting advanced, they build HTML with links (and its ok) but the links may spawn new browser windows (IExplorer), and my button needs to close these windows, but how?
I have made some code to traverse all eht windows that ends with "Windows Internet Explorer", ahd it all seems to work - but how do i close them? I am trying to do it like this:
SendMessage((int)hWnd, WM_SYSCOMMAND, SC_CLOSE, 0);
It seems to work, but the browser pops up a dialog asking me if i want to close the tab of all the tabs...how to work around/solve this?
Cheers,
walking over all top level iE windows and closing them is a bad idea, unless you are guaranteed users can't launch ie and browse the Internet on their own. Otherwise, you might actually lose user data (say an email or a blog post the user has been working on in the last half an hour)
You can't easily work around that dialog without modifying the per-user IE settings. Your other option is to look for that dialog and click the yes button, but that would be fragile and is not guaranteed to continue working if the user upgrades to IE9.
You could potentially prevent opening links in new window by listening to BeforeNavigate event and allowing only navigations that are guaranteed to happen in your control. However, there are scenarios where IE might still decide to open new window.

Categories