We Keep Coding

Selenium can't handle multiple ChromiumWebBrowser instances in C#

I have two instances of the ChromiumWebBrowser in my WinForms project (Visual Studio 2012). My goal is to have the second browser instance "copy" the behavior of the user input in the first browser instance. I can successfully retrieve the input from the first browser, and I managed to hook up Selenium in the project as well.
However, I'm having one issue. Whenever Selenium sends its commands, the first browser is the one that responds to them. For the life of me, I can't seem to figure out how to make the second browser respond. Whenever I completely remove the first browser, the second one starts responding correctly, but adding the first browser again will make only have the first browser use the Selenium commands. I even tried to switch out the moments the browsers are added to the form, but to no avail: whenever there are two available, the wrong one is responsive.
Relevant code:
public BrowserManager(Controller controller, string startingUrl)
{
_controller = controller;
var settings = new CefSettings { RemoteDebuggingPort = 9515 };
Cef.Initialize(settings);
// Input browser
inputBrowser = new ChromiumWebBrowser(startingUrl);
var obj = new XPathHelper(this);
inputBrowser.RegisterJsObject("bound", obj); //Standard object registration
inputBrowser.FrameLoadEnd += obj.OnFrameLoadEnd;
// Output browser
var browserSettings = new BrowserSettings();
var requestContextSettings = new RequestContextSettings { CachePath = "" };
var requestContext = new RequestContext(requestContextSettings);
outputBrowser = new ChromiumWebBrowser(startingUrl);
outputBrowser.RequestContext = requestContext;
outputBrowser.AddressChanged += InitializeOutputBrowser;
outputBrowser.Enabled = false;
outputBrowser.Name = "outputBrowser";
}
The selenium part:
public class SeleniumHelper
{
public SeleniumHelper()
{
DoWorkAsync();
}
private Task DoWorkAsync()
{
Task.Run(() =>
{
string chromeDriverDir = #"ActionRecorder\bin\x64\Debug\Drivers";
var chromeDriverService = ChromeDriverService.CreateDefaultService(chromeDriverDir);
chromeDriverService.HideCommandPromptWindow = true;
ChromeOptions options = new ChromeOptions();
options.BinaryLocation = #"ActionRecorder\bin\x64\Debug\ActionRecorder.exe";
options.DebuggerAddress = "127.0.0.1:9515";
options.AddArguments("--enable-logging");
using (IWebDriver driver = new OpenQA.Selenium.Chrome.ChromeDriver(chromeDriverService, options))
{
driver.Navigate().GoToUrl("http://www.google.com");
var query = driver.FindElement(By.Name("q"));
query.SendKeys("A google search test");
query.Submit();
}
});
return null;
}
}
And finally, a screenshot for some visualization:
Some help with the issue would be very much appreciated. If i missed some crucial info, feel free to ask for it. Thanks in advance!
Greetz,
Tybs

The behavior is correct. You have one debug address and you can only have one debug address for CEF. Which means when you use Selenium it is only seeing one browser.
By default Selenium will send an command to current active Tab or Window. Now in your case you have multiple Chrome view embedded, but they are technically Chrome Tab/Windows which you have placed on the same form.
So if you are in luck below code in should be able to move you to the Window you are interested in
driver.SwitchTo().Window(driver.WindowHandles.Last());
See if it works. If it doesn't then your only other workaround would be to change the order of Adding ChromiumWebBrowser and that should reverse the window it works on.
Below are some important threads that you should read from top to bottom. Very relevant to your issue/request
https://code.google.com/archive/p/chromiumembedded/issues/421
https://github.com/cefsharp/CefSharp/issues/1076

Element obscured when Selenium testing Edge webdriver for Azure Active Directory login Sign in to your account page

I'm using Selenium to make an integration test. Part of the test is the logon process that Microsoft Azure Active Directory uses
The test passes when using InternetExplorerWebDriver, but fails with EdgeWebDriver with the error:
Element is obscured
The relevant part of the code:
var wait = new WebDriverWait(webDriver, TimeSpan.FromSeconds(10));
webDriver.Url = Settings.UrlHome();
var signInElement = webDriver.FindElement(By.Id("SignInLink"));
signInElement.Click();
wait.Until(ExpectedConditions.ElementToBeClickable(By.Id("my_login_name")));
var loginLogoMicrosoft = webDriver.FindElement(By.Id("my_login_name"));
loginLogoMicrosoft.Click();
The error occurs when executing the last Click(). I've tried several other elements, but none of them are working. Also 'inspect element' to determine which elements receives the click get this error when implementing in code.
How can I make the Microsoft Azure Active Directory logon process part of my Selenium test?
A same kind of issue is found here.

I use the following code for filling out the Azure AD login form. The caveat is that when you fill the username field the "Sign In" button is disabled until some ajax operation is done in the background. The trick is to wait until there's a Sign In button on the page without this class.
private void SubmitLoginForm()
{
var useAnotherAccount = Driver.FindElements(By.ClassName("use_another_account")).FirstOrDefault();
if (useAnotherAccount != null)
{
useAnotherAccount.Click();
}
var loginInput = Driver.FindElements(By.Id(Site.Login.UserNameInput)).FirstOrDefault();
if (loginInput != null)
{
loginInput.SendKeys(TestingData.UserName);
loginInput.SendKeys(Keys.Tab);
}
var passwordInput = Driver.FindElements(By.Id(Site.Login.PasswordInput)).FirstOrDefault();
if (passwordInput != null)
{
passwordInput.Clear();
passwordInput.SendKeys(TestingData.PassWord);
passwordInput.SendKeys(Keys.Enter);
}
var wait = new WebDriverWait(Driver, TimeSpan.FromSeconds(5));
wait.Until(f => f.FindElement(By.CssSelector("#cred_sign_in_button:not(.disabled_button")));
var loginButton = Driver.FindElements(By.Id(Site.Login.SigninButton)).FirstOrDefault();
if (loginButton != null)
{
loginButton.Click();
return;
}
throw new InvalidOperationException("Could not click the login button");
}

Get the Control on a page by passing URL? Also get the current page URL?

My Requirement: I want to know which page I am currently in so that if any test fails I want to pass the current page's URL to a method and get the home button link. Ultimately navigating to the home link in case of any exception.
Is there a way to achieve it ?

The URL should be in the address bar of the browser, just read it out of there.
One way of reading out the value is to record an assertion on the value in the address bar, then copy the part of the code in the recorded assertion method that accesses the value.
Another way is to use the cross-hairs tool to select the address area, then (click the double-chevron icon to open the left hand pane and) add the UI control for the selected area. Then access the value.

This will return the top Browser:
BrowserWindow bw = null;
try
{
Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.AllThreads;
var browser = new BrowserWindow() /*{ TechnologyName = "MSAA" }*/;
PropertyExpressionCollection f = new PropertyExpressionCollection();
f.Add("TechnologyName", "MSAA");
f.Add("ClassName", "IEFrame");
f.Add("ControlType", "Window");
browser.SearchProperties.AddRange(f);
UITestControlCollection coll = browser.FindMatchingControls();
// get top of browser stack
foreach (BrowserWindow win in coll)
{
bw = win;
break;
}
String url = bw.Uri.ToString(); //this is the value you want to save
}
catch (Exception e)
{
throw new Exception("Exception getting active (top) browser: - ------" + e.Message);
}
finally
{
Playback.PlaybackSettings.WaitForReadyLevel = WaitForReadyLevel.UIThreadOnly;
}

using C# and UI Automation to grab contents of unknown control-type

In the image below there is an area, which has an unknown (custom) class. That's not a Grid or a Table.
I need to be able:
to select Rows in this area
to grab a Value from each cell
The problem is since that's not a common type element - I have no idea how to google this problem or solve it myself. So far the code is following:
Process[] proc = Process.GetProcessesByName("programname");
AutomationElement window = AutomationElement.FromHandle(proc [0].MainWindowHandle);
PropertyCondition xEllist2 = new PropertyCondition(AutomationElement.ClassNameProperty, "CustomListClass", PropertyConditionFlags.IgnoreCase);
AutomationElement targetElement = window.FindFirst(TreeScope.Children, xEllist2);
I've already tried to threat this Area as a textbox, as a grid, as a combobox, but nothing solved my problem so far. Does anybody have any advice how to grab data from this area and iterate through rows?
EDIT: sorry I've made a wrong assumption. Actually, the header(column 1, column 2, column 3) and the "lower half" of this area are different control-types!!
Thanks to Wininspector I was able to dig more information regarding these control types:
The header has following properties: HeaderControl 0x056407DC (90441692) Atom: #43288 0xFFFFFFFF (-1)
and the lower half has these: ListControl 0x056408A4 (90441892) Atom: #43288 0x02A6FDA0 (44498336)
The code that I've showed earlier - retrieved the "List" element only, so here is the update:
Process[] proc = Process.GetProcessesByName("programname");
AutomationElement window = AutomationElement.FromHandle(proc [0].MainWindowHandle);
//getting the header
PropertyCondition xEllist3 = new PropertyCondition(AutomationElement.ClassNameProperty, "CustomHeaderClass", PropertyConditionFlags.IgnoreCase);
AutomationElement headerEl = XElAE.FindFirst(TreeScope.Children, xEllist3);
//getting the list
PropertyCondition xEllist2 = new PropertyCondition(AutomationElement.ClassNameProperty, "CustomListClass", PropertyConditionFlags.IgnoreCase);
AutomationElement targetElement = window.FindFirst(TreeScope.Children, xEllist2);
After giving it a further thought I've tried to get all column names:
AutomationElementCollection headerLines = headerEl.FindAll(TreeScope.Children, new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.HeaderItem));
string headertest = headerLines[0].GetCurrentPropertyValue(AutomationElement.NameProperty) as string;
textBox2.AppendText("Header 1: " + headertest + Environment.NewLine);
Unfortunately in debug mode element count in "headerLines" is 0 so the program throws an error.
Edit 2: Thanks to the answer below - I've installed Unmanaged UI Automation, which holds better possibilities than the default UIA. http://uiacomwrapper.codeplex.com/
How do you use the legacy pattern to grab data from unknown control-type?
if((bool)datagrid.GetCurrentPropertyValue(AutomationElementIdentifiers.IsLegacyIAccessiblePatternAvailableProperty))
{
var pattern = ((LegacyIAccessiblePattern)datagrid.GetCurrentPattern(LegacyIAccessiblePattern.Pattern));
var state = pattern.Current.State;
}
Edit 3. IUIAutoamtion approach (non-working as of now)
_automation = new CUIAutomation();
cacheRequest = _automation.CreateCacheRequest();
cacheRequest.AddPattern(UiaConstants.UIA_LegacyIAccessiblePatternId);
cacheRequest.AddProperty(UiaConstants.UIA_LegacyIAccessibleNamePropertyId);
cacheRequest.TreeFilter = _automation.ContentViewCondition;
trueCondition = _automation.CreateTrueCondition();
Process[] ps = Process.GetProcessesByName("program");
IntPtr hwnd = ps[0].MainWindowHandle;
IUIAutomationElement elementMailAppWindow = _automation.ElementFromHandle(hwnd);
List<IntPtr> ls = new List<IntPtr>();
ls = GetChildWindows(hwnd);
foreach (var child in ls)
{
IUIAutomationElement iuiae = _automation.ElementFromHandle(child);
if (iuiae.CurrentClassName == "CustomListClass")
{
var outerArayOfStuff = iuiae.FindAllBuildCache(interop.UIAutomationCore.TreeScope.TreeScope_Children, trueCondition, cacheRequest.Clone());
var outerArayOfStuff2 = iuiae.FindAll(interop.UIAutomationCore.TreeScope.TreeScope_Children, trueCondition);
var countOuter = outerArayOfStuff.Length;
var countOuter2 = outerArayOfStuff2.Length;
var uiAutomationElement = outerArayOfStuff.GetElement(0); // error
var uiAutomationElement2 = outerArayOfStuff2.GetElement(0); // error
//...
//I've erased what's followed next because the code isn't working even now..
}
}
The code was implemented thanks to this issue:
Read cell Items from data grid in SysListView32 of another application using C#
As the result:
countOuter and countOuter2 lengths = 0
impossible to select elements (rows from list)
impossible to get ANY value
nothing is working

You might want to try using the core UI automation classes. It requires that you import the dll to use it in C#. Add this to your pre-build event (or do it just once, etc):
"%PROGRAMFILES%\Microsoft SDKs\Windows\v7.0A\bin\tlbimp.exe" %windir%\system32\UIAutomationCore.dll /out:..\interop.UIAutomationCore.dll"
You can then use the IUIAutomationLegacyIAccessiblePattern.
Get the constants that you need for the calls from:
C:\Program Files\Microsoft SDKs\Windows\v7.1\Include\UIAutomationClient.h
I am able to read Infragistics Ultragrids this way.
If that is too painful, try using MSAA. I used this project as a starting point with MSAA before converting to all UIA Core: MSSA Sample Code
----- Edited on 6/25/12 ------
I would definitely say that finding the proper 'identifiers' is the most painful part of using the MS UIAutomation stuff. What has helped me very much is to create a simple form application that I can use as 'location recorder'. Essentially, all you need are two things:
a way to hold focus even when you are off of your form's window Holding focus
a call to ElementFromPoint() using the x,y coordinates of where the mouse is. There is an implementation of this in the CUIAutomation class.
I use the CTRL button to tell my app to grab the mouse coordinates (System.Windows.Forms.Cursor.Position). I then get the element from the point and recursively get the element's parent until I reach the the desktop.
var desktop = auto.GetRootElement();
var walker = GetRawTreeWalker();
while (true)
{
element = walker.GetParentElement(element);
if (auto.CompareElements(desktop, element) == 1){ break;}
}
----- edit on 6/26/12 -----
Once you can recursively find automation identifiers and/or names, you can rather easily modify the code here: http://blog.functionalfun.net/2009/06/introduction-to-ui-automation-with.html to be used with the Core UI Automation classes. This will allow you to build up a string as you recurse which can be used to identify a control nested in an application with an XPath style syntax.

Datagrid contents with UI Automation and .net

I'm having some trouble reading the contents of a Datagrid in an external application using UI Automation and could use some pointers. Here's what I have so far:
int id = System.Diagnostics.Process.GetProcessesByName("Book")[0].Id;
AutomationElement desktop = AutomationElement.RootElement;
AutomationElement bw = desktop.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.ProcessIdProperty, id));
AutomationElement datagrid = bw.FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.AutomationIdProperty, "lv"));
AutomationElementCollection lines = datagrid.FindAll(TreeScope.Children, new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.DataItem));
AutomationElementCollection items = lines[1].FindAll(TreeScope.Children, new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Custom));
GridItemPattern pattern = items[1].GetCurrentPattern(GridItemPattern.Pattern) as GridItemPattern;
TableItemPattern tablePattern = items[1].GetCurrentPattern(TableItemPattern.Pattern) as TableItemPattern;
This works in as much as I can access the column ids and row ids from the GridItemPattern and TableItemPattern but how do I access the value that is in that specific cell? Is it even possible?
Thanks.

I think you need to use ValuePattern for it. Just like that:
ValuePattern pattern = items[0].GetCurrentPattern(ValuePattern.Pattern) as ValuePattern;
string value = pattern.Current.Value;

I finally figured this out, it requires the use of CacheRequest to request the Name property on the AutomationElement. Here's the final code:
var cacheRequest = new CacheRequest
{
AutomationElementMode = AutomationElementMode.None,
TreeFilter = Automation.RawViewCondition
};
cacheRequest.Add(AutomationElement.NameProperty);
cacheRequest.Add(AutomationElement.AutomationIdProperty);
cacheRequest.Push();
var targetText = loginLinesDetails[i].FindFirst(TreeScope.Children, new PropertyCondition(AutomationElement.ClassNameProperty, "TextBlock"));
cacheRequest.Pop();
var myString = targetText.Cached.Name;

I am not familiar with the AutomationElement classes but I have used AutoIT to automate some simple windows stuff in the past (find a dialog, click a button, etc) and it was cake. You might consider it. The download contains a .dll you can reference from a .Net solution.
I'm not sure if the external app is a WinForm grid or not but here is an ASP.Net grid example: http://www.autoitscript.com/forum/topic/13709-how-to-get-the-contents-of-datagrid-control/
Then again, if you are scraping the info from the web I would recommend WatiN or Selenium

You can try using RawViewWalker on a element to get raw values (On controlview you might not be able to get few properties)