c# headless browser with javascript support for crawler - c#

Could anyone suggest headless browser for .NET that supports cookies and authomatically javascript execution?

Selenium+HtmlUnitDriver/GhostDriver is exactly what you are looking for. Oversimplified, Selenium is library for using variety of browsers for automation purposes - testing, scraping, task automation.
There are different WebDriver classes with which you can operate an actual browser. HtmlUnitDriver is a headless one. GhostDriver is a WebDriver for PhantomJS, so you can write C# while actually PhantomJS will do the heavy lifting.
Code snippet from Selenium docs for Firefox, but code with GhostDriver (PhantomJS) or HtmlUnitDriver is almost identical.
using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium.Support.UI;
class GoogleSuggest
{
static void Main(string[] args)
{
// driver initialization varies across different drivers
// but they all support parameter-less constructors
IWebDriver driver = new FirefoxDriver();
driver.Navigate().GoToUrl("http://www.google.com/");
IWebElement query = driver.FindElement(By.Name("q"));
query.SendKeys("Cheese");
query.Submit();
WebDriverWait wait = new WebDriverWait(driver, TimeSpan.FromSeconds(10));
wait.Until((d) => { return d.Title.ToLower().StartsWith("cheese"); });
System.Console.WriteLine("Page title is: " + driver.Title);
driver.Quit();
}
}
If you run this on Windows machine you can use actual Firefox/Chrome driver because it will open an actual browser window which will operate as programmed in your C#. HtmlUnitDriver is the most lightweight and fast.
I have successfully ran Selenium for C# (FirefoxDriver) on Linux using Mono. I suppose HtmlUnitDriver will also work as fine as the others, so if you require speed - I suggest you go for Mono (you can develop, test and compile with Visual Studio on Windows, no problem) + Selenium HtmlUnitDriver running on Linux host without desktop.

I am not aware of a .NET based headless browser but there is always PhantomJS which is C/C++ and it works fairly well for assisting in unit testing of JS with QUnit.
There is also another relevant question here which might help you - Headless browser for C# (.NET)?

Related

Is there a way to activate IE mode in Edge Options?

Hello,
I want to achieve this by having an option inside the EdgeDriver but I cant seem to find it anywhere on the map?
I am trying to open a page in IE mode inside Edge with Selenium and EdgeDriver.
Is there a way to achieve this great thing? [pun intented]
I can see 2 questions in this thread.
Is there a way to activate IE mode in Edge Options?
There is no way to activate IE mode bypassing the Edge options parameter in the Selenium Edge driver.
I am trying to open a page in IE mode inside Edge with Selenium and EdgeDriver. Is there a way to achieve this great thing?
Yes, it is possible to automate the IE mode in the new MS Edge browser using the Selenium web driver.
The new Microsoft Edge allows you to run IE11 validation for legacy sites in addition to your modern experiences. To run your IE11 tests in Microsoft Edge, download the IEDriverServer from Selenium. Then you must pass in a capability to put Microsoft Edge into IE Mode and then run your tests.
Because this capability puts the whole browser into IE11 Mode, you cannot simultaneously test content that should render in the modern Chromium engine, but you should be able to run all of your IE11 tests and validate the rendering in Microsoft Edge. Note that this code requires an update to IEDriverServer which should be included in the next release of Selenium.
After you download the new IEDriverServer from SeleniumHQ and follow the directions for the “Required Configuration” as documented here, you can run the following code to launch the new Microsoft Edge in IE11 mode and run some tests:
static void Main(string[] args)
{
var dir = "{FULL_PATH_TO_IEDRIVERSERVER}";
var driver = "IEDriverServer.exe";
if (!Directory.Exists(dir) || !File.Exists(Path.Combine(dir, driver)))
{
Console.WriteLine("Failed to find {0} in {1} folder.", dir, driver);
return;
}
var ieService = InternetExplorerDriverService.CreateDefaultService(dir, driver);
var ieOptions = new InternetExplorerOptions{};
ieOptions.AddAdditionalCapability("ie.edgechromium", true);
ieOptions.AddAdditionalCapability("ie.edgepath", #"\\msedge.exe");
var webdriver = new InternetExplorerDriver(ieService, ieOptions, TimeSpan.FromSeconds(30));
webdriver.Url = "http://www.example.com";
}
Output:
Notes:
Make sure you are using the latest version of the IE driver server.
I suggest making a test with the latest version of the Stable Edge browser.
Try to pass the full path of the Edge browser in the 'ie.edgepath' capability. For example:
ieOptions.AddAdditionalCapability("ie.edgepath", #"C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe");
Make sure you close all the already opened instances and tabs of the Edge browser before running the code. Otherwise, it will generate an error.
References:
Scroll to the Automating Internet Explorer mode point in this link.
kypflug/webdriver-edge-ie-mode.cs
Below code (which is in VB.NET, but you can easily modify it to C#) will start Chromium Edge in IE Mode
Dim ieService = InternetExplorerDriverService.CreateDefaultService("DIRECTORY_PATH_HAVING_IEDriverServer.exe", "IEDriverServer.exe")
Dim ieOptions = New InternetExplorerOptions
ieOptions.IgnoreZoomLevel = True
ieOptions.AddAdditionalCapability("ie.edgechromium", True)
ieOptions.AddAdditionalCapability("ie.edgepath", "C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe")
Dim driver = New InternetExplorerDriver(ieService, ieOptions, TimeSpan.FromSeconds(60))
driver.Navigate().GoToUrl("https://example.com")
You can download IEDriverServer from https://www.selenium.dev/downloads/

Selenium C# integration With Neoload performance testing tool

Could anyone help me to elaborate procedure for integrating selenium C# script with neoload tool.i wanted to integrate selenium C# and Appium C# coding with neoload .So I went though some steps in documentation and unable to run the example code and didn't get the point how we can integrated with selenium/appium C# script with neoload for chrome or any browser and perform load testing for web or mobile application with same tool .Could you please have a look on shared screen shot and help me to share valuable document with any simple example for understanding the concept/getting some confidence.Thanks for giving valuable time
I modified the code found here for the chrome driver: https://www.neotys.com/documents/doc/neoload/latest/en/html/#24364.htm
DesiredCapabilities cap = new DesiredCapabilities();
NLWebDriverFactory.AddProxyCapabilitiesIfNecessary(cap);
object proxyCapability = cap.GetCapability(CapabilityType.Proxy);
ChromeProfile profile = new ChromeProfile();
if (proxyCapability != null) {
profile.SetProxyPreferences(proxyCapability as Proxy);
}
ChromeDriver webDriver = new ChromeDriver(profile);
Your problem is that you need to provide the ChromeDriver with a profile which has the DesiredCapabilities whereas you try to apply them directly to the driver.

Selenium: switch tab in RemoteWebDriver with C#

Currently, I'm trying to run my Selenium tests on Safari using Selenium Grid and RemoteWebDriver. This is my setup:
Mac OS Sierra 10.12.6 as a machine for running tests.
Selenium server 3.5.3.
Safari 11.
C# Selenium WebDriver and Selenium Support (latest version).
I'm using port forwarding on my host OS (Windows 10) to forward requests to Mac, running on my Virtual Machine. On my Mac I have Selenium Grid hub, which I run like this:
java -jar selenium-server-standalone-3.5.3.jar -role hub -port 4723
Also, there is a node:
java -jar selenium-server-standalone-3.5.3.jar -role node -hub http://10.0.2.15:4723/grid/register
In code, I start my driver like this:
SafariOprions options = new SafariOptions();
IWebDriver driver = new RemoteWebDriver(new Uri(hubURL), options.ToCapabilities());
My tests are running fine with current setup. But when it comes to clicking a link with attribute target='_blank' I'm starting to face some troubles. For other drivers, which I run locally, I can switch tab without any trouble: I'm getting driver.WindowHandles before I click a link, then I click a link and again retrieve Window Handles to compare with previous handles. After that I use driver.SwitchTo().Window(newHandle) and everything is ok.
But when it comes to RemoteWebDriver (or SafariDriver from Apple, I cannot say more precisely) I'm always getting only one Window Handle, even if the new tab is opened and i can see it.
I'm trying to avoid switching tab with "Command + T" as one of solutions suggested, because my tests are meant to be run on all browsers (Chrome, Firefox, Opera, Edge, Safari) and this wont work.
UPDATE: I've tried running Chrome and other browsers in Selenium Grid via RemoteWebDriver and I can say that this is not an issue of RemoteWebDriver. Next, I've installed Visual Studio for Mac and rewrite several things to run my tests without Selenium Grid, just using this code:
//if memory serves, just like this
SafariOptions options = new SafariOptions();
SafariDriver driver = new SafariDriver(options);
But, unfortunately, this didn't help. Driver navigated me to the page, clicked the link and opened a new tab, but without any switch. When I checked for driver.WindowHandles I've only got one, although there was two visible tabs. Neither driver.SwitchTo().ActiveElement nor driver.SwitchTo().Frame(hardcoded_frame_name) doesn't seem to work. Pretty long waits (for 60 seconds after opening the link and another one after that) aren't working too. Now I think that this is really a bug and I will try to report this to Apple as soon as I can.
But for now, maybe someone has a fancy workaround for this?
As we discussed in the comments, it seems to be a timing issue. So we will induce
WebDriverWait to sync up with the trailing Browser instance. I am providing a code block as a solution through Selenium-Java, consider implementing it in C# and update me if it works for you.
driver.get("http://www.google.com");
System.out.println("Page Title is : "+driver.getTitle());
String parent_window = driver.getWindowHandle();
((JavascriptExecutor) driver).executeScript("window.open('http://facebook.com/');");
WebDriverWait wait = new WebDriverWait(driver,3);
wait.until(ExpectedConditions.numberOfWindowsToBe(2));
Set<String> allWindows_1 = driver.getWindowHandles();
ArrayList<String> tabs = new ArrayList<>(allWindows_1);
driver.switchTo().window(tabs.get(1));
wait.until(ExpectedConditions.titleContains("Facebook"));
System.out.println("First Child Handle : "+driver.getTitle());
I could help you with the Java version:
After the actions done, do this below.
//Store the parent window
String parentWindow = driver.getWindowHandle();
//Open a new Windows(Mailtrap)
String a = "window.open('https://mailtrap.io/signin','_blank');";
((JavascriptExecutor)driver).executeScript(a);
//This Thread.sleep is useful with Safari. Do not remove.
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
//Take control over new browser
for(String handle: driver.getWindowHandles()){
driver.switchTo().window(handle);
}

Seleniums Gecko driver crashes when going to new web pages

I am using the new Gecko Driver to test in Firefox. When I change the url it often fails. I have a test that goes to 10 different pages to perform a very basic health check. The pages always load when using Firefox manually (not Gecko/Selenium), but when using the Gecko driver it fails on the GoToUrl().
It's not consistent which of the ten pages it fails on, but it is always consistently failing on one of the ten. When it fails Firefox closes and an error message pops up asking if I want to submit the details to Mozilla.
Am I doing something wrong, or is there some restriction with the Gecko driver that I am unaware of? The Chrome and internet explorer drivers handle the exact same test just fine.
var service = FirefoxDriverService.CreateDefaultService(DriverPathOnSystem);
var driver = new FirefoxDriver(service);
driver.Navigate().GoToUrl(url);
EDIT
Selenium.Support v3.4.0 (nuget)
Selenium.WebDriver v3.4.0 (nuget)
Firefox 51.0.1 (32-bit) but also using the most recent version on the
build machine and it has the same failure there
Gecko 0.16.0
Upon further research I discovered I was actually using WebDriver v3.2.0.
I updated WebDriver to v3.4.0,
updated FireFox to 53.0.0,
updated GeckoDriver to 0.16.1
cleaned the solution and rebuilt and it worked.

How to take screenshot using Selenium RC using c# in IE8?

I am using Selenium RC using c# and need to take screenshot for the test case failed, I have to perform these test on IE 8.
I got lot of tutorials for Firefox but didn't find any for the IE8.
Thanks in advance.
You can use the Selenium Web Driver to take a screenshot of IE pretty easily. The code would look something like this:
IWebDriver driver = new InternetExplorerDriver();
driver.Navigate().GoToUrl("http://www.google.com");
TakeScreenshot(driver, #"C:\screenshot.png");
You can download the Web Driver here: https://code.google.com/p/selenium/downloads/list
Add it to your project and you are off and running.

Categories