Firefox not able to assume untrusted certificate issuer - c#

I am a C# developer and i am using Selenium to test web applications. With the newest updates, firefox is not able to assume untrusted certificate issuer. Firefox does not work regardless, it does not matter what i am trying to do. I am testing on a seperated server via RemoteWebriver. Chrome and internet explorer work fine.
Here is a little snippet of my actual code
[...]
var uri = new Uri($"http://{ServerProperties.Address}:4444/wd/hub");
[...]
case "InternetExplorer":
return new RemoteWebDriver(uri, DesiredCapabilities.InternetExplorer());
case "Firefox":
FirefoxProfile profile = new FirefoxProfile();
profile.AcceptUntrustedCertificates = true;
profile.AssumeUntrustedCertificateIssuer = true;
DesiredCapabilities dc = DesiredCapabilities.Firefox();
dc.SetCapability(FirefoxDriver.ProfileCapabilityName, profile);
return new RemoteWebDriver(uri, dc);
case "Chrome":
return new RemoteWebDriver(uri, DesiredCapabilities.Chrome());
[...]
Thanks for your help

Related

C# Selenium - Failed to Start Tor

I'm trying to launch Tor browser through Selenium in C# using the following code:
using OpenQA.Selenium.Firefox;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace AutomatorApp
{
public class BrowserAutomator
{
public void Automate()
{
String torPath = "D:\\Tor Browser\\Browser\\firefox.exe";
String profilePath = "D:\\Tor Browser\\Browser\\TorBrowser\\Data\\Browser\\profile.default\\";
FirefoxProfile profile = new FirefoxProfile(profilePath);
profile.SetPreference("network.proxy.type", 1);
profile.SetPreference("network.proxy.socks", "127.0.0.1");
profile.SetPreference("network.proxy.socks_port", 9153);
profile.SetPreference("network.proxy.socks_remote_dns", false);
FirefoxDriverService firefoxDriverService = FirefoxDriverService.CreateDefaultService("D:\\geckodriver-v0.26.0-win64", "geckodriver.exe");
firefoxDriverService.FirefoxBinaryPath = torPath;
var firefoxOptions = new FirefoxOptions
{
Profile = profile,
LogLevel = FirefoxDriverLogLevel.Trace
};
FirefoxDriver driver = new FirefoxDriver(firefoxDriverService, firefoxOptions);
}
}
}
However, this shows the error 'Tor Failed to Start' and the exception simply contains 'Permission denied'. I tried launching the app in administrator mode and ensuring that the folder is accessible by all users but this did not solve the issue.
Interestingly, when I try to launch a Firefox browser using the same setup it works well.
Any help is very much appreciated.
Update - Solved: After updating to the latest Tor version (9.5.1) The final working code:
FirefoxProfile profile = new FirefoxProfile(profilePath);
profile.SetPreference("network.proxy.type", 1);
profile.SetPreference("network.proxy.socks", "127.0.0.1");
profile.SetPreference("network.proxy.socks_port", 9153);
profile.SetPreference("network.proxy.socks_remote_dns", false);
FirefoxDriverService firefoxDriverService = FirefoxDriverService.CreateDefaultService(geckoDriverDirectory);
firefoxDriverService.FirefoxBinaryPath = torPath;
firefoxDriverService.BrowserCommunicationPort = 2828;
var firefoxOptions = new FirefoxOptions
{
Profile = null,
LogLevel = FirefoxDriverLogLevel.Trace
};
firefoxOptions.AddArguments("-profile", profilePath);
FirefoxDriver driver = new FirefoxDriver(firefoxDriverService, firefoxOptions);
driver.Navigate().GoToUrl("https://www.google.com");
Important notes:
The following TOR configs need to be changed in about:config :
marionette.enabled: true
marionette.port: set to an unused port, and set this value to firefoxDriverService.BrowserCommunicationPort in your code. This was set to 2828 in my example.
As far as I remember from my attempts a few years ago, TOR with WebDriver didn't work when you set that "Profile" option. Normal Firefox works, but TOR simply doesn't.
If you get a timeout error after this, make sure marionette is enabled on about:config. If it's already enabled, follow what is going on with TOR on start up. Like if the actual firefox browser doesn't load up, stuck at connection launhcer etc, or there is a message box at the browser startup or something... And also make sure no other firefox.exe, geckodriver.exe or tor.exe is running on the background. If multiple executubles are not configured to listen different port numbers, it might cause problems.
Environment.SetEnvironmentVariable("webdriver.gecko.driver", "C:\\TorBrowser\\Browser\\geckodriver.exe");
var gekcoService = FirefoxDriverService.CreateDefaultService("C:\\TorBrowser\\Browser", "geckodriver.exe");
var gekcoService.FirefoxBinaryPath = "C:\\TorBrowser\\Browser\\firefox.exe";
// marionette port that browser listens to. I had it set to a custom port on about:config
var gekcoService.BrowserCommunicationPort = 50111;
// also had given a custom port for geckodriver listen port
var gekcoService.Port = 9881;
var gekcoService.Host = 127.0.0.1;
var gekcoService.HostName = 127.0.0.1;
var gekcoService.Start();
var ffOptions = new FirefoxOptions
{
AcceptInsecureCertificates = true,
BrowserExecutableLocation = "C:\\TorBrowser\\Browser\\firefox.exe",
Profile = null,
UnhandledPromptBehavior = UnhandledPromptBehavior.Dismiss
};
ffOptions.AddArguments("-profile", "C:\\TorBrowser\\Browser\\TorBrowser\\Data\\Browser\\profile.default");
ffOptions.LogLevel = FirefoxDriverLogLevel.Info;
ffOptions.PageLoadStrategy = PageLoadStrategy.Eager;
var ffDriver = new FirefoxDriver(gekcoService, ffOptions);
ffDriver.Manage().Timeouts().PageLoad = TimeSpan.FromSeconds(10);
ffDriver.Manage().Timeouts().AsynchronousJavaScript = TimeSpan.FromSeconds(10);
ffDriver.Manage().Timeouts().ImplicitWait = TimeSpan.FromSeconds(30);
Your code block looks perfect to me.
However there seems to be an issue opening the tor Browser 9.5 which uses the default Firefox v68.9.0esr.
You can find a detailed discussion in How to initiate a Tor Browser 9.5 which uses the default Firefox to 68.9.0esr using GeckoDriver and Selenium through Python
Solution
The solution will be to install and use either of the following browsers:
Firefox v77.0.1
Firefox Nightly v79.0a1
In case you use the above mentioned browsers, you need to:
Firefox v77.0.1:
String torPath = "C:\\Program Files\\Mozilla Firefox\\firefox.exe";
Firefox Nightly v79.0a1
String torPath = "C:\\Program Files\\Firefox Nightly\\firefox.exe";

Internet Explorer 11 Click problem on Xunit

I am using Windows 10 with IE11 on selenium XUNIT test. Selenium doesn't click at a desired IE element. Can someone suggest a solution please?
We had similar problem back in the day when the change the webdriver for IE, whit this code I can run it only in some computers, because in another's we have a lost of comunication between IE and the driver/code due another problem.
public static IWebDriver ExplorerDriverSetUp()
{
var options = new InternetExplorerOptions();
options.RequireWindowFocus = false;
options.IntroduceInstabilityByIgnoringProtectedModeSettings = true;
options.IgnoreZoomLevel = true;
options.EnsureCleanSession = true;
options.InitialBrowserUrl = URL;
IWebDriver driver = new InternetExplorerDriver(options);
driver.Manage().Window.Maximize();
return driver;
}

Unable to get Chrome Performance logs in Selenium C#

I am using following nuget packages in my solution
Selenium.WebDriver - v3.141.0
Selenium.WebDriver.ChromeDriver - v79.0.3945.3600
using following code I am creating a Chrome driver instance
ChromeOptions options = new ChromeOptions();
//Get Performance Logs from Network tab
ChromePerformanceLoggingPreferences perfLogPrefs = new ChromePerformanceLoggingPreferences();
options.PerformanceLoggingPreferences = perfLogPrefs;
options.SetLoggingPreference("performance", LogLevel.All);
(or)
ChromePerformanceLoggingPreferences perfLogPrefs = new
ChromePerformanceLoggingPreferences();
perfLogPrefs.AddTracingCategories(new string[] { "devtools.timeline" });
options.PerformanceLoggingPreferences = perfLogPrefs;
options.SetLoggingPreference("goog:loggingPrefs", LogLevel.All);
options.AddAdditionalCapability(CapabilityType.EnableProfiling, true, true);
and combining with this
options.AddUserProfilePreference("intl.accept_languages", "en-US");
options.AddUserProfilePreference("disable-popup-blocking", "true");
options.AddArgument("test-type");
options.AddArgument("--disable-gpu");
options.AddArgument("no-sandbox");
options.AddArgument("start-maximized");
options.LeaveBrowserRunning = true;
IWebDriver driver = new ChromeDriver(options);
but while creating Chrome driver instance, I am getting following error message
invalid argument: entry 0 of 'firstMatch' is invalid
from invalid argument: perfLoggingPrefs specified, but performance logging was not enabled
May I know what changes do I need to make please to get the performance logs with latest version of Chrome and Selenium driver
I am able to retrieve Performance Logs using the below code when I was using lower versions of Chrome driver (2.35.0)
var logs = driver.Manage().Logs.GetLog("performance");
for (int i = 0; i < logs.Count; i++)
{
Console.WriteLine(logs[i].Message);
}
With Selenium WebDriver (v4.0.0-alpha04) and Selenium.Chrome.WebDriver (v79.0.0) and using the following code, I am able to retrieve the performance logs.
ChromeOptions options = new ChromeOptions();
//Following Logging preference helps in enabling the performance logs
options.SetLoggingPreference("performance", LogLevel.All);
//Based on your need you can change the following options
options.AddUserProfilePreference("intl.accept_languages", "en-US");
options.AddUserProfilePreference("disable-popup-blocking", "true");
options.AddArgument("test-type");
options.AddArgument("--disable-gpu");
options.AddArgument("no-sandbox");
options.AddArgument("start-maximized");
options.LeaveBrowserRunning = true;
//Creating Chrome driver instance
IWebDriver driver = new ChromeDriver(options);
//Extracting the performance logs
var logs = driver.Manage().Logs.GetLog("performance");
for (int i = 0; i < logs.Count; i++)
{
Console.WriteLine(logs[i].Message);
}
Hope this helps.
All these days, I was using the following two lines with latest versions of Selenium WebDriver and Chrome Driver and couldn't figure out what the issue was and now with the latest versions, the following two lines of code is not required.
var perfLogPrefs = new ChromePerformanceLoggingPreferences();
options.PerformanceLoggingPreferences = perfLogPrefs;

FireFox & Selenium not working on specific version

In a project, we have exact guidelines, which Selenium & FireFox Versions to run for UI Tests:
- FireFox: 33.1 (some have 33.1.1, which works also)
- NuGet Selenium.WebDriver 3.3.0
- NuGet Selenium.Support 3.3.0
The FireFoxWebDriver is initialized like this:
var firefoxDirectory = #"C:\Program Files (x86)\Mozilla Firefox\";
var driverExecutableFileName = "firefox.exe";
var profileManager = new FirefoxProfileManager();
var profile = profileManager.GetProfile("default");
profile.EnableNativeEvents = false;
profile.SetPreference("intl.accept_languages", "en-US");
profile.SetPreference("browser.download.folderList", 2);
profile.SetPreference("browser.download.dir", "C:\\Temp");
profile.SetPreference("browser.helperApps.neverAsk.saveToDisk", "text/csv,application/vnd.openxmlformats-officedocument.spreadsheetml.sheet,application/octet-stream");
var defaultPath = $"{firefoxDirectory}{driverExecutableFileName}";
var options = new FirefoxOptions
{
Profile = profile,
UseLegacyImplementation = true
};
var service = FirefoxDriverService.CreateDefaultService(firefoxDirectory, driverExecutableFileName);
if (File.Exists(defaultPath))
{
options.BrowserExecutableLocation = defaultPath;
}
var fireFoxDriver = new FirefoxDriver(service, options, TimeSpan.FromSeconds(30));
return fireFoxDriver;
My problem: It works on every other developer machine, but on mine, the following happens:
As soon as
var fireFoxDriver = new FirefoxDriver(service, options, TimeSpan.FromSeconds(30));
Is hit, an empty FireFox window opens, but then it stops until the Timeout is reached. The length of the timeout doesn't matter, Selenium just doesn't seem to connect.
I uninstalled FireFox, the NuGet cache etc., imported the default-profile from other developers and checked all topics regarding that problem, but most topics are related to version incompatibility, which can't be the problem, since other devs have the same environment.
Are there other known issues or possibilities, what on my machine could influence this behavior?
add this two lines to configuration
profile.SetPreference("browser.startup.homepage_override.mstone", "ignore");
profile.SetPreference("startup.homepage_welcome_url.additional", "about:blank");

How to authenticate (user/password) proxy in c# selenium chrome driver

I have a HTTP/HTTPS proxy which need to authenticate using username and password. How do I do that using C# selenium chrome webdriver?
string host = proxies[count].Split(':')[0];
int port = Convert.ToInt32(proxies[count].Split(':')[1]) + 1;
string prox = host + ":" + port.ToString();
OpenQA.Selenium.Proxy proxy = new OpenQA.Selenium.Proxy();
proxy.HttpProxy = prox;
proxy.SslProxy = prox;
options.Proxy = proxy;
options is the ChromeOptions class which I assign to the driver.
I created little package for yor problem (https://github.com/RDavydenko/OpenQA.Selenium.Chrome.ChromeDriverExtensions)
Install Package:
Install-Package OpenQA.Selenium.Chrome.ChromeDriverExtensions -Version 1.2.0
Use for your ChromeOptions:
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Chrome.ChromeDriverExtensions;
...
var options = new ChromeOptions();
// Add your HTTP-Proxy
options.AddHttpProxy(PROXY_HOST, PROXY_PORT, PROXY_USER, PROXY_PASSWORD);
var driver = new ChromeDriver(options); // or new ChromeDriver(AppDomain.CurrentDomain.BaseDirectory, options);
driver.Navigate().GoToUrl("https://whatismyipaddress.com/"); // Check your IP
Instead of PROXY_HOST, PROXY_PORT, PROXY_USER, PROXY_PASSWORD, use the parameters of your proxy
The only successful approach I have found is to use AutoIT ( https://www.autoitscript.com/site/autoit ).
I have mine setup for proxy authentication for all of the major browsers, but this should work for Chrome.
I wrote this on my phone from memory. If it doesn't work let me know and I'll correct.
WinWait("data:, - Google Chrome","","10")
If WinExists("data:, - Google Chrome","")Then
WinActivate("data:, - Google Chrome")
Send("USERNAMEGOESHERE"{TAB}")
Send("USERPASSWORDGOESHERE"{ENTER}")
Using AutoIT, create this as a script, compile it to exe, save it somewhere and reference the file with the following (Java code):
Runtime.getRuntime().exec("C:\\users\\USERID\\desktop\\FILENAME.exe");
I find it best to call the proxy script a step BEFORE you call the URL that triggers the proxy auth.
2019 Update
After multiple unfortunate attempts, the easiest solution is to create an extension for Chrome as explained by Mike in this post.
It sounds freaky, but it is actually really straightforward.
2021 Update
I have created a simple library (nuget package) that helps you with selenium proxy authentication. You can check the source at Github repo. It also works when using driver headless.
Usage:
public void Test()
{
// Create a local proxy server
var proxyServer = new SeleniumProxyAuth();
// Don't await, have multiple drivers at once using the local proxy server
TestSeleniumProxyServer(proxyServer, new ProxyAuth("123.123.123.123", 80, "prox-username1", "proxy-password1"));
TestSeleniumProxyServer(proxyServer, new ProxyAuth("11.22.33.44", 80, "prox-username2", "proxy-password2"));
TestSeleniumProxyServer(proxyServer, new ProxyAuth("111.222.222.111", 80, "prox-username3", "proxy-password3"));
while (true) { }
}
private async Task TestSeleniumProxyServer(SeleniumProxyAuth proxyServer, ProxyAuth auth)
{
// Add a new local proxy server endpoint
var localPort = proxyServer.AddEndpoint(auth);
ChromeOptions options = new ChromeOptions();
//options1.AddArguments("headless");
// Configure the driver's proxy server to the local endpoint port
options.AddArgument($"--proxy-server=127.0.0.1:{localPort}");
// Optional
var service = ChromeDriverService.CreateDefaultService();
service.HideCommandPromptWindow = true;
// Create the driver
var driver = new ChromeDriver(service, options);
// Test if the driver is working correctly
driver.Navigate().GoToUrl("https://www.myip.com/");
await Task.Delay(5000);
driver.Navigate().GoToUrl("https://amibehindaproxy.com/");
await Task.Delay(5000);
// Dispose the driver
driver.Dispose();
}
This is for firefox but the top answer might help.
C# Selenium WebDriver FireFox Profile - using proxy with Authentication
What you can do is to create a profile and save the authentication data in it. If your profile is called "webdriver" you can select it from your code in the initialization:
ProfilesIni allProfiles = new ProfilesIni();
FirefoxProfile profile = allProfiles.getProfile("WebDriver");
profile.setPreferences("foo.bar",23);
WebDriver driver = new FirefoxDriver(profile);
Chrome won't let you use any extension for selenium web testing these days. And selenium doesn't have any built in username & password input for proxies.
The only solution I found is using AutoIt. It will automates windows level actions. While selenium only automates browser level actions.
Passing username and password to the proxy dialog box is A WINDOWS LEVEL ACTION.
To download and install AutoIt, you can go to : https://www.guru99.com/use-autoit-selenium.html
I wrote a simple AutoIt script as below :
; option to read substring in a window title
Opt("WinTitleMatchMode", 2)
; wait for proxy username & password dialog box
WinWait("- Google Chrome","","10")
; if the box shows up, write the username and password
; username & password are passed as program parameters
If WinExists("- Google Chrome","") Then
WinActivate("- Google Chrome")
; $CmdLine is a special array that holds parameters
if $CmdLine[0] = 2 Then
Send($CmdLine[1] & "{TAB}")
Send($CmdLine[2] & "{ENTER}")
EndIf
; any request dialog to save credential?
WinWait("Save password for ","","10")
; if any, close it
If WinExists("Save password for ","") Then
WinActivate("Save password for ")
Send("{TAB}")
Send("{SPACE}")
EndIf
EndIf
Exit
Compile the script to be an executable program as ProxyAuth.exe.
The program will receive two parameters : username and password.
With parameters, you can use dynamic username & password in your C# script.
Then you need to use the program in C# code as below :
ChromeOptions ChromeOptions = new ChromeOptions();
Proxy proxy = new Proxy();
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.SslProxy = $"{ProxyIP}:{ProxyPort}";
proxy.HttpProxy = $"{ProxyIP}:{ProxyPort}";
ChromeOptions.Proxy = proxy;
ChromeOptions.AddArgument("ignore-certificate-errors");
Driver = new ChromeDriver(ChromeOptions);
Driver.Navigate().GoToUrl(Url);
string cmd = string.Format($"{ProxyUsername} {ProxyPassword}");
Process proc = new Process();
proc.StartInfo.FileName = "ProxyAuth.exe";
proc.StartInfo.Arguments = cmd;
proc.StartInfo.CreateNoWindow = true;
proc.StartInfo.RedirectStandardOutput = true;
proc.StartInfo.RedirectStandardError = true;
proc.StartInfo.UseShellExecute = false;
proc.Start();
You will need to use OpenQA.Selenium, OpenQA.Selenium.Chrome, and System.Diagnostics namespaces.
** EDIT **
If you interested in using AutoIt, you can also use their library in NuGet. Just search in "Manage NuGet Packages" : AutoIt. Choose the one and only one AutoIt package shown and install it.
With this library, you don't need to create ProxyAuth.exe application as described above.
You can use the AutoIt library and make modifications in the C# code :
ChromeOptions ChromeOptions = new ChromeOptions();
Proxy proxy = new Proxy();
proxy.Kind = ProxyKind.Manual;
proxy.IsAutoDetect = false;
proxy.SslProxy = $"{ProxyIP}:{ProxyPort}";
proxy.HttpProxy = $"{ProxyIP}:{ProxyPort}";
ChromeOptions.Proxy = proxy;
ChromeOptions.AddArgument("ignore-certificate-errors");
Driver = new ChromeDriver(ChromeOptions);
Driver.Navigate().GoToUrl(Url);
// use the AutoIt library
AutoItX.AutoItSetOption("WinTitleMatchMode", 2);
AutoItX.WinWaitActive("- Google Chrome", "", 10);
AutoItX.WinActivate("- Google Chrome");
AutoItX.Send(ProxyUsername);
AutoItX.Send("{TAB}");
AutoItX.Send(ProxyPassword);
AutoItX.Send("{ENTER}");
// dismiss save password prompt
AutoItX.WinWaitActive("Save password for ", "", 10);
AutoItX.WinActivate("Save password for ");
AutoItX.Send("{TAB}");
AutoItX.Send("{SPACE}");
Answered in another thread C# Selenium Proxy Authentication with Chrome Driver.
Main idea - use Selenium 4.0 and BiDi APIs.

Categories