We Keep Coding

C# Automation Edge Browser - using Edge Driver - auto testing program - Failure: No matching capabilities found (SessionNotCreated)

Greetings stackoverflow community,
I am trying to compile and run the programcode from this website:
https://social.msdn.microsoft.com/Forums/en-US/7bdafd2a-be91-4f4f-a33d-6bea2f889e09/c-sample-for-automating-ms-edge-chromium-browser-using-edge-web-driver
I followed all the instructions listed in the link and set my paths were I wanted them.
The program and the edge driver starts running, but then an error appears.
"An error exeption "System.InvalidOperationException" appeared in WebDriver.dll.
Further Inforamtion: session not created: No matching capabilities found (SessionNotCreated)"
This is the code from my program, more or less copied from the link above:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using OpenQA.Selenium;
using OpenQA.Selenium.Chrome;
using OpenQA.Selenium.Edge;
using OpenQA.Selenium.Remote;
using OpenQA.Selenium.Support.UI;
namespace ConsoleApplication2
{
class Program
{
static void Main(string[] args)
{
var anaheimService = ChromeDriverService.CreateDefaultService(#"C:\edgedriver_win64", "msedgedriver.exe");
// user need to pass the driver path here....
var anaheimOptions = new ChromeOptions
{
// user need to pass the location of new edge app here....
BinaryLocation = #"
C: \Program Files(x86)\Microsoft\Edge\Application\msedge.exe "
};
IWebDriver driver = new ChromeDriver(anaheimService, anaheimOptions); -- error appears at this line
driver.Navigate().GoToUrl("https: //google.com/");
Console.WriteLine(driver.Title.ToString());
driver.Close();
}
}
}
I would really appreciate your help!
Best Regards
Max

The article you refer to is a bit out of date. Now we don't need to use ChromeDriver to automate Edge. You can refer to the official doc about how to use WebDriver to automate Microsoft Edge.
I recommend using Selenium 4. Here I install Selenium 4.1.0 NuGet package and the sample C# code is like below:
using System;
using OpenQA.Selenium.Edge;
namespace WebDriverTest
{
class Program
{
static void Main(string[] args)
{
var options = new EdgeOptions();
options.BinaryLocation = #"C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe";
var driver = new EdgeDriver(#"C:\edgedriver_win64", options);
driver.Navigate().GoToUrl("https://www.google.com");
Console.WriteLine(driver.Title.ToString());
driver.Close();
}
}
}

Load multiple pages with Htmlagilitypack - C#

I've scraped a website with HTMLAgilityPack in C#, and I'm trying to open all link inside it and scrape them with same method.
But when I try to call this method bottom, page is downloaded from library as I have AdBlock active. In fact, I can't find any tables and HTML code downloaded says "ADblock detected".
This is strange because I've filter oddsmath website on my Google Chrome and I can download the master page withouth any problem. Anyone has faced with this problem?
This is the function and the "Console.WriteLine" is just for testing and see full HTML code.
public void GetMatchesDetails()
{
List<String> matchDetails = new List<string>();
foreach (Oddsmath om in oddsmathGoodMatches)
{
matchDetails.Add("http://www.oddsmath.com" + om.matchUrl);
}
foreach (String om in matchDetails)
{
HtmlDocument doc = new HtmlWeb().Load(om);
foreach (HtmlNode table in doc.DocumentNode.SelectNodes("html"))
{
Console.WriteLine("Found: " + table.OuterHtml);
foreach (HtmlNode row in table.SelectNodes("tr"))
{
Console.WriteLine("row");
foreach (HtmlNode cell in row.SelectNodes("th|td"))
{
Console.WriteLine("cell: " + cell.InnerText);
}
}
}
}
}
EDIT
Going little deeper, I've noticed that maybe is not a problem of my application or something related to Adblock, but seems connected to website i'm trying to scrape... In fact, if you see a page like this: oddsmath.com/football/international/afc-champions-league-1053/… you can see that content are correctly loaded in browser, but tables are empty inside source code. Why? It's Javascript that prevents loading of page?

First: Use whatever you are most comfortable with HAP vs AngleSharp unless time is really a factor in your application. And in this case it is not.
Second: Use a Web Debugger like Fiddler or Charles to understand what it is that you are actually getting from the when you make a request. Since you are not actually getting any html created with javascript or api calls. You only get the page source. Which is why the tables are empty. They are generated with either javascript.
For instance. I just used a web debugger to see that the site makes an api call to:
http://www.oddsmath.com/api/v1/dropping-odds.json/?sport_type=soccer&provider_id=7&cat_id=0&interval=60&sortBy=1&limit=30&language=en
Then javascript will use this json object to create the rest of page.
And this returns a nice json object that is easier to navigate than with eithr HAP or AngleSharp. I recommend using NewtonSoft JSON.
If you are adamant on using HtmlAgilityPack then you need to combine it with Selenium. Because then you can wait until the page is fully loaded before parsing the HTML.
[Edit]
Further digging:
Api-request to get all the leagues and their ids:
http://www.oddsmath.com/api/v1/menu-leagues.json/?language=en
Api-request for just the asian champions league:
http://www.oddsmath.com/api/v1/events-by-league.json/?language=en&country_code=GB&league_id=1053
Other solution with Selenium with Firefox driver.
Eventhough I highly recommend that you use API and NewtonSoft-JSON to your solution I will provide how it can be done with Selenium.
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using HtmlAgilityPack;
using OpenQA.Selenium.Firefox;
using OpenQA.Selenium;
using System.Threading;
namespace SeleniumHap {
class Program {
static void Main(string[] args)
{
HtmlDocument doc = new HtmlDocument();
string url = "http://www.oddsmath.com/football/sweden/division-1-1195/2019-04-26/if-sylvia-vs-nykopings-bis-2858046/";
//string url = "http://www.oddsmath.com/";
FirefoxOptions options = new FirefoxOptions();
//options.AddArguments("--headless");
IWebDriver driver = new FirefoxDriver(options);
driver.Navigate().GoToUrl(url);
while (true) {
doc.LoadHtml(driver.PageSource);
HtmlNode n = doc.DocumentNode.SelectSingleNode("//table[#id='table-odds-cat-0']//*[self::th or self::td]");
if (n != null) {
n = n.SelectSingleNode(".//div[#class='live-odds-loading']");
if (n == null) {
break;
}
}
Thread.Sleep(1000);
}
Console.WriteLine("Exited loop. Meaning the page is done loading since we could get a td. A Crude method but it works");
HtmlNodeCollection tables = doc.DocumentNode.SelectNodes("//table");
foreach(HtmlNode table in tables) {
Console.WriteLine(table.GetAttributeValue("id", "No id"));
HtmlNodeCollection tableContent = table.SelectNodes(".//*[self::th or self::td]");
foreach(HtmlNode n in tableContent) {
Console.WriteLine(n.InnerHtml);
}
break;
}
Console.ReadKey();
}
}
}
As you can see I use Firefox as my driver instead of chrome. When using either you might have to edit the options where you edit the variable 'BrowserExecutableLocation' to tell where the browser's executable is.
As you can see I am using a while loop in a crude way to make sure that the browser fully loads page before continuing on reading html.

Selenium C# Can't find ID or Title

I manage to open a firefox browser, go to http://www.google.com/ search for "Bath Fitter". When i see a bunch of links, i want to in fact click on an item of the top menu provided by Google, Images. Images is located next to Map Videos News...
How can i have it click on Images?
Below is my code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using OpenQA.Selenium;
using OpenQA.Selenium.Firefox;
namespace SeleniumHelloWorld
{
class Program
{
static void Main(string[] args)
{
IWebDriver driver = null;
try
{
driver = new FirefoxDriver();
driver.Navigate().GoToUrl("http://www.google.com/");
driver.Manage().Window.Maximize();
IWebElement searchInput = driver.FindElement(By.Id("gbqfq"));
searchInput.SendKeys("Bath Fitter");
searchInput.SendKeys(Keys.Enter);
searchInput.FindElement(By.Name("Images"));
searchInput.Click();
driver.Close();
}
catch (Exception e)
{
Console.WriteLine("Exception ****" + e.ToString());
}
}
}
}

More specifically you can also write your selector pointing from Top Navigation. This is the XPath.
.//*[#id='hdtb_msb']//a[.='Images']
try this;
driver.FindElement(By.XPath(".//*[#id='hdtb_msb']//a[.='Images']"));
EDIT:
Even though the selectors above were correct your code was not working because of the second page was taking too long to load. There you need to wait for the the element to be in ready state and an implicit wait is needed. Change the code in your try block and replace with mine and try
driver = new FirefoxDriver();
driver.Navigate().GoToUrl("http://www.google.com/");
driver.Manage().Window.Maximize();
IWebElement searchInput = driver.FindElement(By.Id("gbqfq"));
searchInput.SendKeys("Bath Fitter");
searchInput.SendKeys(Keys.Enter);
//this is the magic
driver.Manage().Timeouts().ImplicitlyWait(TimeSpan.FromSeconds(10));
By byImage = By.XPath(".//*[#id='top_nav']//a[.='Images']");
IWebElement imagElement =
new WebDriverWait(driver, TimeSpan.FromSeconds(10)).Until(ExpectedConditions.ElementExists(byImage));
imagElement.Click();

Try something like this...
IList<IWebElement> links = driver.FindElements(By.TagName("a"));
links.First(element => element.Text == "Images").Click();

How to retrieve a value/url?

I apologize in advance as I have not used visual studio much before and I used a template for the most part to create the code below. But I really need some help.
I'm trying to generate a report from a system using visual studio through an API. The code below works and doesn't give me any errors when debugging, but I need to find a way to retrieve the reportURL variable at the end (the report is generated with the specifications below and I should receive a URL to download the report). I am building as a windows application.
Is there anything like console.log or console.writeline in visual studio I can use? Or can I output it to a textbox of some kind? (Again, I am building as a windows form and not a console application). Can anyone help me figure out some kind of code I can use to retrieve the URL based on what is provided below? (Please be detailed if possible as I am still getting used to the program). Thanks!
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows.Forms;
namespace ReportsApplication2
{
using ServiceReference1;
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
ReportExecutionServiceClient client = new ReportExecutionServiceClient();
ReportSubmission submissionRequest = new ReportSubmission();
ReportSubmissionResponse submissionResponse = new ReportSubmissionResponse();
PollReportResponse pollResponse = new PollReportResponse();
WebMediaReportRequest webRepReq = new WebMediaReportRequest();
UserCredentials userCredentials = new UserCredentials();
DateFilter dateFilter = new DateFilter();
userCredentials.UserName = "xxxxx";
userCredentials.Password = "xxxxx";
submissionRequest.UserCredentials = userCredentials;
submissionRequest.DeveloperToken = "xxxxxx";
dateFilter.DateFilterType = DateFilterType.Total;
dateFilter.RelativeDateRange = RelativeDateRange.LastMonth;
webRepReq.Columns = new WebMediaReportColumn[2] { WebMediaReportColumn.MediaPlanName, WebMediaReportColumn.Impressions };
List<WebMediaFilter> webRepFilterList = new List<WebMediaFilter>();
WebMediaFilter webRepFilter = new WebMediaFilter();
webRepFilter.Column = WebMediaReportFilter.ClientGUID;
webRepFilter.Values = new string[1] {"xxxxxx"};
webRepFilterList.Add(webRepFilter);
webRepFilter = new WebMediaFilter();
webRepFilter.Column = WebMediaReportFilter.BuyGUID;
webRepFilter.Values = new string[1] { "xxxxxxxx" };
webRepFilterList.Add(webRepFilter);
webRepReq.ReportName = "test";
webRepReq.Filters = webRepFilterList.ToArray();
webRepReq.Format = FormatType.CSV;
webRepReq.DateFilter = dateFilter;
submissionRequest.ReportRequest = webRepReq;
submissionResponse = client.SubmitReport(submissionRequest);
string reportURL = string.Empty;
do { // Loop until report complete or failed
PollReportRequest pollRequest = new PollReportRequest();
pollRequest.DeveloperToken = "xxxxxxx";
pollRequest.UserCredentials = userCredentials;
pollRequest.ReportId = submissionResponse.ReportId;
pollResponse = client.PollReport(pollRequest);
reportURL = pollResponse.Url;
} while ((pollResponse.Status != ReportStatus.Failed) || ((pollResponse.Status != ReportStatus.Complete)));
}//end

You actually already know what you need. The Console.WriteLine method will write anything you want to the Console.
While you are debugging in Visual Studio, you can view the Console window by turning on the "Output" window. You can find it in the View menu
Even though you are writing a WinForms application, this will still write to the Output window while debugging. It can be an effective tool when debugging, but of course, when you actually publish the application, the command will be meaningless since you won't have a Console to write too.

If you are using Winforms, the simplest approach to show messages is MessageBox.Show():
MessageBox.Show(reportURL);
It will block the program until you click Ok.
http://msdn.microsoft.com/en-us/library/0x49kd7z(v=vs.110).aspx
If you want to output debug info during development, then Console.WriteLine() will work:
Console.WriteLine(reportURL);
It will show in the Output window. If you don't see that, go to the main menu in Visual Studio and select View -> Output, while in debug mode.
Console.WriteLine() will only work within Visual Studio for a Winforms app; after deploying it, the console window doesn't show, so you'll need to show it via a GUI method.
If you need to output it at runtime, either drag a TextBox from the toolbox onto your main form, and do:
textBox.Text = reportURL;
If you can't show it on the main form, then you may want to create your own custom message dialog that includes a TextBox and show it with ShowDialog()

The simplest solution is to use Console.WriteLine.

How would one simulate proxy functions without the HttpListener or TCPListener components in C#?

How can I simulate the functions/actions of a proxy server but without calling elements like HttpListener or TcpListener? How can I generate them from within my C# application?
I've been able to get as far as getting actual data streamed back to my WebBrowser element in my C# application but upon viewing the results, it gives me errors. The reason being is because I'm viewing the LITERAL string and there are JS/CSS components within the resulting HTML stream that makes references to objects via relative URIs. Obviously, my solution thinks they're local and, as such, can't resolve them.
I'm missing proxy-like functions where it should just hand off the stream back to my mock browser and display properly. However, looking at sample proxy server codes built on C#, they're all built as servers using listeners. I'd like it to be something that I can instantiate locally without the need to create a listening interface.
Now, you may be wondering why I'm trying to do this? Well, there are a couple of reasons:
To be able to inject headers ad-hoc so I can test internal web servers
To run as a headless (no GUI) component that can take either HTTP or HTTPS streams from other .NET components and inject headers from, yet, other .NET components.
Some other back-end stuff that I think might but won't know until I have this in place.
Here's what I have so far:
using System;
using System.Collections.Generic;
using System.ComponentModel;
using System.Data;
using System.Drawing;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using HtmlAgilityPack;
using System.Net;
namespace WindowsFormsApplication1
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
}
private void button1_Click(object sender, EventArgs e)
{
WebClient client = new WebClient();
var baseUrl = new Uri(textBox1.Text);
client.Headers.Add("Token1", textBox2.Text);
client.Headers.Add("Token2",textBox3.Text);
byte[] requestHTML = client.DownloadData(textBox1.Text);
string sourceHTML = new UTF8Encoding().GetString(requestHTML);
HtmlAgilityPack.HtmlDocument htmlDoc = new HtmlAgilityPack.HtmlDocument();
htmlDoc.LoadHtml(sourceHTML);
//"//*[#background or #lowsrc or #src or #href]"
foreach (HtmlNode link in htmlDoc.DocumentNode.SelectNodes("//*[#href]"))
{
//Console.Out.WriteLine(link.ToString());
if (!string.IsNullOrEmpty(link.Attributes["href"].Value))
{
HtmlAttribute att = link.Attributes["href"];
Console.WriteLine("Before: " + att.Value);
//Console.Out.WriteLine(att.Value.ToString());
Console.WriteLine(new Uri(baseUrl, att.Value));
link.Attributes["href"].Value = new Uri(baseUrl, att.Value).ToString();
Console.WriteLine("After: " + link.Attributes["href"].Value);
//att.Value = this.AbsoluteUrlByRelative(att.Value);
}
}
foreach (HtmlNode link2 in htmlDoc.DocumentNode.SelectNodes("//*[#src]"))
{
//Console.Out.WriteLine(link.ToString());
if (!string.IsNullOrEmpty(link2.Attributes["src"].Value))
{
HtmlAttribute att = link2.Attributes["src"];
Console.WriteLine("Before: " + att.Value);
// //Console.Out.WriteLine(att.Value.ToString());
Console.WriteLine(new Uri(baseUrl, att.Value));
if (!att.Value.Contains("/WS"))
{
Console.WriteLine("HIT ME!");
var output = "/WS/" + att.Value;
link2.Attributes["src"].Value = new Uri(baseUrl, output).ToString();
Console.WriteLine("After HIT: " + link2.Attributes["src"].Value);
}
else
{
link2.Attributes["src"].Value = new Uri(baseUrl, att.Value).ToString();
Console.WriteLine("After: " + link2.Attributes["src"].Value);
}
// //att.Value = this.AbsoluteUrlByRelative(att.Value);
}
}
Console.WriteLine(htmlDoc.DocumentNode.OuterHtml);
Console.WriteLine("+========================+");
webBrowser1.DocumentText = htmlDoc.DocumentNode.OuterHtml;
}
}
}
Again, this is just prototyped code so forgive the wacky spacing and commenting. In the end, it will be more formal. Right now, this monkey is killing my back.

How about using something like NMock or similar? It would mean having to introduce interfaces so that the mocks can be injected, but still beats doing it almost any other way, IMHO...
From the NMock site:
NMock is a dynamic mock object library for .NET. Mock objects make it
easier to test single components—often single classes—without relying
on real implementations of all of the other components. This means we
can test just one class, rather than a whole tree of objects, and can
pinpoint bugs much more clearly. Mock objects are often used during
Test Driven Development.
You would mock the proxy server more or less like this:
var mocks = new Mockery();
var mockProxyServer = mocks.NewMock<IMyProxyServer>();
That's all you need to do. As you can see, it's interface-dependent. But usually all that I've needed to do is Refactor->Extract Interfaces from the relevant class in VS.
Setting up the simulation is usually done within the context of the unit test, like:
public class TransferFundsPresenterTest
{
private Mockery mocks;
private IMyProxyServer mockProxyServer
[SetUp]
public void SetUp()
{
mocks = new Mockery();
mockProxyServer = mocks.NewMock<IMyProxyServer>();
}
[Test]
public void TestProxyFunction()
{
Expect.Once.On(mockProxyServer).
Method("ProxyFunctionA").
With("1234"). // <-- simulate the input params here
Will(Return.Value("Test")); // <-- simulate the output from server here
}
This is just a basic example. You can do a lot more, it's a very flexible library.
You really should take a look at the NMock site, it's pretty easy to get fully up to speed with the library.
http://www.nmock.org/index.html