We Keep Coding

Selenium ChromeDriver returns 403 errors on build server

We've created a Selenium test project that starts the (ASP.NET) web application and runs a couple of tests using the ChromeDriver. Locally this all runs fine (in headless and non-headless mode).
But on the build server (using an Azure DevOps agent) this fails without ever starting the tests. It looks like it fails when starting the ChromeDriver: the driver starts, but then it's immediately followed by 403 errors. It never gets to the part where it actually loads a webpage.
Any ideas where to look?

Answering my own question to document possible solutions.
After some rigorous investigation (which included using the source code to get to the bottom of things) we found out that the proxy server somehow got in the way. It turned out that the ChromeDriver tries to communicate over a local port (e.g. http://localhost:12345), which was redirected through the proxy server. This failed with a 403 error.
This gave us a lead on possible solutions. First we tried to use the .proxybypass file to exclude localhost addresses. This didn't work -- it turns out that this proxy bypass only works for https requests. And the ChromeDriver control commands are sent over http :-(
We then made sure that no proxy was used in our test code. We did this with the following lines:
var options = new ChromeOptions();
options.AddArgument("--no-sandbox");
options.AddArgument("headless");
options.AddArgument("ignore-certificate-errors");
options.Proxy = new Proxy()
{
Kind = ProxyKind.Direct
};
var driver = new ChromeDriver(options);
In addition to these settings (note that some arguments were added to solve other issues and might not apply to your own situation), we also disabled the proxy for other requests:
WebRequest.DefaultWebProxy = null;
HttpClient.DefaultProxy = new WebProxy()
{
BypassProxyOnLocal = true,
};
This allowed our tests to finally run on the build server without the 403 errors.
One last remark (which might be obvious) is to always run your tests in non-headless mode if you encounter any issues. This allowed us to see the "invalid certificate error" which would otherwise be hidden.

Recurring ERR_CONNECTION_RESET in C# implementation of Selenium ChromeDriver

I've been battling the following error over the past few weeks trying to run UI tests with Selenium (Chrome 92):
WebDriverException unknown error: net::ERR_CONNECTION_RESET. (Session info: chrome=92.0.4515.107)
This occurs most often trying to locate HTML elements using XPath.
Originally, I added a retry loop which checks if the WebDriverException is thrown three times, and if so, recycle the IWebDriver and try again (via IWebDriver.Quit() and IWebDriver.Dispose(). This does not fix the problem, nor does having my application restart after exceeding the retry limit. Thus, I'm starting to think this is a problem at the system-level (Windows Server 2016). Even a reboot does not always fix the issue - when it starts to fail, it just totally freezes and I end up having to completely rebuild the environment.
I'm using the Selenium WebDriver NuGet v92.0.4515.4300; I realize this isn't in total parity with the version listed above, but having the exact same versions does not fix the problem.
I know 'unknown error' is vague. Does anyone have suggestions for what could be causing the problem?
Thanks in advance.
EDIT: Posting browser arguments here, rather than cramming them into a comment. I forgot to mention, there is an instance of Fiddler which we run as a proxy on the local system and use in conjunction with some tests. It's started and ended with the testing application.
The following are injected as options when creating the Chrome instance:
--user-data-dir=C:\temp\{uniquedir}
options.AddUserProfilePreference("credentials_enable_service", false);
options.AddUserProfilePreference("profile.password_manager_enabled", false);
And these are added as command-line arguments through Selenium:
--lang=en-GB
--ignore-certificate-errors
--no-experiments
--disable-translate
--disable-plugins
--enable-logging
--no-sandbox
--dns-prefetch-disable
--disable-gpu
--disable-field-trial-config
(We are doing some security-related detections, hence the disabling of certificate errors.)

You mentioned that you use Fiddler, if the issue is reproducible only when it is started, you can try to regenerate the HTTPS certificates
The issue can be due to the application itself. If the error is shown when you are not using Selenium/ChromeDriver you can ask your developers/devops to check the IIS settings or follow one of the suggestions below.
Check your Internet options from computer Settings->Network&Internet->Proxy. On my computer "Automatically detect Settings" is enabled.
You can also try to delete the Chrome browser cache.
Flush the DNS (ipconfig /flushdns)
It could be some rule (firewall, internal DNS record) configured for the network where is your Windows Server.
Check if the issue is reproducible in incognito mode (without any browser extensions enabled). You can pass --incognito argument when starting the driver.
Issue could be caused by a certificate or SSL/TLS problem.

PushSharp - ios - StopAllServices() hangs with no errors

I am trying to send a push notification to IOS device via PushSharp. For android, it works. For IOS, the call to StopAllServices() hangs forever, without calling any exception handlers.
Could the problem be that I was given a .pem certificate file, and pushsharp requires a .p12 file?
The code is the following:
var br = new PushBroker();
br.OnNotificationSent += br_OnNotificationSent;
br.OnNotificationFailed += br_OnNotificationFailed;
br.OnChannelException += br_OnChannelException;
br.OnServiceException += br_OnServiceException;
br.OnDeviceSubscriptionChanged += br_OnDeviceSubscriptionChanged;
br.OnDeviceSubscriptionExpired += br_OnDeviceSubscriptionExpired;
br.OnChannelCreated += br_OnChannelCreated;
br.OnChannelDestroyed += br_OnChannelDestroyed;
var appleCert = Resource1.ck; // this is a pem file, not a p12 file!!! could this be the problem?
var sandbox = true;
br.RegisterAppleService(new ApplePushChannelSettings(!sandbox, appleCert, "223684"));
// password given to me by ios developer
var deviceIds = new string[] { "09eddcb8b89494adf802a0caf97d5daaa789a53f52d8c544dbdcf39f2c0b619a" };
foreach (var did in deviceIds)
{
br.QueueNotification(
new AppleNotification()
.ForDeviceToken(did)//the recipient device id
.WithAlert("test: " + DateTime.Now.ToString())//the message
.WithBadge(1)
.WithSound("sound.caf"));
}
br.StopAllServices(waitForQueuesToFinish: true); // hangs forever, no callbacks are called
I am using PushSharp taken via Git, and compiled by myself with Visual Studio 2013, as of yesterday.
The hang happens both if the code is in a console application and in an asp.net application.
I am using the sandbox, because I was told to. If I use the production server, I get an exception telling me that the certificate is for the sandbox.
Thanks for any hint as to the cause of the freeze.

We spent a whole day trying to guess the problem!
In the end it was in the wrong Newtonsoft.Json version
Some of the projects in our solution were dependant on the older version of this library as a result we had bad luck to get wrong version in the /bin folder of the Web project.

You can wait few seconds for br_OnNotificationFailed or any other event probably. It should contain some error description.
Nevertheless, I've found out PushSharp has strict requirements about certificates usage. PEM should be OK but it is not enough, even if you import it from file - you should have all necessary certificates in Windows certificates store (pem itself and its dependecies):
Import your PEM to Local Machine\Root storage and give read access rights of its private key to the user of your running application
Import from Apple site certificates Apple Worldwide Developer Relations Certification Authority and Apple Root CA into Local Machine\Trusted Root Certification Authorities
Import Entrust Secure CA certificate (for SSL as described in iOS Developer Library) into Local Machine\Trusted Root Certification Authorities

In the end it was a certificate problem. The .pem I was given is not accepted by PushSharp. Only when I was given a .p12 created with this guide
https://code.google.com/p/apns-sharp/wiki/HowToCreatePKCS12Certificate
, the problem was solved.
However, PushSharp should have raised an exception instead of hanging.

An ASP.NET application is NOT the ideal place to use PushSharp. You'd be better off using a Windows Service, or some other infrastructure if at all possible. The reason is that in an ASP.NET application, the Application Pool (AppPool) can be restarted on you and is usually not under your direct control, which means all the Queued notifications that PushSharp may be in the process of sending could be lost if PushSharp is not cleaned up gracefully.
If you MUST run PushSharp in an ASP.NET application, the best way is to create a singleton PushBroker instance in your Global.asax file. You should keep this singleton instance around for the lifespan of your web application, including a call to pushBroker.StopAllServices() when your Application is ending (Application_End in global.asax).
You can help mitigate losing messages due to unforeseen App Pool terminations or restarts by persisting notifications you want to send in some other way, and only removing them from that persistent storage once the OnNotificationSent event has fired. This is still not perfect (you may risk multiple notification deliveries), but it's probably adequate for most.
You should not be creating and destroying instances of PushBroker each time you send a notification, as this uses unnecessary resources and if you're using Apple APNS, they require you to keep the connection to their servers open as long as possible when sending notifications. You should also call pushBroker.StopAllServices() in your Application_Ended event in your Global.asax. Keep in mind that PushSharp works.
Ref: https://github.com/Redth/PushSharp/issues/240

WCF C# webservice returns nothing, ends client unexpectedly

I have several web services residing on a LIVE server that return data to a client. For some reason since I have moved the services from TEST server to the LIVE server the services are breaking.
My logs indicate that the services are executing correctly serverside but on the client side 'nothing' is returned. I.e. the client exists unexpectedly and the local variables defined 'do not exist in the correct context' when debugging.
PDFLive.PDFServiceClient client = new PDFLive.PDFServiceClient();
string[] response = client2.ReceiptPDF("document string");
string foo = "othercodetobeexecuted";
If I debug over this code, it will attempt to execute 'string[] response ...' and then just skip over the other code and end the client without returning any errors or exceptions and making the local variables
string[] response
and
string foo
'Non existent in the current context'
I have enabled Diagnostics in my web.config and they indicate no errors. Again on the serverside the code executes correctly its the return data that isnt reaching the client or reaching the client but not being processed correctly on the client side.
Ive tried creating the default WCF project and deploying it on my LIVE server and consuming it externally alas, same issue. Could it be something on the Server that is disallowing data to be sent?
I have searched high and low but without any meaningful error being returned its difficult to know what the issue could be. Has anyone had any experience with this issue or come across it in anyway?
I can consume the service via an external test client but when adding a service reference to a C# project it displays the behaviour mentioned above.
SOLVED:
I had to uncheck 'optimize' code in the project settings and my debugger works perfectly now. I got the idea from this post A curious case of Visual Studio 2010 debugger

SOLVED: I had to uncheck 'optimize' code in the project settings and my debugger works perfectly now. I got the idea from this post A curious case of Visual Studio 2010 debugger

Mathematica .Net/Link in an Asp.Net application

I am using the Mathematica .Net/Link platform to create a web service to format and calculate math problems. However I am unable to get it working.
I create it using this code:
_Log.IpDebug("Starting the Kernel Link");
if (string.IsNullOrEmpty(_MathLinkArguments))
_InternelKernel = MathLinkFactory.CreateKernelLink();
else
_InternelKernel = MathLinkFactory.CreateKernelLink(_MathLinkArguments);
_Log.IpDebug("Kernel Link Started");
_InternelKernel.WaitAndDiscardAnswer();
The value of _MathLinkArguments is -linkmode launch -linkname \"C:\\Program Files\\Wolfram Research\\Mathematica\\7.0\\Math.exe\".
This piece of code is called from the Application_Start method of the global.asax.cs file.
When it gets to the WaitAndDiscardAnswer() call it gives the server error:
Error code: 11. Connected MathLink program has closed the link, but there might still be data underway.
Note: The SampleCode given with the .NET/Link package (both a console app and a WinForms app) works.
Edit:
I copied the console app sample code given with Mathematica into an asp.net page and it gave me the same error the first load and then on subsequent loads it gave me:
Error code: 1. MathLink connection was lost.
Edit2:
I forgot to mention that when I have procmon and task manager open while running my app, I can tell that Math.exe starts but it immediately exits, which makes those error code make complete sense...but doesn't explain why that happened.

To allow the .Net/Link to work in Asp.net (at least in IIS 7.5) you need to enable the property loadUserProfile on the app pool for the web site.
I am not entirely sure why this is the case, but from what I found while trying to debug this, there are some things that are gotten from the user's profile. I know for a fact that the default location of the kernel is, which explains why I couldn't use it with no arguments, and so I can only assume that other things are needed as well and without the profile it couldn't determine that.
But whatever the reason is this is required, it is, or at least it is a fix if you are getting similar problems like this in your own application.

I got the same error in a .Net WinForm application.
mathKernel = new MathKernel();
mathKernel.Compute("<< XYZ`XYZGraphs`");
The error occurred on loading the package straight after instantiating the MathKernel.
To resolve it you can wait a couple of seconds and then instantiating the MathKernel works fine. During this state where there might still be data underway the following conditions are both false:
if (!MathKernel.IsConnected)
{
MathKernel.Connect();
}
if (MathKernel.IsComputing)
{
MathKernel.Abort();
}
Edit:
I've recieved the error again and this time was able to determine the problem.
Using a command line open the MathKernel.exe and view the error message: