Selenium WebDriver exception when multiple processes run in parallel - c#

I have a process, using Selenium Web driver for Firefox, for automatic look up for information on one specific site. Due to the volume of the data being looked up, I am running in parallel several copies of this process. Sometimes they are ok, sometimes I get an exception, which I do not understand the cause for. The Process is written in C#, runs on Windows. Each process creates its own instance of Selenium, FF web driver, I see each process has a separate FireFox process. No multi-treading (inside one process). I do not understand what exactly is the issue. Could not find any documentation that could tell me more.
If anyone has an idea, I'd appreciate any info/suggestions
The exception is:
"OpenQA.Selenium.WebDriverException: Unexpected error. System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted 127.0.0.1:7056"
I can not see what can be trying to make more than one "usage" of the socket address in one given process. And from the logs I see that the different processes use different ports ( the "7056" above is a different number in each process).
Thank you in advace

I recently ran into this and found a pretty interesting article about fine tuning your network settings.
https://blogs.msdn.microsoft.com/dgorti/2005/09/18/only-one-usage-of-each-socket-address-protocolnetwork-addressport-is-normally-permitted/
so pretty much if you run a high degree of parallelism in your tests with not enough ports open and a high timeout you will run out of sockets and this will happen. either increase the ports, reduce the wait time and check what can be leaving your instances hanging.

Related

Number of web socket clients that could be opened at once

I am simply looking at the sample code found here:
When I run the server portion and start multiple instances of the client I notice that when I start around 40-50 of them at the same time (using Process.Start()) that sometimes some clients fail to connect.
Why does this happen? What actually stops all these clients from connecting at once? Is there a request limit hidden somewhere?
Are you sure the limitation is not on the server?
I use ClientWebSocket to do some simplistic stress test on my WebSocket component and I can reach thousands or connections and almost a 100% throughput of my NIC. However, I do not create a process for each call. You can see the test console app source code or just download the executable here.

C# A quick way to find out if a network path is available using NetBEUI protocol

Im writing a application that reads logs from 1-many computers in the network. The network computers with the logs dont have tcp/ip installed, they are using NetBEUI protocol instead.
So i access them with "\\computername\c$\path-to-logs"
My question is, how can i access them without having to wait for the long network wait if the network is not available? It could be 1 computer with logs...and it could be up to 5
Example:
check \\computer1\c$\path-to-logs ...found it, copy logs
check \\computer2\c$\path-to-logs ...found it, copy logs
check \\computer3\c$\path-to-logs ...didnt find it (here is normally a long wait before i get the timeout that it doesnt exist)
Best regards Andreas
Andreas,
simplest solution is to make it multi-thread, to open a thread per remote PC.
In communication, you always need to pay attention to communication- time - out when one of the PC's is not available. Multi threading with limiting communication time out is the solution I usually using.

High performance C# TCP server problem: No connection could be made because the target machine actively refused it

I have developed a TCP server according to your advises: High performance TCP server in C#
It is based on asynchron pattern.
I also developed a stress test application to test its performance. My server can get thousands of connections paralelly from my stress test app, can parse data and save it to my database.
When I stress my server, I can get "System.Net.Sockets.SocketException "No connection could be made because the target machine actively refused it" error from my server, so I have to reconnect to it. If I test it with 5000 concurrent connections, I have to try connect again because of this problem 10-20% of the connections, if I test it with 10K concurrent connections, it can be 30-40%. Sometimes it can be - very rarely - more, than 50%. It seems it can not handle connection accepts: I make new connections from my stress test as heavily as my test machine can - about 120 connections/sec.
So, what can cause this kind of exception? How to handle it? What to do in server side implementation to avoid this problem? How to tune TCP connection accept?
Thanks in advance!
You might be running out of available ports every now and then. You can view this easily using SysInternals' TcpView utility.
On Windows, when you release a port, it doesn't immediately go into an available state, but instead sits in a TIME_WAIT state for some interval. Until it leaves this state, no app can use this port. The time delay, the max number of ports, and the available port ranges are all different to the OS, XP vs Win7 vs Win2008 Server.
There are two registry entries that can reduce this time interval:
HKLM/System/CurrentControlSet/Services/Tcpip/Parameters/TCPTimedWaitDelay
and increase the max number of ports that can be opened by an app:
HKLM/System/CurrentControlSet/Services/Tcpip/Parameters/MaxUserPort
EDIT: MaxFreeTcbs seems to be a third setting which could help (I haven't tried this yet), mentioned in this TechNet article which has more advice on tracking down odd network problems. HTH.
You are making connections faster than the software can listen for new connections, or in other words you are reaching the connections per second limit of that port. I think you can double the amount of connections per second by listening to a second port, client side you should just reconnect when you get the exception.
There are also limits applied to the amount of connection, for these see Chris O's answer.

Testing Network/Internet connection issues

What methods are there to test how well a C# desktop application handles various network/internet failures, and which, if any, failures it doesn't handle?
Thanks in advance.
Not an official test but still useful is the good old "yank the cat5e from the wall" test. You'd be surprised how many applications simply crash.
I recommend at least the following:
1) As Peter recommended, yank the cable. This should be done on both server and client machines.
2) A more subtle test is to yank the cable on an intermediate router, so both computers still "see" the cable is connected. This often results in a half-open connection.
3) Another test is to crash one side but leave the OS connected (e.g., Task Manager's End Process). This should also be done on both server and client machines.
These all test different errors and are pretty simple to do.
In addition, you could write an "error simulator" that acts like a badly-behaved client or server implementation. Have it try to send huge messages, or violate the protocol, or send data when it's not supposed to... There are many possibilities, depending on the protocol.

Fast way to check if a server is accessible over the network in C#

I've got a project where I'm hitting a bunch of custom Windows Performance Counters on multiple servers and aggregating them into a database. If a server is down, I want to skip it, and just continue on with my day.
Currently I'm checking to see if a server is live by doing a DirectoryInfo on a share that I've got to look at later in the process anyways, then checking the .Exists property.This is my current code snippet for testing:
DirectoryInfo di = new DirectoryInfo(machine.Share_Path);
if (!di.Exists)
{
log.Warn("Could not access " + machine.Name + "! Maybe its down?");
continue; // Skips to the next server in my loop where this snippet exists.
}
This works, but its pretty slow. It takes about 68 seconds on average for the di.Exists bit to finish its work, and I ideally need to know within a second whether or not a server is accessible. Pinging also isn't an option since a server can be pingable but not "live" in our environment.
I'm still kind of fresh to the .NET world, so I'm open to any advice people can offer.
Thanks in advance.
-Weegee
Ping First, Ask Questions Later
Why not ping first, and then do the di.Exists if you get a response?
That would allow you to fail early in the case that is not reachable, and not waste the time for machines that are down hard.
I have, in fact, used this method successfully before.
MSDN Ping Documentation
Paralellize
Another option you have is to paralellize the checking, and action on the servers as they are known to be available.
You could use the Paralell.ForEach() method, and use a thread-safe queue along with a simple consumer thread to do the required action. Combined with the checking method above, this could alleviate almost all of your bottleneck on the up/down checking.
Knock on the Door
Yet another method would be to ckeck if the required remote service is running (either by hitting its port directly or by querying it with WMI).
Since WMI is almost always running when a machine is up, your connection should be very quick to either succeed or fail.
The only "quick" way I think to see if it's up without relying on ping would be to create a socket, and see if you can actually connect to the port of the service you're trying to reach.
This would be the equivalent of telnet servername 135 to see if it's up.
Specifically...
Create a .NET TCP socket client (System.Net.Sockets.TcpClient)
Call BeginConnect() as an asynchronous operation, to connect to the server in question on one of the RPC ports that your directory exists code would use anyway (TCP 135, 139, or 445).
If you don't hear back from it within X milliseconds, call Close() to cancel the connection.
Disclaimer: I have no idea what effect this would have on any threat/firewall protection that may see this type of Connect / Disconnect with no data sent activity as a threat.
Opening Socket to a specific port usually does the trick. If you really want it to be fast, be sure to set the NoDelay property on the new socket (Nagle algorithm) so there is no buffering.
Fast will largely depend on latency but this is probably the fastest way I know to connect to an endpoint. It's pretty simple to parallelize using the async methods. How fast you can check will largely depend on your network topology but in tests for 1000 servers (latency between 0-75ms) I've been able to get connectivity state in ~30 seconds. Not scientific data at all but should give you the idea.
Also, don't ever do this through UNC file shares because if the server no longer exists you will have a lot of dangling connections that take forever to timeout. So if you have a lot of servers with invalid DNS records and you try to poll them you will bring Windows down completely over time. Things like File.Exists and any file access will cause this.
The "Full-Blown" option would be to install a monitoring tool like SCOM (System Center Operations Manager), this has an SDK you can use to query SCOM for (performance) and maintenance information avout machines being monitored. Might be a bridge to far though....
Telnet is another option. Try telnetting to the target machine to see if it responds.
Create a small Windows Service that you install on your target machine, have the sys admin stop it when they perform maintenance on the target machine (just use batch file to net stop / net start the service)

Categories