We Keep Coding

Exception in cluster.Connect(<keyspace>) using CassandraCSharpDriver v3.9.0

I am getting the following exception in cluster.Connect while performing load testing in my application.
All hosts tried for query failed (tried :: SocketException 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'; :: SocketException 'A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond'; ...)
The exception is thrown only when I overwhelm my application with a large number of requests (1000 or more). However, it seems to work fine for small number of requests because of which I am eliminating the possibility of a firewall issue. For some reason, this call gets stuck which leads to a deadlock in my application. Any help is greatly appreciated. Thanking in advance.
Regards,
Sitakanta Mishra

.NET/C# to MySql running on linux - exception on first command, but subsequent commands do work

Have a really crazy situation. I can't post specifics, so I'm just looking for general guidance. We have already opened a ticket with Oracle/MySql support. I'm just looking to see if anyone else has run into this situation or anything similar. Here is our scenario:
Windows 2012 R2 Server with .NET 4.7.1 running.
Simple Windows Forms .NET application.
We are trying to run a simple query against a Linux MySql Server. MySql is Enterprise Version 5.7.x.
On the first attempted connection, the Windows Forms app locks the UI, waits about 15 seconds, and then reports back that there is an error running the command. The error is shown below.
System.ApplicationException: An exception occurred on the following sql command:select * from tablename where compl_date >= '2019-12-17 04:44:34 PM' ---> MySql.Data.MySqlClient.MySqlException: Authentication to host 'ip address' for user 'userid' using method 'mysql_native_password' failed with message: Reading from the stream has failed. ---> MySql.Data.MySqlClient.MySqlException: Reading from the stream has failed. ---> System.IO.EndOfStreamException: Attempted to read past the end of the stream.
When this error pops up, if I click on the "Continue" button, subsequent calls to the database work as intended (at about a 95% rate).
On the server, the mysqld error logs are shown below for the first call. Subsequent calls do work.
2019-12-16T22:06:29.554171Z 3496 [Warning] IP address 'client ip address' could not be resolved: Name or service not known
2019-12-16T22:06:50.188443Z 3496 [Note] Aborted connection 3496 to db: 'drupaldb' user: 'userid' host: 'ip address' (Got an error reading communication packets)
2019-12-17T02:53:17.832725Z 0 [Note] InnoDB: page_cleaner: 1000ms intended loop took 11355ms. The settings might not be optimal. (flushed=0 and evicted=0, during the time.)
2019-12-17T03:25:18.200855Z 3527 [Note] Got an error reading communication packets
2019-12-17T03:25:37.167395Z 3528 [Note] Got packets out of order
2019-12-17T03:25:37.382512Z 3529 [Note] Got packets out of order
2019-12-17T03:25:47.688836Z 3530 [Note] Bad handshake
2019-12-17T14:26:33.619967Z 4803 [Note] Got timeout reading communication packets
2019-12-17T19:34:34.741441Z 4851 [Note] Got timeout reading communication packets
2019-12-17T19:47:47.595426Z 4853 [Note] Got timeout reading communication packets
2019-12-17T19:48:45.586357Z 4854 [Note] Got timeout reading communication packets
If you have some general ideas, let me know.
FYI, we have some other linux/mysql instances, and this runs just fine.

At this point, we think we have solved the problem, at least for the short term. Both server and client are sitting on a private network. We think that the database server is trying to send a certificate to the windows client. The windows client is also on this private network. We think the Windows Client is not accepting the ssl certificate and that this is causing the failure on the first connection attempt. By adding the option "SslMode=None", this seems to resolve the issue.
Blog post we found that helped us: https://blog.csdn.net/fancyf/article/details/78295964

TcpClient SocketException with timeout after 20s no matter what

I'd like to wait for a slow response from a client with TcpClient but get a timeout after about 20s no matter how I configure it. This is my attempt:
using (var client = new TcpClient { ReceiveTimeout = 9999999, SendTimeout = 9999999 })
{
await client.ConnectAsync(ip, port);
using (var stream = client.GetStream())
{
// Some quick read/writes happen here via the stream with stream.Write() and stream.Read(), successfully.
// Now the remote host is calculating something long and will reply if finished. This throws the below exception however instead of waiting for >20s.
var bytesRead = await stream.ReadAsync(new byte[8], 0, 8);
}
}
The exception is an IOException:
Unable to read data from the transport connection: A connection
attempt failed because the connected party did not properly respond
after a period of time, or established connection failed because
connected host has failed to respond.
...which contains a SocketException inside:
A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection
failed because connected host has failed to respond
SocketErrorCode is TimedOut.
The 20s seems to be an OS default on Windows but isn't it possible to override it from managed code by interacting with TcpClient? Or how can I wait for the response otherwise?
I've also tried the old-style BeginRead-EndRead way and the same happens on EndRead. The problem is also not caused by Windows Firewall or Defender.

I'd like to wait for a slow response from a client
It's important to note that it's the connection that is failing. The connection timeout is only for establishing a connection, which should always be very fast. In fact, the OS will accept connections on behalf of an application, so you're literally just talking about a packet round-trip. 21 seconds should be plenty.
Once the connection is established, then you can just remove the ReceiveTimeout/SendTimeout and use asynchronous reads to wait forever.

It turns out that the remote host wasn't responding in a timely manner, hence the problem. Let me elaborate, and though this will be a solution very specific to my case maybe it will be useful for others too.
The real issue wasn't a timeout per se, as the exception indicated, but rather what exceptions thrown on subsequent Read() calls have shown: "An existing connection was forcibly closed by the remote host"
The remote host wasn't purposely closing the connection. Rather what happened is that when it was slow to respond it was actually so busy that it wasn't processing any TCP traffic either. While the local host wasn't explicitly sending anything while waiting for a response this still was an issue: the local host tried to send ACKs for previous transmissions of the remote host. Since these couldn't be delivered the local host determined that the remote host "forcibly closed" the connection.
I got the clue from looking at the traffic with Wireshark (always good to try to look at what's beneath the surface instead of guessing around): it was apparent that while the remote host was busy it showed complete radio silence. At the same time Wireshark showed retransmission attempts carried out by the local host, indicating that this is behind the issue.
Thus the solution couldn't be implemented on the local host either, the behavior of the remote host needed to be changed.

If no data is received over stream tcpclient close the connection

I would like to know on TcpClient's NetworkStream what exactly happen if timeout occurs.
While debugging the code i found that after request is sent and if no data is received within mention timeout period it throws below exception and unfortunately closes the connection (TcpClient.Connected become false):
Unable to read data from the transport connection: A connection
attempt failed because the connected party did not properly respond
after a period of time, or established connection failed because
connected host has failed to respond.
It throws the exception is okay, but i would like to know how i can prevent it from closing the connection.
It would be great if someone can provide more insights on this.

Have you checked this one? Reconnect TCPClient after interruption I think if you have a long enough TTL of your TCP Connection, should an exception occurs (I believe you would get thrown a SocketException) you can catch that up and initiate your retry logic. There are several implementations for this and obviously that would depend on the use case but normally there is a number of attempts (configuration value) before "giving up" connecting. That way your manager will retry connecting X number of times and will carry on if there is a successful connection otherwise will propagate up in the chain the exception.

Transport errors

I'm working on a crawler and my attempt at fixing an issue with this exception:
System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host at System.Net.Sockets.NetworkStream.BeginRead(Byte[] buffer, Int32 offset, Int32 size, AsyncCallback callback, Object state)
was to implement a retry pattern after using wireshark and looking at network logs I concluded that these errors are most likely transient.
However these exceptions are really bugging me now and I would really like to get to the bottom of why I am getting these errors. Can anyone suggest a good strategy to adopt and tools I can use or reasons you can think of why the connection is being forcibly closed?
Thanks

I see two questions here:
Why is the Exception being thrown?
Why is the connection being forcibly closed?
Why is the Exception being thrown?
This is a problem with the transport implementation you have chosen to consume. Apparently, microsoft decided to communicate the error by wrapping it up in an exception and throw it up the stack. The corresponding source code can be found here: http://referencesource.microsoft.com/#System/net/System/Net/Sockets/NetworkStream.cs,766
In the source code, you can also see that the InnerException is set and contains a localization independent representation of the errorCode.
The bottom line is that this exception being thrown does not mean anything exceptional happened, it can happen just because the connection was dropped.
Which brings us to the next question:
Why is the connection being forcibly closed?
Just as the exceptions message hints, the reason could well be the remote host. Therefore, looking at the remote hosts implementation could be required to get to the bottom of this.
I suspect though, that just judging by the exception, you cannot rule out the reason to be somewhere in between the hosts (sharks have shown an appetite for fiber cables).
I suggest the following experiment:
Set up the two hosts residing on different machines and let them connect through a cable.
While the connection is established, unplug the cable.
This could not disproof but at least proof the possibility.
However, "working on a crawler" suggests that you might encounter a variety of different hosts and it is to be expected that some of them turn taciturn sometimes for whatever reason you would care to imagine.
EDIT:
I remember catching this exception when using TCP over IP when the remote host sent a packet with the RST Flag set. The value of the RST Flag is displayed in Wireshark.
networkstream

My 50 cents: This is the normal behaviour when using networkstreams for reading data from a socket. It is not a user error, the exception thrown just causes the data processing in the reading thread to be interrupted. Just wrap it up with a try/catch-handler accordingly.
You could try to use the DebuggerNonUserCode attribute (https://msdn.microsoft.com/de-de/library/system.diagnostics.debuggernonusercodeattribute%28v=vs.110%29.aspx) to suppress debugger alerts when an exception is triggered. Be aware that this may also "hide" other exceptions...