I have a console program that runs perpetually to process background jobs. Every day, it uploads files via FTP to other servers. After a while it will begin to fail to transfer files with:
SocketException: System.Net.Sockets.SocketException: An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full
Rebooting the server solves the issue but it's annoying. Other answers (like this one) suggest maybe my sockets are not being closed after use but I would expect FtpClient.Dispose() to take care of this, since it does not expose a .Close() method. Here's my code:
using (FtpClient client = new FtpClient())
{
client.Host = "ftp.host.com";
client.Credentials = new System.Net.NetworkCredential(userName: "foo", password: "bar");
client.Connect();
client.UploadFile(localPath: localPath, remotePath: remotePath, overwrite: true);
client.Disconnect();
}
Are there other possible causes for this error that I should investigate?
Related
I'd like to wait for a slow response from a client with TcpClient but get a timeout after about 20s no matter how I configure it. This is my attempt:
using (var client = new TcpClient { ReceiveTimeout = 9999999, SendTimeout = 9999999 })
{
await client.ConnectAsync(ip, port);
using (var stream = client.GetStream())
{
// Some quick read/writes happen here via the stream with stream.Write() and stream.Read(), successfully.
// Now the remote host is calculating something long and will reply if finished. This throws the below exception however instead of waiting for >20s.
var bytesRead = await stream.ReadAsync(new byte[8], 0, 8);
}
}
The exception is an IOException:
Unable to read data from the transport connection: A connection
attempt failed because the connected party did not properly respond
after a period of time, or established connection failed because
connected host has failed to respond.
...which contains a SocketException inside:
A connection attempt failed because the connected party did not
properly respond after a period of time, or established connection
failed because connected host has failed to respond
SocketErrorCode is TimedOut.
The 20s seems to be an OS default on Windows but isn't it possible to override it from managed code by interacting with TcpClient? Or how can I wait for the response otherwise?
I've also tried the old-style BeginRead-EndRead way and the same happens on EndRead. The problem is also not caused by Windows Firewall or Defender.
I'd like to wait for a slow response from a client
It's important to note that it's the connection that is failing. The connection timeout is only for establishing a connection, which should always be very fast. In fact, the OS will accept connections on behalf of an application, so you're literally just talking about a packet round-trip. 21 seconds should be plenty.
Once the connection is established, then you can just remove the ReceiveTimeout/SendTimeout and use asynchronous reads to wait forever.
It turns out that the remote host wasn't responding in a timely manner, hence the problem. Let me elaborate, and though this will be a solution very specific to my case maybe it will be useful for others too.
The real issue wasn't a timeout per se, as the exception indicated, but rather what exceptions thrown on subsequent Read() calls have shown: "An existing connection was forcibly closed by the remote host"
The remote host wasn't purposely closing the connection. Rather what happened is that when it was slow to respond it was actually so busy that it wasn't processing any TCP traffic either. While the local host wasn't explicitly sending anything while waiting for a response this still was an issue: the local host tried to send ACKs for previous transmissions of the remote host. Since these couldn't be delivered the local host determined that the remote host "forcibly closed" the connection.
I got the clue from looking at the traffic with Wireshark (always good to try to look at what's beneath the surface instead of guessing around): it was apparent that while the remote host was busy it showed complete radio silence. At the same time Wireshark showed retransmission attempts carried out by the local host, indicating that this is behind the issue.
Thus the solution couldn't be implemented on the local host either, the behavior of the remote host needed to be changed.
I have a client using NamedPipeClientStream and a server using NamedPipeServerStream.
The client may start before the server, and when it call clientStream.Connect(timeout) i get TimeoutException as expected.
Is there any way I can check if there is a NamedPipeServerStream listening before calling the Connect to prevent an exception?
If someone bumps into this question five years later, this might be helpful:
var isPipeRunning = Directory.GetFiles( #"\\.\pipe\" ).Contains( $"\\.\pipe\{pipeName}" )
I suggest you should use an EventWaitHandle. On all clients, call WaitOne () and on server after opening the stream, call Set ().
So, on "server" side, write this:
EventWaitHandle handle = new EventWaitHandle(false, EventResetMode.ManualReset, String.Format(#"Global\{0}", "SERVER_OPENED_HANDLE"));
OpenStream (); // inside this method you would have code that opens your named pipe for incomming messages
// finally, signal that you are done
handle.Set ();
On client side, write something like this:
EventWaitHandle handle = new EventWaitHandle(false, EventResetMode.ManualReset, String.Format(#"Global\{0}", "SERVER_OPENED_HANDLE"));
// here your thread will sleep until the server calls "Set"
handle.WaitOne ();
// then you can safelly connect to the server here
ConnectToServer ();
There remains just a few situations to handle:
1) The pipe can't be opened on server becase there is already an opened pipe with the same name (will throw an exception).
2) You successfully opened the pipe, you notified clients that you are ready but after that, a milisecond after, the server crushes from some unexpected reason and the client cannot reach the server.
3) User rights problems
In all these situations, you should handle these exceptions using a try / catch and normally, if all goes well, this code will ensure that the client will NOT try to connect before the server successfully opened the pipe.
Anyway, I suggest using a more advanced technique for making IPC through Named Pipes such as using WCF or even .NET Remoting wich besides the fact that it's somehow considered obsolete by many (don't include me) is very decent at least for IPC communication. This will give you the freedom and scalability (maybe one day you will need your clients to be able to reside on other machines, and you will have to switch from IPC to a LAN communication or even in WAN / internet).
There is no way to check this only using a NamedPipeClientStream. However, you can use a Mutex like so
// In the server
var mutex = new System.Threading.Mutex(false, "MyPipeMutex");
OpenPipeAndRunServer();
mutex.Close();
// In the client process
var mutex = new System.Threading.Mutex(false, "MyPipeMutex");
if (!mutex.WaitOne(0, false))
{
OpenPipe();
}
mutex.Close();
You will probably want to wrap the Close calls in a try-finally block to make sure it always closes. In the client you can use a different timeout to actually wait for the NamedPipe to be opened.
You can also catch the exception as a work around.
I have made written a c# program which is successfully connecting to the remote host with and without proxy. We work in two different networks, home and office networks, which uses proxy. Here is the code snippet.
while(true) {
Thread.sleep(5000);
using (var client = new WebClient()) {
client.Headers[HttpRequestHeader.Accept] = "application/json";
client.Headers[HttpRequestHeader.ContentType] = "application/json";
string result = client.UploadString(Event.GetInsertURL(), "POST", json);
if (result.Contains("SUCCESS")) {
// Console.WriteLine("SUCCESS");
}
}
}
The above code runs in a loop to keep making the request to the same api. It is working in both the networks if the program is started in those networks. However, if I start the program in home and hibernate or sleep the computer and restart it in office, I'm getting the following exception.
System.Net.WebException: Unable to connect to the remote server ---> System.Net.Sockets.SocketException: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 74.125.130.141:443
at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
The reason is that the connection made for first time is reused in subsequent requests. Is there any way for me to force creation of connection when I get this exception?
P.S:
Code of Event.GetInsertURL()
public static string GetInsertURL(){
return "https://my-app.appspot.com/_ah/api/"eventendpoint/v1/insertEvents";
}
The code already creates a new client session for each connection in new WebClient().
It may be that the sleep occurs during the course of one of the sessions and then triggers the fault.
In general, any of the network methods could throw under unexpected conditions. The only real solution is to wrap the code in a try/catch block and retry under the correct conditions a few times before reporting a permanent failure.
Based on experience more than knowledge:
use HttpWebRequest instead of webClient , it's just more robust.
my examples are too messy but that's the base:
var httpWebRequest = (HttpWebRequest)System.Net.WebRequest.Create(URI);
I have a program that run in a loop, each iteration runs in a different thread and I'm creating new process that open new service host:
ServiceHost _host = new ServiceHost(_service, new Uri("net.pipe://localhost/" + i_PipeName));
_host.AddServiceEndpoint(typeof(ICommandService), new NetNamedPipeBinding() { TransferMode = TransferMode.Buffered }, i_PipeName);
_host.Open();
from my main program I connect to the open .net pipe at the following way:
ICommandService ServiceProxy = ChannelFactory<ICommandService>.CreateChannel
(new NetNamedPipeBinding(), new EndpointAddress(#"net.pipe://localhost/" + i_PipeName" + #"/" + i_PipeName));
So my problem is that for the first 200+ proccess/iterations it working fine, I can open connection and pass messages but later on there errors that starts to appear:
There was no endpoint listening at
net.pipe://localhost/pipea0360/pipea0360 that could accept the
message. This is often caused by an incorrect address or SOAP action.
See InnerException if present, for more details.
My question is if there are any limitations on the number of pipes I can open in parallel?
Is this because I open so many proccesses?
Have you ruled out a race condition, where the client is attempting to connect to a pipe that the server hasn't established yet? If you have a lot of active threads on the server, it could easily delay the thread that is supposed to start listening. That could explain why it works in the beginning.
In general, having a server thread for each client doesn't scale well; you'll get better performance with a thread pool approach.
Error:
Unable to read data from the transport connection: A blocking operation was interrupted by a call to WSACancelBlockingCall
Situation
There is a TCP Server
My web application connects to this TCP Server
Using the below code:
TcpClientInfo = new TcpClient();
_result = TcpClientInfo.BeginConnect(<serverAddress>,<portNumber>, null, null);
bool success = _result.AsyncWaitHandle.WaitOne(20000, true);
if (!success)
{
TcpClientInfo.Close();
throw new Exception("Connection Timeout: Failed to establish connection.");
}
NetworkStreamInfo = TcpClientInfo.GetStream();
NetworkStreamInfo.ReadTimeout = 20000;
2 Users use the same application from two different location to access information from this server at the SAME TIME
Server takes around 2sec to reply
Both Connect
But One of the user gets above error
"Unable to read data from the transport connection: A blocking operation was interrupted by a call to WSACancelBlockingCall"
when trying to read data from stream
How can I resolve this issue?
Use a better way of connecting to the server
Can't because it's a server issue
if a server issue, how should the server handle request to avoid this problem
This looks Windows-specific to me, which isn't my strong point, but...
You don't show us the server code, only the client code. I can only assume, then, that your server code accepts a socket connection, does its magic, sends something back, and closes the client connection. If this is your case, then that's the problem.
The accept() call is a blocking one that waits for the next client connection attempt and binds to it. There may be a queue of connection attempts created and administered by the OS, but it can still only accept one connection at a time.
If you want to be able to handle multiple simultaneous requests, you have to change your server to call accept(), and when a new connection comes in, launch a worker thread/process to handle the request and go back to the top of the loop where the accept() is. So the main loop hands off the actual work to another thread/process so it can get back to the business of waiting for the next connection attempt.
Real server applications are more complex than this. They launch a bunch of "worker bee" threads/processes in a pool and reuse them for future requests. Web servers do this, for instance.
If my assumptions about your server code are wrong, please enlighten us as to what it looks like.
Just a thought.
If your server takes 2seconds to response, shouldn't the Timeout values be 2000, instead of 20000 (which is 20 seconds)? First argument for AsyncWaitHandle.WaitOne() is in milliseconds.
If you are waiting 20 seconds, may be your server is disconnecting you for being idle?