I have a HttpListener that listen for any file requests on the localhost (i.e 192.168.0.10/Foobar.ext) on the specified port (in this case specifically it is a HLS stream using .m3u8 header files and .ts video files. But the system should work for any type of file).
IAsyncResult result = listener.BeginGetContext(new AsyncCallback(HttpRequestListenerCallback), listener);
result.AsyncWaitHandle.WaitOne(1);
When a request is made the callback creates a HttpListenerContext for the requested file (and that file only), and extracts the file name like this:
HttpListenerContext context = listener.EndGetContext(result);
string fileName = context.Request.Url.AbsolutePath.Substring(1);
the context gets added to a dictionary called httpContexts and linked to an int commandSequenceNumber to keep track of requests.
If the Filename is valid a requests gets send to the server to download the file. The file gets downloaded and gets put into a byte array called totalBufferdata. up to here everything works perfectly.
Now I want to write the byte data of the requested (video)file back to the response (HttpListenerContext.Response) of the context on which the file was requested
To do this i use the following piece of code (this happens after the file has been completely downloaded):
HttpListenerResponse response = httpContexts[commandSequenceNumber].Response; //Get the appropriate context from the dictionary
response.ContentLength64 = totalBufferdata.Count;//set the length response body
Stream responseStream = response.OutputStream; //The stream to which the repsonse needs to be written
try
{
currentContext.Response.OutputStream.Write(dataNoResponseHeader, 0, dataNoResponseHeader.Length);
}
catch (IOException e)
{
Debug.LogError("Exception in writing to response::" + e);
Debug.LogError("innerException:: " + e.InnerException);
}
currentContext.Close();//close the request and response stream
This sends the response back over the context of the request (I.e 192.168.0.10/Foobar.ext, over the same port.)
Now this works fine, aslong as there is a fast, reliable internet connection. When the internet connnection is slow, or inconsistent I start getting the exception:
System.IO.IOException: Unable to write data to the transport connection: The socket has been shut down. ---> System.Net.Sockets.SocketException: The socket has been shut down
With the inner exception being:
System.Net.Sockets.SocketException (0x80004005): The socket has been shut down
I've looked up what the HResult of 0x80004005 corrolated to on msdn, but that is just "E_FAIL Unspecified failure", so no luck there.
I've been unable to figure out why it would throw an expection of the socket closing down (and why it happens on the localhost part, but only when connectivity is bad). I make sure all the data needed is in the totalBufferData, so a low internet speed shouldn't influence this, as all data is already downloaded before i write it to the response. I made sure i do not close the context prematurely anywhere in my code either.
So far i've been unsuccesful in finding a way to get to the underlying socket of HttpListener. I also tried casting response.OutputStream to a NetworkStream, and get the socket from the NetworkStream, but that cast is not valid (Which confused me as they're both IO streams?). Thinking it may be a closing issue i also tried
using(Stream testStream = response.OutputStream)
{
testStream.Write(totalBufferdata.ToArray(), 0, totalBufferdata.Count);
}
I feel like this issue is related to a timeout somewhere. But the Listener doesn't hit the default time-out time. According to MSDN all default time-outs should be 2 minutes. To which I get nowhere close. And i think that the exception returned in case of a time-out should be ObjectDisposedException Write(Byte[], Int32, Int32) was called after the stream was closed. as I would imagine timing out would dispose of the connection, instead of crash it? Is that a misunderstanding?
The requests and reponses are done on seperate threads. But a request that gets done while a response is still in progress gets queued up, and waits for the response to finish before it starts a new response.
Is there a way to get and debug the underlying socket to find out why it closes/crashes? Or maybe an alternative approach to requesting a file over localhost, and responding to this that doesn't use HttpListener?
Some additional information: It is for a Unity application (Unity 2019.1) using scripting runtime version .Net 4.x equivalent, with .Net 4.x api compatibility level and IL2CPP scripting backend. But neither of the classes handling the request or response inherit from monobevahiour (which isn't even possible on threads in unity). Building for Android.
The bounty has ended, but I will open a new one fpr whoever has some valuable information!
See this blog posts that show how to stream a video file using aspnet webapi with the PushStreamContent.
https://www.strathweb.com/2013/01/asynchronously-streaming-video-with-asp-net-web-api/
https://www.c-sharpcorner.com/article/asynchronous-videos-live-streaming-with-asp-net-web-apis-2-0/
This sounds like a problem that I have run into before, with the information given that the internet is slow and or unreliable, I would suggest possibly looking to using a UDP socket instead of TCP as they don't throw exceptions when the connection is cut briefly, or if small amounts of data are lost during transmission, see here. The api is very similar, see here. It would probably be a bit cumbersome to reimplement, but I think it'll solve your problem.
My other insight would be that you're try catch block is specifying that it only accepts IOExceptions, even though it's catching a SocketException, most of the time I just use the generic Exception class to avoid trying to determine which exceptions will be thrown from where.
Just change:
catch (IOException e)
to
catch (Exception e)
Both IOException and SocketException inherit from the Exception class, so the rest of the code remains unmodified. This should give you some more problem specific information hopefully.
You may have some problems there.
First would be a TimeOut situation. Is possible that, because you're experiencing some problems on the internet, that the time to between request and response si bigger than the specified (i believe if you don't, it's set by default to 60 seconds).
Another thing would be that the file size may be to big to write completly at one single package response. But that would happen in any request, not only on the "bad" internet connnection moments.
It's also possible that, because the internet connection is unstable, your "server" detects that the "client" disconected (even briefly), therefore closed the socket.
Related
When using Socket for TCP, trying to read data when the connection has been lost will throw an exception.
I'm used to using exceptions to be aware when a TCP operation fails, since you cannot reliably tell a connection is broken except by trying to read/write.
But I just noticed that NetworkStream.DataAvailable simply returns false in this case, and TcpClient.GetStream also returns successfully. I have code like the following (used in a poll-loop) - is there a way to expand this approach to detect failed connections?
public int Read()
{
var stream = tcp.GetStream();
if (!stream.DataAvailable)
return 0;
else
return stream.Read(InBuffer, 0, InBuffer.Length);
}
If not then I would have to call one of the Read methods using a timeout - which makes me wonder what is the point of DataAvailable as it would easily lead to bugs where failed connections are not detected.
No.
See, you get an exception when the connection is broken.
DataAvailable may simply not have data available AT THE MOMENT. Packets take time to travel and packets may get lost. TCP is having a mechnism to resend list packets, but it causes a delay.
GetStream returns the stream. This is your endpoint. It returns it when the connection is available, which makes sense. DataAvailable is there to check whether there is data waiting. Data not waiting is NOT a lost connection - it is data not waiting.
Basically the title... I'd like to have same feedback on weather NamedPipeServerStream object successfully received a value. This is the starting code:
static void Main(string[] args){
Console.WriteLine("Client running!");
NamedPipeClientStream npc = new NamedPipeClientStream("somename");
npc.Connect();
// npc.WriteTimeout = 1000; does not work, says it is not supported for this stream
byte[] message = Encoding.UTF8.GetBytes("Message");
npc.Write(message);
int response = npc.ReadByte();
Console.WriteLine("response; "+response);
}
I've implemented a small echo message from the NamedPipeServerStream on every read. I imagine I could add some async timeout to check if npc.ReadByte(); did return a value in lets say 200ms. Similar to how TCP packets are ACKed.
Is there a better way of inspecting if namedPipeClientStream.Write() was successful?
I'd like to have same feedback on weather NamedPipeServerStream object successfully received a value
The only way to know for sure that the data you sent was received and successfully processed by the client at the remote endpoint, is for your own application protocol to include such acknowledgements.
As a general rule, you can assume that if your send operations are completing successfully, the connection remains viable and the remote endpoint is getting the data. If something happens to the connection, you'll eventually get an error while sending data.
However, this assumption only goes so far. Network I/O is buffered, usually at several levels. Any of your send operations almost certainly involve doing nothing more than placing the data in a local buffer for the network layer. The method call for the operation will return as soon as the data has been buffered, without regard for whether the remote endpoint has received it (and in fact, almost never will have by the time your call returns).
So if and when such a call throws an exception or otherwise reports an error, it's entirely possible that some of the previously sent data has also been lost in transit.
How best to address this possibility depends on what you're trying to do. But in general, you should not worry about it at all. It will typically not matter if a specific transmission has been received. As long as you can continue transmitting without error, the connection is fine, and asking for acknowledgement is just unnecessary overhead.
If you want to handle the case where an error occurs, invalidating the connection, forcing you to retry, and you want to make the broader operation resumable (e.g. you're streaming some data to the remote endpoint and want to ensure all of the data has been received, without having to resend data that has already been received), then you should build into your application protocol the ability to resume, where on reconnecting the remote endpoint reports the number of bytes it's received so far, or the most recent message ID, or whatever it is your application protocol would need to understand where it needs to start sending again.
See also this very closely-related question (arguably maybe even an actual duplicate…though it doesn't mention named pipes specifically, pretty much all network I/O will involve similar issues):
Does TcpClient write method guarantees the data are delivered to server?
There's a good answer there, as well as links to even more useful Q&A in that answer.
Hello and thanks for your help.
This time I would like to ask about TcpClient.
I have a server program and I am writing a client program.
This client uses TcpClient. It starts by creating a new client
clientSocket=new TcpClient();
(By the way, can this cause exceptions? just in case I put it inside a try-catch but I am not sure if that is really necessary)
Anyway, later I enter a loop and inside this loop I connect to the server
clientSocket.Connect("xx.xx.xx.xx",port);
Then I create a NetworkStream with
clientStream=clientSocket.GetStream();
and then start waiting for data from the server through Read. I know this is blocking so I also set a ReadTimeOut (say 1 second)
Anyway, so far so good.
Later if I don't receive anything from the server, I attempt to send something to it. If this keeps happening for say 3 times I want to close the connection and reconnect to the server again
(notice that a whole different problem is when the server somehow is down, cause that causes other kinds of errors in the client-perhaps I will ask about that later)
So, what do I do?
if(clientSocket.Connected)
{
Console.WriteLine("Closing the socket");
clientSocket.Close();
}
I close the socket.
The loop is finished so I go again to the beginning and try to connect to the server.
clientSocket.Connect("xx.xx.xx.xx",port);
However this causes an error(an unhandled exception actually) "Can not access a disposed object"
So my question is How can I close and reconnect to the server again??
Thanks again for any help
A TcpClient instance can only be used to connect once. You can simply instantiate a new TcpClient, rather than trying to re-open a closed one.
As explained in the other answer, a TcpClient object can only be connected once. If you want to reconnect to the server, you have to create a new TcpClient object and call Connect() again.
That said, you have a number of apparent misconceptions in your question:
First and most important, you should not use ReceiveTimeout if you have any intention whatsoever of trying to use the TcpClient object again, e.g. to send some data to the server. Once the timeout period has expired, the underlying socket is no longer usable.If you want to periodically send data to the server when the server hasn't sent data to you, you should use asynchronous I/O (which you should do anyway, in spite of the learning curve) and use a regular timer object to keep track of how long it's been since you received data from the server.
The TcpClient constructor certainly can throw an exception. At the very least, any attempt to new a reference type object could throw OutOfMemoryException, and in the case of TcpClient, it ultimately tries to create a native socket handle, which could also fail.While all I/O objects and methods can throw exceptions, you should only ever catch exceptions that you have a way to handle gracefully. So before you add a try/catch block to your code, decide what it is you want to do in the case of an exception that will ensure that your code doesn't corrupt any data and continues to operate correctly. It is generally not possible to gracefully handle OutOfMemoryException (and impractical to protect all uses of new in any case), but you certainly can catch SocketException, which could be thrown by the constructor. If that exception is thrown, you should immediately abandon the attempt to create and use TcpClient, and report the error the user so that they can attempt to correct whatever problem prevented the socket's creation.
If your server is expected to be sending you data, and you don't receive it, then closing the connection and retrying is unlikely to improve the situation. That will only cause additional load on the server, making it even more likely it will fail to respond. Likewise sending the same data over and over. You should your request once, wait as long as is practical for a response from the server, and if you get no response within the desired time, report the error to the user and let them decide what to do next.Note that in this case, you could use the ReceiveTimeout property, because all you're going to do if you don't get a response in time is abandon the connection, which is fine.
Very simple:
client.Close();
client = new TcpClient();
client.Connect(host, port);
I'm using TcpListener to accept & read from TcpClient.
The problem is that when reading from a TcpClient, TcpClient.BeginRead / TcpClient.EndRead doesn't throw exception when the internet is disconnected. It throws exception only if client's process is ended or connection is closed by server or client.
The system generally has no chance to know that connection is broken. The only reliable way to know this is to attempt to send something. When you do this, the packet is sent, then lost or bounced and your system knows that connection is no longer available, and reports the problem back to you by error code or exception (depending on environment). Reading is usually not enough cause reading only checks the state of input buffer, and doesn't send the packet to the remote side.
As far as I know, low level sockets doesn't notify you in such cases. You should provide your own time out implementation or ping the server periodically.
If you want to know about when the network status changes you can subscribe to the System.Net.NetworkInformation.NetworkChange.NetworkAvailabilityChanged event. This is not specific to the internet, just the local network.
EDIT
Sorry, I misunderstood. The concept of "connected" really doesn't exist the more you think about it. This post does a great job of going into more details about that. There is a Connected property on the TcpClient but MSDN says (emphasis mine):
Because the Connected property only
reflects the state of the connection
as of the most recent operation, you
should attempt to send or receive a
message to determine the current
state. After the message send fails,
this property no longer returns true.
Note that this behavior is by design.
You cannot reliably test the state of
the connection because, in the time
between the test and a send/receive,
the connection could have been lost.
Your code should assume the socket is
connected, and gracefully handle
failed transmissions.
Basically the only way to check for a client connection it to try to send data. If it goes through, you're connected. If it fails, you're not.
I don't think you'd want BeginRead and EndRead throwing exceptions as these should be use in multi threaded scenarios.
You probably need to implement some other mechanism to respond to the dropping of a connection.
I have some fairly simple code that uploads a photo or video to an endpoint (using HTTP PUT or POST). Every so often I see connection closed exceptions thrown, and in reality the photo/video was uploaded just fine, it's calling GetResponse where the exception occurs.
One thing I've noticed is that GetResponse can take an awful long time to process. Often longer than the actual upload time of the photo to the server. My code writes to the web server using RequestStream.Write.
I did a little test and uploaded about 40 photos/videos to the server that range in size from 1MB to 85MB and the time for GetResponse to return was anywhere from 3 to 40 seconds.
My question is, is this normal? Is this just a matter of how long the server I am uploading these files to is taking to process my request and respond? In looking at Fidder HTTP traces it seems to be the case.
FYI, my uploads are HTTP 1.0, Timeout values set to Infinite (both Timeout and ReadWriteTimeout)
If the server is genuinely taking a long time to return any data (as shown in Fiddler) then that's the cause of it. Uploading an 85MB attachment would take a long time to start with, and then the server has to process it. You can't do a lot about that - other than to use an asynchronous method if you're able to get on with more work before the call returns.
It's not entirely clear what Fiddler's showing you though - is it showing a long time before the server sends the response? If so, there's not much you can do. I'm surprised that the connection is being closed on you, admittedly. If, however, you're not seeing your data being written to the server for a while, that's a different matter.
Are you disposing the response returned? If not, you may have connections which are being kept alive. This shouldn't be a problem if it's explicitly HTTP 1.0, but it's the most common cause of "hanging" web calls in my experience.
Basically, if you don't dispose of a WebResponse it will usually (at least with HTTP 1.1 and keepalive) hold on to the connection. There's a limit to the number of connections which can be open to a single host, so you could end up waiting until an earlier response is finalized before the next one can proceed.
If this is the problem, a simple using statement is the answer:
using (WebResponse response = request.GetResponse())
{
...
}
Yes, the response time may be a lot longer than just the upload time. After the request has been sent to the server it has to be processed and a response has to be returned. There may be some time before the request is processed, and then the file typically is going to be saved somewhere. After that the server will create the response page that is sent back.
IIS handles only one request at a time from each user, so if you start another upload before the first one is completed, it will wait until the first one completes before it even starts to process the next.