We Keep Coding

Check if NamedPipeClientStream write is successful

Basically the title... I'd like to have same feedback on weather NamedPipeServerStream object successfully received a value. This is the starting code:
static void Main(string[] args){
Console.WriteLine("Client running!");
NamedPipeClientStream npc = new NamedPipeClientStream("somename");
npc.Connect();
// npc.WriteTimeout = 1000; does not work, says it is not supported for this stream
byte[] message = Encoding.UTF8.GetBytes("Message");
npc.Write(message);
int response = npc.ReadByte();
Console.WriteLine("response; "+response);
}
I've implemented a small echo message from the NamedPipeServerStream on every read. I imagine I could add some async timeout to check if npc.ReadByte(); did return a value in lets say 200ms. Similar to how TCP packets are ACKed.
Is there a better way of inspecting if namedPipeClientStream.Write() was successful?

I'd like to have same feedback on weather NamedPipeServerStream object successfully received a value
The only way to know for sure that the data you sent was received and successfully processed by the client at the remote endpoint, is for your own application protocol to include such acknowledgements.
As a general rule, you can assume that if your send operations are completing successfully, the connection remains viable and the remote endpoint is getting the data. If something happens to the connection, you'll eventually get an error while sending data.
However, this assumption only goes so far. Network I/O is buffered, usually at several levels. Any of your send operations almost certainly involve doing nothing more than placing the data in a local buffer for the network layer. The method call for the operation will return as soon as the data has been buffered, without regard for whether the remote endpoint has received it (and in fact, almost never will have by the time your call returns).
So if and when such a call throws an exception or otherwise reports an error, it's entirely possible that some of the previously sent data has also been lost in transit.
How best to address this possibility depends on what you're trying to do. But in general, you should not worry about it at all. It will typically not matter if a specific transmission has been received. As long as you can continue transmitting without error, the connection is fine, and asking for acknowledgement is just unnecessary overhead.
If you want to handle the case where an error occurs, invalidating the connection, forcing you to retry, and you want to make the broader operation resumable (e.g. you're streaming some data to the remote endpoint and want to ensure all of the data has been received, without having to resend data that has already been received), then you should build into your application protocol the ability to resume, where on reconnecting the remote endpoint reports the number of bytes it's received so far, or the most recent message ID, or whatever it is your application protocol would need to understand where it needs to start sending again.
See also this very closely-related question (arguably maybe even an actual duplicate…though it doesn't mention named pipes specifically, pretty much all network I/O will involve similar issues):
Does TcpClient write method guarantees the data are delivered to server?
There's a good answer there, as well as links to even more useful Q&A in that answer.

HttpContext.Response underlying socket shuts down unexpectedly

I have a HttpListener that listen for any file requests on the localhost (i.e 192.168.0.10/Foobar.ext) on the specified port (in this case specifically it is a HLS stream using .m3u8 header files and .ts video files. But the system should work for any type of file).
IAsyncResult result = listener.BeginGetContext(new AsyncCallback(HttpRequestListenerCallback), listener);
result.AsyncWaitHandle.WaitOne(1);
When a request is made the callback creates a HttpListenerContext for the requested file (and that file only), and extracts the file name like this:
HttpListenerContext context = listener.EndGetContext(result);
string fileName = context.Request.Url.AbsolutePath.Substring(1);
the context gets added to a dictionary called httpContexts and linked to an int commandSequenceNumber to keep track of requests.
If the Filename is valid a requests gets send to the server to download the file. The file gets downloaded and gets put into a byte array called totalBufferdata. up to here everything works perfectly.
Now I want to write the byte data of the requested (video)file back to the response (HttpListenerContext.Response) of the context on which the file was requested
To do this i use the following piece of code (this happens after the file has been completely downloaded):
HttpListenerResponse response = httpContexts[commandSequenceNumber].Response; //Get the appropriate context from the dictionary
response.ContentLength64 = totalBufferdata.Count;//set the length response body
Stream responseStream = response.OutputStream; //The stream to which the repsonse needs to be written
try
{
currentContext.Response.OutputStream.Write(dataNoResponseHeader, 0, dataNoResponseHeader.Length);
}
catch (IOException e)
{
Debug.LogError("Exception in writing to response::" + e);
Debug.LogError("innerException:: " + e.InnerException);
}
currentContext.Close();//close the request and response stream
This sends the response back over the context of the request (I.e 192.168.0.10/Foobar.ext, over the same port.)
Now this works fine, aslong as there is a fast, reliable internet connection. When the internet connnection is slow, or inconsistent I start getting the exception:
System.IO.IOException: Unable to write data to the transport connection: The socket has been shut down. ---> System.Net.Sockets.SocketException: The socket has been shut down
With the inner exception being:
System.Net.Sockets.SocketException (0x80004005): The socket has been shut down
I've looked up what the HResult of 0x80004005 corrolated to on msdn, but that is just "E_FAIL Unspecified failure", so no luck there.
I've been unable to figure out why it would throw an expection of the socket closing down (and why it happens on the localhost part, but only when connectivity is bad). I make sure all the data needed is in the totalBufferData, so a low internet speed shouldn't influence this, as all data is already downloaded before i write it to the response. I made sure i do not close the context prematurely anywhere in my code either.
So far i've been unsuccesful in finding a way to get to the underlying socket of HttpListener. I also tried casting response.OutputStream to a NetworkStream, and get the socket from the NetworkStream, but that cast is not valid (Which confused me as they're both IO streams?). Thinking it may be a closing issue i also tried
using(Stream testStream = response.OutputStream)
{
testStream.Write(totalBufferdata.ToArray(), 0, totalBufferdata.Count);
}
I feel like this issue is related to a timeout somewhere. But the Listener doesn't hit the default time-out time. According to MSDN all default time-outs should be 2 minutes. To which I get nowhere close. And i think that the exception returned in case of a time-out should be ObjectDisposedException Write(Byte[], Int32, Int32) was called after the stream was closed. as I would imagine timing out would dispose of the connection, instead of crash it? Is that a misunderstanding?
The requests and reponses are done on seperate threads. But a request that gets done while a response is still in progress gets queued up, and waits for the response to finish before it starts a new response.
Is there a way to get and debug the underlying socket to find out why it closes/crashes? Or maybe an alternative approach to requesting a file over localhost, and responding to this that doesn't use HttpListener?
Some additional information: It is for a Unity application (Unity 2019.1) using scripting runtime version .Net 4.x equivalent, with .Net 4.x api compatibility level and IL2CPP scripting backend. But neither of the classes handling the request or response inherit from monobevahiour (which isn't even possible on threads in unity). Building for Android.
The bounty has ended, but I will open a new one fpr whoever has some valuable information!

See this blog posts that show how to stream a video file using aspnet webapi with the PushStreamContent.
https://www.strathweb.com/2013/01/asynchronously-streaming-video-with-asp-net-web-api/
https://www.c-sharpcorner.com/article/asynchronous-videos-live-streaming-with-asp-net-web-apis-2-0/

This sounds like a problem that I have run into before, with the information given that the internet is slow and or unreliable, I would suggest possibly looking to using a UDP socket instead of TCP as they don't throw exceptions when the connection is cut briefly, or if small amounts of data are lost during transmission, see here. The api is very similar, see here. It would probably be a bit cumbersome to reimplement, but I think it'll solve your problem.
My other insight would be that you're try catch block is specifying that it only accepts IOExceptions, even though it's catching a SocketException, most of the time I just use the generic Exception class to avoid trying to determine which exceptions will be thrown from where.
Just change:
catch (IOException e)
to
catch (Exception e)
Both IOException and SocketException inherit from the Exception class, so the rest of the code remains unmodified. This should give you some more problem specific information hopefully.

You may have some problems there.
First would be a TimeOut situation. Is possible that, because you're experiencing some problems on the internet, that the time to between request and response si bigger than the specified (i believe if you don't, it's set by default to 60 seconds).
Another thing would be that the file size may be to big to write completly at one single package response. But that would happen in any request, not only on the "bad" internet connnection moments.
It's also possible that, because the internet connection is unstable, your "server" detects that the "client" disconected (even briefly), therefore closed the socket.

Self-healing SslStream

I'm writing a service that needs to maintain a long running SSL connection to a remote server. I need this server to be self-healing, that is if it's disconnected for any reason then the next time it's written to it will reconnect. I've tried this:
bool isConnected = client.Connected && client.Client.Poll(0, SelectMode.SelectWrite) && stream.CanWrite;
if (!isConnected )
{
this.connected = false;
GetConnection();
}
stream.Write(bytes, 0, bytes.Length);
stream.Flush();
But I find it doesn't act as I would expect it. If I simulate a network outage by disabling my wifi, I'm still able to write to the stream with stream.Write() for approximately 20 seconds. Then next time I try to write to it, none of client.Connected, client.Client.Poll(), or stream.CanWrite() return false, but when I go to write to the stream I get a socket exception. Finally, if I try to recreate the connection, I get this exception: An existing connection was forcibly closed by the remote host.
I would appreciate any help create a long running SslStream that can withstand network failure. Thanks!

From a 10.000 feet point of view:
The reason you can still write to the stream after shutting down your wifi is because there is a network buffer that is holding the data for transmission, stream.Write/stream.Flush success means the network interface (TCP/IP stack) has accepted the data and has been buffered for transmission, not that the data has reach its target.
It takes time to the TCP/IP Stack to notice a full media disconnection, (connection lost/reset) because even if there is no physical link TCP/IP will see this as a temporary issue in the network and will keep retrying for a while (the network could be dropping packets at some point and the stack will keep retrying)
If you think about this in the reverse way, you won't like all your programs to fail if there is a network hiccup (this happen too often on internet), so TCP/IP takes its time to notify to the app layer that the connection has become invalid (after retry several times and wait a reasonable amount of time)
You can always reconnect to the server when the SslStream fails and continue sending data, although you will find is not as easy as this because there are several scenarios where you send and data is not received by server and others where server receive the data and you do not receive any ACK from server at all... So depending on your needs, self-healing alone could be not enough.
Self-Healing is simple to implement, data consistency and reliability is harder and usually requires the server to be ready to support some kind of reliable messaging mechanism to ensure all data has been sent and received.

The underlying protocol for SSL is TCP. TCP will usually only send data if the application wants it to deliver data, or if it needs to reply to data received from the other side by sending an ACK. This means, that a broken connection like a lost link will not be noticed until you are trying to send any data. But you will not notice immediatly, because:
A write to the socket will only deliver the data to the OS kernel and return success if this delivery was successful.
The kernel will then try to deliver the data to the peer and will wait for the ACK from the client.
If it does not get any ACK it will retry again to deliver the data and only after some unsuccessful retries the kernel will declare the connection broken.
Only after the connection is marked broken by the kernel the next write or read will return the error from kernel to user space, like with returning EPIPE when doing a write.
This means, if you want to know up-front if the connection is still alive you have to make sure that you get a regular data exchange on the connection. At the TCP level you might set TCP_KEEPALIVE, but this might use an interval of some hours between exchanges packets. At the SSL layer you might try to use the infamous heartbeat extension, but most peers will not understand it. The last choice is to implement some kind of heartbeat in your own application.
As for the self healing: When reconnecting you get a new TCP connection and you also need to do a full SSL handshake, because the last SSL connection was not cleanly closed and thus cannot be resumed. The server has no idea that this new connection is just a continuation of the old one so you have to implement some kind of meta-connection spanning multiple TCP connections inside your application layer on both client and server. Inside this meta-connection you need to have your own data tracking to detect, which data are really accepted from the peer and which were only send but never explicitly accepted because the connection broke. Sound like a kind of TCP on top of TCP.

Most accurate way to redirect HttpRequests

I'm working on a solution to redirect http requests from my browser to another pc.
In a nutshell: Is there a more accurate way to send an incoming HttpListenerRequest as an HttpRequest to a client?
Background: The browser sends http requests to 127.0.0.1:9666 (yes, click'n'load), which should be answered by a download program.
In my case, the download program isn't running on the machine which runs the browser.
Port Forwarding tools like these don't work for me: http://www.quantumg.net/portforward.php
So I decided to write a specific click'n'load redirector in C#.
My problem is to redirect the browser's request. ATM, I listen on 127.0.0.1:9666, send some information to a client, the client redirects this to the downloader program, which answers. (long term short: this is redirected to the browser).
But using pre-defined cases to redirect only the core-information doesn't seem quite right for me. It should be possible to redirect the whole request. So I don't need to differ between various cases, and this should be more accurate.
Is there any other way to redirect this request, except from copying all headers and properties?
The Server uses an HttpListener to listen for the browser's requests.
Then, the server sends an HttpReqest to the Client.
The Client uses an HttpListener to listen for the server's requests.
The client sends an HttpRequest to the Downloader,
uses GetResponse, and sends this as a response to the server, which responds to the browser.
I've looked up some functions in the MSDN, but didn't find a good way to "copy a whole request".
So here is my problem code:
Thread.Sleep(1500);
//Read client's/JDownloader's stream, send to browser/redirector
do
{
Read = Outstream.Read(Buffer, 0, Buffer.Length);
Totalr += Read;
Instream.Write(Buffer, 0, Read);
Sent += Read;
// Bufferstr += ASCIIEncoding.ASCII.GetString(Buffer, 0, Read); debugging
} while (Read != 0);
Without the Thread.Sleep, the first read returns 171 bytes were read (just and exactly the HTTP header, buffer-length is 1024bytes).
second iteration: when executing Outstream.Read, nothing happens. No matter how long I wait. It seems, the reader is waiting for traffic to receive, but there is no traffic to receive (weird...) When the thread sleeps for 500-1500ms, the first read returns 351bytes were read (the complete http request), and then again, second iteration, nothing.
This happens when reading from the browser's or JDownloader's NetworkStream. They never return 0. A dirty method to get this working is to replace the while-argument with Read == Buffer.Length, dirty because it will fail when exactly Buffer.Length bytes are received (endless waiting again, yay).
The DataAvailable property also dosn't always seem to be right, sometimes it is set to false, when the program didn't even read something from the stream, but there were bytes to receive.(also weird...)
Any other ideas for proper receiving loops?
Nobody? short summary of my problem: neither the browser's request stream, nor JDownloaders response stream return 0. When attempting another read, the program just waits for more bytes to receive.
So I don't know a failsafe method to read the entire stream.
Just repeating until the amount of bytes read is lower than the buffer's length will result in previously mentioned endless wait for bytes, when streamlength % bufferlength == 0.
Also, it seems JDownloader needs more time to generate an answer and write it into the stream than my program attempts to read the stream. So I will only receive a part of the stream. Another read attempt will result in, hooray, an endless wait for bytes.
Is there another way to receive the whole stream without a static delay (via thread sleep)?

Can you just open a TcpListener and forward the raw bytes? That way you do not need to concern yourself with HTTP at all.

What you ask for is basically a proxy implementation - building a well working proxy is no simple task as you will have to understand and handle HTTP etc. in both directions.
I would recommend to either use an existing library for that OR some configurable proxy:
http://www.mentalis.org/soft/projects/proxy/ (with source)
http://fusion.corsis.eu/ (with source)
http://www.wingate.com/
http://www.squid-cache.org/

Socket Server Disconnect and Reconnect Buffer Error?

I'm using the following TCP Sockets Server/Client example: http://www.codeguru.com/Csharp/Csharp/cs_network/sockets/article.php/c8781/
I'm also using the following CryptoStream example:
http://www.obviex.com/samples/Encryption.aspx
Problem: Both Server and Clients communicate perfectly until I Stop the Server socket, wait a minute or so, then start the Server socket. Sometimes but not always I recieve a base64 error in the Encryption.aspx on line 'Convert.FromBase64String(cipherText);'...
I know there is incorrect / corrupted data in the buffer probably left-over from stopping the socket. Then the new data comes in and the error occurs.
Q. Will clearing the 'class SocketPacket; solve this issue?
Q. How do I clear the 'class SocketPacketsocketBuffer'?
Other suggestions are greatly appreciated..

One of the things you may find is happening is that one of the sockets is not being closed down properly. The thing with sockets is that you need to make sure they get correctly closed or you set the server socket to reuse the address.
Try tcpview from sysinternals to view the status of sockets. You can also use netstat to view the status of the sockets.