We Keep Coding

Connection closing while downloading extremely large file

We have a webservice that serves up files. Recently, we have come across a Very Large File - more than 2 GB - that can't be copied into the buffer. I've modified the code to use HttpCompletionOptions.ResponseHeadersRead to not use the buffer and copy directly to a stream. However, most of the time I get
System.IO.IOException: 'Unable to read data from the transport connection: The connection was closed.'
Curl is able to download it without problem. The exception doesn't happen every time, but it's most of the time. I set HttpClient.Timeout to an hour, so that's not the problem. The Exception itself is very ambiguous and I can't find any reason that it would be closing the connection. The logs on the web server also say
info: Microsoft.AspNetCore.Server.Kestrel[34]
Connection id "0HLO4L4D3UAMS", Request id "0HLO4L4D3UAMS:00000001": the application aborted the connection.
so it seems to be something on the client side.
var requestMessage = GetVeryLargeFile(asset, HttpMethod.Get);
using (var result = await _client.SendAsync(requestMessage, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false))
{
result.EnsureSuccessStatusCode();
cancellationToken.ThrowIfCancellationRequested();
using (var stream = await result.Content.ReadAsStreamAsync().ConfigureAwait(false))
{
cancellationToken.ThrowIfCancellationRequested();
using (var fileStream = _fileProvider.Create(filePath))
{
await stream.CopyToAsync(fileStream.StreamInstance).ConfigureAwait(false);
if (fileStream.Length == 0)
{
throw new DataException($"No data retrieved for {asset.Url}");
}
}
}
}
UPDATE:
Based on comments here, I changed the copy line to be synchronous, and that fixed the error. That's certainly less than optimal, but I'm still struggling to figure out why the async will randomly close the connection.
stream.CopyTo(fileStream.StreamInstance);

Could somebody explain how this server program closes it's socket after every connection? And how could I keep the socket open?

So I want to make an TCP connection between 2 UWP apps using streamsockets. I found this example on the microsoft webpage and it works. The problem is that it closes it's sockets after every connection that's been established. I want to understand when it closes(can't find it in the code and that confuses me a bit) and I also want to know how I could keep the connection between server and client open so I don't have to reconnect every time I want to send something.
Example: https://learn.microsoft.com/en-us/windows/uwp/networking/sockets#build-a-basic-tcp-socket-client-and-server
I have looked in the StreamSocket documentation on Windows and can't really find things about closing the socket.
I assume it happens somewhere in this method. It's the server side of the program that is executed when a connection is received.
private async void StreamSocketListener_ConnectionReceived
(Windows.Networking.Sockets.StreamSocketListener sender,
Windows.Networking.Sockets.
StreamSocketListenerConnectionReceivedEventArgs args)
{
string request;
using (var streamReader = new
StreamReader(args.Socket.InputStream.AsStreamForRead()))
{
request = await streamReader.ReadLineAsync();
}
await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () => this.serverListBox.Items.Add(string.Format("server received the request: \"{0}\"", request)));
// Echo the request back as the response.
using (Stream outputStream = args.Socket.OutputStream.AsStreamForWrite())
{
using (var streamWriter = new StreamWriter(outputStream))
{
await streamWriter.WriteLineAsync(request);
await streamWriter.FlushAsync();
}
}
string request;
using (var streamReader = new
StreamReader(args.Socket.InputStream.AsStreamForRead()))
{
request = await streamReader.ReadLineAsync();
}
await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
this.serverListBox.Items.Add(string.Format("server received the
request: \"{0}\"", request)));
// Echo the request back as the response.
using (Stream outputStream =
args.Socket.OutputStream.AsStreamForWrite())
{
using (var streamWriter = new StreamWriter(outputStream))
{
await streamWriter.WriteLineAsync(request);
await streamWriter.FlushAsync();
}
}
await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
this.serverListBox.Items.Add(string.Format("server sent back the
response: \"{0}\"", request)));
sender.Dispose();
await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
this.serverListBox.Items.Add("server closed its socket"));
}
Any help would greatly be appreciated!

I found this example on the microsoft webpage and it works.
Unfortunately, all the Microsoft socket examples are not good examples of how to write socket applications. They are only examples on how to call those APIs. Building a production-quality socket application is non-trivial, and the Microsoft socket examples will mislead you.
For example, this socket server:
Uses Dispatcher to update the UI rather than modern solutions like IProgress<T>.
Reads from its input stream until a newline is found. This is a problem because:
There is no timeout for the request to arrive.
The input buffer grows without bounds.
There's no handling of the half-open scenario.
Most socket examples from Microsoft have the same problems, all of which have to be addressed when writing production-quality socket code. And writing production-quality socket code is much harder than it first appears.
For this reason, I always recommend using an alternative technology (e.g., self-hosted SignalR) if possible.
But to answer your actual question:
I want to understand when it closes
With sockets, there are actually two streams: an input stream and output stream. Both are closed when sender.Dispose(); is called. However, the input stream is also closed when the StreamReader is disposed, and the output stream is also closed when the StreamWriter is disposed. These happen at the end of their using blocks. This is why you cannot read the second message after closing the StreamReader.

TCP server stops responding after a few requests from client [Windows 10 Universal app, C#, XAML]

I tried to make a little UWP card game to learn how to use TCP, but my server always stops responding to the client after a few connections.
The client sends different messages to the server, like "DrawCard;(name of card)". Interestingly, the first 30-40 messages always reach the server without a problem, but after certain types of messages the server just stops accepting any new ones: For instance, I can draw as many cards as I like, but when I play one, the server stops listening.
I have tried to solve this problem for a few hours now, so I hope someone can help me.
When the app starts, this function is called:
public async static Task StartListening()
{
Windows.Networking.Sockets.StreamSocketListener socketListener = new Windows.Networking.Sockets.StreamSocketListener();
socketListener.ConnectionReceived += SocketListener_ConnectionReceived;
await socketListener.BindServiceNameAsync("1337");
//If I leave this line out, it doesn´t even start listening for connections. Any idea why?
await System.Threading.Tasks.Task.Delay(500);
}
This is the event that gets triggered when a connection is received:
public async static void SocketListener_ConnectionReceived(Windows.Networking.Sockets.StreamSocketListener sender,
Windows.Networking.Sockets.StreamSocketListenerConnectionReceivedEventArgs args)
{
Stream inStream = args.Socket.InputStream.AsStreamForRead();
StreamReader reader = new StreamReader(inStream);
string request = await reader.ReadLineAsync();
HandleData(request);
////Send the line back to the remote client.
Stream outStream = args.Socket.OutputStream.AsStreamForWrite();
StreamWriter writer = new StreamWriter(outStream);
await writer.WriteLineAsync(request);
}
HandleData() uses
await CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(CoreDispatcherPriority.Normal,
() =>
{
/...
});
to edit some ObservableCollections that are bound to UI elements.
The client code is:
public async Task SendDataToServer(string data)
{
Windows.Networking.Sockets.StreamSocket socket = new Windows.Networking.Sockets.StreamSocket();
Windows.Networking.HostName serverHost = new Windows.Networking.HostName(hostname);
string serverPort = "1337";
await socket.ConnectAsync(serverHost, serverPort);
//Write data to the server.
Stream streamOut = socket.OutputStream.AsStreamForWrite();
StreamWriter writer = new StreamWriter(streamOut);
await writer.WriteLineAsync(data);
//Read data from the server.
Stream streamIn = socket.InputStream.AsStreamForRead();
StreamReader reader = new StreamReader(streamIn);
string response = await reader.ReadLineAsync();
}
}
In my test scenario, the client app runs on my Windows Mobile phone and the server runs on my PC. The PC doesn´t send any data back to the client apart from returning the received message to make sure it was received correctly.
Thank you very much for any help you can offer. I have tried everything I could find on the Internet, but I didn´t find anything that worked for me.

Thank you very much #Jay Zuo - MSFT!
I looked at the official samples and found out that I missed the following line of code:
CoreApplication.Properties.Add("listener", socketListener);
According to the MS sample, this "save[s] the socket, so subsequent steps can use it."
The app works as intended now. Thank you very much for your help, #Rafael and #Jay Zuo - MSFT !

Closing a 'multipart/x-mixed-replaced' HttpClient connection

I'm unable to close a connection to a 'multipart/x-mixed-replaced' MJPEG video stream. It has an unlimited content length because it's live video so the stream will never finish. Here's a sample of the follow code I use to grab the stream,
var httpClient = new HttpClient();
var response = await httpClient.GetAsync(url, HttpCompletionOption.ResponseHeadersRead
response.EnsureSuccessStatusCode();
var stream = await response.Content.ReadAsStreamAsync();
It's odd, if I go to this url (http://212.42.54.136:8008/mjpg/video.mjpg?camera=1) 15 times in chrome I will receive this error 'The maximum number of clients are already connected'. If I open up Fiddler and choose 'Abort session' I will be able to open another connection.
How do I do 'Abort session' with HttpClient like they do in Fiddler?

You should do a dispose of all the objects, like in:
stream.Dispose();
response.Dispose();
httpClient.Dispose();
Btw. if you are interested there is a new c# library on CodePlex for a MJPEGDecoder running on Windows RT (Winodws 8.1 and Windows Phone). You can find the project here.

Troubles reading from Internet Live Audio Stream via HTTP in Windows Phone 8.1 application

I am experimenting some troubles while developing a Windows phone 8.1 application in C# which attempts to read Internet Audio Streams‏ from different radiostream servers.
Among the methods to set the source of the Background Audio Player I have proved with success the method MediaPlayer.SetUriSource, but unfortunately the media Player experiments missfunctions when playing Audio streams with shoutcast metadata‏.
‎
Especifically the Audio Player eventually stops playing when reading a shoutast stream.‏
‎
For this reason I tried to use the setStreamSource method, by establishing a HTTP conection with the audiostreaming Server and requesting the desired stream to Play, but unfortunately the application can only read the first 64Kbytes of the received audiostream. After that, the thread that manages the stream request dies, without providing errors or debugging Information.
I would like to add that I tried a normal Console Application with a similar code, and it worked fine!
Here some examples of how I implemented the stream request:
var HClient = new HttpClient();
HttpResponseMessage response = await HClient.GetAsync(streamUrl,HttpCompletionOption.ResponseHeadersRead);
_dataStream = await response.Content.ReadAsStreamAsync();
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(streamUrl);
HttpWebResponse response = (HttpWebResponse) await request.GetResponseAsync();
_dataStream = response.GetResponseStream();
HttpClient hcb = new HttpClient();
_dataStream = await hcb.GetStreamAsync(streamUrl);
var request = new HttpRequestMessage(HttpMethod.Get, streamUrl);
HttpResponseMessage response = await hcb.SendAsync(request, HttpCompletionOption.ResponseHeadersRead);
_dataStream = await response.Content.ReadAsStreamAsync();
The result is in all cases the same: I receive a short chunk of audiostream (about 5-8 seconds) which‏ I can Play and hear, and after some seconds the thread stops. I was able to Count the number of Bytes I was able to read from the stream and it is in all cases 65536.