I'm using WebSockets to stream values from server to client. The connection should be closed when the stream of values is completed (server-side termination), or when the client stops subscribing (client-side termination).
How can the client gracefully close the connection?
A rough sample to demonstrate the issue in AspNetCore; server pushes a (potentially) infinite stream of values; client subscribes to the first 5 values, and should then close the connection.
app.Use(async (context, next) =>
{
if (context.Request.Path == "/read")
{
var client = new ClientWebSocket();
await client.ConnectAsync(new Uri(#"wss://localhost:7125/write"), CancellationToken.None);
for (int i = 0; i < 5; i++)
await client.ReceiveAsync(new ArraySegment<byte>(new byte[sizeof(int)]), CancellationToken.None);
// This does not seem to have any particular effect on writer
// await client.CloseOutputAsync(WebSocketCloseStatus.NormalClosure, string.Empty, CancellationToken.None);
// This freezes in AspNetcore on .NET6 core (because the implementation waits for the connection to close, which never happens)
// In AspNet (non-core, .Net Framework 4.8), this seems to throw an exception that data cannot be read after the connection has been closed (i.e. the socket seems to only be closeable if no data is pending to be read)
await client.CloseAsync(WebSocketCloseStatus.NormalClosure, string.Empty, CancellationToken.None);
}
if (context.Request.Path == "/write")
{
var ws = await context.WebSockets.AcceptWebSocketAsync();
await foreach (var number in GetNumbers())
{
var bytes = BitConverter.GetBytes(number);
if (ws.State != WebSocketState.Open)
throw new Exception("I wish we'd hit this branch, but we never do!");
await ws.SendAsync(new ArraySegment<byte>(bytes), WebSocketMessageType.Binary, true, CancellationToken.None);
}
}
});
static async IAsyncEnumerable<int> GetNumbers()
{
for (int i = 0; i <= int.MaxValue; i++)
{
yield return i;
await Task.Delay(25);
}
}
The general issue seems to be that the close message isn't picked up by the /write method, i.e. ws.State remains WebSocketState.Open. I'm assuming that onle receive operations update the connection status?
Is there any good way to handle this situation / for the server to pick up the client's request to close the connection?
I would quite like to avoid the client having to send any explicit messages to the server / for the server to have to read the stream explicitly. I'm increasingly wondering if that is possible, though.
Way the WebSocket protocol works is similar to TCP - via connection establishment, the only difference - initiation is done via http[s].
One send action from one side matches one receive action from another, and vice versa.
You can notice this detail (if i am not mistaken) in remarks of documentation:
Exactly one send and one receive is supported on each WebSocket object in parallel.
So, you should receive at least one data segment and recognize CloseIntention message from client. And the same on client side.
How to receive message, recognize close intention, and properly react to it - see here.
How to send close intention message - see here.
Suspect you should call webSocket.ReceiveAsync at least once in background on your server.
Then, in ContinueWith task call CancellationTokenSource.Cancel() for current server socket session.
That repo is working, except of docker-compose. - I am kinda newcomer in complex DevOps things. )
UPDATE
Remarks part of docs is not about matching of send-receive actions on different sides of conversation. Just wanted you to notice how this TCP-concept works, i.e you should receive data at least once.
Related
We have a webservice that serves up files. Recently, we have come across a Very Large File - more than 2 GB - that can't be copied into the buffer. I've modified the code to use HttpCompletionOptions.ResponseHeadersRead to not use the buffer and copy directly to a stream. However, most of the time I get
System.IO.IOException: 'Unable to read data from the transport connection: The connection was closed.'
Curl is able to download it without problem. The exception doesn't happen every time, but it's most of the time. I set HttpClient.Timeout to an hour, so that's not the problem. The Exception itself is very ambiguous and I can't find any reason that it would be closing the connection. The logs on the web server also say
info: Microsoft.AspNetCore.Server.Kestrel[34]
Connection id "0HLO4L4D3UAMS", Request id "0HLO4L4D3UAMS:00000001": the application aborted the connection.
so it seems to be something on the client side.
var requestMessage = GetVeryLargeFile(asset, HttpMethod.Get);
using (var result = await _client.SendAsync(requestMessage, HttpCompletionOption.ResponseHeadersRead, cancellationToken).ConfigureAwait(false))
{
result.EnsureSuccessStatusCode();
cancellationToken.ThrowIfCancellationRequested();
using (var stream = await result.Content.ReadAsStreamAsync().ConfigureAwait(false))
{
cancellationToken.ThrowIfCancellationRequested();
using (var fileStream = _fileProvider.Create(filePath))
{
await stream.CopyToAsync(fileStream.StreamInstance).ConfigureAwait(false);
if (fileStream.Length == 0)
{
throw new DataException($"No data retrieved for {asset.Url}");
}
}
}
}
UPDATE:
Based on comments here, I changed the copy line to be synchronous, and that fixed the error. That's certainly less than optimal, but I'm still struggling to figure out why the async will randomly close the connection.
stream.CopyTo(fileStream.StreamInstance);
So I want to make an TCP connection between 2 UWP apps using streamsockets. I found this example on the microsoft webpage and it works. The problem is that it closes it's sockets after every connection that's been established. I want to understand when it closes(can't find it in the code and that confuses me a bit) and I also want to know how I could keep the connection between server and client open so I don't have to reconnect every time I want to send something.
Example: https://learn.microsoft.com/en-us/windows/uwp/networking/sockets#build-a-basic-tcp-socket-client-and-server
I have looked in the StreamSocket documentation on Windows and can't really find things about closing the socket.
I assume it happens somewhere in this method. It's the server side of the program that is executed when a connection is received.
private async void StreamSocketListener_ConnectionReceived
(Windows.Networking.Sockets.StreamSocketListener sender,
Windows.Networking.Sockets.
StreamSocketListenerConnectionReceivedEventArgs args)
{
string request;
using (var streamReader = new
StreamReader(args.Socket.InputStream.AsStreamForRead()))
{
request = await streamReader.ReadLineAsync();
}
await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () => this.serverListBox.Items.Add(string.Format("server received the request: \"{0}\"", request)));
// Echo the request back as the response.
using (Stream outputStream = args.Socket.OutputStream.AsStreamForWrite())
{
using (var streamWriter = new StreamWriter(outputStream))
{
await streamWriter.WriteLineAsync(request);
await streamWriter.FlushAsync();
}
}
string request;
using (var streamReader = new
StreamReader(args.Socket.InputStream.AsStreamForRead()))
{
request = await streamReader.ReadLineAsync();
}
await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
this.serverListBox.Items.Add(string.Format("server received the
request: \"{0}\"", request)));
// Echo the request back as the response.
using (Stream outputStream =
args.Socket.OutputStream.AsStreamForWrite())
{
using (var streamWriter = new StreamWriter(outputStream))
{
await streamWriter.WriteLineAsync(request);
await streamWriter.FlushAsync();
}
}
await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
this.serverListBox.Items.Add(string.Format("server sent back the
response: \"{0}\"", request)));
sender.Dispose();
await this.Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () =>
this.serverListBox.Items.Add("server closed its socket"));
}
Any help would greatly be appreciated!
I found this example on the microsoft webpage and it works.
Unfortunately, all the Microsoft socket examples are not good examples of how to write socket applications. They are only examples on how to call those APIs. Building a production-quality socket application is non-trivial, and the Microsoft socket examples will mislead you.
For example, this socket server:
Uses Dispatcher to update the UI rather than modern solutions like IProgress<T>.
Reads from its input stream until a newline is found. This is a problem because:
There is no timeout for the request to arrive.
The input buffer grows without bounds.
There's no handling of the half-open scenario.
Most socket examples from Microsoft have the same problems, all of which have to be addressed when writing production-quality socket code. And writing production-quality socket code is much harder than it first appears.
For this reason, I always recommend using an alternative technology (e.g., self-hosted SignalR) if possible.
But to answer your actual question:
I want to understand when it closes
With sockets, there are actually two streams: an input stream and output stream. Both are closed when sender.Dispose(); is called. However, the input stream is also closed when the StreamReader is disposed, and the output stream is also closed when the StreamWriter is disposed. These happen at the end of their using blocks. This is why you cannot read the second message after closing the StreamReader.
Is there a way, using the Microsoft.Azure.ServiceBus package, to wait on your current thread to receive a message from a queue?
This may more be a problem with my understanding and a desire to use the technology in a way it is not intended to be used, but what I would like to do is combine the send and receive examples from the following Microsoft example so that you can send message(s) off to various queues, and be able to listen in and handle "replies" (just messages that you're listening to on a queue) and close the connection when you are done receiving messages.
Some pseudo-code here:
// send message(s) that will be consumed by other processes / applications, and by doing so later on we will expect some messages back
await SendMessagesAsync(numberOfMessages);
var receivedMessages = 0;
while (receivedMessages < numberOfMessages)
{
// there is no "ReceiveAsync" method, this is what I would be looking for
Message message = await queueClient.ReceiveAsync(TimeSpan.FromSeconds(30));
receivedMessages++;
// do something with the message here
}
await queueClient.CloseAsync();
Is this possible or am I "doing it wrong"?
In the new library ReceiveAsync method is available on MessageReceiver class:
var messageReceiver = new MessageReceiver(SBConnString, QueueName, ReceiveMode.PeekLock);
Message message = await messageReceiver.ReceiveAsync();
See a full example at Get started sending and receiving messages from Service Bus queues using MessageSender and MessageReceiver.
In Microsoft.Azure.ServiceBus library, there is no such a thing calledReceiveAsync. In this, you can process or receive the message by using RegisterOnMessageHandlerAndReceiveMessages(). With this you can receive the message with an event. With this RegisterOnMessageHandlerAndReceiveMessages() like queueClient.RegisterMessageHandler(ReceiveOrProcessMessagesAsync, messageHandlerOptions); and you have to seprately create this event for receiveMessages, in our case it is ReceiveOrProcessMessagesAsync
static async Task ReceiveOrProcessMessagesAsync(Message message, CancellationToken token)
{
// Process the message
Console.WriteLine($"Received message: SequenceNumber:{message.SystemProperties.SequenceNumber} Body:{Encoding.UTF8.GetString(message.Body)}");
// Complete the message so that it is not received again.
// This can be done only if the queueClient is created in ReceiveMode.PeekLock mode (which is default).
await queueClient.CompleteAsync(message.SystemProperties.LockToken);
// Note: Use the cancellationToken passed as necessary to determine if the queueClient has already been closed.
// If queueClient has already been Closed, you may chose to not call CompleteAsync() or AbandonAsync() etc. calls
// to avoid unnecessary exceptions.
}
and you refer the below link for know about Microsoft.Azure.ServiceBus
https://learn.microsoft.com/en-us/azure/service-bus-messaging/service-bus-dotnet-get-started-with-queues
I have a windows service that uses a websocket (from http://sta.github.io/websocket-sharp/) to conenct to Slack and monitor messages.
My connection code looks something like this:
ws = new WebSocket(connection.Url);
ws.OnMessage += async (sender, e) =>
{
var msg = JsonConvert.DeserializeObject<MessageFromSlack>(e.Data);
if (msg.Type == "message" && msg.Text != null && msg.User != UserId)
{
if (userMatcher.IsMatch(msg.Text))
{
await ProcessDirectMessage(msg);
}
await ProcessMessage(msg);
}
if (msg.Type == "channel_joined")
{
await ChannelJoined(msg.ChannelModel.Id);
}
};
ws.OnClose += (sender, e) =>
{
var reason = e.Reason;
var code = e.Code;
System.Diagnostics.Debug.WriteLine($"{code}:{reason}");
};
ws.Connect();
Basically it waits for a message and then if it's directed # my bot, it'll call ProcessDirectMessage and if not it'll call ProcessMessage. The details of those functions are, I think, unimportant (they do some matching looking for key phrases and respond by sending a message back).
This all works fine. For a while. But after some period of time (usually more than a day), it just stops responding altogether. My OnMessage handler never gets hit. I thought that maybe what is happening is the websocket is getting closed on the server side, so I added the OnClose handler, but that never seems to get hit either.
Does anybody have an idea what might be happening here? Is there a way to keep the connection alive, or else reconnect it when it dies?
By the nature of TCP connection - the only reliable way to detect its gone is to write something to it. If you are just reading (waiting for data to arrive) - you can do that for a very long time while the other side is long time dead. That happens if that other side did not close connection gracefully (which involves an exchange of some TCP packets).
Web socket protocol defines special Ping frame, and corresponding Pong frame, which you should use to avoid situation described in the question. From time to time you should send Ping frame and wait (for a certain timeout) for server to respond with Pong frame. If it did not respond in given timeout - assume connection is dead and reconnect.
As far as I know - library you use does not automatically send ping requests on your behalf. However, it allows you to do that via Ping method.
You need to configure timeout with
ws.WaitTime = TimeSpan.FromSeconds(5);
And then, from time to time (for example - when you did not receive any new messages in last X seconds), do:
bool isAlive = ws.Ping();
There is also boolean property which does the same:
bool isAlive = ws.IsAlive;
This is a blocking call (both of the above). It will return true if server replied with Pong during ws.WaitTime interval, and false otherwise. Then you can act accordingly.
I am experimenting in creating a small IRC server to learn some new programming concepts (and others I haven't used in forever). The first step is to get a basic client connecting via TCP to send plaintext commands to the server.
To listen for connection I have the following code:
public NetworkClient(Server server, TcpClient socket, int id)
{
_socket = socket;
_id = id;
_server = server;
}
private async void ListenForClients()
{
int numClients = 0;
while (IsRunning)
{
var tcpClient = await _listener.AcceptTcpClientAsync();
var netClient = new NetworkClient(this, tcpClient, numClients);
netClient.Start();
Console.WriteLine("Client Connected");
numClients++;
}
}
Then in my NetworkClient class my Start() method looks like:
public async void Start()
{
using (var reader = new StreamReader(_socket.GetStream()))
{
while (_server.IsRunning)
{
var line = await reader.ReadLineAsync();
Console.WriteLine("Client {0} wrote: {1}", _id, line);
}
}
}
This works well while a telnet client is connected, however once I close my telnet client reader.ReadLineAsync(); constantly returns null. I would add a check to see if line == null but I'm not sure that is the correct way to detect if a client has disconnected.
To make matters worse, _socket.Connected is constantly returning true all while nulls are getting "received" by reader.ReadLineAsync().
What is the proper way to detect when tcp clients have been disconnected?
A read on a TCP/IP socket will return 0 bytes when the connection has been gracefully closed. This situation causes ReadLineAsync to return null. So, yes, you should check for null and treat it as a graceful socket closure.
Sockets may be closed other ways, too; any socket operation may throw an exception if the socket is abortively closed. If the exception happens at an acceptable part of the protocol (where a close is not considered an error), then you should treat that exception as though it were a graceful closure as well.
Oh, and TcpClient.Connected (like Socket.Connected) is practically useless; it only tells you whether the socket was connected, not whether it is connected. Just pretend that property doesn't exist.
Finally, a couple of notes:
Avoid async void. If your methods return Task, then you have a "handle" to see when they complete (and whether they raised exceptions). My recent MSDN article explains why async void is not recommended.
It's best to periodically send data over the connection to determine if it is still viable. I wrote a TCP/IP .NET sockets FAQ that covers this in more detail.