We Keep Coding

Restricting SendAsync calls to 5 messages per second

I'm implementing Binance's API.
The documentation says:
WebSocket connections have a limit of 5 incoming messages per second. A message is considered:
A PING frame
A PONG frame
A JSON controlled message (e.g. subscribe, unsubscribe)
For ex. there is a simple web socket wrapper such as the one from the official Binance Connector. According to the limitation above, SendAsync should be restricted 5 messages per second. If a few threads call SendAsync 5 times at the same time (including PING frame which is built-in the ClientWebSocket class), it's going to fail. How can I solve the issue with that limitation gracefully? Using bounded channels is a solution?
public class BinanceWebSocket : IDisposable
{
private IBinanceWebSocketHandler handler;
private List<Func<string, Task>> onMessageReceivedFunctions;
private List<CancellationTokenRegistration> onMessageReceivedCancellationTokenRegistrations;
private CancellationTokenSource loopCancellationTokenSource;
private Uri url;
private int receiveBufferSize;
public BinanceWebSocket(IBinanceWebSocketHandler handler, string url, int receiveBufferSize = 8192)
{
this.handler = handler;
this.url = new Uri(url);
this.receiveBufferSize = receiveBufferSize;
this.onMessageReceivedFunctions = new List<Func<string, Task>>();
this.onMessageReceivedCancellationTokenRegistrations = new List<CancellationTokenRegistration>();
}
public async Task ConnectAsync(CancellationToken cancellationToken)
{
if (this.handler.State != WebSocketState.Open)
{
this.loopCancellationTokenSource = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken);
await this.handler.ConnectAsync(this.url, cancellationToken);
await Task.Factory.StartNew(() => this.ReceiveLoop(loopCancellationTokenSource.Token, this.receiveBufferSize), loopCancellationTokenSource.Token, TaskCreationOptions.LongRunning, TaskScheduler.Default);
}
}
public async Task DisconnectAsync(CancellationToken cancellationToken)
{
if (this.loopCancellationTokenSource != null)
{
this.loopCancellationTokenSource.Cancel();
}
if (this.handler.State == WebSocketState.Open)
{
await this.handler.CloseOutputAsync(WebSocketCloseStatus.NormalClosure, null, cancellationToken);
await this.handler.CloseAsync(WebSocketCloseStatus.NormalClosure, null, cancellationToken);
}
}
public void OnMessageReceived(Func<string, Task> onMessageReceived, CancellationToken cancellationToken)
{
this.onMessageReceivedFunctions.Add(onMessageReceived);
if (cancellationToken != CancellationToken.None)
{
var reg = cancellationToken.Register(() =>
this.onMessageReceivedFunctions.Remove(onMessageReceived));
this.onMessageReceivedCancellationTokenRegistrations.Add(reg);
}
}
public async Task SendAsync(string message, CancellationToken cancellationToken)
{
byte[] byteArray = Encoding.ASCII.GetBytes(message);
await this.handler.SendAsync(new ArraySegment<byte>(byteArray), WebSocketMessageType.Text, true, cancellationToken);
}
public void Dispose()
{
this.DisconnectAsync(CancellationToken.None).Wait();
this.handler.Dispose();
this.onMessageReceivedCancellationTokenRegistrations.ForEach(ct => ct.Dispose());
this.loopCancellationTokenSource.Dispose();
}
private async Task ReceiveLoop(CancellationToken cancellationToken, int receiveBufferSize = 8192)
{
WebSocketReceiveResult receiveResult = null;
try
{
while (!cancellationToken.IsCancellationRequested)
{
var buffer = new ArraySegment<byte>(new byte[receiveBufferSize]);
receiveResult = await this.handler.ReceiveAsync(buffer, cancellationToken);
if (receiveResult.MessageType == WebSocketMessageType.Close)
{
break;
}
string content = Encoding.UTF8.GetString(buffer.ToArray());
this.onMessageReceivedFunctions.ForEach(omrf => omrf(content));
}
}
catch (TaskCanceledException)
{
await this.DisconnectAsync(CancellationToken.None);
}
}
}
Second way which I'm not 100% sure it solves it
SendAsync is being called in a loop using Channels. SingleReader is set to true, which means there will be only one consumer at a time. It technically should solve the issue, but I'm not 100% sure because the channel might only be limiting the amount in the buffer.
private readonly Channel<string> _messagesTextToSendQueue = Channel.CreateUnbounded<string>(new UnboundedChannelOptions()
{
SingleReader = true,
SingleWriter = false
});
public ValueTask SendAsync(string message)
{
Validations.Validations.ValidateInput(message, nameof(message));
return _messagesTextToSendQueue.Writer.WriteAsync(message);
}
public void Send(string message)
{
Validations.Validations.ValidateInput(message, nameof(message));
_messagesTextToSendQueue.Writer.TryWrite(message);
}
private async Task SendTextFromQueue()
{
try
{
while (await _messagesTextToSendQueue.Reader.WaitToReadAsync())
{
while (_messagesTextToSendQueue.Reader.TryRead(out var message))
{
try
{
await SendInternalSynchronized(message).ConfigureAwait(false);
}
catch (Exception e)
{
Logger.Error(e, L($"Failed to send text message: '{message}'. Error: {e.Message}"));
}
}
}
}
catch (TaskCanceledException)
{
// task was canceled, ignore
}
catch (OperationCanceledException)
{
// operation was canceled, ignore
}
catch (Exception e)
{
if (_cancellationTotal.IsCancellationRequested || _disposing)
{
// disposing/canceling, do nothing and exit
return;
}
Logger.Trace(L($"Sending text thread failed, error: {e.Message}. Creating a new sending thread."));
StartBackgroundThreadForSendingText();
}
}

I would try to keep it as simple as possible and use Semaphore Slim to achieve this, I have created a class to perform this task.
public class ThrottlingLimiter
{
private readonly SemaphoreSlim _semaphore;
private readonly TimeSpan _timeUnit;
public ThrottlingLimiter(int maxActionsPerTimeUnit, TimeSpan timeUnit)
{
if (maxActionsPerTimeUnit < 1)
throw new ArgumentOutOfRangeException(nameof(maxActionsPerTimeUnit));
if (timeUnit < TimeSpan.Zero || timeUnit.TotalMilliseconds > int.MaxValue)
throw new ArgumentOutOfRangeException(nameof(timeUnit));
_semaphore = new SemaphoreSlim(maxActionsPerTimeUnit, maxActionsPerTimeUnit);
_timeUnit = timeUnit;
}
public async Task WaitAsync(CancellationToken cancellationToken = default)
{
await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false);
ScheduleSemaphoreRelease();
}
private async void ScheduleSemaphoreRelease()
{
await Task.Delay(_timeUnit).ConfigureAwait(false);
_semaphore.Release();
}
}
Now to Use this class, all you have to do is set your limit and timeSpan
public async Task SendData(List<string> allMessages)
{
// Limiting 5 calls per second
ThrottlingLimiter throttlingLimiter = new ThrottlingLimiter(5, TimeSpan.FromSeconds(1));
await Task.WhenAll(allMessages.Select(async message =>
{
await throttlingLimiter.WaitAsync();
try {
await SendInternalSynchronized(message);
// I am not sure what this SendInternalSynchronized returns but I would return some thing to keep a track if this call is successful or not
}
catch (Exception e)
{
Logger.Error(e, L($"Failed to send text message: {message}'. Error: {e.Message}"));
}
});
}
so basically what will happen here is, no matter how big your list is, the ThrottlingLimiter will only send 5 messages per second and wait for the next second to send the next 5 messages.
so, in your case, get all the data from your call to
await _messagesTextToSendQueue.Reader.WaitToReadAsync();
store that into a list or any collection and pass that to the SendData function.

gRPC Keep stream alive

I am experimenting with gRPC for long-lived streaming session as I need to guarantee message ordering from server to client.
I have the following .proto:
service Subscriber {
rpc Subscribe(SubscriptionRequest) returns (stream SubscriberEvent);
}
My current service (hosted in ASP.NET / .NET 5.0) looks like this:
public class SubscriberService : Subscriber.SubscriberBase
{
private readonly ILogger<SubscriberService> _logger;
private readonly ConcurrentDictionary<string, IServerStreamWriter<SubscriberEvent>> _subscriptions = new();
private int _messageCount = 0;
private Timer _timer;
public SubscriberService(ILogger<SubscriberService> logger)
{
_logger = logger;
_timer = new Timer(o => TimerCallback(), null, TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(1));
}
private void TimerCallback()
{
Broadcast($"Current time is {DateTime.UtcNow}");
}
public override Task Subscribe(SubscriptionRequest request, IServerStreamWriter<SubscriberEvent> responseStream, ServerCallContext context)
{
_subscriptions.TryAdd(request.ClientId, responseStream);
return responseStream.WriteAsync(new SubscriberEvent() {Id = 0, Message = "Subscribe successful"});
}
public void Broadcast(string message)
{
var count = ++_messageCount;
foreach (var sub in _subscriptions.Values)
{
sub.WriteAsync(new SubscriberEvent() { Id = count, Message = message });
}
_logger.LogInformation($"Broadcast message #{count}: {message}");
}
}
My client only receives the initial 'Subscribe Successful' message, but never those triggered by the timer. Not do I get any exceptions when calling WriteAsync.
Am I trying to use gRPC for something it was never designed to do (a SignalR/WebSocket substitute), or am I merely missing something obvious?

For a long-running gRPC streaming, you have to wait for a client to say the connection is closed. Something like this:
while (!context.CancellationToken.IsCancellationRequested)
{
// event-based action
responseStream.WriteAsync(new SubscriberEvent() {Id = 0, Message = "Subscribe successful"});
}

I think the previous answer is doing busy-wait. So I want to show you an async version of it.
public override Task Subscribe(RequestMessage, IServerStreamWriter<ReplyMessage> responseStream, ServerCallContext context)
{
// your event-based code here
var tcs = new TaskCompletionSource();
context.CancellationToken.Register(() => tcs.TrySetCanceled(), false);
return tcs.Task;
}
BTW, I think I have been doing a project just like yours. And I have used Observables, Subjects, and ReplaySubjects from Rx.Net. These are very helpful for event-based code.

How to consume multiple message types through one queue with C# and RabbitMQ

currently I try to implement a MessagingClient, working with RabbitMQ and C#.
My idea is that every service uses its own queue for receiving messages and there is an own exchange for each message type. For example a message type could be "download/requested" or "download/started". These exchanges are of type "fanout". Now the clients, that want to listen on some message, bind their queue to the corresponding exchange. For example the download service binds its queue "download-bot" to the exchange "download/requested".
My problem is now that I can't really imagine, how to deal with different types of messages on the parsing site. I use Newtonsoft.Json for encoding/decoding the message objects. For each different message type, there is separate "handler" I want to execute. This handler should get the message (deserialized) as a parameter. My problem is, that with mulitple message types over one queue, there seems to be just one handler qer queue. How do I find out, which concrete handler to execute and how to parse the message, if I don't know the type at compile time? I know an important keyword will be "reflection", but I could not tinker something working together. I should say that i'm pretty new with C#. Could someone provide a working example of this ? Or is this even something, I should do like that?
As a workarround, currently I got the following (reduced) example (using one queue per message type and service)
namespace Lib.Message {
public class RabbitMqMessageClient : IMessageClient {
private static NLog.ILogger logger = Common.Logger.GetLogger ();
private RabbitMqConfig config;
private IModel model;
private const string DOWNLOAD_REQUESTED = "download/requested";
public RabbitMqMessageClient (RabbitMqConfig config) {
this.config = config;
var factory = new ConnectionFactory () { HostName = config.Hostname };
var connection = factory.CreateConnection ();
this.model = connection.CreateModel ();
}
public Task PublishDownloadRequested (string userRef, string id, string url) {
var message = new DownloadRequestMessage (userRef, id, url);
return this.publish (DOWNLOAD_REQUESTED, message);
}
public Task SubscribeToDownloadRequested (Action<DownloadRequestMessage> action) {
return this.subscribe<DownloadRequestMessage> (DOWNLOAD_REQUESTED, action);
}
public Task SubscribeToDownloadRequested (Func<DownloadRequestMessage, Task> action) {
return this.subscribe<DownloadRequestMessage> (DOWNLOAD_REQUESTED, action);
}
public Task Start () {
return Task.CompletedTask;
}
private Task subscribe<TPayloadType> (string topic, Action<TPayloadType> action) {
Func<TPayloadType, Task> wrappedAction = (TPayloadType args) => {
action (args);
return Task.CompletedTask;
};
return this.subscribe<TPayloadType> (topic, wrappedAction);
}
private Task subscribe<TPayloadType> (string topic, Func<TPayloadType, Task> action) {
var queue = $"{config.QueueName}--{topic}";
model.ExchangeDeclare (topic, ExchangeType.Fanout, true);
model.QueueDeclare (
queue,
durable : true,
exclusive : false,
autoDelete : false,
arguments : null);
model.QueueBind (config.QueueName, topic, "foo");
var consumer = new EventingBasicConsumer (model);
// this consumer will be shared for multiple queues right ?
// maybe even not here in this method, but in the "Start" method
consumer.Received += async (model, ea) => {
logger.Info ($"handling {ea.Exchange}");
var jsonString = Encoding.UTF8.GetString (ea.Body.Span.ToArray ());
// here I need to know how to deserialize the payload from the value of "ea.Exchange"
var message = JsonConvert.DeserializeObject<TPayloadType>(jsonString);
// I think i have to put the action in a map (?) instead and then find the concrete handler by the exchange name?
await action(message);
};
model.BasicConsume (config.QueueName, true, consumer);
return Task.CompletedTask;
}
private Task publish (string topic, object payload) {
logger.Info ($"publishing {topic}");
string message = JsonConvert.SerializeObject (payload);
var bytes = Encoding.UTF8.GetBytes (message);
model.BasicPublish (
exchange: topic,
routingKey: "foo",
basicProperties : null,
body : bytes
);
return Task.CompletedTask;
}
}
}

Async wait for multiple threads to finish

I have a code block which is eventually accessed by multiple threads. I search for an up to date async mechanism to continue executing when all threads have passed.
Currently I do the following with a CountDownEvent which works just fine (without async support).
public class Watcher
{
private static readonly Logger Log = LogManager.GetCurrentClassLogger();
private readonly CountdownEvent _isUpdating = new CountdownEvent(1);
private readonly IActivity _activity;
public Watcher([NotNull] IActivity activity)
{
_activity = activity ?? throw new ArgumentNullException(nameof(activity));
_activity.Received += OnReceived;
}
private void OnReceived(IReadOnlyCollection<Summary> summaries)
{
_isUpdating.AddCount();
try
{
// Threads processing
}
finally
{
_isUpdating.Signal();
}
}
private void Disable()
{
_activity.Received -= OnReceived;
_isUpdating.Signal();
/* await */ _isUpdating.Wait();
}
}
Do I need to use any of those AsyncCountdownEvent implementations or is there any other built-in mechanism? I already thought about using a BufferBlock because it has async functionality but I think it's a bit overkill.
Additional to the comments:
IActivity is a WebService call (but shouldn't effect the implementation on top or vice versa)
public async Task Start(bool alwayRetry = true, CancellationToken cancellationToken = new CancellationToken())
{
var milliseconds = ReloadSeconds * 1000;
do
{
try
{
var summaries = await PublicAPI.GetSummariesAsync(cancellationToken).ConfigureAwait(false);
OnSummariesReceived(summaries);
}
catch (Exception ex)
{
Log.Error(ex.Message);
OnErrorOccurred(ex);
}
await Task.Delay(milliseconds, cancellationToken).ConfigureAwait(false);
// ReSharper disable once LoopVariableIsNeverChangedInsideLoop
} while (alwayRetry);
}

It's not clear the IActivity signatures; but you can wait for a range of tasks to be completed:
class MultiAsyncTest {
Task SomeAsync1() { return Task.Delay(1000); }
Task SomeAsync2() { return Task.Delay(2000);}
Task EntryPointAsync() {
var tasks = new List<Task>();
tasks.Add(SomeAsync1());
tasks.Add(SomeAsync2());
return Task.WhenAll(tasks);
}
}
What's IActivity's signature? Does it support Task? Or you are using Thread? More explanation would help to a more specified answer.

Is CancellationTokenSource shutting down correctly

Here's the entry point:
public class Program
{
private static readonly CancellationTokenSource TokenSource = new CancellationTokenSource();
public static void Main(string[] args)
{
// start the app
new Bootstrap()
.RunAsync(TokenSource.Token)
.Wait();
Console.CancelKeyPress += (sender, eventArgs) =>
{
TokenSource.CancelAfter(500);
};
}
}
Here's the bootstrap:
public class Bootstrap : IBootstrap
{
private readonly IServer server;
private readonly ILogger logger;
public Bootstrap(
IServer server,
ILogger logger)
{
this.server = server;
this.logger = logger;
}
public async Task RunAsync(CancellationToken token)
{
this.logger.Info("Application Starting...");
await this.server.StartAsync(token);
}
}
Here's the server:
public abstract class BaseServer : IServer
{
private readonly IPAddress ipAddress;
private readonly int port;
private readonly ILogger logger;
protected BaseServer(
string ipAddress,
int port,
ILogger logger)
{
this.ipAddress = IPAddress.Parse(ipAddress);
this.port = port;
this.logger = logger;
}
public async Task StartAsync(CancellationToken token)
{
this.logger.Debug("[{0}] Listening for connections using: {1}:{2}", this.GetType().Name, this.ipAddress.ToString(), this.port);
var tcpListener = new TcpListener(this.ipAddress, this.port);
tcpListener.Start();
while (!token.IsCancellationRequested)
{
await this.ServerProcessorAsync(tcpListener, token);
}
tcpListener.Stop();
Console.WriteLine("Stopped Listener");
}
public abstract Task ServerProcessorAsync(TcpListener listener, CancellationToken token);
}
Here's the Server Processor:
public class Server : BaseServer
{
private readonly ILogger logger;
public Server(
IAppConfiguration configuration,
ILogger logger)
: base(configuration.IpAddress, configuration.Port, logger)
{
this.logger = logger;
}
public override async Task ServerProcessorAsync(TcpListener listener, CancellationToken token)
{
this.logger.Debug("[{0}] Waiting for connection...", this.GetType().Name);
var client = await listener.AcceptTcpClientAsync();
this.logger.Debug("[{0}] Spawning Thread for Connection...", this.GetType().Name);
Parallel.Invoke(new ParallelOptions
{
CancellationToken = token,
MaxDegreeOfParallelism = 10000,
TaskScheduler = TaskScheduler.Current
}, () => this.ListenToClient(client));
}
private void ListenToClient(TcpClient client)
{
var threadName = Thread.CurrentThread.Name;
var bytes = new byte[2048];
var stream = client.GetStream();
int i;
while ((i = stream.Read(bytes, 0, bytes.Length)) != 0)
{
var timeString = DateTime.Now.ToLongTimeString();
var currentRes = Encoding.UTF8.GetString(bytes);
var received = $"Recieved [{threadName}] [{timeString}]: {currentRes}";
this.logger.Info(received);
var responseData = Encoding.UTF8.GetBytes(received);
stream.Write(responseData, 0, responseData.Length);
}
client.Close();
}
}
Will this correctly shut the app down when ctrl+c is pressed?
Is there a way to debug this, or to know that the resources have been released properly.
I assume that the while (!token.IsCancellationRequested) will break when ctrl+c. I also assume that that if there are any threads created by Parallel.Invoke they will disposed of when the Cancel is called.
If in the case that:
Console.CancelKeyPress += (sender, eventArgs) =>
{
TokenSource.CancelAfter(500);
};
doesn't wait for things to be cleared up, is there a better way than a timeout to make sure that everything is properly cleared up?

First, you Wait on RunAsync before subscribing to Console.CancelKeyPress event, so you will subscribe to it when it's too late.
Second, cancellation token won't work anyway in your case. This line:
var client = await listener.AcceptTcpClientAsync();
Will block until new client connects, and because AcceptTcpClientAsync does not has overload which accepts CancellationToken - usage of CancellationTokens in the whole program becomes not needed. What you should do instead is stop your listener instead of cancellation. This will throw exception on the line above, which you should catch and gracefully end the task.
If you really want to continue with CancellationToken, even if it's not really needed here, consider this approach to make it work with AcceptTcpClientAsync: https://stackoverflow.com/a/14524565/5311735. This might also be good idea if you use CancellationToken to cancel many different operations not shown in your question.