I'm trying my hands on NetMQ (3.3.3.4) and creating a pub-sub pattern.
I want a host/server to listen to all incoming data on one port (9000) and forward the data to all clients/subscribers on another port (9001).
The clients will then send data on 9000 and receive all messages sent (by whomever) on 9001.
Following the documentation I created something like the code below, but I can't get it to work. Mainly, I believe, because ReceiveReady is never called!
How I believe it should work:
client.Publish should cause the first line in host.SubscriberSocket_ReceiveReady to unblock and pass the data along to the other socket
When data has been passed along it should appear in the infinite running Task in the client
Results:
Breakpoints on // This line is never reached are never reached
There are no exceptions anywhere.
Switching the ports on the host so that publish = 9000 and subscribe = 9001 has no effect
Windows Firewall is turned off, so there should not be any blocking
It makes no difference if I'm putting the address into PublisherSocket constructor, or if I'm using _publisherSocket.Bind(address) in Host or _publisherSocket.Connect(address) in Client
What am I doing wrong?
Host
public class MyNetMQHost {
private NetMQSocket _publishSocket;
private NetMQSocket _subscribeSocket;
private NetMQPoller _poller;
public MyNetMQHost(string publishAddress = "#tcp://localhost:9001", string subscribeAddress = "#tcp://localhost:9000") {
Task.Factory.StartNew(() => {
using (_publishSocket = new PublisherSocket(publishAddress))
using (_subscribeSocket = new SubscriberSocket(subscribeAddress))
using (_poller = new NetMQPoller { _publishSocket, _subscribeSocket }) {
_subscriberSocket.ReceiveReady += SubscriberSocket_ReceiveReady;
_poller.Run();
}
});
}
private void SubscriberSocket_ReceiveReady(object sender, NetMQSocketEventArgs e) {
var data = e.Socket.ReceiveMultipartBytes(); // This line is never reached
_publishSocket.SendMultipartBytes(data);
}
}
Client
public class MyNetMQClient {
private readonly NetMQSocket _publishSocket;
private readonly NetMQSocket _subscribeSocket;
public MyNetMQClient(string publishAddress = ">tcp://localhost:9000", string subscribeAddress = ">tcp://localhost:9001") {
_publishSocket = new PublisherSocket(publishAddress);
_subscribeSocket = new SubscriberSocket(subscribeAddress);
Task.Factory.StartNew(() => {
while (true) {
byte[] frameBytes = _subscribeSocket.ReceiveFrameBytes();
int one = 1; // This line is never reached
}
});
}
public void Publish(byte[] data) {
_publishSocket.SendFrame(data);
}
}
Tester
public class Tester {
public void MyTester() {
MyNetMQHost host = new MyNetMQHost();
MyNetMQClient client = new MyNetMQClient();
client.Publish(Encoding.Unicode.GetBytes("Hello world!");
}
}
Both your broker and client never call suscribe.
On the broker call suscriber.Subscribe("") to subscribe for all. On your client subscribe to what ever you want.
In your broker you should actually use XSubscriber and XPublisher to move susvriptions around. That way you dont need the subscribe all. You can use Proxy class for that.
Related
I am trying to create a simple application that will publish some messages to a topic with MQTT (library I am using is M2Mqtt.net) and then I want to subscribe to the topic once the messages have already been sent and then have them all be received and then discarded, because they have been received.
I am using mosquitto 2.0.12 as the broker
This is the publisher:
public class MessagePublisher : IMessagePublisher
{
private readonly MqttClient _client;
public MessagePublisher()
{
_client = new MqttClient("localhost");
// clean session needs to be set to false so that it retains all the missed messages, not just the last one
_client.Connect(Guid.NewGuid().ToString(), "username", "password", false, byte.MaxValue);
}
public void Publish(string topic, string message, bool retain = false)
{
Console.Write($"Sent: {topic}, {message}");
_client.Publish(topic, Encoding.UTF8.GetBytes(message), MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE, retain);
Total.SentAndReceived.Add(message, null);
}
}
This is the listener:
public class MessageReceiver : IMessageReceiver
{
private readonly MqttClient _client;
public MessageReceiver()
{
_client = new MqttClient("localhost");
}
public void Subscribe(params string[] topics)
{
_client.Subscribe(topics, new[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });
_client.MqttMsgPublishReceived += client_receivedMessage;
}
public void Connect()
{
// clean session needs to be set to false so that it retains all the missed messages, not just the last one
_client.Connect(Guid.NewGuid().ToString(), "username", "password", false, byte.MaxValue);
}
public void Disconnect()
{
_client.Disconnect();
}
static void client_receivedMessage(object sender, MqttMsgPublishEventArgs e)
{
var message = Encoding.Default.GetString(e.Message);
Console.WriteLine($"Message Received: {message}");
if (Total.SentAndReceived.ContainsKey(message))
Total.SentAndReceived[message] = message;
}
}
And this is the main application:
public static class Program
{
public static void Main(string[] args)
{
var messageReceiver = new MessageReceiver();
var publisher = new MessagePublisher();
for (var i = 1; i <= 10000; i++)
{
publisher.Publish("Devices/", i.ToString(), true);
}
messageReceiver.Subscribe("Devices/");
messageReceiver.Connect();
Thread.Sleep(5000);
var b = Total.SentAndReceived.Where(x => x.Value == null);
Console.WriteLine($"{b.Count()} Missed Messages");
}
}
The problem I am having is that there are missed messages. And the number of missed messages always changes when I run the application. And it's not that last n messages being missed it's the first n messages.
I hope that if I was to build a service that would listen to the published messages. If the services stops for any reason. Once the service comes back online, the messages sent in that downtime would be received.
I think you have a misunderstanding around some terms here.
First, MQTT does not generally queue messages. The only time the broker will queue messages is if the receiving client has already already been connected and subscribed to the topic at QOS > 0. If that client then disconnects before the publisher sends the messages the broker will queue the messages. They will then only be sent to the receiving client if they then reconnect with the same client id and have the clean session flag set to false. This is the only way that messages will be queued up.
Since you appear to be using randomly generated client ids (Guid.NewGuid().ToString()) this will not work. You also appear to be trying to subscribe before you connect, again that won't work.
Secondly, retained messages have nothing to do with message queuing as described above. A message is retained if the retained flag is set at the point of publishing. The broker will then store that specific message and deliver it ever time a client subscribes to the matching topic. This message will be sent before any other messages on the topic. If another message with the retained flag is published it will replace the previous message, there can only be 1 retained message per topic.
Up till now for the past 3 months, I still have 0 clue how SignalR works at the JIT (Just-in-time) level. I'm trying to build a Hub that sends data to the client just in time, and the client will then receive the data and work along with it.
EDIT: Incase you have no idea what I mean by JIT Sending and
Receiving,
I meant it by the server being able to send connected socket clients data when there is new data available. The socket connection will only be closed either when the server is shutdown/has an issue OR the client disconnects from the socket. So in short, no matter what, when new data arises from the server, it will always send that data ONE BY ONE to connection clients.
So here's what I'm missing out/confused about:
Is the SubscribeToAll (Check out TickerHub.cs below) Method the place where I call when I have new data to notify and beep to the clients or where is it?
I know how the asynchronous WriteToChannel works. Basically it sends a collection, item by item to the client. Key issue is, how do I convert this entire function to JIT? And where do I handle the list of clients subscribed to this hub?
Currently, TickerHub.cs keeps retrieving a dataset (named CurrencyPairs) and then broadcasts it to the clients indefinitely. I have a background service that syncs and updates the CurrencyPairs 24/7. I just need a SignalR expert's help to explain/show how I can invoke the Hub from the background service and then allow the hub to broadcast that new data to the connected clients.
TickerHub.cs
public class TickerHub : Hub, ITickerHubClient
{
private IEnumerable<CurrencyPair> _currencyPairs;
private readonly ICurrencyPairService _cpService;
public TickerHub(ICurrencyPairService cpService)
{
_cpService = cpService;
}
public async Task<NozomiResult<CurrencyPair>> Tickers(IEnumerable<CurrencyPair> currencyPairs = null)
{
var nozRes = new NozomiResult<CurrencyPair>()
{
Success = true,
ResultType = NozomiResultType.Success,
Data = currencyPairs
};
return nozRes;
}
// We can use this to return a payload
public async Task<ChannelReader<NozomiResult<CurrencyPair>>> SubscribeToAll()
{
// Initialize an unbounded channel
//
// Unbounded Channels have no boundaries, allowing the server/client to transmit
// limitless amounts of payload. Bounded channels have limits and will tend to
// drop the clients after awhile.
var channel = Channel.CreateUnbounded<NozomiResult<CurrencyPair>>();
_ = WriteToChannel(channel.Writer); // Write all Currency Pairs to the channel
// Return the reader
return channel.Reader;
// This is a nested method, allowing us to write repeated methods
// with the same semantic conventions while maintaining conformity.
async Task WriteToChannel(ChannelWriter<NozomiResult<CurrencyPair>> writer)
{
// Pull in the latest data
_currencyPairs = _cpService.GetAllActive();
// Iterate them currency pairs
foreach (var cPair in _currencyPairs)
{
// Write one by one, and the client receives them one by one as well
await writer.WriteAsync(new NozomiResult<CurrencyPair>()
{
Success = (cPair != null),
ResultType = (cPair != null) ? NozomiResultType.Success : NozomiResultType.Failed,
Data = new[] {cPair}
});
}
// Beep the client, telling them you're done
writer.Complete();
}
}
}
In case you want to find out if my client sided code doesn't work well, here it is
using Microsoft.AspNetCore.SignalR.Client;
using Newtonsoft.Json;
using Nozomi.Client.Data.Interfaces;
using Nozomi.Data;
using Nozomi.Data.CurrencyModels;
using System;
using System.Collections.Generic;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
namespace Nozomi.Client
{
public class NozomiClient
{
private CancellationToken _tickerStreamCancellationToken;
private string ServerPath;
private HubConnection _hubConnection;
public NozomiClient(string serverPath)
{
ServerPath = serverPath;
_hubConnection = new HubConnectionBuilder()
.WithUrl(serverPath)
.Build();
}
public async Task InitializeAsync()
{
await _hubConnection.StartAsync();
}
public async Task StreamTickers()
{
// Setup the channel for streaming
var streamTickerChannel = await _hubConnection.StreamAsChannelAsync<NozomiResult<CurrencyPair>>("SubscribeToAll", CancellationToken.None);
// Setup the asynchronous data stream
// https://learn.microsoft.com/en-us/aspnet/core/signalr/streaming?view=aspnetcore-2.1#net-client
//while (await streamTickerChannel.WaitToReadAsync())
//{
// while (streamTickerChannel.TryRead(out var cp))
// {
// Console.WriteLine(JsonConvert.SerializeObject(cp));
// }
//}
_hubConnection.On<CurrencyPair>("SubscribeToAll", cp =>
{
Console.WriteLine(cp);
});
while (!_tickerStreamCancellationToken.IsCancellationRequested)
{
if (await streamTickerChannel.WaitToReadAsync())
{
while (streamTickerChannel.TryRead(out var cp))
{
Console.WriteLine(JsonConvert.SerializeObject(cp));
}
}
Console.WriteLine("Processing");
Thread.Sleep(1000);
}
}
public ICurrencyPair CurrencyPairs { get; }
public ISource Sources { get; }
}
}
I am working on a windows service that should collect all incoming TCP/IP streams with this class. It gets started from the services OnStart() void.
However I have encountered that when messages come in from a second sender, the whole communication stops working. The service then does not react to a single connection until you restart it.
public class TCPIP
{
public static Receiver rc = new Receiver();
public class Receiver
{
public delegate void ReceivedEventHandler(string ReceivedText, string Sender);
public event ReceivedEventHandler Received;
public void Start()
{
System.Threading.Thread th = new System.Threading.Thread(internals);
th.Start();
}
private void internals()
{
TcpListener _listener = new TcpListener(1994);
_listener.Start();
while (true)
{
TcpClient rcclient = _listener.AcceptTcpClient();
StreamReader reader = new StreamReader(rcclient.GetStream());
string msg = "";
while (reader.Peek() > -1)
{
msg += Convert.ToChar(reader.Read()).ToString();
}
Received(msg, rcclient.Client.RemoteEndPoint.ToString().Split(Convert.ToChar(":"))[0]);
// Cleanup
rcclient.Close();
reader.Close();
}
}
Could anybody help me out improving this class to answer connections from multiple endpoints, and to not be occupied after one?
Thank you very much in advance.
Just create a thread when you accept some connection, so if it's blocked will not afect the main program.
Btw you can try AcceptTcpClientAsync to prevent blocking calls.
I'm working on a Chat server which receives connections from multiple clients and sends/receives messages.
This is how it gets connections from the clients:
public void StartServer()
{
tcpListener = new TcpListener(IPAddress.Any, 60000);
tcpListener.Start();
listenTask = Task.Factory.StartNew(() => ListenLoop());
}
private async void ListenLoop()
{
int i = 0;
for (; ; )
{
var socket = await _tcpListener.AcceptSocketAsync();
if (socket == null)
break;
var c = new Client(socket, i);
i++;
}
}////Got this code from somewhere here, not really what I want to use (discussed down)
This is the Client class:
public class Client
{
//irrelevant stuff here//
public Client(Socket socket, int number)
{
//irrelevant stuff here//
Thread ct = new Thread(this.run);
ct.Start();
}
public void run()
{
writer.Write("connected"); //Testing connection
while (true)
{
try
{
string read = reader.ReadString();
// Dispatcher.Invoke(new DisplayDelegate(DisplayMessage), new object[] { "[Client] : " + read });
}////////Not working, because Client needs to inherit from MainWindow.
catch (Exception z)
{
MessageBox.Show(z.Message);
}
}
}
}
Ok so problem is, to update the UI Client class must inherit from MainWindow, but when it does, I get "the calling thread must be sta because many UI components require this" error. When it doesn't inherit it works just fine.
Another problem is, I want to use a Client[] clients array and then when a user connects, it adds him to the array so that i can individually write/read to/from specific clients.
while (true)
{
try
{
clients[counter] = new Client(listener.AcceptSocket(), counter);
counter ++;
MessageBox.Show("client " + counter.ToString());
}
catch (Exception e) { MessageBox.Show(e.Message); }
}
Problem here is, i get "Object refrence not set to an instance of an object" when a client connects.
Any ideas how to fix both/any of these problems?
Sorry code might be a bit messed up but I tried lots of stuff to get it working so I ended up with lots of junk in the code.
Thanks in advance.
This is my first time playing around with SignalR. I am trying to build a notification system where the server checks at regular intervals to see if there is something (query database) to broadcast and if there is then it broadcasts it to all the clients.
I came across this post on Stackoverflow and was wondering if modifying the code to make a DB call at a particular interval was indeed the right way to do it. If not is there a better way to do it?
I did see a lot of Notification related questions posted here but none with any code in it. Hence this post.
This is the exact code that I am using:
public class NotificationHub : Hub
{
public void Start()
{
Thread thread = new Thread(Notify);
thread.Start();
}
public void Notify()
{
List<CDCNotification> notifications = new List<CDCNotification>();
while (true)
{
notifications.Clear();
notifications.Add(new CDCNotification()
{
Server = "Server A", Application = "Some App",
Message = "This is a long ass message and amesaadfasd asdf message",
ImgURL = "../Content/Images/accept-icon.png"
});
Clients.shownotification(notifications);
Thread.Sleep(20000);
}
}
}
I am already seeing some weird behaviour where the notifications come more often than they are supposed to. Even though I am supposed to get it every 20s I get it around 4-5 secs and I get multiple messages.
Here is my client:
var notifier = $.connection.notificationHub;
notifier.shownotification = function (data) {
$.each(data, function (i, sample) {
var output = Mustache.render("<img class='pull-left' src='{{ImgURL}}'/> <div><strong>{{Application}}</strong></div><em>{{Server}}</em> <p>{{Message}}</p>", sample);
$.sticky(output);
});
};
$.connection.hub.start(function () { notifier.start(); });
Couple of notes:
As soon as a second client connects to your server there will be 2 threads sending the notifications, therefore if you ave more than one client you will have intervals smaller than 20s
Handling thread manually within ASP.NET is considered bad practice, you should avoid this if possible
In general this smells a lot like polling which is kind of the thing SignalR lets you get rid of since you don't need to signal the server/client
In order to solve this you need todo something like this (again, threads in a web application are generally not a good idea):
public class NotificationHub : Hub
{
public static bool initialized = false;
public static object initLock = new object();
public void Start()
{
if(initialized)
return;
lock(initLock)
{
if(initialized)
return;
Thread thread = new Thread(Notify);
thread.Start();
initialized = true;
}
}
public void Notify()
{
List<CDCNotification> notifications = new List<CDCNotification>();
while (true)
{
notifications.Clear();
notifications.Add(new CDCNotification() { Server = "Server A", Application = "Some App", Message = "This is a long ass message and amesaadfasd asdf message", ImgURL = "../Content/Images/accept-icon.png" });
Clients.shownotification(notifications);
Thread.Sleep(20000);
}
}
}
The static initialized flag prevents multiple threads from being created. The locking around it is to ensure that the flag is only set once.
I am working on the same task over here. Instead of continuously checking the database, I created my own events and listener, where an event is RAISED when a NOTIFICATION IS ADDED :) What do you think about that?