I am trying to create a simple application that will publish some messages to a topic with MQTT (library I am using is M2Mqtt.net) and then I want to subscribe to the topic once the messages have already been sent and then have them all be received and then discarded, because they have been received.
I am using mosquitto 2.0.12 as the broker
This is the publisher:
public class MessagePublisher : IMessagePublisher
{
private readonly MqttClient _client;
public MessagePublisher()
{
_client = new MqttClient("localhost");
// clean session needs to be set to false so that it retains all the missed messages, not just the last one
_client.Connect(Guid.NewGuid().ToString(), "username", "password", false, byte.MaxValue);
}
public void Publish(string topic, string message, bool retain = false)
{
Console.Write($"Sent: {topic}, {message}");
_client.Publish(topic, Encoding.UTF8.GetBytes(message), MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE, retain);
Total.SentAndReceived.Add(message, null);
}
}
This is the listener:
public class MessageReceiver : IMessageReceiver
{
private readonly MqttClient _client;
public MessageReceiver()
{
_client = new MqttClient("localhost");
}
public void Subscribe(params string[] topics)
{
_client.Subscribe(topics, new[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });
_client.MqttMsgPublishReceived += client_receivedMessage;
}
public void Connect()
{
// clean session needs to be set to false so that it retains all the missed messages, not just the last one
_client.Connect(Guid.NewGuid().ToString(), "username", "password", false, byte.MaxValue);
}
public void Disconnect()
{
_client.Disconnect();
}
static void client_receivedMessage(object sender, MqttMsgPublishEventArgs e)
{
var message = Encoding.Default.GetString(e.Message);
Console.WriteLine($"Message Received: {message}");
if (Total.SentAndReceived.ContainsKey(message))
Total.SentAndReceived[message] = message;
}
}
And this is the main application:
public static class Program
{
public static void Main(string[] args)
{
var messageReceiver = new MessageReceiver();
var publisher = new MessagePublisher();
for (var i = 1; i <= 10000; i++)
{
publisher.Publish("Devices/", i.ToString(), true);
}
messageReceiver.Subscribe("Devices/");
messageReceiver.Connect();
Thread.Sleep(5000);
var b = Total.SentAndReceived.Where(x => x.Value == null);
Console.WriteLine($"{b.Count()} Missed Messages");
}
}
The problem I am having is that there are missed messages. And the number of missed messages always changes when I run the application. And it's not that last n messages being missed it's the first n messages.
I hope that if I was to build a service that would listen to the published messages. If the services stops for any reason. Once the service comes back online, the messages sent in that downtime would be received.
I think you have a misunderstanding around some terms here.
First, MQTT does not generally queue messages. The only time the broker will queue messages is if the receiving client has already already been connected and subscribed to the topic at QOS > 0. If that client then disconnects before the publisher sends the messages the broker will queue the messages. They will then only be sent to the receiving client if they then reconnect with the same client id and have the clean session flag set to false. This is the only way that messages will be queued up.
Since you appear to be using randomly generated client ids (Guid.NewGuid().ToString()) this will not work. You also appear to be trying to subscribe before you connect, again that won't work.
Secondly, retained messages have nothing to do with message queuing as described above. A message is retained if the retained flag is set at the point of publishing. The broker will then store that specific message and deliver it ever time a client subscribes to the matching topic. This message will be sent before any other messages on the topic. If another message with the retained flag is published it will replace the previous message, there can only be 1 retained message per topic.
Related
I have successfully connected MQTT to my broker both publisher and subscriber are working fine but there is a problem my subscriber is not actively receiving the message, the function mqtt_publishedRecieved triggers only one time,when I restart the subscriber app and only one message receives at a time. in order to get another message, I have to restart my app again. Well, as per my understanding it is because I called the config method of my class at startup. so therefore It checks for the subscribed topic only at startup. But I really want that functionality in my subscriber class to immediately receive the message once it is published by the receiver, which means both should work parallel.
My requirement is
=> if subscriber is connected, receives the message as soon as publisher fires.
=> if the subscriber is disconnected, message should be queued and later on all message will receive when subscriber connected again.
I do a research mqtt clean session set to false will ensure the persistent session so I set that flag to false to occupy persistent session, but it won't work for me.
it seems to me that I should add functionality to receive a message, for example, a button, so when the button is clicked it starts to get a message, but I can't set trigger/callback/button to receive the message for the subscribed topic. My app should start receiving all the published messages when service is started and stopped when it is disconnected.
below is the code of the publisher class.
public class publisher : IDatabaseSubscription
{
public publisher()
{
_mqttClient = new MqttClient("127.0.0.1");
_mqttClient.Connect("clientId", null, null, false, 60);
}
private void MQTT_OnChanged(object sender, RecordChangedEventArgs<EventDto> e)
{
if (_mqttClient != null && _mqttClient.IsConnected)
{
var message = System.Text.Encoding.UTF8.GetBytes("Hello from app 1");
var statusCode = _mqttClient.Publish("Message1",message , MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE, true);
}
}
}
below is the code of subscriber class
public class subscriber
{
private MqttClient _mqttClient;
public subscriber()
{
}
public void configure ()
{
_mqttClient = new MqttClient("127.0.0.1");
_mqttClient.MqttMsgPublishReceived += client_MqttMsgPublishReceived;
var status = _mqttClient.Subscribe(new string[] { "Message1" }, new byte[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });
_mqttClient.Connect("clientId", null, null , false, 60) ;
}
private void client_MqttMsgPublishReceived(object sender, MqttMsgPublishEventArgs e)
{
var message = System.Text.Encoding.UTF8.GetString(e.Message);
}
}
below is the code in startup
if(IsSubscriptionEnabled())
{
var service = _container.GetInstance<subscription>();
service.configure();
}
It was due to the same client id in both publisher and subscriber, broker kicked off the client if the new client connected with the same client id.
I am trying to write an integration / acceptance test to test some code in azure, the code in the question ATM simply subscribes to one topic and publishes to another.
I have written the test but it doesn't always pass, seems as though there could be a race condition in place. I've tried writing it a couple of ways including using OnMessage and also using Receive (example I show here).
When using OnMessage the test seemed to always exit prematurely (around 30 seconds), which I guess perhaps means its inappropriate for this test anyway.
My query concerning my example specifically, I assumed that once I created the subscription to the target topic, that any message sent to it I would be able to pickup using Receive(), whatever point in time that message arrived meaning, if the message arrives at the target topic before I call Receive(), I would still be able to read the message afterward by calling Receive(). Could anyone please shed any light on this?
namespace somenamespace {
[TestClass]
public class SampleTopicTest
{
private static TopicClient topicClient;
private static SubscriptionClient subClientKoEligible;
private static SubscriptionClient subClientKoIneligible;
private static OnMessageOptions options;
public const string TEST_MESSAGE_SUB = "TestMessageSub";
private static NamespaceManager namespaceManager;
private static string topicFleKoEligible;
private static string topicFleKoIneligible;
private BrokeredMessage message;
[ClassInitialize]
public static void BeforeClass(TestContext testContext)
{
//client for publishing messages
string connectionString = ConfigurationManager.AppSettings["ServiceBusConnectionString"];
string topicDataReady = ConfigurationManager.AppSettings["DataReadyTopicName"];
topicClient = TopicClient.CreateFromConnectionString(connectionString, topicDataReady);
topicFleKoEligible = ConfigurationManager.AppSettings["KnockOutEligibleTopicName"];
topicFleKoIneligible = ConfigurationManager.AppSettings["KnockOutIneligibleTopicName"];
//create test subscription to receive messages
namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
if (!namespaceManager.SubscriptionExists(topicFleKoEligible, TEST_MESSAGE_SUB))
{
namespaceManager.CreateSubscription(topicFleKoEligible, TEST_MESSAGE_SUB);
}
if (!namespaceManager.SubscriptionExists(topicFleKoIneligible, TEST_MESSAGE_SUB))
{
namespaceManager.CreateSubscription(topicFleKoIneligible, TEST_MESSAGE_SUB);
}
//subscriber client koeligible
subClientKoEligible = SubscriptionClient.CreateFromConnectionString(connectionString, topicFleKoEligible, TEST_MESSAGE_SUB);
subClientKoIneligible = SubscriptionClient.CreateFromConnectionString(connectionString, topicFleKoIneligible, TEST_MESSAGE_SUB);
options = new OnMessageOptions()
{
AutoComplete = false,
AutoRenewTimeout = TimeSpan.FromMinutes(1),
};
}
[TestMethod]
public void E2EPOCTopicTestLT50()
{
Random rnd = new Random();
string customerId = rnd.Next(1, 49).ToString();
FurtherLendingCustomer sentCustomer = new FurtherLendingCustomer { CustomerId = customerId };
BrokeredMessage sentMessage = new BrokeredMessage(sentCustomer.ToJson());
sentMessage.CorrelationId = Guid.NewGuid().ToString();
string messageId = sentMessage.MessageId;
topicClient.Send(sentMessage);
Boolean messageRead = false;
//wait for message to arrive on the ko eligible queue
while((message = subClientKoEligible.Receive(TimeSpan.FromMinutes(2))) != null){
//read message
string messageString = message.GetBody<String>();
//Serialize
FurtherLendingCustomer receivedCustomer = JsonConvert.DeserializeObject<FurtherLendingCustomer>(messageString.Substring(messageString.IndexOf("{")));
//assertion
Assert.AreEqual(sentCustomer.CustomerId, receivedCustomer.CustomerId,"verify customer id");
//pop message
message.Complete();
messageRead = true;
//leave loop after processing one message
break;
}
if (!messageRead)
Assert.Fail("Didn't receive any message after 2 mins");
}
}
}
As the official document states about SubscriptionClient.Receive(TimeSpan):
Parameters
serverWaitTime
TimeSpan
The time span the server waits for receiving a message before it times out.
A Null can be return by this API if operation exceeded the timeout specified, or the operations succeeded but there are no more messages to be received.
Per my test, if a message sent to the topic and then delivered to your subscription within your specific serverWaitTime, then you could receive a message no matter whether the message arrives at the target topic before or after you call Receive.
When using OnMessage the test seemed to always exit prematurely (around 30 seconds), which I guess perhaps means its inappropriate for this test anyway.
[TestMethod]
public void ReceiveMessages()
{
subClient.OnMessage(msg => {
System.Diagnostics.Trace.TraceInformation($"{DateTime.Now}：{msg.GetBody<string>()}");
msg.Complete();
});
Task.Delay(TimeSpan.FromMinutes(5)).Wait();
}
For SubscriptionClient.OnMessage, I assumed that it basically a loop invoking Receive. After calling OnMessage, you need to wait for a while and stop this method to exit. Here is a blog about the Event-Driven message programming for windows Azure Service Bus, you could refer to here.
Additionally, I found that your topicClient for sending messages and the subClientKoEligible for receiving a message are not targeted at the same topic path.
I'm trying my hands on NetMQ (3.3.3.4) and creating a pub-sub pattern.
I want a host/server to listen to all incoming data on one port (9000) and forward the data to all clients/subscribers on another port (9001).
The clients will then send data on 9000 and receive all messages sent (by whomever) on 9001.
Following the documentation I created something like the code below, but I can't get it to work. Mainly, I believe, because ReceiveReady is never called!
How I believe it should work:
client.Publish should cause the first line in host.SubscriberSocket_ReceiveReady to unblock and pass the data along to the other socket
When data has been passed along it should appear in the infinite running Task in the client
Results:
Breakpoints on // This line is never reached are never reached
There are no exceptions anywhere.
Switching the ports on the host so that publish = 9000 and subscribe = 9001 has no effect
Windows Firewall is turned off, so there should not be any blocking
It makes no difference if I'm putting the address into PublisherSocket constructor, or if I'm using _publisherSocket.Bind(address) in Host or _publisherSocket.Connect(address) in Client
What am I doing wrong?
Host
public class MyNetMQHost {
private NetMQSocket _publishSocket;
private NetMQSocket _subscribeSocket;
private NetMQPoller _poller;
public MyNetMQHost(string publishAddress = "#tcp://localhost:9001", string subscribeAddress = "#tcp://localhost:9000") {
Task.Factory.StartNew(() => {
using (_publishSocket = new PublisherSocket(publishAddress))
using (_subscribeSocket = new SubscriberSocket(subscribeAddress))
using (_poller = new NetMQPoller { _publishSocket, _subscribeSocket }) {
_subscriberSocket.ReceiveReady += SubscriberSocket_ReceiveReady;
_poller.Run();
}
});
}
private void SubscriberSocket_ReceiveReady(object sender, NetMQSocketEventArgs e) {
var data = e.Socket.ReceiveMultipartBytes(); // This line is never reached
_publishSocket.SendMultipartBytes(data);
}
}
Client
public class MyNetMQClient {
private readonly NetMQSocket _publishSocket;
private readonly NetMQSocket _subscribeSocket;
public MyNetMQClient(string publishAddress = ">tcp://localhost:9000", string subscribeAddress = ">tcp://localhost:9001") {
_publishSocket = new PublisherSocket(publishAddress);
_subscribeSocket = new SubscriberSocket(subscribeAddress);
Task.Factory.StartNew(() => {
while (true) {
byte[] frameBytes = _subscribeSocket.ReceiveFrameBytes();
int one = 1; // This line is never reached
}
});
}
public void Publish(byte[] data) {
_publishSocket.SendFrame(data);
}
}
Tester
public class Tester {
public void MyTester() {
MyNetMQHost host = new MyNetMQHost();
MyNetMQClient client = new MyNetMQClient();
client.Publish(Encoding.Unicode.GetBytes("Hello world!");
}
}
Both your broker and client never call suscribe.
On the broker call suscriber.Subscribe("") to subscribe for all. On your client subscribe to what ever you want.
In your broker you should actually use XSubscriber and XPublisher to move susvriptions around. That way you dont need the subscribe all. You can use Proxy class for that.
I am trying to receive all messages for a given subscription to a Service Bus Topic, but for the context of this app I do not want them dead lettered at this time, I just want to view them and leave them on the subscription. Despite instantiating the Client as
SubscriptionClient sc = SubscriptionClient.CreateFromConnectionString(connectionString, sub.topicName, sub.subscriptionName, ReceiveMode.PeekLock);
and making sure that I am using message.Abandon() rather than message.Complete() the message always gets Dead-lettered after accessing the message. I also have options.AutoComplete set to false
full method code below:
public List<ServiceBusMessage> RetrieveSubscriptionMessages(Subscription sub) {
ServiceBusMessage sbm;
List<ServiceBusMessage> list = new List<ServiceBusMessage>();
String connectionString = ConfigurationManager.AppSettings["Microsoft.ServiceBus.ConnectionString"].ToString();
SubscriptionClient sc = SubscriptionClient.CreateFromConnectionString(connectionString, sub.topicName, sub.subscriptionName, ReceiveMode.PeekLock);
OnMessageOptions options = new OnMessageOptions();
options.AutoComplete = false;
sc.OnMessage((message) => {
try {
sbm = new ServiceBusMessage() {
topicName = sub.topicName,
messageText = message.GetBody<String>()
};
list.Add(sbm);
message.Abandon();
}
catch (Exception) {
message.Abandon();
throw;
}
}, options);
return list;
}
Am I missing something ? Or is there an issue with auto dead-lettering with the onMessage() method?
Thanks !
When a message is abandoned the service bus will immediately make it available for re-delivery to any subscriber of the topic.
If you are trying to configure a multicast mechanism in which multiple listeners all receive the same message, then understand that all listeners on a given subscription will be competing for the same message. In order for every listener to receive its own copy of the message, then simply create a unique subscription to the topic for each listener.
If your intent is to delay re-delivery of the abandoned message, you might look at the SO question: What's the proper way to abandon an Azure SB Message so that it becomes visible again in the future in a way I can control?
This is my first time playing around with SignalR. I am trying to build a notification system where the server checks at regular intervals to see if there is something (query database) to broadcast and if there is then it broadcasts it to all the clients.
I came across this post on Stackoverflow and was wondering if modifying the code to make a DB call at a particular interval was indeed the right way to do it. If not is there a better way to do it?
I did see a lot of Notification related questions posted here but none with any code in it. Hence this post.
This is the exact code that I am using:
public class NotificationHub : Hub
{
public void Start()
{
Thread thread = new Thread(Notify);
thread.Start();
}
public void Notify()
{
List<CDCNotification> notifications = new List<CDCNotification>();
while (true)
{
notifications.Clear();
notifications.Add(new CDCNotification()
{
Server = "Server A", Application = "Some App",
Message = "This is a long ass message and amesaadfasd asdf message",
ImgURL = "../Content/Images/accept-icon.png"
});
Clients.shownotification(notifications);
Thread.Sleep(20000);
}
}
}
I am already seeing some weird behaviour where the notifications come more often than they are supposed to. Even though I am supposed to get it every 20s I get it around 4-5 secs and I get multiple messages.
Here is my client:
var notifier = $.connection.notificationHub;
notifier.shownotification = function (data) {
$.each(data, function (i, sample) {
var output = Mustache.render("<img class='pull-left' src='{{ImgURL}}'/> <div><strong>{{Application}}</strong></div><em>{{Server}}</em> <p>{{Message}}</p>", sample);
$.sticky(output);
});
};
$.connection.hub.start(function () { notifier.start(); });
Couple of notes:
As soon as a second client connects to your server there will be 2 threads sending the notifications, therefore if you ave more than one client you will have intervals smaller than 20s
Handling thread manually within ASP.NET is considered bad practice, you should avoid this if possible
In general this smells a lot like polling which is kind of the thing SignalR lets you get rid of since you don't need to signal the server/client
In order to solve this you need todo something like this (again, threads in a web application are generally not a good idea):
public class NotificationHub : Hub
{
public static bool initialized = false;
public static object initLock = new object();
public void Start()
{
if(initialized)
return;
lock(initLock)
{
if(initialized)
return;
Thread thread = new Thread(Notify);
thread.Start();
initialized = true;
}
}
public void Notify()
{
List<CDCNotification> notifications = new List<CDCNotification>();
while (true)
{
notifications.Clear();
notifications.Add(new CDCNotification() { Server = "Server A", Application = "Some App", Message = "This is a long ass message and amesaadfasd asdf message", ImgURL = "../Content/Images/accept-icon.png" });
Clients.shownotification(notifications);
Thread.Sleep(20000);
}
}
}
The static initialized flag prevents multiple threads from being created. The locking around it is to ensure that the flag is only set once.
I am working on the same task over here. Instead of continuously checking the database, I created my own events and listener, where an event is RAISED when a NOTIFICATION IS ADDED :) What do you think about that?