We Keep Coding

Losing messages with MQTT with C# uPLibrary.Networking.M2Mqtt

I have a problem that I lose messages with MQTT although I send them with "QOS_LEVEL_EXACTLY_ONCE".
The loss is only when the receiver is not running and then starts later.
These messages are then not collected.
Version of M2Mqtt is 4.3.0
If both clients, i.e. receiver and transmitter, are running, no messages are lost.
Only if the receiver is not running, the messages are prefetched during this time and do not arrive at the receiver.
I can't find any setting on the server(broker) for how long messages should be saved
sender
public class Programm
{
static MqttClient mqttClient;
static async Task Main(string[] args)
{
var locahlost = true;
var clientName = "Sender 1";
Console.WriteLine($"{clientName} Startet");
var servr = locahlost ? "localhost" : "test.mosquitto.org";
mqttClient = new MqttClient(servr);
mqttClient.Connect(clientName);
Task.Run(() =>
{
if (mqttClient != null && mqttClient.IsConnected)
{
for (int i = 0; i < 100; i++)
{
var Message = $"{clientName} ->Test {i}";
mqttClient.Publish("Application1/NEW_Message", Encoding.UTF8.GetBytes($"{Message}"), MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE, true);
Console.WriteLine(Message);
Thread.Sleep(i * 1000);
}
}
});
Console.WriteLine($"{clientName} End");
}
}
Server
public class Programm
{
static async Task Main(string[] args)
{
Console.WriteLine("Server");
MqttServerOptionsBuilder options = new MqttServerOptionsBuilder()
// set endpoint to localhost
.WithDefaultEndpoint()
// port used will be 707
.WithDefaultEndpointPort(1883);
// handler for new connections
// creates a new mqtt server
IMqttServer mqttServer = new MqttFactory().CreateMqttServer();
// start the server with options
mqttServer.StartAsync(options.Build()).GetAwaiter().GetResult();
// keep application running until user press a key
Console.ReadLine();
}
}
Receiver
public class Programm
{
static MqttClient mqttClient;
static async Task Main(string[] args)
{
var clientName = "Emfänger 1";
var locahlost = true;
Console.WriteLine($"Start of {clientName}");
Task.Run(() =>
{
var servr = locahlost ? "localhost" : "test.mosquitto.org";
mqttClient = new MqttClient(servr);
mqttClient.MqttMsgPublishReceived += MqttClient_MqttMsgPublishReceived;
mqttClient.Subscribe(new string[] { "Application1/NEW_Message" }, new byte[] { MqttMsgBase.QOS_LEVEL_EXACTLY_ONCE });
mqttClient.Connect(clientName);
});
// client.UseConnecedHandler(e=> {Console.WriteLine("Verbunden") });
Console.ReadLine();
Console.WriteLine($"end of {clientName}");
Console.ReadLine();
}
private static void MqttClient_MqttMsgPublishReceived(object sender, uPLibrary.Networking.M2Mqtt.Messages.MqttMsgPublishEventArgs e)
{
var message = Encoding.UTF8.GetString(e.Message);
Console.WriteLine(message);
}
}

The default value for the Clean session flag when connecting to the broker with M2MQTT is true.
This means that the broker will discard any queued messages.
https://m2mqtt.wordpress.com/using-mqttclient/
You need to set this to false to ensure the client receives the queued messages.
mqttClient.Connect(clientName, false);

I found the error, saving was missing.
Here is the new code from the server
static async Task Main(string[] args)
{
Console.WriteLine("Server");
MqttServerOptionsBuilder options = new MqttServerOptionsBuilder()
.WithDefaultEndpoint()
.WithDefaultEndpointPort(1883)
.WithConnectionValidator(OnNewConnection)
.WithApplicationMessageInterceptor(OnNewMessage)
.WithClientMessageQueueInterceptor(OnOut)
.WithDefaultCommunicationTimeout(TimeSpan.FromMinutes(5))
.WithMaxPendingMessagesPerClient(10)
.WithPersistentSessions()
.WithStorage(storage);
// creates a new mqtt server
IMqttServer mqttServer = new MqttFactory().CreateMqttServer();
// start the server with options
mqttServer.StartAsync(options.Build()).GetAwaiter().GetResult();
// keep application running until user press a key
Console.ReadLine();
}

Unable to listen UDP port in C#

I am trying to listen to UDP port 10086, store the data in the string "loggingEvent", combine the string and send it to the UDP port 51999. The UDP client 10086 is used to listen to port 10086 and the Udpclient 10087 is for sending the message to port 51999.
Although I can receive data in the UDP tool "Packet Sender", the C# code failed to listen to the message on port 10086. However, the port "10086", "10087" and "51999" really works, which can be found by using the command "net-stat -ano".
After debugging, I found that the thread exited at the line "var udpResult = await udpClient_listen.ReceiveAsync();", which confused me a lot. I have also tried non-asynchronous and still does not work. However, the "SendAsync" function works well.
In "CharacteristicPage.xaml.cs"
public static string udp_listener_ip;
public static int udp_listener_port;
public static int udp_client_port;
public static string udp_message;
public static UdpClient udpClient_listen;
public static UdpClient udpClient_send;
public static string loggingEvent;
private void button_udp_Click(object sender, Windows.UI.Xaml.RoutedEventArgs e)
{
IPEndPoint endPoint_listen= new IPEndPoint(IPAddress.Parse(udp_listener_ip), 10086);
udpClient_listen = new UdpClient(endPoint_listen);
IPEndPoint endPoint1 = new IPEndPoint(IPAddress.Parse(udp_listener_ip), 10087);
udpClient_send = new UdpClient(endPoint1);
UDPListener();
}
private static async Task UDPListener()
{
try
{
while (true)
{
Debug.WriteLine("###############UDP listener task start###############");
var udpResult = await udpClient_listen.ReceiveAsync();
loggingEvent = Encoding.ASCII.GetString(udpResult.Buffer);
Debug.WriteLine("UDP listener received: " + loggingEvent);
}
}
catch (Exception e)
{
Debug.WriteLine(e.Message);
var messageDialog = new MessageDialog(e.Message, "UDP connect failures");
await messageDialog.ShowAsync();
}
}
In "ObservableGattCharacteristic.cs"
private async void Characteristic_ValueChanged(GattCharacteristic sender, GattValueChangedEventArgs args)
{
await Windows.ApplicationModel.Core.CoreApplication.MainView.CoreWindow.Dispatcher.RunAsync(
Windows.UI.Core.CoreDispatcherPriority.Normal,
() =>
{
SetValue(args.CharacteristicValue);
});
var udpClient_send = Views.CharacteristicPage.udpClient_send;
var loggingEvent = Views.CharacteristicPage.loggingEvent;
try
{
IPEndPoint endPoint = new IPEndPoint(IPAddress.Parse(Views.CharacteristicPage.udp_listener_ip), 51999);
DateTime foo = DateTime.Now;
long unixTime = ((DateTimeOffset)foo).ToUnixTimeSeconds();
byte[] Payload = Encoding.UTF8.GetBytes(unixTime.ToString() + "," + loggingEvent + "," + ParseValue(value));
await udpClient_send.SendAsync(Payload, Payload.Length, endPoint);
Debug.WriteLine("UDP send message: " + Encoding.UTF8.GetString(Payload));
}
catch
{
Debug.WriteLine("FAILED to publish message!" );
}
}

BrokeredMessage disposed after accessing from different thread

This might be a duplicate of this question but that's confused with talk about batching database updates and still has no proper answer.
In a simple example using Azure Service Bus queues, I can't access a BrokeredMessage after it's been placed on a queue; it's always disposed if I read the queue from another thread.
Sample code:
class Program {
private static string _serviceBusConnectionString = "XXX";
private static BlockingCollection<BrokeredMessage> _incomingMessages = new BlockingCollection<BrokeredMessage>();
private static CancellationTokenSource _cancelToken = new CancellationTokenSource();
private static QueueClient _client;
static void Main(string[] args) {
// Set up a few listeners on different threads
Task.Run(async () => {
while (!_cancelToken.IsCancellationRequested) {
var msg = _incomingMessages.Take(_cancelToken.Token);
if (msg != null) {
try {
await msg.CompleteAsync();
Console.WriteLine($"Completed Message Id: {msg.MessageId}");
} catch (ObjectDisposedException) {
Console.WriteLine("Message was disposed!?");
}
}
}
});
// Now set up our service bus reader
_client = GetQueueClient("test");
_client.OnMessageAsync(async (message) => {
await Task.Run(() => _incomingMessages.Add(message));
},
new OnMessageOptions() {
AutoComplete = false
});
// Now start sending
Task.Run(async () => {
int sent = 0;
while (!_cancelToken.IsCancellationRequested) {
var msg = new BrokeredMessage();
await _client.SendAsync(msg);
Console.WriteLine($"Sent {++sent}");
await Task.Delay(1000);
}
});
Console.ReadKey();
_cancelToken.Cancel();
}
private static QueueClient GetQueueClient(string queueName) {
var namespaceManager = NamespaceManager.CreateFromConnectionString(_serviceBusConnectionString);
if (!namespaceManager.QueueExists(queueName)) {
var settings = new QueueDescription(queueName);
settings.MaxDeliveryCount = 10;
settings.LockDuration = TimeSpan.FromSeconds(5);
settings.EnableExpress = true;
settings.EnablePartitioning = true;
namespaceManager.CreateQueue(settings);
}
var factory = MessagingFactory.CreateFromConnectionString(_serviceBusConnectionString);
factory.RetryPolicy = new RetryExponential(minBackoff: TimeSpan.FromSeconds(0.1), maxBackoff: TimeSpan.FromSeconds(30), maxRetryCount: 100);
var queueClient = factory.CreateQueueClient(queueName);
return queueClient;
}
}
I've tried playing around with settings but can't get this to work. Any ideas?

Answering my own question with response from Serkant Karaca # Microsoft here:
Very basic rule and I am not sure if this is documented. The received message needs to be processed in the callback function's life time. In your case, messages will be disposed when async callback completes, this is why your complete attempts are failing with ObjectDisposedException in another thread.
I don't really see how queuing messages for further processing helps on the throughput. This will add more burden to client for sure. Try processing the message in the async callback, that should be performant enough.
Bugger.

Sending data over NetworkStream using multiple threads

I'm trying to build a command line chat room where the server is handling the connections and repeating input from one client back to all the other clients.
Currently the server is able to take in input from multiple clients, but can only send information back to those clients individually. I think my problem is that each connection is being handled on an individual thread. How would I allow for the threads to communicate with each other or be able to send data to each thread?
Server code:
namespace ConsoleApplication
{
class TcpHelper
{
private static object _lock = new object();
private static List<Task> _connections = new List<Task>();
private static TcpListener listener { get; set; }
private static bool accept { get; set; } = false;
private static Task StartListener()
{
return Task.Run(async () =>
{
IPAddress address = IPAddress.Parse("127.0.0.1");
int port = 5678;
listener = new TcpListener(address, port);
listener.Start();
Console.WriteLine($"Server started. Listening to TCP clients at 127.0.0.1:{port}");
while (true)
{
var tcpClient = await listener.AcceptTcpClientAsync();
Console.WriteLine("Client has connected");
var task = StartHandleConnectionAsync(tcpClient);
if (task.IsFaulted)
task.Wait();
}
});
}
// Register and handle the connection
private static async Task StartHandleConnectionAsync(TcpClient tcpClient)
{
// start the new connection task
var connectionTask = HandleConnectionAsync(tcpClient);
// add it to the list of pending task
lock (_lock)
_connections.Add(connectionTask);
// catch all errors of HandleConnectionAsync
try
{
await connectionTask;
}
catch (Exception ex)
{
// log the error
Console.WriteLine(ex.ToString());
}
finally
{
// remove pending task
lock (_lock)
_connections.Remove(connectionTask);
}
}
private static async Task HandleConnectionAsync(TcpClient client)
{
await Task.Yield();
{
using (var networkStream = client.GetStream())
{
if (client != null)
{
Console.WriteLine("Client connected. Waiting for data.");
StreamReader streamreader = new StreamReader(networkStream);
StreamWriter streamwriter = new StreamWriter(networkStream);
string clientmessage = "";
string servermessage = "";
while (clientmessage != null && clientmessage != "quit")
{
clientmessage = await streamreader.ReadLineAsync();
Console.WriteLine(clientmessage);
servermessage = clientmessage;
streamwriter.WriteLine(servermessage);
streamwriter.Flush();
}
Console.WriteLine("Closing connection.");
networkStream.Dispose();
}
}
}
}
public static void Main(string[] args)
{
// Start the server
Console.WriteLine("Hit Ctrl-C to close the chat server");
TcpHelper.StartListener().Wait();
}
}
}
Client Code:
namespace Client2
{
public class Program
{
private static void clientConnect()
{
TcpClient socketForServer = new TcpClient();
bool status = true;
string userName;
Console.Write("Input Username: ");
userName = Console.ReadLine();
try
{
IPAddress address = IPAddress.Parse("127.0.0.1");
socketForServer.ConnectAsync(address, 5678);
Console.WriteLine("Connected to Server");
}
catch
{
Console.WriteLine("Failed to Connect to server{0}:999", "localhost");
return;
}
NetworkStream networkStream = socketForServer.GetStream();
StreamReader streamreader = new StreamReader(networkStream);
StreamWriter streamwriter = new StreamWriter(networkStream);
try
{
string clientmessage = "";
string servermessage = "";
while (status)
{
Console.Write(userName + ": ");
clientmessage = Console.ReadLine();
if ((clientmessage == "quit") || (clientmessage == "QUIT"))
{
status = false;
streamwriter.WriteLine("quit");
streamwriter.WriteLine(userName + " has left the conversation");
streamwriter.Flush();
}
if ((clientmessage != "quit") && (clientmessage != "quit"))
{
streamwriter.WriteLine(userName + ": " + clientmessage);
streamwriter.Flush();
servermessage = streamreader.ReadLine();
Console.WriteLine("Server:" + servermessage);
}
}
}
catch
{
Console.WriteLine("Exception reading from the server");
}
streamreader.Dispose();
networkStream.Dispose();
streamwriter.Dispose();
}
public static void Main(string[] args)
{
clientConnect();
}
}
}

The main thing wrong in your code is that you make no attempt to send data received from one client to the other connected clients. You have the _connections list in your server, but the only thing stored in the list are the Task objects for the connections, and you don't even do anything with those.
Instead, you should maintain a list of the connections themselves, so that when you received a message from one client, you can then retransmit that message to the other clients.
At a minimum, this should be a List<TcpClient>, but because you are using StreamReader and StreamWriter, you'll want to initialize and store those objects in the list as well. In addition, you should include a client identifier. One obvious choice for this would be the name of the client (i.e. what the user enters as their name), but your example doesn't provide any mechanism in the chat protocol to transmit that identification as part of the connection initialization, so in my example (below) I just use a simple integer value.
There are some other irregularities in the code you posted, such as:
Starting a task in a brand new thread, just to execute a few statements that get you to the point of initiating an asynchronous operation. In my example, I simply omit the Task.Run() part of the code, as it's not needed.
Checking the connection-specific task when it's returned for IsFaulted. Since it's unlikely any I/O will actually have occurred by the time this Task object is returned, this logic has very little use. The call to Wait() will throw an exception, which will propagate to the main thread's Wait() call, terminating the server. But you don't terminate the server in the event of any other error, so it's not clear why you'd want to do that here.
There's a spurious call to Task.Yield(). I have no idea what you're trying to accomplish there, but whatever it is, that statement isn't useful. I simply removed it.
In your client code, you only attempt to receive data from the server when you've sent data. This is very wrong; you want clients to be responsive and receive data as soon as it's sent to them. In my version, I included a simple little anonymous method that is called immediately to start a separate message-receiving loop that will execute asynchronously and concurrently with the main user input loop.
Also in the client code, you were sending the "…has left…" message after the "quit" message that would cause the server to close the connection. This means that the server would never actually receive the "…has left…" message. I reversed the order of the messages so that "quit" is always the last thing the client ever sends.
My version looks like this:
Server:
class TcpHelper
{
class ClientData : IDisposable
{
private static int _nextId;
public int ID { get; private set; }
public TcpClient Client { get; private set; }
public TextReader Reader { get; private set; }
public TextWriter Writer { get; private set; }
public ClientData(TcpClient client)
{
ID = _nextId++;
Client = client;
NetworkStream stream = client.GetStream();
Reader = new StreamReader(stream);
Writer = new StreamWriter(stream);
}
public void Dispose()
{
Writer.Close();
Reader.Close();
Client.Close();
}
}
private static readonly object _lock = new object();
private static readonly List<ClientData> _connections = new List<ClientData>();
private static TcpListener listener { get; set; }
private static bool accept { get; set; }
public static async Task StartListener()
{
IPAddress address = IPAddress.Any;
int port = 5678;
listener = new TcpListener(address, port);
listener.Start();
Console.WriteLine("Server started. Listening to TCP clients on port {0}", port);
while (true)
{
var tcpClient = await listener.AcceptTcpClientAsync();
Console.WriteLine("Client has connected");
var task = StartHandleConnectionAsync(tcpClient);
if (task.IsFaulted)
task.Wait();
}
}
// Register and handle the connection
private static async Task StartHandleConnectionAsync(TcpClient tcpClient)
{
ClientData clientData = new ClientData(tcpClient);
lock (_lock) _connections.Add(clientData);
// catch all errors of HandleConnectionAsync
try
{
await HandleConnectionAsync(clientData);
}
catch (Exception ex)
{
// log the error
Console.WriteLine(ex.ToString());
}
finally
{
lock (_lock) _connections.Remove(clientData);
clientData.Dispose();
}
}
private static async Task HandleConnectionAsync(ClientData clientData)
{
Console.WriteLine("Client connected. Waiting for data.");
string clientmessage;
while ((clientmessage = await clientData.Reader.ReadLineAsync()) != null && clientmessage != "quit")
{
string message = "From " + clientData.ID + ": " + clientmessage;
Console.WriteLine(message);
lock (_lock)
{
// Locking the entire operation ensures that a) none of the client objects
// are disposed before we can write to them, and b) all of the chat messages
// are received in the same order by all clients.
foreach (ClientData recipient in _connections.Where(r => r.ID != clientData.ID))
{
recipient.Writer.WriteLine(message);
recipient.Writer.Flush();
}
}
}
Console.WriteLine("Closing connection.");
}
}
Client:
class Program
{
private const int _kport = 5678;
private static async Task clientConnect()
{
IPAddress address = IPAddress.Loopback;
TcpClient socketForServer = new TcpClient();
string userName;
Console.Write("Input Username: ");
userName = Console.ReadLine();
try
{
await socketForServer.ConnectAsync(address, _kport);
Console.WriteLine("Connected to Server");
}
catch (Exception e)
{
Console.WriteLine("Failed to Connect to server {0}:{1}", address, _kport);
return;
}
using (NetworkStream networkStream = socketForServer.GetStream())
{
var readTask = ((Func<Task>)(async () =>
{
using (StreamReader reader = new StreamReader(networkStream))
{
string receivedText;
while ((receivedText = await reader.ReadLineAsync()) != null)
{
Console.WriteLine("Server:" + receivedText);
}
}
}))();
using (StreamWriter streamwriter = new StreamWriter(networkStream))
{
try
{
while (true)
{
Console.Write(userName + ": ");
string clientmessage = Console.ReadLine();
if ((clientmessage == "quit") || (clientmessage == "QUIT"))
{
streamwriter.WriteLine(userName + " has left the conversation");
streamwriter.WriteLine("quit");
streamwriter.Flush();
break;
}
else
{
streamwriter.WriteLine(userName + ": " + clientmessage);
streamwriter.Flush();
}
}
await readTask;
}
catch (Exception e)
{
Console.WriteLine("Exception writing to server: " + e);
throw;
}
}
}
}
public static void Main(string[] args)
{
clientConnect().Wait();
}
}
There is still a lot you'll need to work on. You'll probably want to implement proper initialization of chat user names on the server side. At the very least, for real-world code you'd want to do more error checking, and make sure the client ID is generated reliably (if you only want positive ID values, you can't have more than 2^31-1 connections before it rolls back over to 0).
I also made some other minor changes that weren't strictly necessary, such as using the IPAddress.Any and IPAddress.Loopback values instead of parsing strings, and just generally simplifying and cleaning up the code here and there. Also, I'm not using a C# 6 compiler at the moment, so I changed the code where you were using C# 6 features so that it would compile using C# 5 instead.
To do a full-blown chat server, you still have your work cut out for you. But I hope that the above gets you back on the right track.

Nmodbus and multiple slave

Im trying to make a Energy plant simulator and i like to simulate in my app few devices.
The idea is that the master request for something and the app ask to the needed slave and make something.
I just start using Nmodbus and all work fine with 1 device.
I tryed using SlaveId to ask for each device but when i read or write, i ever got all writed in the same Datastore.
The app and device run in localhost.
This is how i declare each device in the same network:
int port = 502;
IPAddress address = new IPAddress(new byte[] { 127, 0, 0, 1 });
// create and start the TCP slave
slaveTcpListener = new TcpListener(address, port);
slaveTcpListener.Start();
slave1 = ModbusTcpSlave.CreateTcp(1, slaveTcpListener);
slave1.DataStore = DataStoreFactory.CreateDefaultDataStore();
slave1.Listen();
slave2 = ModbusTcpSlave.CreateTcp(2, slaveTcpListener);
slave2.DataStore = DataStoreFactory.CreateDefaultDataStore();
slave2.Listen();
And this is how i request data of each device:
using (TcpClient client = new TcpClient("127.0.0.1", 502))
{
ModbusIpMaster master = ModbusIpMaster.CreateIp(client);
// read five input values
ushort startAddress = 0;
ushort numInputs = 10;
ushort[] inputs = master.ReadHoldingRegisters(1,startAddress, numInputs);
ushort[] inputs2 = master.ReadHoldingRegisters(2, startAddress, numInputs);
}
But the result is ever the Datastore of slave1.

Unfortunately, there is no way to run multiple slave devices, because slave.Listen() method includes infinite cycle:
public override void Listen()
{
while (true)
{
try
{
try
{
// read request and build message
byte[] frame = SerialTransport.ReadRequest();
IModbusMessage request = ModbusMessageFactory.CreateModbusRequest(frame);
if (SerialTransport.CheckFrame && !SerialTransport.ChecksumsMatch(request, frame))
{
string msg = $"Checksums failed to match {string.Join(", ", request.MessageFrame)} != {string.Join(", ", frame)}.";
Debug.WriteLine(msg);
throw new IOException(msg);
}
// only service requests addressed to this particular slave
if (request.SlaveAddress != UnitId)
{
Debug.WriteLine($"NModbus Slave {UnitId} ignoring request intended for NModbus Slave {request.SlaveAddress}");
continue;
}
// perform action
IModbusMessage response = ApplyRequest(request);
// write response
SerialTransport.Write(response);
}
catch (IOException ioe)
{
Debug.WriteLine($"IO Exception encountered while listening for requests - {ioe.Message}");
SerialTransport.DiscardInBuffer();
}
catch (TimeoutException te)
{
Debug.WriteLine($"Timeout Exception encountered while listening for requests - {te.Message}");
SerialTransport.DiscardInBuffer();
}
// TODO better exception handling here, missing FormatException, NotImplemented...
}
catch (InvalidOperationException)
{
// when the underlying transport is disposed
break;
}
}

That same limitation caused a problem for me, so I took over the NModbus4 project (now just NModbus):
https://github.com/NModbus/NModbus
public static async Task StartModbusSerialRtuSlaveNetwork()
{
using (SerialPort slavePort = new SerialPort("COM5"))
{
// configure serial port
slavePort.BaudRate = 19200;
slavePort.DataBits = 8;
slavePort.Parity = Parity.Even;
slavePort.StopBits = StopBits.One;
slavePort.Open();
//Create an instance of the ModbusFactory
IModbusFactory factory = new ModbusFactory();
//Create an adapter for the serial port
var adapter = new SerialPortAdapter(slavePort);
//Create the slave network
IModbusSlaveNetwork modbusSlaveNetwork = factory.CreateRtuSlaveNetwork(adapter);
//Create the slaves
IModbusSlave slave1 = factory.CreateSlave(1);
IModbusSlave slave2 = factory.CreateSlave(2);
//Add the slaves to the network
modbusSlaveNetwork.AddSlave(slave1);
modbusSlaveNetwork.AddSlave(slave2);
await modbusSlaveNetwork.ListenAsync();
await Task.Delay(1);
}
}