We Keep Coding

TCP support in Azure IoT Hub

Azure IoT Hub Supports AMQP, MQTT, HTTP protocols. In order to customize these protocols we have Azure IoT protocol gateway. I can find good samples on MQTT protocol customization. I need some sample codes for TCP based protocol customization using Azure IoT Protocol Gateway.
EDIT (in order to get an answer): what the OP was asking, is an example using the Azure Protocol Gateway to support a proprietary TCP-based protocol. Currently the IoT hub only supports AMQP, MQTT and HTTP. While those protocols actually rely on TCP, the hub doesn't support direct TCP connection without the extra layer of AMQP, MQTT or HTTP.
As explained here, we need a basic example of a custom TCP based protocol.
Imagine a basic device that can only send some proprietary payload through TCP on a given IP address/port: we need an example of a gateway customization that allows this device to send data to the hub.
The current code of the protocol gateway is poorly designed, as it heavily relies on MQTT.
Adding some bounty too.

The default Protocol Gateway sample are indeed somewhat confusing because of all the MQTT code.
The protocol gateway works by 'simulating' a IoTHub connection for each custom protocol device you connect to the gateway.
To do this translation from the TCP device to an IoTHub device you first need to have a connection to the IoTHub on behalf of the device. This is the gateway part.
Below is the core essentials for this IoTHubConnection.
namespace GatewayTest
{
using System;
using System.Text;
using System.Threading;
using System.Threading.Tasks;
using DotNetty.Buffers;
using Microsoft.Azure.Devices.ProtocolGateway.Identity;
using Microsoft.Azure.Devices.ProtocolGateway.IotHubClient;
using Microsoft.Azure.Devices.ProtocolGateway.Messaging;
public class IoTHubConnection : IMessagingChannel<IMessage>
{
private readonly string iotHubHostName;
private readonly Func<IDeviceIdentity, Task<IMessagingServiceClient>> deviceClientFactory;
private readonly Func<string, Task> onMessage;
private IMessagingServiceClient deviceClient;
private IDeviceIdentity deviceIdentity;
public IoTHubConnection(
string iotHubHostName,
Func<IDeviceIdentity, Task<IMessagingServiceClient>> deviceClientFactory,
Func<string, Task> onMessage)
{
this.iotHubHostName = iotHubHostName;
this.deviceClientFactory = deviceClientFactory;
this.onMessage = onMessage;
}
public event EventHandler CapabilitiesChanged;
public async Task OpenAsync(string deviceId, string deviceKey)
{
this.deviceIdentity = this.GetDeviceIdentity(deviceId, deviceKey);
if (this.deviceIdentity != UnauthenticatedDeviceIdentity.Instance)
{
this.deviceClient = await this.deviceClientFactory(this.deviceIdentity);
this.deviceClient.BindMessagingChannel(this);
}
}
public async Task CloseAsync()
{
await this.deviceClient.DisposeAsync(null);
this.deviceClient = null;
}
public void Handle(IMessage message)
{
var messageBody = message.Payload.ToString(Encoding.UTF8);
this.onMessage(messageBody);
this.deviceClient.CompleteAsync(message.Id);
}
public Task SendMessage(string message)
{
var buffer = Unpooled.WrappedBuffer(Encoding.UTF8.GetBytes(message));
var deviceMessage = this.deviceClient.CreateMessage($"devices/{this.deviceIdentity.Id}/messages/events", buffer);
return this.deviceClient.SendAsync(deviceMessage);
}
protected virtual void OnCapabilitiesChanged(EventArgs e)
{
this.CapabilitiesChanged?.Invoke(this, e);
}
private IDeviceIdentity GetDeviceIdentity(string userName, string deviceKey)
{
IotHubDeviceIdentity ideviceIdentity;
if (!IotHubDeviceIdentity.TryParse($"{this.iotHubHostName}/{userName}", out ideviceIdentity))
{
return UnauthenticatedDeviceIdentity.Instance;
}
ideviceIdentity.WithDeviceKey(deviceKey);
return ideviceIdentity;
}
}
}
The deviceClientFactory callback method should be implemented as shown below and in this line in the ProtocolGateway repo in Github.
deviceClientFactory = IotHubClient.PreparePoolFactory(
"IotHubConnectionString",
400,
TimeSpan.FromMinutes(3),
iotHubClientSettings,
PooledByteBufferAllocator.Default,
new ConfigurableMessageAddressConverter("TopicNameConversion"));
When a Tcp Device connects to the protocol, you should create an instance of this IoTHubConnection and send messages from the Device to the IoTHubConnection and vica versa.
The code below shows a very simple version of how this should be done.
private const int BufferSize = 1024;
private byte[] buffer = new byte[BufferSize];
private IoTHubConnection ioTHubConnection;
private NetworkStream stream;
private async Task Start()
{
listener = new TcpListener(IPAddress.Any, port);
listener.Start();
var client = await listener.AcceptTcpClientAsync();
ioTHubConnection = new IoTHubConnection("IoTHubName", deviceClientFactory, OnIoTHubMessage);
stream = client.GetStream();
// Read DeviceId and DeviceKey from some sort of StartConnection-message send by the TcpClient.
await ioTHubConnection.OpenAsync("DeviceId", "DeviceKey");
stream.BeginRead(buffer, 0, BufferSize, ReadTcpStreamCallback, null);
}
private void ReadTcpStreamCallback(IAsyncResult ar)
{
var bytesRead = stream.EndRead(ar);
if (bytesRead > 0)
{
var message = System.Text.Encoding.ASCII.GetString(result);
ioTHubConnection.SendMessage(message);
// Read again.
stream.BeginRead(buffer, 0, BufferSize, ReadTcpStreamCallback, null);
}
}
private async Task OnIoTHubMessage(string message)
{
// Potentially do some translation on the IoTHub message
// and send it to the Device
var byteData = Encoding.UTF8.GetBytes(message);
stream.BeginWrite(byteData, 0, byteData.Length, SendTcpCallback, null);
}
private void SendTcpCallback(IAsyncResult ar)
{
stream.EndWrite(ar);
}

I know I am late to this conversation. However I have interesting add on or might be a solution for some.
Azure IoT Gateway is now known as Azure IoT Edge, this is clearly mentioned in the following Azure github repo
https://github.com/Azure/iot-edge-modbus.git
On the other hand, Azure IoT Edge supports TCP for some protocols which can be found in the following links
https://learn.microsoft.com/en-us/azure/iot-edge/deploy-modbus-gateway
https://learn.microsoft.com/en-us/azure/iot-edge/iot-edge-as-gateway

Can't get NetMQ pub-sub pattern to work with ReceiveReady

I'm trying my hands on NetMQ (3.3.3.4) and creating a pub-sub pattern.
I want a host/server to listen to all incoming data on one port (9000) and forward the data to all clients/subscribers on another port (9001).
The clients will then send data on 9000 and receive all messages sent (by whomever) on 9001.
Following the documentation I created something like the code below, but I can't get it to work. Mainly, I believe, because ReceiveReady is never called!
How I believe it should work:
client.Publish should cause the first line in host.SubscriberSocket_ReceiveReady to unblock and pass the data along to the other socket
When data has been passed along it should appear in the infinite running Task in the client
Results:
Breakpoints on // This line is never reached are never reached
There are no exceptions anywhere.
Switching the ports on the host so that publish = 9000 and subscribe = 9001 has no effect
Windows Firewall is turned off, so there should not be any blocking
It makes no difference if I'm putting the address into PublisherSocket constructor, or if I'm using _publisherSocket.Bind(address) in Host or _publisherSocket.Connect(address) in Client
What am I doing wrong?
Host
public class MyNetMQHost {
private NetMQSocket _publishSocket;
private NetMQSocket _subscribeSocket;
private NetMQPoller _poller;
public MyNetMQHost(string publishAddress = "#tcp://localhost:9001", string subscribeAddress = "#tcp://localhost:9000") {
Task.Factory.StartNew(() => {
using (_publishSocket = new PublisherSocket(publishAddress))
using (_subscribeSocket = new SubscriberSocket(subscribeAddress))
using (_poller = new NetMQPoller { _publishSocket, _subscribeSocket }) {
_subscriberSocket.ReceiveReady += SubscriberSocket_ReceiveReady;
_poller.Run();
}
});
}
private void SubscriberSocket_ReceiveReady(object sender, NetMQSocketEventArgs e) {
var data = e.Socket.ReceiveMultipartBytes(); // This line is never reached
_publishSocket.SendMultipartBytes(data);
}
}
Client
public class MyNetMQClient {
private readonly NetMQSocket _publishSocket;
private readonly NetMQSocket _subscribeSocket;
public MyNetMQClient(string publishAddress = ">tcp://localhost:9000", string subscribeAddress = ">tcp://localhost:9001") {
_publishSocket = new PublisherSocket(publishAddress);
_subscribeSocket = new SubscriberSocket(subscribeAddress);
Task.Factory.StartNew(() => {
while (true) {
byte[] frameBytes = _subscribeSocket.ReceiveFrameBytes();
int one = 1; // This line is never reached
}
});
}
public void Publish(byte[] data) {
_publishSocket.SendFrame(data);
}
}
Tester
public class Tester {
public void MyTester() {
MyNetMQHost host = new MyNetMQHost();
MyNetMQClient client = new MyNetMQClient();
client.Publish(Encoding.Unicode.GetBytes("Hello world!");
}
}

Both your broker and client never call suscribe.
On the broker call suscriber.Subscribe("") to subscribe for all. On your client subscribe to what ever you want.
In your broker you should actually use XSubscriber and XPublisher to move susvriptions around. That way you dont need the subscribe all. You can use Proxy class for that.

NetMQ - messages only work on reply to incoming message, not to other port

On initialize of my router application I call the following code. It binds fine, receives messages fine but refuses to work for the On_ReceiveXXX methods unless it's a direct response. I want to know why
public void Initialize(string frontEndAddress, string backEndAddress)
{
_poller = new Poller();
_timeAllowedBetweenPings = TimeSpan.FromMinutes(1);
_lastPingResponse = DateTime.Now;
using (var ctx = NetMQContext.Create())
{
_frontEnd = ctx.CreateRouterSocket();
_backEnd = ctx.CreateRouterSocket();
_frontEnd.Bind(frontEndAddress);
Console.WriteLine(string.Format("[Router]: Connected to {0}", frontEndAddress));
_backEnd.Bind(backEndAddress);
Console.WriteLine(string.Format("[Router]: Connected to {0}", backEndAddress));
_frontEnd.ReceiveReady += On_ReceiveFrontEnd;
_backEnd.ReceiveReady += On_ReceiveBackEnd;
_poller.AddSocket(_frontEnd);
_poller.AddSocket(_backEnd);
var timer = new NetMQTimer(TimeSpan.FromSeconds(1));
timer.Elapsed += On_Ping;
_poller.AddTimer(timer);
_poller.PollTillCancelled();
}
}
This fails to call the dealer ReceiveReady event:
private void On_ReceiveFrontEnd(object sender, NetMQSocketEventArgs e)
{
_lastPingResponse = DateTime.Now;
var frontEndMsg = e.Socket.ReceiveMultipartBytes();
var streamData = frontEndMsg.Last();
ApplicationMessage msg = PackageHelper.DeserializeOutgoing(streamData);
Console.WriteLine(string.Format("Command received: {0}", msg.CO));
_backEnd.SendMultipartBytes(frontEndMsg);
}
BUT if I change the line
_backEnd.SendMultipartBytes(frontEndMsg);
to
_frontEnd.SendMultipartBytes(frontEndMsg);
It suddenly works... so messages coming from my front end application can only be responded to, not passed on to the back end application. The same is true the other way round, for the back end messages.

When working with router the first frame is the routing id and it specific to the socket. So you can't pass the entire message from router to router. Change the backend to dealer and it will work, or prefix the message with routing id of the backend socket.

Client Bluetooth connection with 32feet.NET fails all the time

I'm trying to get a Bluetooth socket connection up and running but for some reason my client will not connect.
More precisely I get an exception when I try to connect to the stream:
A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
All examples I found online didn't really solve my problem and I'm currently not really sure where the problem comes from.
The scanning and pairing works fine - I see that the Bluetooth device in question gets successfully paired.
I try to connect via first setting the Client and then call connect
Client Bluetooth name, address and pin are known:
public bool SetClient(String clientName, String btAddress, String pin)
{
bool retVal = false;
m_remoteBluetoothClient = new BluetoothDeviceInfo(BluetoothAddress.Parse(btAddress));
m_localBluetoothClient.SetPin(pin);
if (m_remoteBluetoothClient.Authenticated)
{
//m_localBluetoothClient.Authenticate = true;
retVal = true;
}
else
{
if (BluetoothSecurity.PairRequest(m_remoteBluetoothClient.DeviceAddress, pin))
{
retVal = true;
}
}
return retVal;
}
Then an async connect:
private void ClientConnectThread()
{
m_localBluetoothClient.BeginConnect(m_remoteBluetoothClient.DeviceAddress, BluetoothService.SerialPort, Connect, m_localBluetoothClient);
}
private void Connect(IAsyncResult result)
{
if (result.IsCompleted)
{
m_localBluetoothClient.EndConnect(result);
mBtStream = m_localBluetoothClient.GetStream();
}
}
The locals m_localBluetoothEndpoint and m_localBluetoothClient are created like this although the Endpoint is more or less new (before I used BluetoothCLient without parameter):
m_localBluetoothEndpoint = new BluetoothEndPoint(BluetoothRadio.PrimaryRadio.LocalAddress, BluetoothService.SerialPort);
m_localBluetoothClient = new BluetoothClient(m_localBluetoothEndpoint);
I also tried to work with a Listener in case the remote devices wants to connect but the callback gets never called:
public void SetupListener()
{
var listener = new BluetoothListener(BluetoothService.SerialPort);
listener.Start();
listener.BeginAcceptBluetoothClient(this.BluetoothListenerAcceptClientCallbackTwo, listener);
}
Can anyone tell me if there is anything wrong with my connection approach above and how I can figure out why I get the exception mentioned above?
The exception gets thrown here:
m_localBluetoothClient.EndConnect(result);
A thing I also don't understand is that the SupportedServices call to the remoteCLient returned 0 guids - so the device did not list any Bluetooth services.
m_remoteBluetoothClient.InstalledServices()
Thank you

UDP in C# works on Windows but not Linux

I'm using mono to build a C# program that needs to send and receive using UDP. Currently my implementation works as expected on Windows but I have issues getting communication to work with my Ubuntu or Fedora systems.
Windows can broadcast and receive it's own datagrams.
Ubuntu can broadcast and receive it's own datagrams. It's broadcasts are received by Windows but it doesn't see datagrams broadcast by Windows.
Fedora can broadcast but does not receive datagrams from anywhere (not even itself). It's broadcasts are received by Windows.
When datagrams fail to reach either of the linux machines, the 'receive' function is never fired.
This is what I have so far:
int _port = 4568;
var server = new UdpClient(_port);
var send_UDP = new UdpClient();
The receive method uses the asynchronous calls of the UDPClient;
private static void receive()
{
server.BeginReceive(new AsyncCallback(receive), null);
}
private static void receive(IAsyncResult o)
{
try
{
// I'm told that port = 0 should receive from any port.
var sender = new IPEndPoint(IPAddress.Any, 0);
var data = server.EndReceive(o, ref sender);
receive();
var str = new string(Encoding.ASCII.GetChars(data));
postmessage(sender.Address.ToString() + ":" + sender.Port.ToString() + " > " + str);
}
catch {}
}
And the send method;
public static void send(string message)
{
var target = new IPEndPoint(IPAddress.Parse("255.255.255.255"), _port);
byte[] data = Encoding.ASCII.GetBytes(message);
send_UDP.Send(data, data.Length, target);
}
After some testing with Fedora, it seems to be an issue with the use of 255.255.255.255 to broadcast. Is there some other way to do this?

I already specified this in a comment but placing this as an answer since you may have overlooked it and no answers seem to be forthcoming.
Instead of using 255.255.255.255 for broadcast use your local IP subnet's broadcasting address (for instance 192.168.0.255 on a 192.168.0.1/24 subnet). The 255.255.255.255address will not be forwarded by a router (this is relevant if there are multiple subnets at your clients' sites) whereas a directed broadcast can be forwarded (if so configured). It used to be the case that routers would forward directed broadcasts per default but this was changed in RFC2644 so don't bet the farm on it ;).
Here's an example of calculating the directed IPV4 broadcast address per adapter:
public static void DisplayDirectedBroadcastAddresses()
{
foreach (var iface in NetworkInterface.GetAllNetworkInterfaces()
.Where(c => c.NetworkInterfaceType != NetworkInterfaceType.Loopback))
{
Console.WriteLine(iface.Description);
foreach (var ucastInfo in iface.GetIPProperties().UnicastAddresses
.Where(c => !c.Address.IsIPv6LinkLocal))
{
Console.WriteLine("\tIP : {0}", ucastInfo.Address);
Console.WriteLine("\tSubnet : {0}", ucastInfo.IPv4Mask);
byte[] ipAdressBytes = ucastInfo.Address.GetAddressBytes();
byte[] subnetMaskBytes = ucastInfo.IPv4Mask.GetAddressBytes();
if (ipAdressBytes.Length != subnetMaskBytes.Length) continue;
var broadcast = new byte[ipAdressBytes.Length];
for (int i = 0; i < broadcast.Length; i++)
{
broadcast[i] = (byte)(ipAdressBytes[i] | ~(subnetMaskBytes[i]));
}
Console.WriteLine("\tBroadcast: {0}", new IPAddress(broadcast).ToString());
}
}
}