We Keep Coding

NetMQ (ZeroMQ) how to make "Brokerless Reliability (Freelance Pattern)" works

I facing some problems with the example I got from the ZeroMQ Guide, looks like the class ZSocket and ZContext doesn't exist.
I'm totally new with ZeroMQ (just start lo learn) and I'm following the "ØMQ - The Guide". The first example about REQ-REP, which is very simple, worked well. But now I'm trying something more similar to my objective, the "Brokerless Reliability (Freelance Pattern)" and this one didn't work.
I'm using Visual Studio 2019 with C# code, I created a new project, added NetMQ V4.0.1.6 via Nuget and copied the server code to my project. I got errors with ZContext and ZSocket. I already check the API V3 and API V4, they are clear different. The guide is totally based on version 3 and I'm using V 4. I didn't find any document about the changes or updates or equivalent function/classes/methods and I don't know how to convert the example to the NetMQ V4.
This is my test code:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading;
using NetMQ;
namespace Examples
{
static partial class Program
{
public static void FLServer1(string[] args)
{
//
// Freelance server - Model 1
// Trivial echo service
//
// Author: metadings
//
if (args == null || args.Length < 1)
{
Console.WriteLine();
Console.WriteLine("Usage: ./{0} FLServer1 [Endpoint]", AppDomain.CurrentDomain.FriendlyName);
Console.WriteLine();
Console.WriteLine(" Endpoint Where FLServer1 should bind on.");
Console.WriteLine(" Default is tcp://127.0.0.1:7780");
Console.WriteLine();
args = new string[] { "tcp://127.0.0.1:7780" };
}
using (var context = new ZContext())
using (var server = new ZSocket(context, ZSocketType.REP))
{
server.Bind(args[0]);
Console.WriteLine("I: echo service is ready at {0}", args[0]);
ZMessage message;
ZError error;
while (true)
{
if (null != (message = server.ReceiveMessage(out error)))
{
using (message)
{
server.Send(message);
}
}
else
{
if (error == ZError.ETERM)
return; // Interrupted
throw new ZException(error);
}
}
}
}
}
}

After long hours trying to understand that logic, I found a list of differences from ZeroMQ V3 and V4:
https://github.com/zeromq/netmq/wiki/Migrating-to-v4
Also, accidentally I found the example I was looking for:
https://github.com/NetMQ/Samples/tree/master/src/Brokerless%20Reliability%20(Freelance%20Pattern)/Model%20One

Why is no message received in WS communication on a UWP device?

I have a WS server and I would like to broadcast messages from that server (using another web app) to all HoloLens devices that are connected to the session.
First I have implemented a MessageWebSocket client in the Hololens app that initiated a connection with a sample public WS server echo.websocket.org just to check if the setup is right on the client side. Here is the code:
using System.Collections;
using System.Collections.Generic;
using UnityEngine;
#if WINDOWS_UWP
using System.Threading.Tasks;
using Windows.Networking.Sockets;
using Windows.Storage.Streams;
using Windows.Web;
using System;
#endif
public class WebSocketClient : MonoBehaviour
{
void Start()
{
#if WINDOWS_UWP
int msgTime = 5;
int fadeTime = 1;
guiPhraseReporter.QueueRaport("START", msgTime, fadeTime);
MessageWebSocket ws = new MessageWebSocket();
ws.Control.MessageType = SocketMessageType.Utf8;
ws.MessageReceived += (MessageWebSocket sender, MessageWebSocketMessageReceivedEventArgs args) =>
{
guiPhraseReporter.QueueRaport("Trying to receive message...", msgTime, fadeTime);
try
{
using (DataReader dataReader = args.GetDataReader())
{
dataReader.UnicodeEncoding = UnicodeEncoding.Utf8;
string message = dataReader.ReadString(dataReader.UnconsumedBufferLength);
Debug.Log(message);
}
}
catch (Exception ex)
{
Debug.Log("Error occurred");
}
};
ws.Closed += (IWebSocket sender, WebSocketClosedEventArgs args) => {
Debug.Log("WS closed");
};
try
{
Task connectTask = ws.ConnectAsync(new Uri("ws://echo.websocket.org")).AsTask();
connectTask.ContinueWith(async _ =>
{
string message = "Hello, World!";
using (DataWriter dataWriter = new DataWriter(ws.OutputStream))
{
dataWriter.WriteString(message);
await dataWriter.StoreAsync();
dataWriter.DetachStream();
}
Debug.Log("Sending Hello World");
});
}
catch (Exception ex)
{
WebErrorStatus webErrorStatus = WebSocketError.GetStatus(ex.GetBaseException().HResult);
// Add additional code here to handle exceptions.
Debug.Log(ex);
}
#endif
}
}
And it works fine, I'm able to send a message to the server, and it is echoed back and received correctly by the client.
Things however mess up when I use the actual server I'll be testing on. On my server, I have replicated the behavior from the echo.websocket.org and I echo back any message sent. I'm able to connect, the connection is not closed (Closed is never called), but I don't receive any messages.
If I test both servers using the web browser (with chrome's Smart Websocket Extension), they both work. The only difference (and only possible lead I got) is that the sample server (the one that works on Hololens) sends more headers upon connection:
vs my server:
Maybe there is some easier way to do this, but so far I didn't find any good WS wrappers that would work on UWP. Any help appreciated.

It was faulty logic on my server app after all. So there was no problem with WS communication to begin with, thank you for your time.

Cannot get ReadyReceive pub-sub to work using NetMQ 4.x

I created 2 simple C# Console Projects (.net 4.5.2), added the v4.0.0.1 NetMQ Nuget package to each, loaded each program up into separate Visual Studio 2017 Community Editions, put a breakpoint on the 1 line contained within the OnReceiveReady callback method, started the subscriber program first, then started the publisher program. The ReceieveReady event is not being triggered in the subscriber. What am I doing wrong? Even if I chose subSocket.Subscribe("") then I still didn't get any messages received. Also, removing/modifying the Send/Receive HighWatermarks didn't change things either. Thanks for your help!
Here's the Publisher code:
using System;
using NetMQ;
using NetMQ.Sockets;
using System.Threading;
namespace SampleNQPub
{
class Program
{
static void Main(string[] args)
{
var addr = "tcp://127.0.0.1:3004";
using (var pubSocket = new PublisherSocket())
{
Console.WriteLine("Publisher socket binding.");
pubSocket.Options.SendHighWatermark = 10;
pubSocket.Bind(addr);
for (int i=0; i < 30; i++)
{
pubSocket.SendMoreFrame("NQ").SendFrame(i.ToString());
Thread.Sleep(1000);
}
pubSocket.Disconnect(addr);
}
}
}
}
Here's the Subscriber code:
using System.Threading;
using NetMQ;
using NetMQ.Sockets;
namespace SampleNQSub
{
class Program
{
static void Main(string[] args)
{
var addr = "tcp://127.0.0.1:3004";
using (var subSocket = new SubscriberSocket())
{
subSocket.ReceiveReady += OnReceiveReady;
subSocket.Options.ReceiveHighWatermark = 10;
subSocket.Connect(addr);
subSocket.Subscribe("NQ");
for (int i=0; i < 20; i++)
{
Thread.Sleep(1000);
}
subSocket.Disconnect(addr);
}
}
static void OnReceiveReady(object sender, NetMQSocketEventArgs e)
{
var str = e.Socket.ReceiveFrameString();
}
}
}

Ok, this is a gotcha question in the NetMQ world and I just figured it out. You MUST setup a NetMQPoller that will wind up calling every ReceiveReady callback which you have added to it (NetMQPoller).
Here is the corrected code which will at least (i.e., ReceiveFrameString still only getting the "NQ" part but that's just another method call to fix) get the ReceiveReady event triggered:
using System.Threading;
using System.Threading.Tasks;
using NetMQ;
using NetMQ.Sockets;
namespace SampleNQSub
{
class Program
{
static void Main(string[] args)
{
var addr = "tcp://127.0.0.1:3004";
NetMQPoller poller = new NetMQPoller();
using (var subSocket = new SubscriberSocket())
{
subSocket.ReceiveReady += OnReceiveReady;
subSocket.Options.ReceiveHighWatermark = 10;
subSocket.Connect(addr);
subSocket.Subscribe("NQ");
poller.Add(subSocket);
poller.RunAsync();
for (int i = 0; i < 20; i++)
{
Thread.Sleep(1000);
}
subSocket.Disconnect(addr);
}
}
static void OnReceiveReady(object sender, NetMQSocketEventArgs e)
{
var str = e.Socket.ReceiveFrameString();
e.Socket.ReceiveMultipartStrings();
}
}
}
I noticed that the authors of NetMQ decided in 4.x to take care of the Context object internally so the user wouldn't have to bare the burden of managing it. It would be nice also if they could hide this "polling pump" code from the user as well for the most simple use case.
As a comparison, take a look at the subscriber using NodeJS (with the zmq library) utilizing the Publisher console app I posted above (save this code to sub.js and, in a Windows console, type 'node sub.js'):
var zmq = require('zmq'), sock = zmq.socket('sub');
sock.connect('tcp://127.0.0.1:3004');
sock.subscribe('NQ');
console.log('Subscriber connected to port 3004');
sock.on('message', function() {
var msg = [];
Array.prototype.slice.call(arguments).forEach(function(arg) {
msg.push(arg.toString());
});
console.log(msg);
});
So where's the poller pump mechanism in this? (Answer: I don't care! I just want the messages supplied to me in a callback that I register. [Obviously, tongue-in-cheek. I get that a NetMQPoller is versatile and handles more complex issues, but for basic "give me a message in a callback when it arrives", it would be nice if it were handled internally by the library.])

find().tolistasync() timeout (System.TimeoutException:) in mongodb

I am trying to connect to a mongodb in a virtual machine on CentOS (selinux and firewall are configured). I created a database called test with a collection called test and a view documents in it.
I use C# and the latest relesed driver 2.0.0 in order to connect to the database.
The code looke like this:
using System;
using System.Dynamic;
using System.Collections.Generic;
using MongoDB.Driver;
using MongoDB.Bson;
namespace mongotest
{
class MainClass
{
public static void Main (string[] args)
{
MongoClientSettings settings = new MongoClientSettings ();
settings.Server = new MongoServerAddress ("mongotest", 21017);
MongoClient client = new MongoClient (settings);
System.Threading.Tasks.Task<List<BsonDocument>> datasetsTask = client.GetDatabase ("test").GetCollection<BsonDocument> ("test").Find (x => true).ToListAsync();
foreach (BsonDocument dataset in datasetsTask.Result) {
try {
Console.WriteLine(dataset["_id"]);
try{
Console.WriteLine("\t" + dataset["test"]);
} catch(Exception e) {}
try{
Console.WriteLine("\t" + dataset["name"]);
} catch(Exception e) {}
} catch(Exception e) {Console.WriteLine ("Error: no property called _id");}
}
}
}
}
I checked out the database with robomongo. The test database and the test collection are there. In the line where I try to get the result (in the head of the foreach loop) the execution pauses for 30 seconds and then throws a System.TimeoutException (I had to post it into pastebin because it was too long): http://pastebin.com/pp0kfgaH
Down in line 58 is the real error I think. But I don't know how it comes to this server address.

Looks like a problem with the connection.
Try setting up the connection just like the "Getting started" example on the Github page:
https://github.com/mongodb/mongo-csharp-driver
Also, remember to add the "mongodb://" to your connection string.

WPF C# application will freeze my whole computer ever 2-3 times I run it

I put a lot of information in this issue because I have no idea what will be relavent
Issue:
I am having an issue with a program I am working on where when running it, it will freeze my whole computer and return no error (I am completely incapable of doing anything CTRL+ALT+DEL doesn't even work). This program accepts a connection from a android client and atm the android client is not configured correctly so the connection is being rejected.
Question:
How can I stop my program from freezing my entire machine?
Conjecture:
I have a few theories as to what is going on but no idea how to fix them. I have read that this may have something to do with me running a single threaded process inside my async worker but I am not sure that the socket is a single threaded process. Also I am not entirely sure how I am supposed to deal with exceptions in a backgroundworker so I just let it fall back to the RunWorkerCompletedEventArgs then retrieve the error message from there.
What I have tried:
- I tried putting try catches every where then removing try catches nothing seems to be able to capture this error
- I checked my systems event log and nothing is showing up except my restarts after my computer freezes
- I have attempted to isolate the issue but it can literally happen at any point from the program starting till when I attempt to connect
Setup:
I am running the program out of visual studio 2012 professional on a windows 8 pro machine. The computer I am on has a i7-3770K 3.50GHz and 32GB of ram. The application that is attempting to make a connection to mine is a Android application and the credentials are incorrect when it is attempting to connect. Visual Studio is running off my main hard drive and building the project on another drive.
Closing:
With all that said would some one please be willing to help me? If you need any more information I will be happy to provide it, please ask.
Main Method:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Data;
using System.Windows.Documents;
using System.Windows.Input;
using System.Windows.Media;
using System.Windows.Media.Imaging;
using System.Windows.Navigation;
using System.Windows.Shapes;
using System.ComponentModel;
namespace Server
{
/// <summary>
/// Interaction logic for MainWindow.xaml
/// </summary>
public partial class SourceServer : Window
{
private BackgroundWorker worker = new BackgroundWorker();
public SourceServer()
{
InitializeComponent();
StartListeningForConnections();
}
private void StartListeningForConnections()
{
worker.DoWork += new DoWorkEventHandler(worker_DoWork);
worker.RunWorkerCompleted += new RunWorkerCompletedEventHandler(worker_RunWorkerCompleted);
worker.ProgressChanged += new ProgressChangedEventHandler(worker_ProgressChanged);
worker.WorkerReportsProgress = true;
if (worker.IsBusy != true)
{
worker.RunWorkerAsync();
}
}
private void worker_DoWork(object sender, DoWorkEventArgs e)
{
worker.ReportProgress(0, "Source server version 0.0.0.1ib started");
LoginServer oLoginServer = new LoginServer();
oLoginServer.StartListening(worker);
}
private void worker_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
try
{
lvOutput.Items.Add(e.UserState.ToString());
}
catch (Exception exception)
{
lvOutput.Items.Add(exception.StackTrace);
}
}
private void worker_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
if (e.Error != null)
{
System.IO.File.WriteAllText(Environment.CurrentDirectory + #"\log.txt", e.Error.StackTrace + " /n " + e.Error.Message);
}
else
{
MessageBox.Show("Error was null");
}
worker.Dispose();
}
}
}
SSL Socket Connection:
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Threading;
using System.Net;
using System.Net.Sockets;
using System.Windows;
using System.Windows.Controls;
using System.ComponentModel;
using System.Net.Security;
using System.Security.Cryptography.X509Certificates;
using MySql.Data.MySqlClient;
using System.IO;
namespace Server
{
public class LoginServer
{
// Incoming data from the client.
public static string data = null;
public static X509Certificate serverCertificate = null;
public delegate void UpdateListView(ListView oOutput);
public void StartListening(BackgroundWorker worker)
{
// Data buffer for incoming data.
byte[] bytes = new Byte[1024];
// Establish the local endpoint for the socket.
IPHostEntry ipHostInfo = Dns.GetHostEntry(Dns.GetHostName());
IPAddress ipAddress = ipHostInfo.AddressList[1];
serverCertificate = X509Certificate.CreateFromCertFile(#"server.crt");
TcpListener oServer = new TcpListener(ipAddress, 12345);
// Bind the socket to the local endpoint and
// listen for incoming connections.
// Start listening for connections.
while (true)
{
Thread.Sleep(100);
worker.ReportProgress(0, "Waiting for connection....");
// Program is suspended while waiting for an incoming connection.
//Socket handler = listener.Accept();
oServer.Start();
TcpClient oClient = oServer.AcceptTcpClient();
Stream oStream = oClient.GetStream();
SslStream oSSLStream = new SslStream(oStream);
data = null;
// An incoming connection needs to be processed.
string sUsername = "place holder";
string sPassword = "place holder";
while (true)
{
bytes = new byte[1024];
int bytesRec = oSSLStream.Read(bytes, 0, bytes.Length);
data += Encoding.ASCII.GetString(bytes, 0, bytesRec);
string[] sCredentials = data.Split("|".ToCharArray()[0]);
sUsername = sCredentials[0];
sPassword = sCredentials[1];
if (data.IndexOf("<EOF>") > -1)
{
break;
}
}
// Show the data on the console.
worker.ReportProgress(0, "Connection Recieved : ");
worker.ReportProgress(0, "Username: " + sUsername);
worker.ReportProgress(0, "Password: " + sPassword);
worker.ReportProgress(0, "");
// Echo the data back to the client.
byte[] msg;
if (sUsername.Equals("test") && sPassword.Equals("test"))
{
msg = Encoding.ASCII.GetBytes("approved<EOF>\n");
worker.ReportProgress(0, "approved");
oSSLStream.Write(msg, 0, msg.Length);
}
else
{
msg = Encoding.ASCII.GetBytes("rejected<EOF>\n");
worker.ReportProgress(0, "rejected");
oSSLStream.Write(msg, 0, msg.Length);
}
}
}
public void VerifyUser()
{
}
}
}

While I don't see any reason for this to lock up your entire computer, I do see a couple of reasons for the application to potentially hang...
Your while loop inside of your SSL server will never break unless your client writes '<EOF>' to the stream; which you would have to force it to do. I would likely do something similar to this:
while(( bytesRec = oSSLStream.Read(bytes,0,bytes.Length)) > 0 )
{
// Compare input & break
}
-- The while loop you have now ( without a thread sleep ) will consume all of your systems resources waiting for ... something that may never occur.
In a related issue - I note that your 'DoWork' method launches the listener - but does not start a new thread for this listener. This means that the listener is running inside of your interface thread - which will cause the interface ( and potentially more... ) to hang until the process is completed - which as stated, may never happen.
... Ahem... This last paragraph may be incorrect - you are running an async worker, so I may be incorrect in my second assessment.
Cheers, hope this is helpful.

I've had some hanging problems on Windows 8 that I never saw on Windows 7 (with VS2012). As you experienced it worked fine the first time but only locked up Visual Studio (and not my whole machine) and I had to force quit.
The Visual Studio 2012 Update 4 (which focuses on bug fixes and compatibility) seemed to fix it, although I didn't scientifically test this.
Note: As of 9/1/13 this is only the RC2 version so please check for newer versions, and edit this answer when RTM happens.