We Keep Coding

ZeroMQ performance issue

I'm having an issue with ZeroMQ, which I believe is because I'm not very familiar with it.
I'm trying to build a very simple service where multiple clients connect to a server and sends a query. The server responds to this query.
When I use REQ-REP socket combination (client using REQ, server binding to a REP socket) I'm able to get close to 60,000 messages per second at server side (when client and server are on the same machine). When distributed across machines, each new instance of client on a different machine linearly increases the messages per second at the server and easily reaches 40,000+ with enough client instances.
Now REP socket is blocking, so I followed ZeroMQ guide and used the rrbroker pattern (http://zguide.zeromq.org/cs:rrbroker):
REQ (client) <----> [server ROUTER -- DEALER --- REP (workers running on different threads)]
However, this completely screws up the performance. I'm getting only around 4000 messages per second at the server when running across machines. Not only that, each new client started on a different machine reduces the throughput of every other client.
I'm pretty sure I'm doing something stupid. I'm wondering if ZeroMQ experts here can point out any obvious mistakes. Thanks!
Edit: Adding code as per advice. I'm using the clrzmq nuget package (https://www.nuget.org/packages/clrzmq-x64/)
Here's the client code. A timer counts how many responses are received every second.
for (int i = 0; i < numTasks; i++) { Task.Factory.StartNew(() => Client(), TaskCreationOptions.LongRunning); }
void Client()
{
using (var ctx = new Context())
{
Socket socket = ctx.Socket(SocketType.REQ);
socket.Connect("tcp://192.168.1.10:1234");
while (true)
{
socket.Send("ping", Encoding.Unicode);
string res = socket.Recv(Encoding.Unicode);
}
}
}
Server - case 1: The server keeps track of how many requests are received per second
using (var zmqContext = new Context())
{
Socket socket = zmqContext.Socket(SocketType.REP);
socket.Bind("tcp://*:1234");
while (true)
{
string q = socket.Recv(Encoding.Unicode);
if (q.CompareTo("ping") == 0) {
socket.Send("pong", Encoding.Unicode);
}
}
}
With this setup, at server side, I can see around 60,000 requests received per second (when client is on the same machine). When on different machines, each new client increases number of requests received at server as expected.
Server Case 2: This is essentially rrbroker from ZMQ guide.
void ReceiveMessages(Context zmqContext, string zmqConnectionString, int numWorkers)
{
List<PollItem> pollItemsList = new List<PollItem>();
routerSocket = zmqContext.Socket(SocketType.ROUTER);
try
{
routerSocket.Bind(zmqConnectionString);
PollItem pollItem = routerSocket.CreatePollItem(IOMultiPlex.POLLIN);
pollItem.PollInHandler += RouterSocket_PollInHandler;
pollItemsList.Add(pollItem);
}
catch (ZMQ.Exception ze)
{
Console.WriteLine("{0}", ze.Message);
return;
}
dealerSocket = zmqContext.Socket(SocketType.DEALER);
try
{
dealerSocket.Bind("inproc://workers");
PollItem pollItem = dealerSocket.CreatePollItem(IOMultiPlex.POLLIN);
pollItem.PollInHandler += DealerSocket_PollInHandler;
pollItemsList.Add(pollItem);
}
catch (ZMQ.Exception ze)
{
Console.WriteLine("{0}", ze.Message);
return;
}
// Start the worker pool; cant connect
// to inproc socket before binding.
workerPool.Start(numWorkers);
while (true)
{
zmqContext.Poll(pollItemsList.ToArray());
}
}
void RouterSocket_PollInHandler(Socket socket, IOMultiPlex revents)
{
RelayMessage(routerSocket, dealerSocket);
}
void DealerSocket_PollInHandler(Socket socket, IOMultiPlex revents)
{
RelayMessage(dealerSocket, routerSocket);
}
void RelayMessage(Socket source, Socket destination)
{
bool hasMore = true;
while (hasMore)
{
byte[] message = source.Recv();
hasMore = source.RcvMore;
destination.Send(message, message.Length, hasMore ? SendRecvOpt.SNDMORE : SendRecvOpt.NONE);
}
}
Where the worker pool's start method is:
public void Start(int numWorkerTasks=8)
{
for (int i = 0; i < numWorkerTasks; i++)
{
QueryWorker worker = new QueryWorker(this.zmqContext);
Task task = Task.Factory.StartNew(() =>
worker.Start(),
TaskCreationOptions.LongRunning);
}
Console.WriteLine("Started {0} with {1} workers.", this.GetType().Name, numWorkerTasks);
}
public class QueryWorker
{
Context zmqContext;
public QueryWorker(Context zmqContext)
{
this.zmqContext = zmqContext;
}
public void Start()
{
Socket socket = this.zmqContext.Socket(SocketType.REP);
try
{
socket.Connect("inproc://workers");
}
catch (ZMQ.Exception ze)
{
Console.WriteLine("Could not create worker, error: {0}", ze.Message);
return;
}
while (true)
{
try
{
string message = socket.Recv(Encoding.Unicode);
if (message.CompareTo("ping") == 0)
{
socket.Send("pong", Encoding.Unicode);
}
}
catch (ZMQ.Exception ze)
{
Console.WriteLine("Could not receive message, error: " + ze.ToString());
}
}
}
}

Could you post some source code or at least a more detailed explanation of your test case? In general the way to build out your design is to make one change at a time, and measure at each change. You can always move stepwise from a known working design to more complex ones.

Most probably the 'ROUTER' is the bottleneck.
Check out these related questions on this:
Client maintenance in ZMQ ROUTER
Load testing ZeroMQ (ZMQ_STREAM) for finding the maximum simultaneous users it can handle
ROUTER (and ZMQ_STREAM, which is just a variant of ROUTER) internally has to maintain the client mapping, hence IMO it can accept limited connections from a particular client. It looks like ROUTER can multiplex multiple clients, only as long as, each client has only one active connection.
I could be wrong here - but I am not seeing much proof to the contrary (simple working code that scales to multi-clients with multi-connections with ROUTER or STREAM).
There certainly is a very severe restriction on concurrent connections with ZeroMQ, though it looks like no one know what is causing it.

I have done done performance testing on calling a native unmanaged DLL function with various methods from C#:
1. C++/CLI wrapper
2. PInvoke
3. ZeroMQ/clrzmq
The last might be interesting for you.
My finding at the end of my performance test was that using the ZMQ binding clrzmq was not useful and produced a factor of 100 performance overhead after I tried to optimize the PInvoke calls within the source code of the binding. Therefore I have used the ZMQ without a binding but with PInvoke calls.these calls must be done with the cdecl convention and with the option "SuppressUnmanagedCodeSecurity" to get most speed.
I had to import just 5 functions which was fairly easy.
At the end the speed was a bit slower than a PInvoke call but with the ZMQ-in my case over "inproc".
This may give you the hint to try it without the binding, if speed is interesting for you.
This is not a direct answer for your question but may help you to increase performance in general.

C# - Avoid getting a SocketException

I was wondering if there's a way to avoid getting a SocketException whenever I cannot connect rather than catching the SocketException using try/catch.
I have this code which checks if a server is available of not:
public bool CheckServerStatus(string IP, int Port)
{
try
{
IPAddress[] IPs = Dns.GetHostAddresses(IP);
using (Socket s = new Socket(AddressFamily.InterNetwork,
SocketType.Stream,
ProtocolType.Tcp))
s.Connect(IPs[0], Port);
return true;
}
catch (SocketException)
{
return false;
}
}
Thanks in advance.

You may subclass Socket and provide your specific implementation:
public class MySocket : Socket{
//...
public boolean TryConnect(...){
}
}
You could also instead of a boolean, return a Result object that save the exception for error handling:
public class Result {
public Exception Error { get; set; }
public boolean Success { get{ return Error != null; } }
}

Getting a SocketException isn't a problem; this is what the exception should be used for. Depending on the type of exception you get, you can handle them different ways. It would have been bad if you just caught Exception rather than the more specific SocketException.
Why do you want to avoid it so much? As the comments say, at some point, somewhere, code will fail if the other end of the connection is not available. Just make sure you catch that failure at the appropriate place, like you appear to be doing now.

I managed to accomplish this using BeginConnect as follows
int connectTimeoutMS = 1000;
IPEndPoint endPoint = GetEndPoint();
var evt = new AutoResetEvent(false);
_socket.BeginConnect(endPoint, (AsyncCallback)delegate { evt.Set(); }, null);
evt.WaitOne(connectTimeoutMS);

Maybe you can solve the problem in the first place by using Ahmed approach, but this simply moves the problem a lever deeper.
The main reason why there exists no such a test method is the possibility of a race condition. Just imagine you would check if such a socket is possible and before you can try to establish this socket in the next line a context switch happens (to another thread or application) that just allocates this socket for himself. Now you still get the exception and you have to check for it (by using the try-catch approach).
So this test simply adds no benefit to your code, cause you still have to be prepared for a failing of this method. And that's the reason with this test doesn't exist.

Threading - Multiple SerialPort and one DataGridView

I have 3 SerialPort components and 2 of them has DataReceived handlers.
When data is received, they are sending output through other SerialPort
(the third one).
I know that the possibility is not high in my application that both SerialPorts will try to send at the same time, but I tried to simulate this and of course I get an error that the COM PORT is being used.
private void sendToCOM(String comNumber, String msg, String speed)
{
try
{
using (SerialPort comPort = new SerialPort(comNumber, Int32.Parse(speed), Parity.None, 8, StopBits.One))
{
comPort.Open();
comPort.Write(msg);
comPort.Close();
comPort.Dispose();
}
}
catch(Exception ex)
{
cstFuncs.errorHandler(ex.Message, SettingsForm);
}
}
I understand that each SerialPort has it's own thread.
How can I make a queue or how can I prevent from both serial port to try access the same resources on the same time (and not loose the data that is supposed to be sent out)

There are a few ways you could get locking in the sendToCOM method. This will stop the method from executing twice by causing any subsequent calls to wait until the first is complete. This isn't going to help if another process tried to use the same SerialPort, but it will stop any local contention.
One way would be to use the lock keyword on an object like so:
private readonly object _myLock = new object();
private void sendToCOM(String comNumber, String msg, String speed)
{
try
{
lock(_myLock)
{
using (SerialPort comPort = new SerialPort(comNumber, Int32.Parse(speed), Parity.None, 8, StopBits.One))
{
comPort.Open();
comPort.Write(msg);
comPort.Close();
comPort.Dispose();
}
}
}
catch(Exception ex)
{
cstFuncs.errorHandler(ex.Message, SettingsForm);
}
}
Another way could be to use a ManualResetEventSlim (or something similar) to achieve the same effect:
private ManualResetEventSlim _myLock = new ManualResetEventSlim(true);
private void sendToCOM(String comNumber, String msg, String speed)
{
_myLock.Wait();
try
{
_myLock.Reset();
using (SerialPort comPort = new SerialPort(comNumber, Int32.Parse(speed), Parity.None, 8, StopBits.One))
{
// ..
}
}
catch(Exception ex)
{
cstFuncs.errorHandler(ex.Message, SettingsForm);
}
finally
{
_myLock.Set();
}
}

You probably don't have a thread per serial port, but rather the DataReceived handler is executed in the thread-pool. It still means you can get two reads at once.
You can use locking to solve this:
Add a OutputPortLock object to your class:
private object OutputPortLock = new object();
And try to lock it before writing to the output port:
lock(OutputPortLock)
{
using(SerialPort comPort=....)
{
... write to com port ...
}
}
This will make sure only one thread tries to write to the output port at a time. You might still get Com Port in Use errors if some other process tries to use it.

Theres a few things you can do;
1) Build a lock mechanism. Threads will wait until this lock is unlocked before trying to access the method, meaning that each of the threads will take it in turn to fire off "sendToCOM"
2) Another method is to use a ConcurrentQueue (see: http://msdn.microsoft.com/en-us/library/dd267265.aspx). This is a thread safe list that you could, if you wanted, add COMport objects too, and fire them off from there. Theres a few questions around about doing so, for example;
How to work threading with ConcurrentQueue<T>
I'm sure theres more ways, but these are 2 i've thought of off the top of my head. I'd say the second is more reliable, but requires more work to get running, so the first may be your best interest.

Tcpclient multiple clients connect to server

A client need to build several tcp connections to server simultaneously.
My Server's code is below.
while (_running)
{
if (!_listener.Pending())
{
Thread.Sleep(100);
continue;
}
TcpClient client = _listener.AcceptTcpClient();
}
And my client's code is below.
for (int i = 0; i < num; i++)
{
TcpClient tcp = new TcpClient();
tcp.Connect(_server);
}
The first connection is success. But the second connection is failed due to server's no response(Actually server are listening tcp connection).
However, if I add Thread.Sleep(1500) after each of tcp.Connect(), all connections are success. But this situation is only true if there are one client and one server. If there are many clients then how can I ensure each connection that can be accepted by server? Also why I add Thread.Sleep can make such connections succeed?

I had the same task. I looked for canonical implementation of this task for .Net with no luck.
The approach I use now is descibed below.
Main idea
We need listener to receive connection, give the connection to the handler, and as soon as possible start listen for a new connection.
Implementation
AutoResetEvent _stopEvent = new AutoResetEvent(false);
object _lock = new object();
public void StartListening()
{
_listener.BeginAcceptTcpClient(ConnectionHandler, null);
_stopEvent.WaitOne();//this part is different in my original code, I don't wait here
}
public void StopListening()
{
lock(_lock)
{
listener.Stop();
listener = null;
}
_stopEvent.Set();//this part is different in my original code
}
void ConnectionHandler(IAsyncResult asyncResult)
{
lock(_lock)
{
if(_listener == null)
return;
var tcpClient = _listener.EndAcceptTcpClient(asyncResult);
var task = new MyCustomTask(tcpClient);
ThreadPool.QueueUserWorkItem(task.Execute);
_listener.BeginAcceptTcpClient(ConnectionHandler,null);
}
}
I am still not very confident in calling _listener.BeginAcceptTcpClient in ConnectionHandler, but I haven't found alternative way.

Since there are still no satisfied answers and I finally use different approach to handle my case. I found out that using class Socket is faster and more stable than using TcpListener and TcpClient. I tried different approach to use TcpListener and TcpClient. Firstly, I used TcpListener.AcceptTcpClient to listen client with and without TcpListener.Pending but there is still possibly ignoring some client connection. Sencondly, I used asynchronous method, TcpListener.BeginAcceptTcpClient and TcpListener.EndAcceptTcpClient but still no succeeded, still ignoring some client connection. Finally using Socket.Accept instead of TcpListener.AcceptTcpClient, the former one has nearly no delay and really fast to response to client.

Socket connection issues

My answer:
After getting annoyed, I have found a solution. The problem was indeed C# either C#'s garbage collector or C#'s multithreading, it probably thought the object was no longer needed within THAT thread, and deleted it. The solution was found as follows:
I implemented the ClientThread into the Server class, passing the Client object as a parameters, this minor change made it work. Thank you for all your responses, if anyone in the future has this problem maybe it wasn't C#'s garbage collector. But C# mutithreading OR networking must be done within the same class. I kept my client class and just made the thread object run the function within the Server class.
If anyone can figure out what my problem was, feel free to comment so I can expand my little knowledge of C#'s memory management.
Thanks again to all the people who attempted to help me in this thread.
Original Question
I'm a C++ programmer so I'm used to managing memory myself, and I'm really not sure how to solve this problem.
For instance in C++:
while(true)
{
void* ptr = new char[1000];
}
This would be an obvious memory leaking program, so I need to go ahead and clean it up with:
delete ptr;
But there are cases when I want to create memory for use in a different thread and I DO NOT WANT IT DELETED AFTER THE LOOP.
while(true)
{
socket.Accept(new Client());
}
//////////Client Constructor////////////
Client()
{
clientThread.Start();
}
This snippet is basically what I want to do in C#, but my client connects then disconnects immediately, I'm assuming this is because at the end of the while loop my new Client() is being deleted by our favorite Garbage Collector.
So my question is, how do I get around this and make it NOT delete my object.
Many have replied saying various things about having other links to it in the code. I forgot to mention that I also save the new client in a list of clients located globally
List<Client> clients;
//inside loop
clients.Add(new Client(socket.Accept()));
Ok because I'm unsure if I'm missing more information here is the ACTUAL code snippet
// Server class
internal Socket socket { get; set; }
internal Thread thread { get; set; }
internal List<Client> clients { get; set; }
internal void Init()
{
socket = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
thread = new Thread(AcceptThread);
}
internal void Start(int port,List<Client> clients)
{
var ipep = new IPEndPoint(IPAddress.Any, port);
this.socket.Bind(ipep);
this.socket.Listen(10);
this.clients = clients;
this.thread.Start();
}
internal void End()
{
socket.Close();
thread.Abort();
}
internal void AcceptThread()
{
int ids = 0;
while (true)
{
Client client = new Client();
client.Init(socket.Accept());
client.clientid = ids++;
client.Start();
clients.Add(client);
}
}
// Client class
public class Client
{
.....
#region Base Code
internal void Init(Socket socket)
{
this.socket = socket;
this.status = new SocketStatus();
this.thread = new Thread(ClientThread);
this.stream = new Stream();
}
internal void Start()
{
thread.Start();
}
internal void Close()
{
socket.Close();
status = SocketStatus.Null;
thread.Abort();
}
internal void ClientThread()
{
try
{
while (true)
{
byte[] data = new byte[1];
int rec = socket.Receive(data);
if (rec == 0)
throw new Exception();
else
stream.write(data[0]);
}
}
catch(Exception e)
{
Close();
}
}
#endregion
}
I thank you for all your replies.

That's not how it works at all. If there exists any reference to the instance of Client you created, it is not garbage-collected. This doesn't just apply to your own code, either. Therefore, if GCing is indeed the source of your issue, you never could have accessed it in the first place!
If you weren't intending to access it, you can hold on to them anyway by putting them in a List. However, I believe that once you actually use them in the other thread you're talking about, your problems will go away.

I've been out of the c# game for a while but I don't see anything immediately wrong there. Garbage collection shouldn't kick in until objects are actually not referenced anymore. if your socket.Accept() doesn't keep a reference, perhaps you could do this manually:
var clients = new List<Client>();
while(true)
{
client = new Client();
clients.Add(client);
socket.Accept(client);
}
////////// Client Constructor ////////////
Client()
{
clientThread.Start();
}

From MSDN:
If no data is available for reading, the Receive method will block until data is
available, unless a time-out value was set by using
Socket.ReceiveTimeout. If the time-out value was exceeded, the Receive
call will throw a SocketException. If you are in non-blocking mode,
and there is no data available in the in the protocol stack buffer,
the Receive method will complete immediately and throw a
SocketException. You can use the Available property to determine if
data is available for reading. When Available is non-zero, retry the
receive operation.
If you are using a connection-oriented Socket, the Receive method will
read as much data as is available, up to the size of the buffer. If
the remote host shuts down the Socket connection with the Shutdown
method, and all available data has been received, the Receive method
will complete immediately and return zero bytes.
This appears to be the only way to get a 0 return value from the Receive method, and not get an exception, so it would appear that whatever is on the other end is closing the connection.

The garbage collector only deletes resources that aren't reachable through any reference in your program. As long as you still have a variable that refers to the object, it'll continue to exist.