I'm having an issue with ZeroMQ, which I believe is because I'm not very familiar with it.
I'm trying to build a very simple service where multiple clients connect to a server and sends a query. The server responds to this query.
When I use REQ-REP socket combination (client using REQ, server binding to a REP socket) I'm able to get close to 60,000 messages per second at server side (when client and server are on the same machine). When distributed across machines, each new instance of client on a different machine linearly increases the messages per second at the server and easily reaches 40,000+ with enough client instances.
Now REP socket is blocking, so I followed ZeroMQ guide and used the rrbroker pattern (http://zguide.zeromq.org/cs:rrbroker):
REQ (client) <----> [server ROUTER -- DEALER --- REP (workers running on different threads)]
However, this completely screws up the performance. I'm getting only around 4000 messages per second at the server when running across machines. Not only that, each new client started on a different machine reduces the throughput of every other client.
I'm pretty sure I'm doing something stupid. I'm wondering if ZeroMQ experts here can point out any obvious mistakes. Thanks!
Edit: Adding code as per advice. I'm using the clrzmq nuget package (https://www.nuget.org/packages/clrzmq-x64/)
Here's the client code. A timer counts how many responses are received every second.
for (int i = 0; i < numTasks; i++) { Task.Factory.StartNew(() => Client(), TaskCreationOptions.LongRunning); }
void Client()
{
using (var ctx = new Context())
{
Socket socket = ctx.Socket(SocketType.REQ);
socket.Connect("tcp://192.168.1.10:1234");
while (true)
{
socket.Send("ping", Encoding.Unicode);
string res = socket.Recv(Encoding.Unicode);
}
}
}
Server - case 1: The server keeps track of how many requests are received per second
using (var zmqContext = new Context())
{
Socket socket = zmqContext.Socket(SocketType.REP);
socket.Bind("tcp://*:1234");
while (true)
{
string q = socket.Recv(Encoding.Unicode);
if (q.CompareTo("ping") == 0) {
socket.Send("pong", Encoding.Unicode);
}
}
}
With this setup, at server side, I can see around 60,000 requests received per second (when client is on the same machine). When on different machines, each new client increases number of requests received at server as expected.
Server Case 2: This is essentially rrbroker from ZMQ guide.
void ReceiveMessages(Context zmqContext, string zmqConnectionString, int numWorkers)
{
List<PollItem> pollItemsList = new List<PollItem>();
routerSocket = zmqContext.Socket(SocketType.ROUTER);
try
{
routerSocket.Bind(zmqConnectionString);
PollItem pollItem = routerSocket.CreatePollItem(IOMultiPlex.POLLIN);
pollItem.PollInHandler += RouterSocket_PollInHandler;
pollItemsList.Add(pollItem);
}
catch (ZMQ.Exception ze)
{
Console.WriteLine("{0}", ze.Message);
return;
}
dealerSocket = zmqContext.Socket(SocketType.DEALER);
try
{
dealerSocket.Bind("inproc://workers");
PollItem pollItem = dealerSocket.CreatePollItem(IOMultiPlex.POLLIN);
pollItem.PollInHandler += DealerSocket_PollInHandler;
pollItemsList.Add(pollItem);
}
catch (ZMQ.Exception ze)
{
Console.WriteLine("{0}", ze.Message);
return;
}
// Start the worker pool; cant connect
// to inproc socket before binding.
workerPool.Start(numWorkers);
while (true)
{
zmqContext.Poll(pollItemsList.ToArray());
}
}
void RouterSocket_PollInHandler(Socket socket, IOMultiPlex revents)
{
RelayMessage(routerSocket, dealerSocket);
}
void DealerSocket_PollInHandler(Socket socket, IOMultiPlex revents)
{
RelayMessage(dealerSocket, routerSocket);
}
void RelayMessage(Socket source, Socket destination)
{
bool hasMore = true;
while (hasMore)
{
byte[] message = source.Recv();
hasMore = source.RcvMore;
destination.Send(message, message.Length, hasMore ? SendRecvOpt.SNDMORE : SendRecvOpt.NONE);
}
}
Where the worker pool's start method is:
public void Start(int numWorkerTasks=8)
{
for (int i = 0; i < numWorkerTasks; i++)
{
QueryWorker worker = new QueryWorker(this.zmqContext);
Task task = Task.Factory.StartNew(() =>
worker.Start(),
TaskCreationOptions.LongRunning);
}
Console.WriteLine("Started {0} with {1} workers.", this.GetType().Name, numWorkerTasks);
}
public class QueryWorker
{
Context zmqContext;
public QueryWorker(Context zmqContext)
{
this.zmqContext = zmqContext;
}
public void Start()
{
Socket socket = this.zmqContext.Socket(SocketType.REP);
try
{
socket.Connect("inproc://workers");
}
catch (ZMQ.Exception ze)
{
Console.WriteLine("Could not create worker, error: {0}", ze.Message);
return;
}
while (true)
{
try
{
string message = socket.Recv(Encoding.Unicode);
if (message.CompareTo("ping") == 0)
{
socket.Send("pong", Encoding.Unicode);
}
}
catch (ZMQ.Exception ze)
{
Console.WriteLine("Could not receive message, error: " + ze.ToString());
}
}
}
}
Could you post some source code or at least a more detailed explanation of your test case? In general the way to build out your design is to make one change at a time, and measure at each change. You can always move stepwise from a known working design to more complex ones.
Most probably the 'ROUTER' is the bottleneck.
Check out these related questions on this:
Client maintenance in ZMQ ROUTER
Load testing ZeroMQ (ZMQ_STREAM) for finding the maximum simultaneous users it can handle
ROUTER (and ZMQ_STREAM, which is just a variant of ROUTER) internally has to maintain the client mapping, hence IMO it can accept limited connections from a particular client. It looks like ROUTER can multiplex multiple clients, only as long as, each client has only one active connection.
I could be wrong here - but I am not seeing much proof to the contrary (simple working code that scales to multi-clients with multi-connections with ROUTER or STREAM).
There certainly is a very severe restriction on concurrent connections with ZeroMQ, though it looks like no one know what is causing it.
I have done done performance testing on calling a native unmanaged DLL function with various methods from C#:
1. C++/CLI wrapper
2. PInvoke
3. ZeroMQ/clrzmq
The last might be interesting for you.
My finding at the end of my performance test was that using the ZMQ binding clrzmq was not useful and produced a factor of 100 performance overhead after I tried to optimize the PInvoke calls within the source code of the binding. Therefore I have used the ZMQ without a binding but with PInvoke calls.these calls must be done with the cdecl convention and with the option "SuppressUnmanagedCodeSecurity" to get most speed.
I had to import just 5 functions which was fairly easy.
At the end the speed was a bit slower than a PInvoke call but with the ZMQ-in my case over "inproc".
This may give you the hint to try it without the binding, if speed is interesting for you.
This is not a direct answer for your question but may help you to increase performance in general.
Related
I need to build an application (a server) that handles data sent from a Client via TCP. I must be able to support (at least) 2000 connections. I've made an attempt to write the TCP Server, but I find when I start to reach 600/700 connections, that the response from my server slows greatly (it actually slows down over time as more and more connections are received). I don't normally write networking code so I'm sure there are many concepts I've not fully comprehended and could be improved upon.
The main purpose of my server is to:
Handle data sent from clients and store it in a sql database.
Decide (based
upon the last message received) what the correct response should be to the client.
Queue up a list of responses and
send them to the client one at a time.
This needs to happen for all clients. Below is the code I have implemented:
private readonly TcpListener tcpListener;
private readonly Thread listenThread;
private bool run = true;
public Server()
{
this.tcpListener = new TcpListener(IPAddress.Any, AppSettings.ListeningPort); //8880 is default for me.
this.listenThread = new Thread(new ThreadStart(ListenForClients));
this.listenThread.Start();
}
private void ListenForClients()
{
this.tcpListener.Start();
while (run) {
TcpClient client = this.tcpListener.AcceptTcpClient();
//create a thread to handle communication with connected client
Thread clientThread = new Thread(new ParameterizedThreadStart(HandleClientComm));
clientThread.Start(client);
}
}
private void HandleClientComm(object client)
{
Queue responseMessages = new Queue();
while (run)
{
try
{
lastMessage = clientStream.GetMessage();
if (lastMessage.Length > 0)
{
// Process logic here...
//an item may be added to the response queue, depending on logic.
}
if (responseMessages.Count > 0)
{
clientStream.WriteMessage(msg);
clientStream.Flush();
// sleep for 250 milliseconds (or whats in the config).
Thread.Sleep(AppSettings.MessageSendDelayMilliseconds);
}
}
catch (Exception ex)
{
break;
}
}
tcpClient.Close();
}
And finally, here's an extension class I wrote to help me out:
namespace System.Net.Sockets
{
using System.Text;
public static class NetworkSteamExtension
{
private static readonly ASCIIEncoding Encoder = new ASCIIEncoding();
public static string GetMessage(this NetworkStream clientStream)
{
string message = string.Empty;
try
{
byte[] bMessage = new byte[4068];
int bytesRead = 0;
while (clientStream.DataAvailable)
{
bytesRead = clientStream.Read(bMessage, 0, 4068);
message += Encoder.GetString(bMessage, 0, bytesRead);
}
}
catch (Exception)
{
}
return message;
}
public static void WriteMessage(this NetworkStream clientStream, string message)
{
byte[] buffer = Encoder.GetBytes(message);
clientStream.Write(buffer, 0, buffer.Length);
clientStream.Flush();
}
}
}
Lots of articles on the subject people are using sockets instead. I've also read that .net 4.5 async / await is the best way to implement a solution.
I would like to make sure I take advantage of the newest features in .net (even 4.5.2 if it will help) and build a server that can handle at least 2000 connections. Can someone advise?
Many thanks!
OK, we need to fix some API usage errors and then the main problem of creating too many threads. It is well established that many connections can only be handled efficiently with async IO. Hundreds of connections counts as "too many".
Your client processing loop must be async. Rewrite HandleClientComm so that it uses async socket IO. This is easy with await. You need to search the web for ".net socket await".
You can continue to use synchronous accept calls. No downside there. Keep the simple synchronous code. Only make async those calls that have a significant avgWaitTime * countPerSecond product. That will be all socket calls, typically.
You are assuming that DataAvailable returns you the number of bytes in any given message. TCP does not preserve message boundaries. You need to do that youself. The DataAvailable value is almost meaningless because it can underestimate the true data that will arrive in the future.
It's usually better to use a higher level serialization protocol. For example, protobuf with length prefix. Don't use ASCII. You probably have done that only because you didn't know how to do it with a "real" encoding.
Dispose all resources using using. Don't use the non-generic Queue class. Don't flush streams, this does nothing with sockets.
Why are you sleeping?
I'm developing an application that manages devices in the network, at a certain point in the applicaiton, I must ping (actually it's not a ping, it's a SNMP get) all computers in the network to check if it's type is of my managed device.
My problem is that pinging all computers in the network is very slow (specially because most of them won't respond to my message and will simply timeout) and has to be done asynchronously.
I tried to use TLP to do this with the following code:
public static void FindDevices(Action<IPAddress> callback)
{
//Returns a list of all host names with a net view command
List<string> hosts = FindHosts();
foreach (string host in hosts)
{
Task.Run(() =>
{
CheckDevice(host, callback);
});
}
}
But it runs VERY slow, and when I paused execution I checked threads window and saw that it only had one thread pinging the network and was thus, running tasks synchronously.
When I use normal threads it runs a lot faster, but Tasks were supposed to be better, I'd like to know why aren't my Tasks optimizing parallelism.
**EDIT**
Comments asked for code on CheckDevice, so here it goes:
private static void CheckDevice(string host, Action<IPAddress> callback)
{
int commlength, miblength, datatype, datalength, datastart;
string output;
SNMP conn = new SNMP();
IPHostEntry ihe;
try
{
ihe = Dns.Resolve(host);
}
catch (Exception)
{
return;
}
// Send sysLocation SNMP request
byte[] response = conn.get("get", ihe.AddressList[0], "MyDevice", "1.3.6.1.2.1.1.6.0");
if (response[0] != 0xff)
{
// If response, get the community name and MIB lengths
commlength = Convert.ToInt16(response[6]);
miblength = Convert.ToInt16(response[23 + commlength]);
// Extract the MIB data from the SNMP response
datatype = Convert.ToInt16(response[24 + commlength + miblength]);
datalength = Convert.ToInt16(response[25 + commlength + miblength]);
datastart = 26 + commlength + miblength;
output = Encoding.ASCII.GetString(response, datastart, datalength);
if (output.StartsWith("MyDevice"))
{
callback(ihe.AddressList[0]);
}
}
}
Your issue is that you are iterating a none thread safe item the List.
If you replace it with a thread safe object like the ConcurrentBag you should find the threads will run in parallel.
I was a bit confused as to why this was only running one thread, I believe it is this line of code:
try
{
ihe = Dns.Resolve(host);
}
catch (Exception)
{
return;
}
I think this is throwing exceptions and returning; hence you only see one thread. This also ties into your observation that if you added a sleep it worked correctly.
Remember that when you pass a string your passing the reference to the string in memory, not the value. Anyway, the ConcurrentBag seems to resolve your issue. This answer might also be relevant
I am new to the windows socket programming using C# and i want to create an application that runs under multiple clients with one server. The server will wait for the incoming connection from client and assign a new port to each connection.
Server should accept the file transfer from multiple clients. The transferring file can be about 10-20 MB.
I went through many tutorials and examples but they do transfer in one-to-one pattern. I was able to connect the multiple clients to one server and sending the text through it. The server is accepting the clients connection and their sent text messages but I have no idea transferring the files in same pattern.
I will be a great help if there is any tutorials, examples or guide that help me understand the file transfer from multiple clients to single server.
it's quite easy actually
Save the incoming connections in an array.
start a new thread that gets the data from the socket and put it in an output buffer (make sure it's threadsafe) and that's it
edit 18-6-2014
here is a c++ example from my network class it is not perfect but you will get the idea
m_clientList is a vector in which I save the connections but I don't know if c# has a vector something like a list will work to
DWORD WINAPI Network::ServerAcceptConnections(LPVOID lpParam)
{
Network* network = (Network*) lpParam;
SOCKET new_socket;
char *message;
int c = sizeof(struct sockaddr_in);
DWORD dwWaitResult;
Sleep(100);
while (true)
{
new_socket = accept(pNetwork->m_s , (struct sockaddr *)&pNetwork->m_client, &c);
if (new_socket != INVALID_SOCKET )
{
dwWaitResult = WaitForSingleObject(
pNetwork->m_ClientListMutex, // handle to mutex
INFINITE); // no time-out interval
if (dwWaitResult == WAIT_OBJECT_0)
{
__try
{
//Reply to the client
if (network->m_clientList.size() < network->m_maxConnections)
{
if(network->m_debugModus) print("DEBUG MODUS: Connection accepted\r\n");
network->m_clientList.push_back(new_socket);
message = NETWORK_CLASS_CONNECTION_SUCCES;
Message out;
out.client = new_socket;
out.message = message;
out.size = strlen(message);
pNetwork->SendData(out);
}
else
{
if(pNetwork->m_debugModus) printf("DEBUG MODUS: Max connections reached\r\n");
message = NETWORK_CLASS_MAX_CONNECTIONS;
pNetwork->Send(new_socket, message, strlen(message));
closesocket(new_socket);
}
}
__finally {
// Release ownership of the mutex object
if (! ReleaseMutex(network->m_ClientListMutex))
{
if(pNetwork->m_debugModus) printf("DEBUG MODUS: AcceptConnections: Cant Relese the Mutex\r\n");
}
}
}
else if (dwWaitResult == WAIT_ABANDONED)
{
if(network->m_debugModus) print("DEBUG MODUS: SendDataThread: The thread got ownership of an abandoned mutex\r\n");
return FALSE;
}
}
else
{
if(pNetwork->m_debugModus) printf("DEBUG MODUS: accept failed with error code : %d\r\n" , WSAGetLastError());
}
}
return TRUE;
}
I working in a managed Windows Service written with C#. It keeps receiving messages from several clients connected over TCP/IP. The Client is basically a router that receive and resend messages from thermometers to the Server. The Server parse the messages and store them in a SQL Server database.
The problem I am facing is that some clients, suddenly, stops sending messages. But, as soon the service is restarted, they connect again and resume sending. I don't have the code of the Client since it is a third party device and I pretty sure the problem is with the Server.
I manage to reduce the problem by implementing a timer that keeps checking if each client is still connected (see code below). Also, I added a Keep Alive mode to the Socket, using the socket.IOControl(IOControlCode.KeepAliveValues, ...) method, but the problem still happening.
I'm posting some code from specific parts I consider relevant. But, if more snippets are needed to understand the problem, please ask me and I'll edit the post. All the try/catch blocks was removed to reduce the ammount of code.
I don't want a perfect solution, just any guidance will be appreciated.
private Socket _listener;
private ConcurrentDictionary<int, ConnectionState> _connections;
public TcpServer(TcpServiceProvider provider, int port)
{
this._provider = provider;
this._port = port;
this._listener = new Socket(AddressFamily.InterNetwork, SocketType.Stream, ProtocolType.Tcp);
this._connections = new ConcurrentDictionary<int, ConnectionState>();
ConnectionReady = new AsyncCallback(ConnectionReady_Handler);
AcceptConnection = new WaitCallback(AcceptConnection_Handler);
ReceivedDataReady = new AsyncCallback(ReceivedDataReady_Handler);
}
public bool Start()
{
this._listener.Bind(new IPEndPoint(IPAddress.Any, this._port));
this._listener.Listen(10000);
this._listener.BeginAccept(ConnectionReady, null);
}
// Check every 5 minutes for clients that have not send any message in the past 30 minutes
// MSG_RESTART is a command that the devices accepts to restart
private void CheckForBrokenConnections()
{
foreach (var entry in this._connections)
{
ConnectionState conn = entry.Value;
if (conn.ReconnectAttemptCount > 3)
{
DropConnection(conn);
continue;
}
if (!conn.Connected || (DateTime.Now - conn.LastResponse).TotalMinutes > 30)
{
byte[] message = HexStringToByteArray(MSG_RESTART);
if (!conn.WaitingToRestart && conn.Write(message, 0, message.Length))
{
conn.WaitingToRestart = true;
}
else
{
DropConnection(conn);
}
}
}
}
private void ConnectionReady_Handler(IAsyncResult ar)
{
lock (thisLock)
{
if (this._listener == null)
return;
ConnectionState connectionState = new ConnectionState();
connectionState.Connection = this._listener.EndAccept(ar);
connectionState.Server = this;
connectionState.Provider = (TcpServiceProvider)this._provider.Clone();
connectionState.Buffer = new byte[4];
Util.SetKeepAlive(connectionState.Connection, KEEP_ALIVE_TIME, KEEP_ALIVE_TIME);
int newID = (this._connections.Count == 0 ? 0 : this._connections.Max(x => x.Key)) + 1;
connectionState.ID = newID;
this._connections.TryAdd(newID, connectionState);
ThreadPool.QueueUserWorkItem(AcceptConnection, connectionState);
this._listener.BeginAccept(ConnectionReady, null);
}
}
private void AcceptConnection_Handler(object state)
{
ConnectionState st = state as ConnectionState;
st.Provider.OnAcceptConnection(st);
if (st.Connection.Connected)
st.Connection.BeginReceive(st.Buffer, 0, 0, SocketFlags.None, ReceivedDataReady, st);
}
private void ReceivedDataReady_Handler(IAsyncResult result)
{
ConnectionState connectionState = null;
lock (thisLock)
{
connectionState = result.AsyncState as ConnectionState;
connectionState.Connection.EndReceive(result);
if (connectionState.Connection.Available == 0)
return;
// Here the message is parsed
connectionState.Provider.OnReceiveData(connectionState);
if (connectionState.Connection.Connected)
connectionState.Connection.BeginReceive(connectionState.Buffer, 0, 0, SocketFlags.None, ReceivedDataReady, connectionState);
}
}
internal void DropConnection(ConnectionState connectionState)
{
lock (thisLock)
{
if (this._connections.Values.Contains(connectionState))
{
ConnectionState conn;
this._connections.TryRemove(connectionState.ID, out conn);
}
if (connectionState.Connection != null && connectionState.Connection.Connected)
{
connectionState.Connection.Shutdown(SocketShutdown.Both);
connectionState.Connection.Close();
}
}
}
2 things I think I see...
If this is a connection you keep for multiple messages, you probably should not return from ReceivedDataReady_Handler when connectionState.Connection.Available == 0 IIRC a 0 length data paket can be received. So if the connection is still open, you should call connectionState.Connection.BeginReceive( ... ) before leaving the handler.
(I hesitate to put this here because I do not remember specifics) There is an event you can handle that tells you when things happen to your underlying connection including errors and failures connecting or closing a connection. For the life of me I cannot remember the name(s)... This would likely be more efficient than a timer every few seconds. It also gives you a way to break out of connections stuck in the connecting or closing states.
Add try/catch blocks around all the IO calls, and write the errors to a log file. As it is, it can't recover on error.
Also, be careful with any lock that doesn't have a timeout. These operations should be given a reasonable TTL.
I have experienced these kind of situation many times. The problem is probably not with your code at all but with the network and the way Windows (on boths ends) or the routers handle the network. What happens quite often is that a temporary network outage "breaks" the socket, but Windows isn't aware of it, so it doesn't close the socket.
The only way to overcome this is exactly what you did - sending keep-alives and monitoring connection health. Once you recognize the the connection is down, you need to restart it. However, in your code you don't restart the listener socket which is also broken and can't accept new connections. That's why restarting the service helps, it restarts the listener.
A client need to build several tcp connections to server simultaneously.
My Server's code is below.
while (_running)
{
if (!_listener.Pending())
{
Thread.Sleep(100);
continue;
}
TcpClient client = _listener.AcceptTcpClient();
}
And my client's code is below.
for (int i = 0; i < num; i++)
{
TcpClient tcp = new TcpClient();
tcp.Connect(_server);
}
The first connection is success. But the second connection is failed due to server's no response(Actually server are listening tcp connection).
However, if I add Thread.Sleep(1500) after each of tcp.Connect(), all connections are success. But this situation is only true if there are one client and one server. If there are many clients then how can I ensure each connection that can be accepted by server? Also why I add Thread.Sleep can make such connections succeed?
I had the same task. I looked for canonical implementation of this task for .Net with no luck.
The approach I use now is descibed below.
Main idea
We need listener to receive connection, give the connection to the handler, and as soon as possible start listen for a new connection.
Implementation
AutoResetEvent _stopEvent = new AutoResetEvent(false);
object _lock = new object();
public void StartListening()
{
_listener.BeginAcceptTcpClient(ConnectionHandler, null);
_stopEvent.WaitOne();//this part is different in my original code, I don't wait here
}
public void StopListening()
{
lock(_lock)
{
listener.Stop();
listener = null;
}
_stopEvent.Set();//this part is different in my original code
}
void ConnectionHandler(IAsyncResult asyncResult)
{
lock(_lock)
{
if(_listener == null)
return;
var tcpClient = _listener.EndAcceptTcpClient(asyncResult);
var task = new MyCustomTask(tcpClient);
ThreadPool.QueueUserWorkItem(task.Execute);
_listener.BeginAcceptTcpClient(ConnectionHandler,null);
}
}
I am still not very confident in calling _listener.BeginAcceptTcpClient in ConnectionHandler, but I haven't found alternative way.
Since there are still no satisfied answers and I finally use different approach to handle my case. I found out that using class Socket is faster and more stable than using TcpListener and TcpClient. I tried different approach to use TcpListener and TcpClient. Firstly, I used TcpListener.AcceptTcpClient to listen client with and without TcpListener.Pending but there is still possibly ignoring some client connection. Sencondly, I used asynchronous method, TcpListener.BeginAcceptTcpClient and TcpListener.EndAcceptTcpClient but still no succeeded, still ignoring some client connection. Finally using Socket.Accept instead of TcpListener.AcceptTcpClient, the former one has nearly no delay and really fast to response to client.