I am currently working on a .NET c# socket server which should be able to scale upto 100K concurrent connections. I am using the socketasynceventargs class and the pattern mentioned here . correct me if I am wrong but I understand that maintaining 100K concurrent connections is different from 100K client hitting the socket server at the exact same time. my question is how many connections can I make simultaneously? is this dependent on the socket backlog variable? if so what is the max backlog value i can set?
Thanks in advance
I am currently working on a .NET c# socket server which should be able to scale upto 100K concurrent connections.
Last time I tested this on Win7 this was an easy goal to reach. The number of connections seems to be limited by memory usage.
I am using the socketasynceventargs class and the pattern mentioned here.
This pattern is used to have a very high frequency of calls. It is not useful to maintain a high number of connections because it uses more memory than a simple BeginRead call that is outstanding. Always ask why and don't just copy sample code. Most sample code about sockets is horribly wrong, even on MSDN.
Have a single BeginRead call outstanding per socket. Until a read call is completed the memory buffer given to it is pinned. This causes GC problems. Either use one big preallocated buffer (64MB or so) or read only one byte at first. Only when that one byte read completes you read the rest with a bigger buffer.
correct me if I am wrong but I understand that maintaining 100K concurrent connections is different from 100K client hitting the socket server at the exact same time.
Not sure I understand. 100k clients coming in in the same millisecond would be hard to handle while maintaining 100k connections that have been established over the course of seconds is much easier.
my question is how many connections can I make simultaneously?
Test that and expect to find a high number. Watch your RAM usage.
is this dependent on the socket backlog variable?
That is for outstanding connections that have not been handed to the application. It is mostly meaningless in practice because an app should have a fast accept loop immediately accepting anything.
if so what is the max backlog value i can set?
Set the default.
Related
I am writing a high-throughput server serving thousands of connections. Suppose I have 400 bytes to send via a socket. Suppose I do it in two ways:
Call the Socket.Send() 40 times, each time sending 10 bytes.
Call the Socket.Send() once, sending 400 bytes.
Do these two ways make much difference in terms of speed, CPU load, etc?
If Socket.NoDelay is left at false, then it will very rarely make any difference - most of the time, you're just going to buffering locally - albeit with a bit more P/Invoke overhead than is absolutely necessary (due to lots of calls through the socket layer). Note that Socket.NoDelay should usually be set to true in anything where you care.
If Socket.NoDelay is true, then if everything is working maximally, then you might introduce additional packet fragmentation by using 40 sends of 10 bytes, which would be avoided when using one send of 400 bytes. However, in many cases, the various abstractions and layers in the OS/hardware stacks means that a lot of the 10 byte chunks will probably end up sharing packets. That's still a lot more packets than 1, in the optimal case, though.
Note also that this is always a trade-off: packet fragmentation will decrease overall throughput, but sending the first bytes sooner could reduce the perceived latency, if the other 390 bytes are going to take a measurable (but presumably small) amount of time to construct.
In most cases: this is unlikely to be a bottleneck. If you can avoid packet fragmentation without causing latency, that may be desirable. If it was me, I'd probably be more concerned with efficient buffer management to maximise scalability while avoiding pauses due to GC; tools like the new "pipelines" IO API can really help with that, and Kestrel can be used to host a TCP server based on "pipelines" in a lot less code than you would be using if you wrote your own socket listener - and it then deals with all the buffer management for you.
Is there a valid reason to not use TcpListener for implementing a high performance/high throughput TCP server instead of SocketAsyncEventArgs?
I've already implemented this high performance/high throughput TCP server using SocketAsyncEventArgs went through all sort of headaches to handling those pinned buffers using a big pre-allocated byte array and pools of SocketAsyncEventArgs for accepting and receiving, putting together using some low level stuff and shiny smart code with some TPL Data Flow and some Rx and it works perfectly; almost text book in this endeavor - actually I've learnt more than 80% of these stuff from other-one's code.
However there are some problems and concerns:
Complexity: I can not delegate any sort of modifications to this server to another
member of the team. That bounds me to this sort of tasks and I can
not pay enough attention to other parts of other projects.
Memory Usage (pinned byte arrays): Using SocketAsyncEventArgs the pools are needed to
be pre-allocated. So for handling 100000 concurrent connections
(worse condition, even on different ports) a big pile of RAM is uselessly hovers there;
pre-allocated (even if these conditions are met just at some times,
server should be able to handle 1 or 2 such peaks everyday).
TcpListener actually works good: I actually had put TcpListener into test (with some tricks like
using AcceptTcpClient on a dedicated thread, and not the async
version and then sending the accepted connections to a
ConcurrentQueue and not creating Tasks in-place and the like)
and with latest version of .NET, it worked very well, almost as good
as SocketAsyncEventArgs, no data-loss and a low memory foot-print
which helps with not wasting too much RAM on server and no pre-allocation is needed.
So why I do not see TcpListener being used anywhere and everybody (including myself) is using SocketAsyncEventArgs? Am I missing something?
I see no evidence that this question is about TcpListener at all. It seems you are only concerned with the code that deals with a connection that already has been accepted. Such a connection is independent of the listener.
SocketAsyncEventArgs is a CPU-load optimization. I'm convinced you can achieve a higher rate of operations per second with it. How significant is the difference to normal APM/TAP async IO? Certainly less than an order of magnitude. Probably between 1.2x and 3x. Last time I benchmarked loopback TCP transaction rate I found that the kernel took about half of the CPU usage. That means your app can get at most 2x faster by being infinitely optimized.
Remember that SocketAsyncEventArgs was added to the BCL in the year 2000 or so when CPUs were far less capable.
Use SocketAsyncEventArgs only when you have evidence that you need it. It causes you to be far less productive. More potential for bugs.
Here's the template that your socket processing loop should look like:
while (ConnectionEstablished()) {
var someData = await ReadFromSocketAsync(socket);
await ProcessDataAsync(someData);
}
Very simple code. No callbacks thanks to await.
In case you are concerned about managed heap fragmentation: Allocate a new byte[1024 * 1024] on startup. When you want to read from a socket read a single byte into some free portion of this buffer. When that single-byte read completes you ask how many bytes are actually there (Socket.Available) and synchronously pull the rest. That way you only pin a single rather small buffer and still can use async IO to wait for data to arrive.
This technique does not require polling. Since Socket.Available can only increase without reading from the socket we do not risk to perform a read that is too small accidentally.
Alternatively, you can combat managed heap fragmentation by allocating few very big buffers and handing out chunks.
Or, if you don't find this to be a problem you don't need to do anything.
If I am not reading data from socket fast enough, the TCP protocol will decrease sliding windows size and sender might get blocked during sending (as discussed here what happens when I don't manage to call `recv` fast enough?).
How do I detect this situation on receiver side on Windows - preferably directly in C# code and without impacting the performance of reading from socket? Other monitoring solution (perfmon, wireshark) is also acceptable but far less optimal for my scenario.
What is the exact scenario? Let's say the server app can transmit data with speed up to 1Mbps, however my client app is able to receive the data only with the speed of 0.5Mbps. How do I find out in the client application that TCP flow control is kicking in and decreasing the transmit speed?
I came over Socket.Available property http://msdn.microsoft.com/en-us/library/system.net.sockets.socket.available.aspx and was wondering if that might be recomendable way of querying this information?
You would be better off reading as fast as you possibly can, rather than wasting time trying to have the system tell you you're not reading fast enough, which can only slow down your reading even further. If you're reading at maximum speed and the sender is still getting blocked, TCP is working correctly and there is nothing you can do about it, except maybe look into a faster machine.
The TCP Window is handled by the Kernel and won't be available to you. I guess you could possibly compare the ReceiveBufferSize with number of bytes Received. If this buffer isn't full, then you are waiting.
We have a TCP Async socket game server written in C# serving many concurrent connections. Everything else works fine except for the problems below:
Frequent disconnections for some users (Not all mind you)
Degraded performance for users on slow connections
Most of the users who face this problem are using GSM connections (portable USB dongles based on CDMA) and often the signal strength is poor. So basically they have rather poor bandwidth. Perhaps under 1KB per sec.
My question: What can we do to make the connections more stable and improve performance even for the slower connections?
I am thinking dynamic TCP buffers on client and server side, but not really sure of the performance degradation due to overhead in dynamically doing this for each connection of my direction is even correct.
Max data packet size is under 1 KB.
Current TCP buffer size on server and client is 16KB
Any pointers or references on how to write stable anync socket code in C# for poor or slow connections would be a great help. Thanks in advance.
"Performance" is a very relative term. It looks like your main concern is with data transfer rates. Unfortunately you can't do much about it given low-bandwidth connections - maybe data compression can help, but actual effect depends on your data, and there's always a tradeoff between transfer rate improvement vs. compression/de-compression delays. There's also latency to consider for any sort of interactive game.
As #Pierre suggested in the comments you might consider UDP for the transport, but that only works if you can tolerate packet loss and re-ordering, and that again depends of the data and what you do with it.
Another approach I would suggest investigating is to provide two different quality-of-service modes. Clients on good links can use full functionality (say, full-resolution game images), while low-bandwidth clients would get reduced options (say, much smaller size low-quality images). You can measure initial round-trip times on client connect/handshake to select the mode.
Hope this helps a bit.
Hi
I have TCP/IP client server application. i want to send large serialized object around 1MB through sockets.
Is it possible to get better performance by splitting byte array to for example 10 chunks of arrays and open a socket for each and send them Async compared to opening one socket and send all large data through it ?
Thanks
Splitting the data to less than the MTU will introduce more overhead as there will be more packets - this will actually slow things down. What you are proposing is already being done as part of the protocol i.e. splitting and re-assembling. I would experiment with sending less data e.g. compression.
No, this doesn't speed up the transfer under normal conditions, it only adds overhead. It would only help if you have a slow network segment which is quite busy otherwise and the traffic is shaped per TCP connection.
Make sure that your sockets code is efficient, because wrong buffer and therefore packet sizes, synchroneous operation and other stuff may slow the transfer down.