We Keep Coding

Porting socket server from Node.js to C#

I've built multiple socket server apps in Node.js for a multi-user artificial intelligence app. We're looking at 1K to 10K active socket connections per box. However even when idle and with 0 active connections, some of my servers consume 50-100 MB of memory when running on Unix. I'm sure with a sensible platform like C# or C++, this should be close to 0 MB. So we are considering a port to a "better" platform. Now let my clarify my use case:
This is not a "web server". No files are served.
We do lots of CPU intensive data processing and certain portions have already been ported to C++ and pulled into node via native modules.
We don't need to access much I/O (in most cases a few files are accessed, in some cases none, we don't use an RDBMS either)
We went with node because it was Unix friendly (unlike .NET) and seemed easy to use. But with its current memory consumption we need to evaluate other options. Many have compared Node.js with ASP.NET but I need to build a socket server in C# or C++.
I have significant experience with .NET and C++. There are libs like SuperSocket (used by Redgate and Telerik) that handle all of the low-level stuff in .NET. I will have to find a similar socket framework for C++.
So putting this all together, what are the advantages of using .NET or C++ over Node.js? And considering my servers are highly CPU-bound (not I/O bound) would the benefits of using .NET/C++ be significant or should I stick with Node.js? Any other comments regarding porting a Node.js app to C# or C++?
Bounty: I need advice and a recommended socket server library/implementation/example app in C# and/or C++. Must be open source. I need it to be high-performance, async and bug-free. Must support binary data transfer. Must run on Windows. Unix is a bonus.

We're looking at 1K to 10K active socket connections per box
the bottleneck here is not the programing language or the technology, it's the hardware and OS support. the thing that limits the amount of concurrent sockets count is basically the machine you're running on. yet, from my experience, the determinisitic object lifetime of C++ can help dramatically for supporting large number of concurrent OS resources.
This is not a "web server". No files are served.
I have done some Node.js in my profesional work, I have done some C# but mostly C++. even with node.js as a web server, most of the client and server code didn't had many much in common besides the language itself. the web server dealt with buisness logic mostly, while the client dealt with fetching and presenting the data interactivly. So, I think the main advantage of node.js as a web server is that it gives purist-JS developers the ability to write server side without using languages/technology they are not familliar with.
We do lots of CPU intensive data processing and certain portions have
already been ported to C++ and pulled into node via native modules.
yep. using strongly typed language can do wonders here. no redunadand runtime-parsing.
We don't need to access much I/O (in most cases a few files are
accessed, in some cases none, we don't use an RDBMS either)
Well, I feel there's a myth in the air that node.js somehow handles IO better than other technologies. this is simply wrong. the main feature of Node.js is the fact that by default, the IO is asynchronous. but Node.js didn't invent any wheel. you have asynchronous IO in Java (aka Java.NIO), C# (async/await) and C++ (as native stuff like epoll/IOCompletionPort, or some higher stuff like Boost.ASIO/ CPP-rest, Proxygen etc.)
We went with node because it was Unix friendly (unlike .NET)
.Net Core is a relativly new technology where .Net can run on Unix-based systems (like linux)
I will have to find a similar socket framework for C++.
Boost.ASIO, or write something yourself, it's really not that hard..
So putting this all together, what are the advantages of using .NET or
C++ over Node.js?
better CPU usage: because C++ and C# are strongly typed languages, and C++ is a statically compiled language, there are huge oppretunities for the compiler to optimize CPU extensive jobs.
lower memory footprint: usually because strongly typed languages have smaller objects without the overhead of keeping a lot of meta-data behind the scences.
with C++, having stack allocation and scoped object life-time usually the memory footprint is low. again, it depends on the quality of the code in any language.
no callback hell: C# has tasks and async await. C++ has futures/promises and some compilers (aka VC++) do supports await as well. the asynchronous code simply becomes pure fun to write as oppossed to callbacks. yes, I do aware of JS promises and the new async/await stuff, but they are relativly new compared to .Net implementation.
Compiler checks : since C# and C++ have to be compiled, a lot of silly bugs are caught in compile time. no "undefiend is not a function" or "cannot read property of undefined".
other than that it's pretty much a matter of choice.

NetMQ is native C# port of zeromq.
Zeromq is lightweight messaging library, the zeromq guide is a great if you want to learn about messaging, it also come as a book. It applicable both to zeromq and NetMQ.
If you are using windows and need to handle a lot of connection I don't recommend zeromq as it not using IOCP.
NetMQ is using IOCP on Windows and works both on windows and linux.
Disclosure - I'm author of NetMQ and maintainer on the zeromq (libzmq) project.
[1] https://github.com/zeromq/netmq
[2] http://netmq.readthedocs.io/en/latest/
[3] http://zguide.zeromq.org/page:all
[4] http://www.amazon.com/ZeroMQ-Messaging-Applications-Pieter-Hintjens/dp/1449334067/ref=sr_1_1?ie=UTF8&qid=1462550951&sr=8-1&keywords=zeromq

We do lots of CPU intensive data processing
Node.js may have been the wrong choice from the start and it would probably never match performances of a C++ server. However, it can be pretty close, if you are doing things right. In addition, writing good C++ and a complete rewrite of a system is difficult and time consuming. So, I want to give some reasons for you to stick to Node.js or at least, completely exhaust all your options before you move.
my servers consume 50-100 MB
Are you using Node.js v0.12? With Node.js v4.2 LTS, idle Node.js server should use around 20 MB of memory. (It would probably never be near 0 MB because of V8) Have you checked for memory leaks?
1K to 10K active socket connections per box
This should be easily achievable. If you are using the most popular socket.io library, here's some relevant benchmarks.
on a 3.3 GHz Xeon X5470 using one core, the max messages-sent-per-second rate is around 9,000–10,000 depending on the concurrency level.
from: http://drewww.github.io/socket.io-benchmarking/
(Since, all these connections are kept alive concurrently, CPU usage matters more)
If you are already using that and having issues, try replacing socket.io with SocketCluster which is faster and more scalable. Replacing this should be easier than a complete rewrite. Here's some benchmarks:
8-core Amazon EC2 m3.2xlarge instance running Linux
at 42K, the CPU use of the busiest worker dropped to around 45%
http://socketcluster.io/#!/performance
Finally, to prove that Node.js can nearly reach C++ performance. Have a look at this:
servers use 12G memory
It supports 1,200,000 active websocket connections
https://github.com/smallnest/C1000K-Servers
My point is you have average performance goals that you should be able to reach with Node.js with little effort. Try to benchmark (https://github.com/machinezone/tcpkali) and find the issue rather than do a complete rewrite.

Concurrency support in python compared to C# and C++

I am looking for an explanation by someone who has worked in depth with both languages. Someone who has worked with C# and C++, I know C# provides a task parallel library from .Net 4.0 onwards. C++ also has a concurrent runtime library http://msdn.microsoft.com/en-us/library/dd504870.aspx.
How can one achieve concurrency in python webapps and not spawn processes but rather threads for tasks. Assuming the user is able to manage locks etc.

Warning: concurrency is not parallelism. Warren is correct that the Global Interpreter Lock (GIL) limits parallelism within a single Python process, but if the process is IO-bound a select-based application can provide sufficient concurrency without parallelism (Python's asyncore documentation has a good discussion of the select-based workflow).
That said, Python's Twisted framework provides future-like objects called deferreds and does multi-threading under the hood (obviously the GIL limits each process to a single thread at any given instant). Though some find Twisted too low level and/or deferreds to complicated for webapp development.
In the extreme it is possible for native code called from Python to release the GIL if the native code is not going to interact with Python objects. For example ctypes provides a rudimentary interface to C functions which releases the GIL while the C function is running. Similar results can be achieved with other extension frameworks like Boost.Python and SIP.

Python has a global interpreter lock that prevents more than one thread from executing at a time, essentially preventing concurrency. That's why you see people spawning processes when they want concurrency in Python. C# and C++ in contrast permit concurrent threads.
Additional information on why Python has a global interpreter lock is available at Why the Global Interpreter Lock?.

select/poll based framework in C#

I like to write an application that opens many sockets and files. Think of it as webserver (which is not true in my case, but to simplify the problem here).
If I would write it in C on Unix I would use poll/select and be quite efficient and because I don't have multiple threads, everything is easy to write, while being very efficient.
If I use multiple threads to use all cores of the CPU (given that I don't wanna use processes) I would use Unix FIFOs to transfer messages and use still poll/select on each thread (which works flawlessly with files/socket/fifos/). Things are still very simple while being quite efficient.
But when using C# it looks like there are different selects and most classes don't support that programming style at all (HttpWebListener just as one example). I don't like the BeginInvoke messiness because there are things happening in the background on which I don't have any control (ThreadPooling, Shutting down a blocking server gracefully, ...).
I wonder if there is any select/poll alike framework available for C#?

You can actually use your same approaches in C# - you just need to use the lower level Socket class, which provides Select and Poll.
That being said, the new asynchronous methods built on top of socket in the higher level classes tend to have many advantages. Once you learn and understand how they function, they can be very efficient and quite a bit nicer to develop against.
This extends all the way up the stack - with the "highest level" abstractions being frameworks like WCF, which provide huge benefits in terms of productivity, reliability, safety, and ease of development for many types of applications.

BeginInvoke (or Tasks based on the Begin/End pattern) are the standard model of async programming on .NET. They indeed force the continuation callbacks to run on the thread-pool. If you are fine with that the Begin/End model is actually very efficient and nice (as nice as callback-based code can be...).
Of the top of my head I cannot see a compelling reason why I wouldn't want to use the thread-pool for completion callbacks. Maybe you can squeeze out a little more efficiency using IOCPs.
Select/poll certainly isn't the way to become more efficient. Although .NET sockets support it.
You said
Shutting down a blocking server gracefully
would be a problem. I don't see why. Can you elaborate?

Pcap.net vs Sharppcap

I just want to listen a network device, capture packets and write the packets to a dummy file. Also i need to filter packets while listening so ill only write packets which passes the filter. I need to do these on .net c#. These are my requirements. So which one should i use? High transfer rate and minimum packet loss is really important.
Thanks for reading.

As the author of SharpPcap I can say that you'll be able to perform all of those operations with the library. Performance was a critical design goal.
Packet.Net has a range of packets that it can parse and is the library bundled along with SharpPcap for packet dissection and generation. It's architecture does lazy evaluation anywhere it is possible in order to be as fast as possible.
Performance is tricky, especially because network packet capture is often a lower priority task for an operating system. The faster your application handles the packet the more packets can be handled without drops. I've been able to capture 3MB/s of packets without any drops. I haven't tried it at higher data rates or written extensive tests to generate and capture data in order to evaluate performance. Tests and real world results are welcome data points to be added to the documentation and website though.

The entire functionality is available in Pcap.Net.
Pcap.Net uses C++/CLI to wrap WinPcap, which is considered more efficient than PInvoke.
The packet library in Pcap.Net is quite big and complex packets can be parsed and created. This includes recursive layers like IP over IP. Parsing of each layer is done lazily and only when you need it.
For your needs, I see only benefits of using Pcap.Net over SharpPcap.

Need C# client to connect to server with low latency bi-direction communication

I'm in the process of learning C# and just need a pointing in the right direction. I want to build a client in C# that communicates with a server running PHP/mySQL. There will need to be almost constant communication between the two. It will be for a game, so low-latency and bi-directional communication. I'm not looking for an exact how-to, but rather what method I need to use to connect the two for the fastest and most reliable connection. I have read others use XML, but that seems like it would be slower if used near-constantly, like once or more a second, but I could be totally wrong. Thanks in advance!

Normally communication with those characteristics is made over a persistent TCP connection. C# offers lots of ready-to-use functionality in the System.Net.Sockets namespace (start looking from TcpClient and TcpListener; there are also more low-level interfaces if you need them).
The important question here is: what do you mean exactly "server running PHP"? If the server offers only an HTTP interface, then you would find it more natural to communicate not with sockets but with the WebClient or the more low-level HttpWebRequest classes instead.

Ah, writing a game in C# as a means to get started with the language! How many have started this way.
Have you defined your client-server protocol yet? I'm not talking about TCP vs. UDP, which TomTom and Jon have discussed. I mean, what is the data stream going to look like?
Packet fragmentation is the enemy of low-latency network code. Learn about MTU and packet fragmentation, Nagle's algorithm, etc. and write down some notes for later when you implement the network code. Make sure you calculate the smallest size packet you would be interested in sending, how big its headers might be, and how large of a payload you can fit into that packet. Then see if you can come up with a protocol that uses the available space efficiently.
You may gain a lot more by optimizing your server application and/or porting it to a different language. Just because you CAN use PHP for everything server side doesn't mean you SHOULD. Keep the part that shows you useful information in a web browser, and evaluate whether you should rewrite the time-critical and game client communication parts in another language. Interpreted languages are not especially well known for their speed when crunching real-time game world data. Sure, I once wrote something like that in Perl using POE, but ultimately it was a lot less performant than the C code I was mimicking.
Finally, I would recommend you look into XNA, since it has a lot of this stuff already.