Everything that I read about sockets in .NET says that the asynchronous pattern gives better performance (especially with the new SocketAsyncEventArgs which saves on the allocation).
I think this makes sense if we're talking about a server with many client connections where its not possible to allocate one thread per connection. Then I can see the advantage of using the ThreadPool threads and getting async callbacks on them.
But in my app, I'm the client and I just need to listen to one server sending market tick data over one tcp connection. Right now, I create a single thread, set the priority to Highest, and call Socket.Receive() with it. My thread blocks on this call and wakes up once new data arrives.
If I were to switch this to an async pattern so that I get a callback when there's new data, I see two issues
The threadpool threads will have default priority so it seems they will be strictly worse than my own thread which has Highest priority.
I'll still have to send everything through a single thread at some point. Say that I get N callbacks at almost the same time on N different threadpool threads notifying me that there's new data. The N byte arrays that they deliver can't be processed on the threadpool threads because there's no guarantee that they represent N unique market data messages because TCP is stream based. I'll have to lock and put the bytes into an array anyway and signal some other thread that can process what's in the array. So I'm not sure what having N threadpool threads is buying me.
Am I thinking about this wrong? Is there a reason to use the Async patter in my specific case of one client connected to one server?
UPDATE:
So I think that I was mis-understanding the async pattern in (2) above. I would get a callback on one worker thread when there was data available. Then I would begin another async receive and get another callback, etc. I wouldn't get N callbacks at the same time.
The question still is the same though. Is there any reason that the callbacks would be better in my specific situation where I'm the client and only connected to one server.
The slowest part of your application will be the network communication. It's highly likely that you will make almost no difference to performance for a one thread, one connection client by tweaking things like this. The network communication itself will dwarf all other contributions to processing or context switching time.
Say that I get N callbacks at almost
the same time on N different
threadpool threads notifying me that
there's new data.
Why is that going to happen? If you have one socket, you Begin an operation on it to receive data, and you get exactly one callback when it's done. You then decide whether to do another operation. It sounds like you're overcomplicating it, though maybe I'm oversimplifying it with regard to what you're trying to do.
In summary, I'd say: pick the simplest programming model that gets you what you want; considering choices available in your scenario, they would be unlikely to make any noticeable difference to performance whichever one you go with. With the blocking model, you're "wasting" a thread that could be doing some real work, but hey... maybe you don't have any real work for it to do.
The number one rule of performance is only try to improve it when you have to.
I see you mention standards but never mention problems, if you are not having any, then you don't need to worry what the standards say.
"This class was specifically designed for network server applications that require high performance."
As I understand, you are a client here, having only a single connection.
Data on this connection arrives in order, consumed by a single thread.
You will probably loose performance if you instead receive small amounts on separate threads, just so that you can assemble them later in a serialized - and thus like single-threaded - manner.
Much Ado about Nothing.
You do not really need to speed this up, you probably cannot.
What you can do, however is to dispatch work units to other threads after you receive them.
You do not need SocketAsyncEventArgs for this. This might speed things up.
As always, measure & measure.
Also, just because you can, it does not mean you should.
If the performance is enough for the foreseeable future, why complicate matters?
Related
so im writing a udp server and client for the first time for a 1v1's game.
My idea is to have the server handling first connections made and creating a new thread every time 2 new players connect to handle all communication between them.
A typical client message would have the threadIndex (i have an array of threads), playerId (which player it came from) and whatever they need to be done.
Is it possible to receive the packet on all threads and analyze if its meant for them? Would this be efficient? How should i approach this?
The suitable approach depends of nature of server tasks, but creating a new thread for every pair of players is not the best idea probably. Basically lets imagine, that your server mostly performs:
I/O bound tasks. In other words most of time it waits for some I\O
opertiton - network respond, query to database or disk operation. In
this case you probably need asynchorous model, when all your
connections are handled in the same thread. It would be efficient
because you actually don't have much to do in your own code. I suppose
you more likely have kinda I/O bound tasks. For example you just need to route messages between players and push\pull some data from DB. All routed messages will have an Id of the game(between to plyers), so you will never miss any of them, and they won't be missent. Take a look on this video to see the ideas and goals of asynchronous approach.
CPU bound tasks. Here server must compute something, perform heavy algorithms or process huge amount of data. In this case you probably need multithreading, but again thread per players pair may not be the most suitable approach, because it is not well scaleable and eats too much resourses. If you have some heavy CPU tasks, try to hanlde them in queue with a set of background workers. And then push the messages in asynchronous manner. Take a look on producer-consumer implementation with BlockingCollection.
You may have a combination of two cases, and of cource you can combine the approaches above. Also see questions 1, 2, 3. Try and return with specific questions. Hope it helps.
ThreadPool utilizes recycling of threads for optimal performance over using multiple of the Thread class. However, how does this apply to processing methods with while loops inside of the ThreadPool?
As an example, if we were to apply a thread in the ThreadPool to a client that has connected to a TCP server, that client would need a while loop to keep checking for incoming data. The loop can be exited to disconnect the client, but only if the server closes or if the client demands a disconnection.
If that is the case, then how would having a ThreadPool help when masses of clients connect? Either way the same amount of memory is used if the clients stay connected. If they stay connected, then the threads cannot be recycled. If so, then ThreadPool would not help much until a client disconnects and opens up a thread to recycle.
On the other hand it was suggested to me to use the Network.BeginReceive and NetworkStream.EndReceive asynchronous methods to avoid threads all together to save RAM usage and CPU usage. Is this true or not?
Either way the same amount of memory is used if the clients stay
connected.
So far this is true. It's up to your app to decide how much state it needs to keep per client.
If they stay connected, then the threads cannot be recycled. If so,
then ThreadPool would not help much until a client disconnects and
opens up a thread to recycle.
This is untrue, because it assumes that all interesting operations performed by these threads are synchronous. This is a naive mode of operation, and in fact real world code is asynchronous: a thread makes a call to request an action and is then free to do other things. When a result is made available as a result of that action, some thread looking for other things to do will run the code that acts on the result.
On the other hand it was suggested to me to use the
Network.BeginReceive and NetworkStream.EndReceive asynchronous methods
to avoid threads all together to save RAM usage and CPU usage. Is this
true or not?
As explained above, async methods like these will allow you to service a potentially very large number of clients with only a small number of worker threads -- but by itself it will do nothing to either help or hurt the memory situation.
You are correct. Slow blocking codes can cause poor performances both on the client-side as well as server-side. You can run slow work on a separate thread and that might work well enough on the client-side but may not help on the server-side. Having blocking methods in the server can diminish the overall performance of the server because it can lead to a situation where your server has a large no of threads running and all blocked. So, even simple request might end up taking a long time. It is better to use asynchronous APIs if they are available for slow running tasks just like the situation you are in. (Note: even if the asynchronous operations are not available, you can implement one by implementing a custom awaiter class) This is better for the clients as well as servers. The main point of asynchronous code is to reduce the no of threads. Because servers can have larger no of requests in progress simultaneously because reducing no of threads to handle a particular no of clients can improve scalability.
If you dont need to have more control over the threads or the thread-pool you can go with asynchronous approach.
Also, each thread takes 1 MB space on the heap. So, asynchronous methods will definitely help reduce memory usage. However, I think the nature of the work you have described here is going to take pretty much the same amount of time in multi-threaded as well as asynchronous approach.
If I have 1 thread for my MMORPG server running a async socket and async packet handler, and in that 1 thread I have a static World that contains all entities in the game.
Would there be any threading issues if say, the async packet handler recieves an Attack message, resulting in a search of the entities in the world to figure out the target.
At the same time the static World Proc method is increasing the size of the Dictionary containing the monster entities adding extra monsters that spawned.
If this is all on the same thread, will the server explode?
will the server explode?
Yes, you can run into problems ("explode") because the async stuff is running on a different thread (even though you didn't create that thread explicitly) and it might access a shared object (world) at the same time as your main thread. Many datastructures (including the Dictionary) are not designed for this scenario and might crash or return the wrong answer.
The typical approach is to use locks to protect your shared objects: take the lock before modifying it, do whatever modification, and then release the lock. This way, only one thread at a time accesses the world (and its dictionary) and so everything remains consistent. Explosion averted.
Another way would be to switch to a more synchronous form of networking, perhaps for example avoiding completion handlers and instead waiting to hear from each of the players, and then acting on the inputs. This can be done very simply, but the simple way has drawbacks: any one slow player can slow the whole thing down. So sadly you're probably going to have to deal with some complexity, one way or another.
If I give answer in one line. Server will explode. As network activity and game logic is in same thread. And as you have mentioned you will be needing high network usage.
I seriously like that if you have a look to F#. It has all the things that you needed. As far as I got it from question. And few things are like collection change, and async is by default in language. Even Nodejs it is also worth trying. But again it all depends on requirement. Though I try to explain few keywords that may help you to take decision.
Non-blocking : It means thread will not be block on events. It will not wait for function to wait for another function to execute. But that doesn't mean you can't block it. In any case it is a single thread.
Async: It is some what like that. But in C# 5 async comes with keyword so you don't have to do threading part of programming.
Parallel Processing: In game development parallel processing is important. Now, that you can do with multiple thread or just use TPL.
In the case of UI based (where there are many objects) game I highly recommended that you separate processing thread and UI thread to improve user experience. Else FPS will go down while you are processing data.
Let me know if any further information needed.
Server will not go down if you take little care of it.
If on the same thread, then no. If you are doing all the work mentioned on a single thread, then there's no issue. However, as stated, if you are accessing a "shared" object instance across threads, then yes, there will be an issue, and locking will be required (using a "lock(){...}" block).
As your user base increases, you will have to keep an eye on the number of threads generated, or event messages if using a non-blocking event model for incoming requests.
On a different, yet related note, keep an eye on this C# based MMO server (with scripting support): https://dreamspace.codeplex.com/ - it may become a big help to MMO game creators very soon (will support Construct 2 by default).
I'm in need of some advice in proper coding:
I'm working on a program where multiple serial connections are used. Each communication line has a controller working as an abstraction layer. Between the controller and the serial port, a protocol is inserted to wrap the data in packages, ready for transfer. The protocol takes care of failed deliveries, resending etc.
To ensure that the GUI won't hang, the each connection line (protocol and serial port) is created on a separate thread. The controller is handled by the main thread, since it has controls in the GUI.
Currently, when I create the threads, I have chosen to create a message loop on them (Application.Run()), so instead polling buffers and yielding if no work, i simply invoke the thread (BeginInvoke) and uses the message loop as a buffer. This currently works nicely, and no serious problems so far.
My question is now: Is this "good coding", or should i use a while loop on the tread and be polling buffers instead?, or some third thing?
I would like to show code, but so far it is several thousand lines of code, so please be specific if you need to see any part of the code. :)
Thank you.
Using message loops in each thread is perfectly fine; Windows is optimized for this scenario. You are right to avoid polling, but you may want to look into other event-based designs that are more efficient still, for example preparing a package for transfer and calling SetEvent to notify a thread it's ready, or semaphore and thread-safe queue as Martin James suggests.
I'm not 100% sure what you are doing here but, with a bit of 'filling in' it doesn't sound bad:)
When your app is idle, (no comms), is CPU use 0%?
Is your app free of sleep(0)/sleep(1), or similar, polling loops?
Does it operate with a reasonably low latency?
If the answers are three 'YES', you should be fine :)
There are a few, (very few!), cases where polling for results etc. is a good idea, (eg. when the frequency of events in the threads is so high that signaling every progress event to the GUI would overwhelm it), but mostly, it's just poor design.
In C#, when receiving network data with the BeginReceive/EndReceive methods, is there any reason you shouldn't process the packets as soon as you receive them? Some of the tasks can be decently cpu intensive. I ask because I've seen some implementations that push the packets off into a processing queue and then handle them there. To me this seems redundant because, as far as I know, the async methods also operate on a thread pool.
Generally, you need to receive 'enough' packets to have a data item that is 'processable'.
IMO, It's better to have one thread whose job is receiving data, and another to actually process it.
As Mitch points out, you need to be able to receive enough packets to have a complete message/frame . But there's no reason why you shouldn't start processing that frame immediately and issue another BeginReceive. In fact, if you believe your processing could take some time, you're better off handing it off to the worker thread-pool rather than block a thread from the i/o pool (which is where your callback will fire).
In addition, unless you're expecting a low number of connections, spawning a thread to handle each connection is not a very scalable approach, although it does have the benefit of some simplicity.
I recently wrote an article on pipelining data-processing off a network socket, which you can find here.