I am developing a multithread server which works nice so far - 1 separate thread for client accepting, threadpool for data reading and processing. Today I have added new thread for doing some stuff and sending messages to client every 500 ms (just 2-5 messages). I have noticed quite massive slowdown but Im not sure why - its separate thread and its not due to iteration and locking collections, because when I add //before SendMessage call, it was still as fast as before.
The SendMessage basically iterates all connected clients and for each of them calls SendData method which writes data to their networkstream.
What am I missing? I still think those are different threads and I hope its not due to stream.write..
Thank you in advance!
If you can try to post a code sample or a summary, your message sending implementation would make a good candidate.
First, purely general advice.
This is a good time to whip out a profiler. This kind of guessing is tempting, and often a good mental excercise, but most of the time programmers are wrong about what they think is making their software slow. A profiler will tell you, for example, if your program is spending 90% of its execution time inside of one method.
Second, a speculative guess.
It sounds like your message command runs off a timer. Make sure that you aren't having issues with reentrancy - for example if your sendmessage loop takes longer than 500ms to complete (and together with creating a new thread and multiple unpredictable latency network calls it could well do that), and you have the whole operation in a lock, then the timer will keep spawning off threadpool threads that are sitting in that lock waiting for the previous operation to complete - and there is a finite number of available threadpool threads. To check if this is a problem you don't even need a profiler, when latency gets bad pause the debugger and check up on your list of currently executing threads.
If this is the case consider doing something else - like have a single thread that runs in an infinite loop using a waithandle as a blocking mechanism and timer that sets the waithandle every 500ms.
But it will be much easier to help you if you post some code snippets, and run a profiler (Ants or DotTrace both work great).
Threads & threadpools for things like socket servers is the old way to do things. It's very unscalable (optimally you would like to not have more threads than cores), and full of locks.
Try converting your code to asynchronous code. You only need 1 thread, and you get callbacks whenever input arrives or when new data can be sent. The resulting code is much faster and doesn't have these bottleneck problems.
I know the advice of: no no, rewrite everything you should do it like this, is not really helpful, since it doesn't answer the exact question you asked. But if you do have the time, I still think it's a good advice. Or else, it's good advice for the next server you'll make ;^)
Related
If I have 1 thread for my MMORPG server running a async socket and async packet handler, and in that 1 thread I have a static World that contains all entities in the game.
Would there be any threading issues if say, the async packet handler recieves an Attack message, resulting in a search of the entities in the world to figure out the target.
At the same time the static World Proc method is increasing the size of the Dictionary containing the monster entities adding extra monsters that spawned.
If this is all on the same thread, will the server explode?
will the server explode?
Yes, you can run into problems ("explode") because the async stuff is running on a different thread (even though you didn't create that thread explicitly) and it might access a shared object (world) at the same time as your main thread. Many datastructures (including the Dictionary) are not designed for this scenario and might crash or return the wrong answer.
The typical approach is to use locks to protect your shared objects: take the lock before modifying it, do whatever modification, and then release the lock. This way, only one thread at a time accesses the world (and its dictionary) and so everything remains consistent. Explosion averted.
Another way would be to switch to a more synchronous form of networking, perhaps for example avoiding completion handlers and instead waiting to hear from each of the players, and then acting on the inputs. This can be done very simply, but the simple way has drawbacks: any one slow player can slow the whole thing down. So sadly you're probably going to have to deal with some complexity, one way or another.
If I give answer in one line. Server will explode. As network activity and game logic is in same thread. And as you have mentioned you will be needing high network usage.
I seriously like that if you have a look to F#. It has all the things that you needed. As far as I got it from question. And few things are like collection change, and async is by default in language. Even Nodejs it is also worth trying. But again it all depends on requirement. Though I try to explain few keywords that may help you to take decision.
Non-blocking : It means thread will not be block on events. It will not wait for function to wait for another function to execute. But that doesn't mean you can't block it. In any case it is a single thread.
Async: It is some what like that. But in C# 5 async comes with keyword so you don't have to do threading part of programming.
Parallel Processing: In game development parallel processing is important. Now, that you can do with multiple thread or just use TPL.
In the case of UI based (where there are many objects) game I highly recommended that you separate processing thread and UI thread to improve user experience. Else FPS will go down while you are processing data.
Let me know if any further information needed.
Server will not go down if you take little care of it.
If on the same thread, then no. If you are doing all the work mentioned on a single thread, then there's no issue. However, as stated, if you are accessing a "shared" object instance across threads, then yes, there will be an issue, and locking will be required (using a "lock(){...}" block).
As your user base increases, you will have to keep an eye on the number of threads generated, or event messages if using a non-blocking event model for incoming requests.
On a different, yet related note, keep an eye on this C# based MMO server (with scripting support): https://dreamspace.codeplex.com/ - it may become a big help to MMO game creators very soon (will support Construct 2 by default).
I wrote some code that mass imports a high volume of users into AD. To refrain from overloading the server, I put a thread.sleep() in the code, executed at every iteration.
Is this a good use of the method, or is there a better alternative (.NET 4.0 applies here)?
Does Thread.Sleep() even aid in performance? What is the cost and performance impact of sleeping a thread?
The Thread.Sleep() method will just put the thread in a pause state for the specified amount of time. I could tell you there are 3 different ways to achieve the same Sleep() calling the method from three different Types. They all have different features. Anyway most important, if you use Sleep() on the main UI thread, it will stop processing messages during that pause and the GUI will look locked. You need to use a BackgroundWorker to run the job you need to sleep.
My opinion is to use the Thread.Sleep() method and just follow my previous advice. In your specific case I guess you'll have no issues. If you put some efforts looking for the same exact topic on SO, I'm sure you'll find much better explanations about what I just summarized before.
If you have no way to receive a feedback from the called service, like it would happen on a typical event driven system (talking in abstract..we could also say callback or any information to understand how the service is affected by your call), the Sleep may be the way to go.
I think that Thread.Sleep is one way to handle this; #cHao is correct that using a timer would allow you to do this in another fashion. Essentially, you're trying to cut down number of commands sent to the AD server over a period of time.
In using timers, you're going to need to devise a way to detect trouble (that's more intuitive than a try/catch). For instance, if your server starts stalling and responding slower, you're going to continue stacking commands that the server can't handle (which may cascade in other errors).
When working with AD I've seen the Domain Controller freak out when too many commands come in (similar to a DOS attack) and bring the server to a crawl or crash. I think by using the sleep method you're creating a manageable and measurable flow.
In this instance, using a thread with a low priority may slow it down, but not to any controllable level. The thread priority will only be a factor on the machine sending the commands, not to the server having to process them.
Hope this helps; cheers!
If what you want is not overload the server you can just reduce the priority of the thread.
Thread.Sleep() do not consume any resources. However, the correct way to do this is set the priority of thread to a value below than Normal: Thread.Current.Priority = ThreadPriority.Lowest for example.
Thread.Sleep is not that "evil, do not do it ever", but maybe (just maybe) the fact that you need to use it reflects some lack on solution design. But this is not a rule at all.
Personally I never find a situation where I have to use Thread.Sleep.
Right now I'm working on an ASP.NET MVC application that uses a background thread to load a lot of data from database into a memory cache and after that write some data to the database.
The only feature I have used to prevent this thread to eat all my webserver and db processors was reduce the thread priority to the Lowest level. That thread will get about to 35 minutes to conclude all the operations instead of 7 minutes if a use a Normal priority thread. By the end of process, thread will have done about 230k selects to the database server, but this do not has affected my database or webserver performance in a perceptive way for the user.
tip: remember to set the priority back to Normal if you are using a thread from ThreadPool.
Here you can read about Thread.Priority:
http://msdn.microsoft.com/en-us/library/system.threading.thread.priority.aspx
Here a good article about why not use Thread.Sleep in production environment:
http://msmvps.com/blogs/peterritchie/archive/2007/04/26/thread-sleep-is-a-sign-of-a-poorly-designed-program.aspx
EDIT Like others said here, maybe just reduce your thread priority will not prevent the thread to send a large number of commands/data to AD. Maybe you'll get better results if you rethink all the thing and use timers or something like that. I personally think that reduce priority could resolve your problem, although I think you need to do some tests using your data to see what happens to your server and other servers involved in the process.
You could schedule the thread at BelowNormal priority instead. That said, that could potentially lead to your task never running if something else overloads the server. (Assuming Windows scheduling works the way the documentation on scheduling threads mentions for "some operating systems".)
That said, you said you're moving data into AD. If it's over the nework, it's entirely possible the CPU impact of your code will be negligible compared to I/O and processing on the AD side.
I don't see any issue with it except that during the time you put the thread to sleep then that thread will not be responsive. If that is your main thread then your GUI will become non responsive. If it is a background thread then you won't be able to communicate with it (eg to cancel it). If the time you sleep is short then it shouldn't matter.
I don't think reducing the priority of the thread will help as 1) your code might not even be running on the server and 2) most of the work being done by the server is probably not going to be on your thread anyway.
Thread.sleep does not aid performance (unless your thread has to wait for some resource). It incurs at least some overhead, and the amount of time that you sleep for is not guaranteed. The OS can decide to have your Thread sleep longer than the amount of time you specify.
As such, it would make more sense to do a significant batch of work between calls to Thread.Sleep().
Thread.Sleep() is a CPU-less wait state. Its overhead should be pretty minimal. If execute Thread.Sleep(0), you don't [necessarily] sleep, but you voluntarily surrender your time slice so the scheduler can let lower priority thread run.
You can also lower your thread's priority by setting Thread.Priority.
Another way of throttling your task is to use a Timer:
// instantiate a timer that 'ticks' 10 times per second (your ideal rate might be different)
Timer timer = new Timer( ImportUserIntoActiveDirectory , null , 0 , 100 ) ;
where ImportUserIntoActiveDirectory is an event handler that will import just user into AD:
private void ImportUserIntoActiveDirectory( object state )
{
// import just one user into AD
return
}
This lets you dial things in. The event handler is called on thread pool worker threads, so you don't tie up your primary thread. Let the OS do the work for you: all you do is decide on your target transaction rate.
I'm in need of some advice in proper coding:
I'm working on a program where multiple serial connections are used. Each communication line has a controller working as an abstraction layer. Between the controller and the serial port, a protocol is inserted to wrap the data in packages, ready for transfer. The protocol takes care of failed deliveries, resending etc.
To ensure that the GUI won't hang, the each connection line (protocol and serial port) is created on a separate thread. The controller is handled by the main thread, since it has controls in the GUI.
Currently, when I create the threads, I have chosen to create a message loop on them (Application.Run()), so instead polling buffers and yielding if no work, i simply invoke the thread (BeginInvoke) and uses the message loop as a buffer. This currently works nicely, and no serious problems so far.
My question is now: Is this "good coding", or should i use a while loop on the tread and be polling buffers instead?, or some third thing?
I would like to show code, but so far it is several thousand lines of code, so please be specific if you need to see any part of the code. :)
Thank you.
Using message loops in each thread is perfectly fine; Windows is optimized for this scenario. You are right to avoid polling, but you may want to look into other event-based designs that are more efficient still, for example preparing a package for transfer and calling SetEvent to notify a thread it's ready, or semaphore and thread-safe queue as Martin James suggests.
I'm not 100% sure what you are doing here but, with a bit of 'filling in' it doesn't sound bad:)
When your app is idle, (no comms), is CPU use 0%?
Is your app free of sleep(0)/sleep(1), or similar, polling loops?
Does it operate with a reasonably low latency?
If the answers are three 'YES', you should be fine :)
There are a few, (very few!), cases where polling for results etc. is a good idea, (eg. when the frequency of events in the threads is so high that signaling every progress event to the GUI would overwhelm it), but mostly, it's just poor design.
Often times I when I see some multi-threaded code, I see Thread.Sleep() statements in the code.
I even had a crash where I was trying to figure out the problem, so commented out most of the multi-threaded code and slowly brought it and for the final piece when I added a for statement like:
for ( int i = 0; i < 1000000; ++i )
++i;
it didn't crash. So now I replaced it Thread.Sleep() and it seems to work. I can't repro it easily to post it here, but is using Thread.Sleep() necessary for multi-threaded applications?
What's the purpose of them? Would it lead to unexpected results if not used?
EDIT: Btw I am using the BackgroundWorker and only implementing my stuff in there, but not sure what causes this. Although I am using an API which is the hosting app where the app is not multi threaded. So for instance I think I can't call it's API functions on several threads at once. Not sure, but that was my guess.
Typically, Thread.Sleep is a sign of a bad design. That being said, its MUCH better than eating 100% of the CPU core time, which is what the for loop above is doing.
A better option is typically to use a WaitHandle, such as a ManualResetEvent, to trigger the continuation of the thread's execution when the "event" (which is the reason to delay) occurs. Alternatively, using a Timer can work as well in many cases.
The Thread.Sleep(1) allows switch to execution another thread. So if you have more threads than cores/processors and you know "now I did in this thread a lot of work and next work can be done little-bit later" you call Thread.Sleep(1) and allows another thread to do some work sooner than the native switcher will "pause" the currently executed thread.
Try this: Write a program that launches 100 threads, and put each of the thread into a for loop as you described. And then write another that launches 100 threads and uses Thread.Sleep instead.
Run them both and compare the CPU usage. You'll see the point. =)
Thread.Sleep() simply causes the executing thread to halt for the specified duration.
I've seen many developers use Thread.Sleep() because they don't probably handle the joining of dependent threads. They simply use Thread.Sleep() to force a thread to wait for some amount of time until the think their other threads would have finished and have their data available.
If you have two threads that need to wait on each other to proceed with their processing, you should really use the mechanisms built in to .NET that are meant to handle situations like that (ie. ManualResetEvent, etc.)
Thread.Sleep() is OK to use in some situations eg. watchdog threads.
However in your case, it may not seem to be the optimal solution as pointed out by others.
Without a code sample, it's hard to tell, but based on your description, I don't think it's a question of Thread.Sleep() or not. I would suspect that you may be suffering from a race condition - that's usually why you experience "random" buggy behavior or even "random" crashes in multithreaded code - as seems to be what you are experiencing.
For whatever reason, your for-loop may cause the subtle critical timings of the race condition to occur less often, but it won't solve the root cause. There are many pitfalls to be aware of when doing multithreaded programming, I can only advice you to read up on the topic if you want to be able to avoid these.
I'll recommend reading http://www.amazon.com/Concurrent-Programming-Windows-Joe-Duffy/dp/032143482X
Everything that I read about sockets in .NET says that the asynchronous pattern gives better performance (especially with the new SocketAsyncEventArgs which saves on the allocation).
I think this makes sense if we're talking about a server with many client connections where its not possible to allocate one thread per connection. Then I can see the advantage of using the ThreadPool threads and getting async callbacks on them.
But in my app, I'm the client and I just need to listen to one server sending market tick data over one tcp connection. Right now, I create a single thread, set the priority to Highest, and call Socket.Receive() with it. My thread blocks on this call and wakes up once new data arrives.
If I were to switch this to an async pattern so that I get a callback when there's new data, I see two issues
The threadpool threads will have default priority so it seems they will be strictly worse than my own thread which has Highest priority.
I'll still have to send everything through a single thread at some point. Say that I get N callbacks at almost the same time on N different threadpool threads notifying me that there's new data. The N byte arrays that they deliver can't be processed on the threadpool threads because there's no guarantee that they represent N unique market data messages because TCP is stream based. I'll have to lock and put the bytes into an array anyway and signal some other thread that can process what's in the array. So I'm not sure what having N threadpool threads is buying me.
Am I thinking about this wrong? Is there a reason to use the Async patter in my specific case of one client connected to one server?
UPDATE:
So I think that I was mis-understanding the async pattern in (2) above. I would get a callback on one worker thread when there was data available. Then I would begin another async receive and get another callback, etc. I wouldn't get N callbacks at the same time.
The question still is the same though. Is there any reason that the callbacks would be better in my specific situation where I'm the client and only connected to one server.
The slowest part of your application will be the network communication. It's highly likely that you will make almost no difference to performance for a one thread, one connection client by tweaking things like this. The network communication itself will dwarf all other contributions to processing or context switching time.
Say that I get N callbacks at almost
the same time on N different
threadpool threads notifying me that
there's new data.
Why is that going to happen? If you have one socket, you Begin an operation on it to receive data, and you get exactly one callback when it's done. You then decide whether to do another operation. It sounds like you're overcomplicating it, though maybe I'm oversimplifying it with regard to what you're trying to do.
In summary, I'd say: pick the simplest programming model that gets you what you want; considering choices available in your scenario, they would be unlikely to make any noticeable difference to performance whichever one you go with. With the blocking model, you're "wasting" a thread that could be doing some real work, but hey... maybe you don't have any real work for it to do.
The number one rule of performance is only try to improve it when you have to.
I see you mention standards but never mention problems, if you are not having any, then you don't need to worry what the standards say.
"This class was specifically designed for network server applications that require high performance."
As I understand, you are a client here, having only a single connection.
Data on this connection arrives in order, consumed by a single thread.
You will probably loose performance if you instead receive small amounts on separate threads, just so that you can assemble them later in a serialized - and thus like single-threaded - manner.
Much Ado about Nothing.
You do not really need to speed this up, you probably cannot.
What you can do, however is to dispatch work units to other threads after you receive them.
You do not need SocketAsyncEventArgs for this. This might speed things up.
As always, measure & measure.
Also, just because you can, it does not mean you should.
If the performance is enough for the foreseeable future, why complicate matters?