We Keep Coding

C# - Compact Framework Windows CE - GUI Thread Blocks Higher Priority Threads

I am having an issue where I have a Windows CE compact framework Application written in C#, where I have the primary GUI thread set to normal priority and a communication thread set to above normal priority to get as close to pseudo real time performance. The issue I am having is within a button handler I run a loop to load config data from a file to the GUI before allowing it to be edited. This takes around 2-3 seconds to complete. While this blocking in the event handler is happening, my higher priority communication thread is being blocked. There are no locks are thread syncs in place. The communicatio thread has no dependencies on the GUI thread.
This is how I spawn my comm thread:
MbWorkerThread = new Thread(MbPollingThread);
MbWorkerThread.IsBackground = true;
MbWorkerThread.Priority = ThreadPriority.AboveNormal;
MbWorkerThread.Start();
It is an MTA application. Also, I have tried to use Thread.Sleep(1) in the GUI event handler to yield to the higher priority thread and it does not work. I also tried using signals to yield to the higher priority thread, and that does not work. The only thing that works is if I place Application.DoEvents() in the loop while loading config in the event handler. This of coarse whas just a test, as I do not want to sprinkle Application.DoEvents() throught my code to make it work since I know Application.DoEvents() is dangerous.
My understanding is that the primary GUI thread is a foreground thread, but a thread none the less. Also, I have made the communication thread a background thread just to allow it to be killed when the primary thread is exited.
I have tried everything, I have search the Internet endlessly before asking this question.
Any help will be greatly appreciated.
P.S. - I though about a form timer but I know it runs in the GUI thread so that would not help. I though about another thread but I really did not what to marshall GUI updates via Invoke.

Your program starts in Main(), where you typically call Application.Run( new MyForm() ). Application.Run() implements the standard Windows Message Pump, which deals with messages from the OS and other applications, including user input, inter-process communication, repaint requests, etc.
GUI events, like Button click, are dispatched via this thread. If you perform long-running work in an event handler, other messages are not being processed.
Application.DoEvents() blocks the calling thread, and waits for all pending messages to be processed. If DoEvents helps your communication thread when Sleep(1) did not, then I suspect there is a dependency between your communication thread and the GUI/Message Pump thread.
Even if this is not the case, it is not a good idea to block the GUI thread. Move your file loading into the background with ThreadPool.QueueUserWorkItem() and marshal the results back to the UI at the end with Invoke or BeginInvoke.

BeginInvoke instead of Invoke fixed the issue. Thanks for the replies.

Alternative to Thread.Sleep that keeps the UI responsive?

I'm doing all this in C#, in Visual Studio 2008.
I want to slow down the work of my algorithm so that the user can watch it's work. There is a periodic change visible at the GUI so I added Thread.Sleep after every instance.
Problem is that Thread.Sleep, when set to at least a second, after a few instances of Thread.Sleep (after few loops) simply freezes entire GUI and keeps it that way till program completion. Not right away, but it always happens. How soon depends on the length of the sleep.
I have proof that entire program does not freeze, it's working it's thing, even the sleep is making pauses of correct length. But the GUI freezes at certain point until the algorithm ends, at which point it shows the correct final state.
How to solve this issue? Alternative to pausing algorithm at certain point?

First off, don't make the user wait for work that is done before they even think about when it will be finished. Its pointless. Please, just say no.
Second, you're "sleeping" the UI thread. That's why the UI thread is "locking up." The UI thread cannot be blocked; if it is, the UI thread cannot update controls on your forms and respond to system messages. Responding to system messages is an important task of the UI thread; failing to do so makes your application appear locked up to the System. Not a good thing.
If you want to accomplish this (please don't) just create a Timer when you start doing work that, when it Ticks, indicates its time to stop pretending to do work.
Again, please don't do this.

I'd guess everything is running out of a single thread. The user probably invokes this algorithm by clicking on a button, or some such. This is handled by your main thread's message queue. Until this event handler returns, your app's GUI cannot update. It needs the message queue to be pumped on regular basis in order to stay responsive.
Sleeping is almost never a good idea, and definitely not a good idea in the GUI thread. I'm not going to recommend that you continue to use sleep and make your GUI responsive by calling Application.DoEvents.
Instead, you should run this algorithm in a background thread and when it completes it should signal so to the main thread.

You are about to commit some fairly common user interface bloopers:
Don't spam the user with minutiae, she's only interested in the result
Don't force the user to work as fast as you demand
Don't forbid the user to interact with your program when you are busy.
Instead:
Display results in a gadget like a ListBox to allow the user to review results at her pace
Keep a user interface interactive by using threads
Slow down time for your own benefit with a debugger

This depends on a lot of things, so its hard to give a concrete answer from what you've said. Still, here are some matters that might be relevant:
Are you doing this on a UI thread (e.g. the thread the form-button or UI event that triggered the work started on)? If so, it may be better to create a new thread to perform the work.
Why do you sleep at all? If the state related to the ongoing work is available to all relevant threads, can the observer not just observe this without the working thread sleeping? Perhaps the working thread could write an indicator of the current progress to a volatile or locked variable (it must be locked if it's larger than pointer size - e.g. int or an object - but not otherwise. If not locked, then being volatile will prevent cache inconsistency between CPUs, though this may not be a big deal). In this case you could have a forms timer (there are different timers in .Net with different purposes) check the status of that variable and update the UI to reflect the work being done, without the working thread needing to do anything. At most it may be beneficial to Yield() in the working thread on occasion, but its not likely that even this will be needed.

interface freezes in c# multi-threaded app

I have a c# .NET multi-threaded application that is freezing the interface. What is unusual about this is that the interface does not freeze unless I let the system sit idle long enough for the screen saver to start (which requires me to reenter my password to re-gain access to the system). When the interface becomes visible again (after I have successfully entered my password) all the windows are white. I can see the window titles, move the windows around, minimize them and such, but the screens are not repainting. When I break all and enter the debugger, the call stack has Application.Run(), external code, and then "in a sleep, wait, or join". I put break points in all four of the threads I open and they are still running, it is just the main app's UI thread that is blocked. When I look at my thread list, what was my main thread and my four worker threads now consists of my main thread and 11 worker threads. I didn't open this many threads so it must be the serialport class.
Now let me describe my program.
My main app allows users to collect and monitor data from serial ports. I have implemented this in the following way. When a connection is desired, a button is pressed on the main app which calls a function in a DLL which opens a status window and then launches a thread which monitors the serial port. When that function returns, the main app launches a thread to monitor a queue created in the DLL when it is initialized. When data is received from the serial port, the data is parsed and then the status window is updated (via a delegate) and the data is pushed onto the queue. When the main apps worker thread sees data in the queue it retrieves it and posts it in a list box on the main app, using a delegate. In all cases I use BeginInvoke to call these delegates.
My DLL contains two libraries for the two different types of equipment it can communicate with.
This problem occurs when I have a connection to two devices; hence the four worker threads two for each device. The DLL itself is setup as a comm object so I can access it easily from a C++/MFC app and a c# app, both of which utilize it.
I found that if I add code to the thread inside the DLL so it calls Application.DoEvents() every 30 seconds, the interface will be frozen for about 30 seconds and then resume activity like normal. I figure something is blocking the main thread and forcing DoEvents() to fire seems to break the lock, but I have no idea what might be causing this lock. This is not a solution, just something of interest.
I would appreciate any suggestions you might have. Thanks.

I found that if I add code to the thread inside the DLL so it calls Application.DoEvents() every 30 seconds, the interface will be frozen for about 30 seconds and then resume activity like normal. I figure something is blocking the main thread and forcing DoEvents() to fire seems to break the lock, but I have no idea what might be causing this lock. This is not a solution, just something of interest.
I would recommend running your program under the new Visual Studio 2010 Concurrency Profiler. This will show you, at runtime, which threads are blocked, and which objects they are waiting on. Thread contention is explicitly marked and highlighted for you.
You can use this to easily determine what code is causing the deadlock on your UI thread.

Try changing your Thread Start code to Thread.Start() instead of BeginInvoke(). BeginInvoke does not keep threads tryky seperate from your UI, as it and it may be interacting strangely with DoEvents. You can read up on BeginInvoke and how it works here: http://www.codeproject.com/KB/cs/begininvoke.aspx
Also, DoEvents is NEVER necessary in an application, and can cause a lot of unexpected behavior. Use threadding with UI calls wrapped in a Control.Invoke(...) statement. If you're using .NET 3.5+, you can make this easy with delegates that look like this: Invoke((Action)delegate() {*code goes here*});

Multithreading on Windows Forms

I want to paralelize a 3D voxel editor built on top of Windows Forms, it uses a raycaster to render so dividing the screen and getting each thread on a pool to render a part of it should be trivial.
The problem arises in that Windows Forms' thread must run as STA - I can get other threads to start and do the work but blocking the main thread while waiting for them to finish causes strange random deadlocks as expected.
Keeping the main thread unblocked would also be a problem - if, for example, the user uses a floodfill tool the input would be processed during the rendering process which would cause "in-between" images (an object partially colored, for example). Copying the entire image before every frame isn't doable either because the volumes are big enough to offset any performance gain if it has to be copied every frame.
I want to know if there is any workaround to get the amin thread to appear blocked to the user in a way that it will not be actually blocked but will delay the processing of input till the next frame.
If it isn't possible, is there a better design for dealing with this?
EDIT: Reading the anwsers I think I wasn't clear that the raycaster runs in real time, so showing progress dialogs won't work at all. Unfortunately the FPS is low enough (5-40 depending on various factors) for the input between frames to produce unwanted results.
I have already tried to implement it blocking the UI thread and using some threads of a ThreadPool to process and it works fine except for this problem with STA.

This is a common problem. With windows forms you can have only one UI thread. Don't run your algorithm on the UI thread because then the UI will appear frozen.
I recommend running your algorithm and waiting for it to finish before updating the UI. A class called BackgroundWorker comes pre-built to do just this very thing.
Edit:
Another fact about the UI thread is that it handles all of the mouse and keyboard events, along with system messages that are sent to the window. (Winforms is really just Win32 surrounded by a nice API.) You cannot have a stable application if the UI thread is saturated.
On the other hand, if you start several other threads and try to draw directly on the screen with them, you may have two problems:
You're not supposed to draw on the UI with any thread but the UI thread. Windows controls are not thread safe.
If you have a lot of threads, context switching between them may kill your performance.
Note that you (and I) shouldn't claim a performance problem until it has been measured. You could try drawing a frame in memory and swapping it in at an appropriate time. Its called double-buffering and is very common in Win32 drawing code to avoid screen flicker.
I honestly don't know if this is feasible with your target frame rate, or if you should consider a more graphics-centered library like OpenGL.

Am I missing something or can you just set your render control (and any other controls that generate input events) to disabled while you're rendering a frame? That will prevent unwanted inputs.
If you still want to accept events while you're rendering but don't want to apply them until the next frame, you should leave your controls enabled and post the detail of the event to an input queue. That queue should then be processed at the start of every frame.
This has the affect that the user can still click buttons and interact with the UI (the GUI thread does not block) and those events are not visible to the renderer until the start of the next frame. At 5 FPS, the user should see their events are processed within 400ms worst case (2 frames), which isn't quite fast enough, but better than threading deadlocks.
Perhaps something like this:
Public InputQueue<InputEvent> = new Queue<InputEvent>();
// An input event handler.
private void btnDoSomething_Click(object sender, EventArgs e)
{
lock(InputQueue)
{
InputQueue.Enqueue(new DoSomethingInputEvent());
}
}
// Your render method (executing in a background thread).
private void RenderNextFrame()
{
Queue<InputEvent> inputEvents = new Queue<InputEvent>();
lock(InputQueue)
{
inputEvents.Enqueue(InputQueue.Dequeue());
}
// Process your input events from the local inputEvents queue.
....
// Now do your render based on those events.
....
}
Oh, and do your rendering on a background thread. Your UI thread is precious, it should only do the most trivial work. Matt Brundell's suggestion of BackgroundWorker has lots of merit. If it doesn't do what you want, the ThreadPool is also useful (and simpler). More powerful (and complex) alternatives are the CCR or the Task Parallel Library.

Show a modal "Please Wait" dialog using ShowDialog, then close it once your rendering is finished.
This will prevent the user from interacting with the form while still allowing you to Invoke to the UI thread (which is presumably your problem).

If you don't want all the features offered by the BackgroundWorker you can simply use the ThreadPool.QueueUserWorkItem to add something to the thread pool and use a background thread. It would be easy to show some kind of progress while the background thread was performing it's operations as you can provide a delegate callback to notify you whenever a particular background thread is done. Take a look at ThreadPool.QueueUserWorkItem Method (WaitCallback, Object) to see what I'm referring you to. If you need something more complex you could always use the APM async method to perform your operations as well.
Either way I hope this helps.
EDIT:
Notify user somehow that changes are being made to the UI.
On a(many) background threads using the ThreadPool perform the ops you need to perform to the UI.
For each operation keep a reference to the state for the operation so that you know when it completed in the WaitCallback. Maybe put them in some type of hash / collection to keep ref to them.
Whenever an operation completes remove it from the collection that contains a ref to the ops that were performed.
Once all operations have completed (hash / collection) has no more references in it render the UI with the changes applied. Or possibly incrementally update the UI
I'm thinking that if you are making so many updates to the UI while you are performing your operations that is what is causing your problems. That's also why I recommended the use of SuspendLayout, PerformLayout as you may have been performing so many updates to the UI the main thread was getting overwhelmed.
I am no expert on threading though, just trying to think it through myself. Hope this helps.

Copying the entire image before every frame isn't doable either because the volumes are big enough to offset any performance gain if it has to be copied every frame.
Then don't copy the off-screen buffer on every frame.

Shift Control from backgrount thread to main thread

I have thread that monitor the status of the device( using i/o ). This will fire the event to several UI (Forms/Dialogs..) parts about the connection status(Connected, Disconnected, Fault). Based on this status, the forms and dialogs are destroyed, created , enabled and disabled.
My Problem :
I getting Cross-Thread exception because the thread doing the operations like Dispose the from, create the from .... . I using smart client, forms and dialogs are part of the WorkItemController and i just terminating that.
I don't want u use Invoke or BeginInvoke . I want this thread to continue only monitoring the status. SO that this thread has to give its control to the main thread and the main thread will close the form or create the form.
Just i want how to shift the control from one thread to another. Is any way is there?

The best solution is to use Invoke to switch back to your mainthread.
or if you can use global variables shared between the main thread and your background thread, but this means your main thread should check the variables in a loop and this may hang GUI.
Or use System.Windows.Forms.Timer as it executes in the main thread.