A c# application I’m developing consists essentially of performing operations on images coming from a camera and printing them on a picturebox.
It has a library written in c++ that retrieves images from a webcam (imgcam), and makes copies of them (imgcam_copy), then they are delivered to the managed code who is requesting them.
In turn the managed part of the application consists of a secondary thread who executes a while cycle that takes images from the unmanaged library and prints them to a picturebox.
Everything works fine and the most delicate part of everything, that is the management of dynamic resources from unmanaged to managed code seems clear to me and I’m aware of what I’m doing (at least I hope), but the problem is with the writing of the code necessary for making threads work properly.
Indeed there are many things that are not clear to me, even after browsing the web for hours.
I’m compelled to print the image with UI thread, so I have to rely on Invoke.
delegate void setImageCallback(Image img);
private void showFrame(Image img)
{
if (pboxCam.InvokeRequired)
{
this.Invoke(new setImageCallback(showFrame), img);
}
else
{
pboxCam.Image = img;
}
}
I have many questions:
1) Is Invoke a costly operation? I’ve made many efforts to reduce the execution time of main operations on images, it would be disappointing to waste part of the gain in just showing the result.
2) Do you think that it is better to use synchronously Invoke or asynchronously BeginInvoke+copy of image?
3) Could it be helpful not having Image as function parameters but accessing it as member of a class ?
private delegate void setImageCallback();
private void showFrame()
{
if (pboxCam.InvokeRequired)
{
this.Invoke(new setImageCallback(showFrame));
}
else
{
pboxCam.Image = cm.bitmap;
}
}
4) maybe you won’t share my worry for the performance, but I would like to know if you share the one for thread safety. The UI thread just wants to display an image while the non-UI thread makes a copy of it, is it really unsafe to rely on an asynchronous mechanism?
Calling Invoke is expensive only in that it involves a thread context switch and (possibly) waiting for the UI thread to become available. But if your UI thread isn't tied up doing a bunch of other stuff, and you're not trying to do hundreds of picturebox updates a second, then it's unlikely to be a noticeable problem. If the image is of any significant size, drawing the pixels is going to take so much time that the microseconds it takes for the Invoke will be insignificant.
More importantly, you must do the update on the UI thread. So either you do all of your processing on the UI thread, or you use Invoke to marshal the update to the UI thread. Since you don't want to tie up the UI thread doing calculations, you don't really have a choice.
The difference between having img as a function parameter or as a member of a class is insignificant.
BeginInvoke can be useful, but you have to be careful. All BeginInvoke does is queue a request so that your background thread can get on with its work and the UI thread can update its display when it has time. It's very easy to queue so many requests with BeginInvoke that the UI thread is spending near all of its time handling those requests and the rest of the UI appears to lock up because user-initiated actions got stuffed in the queue after the update actions. If you're doing many dozens of image updates per second, depending of course on the image size, your UI thread will never catch up. Your UI will lock up and eventually you'll run out of memory to queue requests.
It appears that your major performance bottlenecks here are doing the calculations on the image and then displaying the image. Both of those operations are so time intensive that whatever time you spend in Invoke is likely irrelevant.
If your program's performance using Invoke is good enough, then you're probably better off leaving it as it is. You can try using BeginInvoke but as you say that'll require cloning the image. In addition, you might experience a locked-up UI.
In short, the answer to your questions is, "it depends." I don't know enough about your image processing or the amount of time it takes to do that processing. If you're curious, try it with BeginInvoke and see. Worst that can happen is that your program crashes and you have to go back to using Invoke. But be sure to test thoroughly over an extended period.
Your last question is unclear. If you're asking if it's dangerous to do UI updates on a background thread, the answer is a resounding "Yes!" Updating the UI on any thread other than the UI thread can lead to all manner of strange and wonderful bugs that are nearly impossible to duplicate at times, and very difficult to track down. Windows expects UI elements to be updated on a single thread.
Related
How InvokeRequired and Invoke let us make our apps thread safe.
Let's consider such code:
private void ThreadSafeUpdate(string message)
{
if (this.textBoxSome.InvokeRequired)
{
SetTextCallback d = new SetTextCallback(msg);
this.Invoke
(d, new object[] { message });
}
else
{
// It's on the same thread, no need for Invoke
this.textBoxSome.Text = message;
}
}
Is it possible to change state of InvokeRequired after InvokeRequired and before Invoke? If not, then why?
How does Invoking make it thread safe?
If InvokeRequired illustrate is current thread owning control, how would the thread know that it is or it is not the owner.
Let's consider that SomeMethod() is currently running on Thread1. We would like to call it from Thread2. Internally this method updates some field. Does Method.Invoke contain some kind of lock mechanism internally?
What if SomeMethod() takes very long time and we would like to run something other on the control owner thread. Does Invoking lock the owner thread or is it some kind of a background thread safe task?
ThreadSafeUpdate() //takes 5 minutes in Thread2
ThreadSafeUpdate() //after 2 minutes, we are running it in other thread2
ThreadSafeUpdate() //next run from Thread3
I think it is some kind of general pattern which can be implemented outside of winforms, what's its name?
Is it possible to change state of InvokeRequired
Yes, and it is a pretty common occurrence. Either because you started the thread too soon, before the form's Load event fired. Or because the user closed the window just as this code is running. In both cases this code fails with an exception. InvokeRequired fails when the thread races ahead of the window creation, the invoked code fails when the UI thread races ahead of the thread. The odds for an exception are low, too low to ever diagnose the bug when you test the code.
How Invoking make it thread safe?
You cannot make it safe with this code, it is a fundamental race. It must be made safe by interlocking the closing of the window with the thread execution. You must make sure that the thread stopped before allowing the window to close. The subject of this answer.
how would he know that he is or he is not owner.
This is something that can be discovered with a winapi call, GetWindowsThreadProcessId(). The Handle property is the fundamental oracle for that. Pretty decent test, but with the obvious flaw that it cannot work when the Handle is no longer valid. Using an oracle in general is unwise, you should always know when code runs on a worker thread. Such code is very fundamentally different from code that runs on the UI thread. It is slow code.
We would like to call it from Thread2
This is not in general possible. Marshaling a call from one thread to a specific other thread requires that other thread to co-operate. It must solve the producer-consumer problem. Take a look at the link, the fundamental solution to that problem is a dispatcher loop. You probably recognize it, that's how the UI thread of a program operates. Which must solve this problem, it gets notifications from arbitrary other threads and UI is never thread-safe. But worker threads in general don't try to solve this problem themselves, unless you write it explicitly, you need a thread-safe Queue and a loop that empties it.
What's if SomeMethod() takes very long time
Not sure I follow, the point of using threads is to let code that takes a long time not do anything to harm the responsiveness of the user interface.
I think it is some kind of general pattern
There is, it doesn't look like this. This kind of code tends to be written when you have an oh-shoot moment and discover that your UI is freezing. Bolting threading on top of code that was never designed to support threading is forever a bad idea. You'll overlook too many nasty little details. Very important to minimize the number of times the worker thread interacts with the UI thread, your code is doing the opposite. Fall in the pit of success with the BackgroundWorker class, its RunWorkerCompleted event gives a good synchronized way to update UI with the result of the background operation. And if you like Tasks then the TaskScheduler.FromCurrentSynchronizationContext() method helps you localize the interactions.
Usually, no. But it could happen if you're using await between the InvokeRequired check and Invoke call without capturing the execution context. Of course, if you're already using await, you're probably not going to be using Invoke and InvokeRequired.
EDIT: I just noticed that InvokeRequired will return false when the control handle hasn't been created yet. It shouldn't make much of a difference, because your call will fail anyway when the control hasn't quite been created yet, but it is something to keep in mind.
It doesn't make it thread-safe. It just adds the request to the control's queue, so that it's executed the next available time on the same thread the control was created on. This has more to do with windows architecture than with general thread-safety. The end result, however, is that the code runs on a single thread - of course, this still means you need to handle shared state synchronization manually, if any.
Well, it's complicated. But in the end, it boils down to comparing the thread ID of the thread that created the control, and the current thread ID. Internally, this calls the native method GetWindowThreadProcessId - the operating system keeps track of the controls (and more importantly, their message loops).
Invoke cannot return until the GUI thread returns to its message loop. Invoke itself only posts the command to the queue and waits for it to be processed. But the command is run on the GUI thread, not the Invoke-caller. So the SomeMethod calls in your example will be serialized, and the Invoke call itself will wait until the second call finishes.
This should already be answered. The key point is "only run GUI code on the GUI thread". That's how you get reliable and responsive GUI at all times.
You can use it anywhere you've got a loop or a wait on some queue. It probably isn't all that useful, although I have actually used it already a few times (mostly in legacy code).
However, all of this is just a simple explanation of the workings. The truth is, you shouldn't really need InvokeRequired... well, ever. It's an artifact of a different age. This is really mostly about juggling threads with little order, which isn't exactly a good practice. The uses I've seen are either lazy coding, or hotfixes for legacy code - using this in new code is silly. The argument for using InvokeRequired is usually like "it allows us to handle this business logic safely whether it runs in the GUI thread or not". Hopefully, you can see the problem with that logic :)
Also, it's not free thread-safety. It does introduce delays (especially when the GUI thread is also doing some work that isn't GUI - very likely in code that uses InvokeRequired in the first place). It does not protect you from accesses to the shared state from other threads. It can introduce deadlocks. And don't even get me started on doing anything with code that uses Application.DoEvents.
And of course, it's even less useful once you take await into consideration - writing asynchronous code is vastly easier, and it allows you to make sure the GUI code always runs in the GUI context, and the rest can run wherever you want (if it uses a thread at all).
I have a C# Windows Forms application wicht does some camera control and computer vision. For all the parts which take longer for calculation I used seperate threads. But there are still some parts which are in the callback functions of the GUI. As I understand, all these callback functions are executed in the same thread. Is there a way to see how much time this thread is working or idle? What percentage of idle time is needed such that the GUI is still responsive?
It's recommended that you shouldn't block the UI thread for more than 50ms, otherwise it will affect the UI responsiveness. I.e., two UI callbacks queued with Form.BeginInvoke, each taking ~50ms to complete, may introduce some unpleasant UI experience to the user.
It doesn't make sense to update the UI more often than the user can react to it (i.e, ~24 frames per second). So, you should throttle the UI thread callbacks and give user input events a priority.
I recently posted an example of how it can possibly be done:
https://stackoverflow.com/a/21654436/1768303
For simple tasks you could use a stopwatch and measure the time manually. However I think you'll need to check what a performance profiler is.
Also - there is little situations in which your GUI needs that heavy processing. In most cases the problem comes from putting too much calculations in event handlers instead of implementing them somewhere outside and then update the form when finished. It's less of a single/multi-threading problem and more of using available events properly.
I am working with a fairly complex GUI and am trying to pass a lot of data from the GUI to a backgroudWorker. The problem I am running into is accessing some of the GUI values from the background worker. For example, if I try to get ComboBox.Text I get a InvalidOperationException due to cross-threading. However, if I say do TextBox.Text, everything seems to work fine. Granted I am fairly new to C#, so I'm a little unclear on why some of these are OK and others fail.
I have come up with several ways to fix my issues but am seeking the best practice from someone who is experienced in c#.
Here are a couple ways i can think of fixing this
create class/struct of all the values you want to pass to the background worker and pass this when you call RunworkAsync. I did not find this very attractive as i was having to build a class/struct for every page on my GUI to pass to the backgroundWorker
Create a bunch of different background workers that had specific task. I still had some issues with passing data but the amount of data I had to pass was cut down quite a bit. However, the number of DoWork/ProgressChanged/RunworkerCompleted went up significantly which was less than ideal.
(this lead me to what I'm currently doing)
create a delegate and method to capture the information
private delegate string ReadComboDelegate(ComboBox c);
private string ReadComboBox(ComboBox c)
{
if(c.InvokeRequired)
{
ReadComboDelegate del = new ReadComboDelegate(this.ReadComboBox);
return (string) c.Invoke(del,c);
}
else
{
return c.Text
}
}
then within DoWork, do somthing like string txt = this.ReadComboBox(this.comboBox1);
When you have a simple GUI and you don't have to pass a lot of data this is pretty simple problem. However, the more items and complex the GUI gets the bigger this problem becomes. If anyone has any info that would make this easier, I would appreciate it.
Thanks
The Cross Threading issue you are running into is due to the requirement that only the UI thread is allowed to "touch" UI controls.
I think that the most agreed upon method of passing data to a background worker is your solution #1 - create a simple structure that contains all of the data needed to perform the processing.
This is much simpler than creating ReadXXX methods for every control in the UI, and it defines what the background process needs to perform its task...
It is rather by accident that TextBox doesn't cause this exception. Its Text property is cached in a string. That's not the case for ComboBox.Text, and the vast majority of other control properties, it asks the native Windows control and at that point Windows Forms discovers that you are trying to use a control from a thread other than the UI thread. No can do.
You definitely need to think of a way to restructure this code. It is not only illegal, it is incredibly expensive and fundamentally thread unsafe since the UI could be updated while your worker is running. Collect the info from the controls you need into a little helper class, pass that as an argument to the RunWorkerAsync(object) overload. And get it back in DoWork from e.Argument.
I would definitely avoid #3. Despite the fervor over using Control.Invoke to coordinate worker and UI threads it is often overused and is usually a suboptimal strategy at best. I much prefer #1 and #2 over #3. Here are the reasons why I tend to avoid #3.
It tightly couples the UI and worker threads.
The worker thread gets to dictate how much work the UI thread performs.
The worker thread has to wait for the UI thread to respond before proceeding.
It is an expensive operation.
I know it may require some additional upfront effort on your part to get #1 or #2 going, but the end result will be better in the long run.
As a corollary to my answer the Control.Invoke method tends to be overused when data needs to follow the opposite direction as well (from worker thread to UI thread as in the case of sending progress information to the UI). Sadly this is the method that BackgroundWorker uses internally with its ReportProgress method. It is usually better to have the UI thread poll a shared data structure for this information for some of the same reasons as above plus:
The UI thread gets to dictate when and how often the update should take place.
It puts the responsibility of updating the UI thread on the UI thread where it should belong anyway.
There is no risk of the UI message pump being overrun as would be the case with the marshaling techniques initiated by the worker thread.
However, with that said I am not suggesting that you abandon BackgroundWorker entirely. Just keep some of these points in mind.
I am using c# to integrate with a web cam. I need to generate a snapshot image every x milliseconds and save it to file.
I already have the code up and running to save to file on a button click event, however I wonder what am I supposed to do when taking snapshots in the background - Should this be multi threaded? I'm honestly not sure.
I could just block the UI thread, put Thread.Sleep and then just take the snapshot, but I don't know if this is right.
I thought of using a background worker, but I am now experiencing cross threaded difficulties with SendMessage... So I wonder if I should even go and bother to multi-thread or just block the UI.
There will be a physical hardware limit to how fast the camera can update its pixel buffer. Webcams don't go far above 30fps. Getting the actual image should be more or less instantaneous (unless at very high res), so you would not require threading to start off with. When I did it a while ago I used the approach as given on
http://weblogs.asp.net/nleghari/pages/webcam.aspx
I think you should put this task on a separate thread. The process of creating and saving the image may take more time is some situations and at that time your HMI may freeze. To avoid this put this task on a separate thread.
You could create a timer to kick a delegate every n milliseconds and that delegate could queue a worker thread to do what your OnClick() handler does already.
I would NOT write this as a single-threaded app because, depending on the performance of the user's webcam, you could easily end up in an eternal loop handling timer events, causing your main UI thread to be permanently blocked.
ThreadQueue.QueueUserWorkitem((args) =>
{
// Blah ...
}
should not require much effort to get working correctly.
I want to paralelize a 3D voxel editor built on top of Windows Forms, it uses a raycaster to render so dividing the screen and getting each thread on a pool to render a part of it should be trivial.
The problem arises in that Windows Forms' thread must run as STA - I can get other threads to start and do the work but blocking the main thread while waiting for them to finish causes strange random deadlocks as expected.
Keeping the main thread unblocked would also be a problem - if, for example, the user uses a floodfill tool the input would be processed during the rendering process which would cause "in-between" images (an object partially colored, for example). Copying the entire image before every frame isn't doable either because the volumes are big enough to offset any performance gain if it has to be copied every frame.
I want to know if there is any workaround to get the amin thread to appear blocked to the user in a way that it will not be actually blocked but will delay the processing of input till the next frame.
If it isn't possible, is there a better design for dealing with this?
EDIT: Reading the anwsers I think I wasn't clear that the raycaster runs in real time, so showing progress dialogs won't work at all. Unfortunately the FPS is low enough (5-40 depending on various factors) for the input between frames to produce unwanted results.
I have already tried to implement it blocking the UI thread and using some threads of a ThreadPool to process and it works fine except for this problem with STA.
This is a common problem. With windows forms you can have only one UI thread. Don't run your algorithm on the UI thread because then the UI will appear frozen.
I recommend running your algorithm and waiting for it to finish before updating the UI. A class called BackgroundWorker comes pre-built to do just this very thing.
Edit:
Another fact about the UI thread is that it handles all of the mouse and keyboard events, along with system messages that are sent to the window. (Winforms is really just Win32 surrounded by a nice API.) You cannot have a stable application if the UI thread is saturated.
On the other hand, if you start several other threads and try to draw directly on the screen with them, you may have two problems:
You're not supposed to draw on the UI with any thread but the UI thread. Windows controls are not thread safe.
If you have a lot of threads, context switching between them may kill your performance.
Note that you (and I) shouldn't claim a performance problem until it has been measured. You could try drawing a frame in memory and swapping it in at an appropriate time. Its called double-buffering and is very common in Win32 drawing code to avoid screen flicker.
I honestly don't know if this is feasible with your target frame rate, or if you should consider a more graphics-centered library like OpenGL.
Am I missing something or can you just set your render control (and any other controls that generate input events) to disabled while you're rendering a frame? That will prevent unwanted inputs.
If you still want to accept events while you're rendering but don't want to apply them until the next frame, you should leave your controls enabled and post the detail of the event to an input queue. That queue should then be processed at the start of every frame.
This has the affect that the user can still click buttons and interact with the UI (the GUI thread does not block) and those events are not visible to the renderer until the start of the next frame. At 5 FPS, the user should see their events are processed within 400ms worst case (2 frames), which isn't quite fast enough, but better than threading deadlocks.
Perhaps something like this:
Public InputQueue<InputEvent> = new Queue<InputEvent>();
// An input event handler.
private void btnDoSomething_Click(object sender, EventArgs e)
{
lock(InputQueue)
{
InputQueue.Enqueue(new DoSomethingInputEvent());
}
}
// Your render method (executing in a background thread).
private void RenderNextFrame()
{
Queue<InputEvent> inputEvents = new Queue<InputEvent>();
lock(InputQueue)
{
inputEvents.Enqueue(InputQueue.Dequeue());
}
// Process your input events from the local inputEvents queue.
....
// Now do your render based on those events.
....
}
Oh, and do your rendering on a background thread. Your UI thread is precious, it should only do the most trivial work. Matt Brundell's suggestion of BackgroundWorker has lots of merit. If it doesn't do what you want, the ThreadPool is also useful (and simpler). More powerful (and complex) alternatives are the CCR or the Task Parallel Library.
Show a modal "Please Wait" dialog using ShowDialog, then close it once your rendering is finished.
This will prevent the user from interacting with the form while still allowing you to Invoke to the UI thread (which is presumably your problem).
If you don't want all the features offered by the BackgroundWorker you can simply use the ThreadPool.QueueUserWorkItem to add something to the thread pool and use a background thread. It would be easy to show some kind of progress while the background thread was performing it's operations as you can provide a delegate callback to notify you whenever a particular background thread is done. Take a look at ThreadPool.QueueUserWorkItem Method (WaitCallback, Object) to see what I'm referring you to. If you need something more complex you could always use the APM async method to perform your operations as well.
Either way I hope this helps.
EDIT:
Notify user somehow that changes are being made to the UI.
On a(many) background threads using the ThreadPool perform the ops you need to perform to the UI.
For each operation keep a reference to the state for the operation so that you know when it completed in the WaitCallback. Maybe put them in some type of hash / collection to keep ref to them.
Whenever an operation completes remove it from the collection that contains a ref to the ops that were performed.
Once all operations have completed (hash / collection) has no more references in it render the UI with the changes applied. Or possibly incrementally update the UI
I'm thinking that if you are making so many updates to the UI while you are performing your operations that is what is causing your problems. That's also why I recommended the use of SuspendLayout, PerformLayout as you may have been performing so many updates to the UI the main thread was getting overwhelmed.
I am no expert on threading though, just trying to think it through myself. Hope this helps.
Copying the entire image before every frame isn't doable either because the volumes are big enough to offset any performance gain if it has to be copied every frame.
Then don't copy the off-screen buffer on every frame.