I have a C# desktop application in which one thread that I create continously gets an image from a source(it's a digital camera actually) and puts it on a panel(panel.Image = img) in the GUI(which must be another thread as it is the code-behind of a control.
The application works but on some machines I get the following error at random time intervals(unpredictable)
************** Exception Text **************
System.InvalidOperationException: The object is currently in use elsewhere.
Then the panel turns into a red cross, red X - i think this is the invalid picture icon that is editable from the properties. The application keeps working but the panel is never updated.
From what I can tell this error comes from the control's onpaint event where I draw something else on the picture.
I tried using a lock there but no luck :(
The way I call the function that puts the image on the panel is as follows:
if (this.ReceivedFrame != null)
{
Delegate[] clients = this.ReceivedFrame.GetInvocationList();
foreach (Delegate del in clients)
{
try
{
del.DynamicInvoke(new object[] { this,
new StreamEventArgs(frame)} );
}
catch { }
}
}
this is the delegate:
public delegate void ReceivedFrameEventHandler(object sender, StreamEventArgs e);
public event ReceivedFrameEventHandler ReceivedFrame;
and this is how the function inside the control code-behind registers to it:
Camera.ReceivedFrame +=
new Camera.ReceivedFrameEventHandler(camera_ReceivedFrame);
I also tried
del.Method.Invoke(del.Target, new object[] { this, new StreamEventArgs(b) });
instead of
del.DynamicInvoke(new object[] { this, new StreamEventArgs(frame) });
but no luck
Does anyone know how I could fix this error or at least catch the error somehow and make the thread put the images on the panel once again?
This is because Gdi+ Image class is not thread safe. Hovewer you can avoid InvalidOperationException by using lock every time when you need to Image access, for example for painting or getting image size:
Image DummyImage;
// Paint
lock (DummyImage)
e.Graphics.DrawImage(DummyImage, 10, 10);
// Access Image properties
Size ImageSize;
lock (DummyImage)
ImageSize = DummyImage.Size;
BTW, invocation is not needed, if you will use the above pattern.
I had a similar problem with the same error message but try as I might, locking the bitmap didn't fix anything for me. Then I realized I was drawing a shape using a static brush. Sure enough, it was the brush that was causing the thread contention.
var location = new Rectangle(100, 100, 500, 500);
var brush = MyClass.RED_BRUSH;
lock(brush)
e.Graphics.FillRectangle(brush, location);
This worked for my case and lesson learned: Check all the reference types being used at the point where thread contention is occurring.
Seems to me, that the same Camera object is used several times.
E.g. try to use a new buffer for each received frame. It seems to me, that while the picture box is drawing the new frame, your capture library fills that buffer again. Therefore on faster machines this might not be an issue, with slower machines it might be an issue.
I've programmed something similar once, after each received frame, we had to request to receive the next frame and set the NEW frame receive buffer in that request.
If you can not do that, copy the received frame from the camera first to a new buffer and append that buffer to a queue, or just use 2 alternating buffers and check for overruns. Either use myOutPutPanel.BeginInvoke to call the camera_ReceivedFrame method, or better have a thread running, which checks the queue, when it has a new entry it calls mnyOutPutPanel.BeginInvoke to invoke your method to set the new buffer as image on the panel.
Furthermore, once you received the buffer, use the Panel Invoke Method to invoke the setting of the image (guarantee that it runs in the window thread and not the thread from your capture library).
The example below can be called from any thread (capture library or other separate thread):
void camera_ReceivedFrame(object sender, StreamEventArgs e)
{
if(myOutputPanel.InvokeRequired)
{
myOutPutPanel.BeginInvoke(
new Camera.ReceivedFrameEventHandler(camera_ReceivedFrame),
sender,
e);
}
else
{
myOutPutPanel.Image = e.Image;
}
}
I think this is multithreading problem
Use windows golden rule and update the panel in the main thread use panel.Invoke
This should overcome cross threading exception
Related
I'm working on a project using EmguCV which requires capturing images from a camera and processing them, looking for certain things in the image. The processing takes up about ~.2 seconds, and is the biggest time-sink in the application so we're looking into making the capturing/processing different threads to speed up the overall process.
We've already tried the VideoCapture.ImageGrabbed event handler, and calling Retrieve, as well as setting up a threaded loop of our own calling QueryFrame and any other capturing method we can find.
Most of the threaded solutions end up causing empty images (not caught by the .IsEmpty property of the Mat, though, for some reason), which end up as images saved with 0 bytes.
After this, we tried simplifying but ran into an issue where the camera is always a few seconds behind, due to the buffer internal to the library. This leads to my question: is there a way to refresh the buffer, or clear it of memory? We cannot dispose of the capture object because of the time overhead for creating the object. Any tips about threading the capturing and processing are welcome as well, though the other suggestions I've found around this site about similar situations have not led to much success.
A brief example of the code we're using
_capture.ImageGrabbed += GrabFrames;
...
public void GrabFrames()
{
Mat image = new Mat();
_capture.Retrieve(image);
ProcessThread = new Thread(new ParameterizedThreadStart(StartProcess));
ProcessThread.Start(image);
}
...
public void StartProcess(object image)
{
Mat img = (Mat)image;
Process(image);
img.Dispose();
}
I have two high speed USB3 cameras (Ximea) and want to code an application for image recording. Framerates are up to 500fps at VGA resolution but I also want to use the 2Mpx resolution at 170fps.
Their .Net SDK tells me that I should simply "get" the images in a loop.
My problem is that I have no idea how to get the images and save them while still showing a live preview. Everytime I add some code to actually update the picturebox the frame rate drops drastically.
At the moment I utilize a recording function that is called with
Task.Run(() => Record());
and inside the Record() I have a loop getting the bitmaps
while(record == true)
{
Camera.GetImage(out myImage, timeout); //From Ximea .Net SDK
Info = Camera.GetLastImageParams();
Timestamp = Info.GetTimeStamp();
ThreadPool.QueueUserWorkItem(state => SaveImage(myImage, filepath, Timestamp));
}
with the SaveImage being
private void SaveImage(Bitmap myImage, string filepath, double Timestamp)
{
try
{
lock(myImage)
{
myImage.Save(filepath + Timestamp.ToString("0.00000") + ".tif");
}
}
catch{}
}
How can I show a live preview while recording and how can I make the entire code more stable (at the moment there are some dropped frames because of "object already in use"-errors or "generic error in GDI+" at the Image.Save() call, that I skip with the try/catch statement)?
I believe you can tell the Ximea API how many image buffers you want in the incoming queue... use XI_PRM_BUFFER_POLICY and XI_PRM_BUFFERS_QUEUE_SIZE appropriately to make that queue length somewhat long.
Then, have a thread that, when activated, copies an image out of a XI_IMG struct into your own buffer. Activate that thread every n frames (based on the size of the Ximea image buffer queue size)... but don't do any memory copies in the loop that actually calls xiGetImage. You should probably block in your thread to avoid tearing (because the Ximea code could get back around to using the same buffer again if you're not fast enough at copying the data out)... but you could then dynamically adjust the number of buffers so you can finish your copy within the time you have. Also, you may consider copying the image data to ANOTHER buffer if you're doing something that takes a long time...
pseudo-code (sorry, it's C-ish):
// sync objects and a global image buffer pointer
CRITICAL_SECTION cs;
void *buf;
HANDLE ev;
int CopyImageThreadProc(...)
{
while (true)
{
if (WaitOnSingleObject(ev) == WAIT_OBJ_0)
{
EnterCriticalSection(cs);
// copy the image data at buf where ever you want
LeaveCriticalSection(cs);
}
}
}
int main(...)
{
// set up ximea api with appropriate buffering
// create event and critsec, start thread
while (!done)
{
XI_IMG img;
xiGetImage(dev, 10, &img);
// every 15 frames, tell your thread to go...
// if you find that the critsec is causing a hiccup, you can adjust this
// but remember to adjust the queue length, too
// if you change this to TRY entercriticalsection, you can determine that
if ((img.acq_nframe % 15) == 0)
{
EnterCriticalSection(cs);
buf = img.bp;
SetEvent(ev);
LeaveCriticalSection(cs);
}
}
// clean up
}
Add each captured frame to a queue, then have a worker thread that takes those images, one at a time, and saves them. Trying to write multiple images to disk at the same time will most likely be slower. Also, always Dispose of any GDI objects or you will run into trouble really fast. I would think not doing that is what is giving you the exceptions.
As for showing the images, make sure you are not trying to display every image. Your monitor most likely runs at 60 Hz, so anything faster than that will be a waste. I also suspect that (with the performance of GDI), you won't necessarily be able to achieve even that. So I suggest you have a second queue with images to display, and if you see that queue getting too large, your program will need to slow down a bit and not push as many frames to the queue.
Edit: And of course, as #Franck mentions, if your disk can't keep up, your queue/buffer will fill up quickly. Compressing the images might help, assuming they have suitable content for compression and that your processor can keep up.
Edit: What you need is a producer-consumer pattern. There are many ways of doing this, but one might be something like:
// blocking collection
private BlockingCollection<Bitmap> m_Queue = ...
// camera thread
while( run )
{
var bitmap = GrabFrame();
m_Queue.Add( bitmap );
}
// worker thread
try
{
while( true )
{
// Take() will block if the queue is empty
var bitmap = m_Queue.Take();
bitmap.Save( ... );
bitmap.Dispose();
}
catch( InvalidOperationException )
{
// you'll end up here if you call `m_Queue.CompleteAdding()`
// (after the queue has been emptied, of course)
}
As for displaying the images, you could probably use something similar, with the addition of having some code that determines if it's time to push a new image or not.
Hi Stack Overflow members,
I'm a newbie to C# programming. I am developing a basic camera streaming and still capture application. Once user takes still, I will be displaying it on overlay using VMR9's bitmap mixing concept.
What I did?
I am making use of C# direct show library from here
First I get all required filters interfaces. Find the attached capture device. Called Render stream with source filter and vmr9 for PREVIEW pin. Source filter, sample grabber and null renderer for STILL PIN.
I am having three menu buttons -> take still, show overlay and hide overlay.
I am making use of bitmap mixer sample provided in that library.
Each time user presses Take Still menu, image will be saved in desktop and will be re-sized to small resolution and displayed on video overlay.
Show Overlay and hide overlay calls ShowHideBitmap() which perform operation of querying VMR9BitmapMixer interface from vmr9 filter, fills VMR9AlphaBitmap structure and then calls IVMRMixerBitmap9.SetAlphaBitmap function.
What issue I face?
After taking still, if I call ShowHideBitmap() through menu option, the still image taken is being updated perfectly on overlay.
This is another option that performs automatic update of overlay as soon as still image is saved. I create event based thread and made it to wait for update event created using EventWaitHandle. Before returning from samplegrabber BufferCB function, I set this update event. Which in turn proceeds with waiting thread. Inside thread I call ShowHideBitmap function. In this scenario, I receive error message as follows.
Unable to case COM object of type 'DirectShowLib.VideoMixingRenderer9' to interface type 'DirectShowLib.IVMRMixerBitmap9'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{ced175e5-1935-4820-81bd-ff6ad00c9108}' failed due to the following error: No such interface supported (Exception from HRESULT: 0X80040002 (E_NOINTERFACE)
Here is the code block of ShowHideBitmap function
//Declarations
private static IBaseFilter vmr9 = null;
private static IVMRMixerBitmap9 vmr9mixerBitmap = null;
private IVMRWindowlessControl9 vmr9windowlessCtrl = null;
private static void ShowHideBitmap(Boolean bEnable)
{
int hr = 0;
VMR9AlphaBitmap alphaBmp;
if (!bEnable)
{
if (vmr9mixerBitmap != null)
{
// Get current Alpha Bitmap Parameters
hr = vmr9mixerBitmap.GetAlphaBitmapParameters(out alphaBmp);
DsError.ThrowExceptionForHR(hr);
// Disable them
alphaBmp.dwFlags = VMR9AlphaBitmapFlags.Disable;
// Update the Alpha Bitmap Parameters
hr = vmr9mixerBitmap.UpdateAlphaBitmapParameters(ref alphaBmp);
DsError.ThrowExceptionForHR(hr);
// Create a surface from our alpha bitmap
surface.Dispose();
vmr9mixerBitmap = null;
//Release this alpha bitmap source.
if (alphaBitmap != null)
{
alphaBitmap.Dispose();
}
}
return;
}
else
{
try
{
alphaBitmap = BitmapGenerator.GenerateAlphaBitmap();
// Create a surface from our alpha bitmap
if(surface == null)
surface = new Surface(device, alphaBitmap, Pool.SystemMemory);
// Get the unmanaged pointer
unmanagedSurface = surface.GetObjectByValue(DxMagicNumber);
if (vmr9mixerBitmap == null)
vmr9mixerBitmap = (IVMRMixerBitmap9)vmr9;
// Set Alpha Bitmap Parameters for using a Direct3D surface
alphaBmp = new VMR9AlphaBitmap();
alphaBmp.dwFlags = VMR9AlphaBitmapFlags.EntireDDS;
alphaBmp.pDDS = unmanagedSurface;
alphaBmp.rDest = GetDestRectangle();
alphaBmp.fAlpha = 1.0f;
// Set Alpha Bitmap Parameters
hr = vmr9mixerBitmap.SetAlphaBitmap(ref alphaBmp);
DsError.ThrowExceptionForHR(hr);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
}
And here is the thread the waits for update event.
Thread overlayupdatethreadhandle = new Thread(new ThreadStart(overlayupdatethread));
overlayupdatethreadhandle.Start();
private void overlayupdatethread()
{
do
{
overlayupdateeventhandle.WaitOne();
ShowHideBitmap(GlobalVar.m_ShowOverlay);
} while (true);
}
I have tried updating this overlay using timer which was running at background with interval of 100ms. Using timer was working good, but for this operation, using timer is of bad choice. So i moved with threading concept.
Why is that getting interface failed while calling from thread and works good when calling from menu options? Should I have to take care of any special thing? I have even tried parametrized thread, but no luck.
Thanks in advance for your help.
EDIT: If ShowHideBitmap is called from Main Thread, every thing works fine. If ShowHideBitmap is called from worker thread, COM object creates Exception. How to handle this cross-thread operation?
The exception is ratty, not uncommon in COM. What it really means is "I have no idea how to give you an interface reference that you can use from a worker thread". Which is a common kind of mishap, these kind of COM components just are not thread-safe at all. And they enforce that by either taking care of it, marshaling the calls from the worker thread to the owner thread automatically. Or by not letting you use them from another thread at all because marshaling would be pointless, making it way too slow. VMR falls in the latter category.
This is very unlike .NET, it also has a lot of classes that are completely thread-unsafe. Basic stuff too, none of the collection classes are for example. But it lets you use these classes in a thread anyway, leaving it up to you to make it thread-safe. This quite often goes wrong of course, using proper locking is a skill.
COM has always been thread-aware by design. With the philosophy that threading is very hard to get right so it should be taken care of by the smart people. Which works fantastically 95% of the time. And gives you a major migraine the rest of the time. The kind of migraine induced by the hard to diagnose poor perf when COM takes care of threading. And the crappy error reporting when it doesn't.
Well, no can do, you really do have to use that interface from the same thread that created the VMR instance. No way around that.
I had error E_NOINTERFACE when trying to use Listener / Event handler object from Delphi library. To overcome issue with marshaling & different threads I saved dispatcher of thread that assigns listener and then use it to fire events.
Interfaces:
[ComVisible(true)]
[Guid("2FFC2C20-A27B-4D67-AEA3-350223D3655F")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface IDataSystemInterfaceEventListener
{
void OnIntializeCompleted(int status);
void OnTerminateCompleted(int status);
void OnRunCompleted(int status);
}
[ComVisible(true)]
[Guid("B9953413-A8C9-4CE2-9263-B488CA02E7EC")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface IDataSystemInterface
{
void Intialize(string config);
void StartRun(string conditions);
void StopRun();
void Terminate();
IDataSystemInterfaceEventListener Listener { get; set; }
}
Then implementation (notice Dispatcher.CurrentDispatcher stored)
[ComVisible(true)]
[Guid("0818F830-DA37-4167-BF31-3A2C55A9BF2B")]
public class DataSystemModule : IDataSystemInterface
{
private Dispatcher m_dispatcherListener = null;
private IDataSystemInterfaceEventListener m_listener = null;
public IDataSystemInterfaceEventListener Listener
{
get
{
return m_listener;
}
set
{
m_dispatcherListener = Dispatcher.CurrentDispatcher;
m_listener = value;
}
}
}
Then in code:
if (Listener != null)
{
m_dispatcherListener.Invoke((Action)delegate()
{
Listener.OnTerminateCompleted((int)TerminateStatus.Completed);
});
}
Without dispacther if Listener is called in different thread it will produce error
I have a Winform with 4 PictureBox controls, each control will contain a diferent image. The process is:
An event x is raised, the eventargs from this event, contain the filenames of each image (4), an so on (file exists etc..). Then, I have to update the UI.
Commonly I use Invoke:
Invoke((ThreadStart)delegate()
{
picBig.Image = new Bitmap(strImageBig);
picLittle1.Image = new Bitmap(saLittle[0]);
picLittle2.Image = new Bitmap(saLittle[1]);
picLittle3.Image = new Bitmap(saLittle[2]);
});
// saLittle[] is a string array, contains, filenames: "image1.jpg"
But when this executes, the form freezes for a little time, about 500ms, I know it's a small interval but it's noticeable, then the app continues normally.
I was trying to find out the reason of that 'UI freeze', then, after research, I found BeginInvoke. Now my code looks like this:
BeginInvoke((MethodInvoker)delegate
{
picBig.Image = new Bitmap(strImageBig);
picLittle1.Image = new Bitmap(saLittle[0]);
picLittle2.Image = new Bitmap(saLittle[1]);
picLittle3.Image = new Bitmap(saLittle[2]);
});
This is a little faster. But the UI is still freezing for 200~300ms.
In the articles I've read, they say that BeginInvoke is a better way than Invoke.
The code is working OK, there is no problem with logic or anything else. I just want to know why this happens. I don't want to leave this doubt unclear. The project is already finished. Hope this be useful for someone else.
Maybe this is not the correct approach. I know there are many ways to update the UI from a background thread, but is there another way to make an update faster?
Or, do you think, the image loading is the reason? Is there another way to do a faster loading of images?
Thanks in advance.
This is because you're actually loading your images from disk on the UI thread along with setting the contents of the control. Calling the Bitmap constructor with a file path will go to the hard drive and load the image into memory.
Invoke and BeginInvoke will run the delegate that you provide on the thread that the control was created on, which is most likely going to be the UI thread.
...but is there another way to make an update faster?
Load the images on your background thread and, when they're actually loaded, invoke and set the images into the controls.
var big = new Bitmap(strImageBig);
var little1 = new Bitmap(saLittle[0]);
var little2 = new Bitmap(saLittle[1]);
var little3 = new Bitmap(saLittle[2]);
Invoke((ThreadStart)delegate()
{
picBig.Image = big;
picLittle1.Image = little1;
picLittle2.Image = little2;
picLittle3.Image = little3;
});
But when this executes, the form freezes for a little time, about 500ms, I know it's a small interval but it's noticeable, then the app continues normally.
Eventually, the UI thread needs to actually update the images. When the images are generated and updated on the UI thread, this will cause a (short) delay.
Currently my program reads images and text in a record from an xml file, displays them on the screen, and then the click of the previous/next buttons moves to the next record. However, it seems to need a few seconds loading time between each photo and I'd like it to be instant, like how Windows Photo Gallery would...or Facebook photos (bear in mind this is not a web app).
I searched found a few similar situations to mine but none seemed to fit my situation. I tried making a class, based on my search, to deal with background loading and calling it in my program, but it's fraught with error and probably won't even do what I want it do:
//ImageManager.cs
class ImageManager
{
private Dictionary<string, Image> images = new Dictionary<string, Image>();
public Image get(string s)
{ // blocking call, returns the image
return load(s);
}
private Image load(string s)
{ // internal, thread-safe helper
lock (images)
{
if (!images.ContainsKey(s))
{
Image img = images.Add(s, img); //load the image s - ERROR cannot implicitly convert type void to image. Void??
return img;
}
return images[s];
}
}
public void preload(params string[] imgs)
{ // non-blocking preloading call
foreach (string img in imgs)
{
BackgroundWorker bw = new BackgroundWorker();
bw.DoWork += (s, e) => { load(img); }; // discard the actual image return
bw.RunWorkerAsync();
}
}
}
//MainWindow.cs
ImageManager im = new ImageManager();
im.preload("Data/Images"); // Errors - im is a field but used like a type/token '('
Many thanks in advance
Your ImageManager should work with ImageSources, not Images. Even if you get your current code to work you'll find that your UI still hangs because you have no choice but to perform the work on the UI thread. If you instead deal with ImageSources, you can load them on a background thread and then freeze them in order to use them from the UI thread. This frees you to pre-emptively load images, or to show a loading animation whilst they load.
BitmapFrame.Create is likely the method you want to be using to load the images.
Consider caching scaled down images - 1:1 of what you want to show, or even smaller. This way loading of preview will be much faster and if user looks at the image long enough you can load full image.
With modern photos original size of the image is usually way bigger than can be normally diaplayed. So if you always read original images you spend large amount of disk IO on something that will never be shown.
Usual note: it may not be case in your program. As with any performance issues measure, than optimize.