Hi Stack Overflow members,
I'm a newbie to C# programming. I am developing a basic camera streaming and still capture application. Once user takes still, I will be displaying it on overlay using VMR9's bitmap mixing concept.
What I did?
I am making use of C# direct show library from here
First I get all required filters interfaces. Find the attached capture device. Called Render stream with source filter and vmr9 for PREVIEW pin. Source filter, sample grabber and null renderer for STILL PIN.
I am having three menu buttons -> take still, show overlay and hide overlay.
I am making use of bitmap mixer sample provided in that library.
Each time user presses Take Still menu, image will be saved in desktop and will be re-sized to small resolution and displayed on video overlay.
Show Overlay and hide overlay calls ShowHideBitmap() which perform operation of querying VMR9BitmapMixer interface from vmr9 filter, fills VMR9AlphaBitmap structure and then calls IVMRMixerBitmap9.SetAlphaBitmap function.
What issue I face?
After taking still, if I call ShowHideBitmap() through menu option, the still image taken is being updated perfectly on overlay.
This is another option that performs automatic update of overlay as soon as still image is saved. I create event based thread and made it to wait for update event created using EventWaitHandle. Before returning from samplegrabber BufferCB function, I set this update event. Which in turn proceeds with waiting thread. Inside thread I call ShowHideBitmap function. In this scenario, I receive error message as follows.
Unable to case COM object of type 'DirectShowLib.VideoMixingRenderer9' to interface type 'DirectShowLib.IVMRMixerBitmap9'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{ced175e5-1935-4820-81bd-ff6ad00c9108}' failed due to the following error: No such interface supported (Exception from HRESULT: 0X80040002 (E_NOINTERFACE)
Here is the code block of ShowHideBitmap function
//Declarations
private static IBaseFilter vmr9 = null;
private static IVMRMixerBitmap9 vmr9mixerBitmap = null;
private IVMRWindowlessControl9 vmr9windowlessCtrl = null;
private static void ShowHideBitmap(Boolean bEnable)
{
int hr = 0;
VMR9AlphaBitmap alphaBmp;
if (!bEnable)
{
if (vmr9mixerBitmap != null)
{
// Get current Alpha Bitmap Parameters
hr = vmr9mixerBitmap.GetAlphaBitmapParameters(out alphaBmp);
DsError.ThrowExceptionForHR(hr);
// Disable them
alphaBmp.dwFlags = VMR9AlphaBitmapFlags.Disable;
// Update the Alpha Bitmap Parameters
hr = vmr9mixerBitmap.UpdateAlphaBitmapParameters(ref alphaBmp);
DsError.ThrowExceptionForHR(hr);
// Create a surface from our alpha bitmap
surface.Dispose();
vmr9mixerBitmap = null;
//Release this alpha bitmap source.
if (alphaBitmap != null)
{
alphaBitmap.Dispose();
}
}
return;
}
else
{
try
{
alphaBitmap = BitmapGenerator.GenerateAlphaBitmap();
// Create a surface from our alpha bitmap
if(surface == null)
surface = new Surface(device, alphaBitmap, Pool.SystemMemory);
// Get the unmanaged pointer
unmanagedSurface = surface.GetObjectByValue(DxMagicNumber);
if (vmr9mixerBitmap == null)
vmr9mixerBitmap = (IVMRMixerBitmap9)vmr9;
// Set Alpha Bitmap Parameters for using a Direct3D surface
alphaBmp = new VMR9AlphaBitmap();
alphaBmp.dwFlags = VMR9AlphaBitmapFlags.EntireDDS;
alphaBmp.pDDS = unmanagedSurface;
alphaBmp.rDest = GetDestRectangle();
alphaBmp.fAlpha = 1.0f;
// Set Alpha Bitmap Parameters
hr = vmr9mixerBitmap.SetAlphaBitmap(ref alphaBmp);
DsError.ThrowExceptionForHR(hr);
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
}
And here is the thread the waits for update event.
Thread overlayupdatethreadhandle = new Thread(new ThreadStart(overlayupdatethread));
overlayupdatethreadhandle.Start();
private void overlayupdatethread()
{
do
{
overlayupdateeventhandle.WaitOne();
ShowHideBitmap(GlobalVar.m_ShowOverlay);
} while (true);
}
I have tried updating this overlay using timer which was running at background with interval of 100ms. Using timer was working good, but for this operation, using timer is of bad choice. So i moved with threading concept.
Why is that getting interface failed while calling from thread and works good when calling from menu options? Should I have to take care of any special thing? I have even tried parametrized thread, but no luck.
Thanks in advance for your help.
EDIT: If ShowHideBitmap is called from Main Thread, every thing works fine. If ShowHideBitmap is called from worker thread, COM object creates Exception. How to handle this cross-thread operation?
The exception is ratty, not uncommon in COM. What it really means is "I have no idea how to give you an interface reference that you can use from a worker thread". Which is a common kind of mishap, these kind of COM components just are not thread-safe at all. And they enforce that by either taking care of it, marshaling the calls from the worker thread to the owner thread automatically. Or by not letting you use them from another thread at all because marshaling would be pointless, making it way too slow. VMR falls in the latter category.
This is very unlike .NET, it also has a lot of classes that are completely thread-unsafe. Basic stuff too, none of the collection classes are for example. But it lets you use these classes in a thread anyway, leaving it up to you to make it thread-safe. This quite often goes wrong of course, using proper locking is a skill.
COM has always been thread-aware by design. With the philosophy that threading is very hard to get right so it should be taken care of by the smart people. Which works fantastically 95% of the time. And gives you a major migraine the rest of the time. The kind of migraine induced by the hard to diagnose poor perf when COM takes care of threading. And the crappy error reporting when it doesn't.
Well, no can do, you really do have to use that interface from the same thread that created the VMR instance. No way around that.
I had error E_NOINTERFACE when trying to use Listener / Event handler object from Delphi library. To overcome issue with marshaling & different threads I saved dispatcher of thread that assigns listener and then use it to fire events.
Interfaces:
[ComVisible(true)]
[Guid("2FFC2C20-A27B-4D67-AEA3-350223D3655F")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface IDataSystemInterfaceEventListener
{
void OnIntializeCompleted(int status);
void OnTerminateCompleted(int status);
void OnRunCompleted(int status);
}
[ComVisible(true)]
[Guid("B9953413-A8C9-4CE2-9263-B488CA02E7EC")]
[InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface IDataSystemInterface
{
void Intialize(string config);
void StartRun(string conditions);
void StopRun();
void Terminate();
IDataSystemInterfaceEventListener Listener { get; set; }
}
Then implementation (notice Dispatcher.CurrentDispatcher stored)
[ComVisible(true)]
[Guid("0818F830-DA37-4167-BF31-3A2C55A9BF2B")]
public class DataSystemModule : IDataSystemInterface
{
private Dispatcher m_dispatcherListener = null;
private IDataSystemInterfaceEventListener m_listener = null;
public IDataSystemInterfaceEventListener Listener
{
get
{
return m_listener;
}
set
{
m_dispatcherListener = Dispatcher.CurrentDispatcher;
m_listener = value;
}
}
}
Then in code:
if (Listener != null)
{
m_dispatcherListener.Invoke((Action)delegate()
{
Listener.OnTerminateCompleted((int)TerminateStatus.Completed);
});
}
Without dispacther if Listener is called in different thread it will produce error
Related
Here is the basic idea of my code:
private void CaptureCameraFrame()
{
Capture = new VideoCapture();
CameraModel.Instance.CameraViewMat = Capture.QueryFrame();
// do stuff with queried matrix here
if(noAbortCondition)
{
CaptureCameraFrame();
}
}
The method should run in a separate thread updating my GUI with the current image after processing.
Only Problem is, that I get two different kinds of error:
Attempt to read/write protected memory: This happens on the second runthrough
of the method.
I get an null-reference error using `CameraModel.Instance.CameraViewMat right after querying the frame.
The two issues seem to be connected, seems like QueryFrame() runs asynchronously from the rest of the code and isn't done when the program jumps to the next step.
Question is: How can I make sure, that querying the image from the camera is finished, and I can use the information in the matrix as well as start a new query?
In all the examples I have found this is done by using time, but I would like to start with a new frame as soon as processing on the last frame is done.
I haven't really done much in C# when it comes to threading, but what I understand in such cases one would use the asyncand awaitkeywords to make sure a method in an asynchronous method is finished. However I wasn't able to make a working implementation in this case.
You are creating VideoCapture class instance repeatedly and even not disposing of it. Create your VideoCapture instance only once and use them for your task. At the end dispose it.
public YourConstructor()
{
Capture = new VideoCapture();
}
private void CaptureCameraFrame()
{
CameraModel.Instance.CameraViewMat = Capture.QueryFrame();
// do stuff with queried matrix here
if(noAbortCondition)
{
CaptureCameraFrame();
}
}
Hopefully, it will work for you!
Im using System.Windows.Application.Current.Dispatcher.BeginInvoke(DispatcherPriority.Input, new Action(() => ... for a wpf graphic refresh.
It works in my other function greatfully, but in my SQL delete function it wount be triggered/executed.
I tried it with System.Windows.Forms.Application.DoEvents(); but it wount do anything.
Set_Loading_Changed()
{
System.Windows.Application.Current.Dispatcher.BeginInvoke(
DispatcherPriority.Input,
new Action(() =>
{
if (BLoading)
{
DataGrid_Anzeige.IsEnabled = false;
Mouse.OverrideCursor = System.Windows.Input.Cursors.Wait;
}
else
{
DataGrid_Anzeige.IsEnabled = true;
Mouse.OverrideCursor = null;
}
}));
}
Btn_Remove()
{
...
Set_Loading_Changed();
using (OleDbConnection ODC = new OleDbConnection("..."))
{
foreach (var selectedRow in DataGrid_Anzeige.SelectedItems.OfType<DataRowView>())
{
sSQL_Statement = "...";
ODC.Open();
OleDbCommand ODCmd = new OleDbCommand(sSQL_Statement, ODC);
ODCmd.ExecuteNonQuery();
ODC.Close();
EDIT:
I insert the complete part of my Set_Load_Changed() function, hope you can get a clue with this informations.
Im using it primarly in my search Thread (Task.Factory.StartNew(() => { ... }));) so it must be the DispatcherPriority.Input.
You're running into a common issue with misunderstanding the WPF threading system. The way WPF is structured is with one thread for the program to run and modify the UI in, usually called the UI thread, and a second thread which you have no normal way of using, which automatically renders the UI, commonly called the rendering or compositing thread.
The key point you need to know here is that if you stall the UI thread with a large operation (like a database read or a large calculation) immediately after BeginInvoke(), then you're preventing the UI thread from running those commands until you allow it to invoke the next action. BeginInvoke() simply queues the action to be performed the next time the dispatcher is allowed - the dispatcher will not interrupt what is currently being done. Setting the priority to Input ensures that it will be handled ahead of other lower priority work, but still will not cause it to interrupt your current method.
If you instead call Invoke(), you are interrupting your work to ask the dispatcher to perform the action and then return to what you're doing when it's finished.
While this is preferable to the behavior you're currently getting, this isn't how you're intended to use the dispatcher, and will still cause your app to appear 'frozen' while it completes the long operation. To avoid this, the easiest thing to do is run the long operation in a Task, using the async/await keywords and the Task Parallel Library.
Stephen Cleary has an excellent blog where he covers a lot of topics related to this. His introductory post (dating back to the keywords' introduction) is here.
I would encourage poking around his blog if you have more issues in this area - he's one of the leading experts in explaining this area, and has covered most of the problems you run into.
Further reading:
What's the difference between Invoke() and BeginInvoke()?
WPF Threading Model
To change the cursor in WPF is unfortunately not as straightforward as in WinForms. I remember struggling with it myself until I stumbled upon the following solution. I didn't come up with this myself, I'll try and find the source to give credit where it is due.
using System;
using System.Collections.Generic;
using System.Windows.Input;
namespace MyNamespace
{
public class OverrideCursor : IDisposable
{
static Stack<Cursor> s_Stack = new Stack<Cursor>();
public OverrideCursor(Cursor changeToCursor = null)
{
if (changeToCursor == null)
changeToCursor = Cursors.Wait;
s_Stack.Push(changeToCursor);
if (Mouse.OverrideCursor != changeToCursor)
Mouse.OverrideCursor = changeToCursor;
}
public void Dispose()
{
s_Stack.Pop();
var cursor = _stack.Count > 0 ? _stack.Peek() : null;
if (Mouse.OverrideCursor != cursor)
Mouse.OverrideCursor = cursor;
}
}
}
Now this disposable class can be used anywhere in your project to change the cursor temporarily.
using (new OverrideCursor())
{
//your code
}
This will change the cursor to anything you want by passing the cursor as parameter of the constructor, or nothing to use Cursors.Wait by default.
For the time needed to execute any code placed inside the using-block the cursor will be changed turning back to normal afterwards.
You can also initiate an object of the class without the using-block to set it indefinitely but you shouldn't forget to call Dispose() when done.
Edit: source: https://stackoverflow.com/a/675686/4579864
If want to do whatever you are doing in Set_Loading_Changed() before you connect to the database, you should call Invoke instead of BeginInvoke:
Set_Loading_Changed()
{
System.Windows.Application.Current.Dispatcher.Invoke(...);
}
What's the difference between Invoke() and BeginInvoke()
I've just "earned" the privilege to maintain a legacy library coded in C# at my current work.
This dll:
Exposes methods for a big legacy system made with Uniface, that has no choice but calling COM objects.
Serves as a link between this legacy system, and another system's API.
Uses WinForm for its UI in some cases.
More visually, as I understand the components :
*[Big legacy system in Uniface]* ==[COM]==> [C# Library] ==[Managed API]==> *[Big EDM Management System]*
The question is: One of the methods in this C# Library takes too long to run and I "should" make it asynchronous!
I'm used to C#, but not to COM at all. I've already done concurrent programming, but COM seems to add a lot of complexity to it and all my trials so far end in either:
A crash with no error message at all
My Dll only partially working (displaying only part of its UI, and then closing), and still not giving me any error at all
I'm out of ideas and resources about how to handle threads within a COM dll, and I would appreciate any hint or help.
So far, the biggest part of the code I've changed to make my method asynchronous :
// my public method called by the external system
public int ComparedSearch(string application, out string errMsg) {
errMsg = "";
try {
Action<string> asyncOp = AsyncComparedSearch;
asyncOp.BeginInvoke(application, null, null);
} catch (ex) {
// ...
}
return 0;
}
private int AsyncComparedSearch(string application) {
// my actual method doing the work, that was the called method before
}
Any hint or useful resource would be appreciated.
Thank you.
UPDATE 1:
Following answers and clues below (especially about the SynchronizationContext, and with the help of this example) I was able to refactor my code and making it to work, but only when called from another Window application in C#, and not through COM.
The legacy system encounters a quite obscure error when I call the function and doesn't give any details about the crash.
UPDATE 2:
Latest updates in my trials: I managed to make the multithreading work when the calls are made from a test project, and not from the Uniface system.
After multiple trials, we tend to think that our legacy system doesn't support well multithreading in its current config. But that's not the point of the question any more :)
Here is a exerpt of the code that seems to work:
string application;
SynchronizationContext context;
// my public method called by the external system
public int ComparedSearch(string application, out string errMsg) {
this.application = application;
context = WindowsFormsSynchronizationContext.Current;
Thread t = new Thread(new ThreadStart(AsyncComparedSearchAndShowDocs));
t.Start();
errMsg = "";
return 0;
}
private void AsyncComparedSearch() {
// ANY WORK THAT AS NOTHING TO DO WITH UI
context.Send(new SendOrPostCallback(
delegate(object state)
{
// METHODS THAT MANAGE UI SOMEHOW
}
), null);
}
We are now considering other solutions than modifying this COM assembly, like encapsulating this library in a Windows Service and creating an interface between the system and the service. It should be more sustainable..
It is hard to tell without knowing more details, but there are few issues here.
You execute the delegate on another thread via BeginInvoke but you don't wait for it. Your try\catch block won't catch anything as it has already passed while the remote call is still being executed. Instead, you should put try\catch block inside AsyncComparedSearch.
As you don't wait for the end of the execution of remote method (EndInvoke or via callback) I am not sure how do you handle the results of the COM call. I guess then that you update the GUI from within AsyncComparedSearch. If so, it is wrong, as it is running on another thread and you should never update GUI from anywhere but the GUI thread - it will most likely result with a crash or other unexpected behavior. Therefore, you need to sync the GUI update work to GUI thread. In WinForms you need to use Control.BeginInvoke (don't confuse it with Delegate.BeginInvoke) or some other way (e.g. SynchronizationContext) to sync the code to GUI thread. I use something similar to this:
private delegate void ExecuteActionHandler(Action action);
public static void ExecuteOnUiThread(this Form form, Action action)
{
if (form.InvokeRequired) { // we are not on UI thread
// Invoke or BeginInvoke, depending on what you need
form.Invoke(new ExecuteActionHandler(ExecuteOnUiThread), action);
}
else { // we are on UI thread so just execute the action
action();
}
}
then I call it like this from any thread:
theForm.ExecuteOnUiThread( () => theForm.SomeMethodWhichUpdatesControls() );
Besides, read this answer for some caveats.
I have a C# desktop application in which one thread that I create continously gets an image from a source(it's a digital camera actually) and puts it on a panel(panel.Image = img) in the GUI(which must be another thread as it is the code-behind of a control.
The application works but on some machines I get the following error at random time intervals(unpredictable)
************** Exception Text **************
System.InvalidOperationException: The object is currently in use elsewhere.
Then the panel turns into a red cross, red X - i think this is the invalid picture icon that is editable from the properties. The application keeps working but the panel is never updated.
From what I can tell this error comes from the control's onpaint event where I draw something else on the picture.
I tried using a lock there but no luck :(
The way I call the function that puts the image on the panel is as follows:
if (this.ReceivedFrame != null)
{
Delegate[] clients = this.ReceivedFrame.GetInvocationList();
foreach (Delegate del in clients)
{
try
{
del.DynamicInvoke(new object[] { this,
new StreamEventArgs(frame)} );
}
catch { }
}
}
this is the delegate:
public delegate void ReceivedFrameEventHandler(object sender, StreamEventArgs e);
public event ReceivedFrameEventHandler ReceivedFrame;
and this is how the function inside the control code-behind registers to it:
Camera.ReceivedFrame +=
new Camera.ReceivedFrameEventHandler(camera_ReceivedFrame);
I also tried
del.Method.Invoke(del.Target, new object[] { this, new StreamEventArgs(b) });
instead of
del.DynamicInvoke(new object[] { this, new StreamEventArgs(frame) });
but no luck
Does anyone know how I could fix this error or at least catch the error somehow and make the thread put the images on the panel once again?
This is because Gdi+ Image class is not thread safe. Hovewer you can avoid InvalidOperationException by using lock every time when you need to Image access, for example for painting or getting image size:
Image DummyImage;
// Paint
lock (DummyImage)
e.Graphics.DrawImage(DummyImage, 10, 10);
// Access Image properties
Size ImageSize;
lock (DummyImage)
ImageSize = DummyImage.Size;
BTW, invocation is not needed, if you will use the above pattern.
I had a similar problem with the same error message but try as I might, locking the bitmap didn't fix anything for me. Then I realized I was drawing a shape using a static brush. Sure enough, it was the brush that was causing the thread contention.
var location = new Rectangle(100, 100, 500, 500);
var brush = MyClass.RED_BRUSH;
lock(brush)
e.Graphics.FillRectangle(brush, location);
This worked for my case and lesson learned: Check all the reference types being used at the point where thread contention is occurring.
Seems to me, that the same Camera object is used several times.
E.g. try to use a new buffer for each received frame. It seems to me, that while the picture box is drawing the new frame, your capture library fills that buffer again. Therefore on faster machines this might not be an issue, with slower machines it might be an issue.
I've programmed something similar once, after each received frame, we had to request to receive the next frame and set the NEW frame receive buffer in that request.
If you can not do that, copy the received frame from the camera first to a new buffer and append that buffer to a queue, or just use 2 alternating buffers and check for overruns. Either use myOutPutPanel.BeginInvoke to call the camera_ReceivedFrame method, or better have a thread running, which checks the queue, when it has a new entry it calls mnyOutPutPanel.BeginInvoke to invoke your method to set the new buffer as image on the panel.
Furthermore, once you received the buffer, use the Panel Invoke Method to invoke the setting of the image (guarantee that it runs in the window thread and not the thread from your capture library).
The example below can be called from any thread (capture library or other separate thread):
void camera_ReceivedFrame(object sender, StreamEventArgs e)
{
if(myOutputPanel.InvokeRequired)
{
myOutPutPanel.BeginInvoke(
new Camera.ReceivedFrameEventHandler(camera_ReceivedFrame),
sender,
e);
}
else
{
myOutPutPanel.Image = e.Image;
}
}
I think this is multithreading problem
Use windows golden rule and update the panel in the main thread use panel.Invoke
This should overcome cross threading exception
I'm working on implementing directshow capability in a C# dll. I'm basing my work off of the C++ based "CameraCapture" example Microsoft provides with the Windows Mobile 6 sdk.
Things were going well (thanks to earlier help on this site), but I've run into a bit of a snag while trying to listen for directshow events in C# land:
I have a thread that loops to listen for dshow events. It waits based on a manual reset event that gets defined here (this is defined at the end of graph initialization: the graph is built, renderstream is called, and controlstream is already blocking dataflow):
DshowRequestMan = new ManualResetEvent(false);
MediaEvent = (IMediaEvent)FilterGraph;
chk(MediaEvent.GetEventHandle(out DshowEventHandle));
DshowRequestMan.Handle = DshowEventHandle; //note: no "SafeHandle" in cf
There are 2 related problems I'm experiencing with this:
When my dshow event handler loop pulls the event via IMediaEvent.GetEvent() using a timeout of 0, I get a "timeout exceeded" hresult (-2147467260) on the third iteration. That third event trigger (and subsequent error) don't occur in the C++ example.
If I ignore the timeout case mentioned above, it will constantly trigger with a 73 event. This kills the processor since the loop basically never pauses.
When the C++ example runs its graph in preview mode, it gets two IMediaEvents: first 13, then 73. After that, it stops triggering until actual capturing is started. My C# version pulls 13, then 73, then 73 again, but with the timeout error.
In short, it seems like that third triggering of the DshowRequestMan shouldn't be happening because there is no dshowevent to actually "get", hence, the timeout.
Not sure what I'm doing wrong- I'm calling "FreeEventParams()" with each iteration... I'm suspecting the ManualResetEvent object is being used incorrectly since I'm just assigning something to its handle property, but the same thing happens when I use a Pinvoked "WaitForSingleObject" to listen to the DshowEventHandle... I'm confused at this point.
Any ideas would be greatly appreciated. Thanks in advance!
Here is my code for filter graph event handling:
while (!_stopReceivingEvents) {
IntPtr eventHandle;
var hr = _mediaEvent.GetEventHandle(out eventHandle);
DsError.ThrowExceptionForHR(hr);
//NOTE: Do not close the event handle, because it is used internally by the filter graph
using (var safeWaitHandle = new SafeWaitHandle(eventHandle, false)) {
using (var waitHandle = new AutoResetEvent(false) {SafeWaitHandle = safeWaitHandle}) {
if (WaitHandle.WaitAll(new[] {waitHandle}, 500)) {
//receive all available events
do {
EventCode eventCode;
IntPtr param1;
IntPtr param2;
hr = _mediaEvent.GetEvent(out eventCode, out param1, out param2, 500);
_mediaEvent.FreeEventParams(eventCode, param1, param2);
if (hr == 0) {
switch (eventCode) {
//add handling code here
}
}
} while (hr == 0);
}
}
}
}
Are you making sure to Reset you event after you get and handle it?
Whatr does your actual wait code look like? The problem is likely that you're waiting for 0ms, which basically means "check the event and return immediately." If you want it to block on the wait (like a native call with WAIT_INFINITE) you need to pass in System.Threading.Timeout.Infinite (which is actually -1).
If you want the checking thread to periodically yield (which is way safer than an infinite wait and will actually allow your app to shut down), then you need to check the return from the wait, and if it's a timeout (0x80004004) then just go back and wait again:
while(!shutdown)
{
if(DshowRequestMan.WaitOne(1000))
{
// handle the event
}
}