We Keep Coding

Dealing with a 3rd party library who's function call causes heap corruptions when P/Invoking

I have created a library wrapper which invokes FlexNet Publisher. It is written in C. I am trying to P/Invoke it:
[DllImport("lmgr11.dll")]
public static extern void free_job();
Its implementation is fairly simple:
void WINAPI free_job()
{
if (jobPtr != NULL)
lc_free_job(jobPtr);
jobPtr = NULL;
}
The documentation says that lc_free_job should free a job as well as all resources. Calling lc_free_job works just fine from native code (I made an ATL object expose a wrapper to it through a COM object and I can consume this method all day long from a Visual C++ console application, so I know it must work).
However, from C#, when I try to P/Invoke this method, I get the following error which crashes my application:
Unhandled exception at 0x00007FFA39358283 (ntdll.dll) in
CerberusTestHarness.exe: 0xC0000374: A heap has been corrupted
(parameters: 0x00007FFA393AF6B0).
Why is this? Can I catch it or circumvent it in any way? I can't seem to catch the exception if I wrap the call in a try-catch because its not throwing a .NET exception. Note that I don't have the source code for lc_free_job so I can't inspect it or view its source code, unfortunately. Its written in C.

The issue was due to my C# P/Invoke code from a previous invocation. Hopefully this helps anyone else who comes across the same issue.
I had defined this P/Invoke to get an error string back from a function which returns char *:
[DllImport("lmgr11.dll")]
public static extern string errstring();
I was calling it every time something failed in my 3rd party library as above. This is not the correct way to P/Invoke a method which returns char *. Otherwise, when freeing the error from the native side, it will cause a heap corruption due to the way this string was marshaled.
It needed to be defined like this:
[DllImport("lmgr11.dll")]
public static extern IntPtr errstring();
And called as follows:
var errorMessage = Marshal.PtrToStringAnsi(errstring());

Exceptions on unmanaged threads in .NET

How do I handle situations when the my app is terminating, using a callback prior to termination?
The .NET handlers do not work in the following scenario, is SetUnhandledExceptionHandler the correct choice? It appears to have the shortcomings discussed in the following.
Scenario
I want to respond to all cases of app termination with a message and error report to our service in our .net app.
However, I have a WPF app in which two of our testers get unhandled exceptions that bypass:
AppDomain.UnhandledException (most importantly)
Application.ThreadException
Dispatcher.UnhandledException
They are marked SecuirtyCritical and HandleProcessCorruptedStateExceptions.
legacyCorruptedStateExceptionsPolicy is set to true in the app.config
My two examples in the wild
VirtualBox running widows10 throws inside some vboxd3d.dll when initialising WPF somewhere (turning off vbox 3d accel "fixes it")
Win8 machine with suspicious option to "run on graphics card A/B" in system context menu, crashes somewhere (:/) during WPF startup but only when anti-cracking tools are applied.
Either way, when live, the app must to respond to these kinds of failures prior to termination.
I can reproduce this with an unmanaged exception, that occurs in an unmanaged thread of a PInvoked method in .net:
test.dll
BOOL APIENTRY DllMain( HMODULE hModule,
DWORD ul_reason_for_call,
LPVOID lpReserved
)
{
switch (ul_reason_for_call)
{
case DLL_PROCESS_ATTACH:
case DLL_THREAD_ATTACH:
case DLL_THREAD_DETACH:
case DLL_PROCESS_DETACH:
break;
}
return TRUE;
}
DWORD WINAPI myThread(LPVOID lpParameter)
{
long testfail = *(long*)(-9022);
return 1;
}
extern "C" __declspec(dllexport) void test()
{
DWORD tid;
HANDLE myHandle = CreateThread(0, 0, myThread, NULL, 0, &tid);
WaitForSingleObject(myHandle, INFINITE);
}
app.exe
class TestApp
{
[DllImport("kernel32.dll")]
static extern FilterDelegate SetUnhandledExceptionFilter(FilterDelegate lpTopLevelExceptionFilter);
[UnmanagedFunctionPointer(CallingConvention.StdCall)]
delegate int FilterDelegate(IntPtr exception_pointers);
static int Win32Handler(IntPtr nope)
{
MessageBox.Show("Native uncaught SEH exception"); // show + report or whatever
Environment.Exit(-1); // exit and avoid WER etc
return 1; // thats EXCEPTION_EXECUTE_HANDLER, although this wont be called due to the previous line
}
[DllImport("test.dll")]
static extern void test();
[STAThread]
public static void Main(string[] args)
{
AppDomain.CurrentDomain.UnhandledException += new UnhandledExceptionEventHandler(CurrentDomain_UnhandledException);
SetUnhandledExceptionFilter(Win32Handler);
test(); // This is caught by Win32Handler, not CurrentDomain_UnhandledException
}
[SecurityCritical, HandleProcessCorruptedStateExceptions ]
static void CurrentDomain_UnhandledException(object sender, UnhandledExceptionEventArgs e)
{
Exception ex = e.ExceptionObject as Exception;
MessageBox.Show(ex.ToString()); // show + report or whatever
Environment.Exit(-1); // exit and avoid WER etc
}
}
This handles the failure in the vboxd3d.dll in a bare WPF test app, which of course also has the WCF Dispatcher and WinForms Application (why not) exception handlers registered.
Updates
In the production code I am trying to use this on, the handler appears to get overwritten by some other caller, I can get around that by calling the method every 100ms which is stupid of course.
On the machine with the vbox3d.dll problem, doing the above replaces the exception with one in clr.dll.
It appears at the time of crash, the managed function pointer passed into kernel32 is no longer valid. Setting the handler with a native helper dll, which calls a native function inside appears to be working. The managed function is a static method - I'm not sure pinning applies here, perhaps the clr is in the process of terminating...
Indeed the managed delegate was being collected. No "overwriting" of the handler was occuring. I've added as an answer..not sure what to accept or what the SO convention is here...

The problem with the code in the question was this:
SetUnhandledExceptionFilter(Win32Handler);
Which since a delegate is automatically created, is eqivilant to:
FilterDelegate del = new FilterDelegate(Win32Handler);
SetUnhandledExceptionFilter(del);
Problem being, that the GC can collect it, and the native->managed thunk that is created, at any point after it's final reference. So:
SetUnhandledExceptionFilter(Win32Handler);
GC.Collect();
native_crash_on_unmanaged_thread();
Will always cause a nasty crash where the handler passed into kernel32.dll is no longer a valid function pointer. This is remedied by not allowing the GC to collect:
public class Program
{
static FilterDelegate mdel;
public static void Main(string[] args)
{
FilterDelegate del = new FilterDelegate(Win32Handler);
SetUnhandledExceptionFilter(del);
GC.KeepAlive(del); // do not collect "del" in this scope (main)
// You could also use mdel, which I dont believe is collected either
GC.Collect();
native_crash_on_unmanaged_thread();
}
}
The other answers are also a great resource; not sure what to mark as the answer right now.

I've had to deal with, shall we say, unpredictable unmanaged libraries.
If you're P/Invoking into the unmanaged code, you may have problems there. I've found it easier to use C++/CLI wrappers around the unmanaged code and in some cases, I've written another set of unmanaged C++ wrappers around the library before getting to the C++/CLI.
You might be thinking, "why on earth would you write two sets of wrappers?"
The first is that if you isolate the unmanaged code, it makes it easier to trap exceptions and make them more palatable.
The second is purely pragmatic - if you have a library (not a dll) which uses stl, you will find that the link will magically give all code, managed and unmanaged, CLI implementation of the stl functions. The easiest way to prevent that is to completely isolate the code that uses stl, which means that everytime you access a data structure through stl in unmanaged code you end up doing multiple transitions between managed and unmanaged code and your performance will tank. You might think to yourself, "I'm a scrupulous programmer - I'll be super careful to put #pragma managed and/or #pragma unmanaged wrappers in the right places and I'm all set." Nope, nope, and nope. Not only is this difficult and unreliable, when (not if) you fail to do it properly, you won't have a good way to detect it.
And as always, you should ensure that whatever wrappers you write are chunky rather than chatty.
Here is a typical chunk of unmanaged code to deal with an unstable library:
try {
// a bunch of set up code that you don't need to
// see reduced to this:
SomeImageType *outImage = GetImage();
// I was having problems with the heap getting mangled
// so heapcheck() is conditional macro that calls [_heapchk()][1]
heapcheck();
return outImage;
}
catch (std::bad_alloc &) {
throw MyLib::MyLibNoMemory();
}
catch (MyLib::MyLibFailure &err)
{
throw err;
}
catch (const char* msg)
{
// seriously, some code throws a string.
throw msg;
}
catch (...) {
throw MyLib::MyLibFailure(MyKib::MyFailureReason::kUnknown2);
}

An exception that can't be handled properly can always happen, and the process may die unexpectedly no matter how hard you try to protect it from within. However, you can monitor it from the outside.
Have another process that monitors your main process. If the main process suddenly disappears without logging an error or reporting things gracefully, the second process can do that. The second process can be a lot simpler, with no unmanaged calls at all, so chances of it disappearing all of a sudden are significantly smaller.
And as a last resort, when your processes start check if they've shut down properly. If not, you can report a bad shutdown then. This will be useful if the entire machine dies.

DirectShow ISampleGrabber.SetCallback throws InvalidCastException after showing a C# MessageBox

I am using DirectShow's SampleGrabber to capture an image from a webcam video. Codes are writing in C# and use .NET wrappers of DirectShow's interfaces to do COM communication. The following BufferCB, first, copies image data to a local array variable, disables SampleGrabber callback, processes a received image data, and uses MessageBox to shows a result.
public int BufferCB(double sampleTime, IntPtr pBuffer, int bufferLen)
{
if (Monitor.TryEnter(lockSync))
{
try
{
if ((pBuffer != IntPtr.Zero))
{
this.ImageData = new byte[bufferLen];
Marshal.Copy(pBuffer, this.ImageData, 0, bufferLen);
// Disable callback
sampleGrabber.SetCallback(null, 1);
// Process image data
var result = this.Decode(new Bitmap(this.ImageData));
if (result != null)
{
MessageBox.Show(result.ToString());
}
// Enable callback
sampleGrabber.SetCallback(this,1);
}
}
finally
{
Monitor.Exit(this.parent.lockSync);
}
}
return 0;
}
Now, if the result = null, hence the MessageBox.Show never runs, both clamping calls sampleGrabber.SetCallback() will run without any issue. Once the result != null, and the MessageBox shows up, the call sampleGrabber.SetCallback(this,1) will throw the InvalidCastException as below:
Unable to cast COM object of type 'System.__ComObject' to interface type 'DirectShowInterfaces.ISampleGrabber'. This operation failed because the QueryInterface call on the COM component for the interface with IID '{6B652FFF-11FE-4FCE-92AD-0266B5D7C78F}' failed due to the following error: No such interface supported (Exception from HRESULT: 0x80004002 (E_NOINTERFACE)).
If I stop in a VS debugger and add a watch of ((ISampleGrabber)sampleGrabber).SetCallback(sampleGrabber, 1), I will get the ArgumentException with the message of "Cannot find the method on the object instance." instead.
May someone experience the same issue can give me some advise. Thank you.

BufferCB and SampleCB calls take place on worker threads, which typically belong to MTA. Your graph initialization on the other hand typically takes place on STA thread. DirectShow API and filters volantarily ignore COM threading rules while .NET enforces thread checks an raises exception on attempts to use COM interface pointer on a wrong thread. You are hitting exactly this issue.
You don't need to reset callback and then set it back. Use SampleCB callback instead and it happens as a blocking call. Before you complete the processing the rest of the streaming is on hold.

Fatal execution error when a file is opened, closed and disposed multiple times in C#

In a thread, a file is being opened, closed and disposed continuosly. Does this cause an issue ?
Here is the code
StreamWriter file1 = new StreamWriter(filepath4, true);
for (int i = 0; i < ChannelValueForTcp; i++)
{
file1.WriteLine(data[i]);
}
file1.WriteLine(data[data.Length-1]);
file1.WriteLine(data[data.Length - 2]);
file1.Close();
file1.Dispose();
Please help I am stuck. (This comes up randomly we are trying to run the code for 8 hours continuously.)
Edit:
No no other thread works or does anything associated with this file. It is being used only here.
There are other threads running, which are giving the same error but randomly after 45 minutes - 5 hours of testing.
Here is the c code. Please download it
[DllImport("ConsoleApplication2.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern int main_c();
public string[] tcp(string peer, int port)
{
int i = main_c();//the c code writes to a file called akash.txt and returns = 0 if it is successful. Then I read the file and do some functions on it.
if (i == 0)
{
StreamReader objReader = new StreamReader("akash.txt");

Random crashes and FatalExecutionEngineError exceptions are normally associated with stack or heap corruption which can remain hidden until further down in your code. Ensure that you have marshalled all your C++ functions correctly using the right calling convention, parameter types and return types.
Microsoft specifies the probable cause of the message is:
The CLR has been fatally corrupted. This is most often caused by data corruption, which can be caused by a number of problems, such as calls to malformed platform invoke functions and passing invalid data to the CLR.
Judging from the code you've supplied, your declaration looks correct so it could be another function that you've marshalled that is causing an issue.
Ensure that your C++ code is stable and not the cause of the problem. I think that it might be associated with the deletion or filling of the 'res' buffer.
You could be compiling the DLL with a flag which sets the calling convention to something apart from __cdecl. You can verify this by right-clicking on the project > Properties > C/C++ > Advanced > Calling Convention.

How to raise Exception from C# .NET COM

How to raise Exception from a C# .NET COM. In fact I'm inside a Win32 application build using Borland Delphi 7.0 and I'm try to consume a function in .NET. Everything is working fine except the fact raising exception in C# are not re-routed to Delphi, I guess I miss some kind of method decorator. Here a piece of code:
void iLinkV2.FindLinkedClientEng(string SourceApp, string SourceClientID,
ref object ClientID, ref object Engagement)
{
throw new COMException("errClientNotFound",
(int)ExceptionsType.errClientNotFound);
ClientID = SourceClientID;
}
[ComVisible(true),
Guid("D731C821-EEA2-446E-BC5A-441DCA0477F5"),
InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
public interface iLinkV2
{ ...
}

You don't need to. Exceptions are not part of COM Interop specification and should not cross module boundaries. You can use status codes to indicate any errors.
Catch the exception just before method exits (in each COM interface method). Turn the result into an error code. If you use an HRESULT then you can consume it as safecall and get the Delphi compiler to re-raise an exception on the other side of the fence.