System.ExecutionEngineException only when debugging - c#

I am a C++ developer so I do not have much experience with C#, however, I have to fix this problem anyway.
There is a C# GUI which is using some C++ DLLs. When calling one of the methods from one of the DLLs a System.ExecutionEngineException occurs. Obviously I cant step into the C++ code to see if there is a problem with this code. So I build a small C++ testing application and this one worked fine. The parameters the method is called with in the C# code also look fine to me (used the same values in my C++ testing app). Although there is a problem somewhere in the GUI which may or may not be related to this issue, the same executable started without Visual Studio works just fine.
I have currently no idea on how to find out what causes the problem. Any hints will be highly appreciated.
Call to method in C# code where caseID is of type int, paramID is of type System.Int32 and paramData is a custom struct.
getParameter(caseID, dataID, paramID, ref paramData)
The signature of the method in the C# code looks like this:
[ DllImport ("helper.dll", CallingConvention=CallingConvention.Cdecl/*, CharSet = CharSet.Auto*/ )]
public static extern ERROR getParameter(System.Int32 caseID, System.Int32 dataID, System.Int32 paramID, ref PARAMETER_DATA data);
The signature of the corresponding method in the C++ DLL looks like this:
ERROR _stdcall getParameter(const long caseID, const long dataID, long number, PARAMETER_DATA *data);
What does look odd to me is that the C++ code explicitly states that the method should be called with calling convention "stdcall" whereas the C# code want to use Cdecl. However, this code seems to have worked for quite a while now. I also tried to change both calling conventions to stdcall but that did not work either.

The problem was that a struct member which is a char[] in C++ changed its size. In the C# code there was this attribute
[MarshalAs(System.Runtime.InteropServices.UnmanagedType.ByValTStr, SizeConst=11)]
So I had to change the value of the SizeConst property. Don't know why I didn' see this when I first checked the structs for differences...
I also fixed the calling convention issue and now everything seems to work fine.

Related

AppDomain.GetCurrentThreadId(); made it work, not sure how?

Warning: I started coding in c# 1 month ago with no pre-existent programming knowledge.
I'm using the following method to center the MessageBox inside the parent window:
How to get MessageBox.Show() to pop up in the middle of my WPF application?
I received warning claiming that the AppDomain.GetCurrentThreadId(); method is obsolete, so I found a "solution" here: http://www.pinvoke.net/default.aspx/kernel32/GetCurrentThreadId.html
As the code wouldn't work with uint, I changed it into int and then changed the AppDomain.GetCurrentThreadId(); into GetCurrentThreadId();
[DllImport("kernel32.dll")]
static extern int GetCurrentThreadId();
I've written a fairly complex database program that seems to work flawlessly so far and I understand most of the code behind it, so I would really appreciate if someone could explain to me what exactly I did with the mentioned "solution".
GetCurrentThreadId is a WinApi function, located in kernel32.dll (implemented in C). C# allows you to specify how to call this function via a mechanism called P/Invoke which dynamically finds the function in the specified library by its name, and properly marshals the arguments and return value, if any.
If .NET cannot provide what you need, it is generally fine to resolve to using P/Invoke, but note that this is not a 100% portable solution. On other platforms, the function is likely not to be present, but if you limit yourself to Windows (as you already do, looking at the answer you linked), it is okay.

PInvoke Stack Imbalance passing 2D array from C# to C++ [duplicate]

I am working on a rather large codebase in which C++ functionality is P/Invoked from C#.
There are many calls in our codebase such as...
C++:
extern "C" int __stdcall InvokedFunction(int);
With a corresponding C#:
[DllImport("CPlusPlus.dll", ExactSpelling = true, SetLastError = true, CallingConvention = CallingConvention.Cdecl)]
private static extern int InvokedFunction(IntPtr intArg);
I have scoured the net (insofar as I am capable) for the reasoning as to why this apparent mismatch exists. For example, why is there a Cdecl within the C#, and __stdcall within the C++? Apparently, this results in the stack being cleared twice, but, in both cases, variables are pushed onto the stack in the same reverse order, such that I do not see any errors, albeit the possibility that return information is cleared in the event of attempting a trace during debugging?
From MSDN: http://msdn.microsoft.com/en-us/library/2x8kf7zx%28v=vs.100%29.aspx
// explicit DLLImport needed here to use P/Invoke marshalling
[DllImport("msvcrt.dll", EntryPoint = "printf", CallingConvention = CallingConvention::Cdecl, CharSet = CharSet::Ansi)]
// Implicit DLLImport specifying calling convention
extern "C" int __stdcall MessageBeep(int);
Once again, there is both extern "C" in the C++ code, and CallingConvention.Cdecl in the C#. Why is it not CallingConvention.Stdcall? Or, moreover, why is there __stdcall in the C++?
Thanks in advance!
This comes up repeatedly in SO questions, I'll try to turn this into a (long) reference answer. 32-bit code is saddled with a long history of incompatible calling conventions. Choices on how to make a function call that made sense a long time ago but are mostly a giant pain in the rear end today. 64-bit code has only one calling convention, whomever is going to add another one is going to get sent to small island in the South Atlantic.
I'll try to annotate that history and relevance of them beyond what's in the Wikipedia article. Starting point is that the choices to be made in how to make a function call are the order in which to pass the arguments, where to store the arguments and how to cleanup after the call.
__stdcall found its way into Windows programming through the olden 16-bit pascal calling convention, used in 16-bit Windows and OS/2. It is the convention used by all Windows api functions as well as COM. Since most pinvoke was intended to make OS calls, Stdcall is the default if you don't specify it explicitly in the [DllImport] attribute. Its one and only reason for existence is that it specifies that the callee cleans up. Which produces more compact code, very important back in the days when they had to squeeze a GUI operating system in 640 kilobytes of RAM. Its biggest disadvantage is that it is dangerous. A mismatch between what the caller assumes are the arguments for a function and what the callee implemented causes the stack to get imbalanced. Which in turn can cause extremely hard to diagnose crashes.
__cdecl is the standard calling convention for code written in the C language. Its prime reason for existence is that it supports making function calls with a variable number of arguments. Common in C code with functions like printf() and scanf(). With the side effect that since it is the caller that knows how many arguments were actually passed, it is the caller that cleans up. Forgetting CallingConvention = CallingConvention.Cdecl in the [DllImport] declaration is a very common bug.
__fastcall is a fairly poorly defined calling convention with mutually incompatible choices. It was common in Borland compilers, a company once very influential in compiler technology until they disintegrated. Also the former employer of many Microsoft employees, including Anders Hejlsberg of C# fame. It was invented to make argument passing cheaper by passing some of them through CPU registers instead of the stack. It is not supported in managed code due to the poor standardization.
__thiscall is a calling convention invented for C++ code. Very similar to __cdecl but it also specifies how the hidden this pointer for a class object is passed to instance methods of a class. An extra detail in C++ beyond C. While it looks simple to implement, the .NET pinvoke marshaller does not support it. A major reason that you cannot pinvoke C++ code. The complication is not the calling convention, it is the proper value of the this pointer. Which can get very convoluted due to C++'s support for multiple inheritance. Only a C++ compiler can ever figure out what exactly needs to be passed. And only the exact same C++ compiler that generated the code for the C++ class, different compilers have made different choices on how to implement MI and how to optimize it.
__clrcall is the calling convention for managed code. It is a blend of the other ones, this pointer passing like __thiscall, optimized argument passing like __fastcall, argument order like __cdecl and caller cleanup like __stdcall. The great advantage of managed code is the verifier built into the jitter. Which makes sure that there can never be an incompatibility between caller and callee. Thus allowing the designers to take the advantages of all of these conventions but without the baggage of trouble. An example of how managed code could stay competitive with native code in spite of the overhead of making code safe.
You mention extern "C", understanding the significance of that is important as well to survive interop. Language compilers often decorate the names of exported function with extra characters. Also called "name mangling". It is a pretty crappy trick that never stops causing trouble. And you need to understand it to determine the proper values of the CharSet, EntryPoint and ExactSpelling properties of a [DllImport] attribute. There are many conventions:
Windows api decoration. Windows was originally a non-Unicode operating system, using 8-bit encoding for strings. Windows NT was the first one that became Unicode at its core. That caused a rather major compatibility problem, old code would not have been able to run on new operating systems since it would pass 8-bit encoded strings to winapi functions that expect a utf-16 encoded Unicode string. They solved this by writing two versions of every winapi function. One that takes 8-bit strings, another that takes Unicode strings. And distinguished between the two by gluing the letter A at the end of the name of the legacy version (A = Ansi) and a W at the end of the new version (W = wide). Nothing is added if the function doesn't take a string. The pinvoke marshaller handles this automatically without your help, it will simply try to find all 3 possible versions. You should however always specify CharSet.Auto (or Unicode), the overhead of the legacy function translating the string from Ansi to Unicode is unnecessary and lossy.
The standard decoration for __stdcall functions is _foo#4. Leading underscore and a #n postfix that indicates the combined size of the arguments. This postfix was designed to help solve the nasty stack imbalance problem if the caller and callee don't agree about the number of arguments. Works well, although the error message isn't great, the pinvoke marshaller will tell you that it cannot find the entrypoint. Notable is that Windows, while using __stdcall, does not use this decoration. That was intentional, giving programmers a shot at getting the GetProcAddress() argument right. The pinvoke marshaller also takes care of this automatically, first trying to find the entrypoint with the #n postfix, next trying the one without.
The standard decoration for __cdecl function is _foo. A single leading underscore. The pinvoke marshaller sorts this out automatically. Sadly, the optional #n postfix for __stdcall does not allow it to tell you that your CallingConvention property is wrong, great loss.
C++ compilers use name mangling, producing truly bizarre looking names like "??2#YAPAXI#Z", the exported name for "operator new". This was a necessary evil due to its support for function overloading. And it originally having been designed as a preprocessor that used legacy C language tooling to get the program built. Which made it necessary to distinguish between, say, a void foo(char) and a void foo(int) overload by giving them different names. This is where the extern "C" syntax comes into play, it tells the C++ compiler to not apply the name mangling to the function name. Most programmer that write interop code intentionally use it to make the declaration in the other language easier to write. Which is actually a mistake, the decoration is very useful to catch mismatches. You'd use the linker's .map file or the Dumpbin.exe /exports utility to see the decorated names. The undname.exe SDK utility is very handy to convert a mangled name back to its original C++ declaration.
So this should clear up the properties. You use EntryPoint to give the exact name of the exported function, one that might not be a good match for what you want to call it in your own code, especially for C++ mangled names. And you use ExactSpelling to tell the pinvoke marshaller to not try to find the alternative names because you already gave the correct name.
I'll nurse my writing cramp for a while now. The answer to your question title should be clear, Stdcall is the default but is a mismatch for code written in C or C++. And your [DllImport] declaration is not compatible. This should produce a warning in the debugger from the PInvokeStackImbalance Managed Debugger Assistant, a debugger extension that was designed to detect bad declarations. And can rather randomly crash your code, particularly in the Release build. Make sure you didn't turn the MDA off.
cdecl and stdcall are both valid and usable between C++ and .NET, but they should consistent between the two unmanaged and managed worlds. So your C# declaration for InvokedFunction is invalid. Should be stdcall. The MSDN sample just gives two different examples, one with stdcall (MessageBeep), and one with cdecl (printf). They are unrelated.

Calling DLL in VB.Net using Pinvoke is giving PInvokeStackImbalance Exception (UCHAR*) [duplicate]

I am working on a rather large codebase in which C++ functionality is P/Invoked from C#.
There are many calls in our codebase such as...
C++:
extern "C" int __stdcall InvokedFunction(int);
With a corresponding C#:
[DllImport("CPlusPlus.dll", ExactSpelling = true, SetLastError = true, CallingConvention = CallingConvention.Cdecl)]
private static extern int InvokedFunction(IntPtr intArg);
I have scoured the net (insofar as I am capable) for the reasoning as to why this apparent mismatch exists. For example, why is there a Cdecl within the C#, and __stdcall within the C++? Apparently, this results in the stack being cleared twice, but, in both cases, variables are pushed onto the stack in the same reverse order, such that I do not see any errors, albeit the possibility that return information is cleared in the event of attempting a trace during debugging?
From MSDN: http://msdn.microsoft.com/en-us/library/2x8kf7zx%28v=vs.100%29.aspx
// explicit DLLImport needed here to use P/Invoke marshalling
[DllImport("msvcrt.dll", EntryPoint = "printf", CallingConvention = CallingConvention::Cdecl, CharSet = CharSet::Ansi)]
// Implicit DLLImport specifying calling convention
extern "C" int __stdcall MessageBeep(int);
Once again, there is both extern "C" in the C++ code, and CallingConvention.Cdecl in the C#. Why is it not CallingConvention.Stdcall? Or, moreover, why is there __stdcall in the C++?
Thanks in advance!
This comes up repeatedly in SO questions, I'll try to turn this into a (long) reference answer. 32-bit code is saddled with a long history of incompatible calling conventions. Choices on how to make a function call that made sense a long time ago but are mostly a giant pain in the rear end today. 64-bit code has only one calling convention, whomever is going to add another one is going to get sent to small island in the South Atlantic.
I'll try to annotate that history and relevance of them beyond what's in the Wikipedia article. Starting point is that the choices to be made in how to make a function call are the order in which to pass the arguments, where to store the arguments and how to cleanup after the call.
__stdcall found its way into Windows programming through the olden 16-bit pascal calling convention, used in 16-bit Windows and OS/2. It is the convention used by all Windows api functions as well as COM. Since most pinvoke was intended to make OS calls, Stdcall is the default if you don't specify it explicitly in the [DllImport] attribute. Its one and only reason for existence is that it specifies that the callee cleans up. Which produces more compact code, very important back in the days when they had to squeeze a GUI operating system in 640 kilobytes of RAM. Its biggest disadvantage is that it is dangerous. A mismatch between what the caller assumes are the arguments for a function and what the callee implemented causes the stack to get imbalanced. Which in turn can cause extremely hard to diagnose crashes.
__cdecl is the standard calling convention for code written in the C language. Its prime reason for existence is that it supports making function calls with a variable number of arguments. Common in C code with functions like printf() and scanf(). With the side effect that since it is the caller that knows how many arguments were actually passed, it is the caller that cleans up. Forgetting CallingConvention = CallingConvention.Cdecl in the [DllImport] declaration is a very common bug.
__fastcall is a fairly poorly defined calling convention with mutually incompatible choices. It was common in Borland compilers, a company once very influential in compiler technology until they disintegrated. Also the former employer of many Microsoft employees, including Anders Hejlsberg of C# fame. It was invented to make argument passing cheaper by passing some of them through CPU registers instead of the stack. It is not supported in managed code due to the poor standardization.
__thiscall is a calling convention invented for C++ code. Very similar to __cdecl but it also specifies how the hidden this pointer for a class object is passed to instance methods of a class. An extra detail in C++ beyond C. While it looks simple to implement, the .NET pinvoke marshaller does not support it. A major reason that you cannot pinvoke C++ code. The complication is not the calling convention, it is the proper value of the this pointer. Which can get very convoluted due to C++'s support for multiple inheritance. Only a C++ compiler can ever figure out what exactly needs to be passed. And only the exact same C++ compiler that generated the code for the C++ class, different compilers have made different choices on how to implement MI and how to optimize it.
__clrcall is the calling convention for managed code. It is a blend of the other ones, this pointer passing like __thiscall, optimized argument passing like __fastcall, argument order like __cdecl and caller cleanup like __stdcall. The great advantage of managed code is the verifier built into the jitter. Which makes sure that there can never be an incompatibility between caller and callee. Thus allowing the designers to take the advantages of all of these conventions but without the baggage of trouble. An example of how managed code could stay competitive with native code in spite of the overhead of making code safe.
You mention extern "C", understanding the significance of that is important as well to survive interop. Language compilers often decorate the names of exported function with extra characters. Also called "name mangling". It is a pretty crappy trick that never stops causing trouble. And you need to understand it to determine the proper values of the CharSet, EntryPoint and ExactSpelling properties of a [DllImport] attribute. There are many conventions:
Windows api decoration. Windows was originally a non-Unicode operating system, using 8-bit encoding for strings. Windows NT was the first one that became Unicode at its core. That caused a rather major compatibility problem, old code would not have been able to run on new operating systems since it would pass 8-bit encoded strings to winapi functions that expect a utf-16 encoded Unicode string. They solved this by writing two versions of every winapi function. One that takes 8-bit strings, another that takes Unicode strings. And distinguished between the two by gluing the letter A at the end of the name of the legacy version (A = Ansi) and a W at the end of the new version (W = wide). Nothing is added if the function doesn't take a string. The pinvoke marshaller handles this automatically without your help, it will simply try to find all 3 possible versions. You should however always specify CharSet.Auto (or Unicode), the overhead of the legacy function translating the string from Ansi to Unicode is unnecessary and lossy.
The standard decoration for __stdcall functions is _foo#4. Leading underscore and a #n postfix that indicates the combined size of the arguments. This postfix was designed to help solve the nasty stack imbalance problem if the caller and callee don't agree about the number of arguments. Works well, although the error message isn't great, the pinvoke marshaller will tell you that it cannot find the entrypoint. Notable is that Windows, while using __stdcall, does not use this decoration. That was intentional, giving programmers a shot at getting the GetProcAddress() argument right. The pinvoke marshaller also takes care of this automatically, first trying to find the entrypoint with the #n postfix, next trying the one without.
The standard decoration for __cdecl function is _foo. A single leading underscore. The pinvoke marshaller sorts this out automatically. Sadly, the optional #n postfix for __stdcall does not allow it to tell you that your CallingConvention property is wrong, great loss.
C++ compilers use name mangling, producing truly bizarre looking names like "??2#YAPAXI#Z", the exported name for "operator new". This was a necessary evil due to its support for function overloading. And it originally having been designed as a preprocessor that used legacy C language tooling to get the program built. Which made it necessary to distinguish between, say, a void foo(char) and a void foo(int) overload by giving them different names. This is where the extern "C" syntax comes into play, it tells the C++ compiler to not apply the name mangling to the function name. Most programmer that write interop code intentionally use it to make the declaration in the other language easier to write. Which is actually a mistake, the decoration is very useful to catch mismatches. You'd use the linker's .map file or the Dumpbin.exe /exports utility to see the decorated names. The undname.exe SDK utility is very handy to convert a mangled name back to its original C++ declaration.
So this should clear up the properties. You use EntryPoint to give the exact name of the exported function, one that might not be a good match for what you want to call it in your own code, especially for C++ mangled names. And you use ExactSpelling to tell the pinvoke marshaller to not try to find the alternative names because you already gave the correct name.
I'll nurse my writing cramp for a while now. The answer to your question title should be clear, Stdcall is the default but is a mismatch for code written in C or C++. And your [DllImport] declaration is not compatible. This should produce a warning in the debugger from the PInvokeStackImbalance Managed Debugger Assistant, a debugger extension that was designed to detect bad declarations. And can rather randomly crash your code, particularly in the Release build. Make sure you didn't turn the MDA off.
cdecl and stdcall are both valid and usable between C++ and .NET, but they should consistent between the two unmanaged and managed worlds. So your C# declaration for InvokedFunction is invalid. Should be stdcall. The MSDN sample just gives two different examples, one with stdcall (MessageBeep), and one with cdecl (printf). They are unrelated.

Pinvoke issue in visual C# with C++ code

I'm trying to code the "One-minute" example of reconstructMe SDK, however their SDK is in c++ I think, I succesfully created a console application as directed on their page (here), but I want to create a simple UI, however I'm not very familiar with Visual c++ (I do know something about c++), so I tried their example for C# (I'm pretty much familiar with their UI design) but I get this error when using the C# code provided (here):
A call to PInvoke function 'Lala!Lala.Reme::reme_context_compile' has
unbalanced the stack. This is likely because the managed PInvoke
signature does not match the unmanaged target signature. Check that
the calling convention and parameters of the PInvoke signature match
the target unmanaged signature.
I think is an issue of configuring the project or something like that.
The reconstructMe DLL is using the C calling convention (Cdecl) and not the Windows calling convention (StdCall). They differ in how the stack is handled, which is why having an incorrect calling convention results in a stack imbalance.
The default for p/Invoke is StdCall, you need to manually set the CallingConvension parameter to Cdecl on the DllImport attribute:
[DllImport("example.dll", CallingConvention=CallingConvention.Cdecl)]
public static extern int function(int param);

Writing C++ intended to be called from C#?

So I am doing this as a learning moment and I'm not afraid to say I have no idea what I'm doing here. It might also be worth mentioning that I don't know much about C++ in this scenario.
In C#, I've used DllImport plenty of times to bring in stuff from user32.dll or other DLLs that I haven't written, but I'm looking to better understand how the other half (the C++ half) is implemented to make this happen.
The C++ code I have is simple and just to verify that the call went through successfully:
#include <iostream>
using namespace std;
__declspec(dllexport) void HelloWorld() {
cout << "Hello, World" << endl;
}
I don't know what the importance of __declspec(dllexport) is, but I've seen it on a couple websites that didn't touch much on its importance.
My C# isn't very different than previous DllImports I've done before:
[DllImport("TestDLL.dll")]
static extern void HelloWorld();
static void Main(string[] args) {
HelloWorld();
}
I'm compiled the C++ DLL and put it in the C# project and it's copied to the bin folder. When I run the C# project I get an EntryPointNotFoundException at the call to HelloWorld() inside the main function.
My guess is that I need to either change the C++ code or the compilation flags of the C++ project. Currently "Use of MFC" is set to "Use Standard Windows Libraries" and there's no use of ATL or CLR. Any help would be greatly appreciated.
C++ is a language that supports overloading. In other words, you can have more than one version of HelloWorld(). You could also export HelloWorld(int), a distinct version. It is also a language that requires a linker. In order to not confuzzle the linker about the same name for different functions, the compiler decorates the name of the function. Aka "name mangling".
The tool you want to use to troubleshoot problems like this is Dumpbin.exe. Run it from the Visual Studio Command Prompt on your DLL with the /exports option. You'll see this:
ordinal hint RVA name
1 0 000110EB ?HelloWorld##YAXXZ = #ILT+230(?HelloWorld##YAXXZ)
Bunch of gobbledegook, the exported name is shown in parentheses. Note the ? in the front and ##YAXXZ after the name, that's why the CLR cannot find the exported function. A function that takes an int argument will be exported as ?HelloWorld##YAXH#Z (try it).
The [DllImport] directive supports this, you can use EntryPoint property to give the exported name. Or you can tell the C++ compiler that it should generate code that a C compiler can use. Put extern "C" in front of the declaration and the C++ compiler will suppress the name decoration. And won't support function overloads anymore of course. Dumpbin.exe now shows this:
ordinal hint RVA name
1 0 00011005 HelloWorld = #ILT+0(_HelloWorld)
Note that the name is still not plain "HelloWorld", there's an underscore in front of the name. That's a decoration that helps catch mistakes with the calling convention. In 32-bit code there are 5 distinct ways to call a function. Three of which are common with DLLs, __cdecl, __stdcall and __thiscall. The C++ compiler defaults to __cdecl for regular free functions.
This is also a property of the [DllImport] attribute, the CallingConvention property. The default that's used if it isn't specified is CallingConvention.StdCall. Which matches the calling convention for many DLLs, particularly the Windows ones, but doesn't match the C++ compiler's default so you still have a problem. Simply use the property or declare your C++ function like this:
extern "C" __declspec(dllexport)
void __stdcall HelloWorld() {
// etc..
}
And the Dumpbin.exe output now looks like:
ordinal hint RVA name
1 0 000110B9 _HelloWorld#0 = #ILT+180(_HelloWorld#0)
Note the added #0, it describes the size of the stack activation frame. In other words, how many bytes worth of arguments are passed. This helps catch a declaration mistake at link time, such mistakes are extremely difficult to diagnose at runtime.
You can now use the [DllImport] attribute as you originally had it, the pinvoke marshaller is smart enough to sort out the decoration of the actual function. You can help it with the ExactSpelling and EntryPoint properties, it will be slightly quicker but nothing you'd ever notice.
First question last: __declspec(dllexport) is just a hint to the compiler that you intend to export the function from a DLL. It will generate a wee bit of extra code that can help making the exported function call faster (nothing the CLR uses). And passes an instruction to the linker that the function needs to be exported. Exporting functions can also be done with a .def file but that's doing it the hard way.
This is probably the best way to do it: How to import and use a unmanaged C++ class from C#?
I would recommend that you create a C++/CLI project which statically links with your pure C++. The C++/CLI project will generate the DLL and you will use it just like any other DLL in C#. Again, see the link above.
There ae basically two things that are going to influence name mangling which is why you are having trouble importing the function which are if extern "C" is around your function definition and the calling convention of your function.
extern "C" with a cdecl calling convention will give you a clean name that is easy to import, but you will need to add the calling convention to the DllImportAttribute.

Categories