We Keep Coding

PInvoking AddSecurityPackageA

I am working on a project where I need to PInvoke the secur32!AddSecurityPackageA function, but I am still learning the ins and outs of how to do this by hand and could use some help.
Here are a the references I am working with:
https://learn.microsoft.com/en-us/windows/desktop/api/sspi/nf-sspi-addsecuritypackagea
https://learn.microsoft.com/en-us/windows/desktop/api/sspi/ns-sspi-_security_package_options
And here's a sample of my code where I am trying to define the struct and call the function:
[DllImport("secur32.dll", EntryPoint = "AddSecurityPackageA")]
public static extern void AddSecurityPackageA(
ref string pszPackageName,
ref SECURITY_PACKAGE_OPTIONS[] Options
);
[StructLayout(LayoutKind.Sequential, CharSet =CharSet.Ansi)]
public class SECURITY_PACKAGE_OPTIONS
{
public ulong Size;
public ulong Type;
public ulong Flags;
public ulong SignatureSize;
public IntPtr Signature;
}
string dll = #"c:\temp\test.dll";
SECURITY_PACKAGE_OPTIONS[] pkgOpts = new SECURITY_PACKAGE_OPTIONS();
AddSecurityPackageA(ref dll, ref pkgOpts);
My questions are:
At lines 3 and 4, is this an appropriate use of ref and is this generally correct according to the MSDN docs?
At line 14, the C++ struct on MSDN has this as a void pointer, but while researching I found that the C# equivalent is an IntPtr. Is that correct or do I need to use unsafe?
In general, has anyone found any really good PInvoke tutorials outside of reading other people's code? I'm moving over from Python so it is quite a bit different and much of what I've found is either "draw a circle, draw the rest of the owl" or insanely lengthy MSDN documentation that makes a lot of assumptions.
Thank you!

Some comments:
Use the W function rather than the A function. You don't want to limit yourself to ANSI. This is a Unicode world.
The function has a return value. You must declare the function with a matching return value type. Presumably it is uint or int but you should check in the C++ header file.
ref string is wrong. It should be string.
ref SECURITY_PACKAGE_OPTIONS[] is wrong. It is not an array. It is a pointer to a struct. Since you declared SECURITY_PACKAGE_OPTIONS as a class, a reference type, you can replace ref SECURITY_PACKAGE_OPTIONS[] with SECURITY_PACKAGE_OPTIONS.
C++ unsigned long is 32 bits, so it should be uint in C#.
IntPtr is correct, but that leaves unresolved the question of how to declare the digital signature and obtain a pointer to it. I think it's outside the remit of this question for us to track down an example of how to do that.

This one works for me:
[DllImport("secur32.dll", CharSet = CharSet.Auto, SetLastError = true)]
public static extern uint AddSecurityPackage(
string pszPackageName,
SECURITY_PACKAGE_OPTIONS Options
);
[DllImport("secur32.dll", CharSet = CharSet.Auto, SetLastError = true)]
public static extern uint DeleteSecurityPackage(
string pszPackageName
);

How to PInvoke CMark's cmark_markdown_to_html

I'm trying to PInvoke the following function from GitHub's fork of CMark
char *cmark_markdown_to_html(const char *text, size_t len, int options)
and here's my PInvoke signature:
[DllImport("cmark.dll")]
public static extern IntPtr cmark_markdown_to_html(
[In()] [MarshalAs(UnmanagedType.LPStr)] string text,
[MarshalAs(UnmanagedType.SysUInt)] uint len,
[MarshalAs(UnmanagedType.SysInt)] int options);
I call it as follows:
var retValue = cmark_markdown_to_html(markdown, 0, 0);
However, it throws a Marshaling exception the with the message:
Cannot marshal 'parameter #2':
Invalid managed/unmanaged type combination
(Int32/UInt32 must be paired with I4, U4, or Error).
OK, so I change the signature to:
[DllImport("cmark.dll")]
public static extern IntPtr cmark_markdown_to_html(
[In, MarshalAs(UnmanagedType.LPStr)] string text,
[MarshalAs(UnmanagedType.U4)] uint len,
[MarshalAs(UnmanagedType.I4)] int options);
Now it throws a PInvokeStackImbalance error
The name '$exception' does not exist in the current context
Native stuff is a mystery to me. Can someone help?

Stack imbalance
The reason for stack imbalance is described in Why are Cdecl calls often mismatched in the "standard" P/Invoke Convention?.
Basically you need to specify proper calling convention in the DllImport that will probably be cdecl:
[DllImport("cmark.dll", CallingConvention = CallingConvention.Cdecl)]
Marshalling size_t
That one is discussed in what is equal to the c++ size_t in c# .
Assuming that CMark will always be native (32 bit on x86, 64 bit on x64), you should use (U)IntPtr.
If it is always 32 bit, then Int32 should be used.

PInvokeStackImbalance
Presumably CMark is using the default calling convention. The default calling convention for C code is "Cdecl", while the default used for P/invoke calls is "StdCall". The calling convention specifies how the arguments and return value are placed on the call stack. A mismatch between the used calling convention causes the stack to become imbalanced.
You can specify the calling convention to use from C# code in the DllImport attribute.
[DllImport("cmark.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern IntPtr cmark_markdown_to_html(
[In, MarshalAs(UnmanagedType.LPStr)] string text,
[MarshalAs(UnmanagedType.U4)] uint len,
[MarshalAs(UnmanagedType.I4)] int options);
Marshalling
In case you want your code to be compatible with architectures other than 32-bit, you would want to marshall size_t with UIntPtr. (Read #Eugene Podskal answer)
[DllImport("cmark.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern IntPtr cmark_markdown_to_html(
[In, MarshalAs(UnmanagedType.LPStr)] string text,
[MarshalAs(UnmanagedType.U4)] UIntPtr len,
[MarshalAs(UnmanagedType.I4)] int options);
Then you can call the method like so
var retValue = cmark_markdown_to_html(markdown, UIntPtr.Zero, 0);

C# delegate for C++ callback

I think I have basically understood how to write c# delegates for callbacks, but this one is confusing me.
The c++ definition is as follows:
typedef int (__stdcall* Callback)(
long lCode,
long lParamSize,
void* pParam
);
and my c# approach would be:
unsafe delegate int CallbackDelegate (int lCode, int lParamSize, IntPtr pParam);
Although this seems to be incorrect, because I get a PInvokeStackInbalance error, which means my definition of the delegate is wrong.
The rest of the parameters of the function are strings or ints, which means they cannot cause the error, and if I just pass a IntPtr.Zero instead of the delegate (which would mean I'm pointing to a non-existent callback function) I get an AccessViolation error, which makes sense aswell.
What am I doing wrong?
EDIT:
The full c++ function is:
int
__stdcall
_Initialize (
const char* FileName,
Callback cbFunction,
int Code,
const char* Name,
unsigned int Option,
unsigned int Option2
);
My c# version is:
[DllImport("MyDll.dll", CallingConvention = CallingConvention.StdCall)]
public static extern int _Initialize (string FileName, CallbackDelegate cbFunction, int Code, string Name, uint Options, uint Options2);
The function is (for testing) just called inside of the main routine of a console application:
static void Main(string[] args)
{
CallbackDelegate del = new CallbackDelegate(onCallback);
Console.Write(_Initialize("SomeFile.dat", del, 1000, "", 0, 4));
Console.Read();
}
where onCallback is this:
static int onCallback(int lCode, int lParamSize, IntPtr pParam)
{
return 0;
}
I get the PInvokeStackInbalance error on the line where I call _Initialize, if I pass a IntPtr.Zero instead of the delegate, and change the definition of the function to IntPtr instead of CallbackDelegate then I get a AccessViolationException.

I added your code to my current project where I'm doing a lot of C#/C++ interop in VS2012. And while I hate to use the "it worked on my machine", it worked fine for me. The code as I ran it is listed out below just to illustrate that I didn't make and fundamental changes.
My suggestion is that you build a new native dll with a stub function for _Initialize such as below and see if it works when you can control both sides of the interface. If that works but the real dll doesn't, it comes down to either compiler settings on the native side or their is a bug on the native side and it is stomping on the stack.
extern "C" {
typedef int (__stdcall* Callback)(long lCode,long lParamSize,void* pParam );
TRADITIONALDLL_API int __stdcall _Initialize (const char* FileName,Callback cbFunction, int Code,
const char* Name,unsigned int Option,unsigned int Option2)
{
cbFunction(0, 0, nullptr);
return 0;
}
}
On the C# side, I added the declarations to my interface class:
public delegate int CallbackDelegate(int lCode, int lParamSize, IntPtr pParam);
[DllImport("XXX.dll", CallingConvention = CallingConvention.StdCall)]
public static extern int _Initialize(string FileName, CallbackDelegate cbFunction, int Code, string Name, uint Options, uint Options2);
And then in my main:
private int onCallback(int lCode, int lParamSize, IntPtr pParam)
{
return 0;
}
XXXInterface.CallbackDelegate del = new XXXInterface.CallbackDelegate(onCallback);
Console.Write(XXXInterface._Initialize("SomeFile.dat", del, 1000, "", 0, 4));

A .NET long is 64bits. A C++ long may be just 32bits. Check with the C++ compiler which compiled that definition as to what size it's longs are.

[UnmanagedFunctionPointerAttribute(CallingConvention.StdCall)]
unsafe delegate int CallbackDelegate (int lCode, int lParamSize, IntPtr pParam);
If .NET assumes cdecl instead of stdcall, your stack will most assuredly be hosed.

DllImport, Char*& and StringBuilder C/C#

I have a problem, I tried to look at almost all the poster solutions, unsuccessful to find a suitable one.
The question is easy, Want to have a return string from unmanaged C code in my managed C#.
The c function is:
extern "C" __declspec(dllexport) int process_batch (char *&result);
and in C# I imported the DLL:
[DllImport("mydll.dll")]
public static extern IntPtr process_batch(StringBuilder result);
I run, but the return value in my StringBuilder is a 7-8 character string of non-sense! (I think the memory address)
I tried adding ref before the StringBuilder, this time I get the correct return value in StringBuilder but I get AccessViolationException :
Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
So I need your help to fix this.
Just one more thing, I use malloc in c for allocating the memory to the char* variable.
Thanks,

If you really want to do that, pass the parameter as a ref IntPtr, then call Marshal.PtrToStringAnsi, or similar.
[DllImport("mydll.dll")]
public static extern IntPtr process_batch(ref IntPtr result);
Note that, since you're allocating the string with malloc in C, the .NET program will have to keep track of the returned pointer so that it can pass it back to you to be deallocated. Otherwise you'll have a memory leak.
If you were to pass a ref StringBuilder, you wouldn't be able to deallocate the memory that was allocated by the C program.
Also, as somebody commented in another post, you need to set the calling convention:
[DllImport("mydll.dll", CallingConvention=CallingConvention.Cdecl)]

I've been using the following code successfully:
[DllImport("user32.dll", EntryPoint = "GetClassName", ExactSpelling = false,
CharSet = CharSet.Auto, SetLastError = true)]
private static extern int _GetClassName(IntPtr hwnd, StringBuilder lpClassName,
int nMaxCount);
public static string GetClassName(IntPtr hWnd)
{
StringBuilder title = new StringBuilder(MAXTITLE);
int titleLength = _GetClassName(hWnd, title, title.Capacity + 1);
title.Length = titleLength;
return title.ToString();
}
I'd advise for a more specific declaration of the imported method via the DllImportAttribute. Try the CharSet = CharSet.Auto bit for instance
I know that this is not exactly related to your original problem as this makes use of the Windows API, but maybe it is of help nonetheless.

How to pass PWCHAR to C++ dll from C#

I have an dll written in C++, and I want to call it from C#. The function outputs outputChar and deadChar, the deadChar variable is also read by the C++ function.
I tried to call function from C# in different ways, but all time I got AccessViolationException: "Attempted to read or write protected memory. This is often an indication that other memory is corrupt."
C++ dll:
extern "C" _declspec (dllexport) int convertVirtualKeyToWChar(int virtualKey, PWCHAR outputChar, PWCHAR deadChar);
C# code 1:
[DllImport("keylib.dll")]
static extern int convertVirtualKeyToWChar(int virtualKey,
StringBuilder output,
StringBuilder deadchar);
C# code 2:
static extern int convertVirtualKeyToWChar(int virtualKey,
out char output,
ref char deadchar);

Note: The two PWCHAR arguments to your function convertVirtualKeyToWChar are ambiguous. They could be pointers to a single WCHAR or pointers to aWCHAR string. Given the name of the function and arguments, this answer assumes they are pointers to a single WCHAR.
You want to use the following:
[DllImport("keylib.dll", CharSet=CharSet.Unicode, CallingConvention=CallingConvention.Cdecl)]
static extern int convertVirtualKeyToWChar( int virtualKey,
out char output,
ref char deadchar );
Your crash is caused by two reasons: DllImport defaults to the ANSI character set and StdCall calling convention. Your C++ DLL does not specify any calling convention, so it will default to CDecl.
See DllImportAttribute.CallingConvention and DllImportAttribute.CharSet

This is a stab in the dark, so caveat emptor...
static extern int convertVirtualKeyToWChar(int virtualKey,
char[] output,
char[] deadchar);
Pass a single-element array to each char[] parameter.

Try this (bearing in mind that if an exception is thrown between Alloc and Free you will leak memory, so build some error handling in):
static void Main(string[] args)
{
IntPtr pout = Marshal.AllocHGlobal(2);
IntPtr pdead = Marshal.AllocHGlobal(2);
int ret = convertVirtualKeyToWChar(1, pout, pdead);
char output = (char)Marshal.ReadInt16(pout);
char dead = (char)Marshal.ReadInt16(pdead);
Marshal.FreeHGlobal(pout);
Marshal.FreeHGlobal(pdead);
}
static extern int convertVirtualKeyToWChar(int virtualKey,
IntPtr output,
IntPtr deadchar);