We Keep Coding

Having trouble marshalling an array of structs from c to c#

I'm trying to marshal an array of c structs into C# (Using Unity) but, no matter the method I use, I always get an exception or a crash.
I'm loading dlls (libretro cores) that conform (or should...) to the Libretro API, the c/c++ side is not available to me (more precisely, not allowed to be modified by me), which means I have to handle the data I get back from that dll no matter how it is laid out.
The C API structs are defined as follow (RETRO_NUM_CORE_OPTION_VALUES_MAX is a constant with a value of 128):
struct retro_core_option_value
{
const char *value;
const char *label;
};
struct retro_core_option_definition
{
const char *key;
const char *desc;
const char *info;
struct retro_core_option_value values[RETRO_NUM_CORE_OPTION_VALUES_MAX];
const char *default_value;
};
struct retro_core_options_intl
{
struct retro_core_option_definition *us;
struct retro_core_option_definition *local;
};
My C# mappings look like this at the moment:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct retro_core_option_value
{
public char* value;
public char* label;
}
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct retro_core_option_definition
{
public char* key;
public char* desc;
public char* info;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = RETRO_NUM_CORE_OPTION_VALUES_MAX)]
public retro_core_option_value[] values;
public char* default_value;
}
[StructLayout(LayoutKind.Sequential)]
public struct retro_core_options_intl
{
public IntPtr us;
public IntPtr local;
}
The callback function has the following signature on the C# side:
public unsafe bool Callback(retro_environment cmd, void* data)
retro_environment is an unsigned int converted to an enum, a switch is performed on it and then dictates how to handle the void* data pointer appropriately. Here data is a retro_core_options_intl*.
I'm able to do the void* conversion in 2 ways:
retro_core_options_intl intl = Marshal.PtrToStructure<retro_core_options_intl>((IntPtr)data);
or
retro_core_options_intl* intl = (retro_core_options_intl*)data;
I get a readable address with both approaches (intl.us for the first and intl->us for the second), the "local" part is empty in my particular case but the "us" part is defined as mandatory by the API. intl->us points to an array of retro_core_option_definition of variable length.
The issue I'm having is trying to read the values inside of this mandatory construct.
The array I'm trying to load right now can be seen here: https://github.com/visualboyadvance-m/visualboyadvance-m/blob/master/src/libretro/libretro_core_options.h at line 51.
The API defines a fixed size for the "struct retro_core_option_value values[RETRO_NUM_CORE_OPTION_VALUES_MAX]" struct member, but code that comes in is almost always defined as an array where the last element is "{ NULL, NULL }" to indicate the end, so they don't always (almost never) contain 128 values.
I tried:
retro_core_options_intl intl = Marshal.PtrToStructure<retro_core_options_intl>((IntPtr)data);
retro_core_option_definition us = Marshal.PtrToStructure<retro_core_option_definition>(intl.us);
This gives a NullReferenceException.
retro_core_options_intl intl = Marshal.PtrToStructure<retro_core_options_intl>((IntPtr)data);
retro_core_option_definition[] us = Marshal.PtrToStructure<retro_core_option_definition[]>(intl.us);
This gives a retro_core_option_definition array of 0 length.
retro_core_options_intl intl = Marshal.PtrToStructure<retro_core_options_intl>((IntPtr)data);
retro_core_option_definition us = new retro_core_option_definition();
Marshal.PtrToStructure(intl.us, us);
This gives a "destination is a boxed value".
That's basically where I'm at...
Any help would be much appreciated :)
The entire codebase can be found here: https://github.com/Skurdt/LibretroUnityFE

First thing I see is that you either need to use wchar_t types instead of char types in you C code, or you can use byte instead of char in C#. System.Char in C# is two bytes. char in C code is 1 byte.
You can also use System.String in the C# code and annotate it with a MarshalAs attribute to tell it what type of char data is coming in, such as Ansi or Unicode C strings.

Calling a unmanaged C method with a SAFEARRAY **ppsa argument from managed C# code

I've been struggling on how to migrate this VB6 code into C#. It involves calling a function inside a DLL passing an array of structure, among other things.
So in VB6, the "struct" declaration is like this:
'Define structure for RGETDAT_STR procedure call
Type rgetdat_str_data_str
type As Integer 'data type (set internally)
file As Integer 'file in database
rec As Integer 'record in file
word As Integer 'word offset in record
start_bit As Integer 'UNUSED
length As Integer 'length of string
flags As Integer 'flags
padding1 As Integer 'UNUSED
value As String 'database value
status As Integer 'return status
padding2 As Integer 'UNUSED
End Type
and one function that uses this "struct" has a method declared as this:
Public Declare Function rgetdat_str Lib "hscnetapi.dll" _
Alias "rgetdat_str_vb" _
(ByVal Server As String, ByVal num_points As Integer, _
getdat_str_data() As rgetdat_str_data_str) As Integer
So, I attempted to convert these 2 pieces of code into C#. I had tried so many variations, but I will post here the latest one that I have. The idea is to call the function via P/Invoke.
The C# struct (so far):
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct rgetdat_str_data_str
{
public short type;
public short file;
public short rec;
public short word;
public short start_bit;
public short length;
public short flags;
public short padding1;
[MarshalAs(UnmanagedType.LPStr)]
public string value;
public short status;
public short padding2;
}
and the function import (so far):
[DllImport("hscnetapi.dll", EntryPoint = "rgetdat_str_vb")]
public static extern short rgetdat_str(
[MarshalAs(UnmanagedType.LPTStr)]
string Server,
short num_points,
[In,Out, MarshalAs(UnmanagedType.LPArray)]
ref rgetdat_str_data_str[] getdat_str_data);
Nothing worked so far in my various experiments on marshalling attributes with the parameters.
I managed to find the C header file for this DLL, and the declaration looks like this:
EXTERN_C short __loadds CALLBACK rgetdat_str_vb_ansi
_DECLARE((char *szHostname, short cRequests, SAFEARRAY **ppsa));
and the "struct" in the C world is declared like this:
/* define union used in rgetdat_value in RGETDAT procedure call */
typedef union rgetdat_value_str
{
n_short int2;
n_long int4;
n_float real4;
n_double real8;
n_char *str;
n_ushort bits;
} rgetdat_value;
/* define structure for RGETDAT procedure call */
typedef struct rgetdat_data_str
{
n_ushort type;
n_ushort file;
n_ushort rec;
n_ushort word;
n_ushort start_bit;
n_ushort length;
n_short flags;
rgetdat_value value;
n_short status;
} rgetdat_data;
In my frustration, I tried to open this DLL with the ITypeLib Viewer tool. I was surprised that the DLL file can be opened, even though I cannot add this DLL as a Reference in my project. Anyway, a couple of things that I observed within the viewer.
The function has this signature:
[entry("rgetdat_str_vb"), helpstring("...")]
short _stdcall rGetdat_Str(
[in] LPSTR Server,
[in] short num_points,
[in, out] SAFEARRAY(rGetdat_Str_Data_Str)* getdat_str_data);
and the "struct" looked like this:
typedef struct tagrGetdat_Str_Data_Str {
short type;
short file;
short rec;
short word;
short start_bit;
short length;
short flags;
short padding1;
BSTR value;
short status;
short padding2;
} rGetdat_Str_Data_Str;
Based on these observations, I played around with the marshalling attributes of the C# struct, for example,
1.) Changing the struct's value attribute to [MarshalAs(UnmanagedType.BStr)]
2.) Changing the function's getdat_str_data parameter attribute to MarshalAs(UnmanagedType.SafeArray, SafeArraySubType = VarEnum.VT_RECORD)
and still nothing works.
There's an blog/article talking about a similar topic here: http://limbioliong.wordpress.com/2012/02/28/marshaling-a-safearray-of-managed-structures-by-pinvoke-part-1/ but I can't just wrap my head around it.
It seems that VB6 can do it very simply compared to C# (.Net) with this DLL function call. Any hints or ideas out there on how to DLLImport declare this function in C# (.Net)?

You need to use MarshalAs with UnmanagedType.SafeArray to tell the marshaller that you want the array marshalled as a SAFEARRAY.
[DllImport("hscnetapi.dll", EntryPoint = "rgetdat_str_vb")]
public static extern short rgetdat_str(
[MarshalAs(UnmanagedType.LPStr)]
string Server,
short num_points,
[MarshalAs(UnmanagedType.SafeArray, SafeArraySubType=VT_USERDEFINED)]
ref rgetdat_str_data_str[] getdat_str_data
);
In your C# struct, you handle the BSTR member incorrectly. It should be
[MarshalAs(UnmanagedType.BStr)]
public string value;

Native C-DLL in C# AccessViolation/Marshalling

I have a problem with calling a C DLL fom C#
The C function is (I don't have a c header or a good spec for this :( )
int knr12_read ( char *kn12, char *ik9, char *wok, char *wlc,
char *plz, char *ort, char *woz );
kn12 is a ref parameter
This is what I've tried in C#
[return: MarshalAs(UnmanagedType.U4)]
[DllImport("Knr12.dll", CallingConvention = CallingConvention.StdCall, EntryPoint = "knr12_read", CharSet = CharSet.Ansi)]
unsafe public static extern int knr12_read(out IntPtr buffer, string ik9, string wok, string wlc, string plz, string ort, string woz);
int knr = knr12_read(out pBuffer, knrTemp, "11111", "", "98529", "Suhl", "1");
string data = Marshal.PtrToStringAnsi(pBuffer);
The returning int is always right, how it should be, but I have problems with the ref parameter pBuffer...
Also the sting type for the other variables is working...
When I use a ref,I always get an AccessViolation error knr12_read().In case I use out I get a pointer,but the String is always empty which can't be.I even tried out String as ref for char* but I get an AccessViolation error on knr12_read().
Please guide.

StringBuilder is often a good type to use when P/Invoking to functions with string returning parameters:
static extern int knr12_read(StringBuilder kn12, ...)
You'll need to initialise the string builder before you call the function, something like:
StringBuilder outString = new StringBuilder(100);
You shouldn't need the 'unsafe', and unless the 'C' code holds onto the pointers for longer than the duration of the call, you shouldn't need to worry about pinning - the framework is doing that for you.
Here's a SO question which should help: Marshal "char *" in C#

Probably you have not pinned the buffer. here is the example of how to pin the buffer data.
GCHandle pinnedRawData = GCHandle.Alloc(rawData,
GCHandleType.Pinned);
Pinning the object makes sure that the pointer is valid cause .Net runtime can always reallocate the memory as and when it thinks fit.
Try it out and let me know if it helps you.

Some P/Invoke C# to C marshalling questions working with booleans in structs

I have some problems working with boolean types and marshalling this in a struct back and forth between C# and C. I am very rusty in C but hopefully there's nothing crucially wrong in that part.
As far as I've read/seen, .NET Boolean and C# bool type is 4 bytes long whilst the C type bool is only 1 byte. For memory footprint reasons, I do not which to use the defined BOOL 4 bytes version in the C code.
Here is some simple test code that hopefully will make my questions clear:
C code:
typedef struct
{
double SomeDouble1;
double SomeDouble2;
int SomeInteger;
bool SomeBool1;
bool SomeBool2;
} TestStruct;
extern "C" __declspec(dllexport) TestStruct* __stdcall TestGetBackStruct(TestStruct* structs);
__declspec(dllexport) TestStruct* __stdcall TestGetBackStruct(TestStruct* structs)
{
return structs;
}
I call this code in C# using the following definitions:
[StructLayout(LayoutKind.Explicit)]
public struct TestStruct
{
[FieldOffset(0)]
public double SomeDouble1;
[FieldOffset(8)]
public double SomeDouble2;
[FieldOffset(16)]
public int SomeInteger;
[FieldOffset(17), MarshalAs(UnmanagedType.I1)]
public bool SomeBool1;
[FieldOffset(18), MarshalAs(UnmanagedType.I1)]
public bool SomeBool2;
};
[DllImport("Front.dll", CharSet = CharSet.Auto, SetLastError = true, CallingConvention = CallingConvention.StdCall)]
public static extern IntPtr TestGetBackStruct([MarshalAs(UnmanagedType.LPArray, SizeConst = 1)] TestStruct[] structs);
and here is the actual test function in C#:
[Test]
public void Test_CheckStructParsing()
{
var theStruct = new TestStruct();
theStruct.SomeDouble1 = 1.1;
theStruct.SomeDouble2 = 1.2;
theStruct.SomeInteger = 1;
theStruct.SomeBool1 = true;
theStruct.SomeBool2 = false;
var structs = new TestStruct[] { theStruct };
IntPtr ptr = TestGetBackStruct(structs);
var resultStruct = (TestStruct)Marshal.PtrToStructure(ptr, typeof(TestStruct));
}
This works in the sense that I do get a struct back (using the debugger to inspect it), but with totally wrong values. I.e. the marshalling does not work at all. I've tried different version of the C# struct without success. So here are my questions (1 & 2 most important):
Is the C function correct for this purpose?
How is the struct to be written correctly in order to get me the correct values in the struct back to C#? (Is it even necessary to define the struct with the StructLayout(LayoutKind.Explicit) attribute using the FieldOffset values or can I use StructLayout(LayoutKind.Sequential) instead)?
Since I am returning a pointer to the TestStruct in C, I guess it should be possible to get back an array of TestStructs in C#. But this does not seem to be possible using the Marshal.PtrToStructure function. Would it be possible in some other way?
Apparantly it is possible to use something called unions in C by having multiple struct fields point to the same memory allocation using the same FieldOffset attribute value. I understand this, but I still don't get yet when such scenario would be useful. Please enlighten me.
Can someone recommend a good book on C# P/Invoke to C/C++? I am getting a bit tired of getting pieces of information here and there on the web.
Much obliged for help with these questions. I hope they were not too many.

Stop using LayoutKind.Explicit and get rid of the FieldOffset attributes and your code will work. Your offsets were not correctly aligning the fields.
public struct TestStruct
{
public double SomeDouble1;
public double SomeDouble2;
public int SomeInteger;
[MarshalAs(UnmanagedType.I1)]
public bool SomeBool1;
[MarshalAs(UnmanagedType.I1)]
public bool SomeBool2;
};
Declare the function in C# like this:
public static extern void TestGetBackStruct(TestStruct[] structs);
The default marshalling will match your C++ declaration (your code is C++ rather than C in fact) but you must make sure that your allocate the TestStruct[] parameter in the C# code before calling the function. Normally you would also pass the length of the array as a parameter so that the C++ code knows how many structs there are.
Please don't try to return the array of structures from the function. Use the structs parameter as an in/out parameter.
I know of no book with an emphasis on P/Invoke. It appears to be something of a black art!

How to map a C array to C#?

My question has to do with trying to call a function written in C from C#. I've looked in a header file that came with the C library to understand the functions as they exist in the C dll. Here's what I see:
C code (for a function called "LocGetLocations"):
typedef enum {
eLocNoError,
eLocInvalidCriteria,
eLocNoMatch,
eLocNoMoreLocations,
eLocConnectionError,
eLocContextError,
eLocMemoryError
} tLocGetStatus;
typedef void *tLocFindCtx;
typedef void *tLocation;
PREFIX unsigned int POSTFIX LocGetLocations
(
tLocFindCtx pCtx,
tLocation *pLoc,
unsigned int pNumLocations,
tLocGetStatus *pStatus
);
In C#, I have this:
[DllImport(#"VertexNative\Location.dll")]
public static extern uint LocGetLocations(IntPtr findContext, out byte[] locations, uint numberLocations, out int status);
The problem is that I don't quite know how to handle the pLoc parameter in C#. I'm bringing it over as a byte array, although I'm not sure if that is correct. The C library's documentation says that that parameter is a pointer to an array of handles.
How can I get an array back on the C# side and access its data?
The example I was given in C, looks like this:
tLocation lLocation[20];
// other stuff
LocGetLocations(lCtx, lLocation, 20, &lStatus)
Any help would be much appreciated!

Generally, the only thing that matters is the size of the parameters. As I recall enums are integers in C, so you can simply use that. Or better, recreate the same enum in C#, I think it would work. One thing to remember is that when dealing with complex structs, one needs to use attributes to tell the framework about the desired alignment of members.

In the end, this signature works:
[DllImport(#"VertexNative\Location.dll")]
public static extern uint LocGetLocations(IntPtr findContext, [Out] IntPtr[] locations, uint numberLocations, out int status);
And I can call it like this (some refactoring needed):
IntPtr[] locations = new IntPtr[20];
int status;
// findContext is gotten from another method invocation
uint result = GeoCodesNative.LocGetLocations(findContext, locations, 20, out status);
Thanks for the help!