We Keep Coding

Having trouble marshalling an array of structs from c to c#

I'm trying to marshal an array of c structs into C# (Using Unity) but, no matter the method I use, I always get an exception or a crash.
I'm loading dlls (libretro cores) that conform (or should...) to the Libretro API, the c/c++ side is not available to me (more precisely, not allowed to be modified by me), which means I have to handle the data I get back from that dll no matter how it is laid out.
The C API structs are defined as follow (RETRO_NUM_CORE_OPTION_VALUES_MAX is a constant with a value of 128):
struct retro_core_option_value
{
const char *value;
const char *label;
};
struct retro_core_option_definition
{
const char *key;
const char *desc;
const char *info;
struct retro_core_option_value values[RETRO_NUM_CORE_OPTION_VALUES_MAX];
const char *default_value;
};
struct retro_core_options_intl
{
struct retro_core_option_definition *us;
struct retro_core_option_definition *local;
};
My C# mappings look like this at the moment:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct retro_core_option_value
{
public char* value;
public char* label;
}
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct retro_core_option_definition
{
public char* key;
public char* desc;
public char* info;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = RETRO_NUM_CORE_OPTION_VALUES_MAX)]
public retro_core_option_value[] values;
public char* default_value;
}
[StructLayout(LayoutKind.Sequential)]
public struct retro_core_options_intl
{
public IntPtr us;
public IntPtr local;
}
The callback function has the following signature on the C# side:
public unsafe bool Callback(retro_environment cmd, void* data)
retro_environment is an unsigned int converted to an enum, a switch is performed on it and then dictates how to handle the void* data pointer appropriately. Here data is a retro_core_options_intl*.
I'm able to do the void* conversion in 2 ways:
retro_core_options_intl intl = Marshal.PtrToStructure<retro_core_options_intl>((IntPtr)data);
or
retro_core_options_intl* intl = (retro_core_options_intl*)data;
I get a readable address with both approaches (intl.us for the first and intl->us for the second), the "local" part is empty in my particular case but the "us" part is defined as mandatory by the API. intl->us points to an array of retro_core_option_definition of variable length.
The issue I'm having is trying to read the values inside of this mandatory construct.
The array I'm trying to load right now can be seen here: https://github.com/visualboyadvance-m/visualboyadvance-m/blob/master/src/libretro/libretro_core_options.h at line 51.
The API defines a fixed size for the "struct retro_core_option_value values[RETRO_NUM_CORE_OPTION_VALUES_MAX]" struct member, but code that comes in is almost always defined as an array where the last element is "{ NULL, NULL }" to indicate the end, so they don't always (almost never) contain 128 values.
I tried:
retro_core_options_intl intl = Marshal.PtrToStructure<retro_core_options_intl>((IntPtr)data);
retro_core_option_definition us = Marshal.PtrToStructure<retro_core_option_definition>(intl.us);
This gives a NullReferenceException.
retro_core_options_intl intl = Marshal.PtrToStructure<retro_core_options_intl>((IntPtr)data);
retro_core_option_definition[] us = Marshal.PtrToStructure<retro_core_option_definition[]>(intl.us);
This gives a retro_core_option_definition array of 0 length.
retro_core_options_intl intl = Marshal.PtrToStructure<retro_core_options_intl>((IntPtr)data);
retro_core_option_definition us = new retro_core_option_definition();
Marshal.PtrToStructure(intl.us, us);
This gives a "destination is a boxed value".
That's basically where I'm at...
Any help would be much appreciated :)
The entire codebase can be found here: https://github.com/Skurdt/LibretroUnityFE

First thing I see is that you either need to use wchar_t types instead of char types in you C code, or you can use byte instead of char in C#. System.Char in C# is two bytes. char in C code is 1 byte.
You can also use System.String in the C# code and annotate it with a MarshalAs attribute to tell it what type of char data is coming in, such as Ansi or Unicode C strings.

Access legacy C-dll with C# using unsafe fixed size array of struct within struct

I have a legacy C-dll to access a Hardware Device. The dll uses structures containing structure Arrays as function arguments. And I am having a hard time to get this working together with C# (in unsafe mode which is ok since access Speed is an issue here).
The original declaration from DeviceDll.h of the legacy C-dll Looks like:
typedef struct tag_RESULT_JUDGEMENT
{
short nJudgementResult;
} struc_RESULT_JUDGEMENT;
typedef struct tag_RESULT_FORMULA
{
float fFormulaResult[3];
float fAnalogResult;
} struc_RESULT_FORMULA;
typedef struct tag_RESULT_SCRIPT
{
short nScriptNo;
float fTime;
short nFormulaCount;
struc_RESULT_FORMULA sFormula[32];
short nJudgementCount;
struc_RESULT_JUDGEMENT sJudgement[8];
} struc_RESULT_SCRIPT;
WORD PASCAL GetResult(LPCTSTR pDeviceCode, struc_RESULT_SCRIPT_NO_LIST* pList, struc_RESULT_SCRIPT* pResult[]);
Declaring the C-dll functions in C# is done in the following way:
[DllImport("ExternalDevice.dll")]
public unsafe static extern int GetResult(StringBuilder pDeviceCode, struc_RESULT_SCRIPT_NO_LIST* pList, struc_RESULT_SCRIPT **ppResult);
and the structure which I want to access I have declared in a way which unfortunatly gives an error:
[StructLayout(LayoutKind.Explicit)]
public unsafe struct struc_RESULT_SCRIPT {
[FieldOffset(0)]
public Int16 nScriptNo;
[FieldOffset(2)]
public float fTime;
[FieldOffset(6)]
public Int16 iFormulaCount;
[FieldOffset(8)
public fixed struc_RESULT_FORMULA[32] oFormula;
}
Error: Buffer of a fixed size must be of "bool", "Byte", . . . or "double"
Is there a way to declare a fixed structure Array within a structure so that I can use a structure-type variable afterwards as an Argument for calling the legacy DLL ?
What I have tried so far:
Avoiding a structure Array within a structure (rather clumsy but working)
[StructLayout(LayoutKind.Explicit)]
public unsafe struct struc_RESULT_SCRIPT {
[FieldOffset(0)]
public Int16 nScriptNo;
[FieldOffset(2)]
public float fTime;
[FieldOffset(6)]
public Int16 iFormulaCount;
[FieldOffset(8)
public struc_RESULT_FORMULA oFormula01;
[FieldOffset(8 + 16)
public struc_RESULT_FORMULA oFormula02;
. . .
[FieldOffset(8 + 31*16)
public struc_RESULT_FORMULA oFormula32;
}
. . .
struc_RESULT_SCRIPT** ppResult; //local variable => allocated on the stack => so it's already fixed
. . .
int iRv = GetResult(sbMpmIp, &oScriptList, ppResult);
struc_RESULT_FORMULA oFormulaResult = ppResult[0]->oFormula01;
This is working - but accessing oFormula01 … oFormula32 for 32 structure-variables is rather clumsy. I would strongly prefer to get the result as an Array so that I can Access it like oFormula = arrayFormula[xx]; within a for - loop.
Is there a way to declare an unsafe, fixed structure Array within a structure in C# - or a feasable work-around ?
Thank you very much in advance!

According to the documentation for fixed sized buffers there is no way use structs in them.
The best option seem to do as you have done and declare 1..n fields. The anwer to this related question suggest the same thing. If the problem is simply not being able to loop over the fields, then consider adding a iterator block, like:
public IEnumerable<struc_RESULT_FORMULA> GetResultFormulas(){
yield return oFormula01;
...
yield return oFormula32;
}
An alternative would be a large switch statement/expression if you need random indexing.

interop with nim return Struct Array containing a string /char* member

interoping nim dll from c# i could call and execute the code below
if i will add another function (proc) that Calls GetPacks() and try to echo on each element's buffer i could see the output in the C# console correctly
but i could not transfer the data as it is, i tried everything but i could not accomplish the task
proc GetPacksPtrNim(parSze: int, PackArrINOUT: var DataPackArr){.stdcall,exportc,dynlib.} =
PackArrINOUT.newSeq(parSze)
var dummyStr = "abcdefghij"
for i, curDataPack in PackArrINOUT.mpairs:
dummyStr[9] = char(i + int8'0')
curDataPack = DataPack(buffer:dummyStr, intVal: uint32 i)
type
DataPackArr = seq[DataPack]
DataPack = object
buffer: string
intVal: uint32
when i do same in c/c++ the type i am using is either an IntPtr or char*
that is happy to contain returned buffer member
EXPORT_API void __cdecl c_returnDataPack(unsigned int size, dataPack** DpArr)
{
unsigned int dumln, Index;dataPack* CurDp = {NULL};
char dummy[STRMAX];
*DpArr = (dataPack*)malloc( size * sizeof( dataPack ));
CurDp = *DpArr;
strncpy(dummy, "abcdefgHij", STRMAX);
dumln = sizeof(dummy);
for ( Index = 0; Index < size; Index++,CurDp++)
{
CurDp->IVal = Index;
dummy[dumln-1] = '0' + Index % (126 - '0');
CurDp->Sval = (char*) calloc (dumln,sizeof(dummy));
strcpy(CurDp->Sval, dummy);
}
}
c# signature for c code above
[DllImport(#"cdllI.dll", CallingConvention = CallingConvention.Cdecl), SuppressUnmanagedCodeSecurity]
private static extern uint c_returnDataPack(uint x, DataPackg.TestC** tcdparr);
C# Struct
public unsafe static class DataPackg
{
[StructLayout(LayoutKind.Sequential)]
public struct TestC
{
public uint Id;
public IntPtr StrVal;
}
}
finally calling the function like so:
public static unsafe List<DataPackg.TestC> PopulateLstPackC(int ArrL)
{
DataPackg.TestC* PackUArrOut;
List<DataPackg.TestC> RtLstPackU = new List<DataPackg.TestC>(ArrL);
c_returnDataPack((uint)ArrL, &PackUArrOut);
DataPackg.TestC* CurrentPack = PackUArrOut;
for (int i = 0; i < ArrL; i++, CurrentPack++)
{
RtLstPackU.Add(new DataPackg.TestC() { StrVal = CurrentPack->StrVal, Id = CurrentPack->Id });
}
//Console.WriteLine("Res={0}", Marshal.PtrToStringAnsi((IntPtr)RtLstPackU[1].StrVal));//new string(RtLstPackU[0].StrVal));
return RtLstPackU;
}
how could i produce similar c code as above from Nim ?
it doesn't have to be same code, but same effect, that in c# i would be able to read the content of the string. for now, the int is readable but the string is not
Edit:
this is what i tried to make things simple
struct array of int members
Update:
it seem that the problem is to do with my settings of nim in my windows OS.
i will be updating as soon as i discover what exactly is wrong.

The string type in Nim is not equivalent to the C's const char* type. Strings in Nim are represented as pointers, pointing into a heap-allocated chunk of memory, which has the following layout:
NI length; # the length of the stored string
NI capacity; # how much room do we have for growth
NIM_CHAR data[capacity]; # the actual string, zero-terminated
Please beware that these types are architecture specific and they are really an implementation detail of the compiler that can be changed in the future. NI is the architecture-default interger type and NIM_CHAR is usually equivalent to a 8-bit char, since Nim is leaning towards the use of UTF8.
With this in mind, you have several options:
1) You can teach C# about this layout and access the string buffers at their correct location (the above caveats apply). An example implementation of this approach can be found here:
https://gist.github.com/zah/fe8f5956684abee6bec9
2) You can use a different type for the buffer field in your Nim code. Possible candidates are ptr char or the fixed size array[char]. The first one will require you to give up the automatic garbage collection and maintain a little bit of code for manual memory management. The second one will give up a little bit of space efficiency and it will put hard-limits on the size of these buffers.
EDIT:
Using cstring may also look tempting, but it's ultimately dangerous. When you assign a regular string to a cstring, the result will be a normal char * value, pointing to the data buffer of the Nim string described above. Since the Nim garbage collector handles properly interior pointers to allocated values, this will be safe as long as the cstring value is placed in a traced location like the stack. But when you place it inside an object, the cstring won't be traced and nothing prevents the GC from releasing the memory, which may create a dangling pointer in your C# code.

Try to change your struct to:
public unsafe static class DataPackg
{
[StructLayout(LayoutKind.Sequential)]
public struct TestC
{
public uint Id;
[MarshalAs(UnmanagedType.LPStr)]
public String StrVal;
}
}

Create empty native C++ byte* in C# to PInvoke with

I need to create a binary blob of empty data to PInvoke a native C++ dll that needs a unsigned char* of nulls.
The native C++ program is expecting a structure of data, and there's a nulled area of bytes in the middle, but in C# I can't just make a struct with an initialized byte[] in the middle.
My struct in C++ looks like this
struct myStruct
{
byte command;
byte returncode;
void* Source (a pointer to a string, rather a null term char*)
void* Destination (same thing)
byte filler [99]
byte options;
}
I've already figured that I can take a string and convert it to an array of bytes using myStruct.Source = (void*)Marshal.StringtToGlobalAnsi(source) (correct me if I'm wrong).
But I don't know how to fill out that empty array of bytes in the middle.
This is my C# struct so far.
[StructLayout(LayoutKind.Sequential, Pack = 1)]
unsafe struct Trans_s
{
public byte command;
public byte returnCode;
public void* pSource;
public void* pDest;
public byte* filler;
public byte options;
}

For the array of bytes, you need to mark the field in the C# struct as a byvalarray. The default marshaller uses LPArray if you do not.
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 99)]
public byte[] filler;
You need to allocate the memory for filler when you create an instance of the struct.

No, the C++ code doesn't expect an unsigned char* of nulls. It expects that many padding bytes inside the struct.
The C# fixed keyword might help you here.

C# struct to C++ Marshalling problem

I'm invoking a C++ function from within C#.
This is the function header in C++ :
int src_simple (SRC_DATA *data, int converter_type, int channels) ;
And this is the equivilent C# function :
[DllImport("libsamplerate-0.dll")]
public static extern int src_simple(ref SRC_DATA sd, int converter_type, int channels);
This is the SRC_DATA structure in C++ and then in C# :
typedef struct
{ float *data_in, *data_out ;
long input_frames, output_frames ;
long input_frames_used, output_frames_gen ;
int end_of_input ;
double src_ratio ;
} SRC_DATA ;
This is the C# struct I have defined :
[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct SRC_DATA
{
public IntPtr data_in, data_out;
public long input_frames, output_frames;
public long input_frames_used, output_frames_gen;
public int end_of_input;
public double src_ratio;
}
The big problem is that the last parameter , src_ratio, doesn't get delivered properly to the C++ function, (it sees it as 0 or something invalid).
Are my declarations correct?
Thanks

Are you sure that the problem is in src_ratio member? long in C# is 64 bit. In C++ on Win32 platform long is 32 bit, I think you need to use int in C# for long C++ structure members. Also, Pack = 1 looks a bit strange, do you use the same structure member alignment in C++?

You force packing in C# but not in C++. What could be happening is that the C++ compiler is padding the 7 in32's with four additional bytes to ensure the double is 8-byte aligned.
Check the #pragma pack compiler directives.

Check what size an int is in your C++ compiler. A C# int is always 32 bits.