I'm trying to PInvoke a method which has a struct parameter with nested struct array pointer. The c declaration looks like this:
duckdb_state duckdb_query(duckdb_connection connection, const char *query, duckdb_result *out_result);
typedef struct {
void *data;
bool *nullmask;
duckdb_type type;
char *name;
} duckdb_column;
typedef struct {
idx_t column_count;
idx_t row_count;
duckdb_column *columns;
char *error_message;
} duckdb_result;
I declared them in C# like this:
[DllImport("duckdb.dll", CallingConvention = CallingConvention.Cdecl, EntryPoint = "duckdb_query")]
public static extern DuckdbState DuckdbQuery(IntPtr connection, string query, out DuckdbResult result);
[StructLayout(LayoutKind.Sequential)]
public struct DuckdbColumn
{
IntPtr data;
bool nullmask;
DuckdbType type;
string name;
}
[StructLayout(LayoutKind.Sequential)]
public struct DuckdbResult
{
public long column_count;
public long row_count;
public IntPtr columns;
public string error_message;
}
But when I try to execute the query and read the columns I don't get any meaningful data:
result = DuckdbQuery(connection, "SELECT * FROM integers", out queryResult);
DuckdbColumn[] columns = new DuckdbColumn[queryResult.column_count];
var queryResultColumns = queryResult.columns;
var columnPointer = Marshal.ReadIntPtr(queryResultColumns);
var ptrToStructure = (DuckdbColumn)Marshal.PtrToStructure(columnPointer, typeof(DuckdbColumn));
How should I change the PInvoke declarations so that I can read the columns after executign the query?
There is example c code at: DuckDB c example
Update 1
I can get column names with the following code:
for (int i = 0; i < queryResult.column_count; i++)
{
var column = (DuckdbColumn)Marshal.PtrToStructure(queryResult.columns + 8 + (Marshal.SizeOf<DuckdbColumn>() + 8) * i, typeof(DuckdbColumn));
columns[i] = column;
}
but type field still says DUCKDB_TYPE_INVALID
Update 2
As suggested by David in his answer I changed bool nullmask;to IntPtr nullmask; and I can now read column information like this:
for (int i = 0; i < queryResult.column_count; i++)
{
var column = (DuckdbColumn)Marshal.PtrToStructure(queryResult.columns + Marshal.SizeOf<DuckdbColumn>() * i, typeof(DuckdbColumn));
columns[i] = column;
}
You have translated this field incorrectly
bool *nullmask
This is not a bool it's a pointer. Declare it as
IntPtr nullmask;
There could be other errors because we can't see all the translations. Additionally, the +8 in your array access pointer arithmetic looks suspicious.
Related
Im trying to populate a struct containing multiple layers of other structs from c using c#. I am very confused on how to create the structs in c# using marshalling.
I have been looking at a lot of different examples but I haven't managed to get it working since the examples seem much more simple then the following structure.
I have tried the following but no luck so far.
The structs in c are defined as the following where I want to create a DATASET object i c#.
struct DATASET {
uint64_t n_tables;
uint64_t n_columns;
uint64_t n_segments;
uint64_t n_records;
struct DATATABLE **table;
};
struct DATATABLE {
unsigned int n_headers;
uint64_t n_columns;
uint64_t n_segments;
uint64_t n_records;
struct DATASEGMENT **segment;
};
struct DATASEGMENT {
uint64_t n_rows;
uint64_t n_columns;
double **data;
char *label;
char *header;
char **text;
};
I have tried the following in c#
[StructLayout(LayoutKind.Sequential)]
public class DATASET
{
public ulong n_tables;
public ulong n_columns;
public ulong n_segments;
public ulong n_records;
public DATATABLE[] table;
}
[StructLayout(LayoutKind.Sequential)]
public class DATATABLE
{
public uint n_headers;
public ulong n_columns;
public ulong n_segments;
public ulong n_records;
public DATASEGMENT[] segment;
}
[StructLayout(LayoutKind.Sequential)]
public class DATASEGMENT
{
public ulong n_rows;
public ulong n_columns;
public double[][] data;
[MarshalAs(UnmanagedType.LPStr)] public string label;
[MarshalAs(UnmanagedType.LPStr)] public string header;
[MarshalAs(UnmanagedType.LPStr)] public List<string> text;
}
And for creating the struct in c# i have used
[DllImport("Name_of_dll.dll")]
public static extern void print_dataset(DATASET D, int n_lines);
static void Main(string[] args)
{
DATATABLE[] table_array = { new DATATABLE { n_headers = 23 }, new DATATABLE { n_headers = 3 } };
DATASET D = new()
{
n_tables = 1,
n_columns = 3,
n_segments = 1,
n_records = 9,
table = table_array,
};
print_dataset(D, 2);
}
For testing the created structure in c# i have used the following function from c
extern __declspec(dllexport) void print_dataset(struct DATASET* D, int n_lines);
void print_dataset(struct DATASET* D, int n_lines)
{
// Works
printf_s("n_tables = %" PRIu64 ", n_columns = %" PRIu64 ", n_segments = %" PRIu64 ", n_records = %" PRIu64 "\n", D->n_tables, D->n_columns, D->n_segments, D->n_records);
// Works
printf_s("n_lines = %d\n", n_lines);
// Crashes whes it tries to get n_columns
printf_s("D->table[0]->n_columns = %" PRIu32 "\n", D->table[0]->n_headers);
// I haven't gotten this far yet
for (int i = 0; i < n_lines; i++) {
printf_s("%d | ", i);
for (int j = 0; j < D->n_columns; j++) {
printf_s("%f ", D->table[0]->segment[0]->data[j][i]);
}
printf_s("\n");
}
}
The problem is that it works until i hit the table struct where the program then crashes. The output i get is:
n_tables = 1, n_columns = 3, n_segments = 1, n_records = 9
n_lines = 2
Fatal error. System.AccessViolationException: Attempted to read or write protected memory.
This is often an indication that other memory is corrupt.
Repeat 2 times:
at COMPLEX_STRUCT.Program.print_dataset(C_SHARP.DATASET, Int32)
at COMPLEX_STRUCT.Program.Main(System.String[])
I am still pretty new to marshalling and c# in general and don't really understand what is wrong.
this is a very clean and nice solution to marsahall a struct array from unmanaged C++ code.
it is allmost perfect solution when it comes to simplicity, it took me a while to get to that level of understanding the concept, so that in few lines of code, as you can see C# Main(), i have a populated array of struct ready to be 'harvested'..
typedef struct {
int Id;
BSTR StrVal;
}Package;
extern "C" __declspec(dllexport) void dodata(int requestedLength,int StringSize, Package **Packs){
int count;
count=0;
*Packs = (Package*)LocalAlloc(0, requestedLength * sizeof(Package));
Package *Cur = *Packs;
while(count!= requestedLength)
{
Cur[count].StrVal = NULL;
Cur[count].Id = count;
Cur[count].StrVal=SysAllocString(L"abcdefghij");
Cur[count].StrVal[StringSize-1]=count+'0';
++count;
}
}
C#
[DllImport(#"ExportStructArr.dll", CallingConvention = CallingConvention.Cdecl), SuppressUnmanagedCodeSecurity]
public static extern void dodata(int requestedLength, int StringSize, out IntPtr csPkPtr);
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct csPk
{
public int V;
[MarshalAsAttribute(UnmanagedType.BStr, SizeConst = 10)]
public string testStr;
}
static void Main(string[] args){
int ArrL = 16000;
csPk[] Cpk = new csPk[ArrL];
IntPtr CpkPtr = IntPtr.Zero;
int szPk = Marshal.SizeOf(typeof(csPk));
dodata(ArrL, 10, out CpkPtr);
}
now all i have to do is :
for (int i = 0; i < Cpk.Length; i++)
{
Cpk[i] = (csPk)Marshal.PtrToStructure(new IntPtr(CpkPtr.ToInt32() + (szPk * i)), typeof(csPk));
}
the solution is quite easy as you can see
the question is using unsafe or any kind of transformation to the data, even going down to bytes...
how could i optimize it to perform better returning the data ?
Edit:
links i have tried to learn from other answers here in SO:
answer from Hans Passant
answer from AbdElRaheim
answer from supercat
also tried google : wikipedia , a github post by stephentoub
this is a complete blazing fast solution to populate a list of objects, i did my best
and i will be happy to have comments and suggestions.
c++
typedef struct _DataPacket
{
BSTR buffer;
UINT size;
} DataPacket;
extern "C" __declspec(dllexport) void GetPacksUnsafe( int size, DataPacket** DpArray )
{
int szr = size;int count=0;
*DpArray = (DataPacket*)CoTaskMemAlloc( szr * sizeof( DataPacket ));
if ( DpArray != NULL )
{
DataPacket* CurPack = *DpArray;
for ( int i = 0; i < szr; i++, CurPack++ )
{
CurPack->size = i;
CurPack->buffer = SysAllocString(L"SomeText00");
CurPack->buffer[9]=i+'0';
}
}
}
C#
[DllImport(#"ExportStructArr.dll", CallingConvention = CallingConvention.Cdecl), SuppressUnmanagedCodeSecurity]
public static extern void GetPacksUnsafe(int size, PackU** outPackUArr);
[StructLayout(LayoutKind.Sequential)]
public unsafe struct PackU
{
public char* StrVal;
public int IntVal;
}
public static unsafe List<PackU> PopulateLstPackU(int ArrL)
{
PackU* PackUArrOut;
List<PackU> RtLstPackU = new List<PackU>(ArrL);
GetPacksUnsafe(ArrL, &PackUArrOut);
PackU* CurrentPack = PackUArrOut;
for (int i = 0; i < ArrL; i++, CurrentPack++)
{
RtLstPackU.Add(new PackU(){ StrVal = CurrentPack->StrVal, IntVal=CurrentPack->IntVal});
}
Marshal.FreeCoTaskMem((IntPtr)PackUArrOut);
for (int i = 0; i < 10; i++)
{
Console.WriteLine("{0}", new string(RtLstPackU[i].StrVal));
}
return RtLstPackU;
}
using the code is as simple as it could possibly be
static unsafe void Main(string[] args)
{
int ArrL = 100000;
List<PackU> LstPackU;
LstPackU = PopulateLstPackU(ArrL);
}
there you have a list of custom data as fast as a bullet..
EDIT
using pointers instead of strings :
typedef struct _DataPackCharPnt
{
char* buffer;
UINT IntVal;
} DataPackCharPnt;
extern "C" __declspec(dllexport) void GetPacksPnt( int size, DataPackCharPnt** DpArrPnt )
{
int count = 0;
int TmpStrSize = 10;
*DpArrPnt = (DataPackCharPnt*)CoTaskMemAlloc( size * sizeof( DataPackCharPnt ));
DataPackCharPnt* CurPackPnt = *DpArrPnt;
char dummyStringDataObject[]= "abcdefgHi";
for ( int i = 0; i < size; i++,CurPackPnt++ )
{
dummyStringDataObject[9] = i+'0';
CurPackPnt->IntVal=i;
CurPackPnt->buffer = (char*)malloc(sizeof(char)*TmpStrSize);
strcpy(CurPackPnt->buffer, dummyStringDataObject);
}
}
reduced the time taken from 11 to 7 ms populating 100k elements
is there any part of creating the buffer i could omit ?
the duty of dummyStringDataObject is to simulate work, say getting a file name then set the buffer with its value, so except for this extra time which is the whole purpose of this function, to return some unknown values and lengths of the strings...
could you optimize it even further ?
I want to fill a structure with data so I can pass it to my DLL functions. The structures are similar in both C# and C implementations. For instance something like (in C):
typedef struct{
apple[] apples;
leaf[] leaves;
} tree;
typedef struct{
int taste;
} apple;
typedef struct{
int color;
} leaf;
I want to create a tree structure on C#, fill it with apples and leaves and then send it to a function I have on a DLL.
How?
First of all, I think you should change the C struct a little bit:
typedef struct{
UINT cApples;
const apple *pApples;
UINT cLeaves;
const leaf *pLeaves;
} tree;
On the C# side:
[StructLayout(LayoutKind.Sequential, Pack = 1)]
internal struct Tree
{
internal uint cApples;
internal IntPtr pApples;
internal uint cLeaves;
internal IntPtr pLeaves;
}
You can define Leaf and Apple similarly. Then you can populate them on C# side as follows:
private Tree CreateTree(Leaf[] leaves, Apple[] apples)
{
var result = new Tree();
result.cLeaves = (uint)leaves.Length;
result.cApples = (uint)apples.Length;
result.pLeaves = AllocateNativeMemory(leaves);
result.pApples = AllocateNativeMemory(apples);
return result;
}
private IntPtr AllocateNativeMemory<T>(T[] elements) where T : struct
{
int elementSize = Marshal.SizeOf(typeof(T));
IntPtr result = Marshal.AllocHGlobal(elements.Length * elementSize);
for (int i = 0; i < elements.Length; i++)
{
Marshal.StructureToPtr(
elements[i], new IntPtr((long)result + i * elementSize), false);
}
return result;
}
Now you can pass the result of the CreateTree method to the extern method that calls the C side.
Remark: The allocated memory should be freed; otherwise, your app will leak. If you decide that the C# side is responsible to free the allocated memory, you should do it at the end as follows:
private static void FreeTree(Tree tree)
{
FreeNativeMemory<Leaf>(tree.pLeaves, tree.cLeaves);
FreeNativeMemory<Apple>(tree.pApples, tree.cApples);
}
private static void FreeNativeMemory<T>(IntPtr arrayPtr, uint arrayLen) where T : struct
{
int elementSize = Marshal.SizeOf(typeof(T));
for (uint i = 0; i < arrayLen; i++)
{
Marshal.DestroyStructure(new IntPtr((long)arrayPtr + i * elementSize), typeof(T));
}
Marshal.FreeHGlobal(arrayPtr);
}
How to convert the IntPtr to an array. Actually I called the function from unmanaged dll. It returns IntPtr. Now I need to convert it to an array. Please any one give an idea.Code snippet is below.
Unmanaged function declared
[DllImport("NLib.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern unsafe IntPtr N_AllocPt1dArray(NL_INDEX n, ref stacks S);
Calling the function
void Function1()
{
IntPtr PPtr=N_AllocPt1dArray(n, ref S);
}
Now I need to convert PPtr to an array(array is demo[]).where demo is defined by
public unsafe struct demo
{
public int x ;
public int y ;
public int z ;
}demo DEMO;
Try this:
array[0] = (demo)System.Runtime.InteropServices.Marshal.PtrToStructure(PPtr , typeof(demo));
UPDATE :
Solution 2 of this page is what you need.
It depends of what type of data you are pointing to, the next code get an array of strings from an IntPtr:
nstring is the number of elements you expect to get.
You can modify the code to satisfy your needs, but this can give you an idea of how to retrive data from an IntPtr in an unmaganed block code.
private string[] ConvertIntPtrToStringArray(IntPtr p, int nstring)
{
//Marshal.ptr
string[] s = new string[nstring];
char[] word;
int i, j, size;
unsafe
{
byte** str = (byte**)p.ToPointer();
i = 0;
while (i < nstring)
{
j = 0;
while (str[i][j] != 0)
j++;
size = j;
word = new char[size];
j = 0;
while (str[i][j] != 0)
{
word[j] = (char)str[i][j];
j++;
}
s[i] = new string(word);
i++;
}
}
return s;
}
cheers,
Kevin
I am trying to use a DLL written in C in a C# application. I've created a simplified example that replicates the issue I am having.
The C code below creates an array of struct data's and assigns the array pointer to the array parameter passed to the get_data() function. The C# code is supposed to be the boilerplate code needed for marshalling the struct to be used in C#, but is presenting issues for me.
C code
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
struct data {
int32_t value;
union {
struct person {
uint8_t first[10];
uint8_t last[10];
} person;
struct number {
int32_t imaginary;
int32_t real;
} number;
} type;
};
int get_data(int count, struct data ***array)
{
int i;
/* allocate pointers */
*array = calloc(count, sizeof(struct data*));
if (*array == NULL)
return 1;
for (i = 0; i < count; i++) {
/* allocate data struct */
struct data *data = calloc(1, sizeof(struct data));
if (data == NULL)
return 2;
if ((i % 2) == 0) {
/* if even, its human */
data->value = i;
memcpy(data->type.person.first, "john", 4);
memcpy(data->type.person.last, "doe", 3);
} else {
/* if odd its a number */
data->value = i;
data->type.number.imaginary = -1;
data->type.number.real = i + 1;
}
(*array)[i] = data;
}
return 0;
}
C# code
[DllImport("libdata.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern Int32 get_data(Int32 count, ref IntPtr array);
[StructLayout(LayoutKind.Sequential)]
public struct Person
{
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 10)]
public String first;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 10)]
public String last;
}
[StructLayout(LayoutKind.Sequential)]
public struct Number
{
public Int32 imaginary;
public Int32 real;
}
[StructLayout(LayoutKind.Explicit)]
public struct TypeUnion
{
[FieldOffset(0)]
public Person person;
[FieldOffset(0)]
public Number number;
}
[StructLayout(LayoutKind.Sequential)]
public struct Data
{
public Int32 value;
public TypeUnion type;
}
Right now when I run my test program I get an exception:
System.TypeLoadException was unhandled
Message=Could not load type 'WpfRibbonApplication1.TypeUnion' from assembly 'WpfRibbonApplication1, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null' because it contains an object field at offset 0 that is incorrectly aligned or overlapped by a non-object field.
I have tried several different ways of marshalling the Person strings, but get the exception whichever way I try (using this as reference). Am I missing something obvious? Could I get some help to properly read the array created in the C function within my C# application?
Edit (per David Heffernan's comment)
IntPtr arrayPtr = new IntPtr();
int count = 4;
int ret = LibData.get_data(count, ref arrayPtr);
Console.WriteLine("ret=" + ret);
for (int i = 0; i < count; i++)
{
IntPtr dataPtr = (IntPtr)Marshal.ReadIntPtr(arrayPtr) + (i * Marshal.SizeOf(typeof(IntPtr)));
Data data = (Data)Marshal.PtrToStructure(dataPtr, typeof(Data));
Console.WriteLine("value=" + data.value);
if ((i % 2) == 0)
{
// even is human
Console.WriteLine("first=" + data.type.first);
Console.WriteLine("last=" + data.type.last);
}
else
{
// odd is number
Console.WriteLine("imaginary=" + data.type.imaginary);
Console.WriteLine("real=" + data.type.real);
}
Console.WriteLine("");
}
The error message is telling you that you cannot overlay an object field with a non-object field. You are overlaying a string with an int.
There's no way you are going to get around that using FieldOffset to replicate the native union. As I see it you have two main options:
Stop using a union and include both Person and Number structs in the Data struct.
Continue using a union, but marshal it yourself. Use the Marshal class to read the data in the struct. For example, you'd use Marshal.ReadInt32 to read the integers, Marshal.Copy to read the character arrays and so on.