We Keep Coding

faster way to return data as an array interoping c++

this is a very clean and nice solution to marsahall a struct array from unmanaged C++ code.
it is allmost perfect solution when it comes to simplicity, it took me a while to get to that level of understanding the concept, so that in few lines of code, as you can see C# Main(), i have a populated array of struct ready to be 'harvested'..
typedef struct {
int Id;
BSTR StrVal;
}Package;
extern "C" __declspec(dllexport) void dodata(int requestedLength,int StringSize, Package **Packs){
int count;
count=0;
*Packs = (Package*)LocalAlloc(0, requestedLength * sizeof(Package));
Package *Cur = *Packs;
while(count!= requestedLength)
{
Cur[count].StrVal = NULL;
Cur[count].Id = count;
Cur[count].StrVal=SysAllocString(L"abcdefghij");
Cur[count].StrVal[StringSize-1]=count+'0';
++count;
}
}
C#
[DllImport(#"ExportStructArr.dll", CallingConvention = CallingConvention.Cdecl), SuppressUnmanagedCodeSecurity]
public static extern void dodata(int requestedLength, int StringSize, out IntPtr csPkPtr);
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
public struct csPk
{
public int V;
[MarshalAsAttribute(UnmanagedType.BStr, SizeConst = 10)]
public string testStr;
}
static void Main(string[] args){
int ArrL = 16000;
csPk[] Cpk = new csPk[ArrL];
IntPtr CpkPtr = IntPtr.Zero;
int szPk = Marshal.SizeOf(typeof(csPk));
dodata(ArrL, 10, out CpkPtr);
}
now all i have to do is :
for (int i = 0; i < Cpk.Length; i++)
{
Cpk[i] = (csPk)Marshal.PtrToStructure(new IntPtr(CpkPtr.ToInt32() + (szPk * i)), typeof(csPk));
}
the solution is quite easy as you can see
the question is using unsafe or any kind of transformation to the data, even going down to bytes...
how could i optimize it to perform better returning the data ?
Edit:
links i have tried to learn from other answers here in SO:
answer from Hans Passant
answer from AbdElRaheim
answer from supercat
also tried google : wikipedia , a github post by stephentoub

this is a complete blazing fast solution to populate a list of objects, i did my best
and i will be happy to have comments and suggestions.
c++
typedef struct _DataPacket
{
BSTR buffer;
UINT size;
} DataPacket;
extern "C" __declspec(dllexport) void GetPacksUnsafe( int size, DataPacket** DpArray )
{
int szr = size;int count=0;
*DpArray = (DataPacket*)CoTaskMemAlloc( szr * sizeof( DataPacket ));
if ( DpArray != NULL )
{
DataPacket* CurPack = *DpArray;
for ( int i = 0; i < szr; i++, CurPack++ )
{
CurPack->size = i;
CurPack->buffer = SysAllocString(L"SomeText00");
CurPack->buffer[9]=i+'0';
}
}
}
C#
[DllImport(#"ExportStructArr.dll", CallingConvention = CallingConvention.Cdecl), SuppressUnmanagedCodeSecurity]
public static extern void GetPacksUnsafe(int size, PackU** outPackUArr);
[StructLayout(LayoutKind.Sequential)]
public unsafe struct PackU
{
public char* StrVal;
public int IntVal;
}
public static unsafe List<PackU> PopulateLstPackU(int ArrL)
{
PackU* PackUArrOut;
List<PackU> RtLstPackU = new List<PackU>(ArrL);
GetPacksUnsafe(ArrL, &PackUArrOut);
PackU* CurrentPack = PackUArrOut;
for (int i = 0; i < ArrL; i++, CurrentPack++)
{
RtLstPackU.Add(new PackU(){ StrVal = CurrentPack->StrVal, IntVal=CurrentPack->IntVal});
}
Marshal.FreeCoTaskMem((IntPtr)PackUArrOut);
for (int i = 0; i < 10; i++)
{
Console.WriteLine("{0}", new string(RtLstPackU[i].StrVal));
}
return RtLstPackU;
}
using the code is as simple as it could possibly be
static unsafe void Main(string[] args)
{
int ArrL = 100000;
List<PackU> LstPackU;
LstPackU = PopulateLstPackU(ArrL);
}
there you have a list of custom data as fast as a bullet..
EDIT
using pointers instead of strings :
typedef struct _DataPackCharPnt
{
char* buffer;
UINT IntVal;
} DataPackCharPnt;
extern "C" __declspec(dllexport) void GetPacksPnt( int size, DataPackCharPnt** DpArrPnt )
{
int count = 0;
int TmpStrSize = 10;
*DpArrPnt = (DataPackCharPnt*)CoTaskMemAlloc( size * sizeof( DataPackCharPnt ));
DataPackCharPnt* CurPackPnt = *DpArrPnt;
char dummyStringDataObject[]= "abcdefgHi";
for ( int i = 0; i < size; i++,CurPackPnt++ )
{
dummyStringDataObject[9] = i+'0';
CurPackPnt->IntVal=i;
CurPackPnt->buffer = (char*)malloc(sizeof(char)*TmpStrSize);
strcpy(CurPackPnt->buffer, dummyStringDataObject);
}
}
reduced the time taken from 11 to 7 ms populating 100k elements
is there any part of creating the buffer i could omit ?
the duty of dummyStringDataObject is to simulate work, say getting a file name then set the buffer with its value, so except for this extra time which is the whole purpose of this function, to return some unknown values and lengths of the strings...
could you optimize it even further ?

How to convert the IntPtr to an array?

How to convert the IntPtr to an array. Actually I called the function from unmanaged dll. It returns IntPtr. Now I need to convert it to an array. Please any one give an idea.Code snippet is below.
Unmanaged function declared
[DllImport("NLib.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern unsafe IntPtr N_AllocPt1dArray(NL_INDEX n, ref stacks S);
Calling the function
void Function1()
{
IntPtr PPtr=N_AllocPt1dArray(n, ref S);
}
Now I need to convert PPtr to an array(array is demo[]).where demo is defined by
public unsafe struct demo
{
public int x ;
public int y ;
public int z ;
}demo DEMO;

Try this:
array[0] = (demo)System.Runtime.InteropServices.Marshal.PtrToStructure(PPtr , typeof(demo));
UPDATE :
Solution 2 of this page is what you need.

It depends of what type of data you are pointing to, the next code get an array of strings from an IntPtr:
nstring is the number of elements you expect to get.
You can modify the code to satisfy your needs, but this can give you an idea of how to retrive data from an IntPtr in an unmaganed block code.
private string[] ConvertIntPtrToStringArray(IntPtr p, int nstring)
{
//Marshal.ptr
string[] s = new string[nstring];
char[] word;
int i, j, size;
unsafe
{
byte** str = (byte**)p.ToPointer();
i = 0;
while (i < nstring)
{
j = 0;
while (str[i][j] != 0)
j++;
size = j;
word = new char[size];
j = 0;
while (str[i][j] != 0)
{
word[j] = (char)str[i][j];
j++;
}
s[i] = new string(word);
i++;
}
}
return s;
}
cheers,
Kevin

Return array of pointers from c++ to c#

Below I have a code snippet from c++.
I need to return array of pointers (to TempStruct).
The problem is that on c# side I get only one element. On the other I get AV.
**C++**
extern "C" __declspec(dllexport) void GetResult(TempStruct** outPtr, long *size)
{
*outPtr = (TempStruct*)new TempStruct*[2];
outPtr[0] = new TempStruct("sdf", 123);
outPtr[1] = new TempStruct("abc", 456);
*size = 2;
}
**C#**
[DllImport("test.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
public static extern void GetResult(out IntPtr outPtr, out int size);
IntPtr ptr = IntPtr.Zero;
int length;
GetResult(out ptr, out length);
int size = Marshal.SizeOf(typeof(TempStruct));
TempStruct[] someData2 = new TempStruct[length];
for (int i = 0; i < length; i++)
{
IntPtr wskptr = (IntPtr)(ptr.ToInt64() + (size * i));
someData2[i] = (TempStruct)Marshal.PtrToStructure(wskptr, typeof(TempStruct));
}

You are doing it wrong.
You are mixing pointer types.
By using the new TempStruct() you are creating an array of pointers to TempStruct. The example I gave you created an array of TempStruct. See the difference?
Now... TempStruct** outPtr should be TempStruct*** outPtr (because you want to return (*) an array (*) of pointers (*)... Or TempStruct**& if you prefer :-)
Change this line
someData2[i] = (TempStruct)Marshal.PtrToStructure(Marshal.ReadIntPtr(wskptr), typeof(TempStruct));
Because you must read the single pointers.
I do hope you are deleting the various TempStruct with delete and using the
delete[] ptr;
operator to delete the array of structures.
Full example:
C++:
struct TempStruct
{
char* str;
int num;
// Note the strdup. You don't know the source of str.
// For example if the source is "Foo", then you can't free it.
// Using strdup solves this problem.
TempStruct(const char *str, int num)
: str(strdup(str)), num(num)
{
}
~TempStruct()
{
free(str);
}
};
extern "C"
{
__declspec(dllexport) void GetResult(TempStruct ***outPtr, int *size)
{
*outPtr = new TempStruct*[2];
(*outPtr)[0] = new TempStruct("sdf", 123);
(*outPtr)[1] = new TempStruct("abc", 456);
*size = 2;
}
__declspec(dllexport) void FreeSomeData(TempStruct **ptr, int size)
{
for (int i = 0; i < size; i++)
{
delete ptr[i];
}
delete[] ptr;
}
}
C#:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi, Pack = 1), Serializable]
internal struct TempStruct
{
public string str;
public int num;
}
[DllImport("NativeLibrary.dll", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi)]
static extern void GetResult(out IntPtr outPtr, out int numPtr);
[DllImport("NativeLibrary.dll", CallingConvention = CallingConvention.Cdecl)]
static extern void FreeSomeData(IntPtr ptr, int num);
// C++ will return its TempStruct array in ptr
IntPtr ptr;
int size;
GetResult(out ptr, out size);
TempStruct[] someData2 = new TempStruct[size];
for (int i = 0; i < size; i++)
{
IntPtr ptr2 = Marshal.ReadIntPtr(ptr, i * IntPtr.Size);
someData2[i] = (TempStruct)Marshal.PtrToStructure(ptr2, typeof(TempStruct));
}
// Important! We free the TempStruct allocated by C++. We let the
// C++ do it, because it knows how to do it.
FreeSomeData(ptr, size);
Note that you don't need [Serializable] and Pack=1 on the C# struct
More correct for the C++:
__declspec(dllexport) void GetResult(TempStruct **&outPtr, int &size)
{
outPtr = new TempStruct*[2];
outPtr[0] = new TempStruct("sdf", 123);
outPtr[1] = new TempStruct("abc", 456);
size = 2;
}
It is more correct because both outPtr and size can't be NULL. See https://stackoverflow.com/a/620634/613130 . The C# signature is the same.

The C++ code is wrong. It's returning an array of pointers to struct. The fact that you cast the value returned by new should have alerted you to the fact that you made a mistake. You want to return an array of struct.
It should be:
*outPtr = new TempStruct[2];
(*outPtr)[0].str = "sdf";
(*outPtr)[0].i = 123;
(*outPtr)[1].str = "abc";
(*outPtr)[1].i = 456;
*size = 2;

Handling Array returned from c++ dll to C#

I have this in my dll created in c++
extern "C" __declspec(dllexport)
char* __stdcall hh()
{
char a[2];
a[0]='a';
a[1]='b';
return(a);
}
And this is how I am trying to handle code in c#
[DllImport(#"mydll.dll",CharSet = CharSet.Ansi,CallingConvention = CallingConvention.StdCall)]
public static extern IntPtr hh();
static void Main(string[] args)
{
IntPtr a = hh();
//How to proceed here???
}
}
Help in proceeding further.

There is no way to handle such arrays. char a[2] is allocated on the stack in your C++ function and is destroyed as soon as you return from it. You should either pass an array from C# and fill it in the C++ code or allocate array in the heap and provide some means for freeing it.
When you have it correct the handling will depend on how you return the data from C++ code. If it's still IntPtr you could use Marshal.ReadByte methods to read characters from memory and use Encoding methods to convert those bytes into string if necessary.
const int bufferSize = 2; // suppose it's some well-known value.
IntPtr p = ...; // get this pointer somehow.
for (var i = 0; i != bufferSize; ++i)
{
var b = Marshal.ReadByte(p, i);
Console.WriteLine(b);
}

I got a solution as follows::
OUR C++ code goes as follows
extern "C" __declspec(dllexport)
char** __stdcall hh()
{
static char* myArray[3] = {"A1", "BB2", "CC3",};
return myArray;
}
And C# goes as follows
[DllImport(#"ourdll.dll",CharSet = CharSet.Ansi,CallingConvention = CallingConvention.StdCall)]
public static extern IntPtr hh();
static void Main(string[] args)
{
IntPtr a = hh();
int j = 0;
string[] s=new string[100];
do
{
s[j] = Marshal.PtrToStringAnsi(Marshal.ReadIntPtr(a,4*j));
j++;
}
while(s[j-1] != null);
}
The only problem now faced is that how can we know size of the array
so that in this statement
string[] s=new string[100];
we neednot waste our memory.

The answer would be
string stra = Marshal.PtrToStringAnsi(a);
But you also have the problem that the dll returns garbage per your code as char* is a local c style string.
Would be ok if you would return something like:
const char* str = "Hello from DLL.";

Try to use not empty StringBuilder as the return value.

passing array of structs from c# to regular dll

I have a regular dll with the following function exported.
extern "C" __declspec(dllexport) int FindNearestStuff(double _latitude, double _longitude , LocationStruct * locations[])
LocationStruct is very simple
struct LocationStruct
{
long positionIndex;
long item;
};
I'm tryign to call it from c# using
[DllImport("myclever.dll", CharSet = CharSet.None)]
private static extern int FindNearestStuff(double _latitude, double _longitude,
ref LocationStruct [] locations);
It's all cool and funky and I can step into the dll function from the debugger.
Inside the dll the LocationStruct array is populated correctly and all is very good.
The problem I have is when it returns back from the dll, the LocationStruct array is not coming back with the data - just empty values...
What am I missing?

thanks so much for your help - you certainly put me onthe right direction and i really appreciate your assistance!
This is the solution which seems to work for me;
[DllImport("myclever.dll", CharSet = CharSet.None)]
private static extern int FindNearestStuff(double _latitude, double _longitude, IntPtr locations);
public static int FindNearestStuff(double _latitude, double _longitude, LocationStruct[] locations)
{
int returnValue = -1;
LocationStruct temp;
temp.roadIndex = 1;
temp.tdist = 1;
int iStructSize = Marshal.SizeOf(temp);
try
{
IntPtr locationsPtr = IntPtr.Zero;
IntPtr buffer = Marshal.AllocHGlobal(iStructSize * 10);
FindNearestRoads(_latitude, _longitude, buffer);
for (int i = 0; i < 10; i++)
{
IntPtr ptr = new IntPtr(buffer.ToInt32() + iStructSize * i);
locations[i] = (LocationStruct)Marshal.PtrToStructure(ptr, typeof(LocationStruct));
}
returnValue = 0;
}
catch
{
}
return returnValue;
}

I'm not sure that you will be able to do this automatically since C# has no way of knowing how many items are returned in the locations variable (I'm assuming that the return value of FindNearestStuff is the number of entries in locations.)
You will have to manually marshal your data using the Marshall class and a process like this:
[DllImport("myclever.dll", CharSet = CharSet.None)]
private static extern int FindNearestStuff(double _latitude, double _longitude,
out IntPtr locations);
public static LocationStruct[] FindNearestStuff(double latitude, double longitude) {
IntPtr locationsPtr = IntPtr.Zero;
int numLocations = FindNearestStuff(latitude, longitude, out locationsPtr);
LocationsStruct[] locations = new LocationsStruct[numLocations];
for (int i = 0; i < numLocations; i++) {
// locationsPtr is a pointer to the struct, so read the value
// at locationPtr which will be the address of the struct and
// then manually marshal the struct from that address
locaitonsStruct[i] = (LocationStruct)Marshal.PtrToStructure(
Marshal.ReadIntPtr(locationsPtr), typeof(LocationsStruct));
// Move to the location pointer to the next address of a
// pointer to a struct
locationsPtr += IntPtr.Size;
}
return locations;
}
I haven't actually tried this so caveat emptor.