I have a 3rd party C library which one of its exported method's is as follows:
#define MAX_INDEX 8
int GetStuff(IN char* index[MAX_INDEX], OUT char* buf, IN size_t size);
The first argument is populated with pointers into the buf argument where specific strings are stored. The size indicates how long each string is expected to be in the buf buffer. My C# P/Invoke method for this currently looks like this:
[DllImport("Path/To/Dll", CharSet = CharSet.Ansi)]
private static extern int GetStuff(IntPtr indecies, IntPtr buf, Int32 size);
The C# P/Invoke method is private because I'm wrapping it's functionality in a public "getter" method which handles the allocation/deallocation of memory for the caller. When I use this method in C++, iterating is quite simple actually. I simply do something like:
char* pIndecies[MAX_INDEX];
char* pBuffer = new char[MAX_INDEX * (256 + 1)]; // +1 for terminating NULL
GetStuff(pIndecies, pBuffer, 256);
// iterate over the items
for(int i(0); i < MAX_INDEX; i++) {
if(pIndecies[i]) {
std::cout << "String for index: " << i << " " << pIndecies[i] << std::endl;
}
}
Because of how these are used in C++, I decided I should probably use IntPtr objects and just allocate the memory that I'll need from the heap, call into the native code, and iterate over it the way I would in C++. Then I remembered that chars in C# are unicode characters and not ASCII characters. Would iteration in C# work the same even if I placed the iteration in an unsafe code block? My first thought was to do the following:
IntPtr pIndecies = Marshal.AllocHGlobal(MAX_INDEX * 4); // the size of a 32-pointer
IntPtr pBuffer = Marshal.AllocHGlobal(MAX_INDEX * (256 + 1)); // should be the same
NativeMethods.GetStuff(pIndecies, pBuffer, 256);
unsafe {
char* pCStrings = (char*)pIndecies.ToPointer();
for(int i = 0; i < MAX_INDEX; i++) {
if(pCStrings[i])
string s = pCStrings[i];
}
}
My question then is, "How should I iterate over what this native code method returns?" Is this the right way to marshal to this function? Should I use a StringBuilder object for the second argument? One constraining problem is that the first argument is a 1:1 mapping of a structure behind the GetStuff() method. The value of each index is crucial for understanding what you're looking at in the second buffer argument.
I appreciate any suggestions.
Thanks,
Andy
I think you are on the right track, but I'd do it without using unsafe code. Like this:
[DllImport("Path/To/Dll", CharSet = CharSet.Ansi)]
private static extern int GetStuff(IntPtr[] index, IntPtr buf, Int32 size);
....
IntPtr[] index = new IntPtr[MAX_INDEX];
IntPtr pBuffer = Marshal.AllocHGlobal(MAX_INDEX * 256 + 1);
try
{
int res = NativeMethods.GetStuff(index, pBuffer, 256);
// check res for errors?
foreach (IntPtr item in index)
{
if (item != IntPtr.Zero)
string s = Marshal.PtrToStrAnsi(item);
}
}
finally
{
Marshal.FreeHGlobal(pBuffer);
}
This is a pretty direct translation of your C++ version.
Related
I am seeing a pretty bizarre issue while trying to pass an array from C++ to C#. I am using Marshal.Copy (specifically: https://msdn.microsoft.com/en-us/library/a53bd6cz(v=vs.110).aspx).
Problem: float array from C++ to C# is yielding a few NaN's in the resulting array.
(Note: I am working in the context of the Unity game engine)
Code
Example C++ code:
extern "C" bool UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API getSomeFloats(float** points, int* count) {
std::vector<float> results;
std::vector<SOME_TYPE> key_points = <SOME_POINTS>
for (auto iter = key_points.begin(); iter < key_points.end(); iter++) {
results.push_back(static_cast<float>(iter->pt.x));
results.push_back(static_cast<float>(iter->pt.y));
}
*points = results.data();
*count = results.size();
//<Print results to csv here>
return true;
}
Example C# code:
[DllImport("NativePlugin")]
private static extern bool getSomeFloats (ref IntPtr ptrResultItems, ref int resultItemsLength);
private static float[] getFloatArrayFromNative() {
IntPtr ptrResultItems = IntPtr.Zero;
int resultItemsLength = 0;
bool success = getSomeFloats (ref ptrResultItems, ref resultItemsLength);
float[] resultItems = null;
if (success) {
// Load the results into a managed array.
resultItems = new float[resultItemsLength];
Marshal.Copy (ptrResultItems
, resultItems
, 0
, resultItemsLength);
// <PRINT out resultItems to csv here>
return resultItems;
} else {
Debug.Log ("Something went wrong getting some floats");
return new float[] { -1, -2 };
}
}
Example Ouput:
Take the following example:
C++ output (print_out.csv):
123, 456, 789
C# output (print_out_cs.csv):
123, NaN, 789
I'm completely stumped on this one. I just don't understand why only some (roughly 7/100) floats are returning NaN. Does anyone have any advice/insight that might help?
Thanks!
Found few problems in your code:
1. std::vector<float> results; is declared on the stack. It will be gone by the time the function has returned. Declare it as a pointer
std::vector<float> *results = new std::vector<float>(10);
but make sure to also declare a function that will free it on the C++ side.
2.The function parameter do not match.
Your C++:
getSomeFloats(float** points, int* count, CameraPose* pose)
Your C#:
getSomeFloats (ref IntPtr ptrResultItems, ref int resultItemsLength);
You either have to remove CameraPose* pose from the C++ side or add IntPtr pose to the C# side.
3. The use of UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API.
You don't need that. This is used when you want to use Unity's built in functions such as GL.IssuePluginEvent. You are not using it in this case.
This should do it:
#define DLLExport __declspec(dllexport)
extern "C"
{
DLLExport void fillArrayNative(float* data, int count, int* outValue)
{
}
}
4.C# has a garbage collector that moves variables around in the memory. You must pin C# array if you want to modify it from C++ side. You only need to pin C# array. Another option is to allocate the array on C++ side, return it to C# copy it to a temporary variable on the C# side then delete it on the C++ side.
5.Copy the result back to the array instead of assigning it.
Recommended method:
There are just many ways to do this and some of them are extremely slow. If you want to use Marshal.Copy, you have to allocate the array on the C++ side or else you will run into some undefined behavior.
The fastest and the most efficient way to do this is to allocate the array on the C# side as a global variable. Pass the array and its length to the native side. Also pass a third parameter which C++ can use to tell C# the amount of index that has been updated or written to.
This is much more better than creating new array, copying it to C# variable then destroying it each time the function is called.
This is what you should be using:
C++:
#define DLLExport __declspec(dllexport)
extern "C"
{
DLLExport void fillArrayNative(float* data, int count, int* outValue)
{
std::vector<float> results;
for (int i = 0; i < count; i++)
{
//Fill the array
data[i] = results[i];
}
*outValue = results.size();
}
}
You can also use: std::copy ( data, data+count, results.begin() ); instead of loop to copy the data too.
C#:
[DllImport("NativePlugin", CallingConvention = CallingConvention.Cdecl)]
private static extern void fillArrayNative(IntPtr data, int count, out int outValue);
public unsafe void getFillArrayNative(float[] outArray, int count, out int outValue)
{
//Pin Memory
fixed (float* p = outArray)
{
fillArrayNative((IntPtr)p, count, out outValue);
}
}
Usage:
const int arraySize = 44500;
float[] arrayToFill = new float[arraySize];
void Start()
{
int length = arrayToFill.Length;
int filledAmount = 0;
getFillArrayNative(arrayToFill, length, out filledAmount);
//You can then loop through it with with the returned filledAmount
for (int i = 0; i < filledAmount; i++)
{
//Do something with arrayToFill[i]
}
}
This is just an example and it is faster than all other methods I've used before. Avoid doing it the way you are currently doing it with Marshal.Copy. If you still want to do it your way or use Marshal.Copy then here is the appropriate way to do it which requires allocation, copying data and de-allocating memory in each call.
The pointer you return in getSomeFloats is owned by results. Before getSomeFloats returns, the vector destructor for results will free that memory. When the C# code tries to use the pointer, you are accessing unallocated memory, which results in Undefined Behavior. In your case most of the data hasn't been changed yet, but some of it has. Potentially any or all of the data could have been changed (if the memory has been reused), or even a program crash (if the freed memory has been returned to the OS).
I am seeing a pretty bizarre issue while trying to pass an array from C++ to C#. I am using Marshal.Copy (specifically: https://msdn.microsoft.com/en-us/library/a53bd6cz(v=vs.110).aspx).
Problem: float array from C++ to C# is yielding a few NaN's in the resulting array.
(Note: I am working in the context of the Unity game engine)
Code
Example C++ code:
extern "C" bool UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API getSomeFloats(float** points, int* count) {
std::vector<float> results;
std::vector<SOME_TYPE> key_points = <SOME_POINTS>
for (auto iter = key_points.begin(); iter < key_points.end(); iter++) {
results.push_back(static_cast<float>(iter->pt.x));
results.push_back(static_cast<float>(iter->pt.y));
}
*points = results.data();
*count = results.size();
//<Print results to csv here>
return true;
}
Example C# code:
[DllImport("NativePlugin")]
private static extern bool getSomeFloats (ref IntPtr ptrResultItems, ref int resultItemsLength);
private static float[] getFloatArrayFromNative() {
IntPtr ptrResultItems = IntPtr.Zero;
int resultItemsLength = 0;
bool success = getSomeFloats (ref ptrResultItems, ref resultItemsLength);
float[] resultItems = null;
if (success) {
// Load the results into a managed array.
resultItems = new float[resultItemsLength];
Marshal.Copy (ptrResultItems
, resultItems
, 0
, resultItemsLength);
// <PRINT out resultItems to csv here>
return resultItems;
} else {
Debug.Log ("Something went wrong getting some floats");
return new float[] { -1, -2 };
}
}
Example Ouput:
Take the following example:
C++ output (print_out.csv):
123, 456, 789
C# output (print_out_cs.csv):
123, NaN, 789
I'm completely stumped on this one. I just don't understand why only some (roughly 7/100) floats are returning NaN. Does anyone have any advice/insight that might help?
Thanks!
Found few problems in your code:
1. std::vector<float> results; is declared on the stack. It will be gone by the time the function has returned. Declare it as a pointer
std::vector<float> *results = new std::vector<float>(10);
but make sure to also declare a function that will free it on the C++ side.
2.The function parameter do not match.
Your C++:
getSomeFloats(float** points, int* count, CameraPose* pose)
Your C#:
getSomeFloats (ref IntPtr ptrResultItems, ref int resultItemsLength);
You either have to remove CameraPose* pose from the C++ side or add IntPtr pose to the C# side.
3. The use of UNITY_INTERFACE_EXPORT UNITY_INTERFACE_API.
You don't need that. This is used when you want to use Unity's built in functions such as GL.IssuePluginEvent. You are not using it in this case.
This should do it:
#define DLLExport __declspec(dllexport)
extern "C"
{
DLLExport void fillArrayNative(float* data, int count, int* outValue)
{
}
}
4.C# has a garbage collector that moves variables around in the memory. You must pin C# array if you want to modify it from C++ side. You only need to pin C# array. Another option is to allocate the array on C++ side, return it to C# copy it to a temporary variable on the C# side then delete it on the C++ side.
5.Copy the result back to the array instead of assigning it.
Recommended method:
There are just many ways to do this and some of them are extremely slow. If you want to use Marshal.Copy, you have to allocate the array on the C++ side or else you will run into some undefined behavior.
The fastest and the most efficient way to do this is to allocate the array on the C# side as a global variable. Pass the array and its length to the native side. Also pass a third parameter which C++ can use to tell C# the amount of index that has been updated or written to.
This is much more better than creating new array, copying it to C# variable then destroying it each time the function is called.
This is what you should be using:
C++:
#define DLLExport __declspec(dllexport)
extern "C"
{
DLLExport void fillArrayNative(float* data, int count, int* outValue)
{
std::vector<float> results;
for (int i = 0; i < count; i++)
{
//Fill the array
data[i] = results[i];
}
*outValue = results.size();
}
}
You can also use: std::copy ( data, data+count, results.begin() ); instead of loop to copy the data too.
C#:
[DllImport("NativePlugin", CallingConvention = CallingConvention.Cdecl)]
private static extern void fillArrayNative(IntPtr data, int count, out int outValue);
public unsafe void getFillArrayNative(float[] outArray, int count, out int outValue)
{
//Pin Memory
fixed (float* p = outArray)
{
fillArrayNative((IntPtr)p, count, out outValue);
}
}
Usage:
const int arraySize = 44500;
float[] arrayToFill = new float[arraySize];
void Start()
{
int length = arrayToFill.Length;
int filledAmount = 0;
getFillArrayNative(arrayToFill, length, out filledAmount);
//You can then loop through it with with the returned filledAmount
for (int i = 0; i < filledAmount; i++)
{
//Do something with arrayToFill[i]
}
}
This is just an example and it is faster than all other methods I've used before. Avoid doing it the way you are currently doing it with Marshal.Copy. If you still want to do it your way or use Marshal.Copy then here is the appropriate way to do it which requires allocation, copying data and de-allocating memory in each call.
The pointer you return in getSomeFloats is owned by results. Before getSomeFloats returns, the vector destructor for results will free that memory. When the C# code tries to use the pointer, you are accessing unallocated memory, which results in Undefined Behavior. In your case most of the data hasn't been changed yet, but some of it has. Potentially any or all of the data could have been changed (if the memory has been reused), or even a program crash (if the freed memory has been returned to the OS).
I have to use a method from a library that has the following signature in the header file:
NKREMOTELIB_API int __stdcall GetEvfFrame(
const unsigned char*& buffer,
size_t& size,
NKRemoteEvfDisplayInfo& displayInfo);
I am calling these from c# using the following:
[DllImport("NKRemoteLib.dll")]
public static extern int GetEvfFrame(out IntPtr buffer, out IntPtr size, out NKRemoteEvfDisplayInfo displayInfo);
private void Test() {
IntPtr size = new IntPtr();
IntPtr buffer = new IntPtr();
NKRemoteEvfDisplayInfo displayInfo;
int res = GetEvfFrame(out buffer, out size, out displayInfo);
}
I have displayInfo defined as a struct in c# as well.
My question is where there is any significance to the ampersand in the function signature in the header file? Why size_t& instead of size_t or char*& instead of char*?
This has nothing to do with C++/C# interop. It's actually about C++.
C++ has the concept of references. When you pass an object by reference, you are actually passing the object itself, and not just a copy. That means any modification to the object in the callee will appear in the caller. C# has a similar concept with the out and ref keywords.
You can test it that way:
void f(int i)
{
i = 5;
}
void g(int &i)
{
i = 5;
}
int i = 0, j = 0;
f(i);
g(j);
std::cout << i; // should print 0
std::cout << j; // should print 5
As to "why" there's an ampersand, well, it's probably because those are out parameters: that is, the caller wants the function to modify these values. This is done to circumvent the "single return value" restriction of C-like languages.
The ampersand denotes pass-by-reference semantics. In other words, the GetEvfFrame function is passed a reference to a size_t value, not the size_t value itself.
Your p/invoke signature should really use ref rather than out.
The ampersand (&) stands for "reference". This is basically the same as your c# out modifier.
I'm following the pinvoke code provided here but am slightly scared by the marshalling of the variable-length array as size=1 and then stepping through it by calculating an offset instead of indexing into an array. Isn't there a better way? And if not, how should I do this to make it safe for 32-bit and 64-bit?
[StructLayout(LayoutKind.Sequential)]
public struct SID_AND_ATTRIBUTES
{
public IntPtr Sid;
public uint Attributes;
}
[StructLayout(LayoutKind.Sequential)]
public struct TOKEN_GROUPS
{
public int GroupCount;
[MarshalAs(UnmanagedType.ByValArray, SizeConst = 1)]
public SID_AND_ATTRIBUTES[] Groups;
};
public void SomeMethod()
{
IntPtr tokenInformation;
// ...
string retVal = string.Empty;
TOKEN_GROUPS groups = (TOKEN_GROUPS)Marshal.PtrToStructure(tokenInformation, typeof(TOKEN_GROUPS));
int sidAndAttrSize = Marshal.SizeOf(new SID_AND_ATTRIBUTES());
for (int i = 0; i < groups.GroupCount; i++)
{
// *** Scary line here:
SID_AND_ATTRIBUTES sidAndAttributes = (SID_AND_ATTRIBUTES)Marshal.PtrToStructure(
new IntPtr(tokenInformation.ToInt64() + i * sidAndAttrSize + IntPtr.Size),
typeof(SID_AND_ATTRIBUTES));
// ...
}
I see here another approach of declaring the length of the array as much bigger than it's likely to be, but that seemed to have its own problems.
As a side question: When I step through the above code in the debugger I'm not able to evaluate tokenInformation.ToInt64() or ToInt32(). I get an ArgumentOutOfRangeException. But the line of code executes just fine!? What's going on here?
Instead of guessing what the offset, is its generally better to use Marshal.OffsetOf(typeof(TOKEN_GROUPS), "Groups") to get the correct offset to the start of the array.
I think it looks okay -- as okay as any poking about in unmanaged land is, anyway.
However, I wonder why the start is tokenInformation.ToInt64() + IntPtr.Size and not tokenInformation.ToInt64() + 4 (as the GroupCount field type is an int and not IntPtr). Is this for packing/alignment of the structure or just something fishy? I do not know here.
Using tokenInformation.ToInt64() is important because on a 64-bit machine will explode (OverflowException) if the IntPtr value is larger than what an int can store. However, the CLR will handle a long just fine on both architectures and it doesn't change the actual value extracted from the IntPtr (and thus put back into the new IntPtr(...)).
Imagine this (untested) function as a convenience wrapper:
// unpacks an array of structures from unmanaged memory
// arr.Length is the number of items to unpack. don't overrun.
void PtrToStructureArray<T>(T[] arr, IntPtr start, int stride) {
long ptr = start.ToInt64();
for (int i = 0; i < arr.Length; i++, ptr += stride) {
arr[i] = (T)Marshal.PtrToStructure(new IntPtr(ptr), typeof(T));
}
}
var attributes = new SID_AND_ATTRIBUTES[groups.GroupCount];
PtrToStructureArray(attributes, new IntPtr(tokenInformation.ToInt64() + IntPtr.Size), sidAndAttrSize);
Happy coding.
From safe, managed code in C#, I would like to call a function in a C API that receives an array of pointers (void**).
I have the corresponding managed array of IntPtr objects, but the Marshal methods advertised in the documentation at MSDN do not seem sufficient to provide an IntPtr to an unmanaged block of memory with the correct content.
I had hoped to obtain an IntPtr with 'Marshal.AllocHGlobal' and then assign the correct content using 'Marshal.Copy', but it seems the function has not been overloaded for an array of IntPtr.
Any thoughts on the best way to do this?
Thanks in advance.
The P/Invoke marshaller already does this, you don't have to help. Just declare the function argument as an array:
[DllImport("blah.dll")]
private static extern void SomeFunction(IntPtr[] array);
Just in case: although you don't have to use the unsafe keyword here, there isn't anything safe about it. The C code can easily corrupt the heap when it writes past the end of the block you allocated.
Pass the array as an IntPtr[], IntPtr are by default marshaled as void*. No
need for unsafe.
[DllImport("containingFoo.dll")]
public static extern void foo( IntPtr[] ptr);
...
// some floats
float[] fpa = {7.2F, 2.3F, 3.3F, 4.5F, 6.5F};
// allocate unmanaged for float[] fpa and int (length of array)
IntPtr fptr = Marshal.AllocHGlobal(fpa.Length *
Marshal.SizeOf(typeof(float)));
IntPtr iptr = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(int)));
// set length of array
Marshal.WriteInt32(iptr, fpa.Length);
// copy the array
Marshal.Copy(fpa, 0, fptr, fpa.Length);
// strore both pointers in IntPtr[]
IntPtr[] pptr = {fptr, iptr};
// call foo passing the IntPtr[] to C
foo(pptr);
//C/C++
// note that stdcall is the default calling convention when using
PInvoke!!!!
void __stdcall foo(void** data)
{
float * fa = (float*)*data; // first element points to float array
int *ip = (int*)data + 1; // pointer to next element in void array
int *pp = (int*)*ip; // get pointer to int
for (int i = 0; i < *pp ; i++)
{
printf("\t: %f\n", *fa++);
}
}