On the unmanaged side I have short* m_pLevels[4], I need to get this array of pointers to the C# side so I can then use each point to copy from the unmanaged side to a managed array.
How do I get an array of short* to C# and would that array be an array of IntPtr ?
I recomend using C++/CLI to communicate between C# (managed) and C++ (native).
In the examples below I used the simplest case where the native data is simply 1 binary buffer.
You can adapt it to pass any form of native data.
You can take one of these 2 approaches:
The most efficient way is to pass the unmanaged data to C# and use it as is.
You will have to use an unsafe methods to handle raw pointers.
It's efficient but more risky.
An example for an unsafe C# method getting a native buffer:
unsafe void HandleNativeData(sbyte* pData, int dataLen)
{
// ...
}
The safest way is to marshal the unmanaged memory to managed one.
If for example you have a C# or C++/CLI method that got a raw pointer from C++ (like in approach 1), you can do the following:
unsafe void HandleNativeData(sbyte* pData, int dataLen)
{
byte[] DataManagedBuf = new byte[dataLen];
Marshal.Copy((IntPtr)pData, DataManagedBuf, 0, dataLen);
// At this point DataManagedBuf is a proper managed buffer,
// containing the data that was passed from native C++.
}
PInvoke is also an option:
// this function pre-supposes that everyone knows there are always
// exactly 4 buffers of data. that's pretty rare for IRL interop.
// normally you'd use a struct with fixed length arrays, but the
// question explicitly states short*[].
[DllImport("UnmanagedCode.dll", EntryPoint = "NameOfUnmanagedCodeFunction"]
private static extern void GetUnmanagedBuffers(IntPtr[] shortPtrs, int[] lengths);
// this function will call the unmanaged function and then
// marshal over the pointers into managed arrays
public List<short[]> GetBuffers()
{
var managedArrays = new List<short[]>(4);
// the unmanaged DLL fills in the buffer addresses for us
var shortPtrs = new IntPtr[4];
// and the lengths of each buffer
var lengths = new int[4];
GetUnmanagedBuffers(shortPtrs, lengths);
// input validation/exception handling omitted for brevity
for (int i = 0; i < 4; i++)
{
var length = bufferLengths[i];
// create the array and add it to the return values
managedArrays.Add(new short[length]);
// the overload of Marshal.Copy that works with int16[]
Marshal.Copy(bufferPtrs[i], //source pointer
managedArrays[i], //destination array
0, //starting index into dest
length) //number of int16 to copy
}
// often you'd see another PInvoke here telling the unmanaged
// code that you're done with the buffers so they can be freed.
return managedArrays;
}
Related
I have a library in C++ containing a library returning a Byte* :
typedef unsigned char Byte;
Byte* RotateImage90(Byte* data, int w, int h);
I'm using this library in a program in C# (Xamarin) :
[DllImport("libCpp", EntryPoint = "RotateImage90")]
public static extern IntPtr rotate90(byte[] data, int w, int h);
Byte[] test(Byte[] data, int w, int h)
{
IntPtr ptr = rotate90(data, w, h);
Byte[] img = ????;// <= function missing
return img;
}
It works good, but I don't know how to convert the pointer to a Byte array. Someone know a function to do that ?
A problem of your function interface is that the memory that the function dynamically allocates to return the rotated image byte array, must be allocated with the same memory allocator that the C# side (or whatever client code) will use to release the memory.
In other words, the module that allocates the memory and the module that frees it must use the same allocator.
When I needed to pass some array data between native code and C# code I successfully used Safe Arrays. On the C++ side, you can use ATL's CComSafeArray to simplify the safe array programming; on the other hand, C# and the CLR understand safe arrays well, so it's easy to get the array data in C# and consume it in managed code.
You can use a function like this to produce a safe array of bytes in C++ (in your case, the safe array will store the rotated image data):
extern "C" HRESULT __stdcall ProduceSafeArrayOfBytes(/* [out] */ SAFEARRAY** ppsa)
{
HRESULT hr = S_OK;
try
{
// Create the safe array to be returned to the caller
CComSafeArray<BYTE> sa( /* Element count */);
// Fill the safe array data.
// You can use a simple sa[i] syntax,
// where 'i' is a 0-based index
...
// Return the safe array to the caller (transfer ownership)
*ppsa = sa.Detach();
}
// Convert exceptions to HRESULT return codes
catch (const CAtlException& e)
{
hr = e;
}
catch (const std::exception& )
{
hr = E_FAIL;
}
return hr;
}
On the C# side, you can use this PInvoke signature:
[DllImport("NativeDll.dll", PreserveSig = false)]
public static extern void ProduceSafeArrayOfBytes(
[Out, MarshalAs(UnmanagedType.SafeArray, SafeArraySubType = VarEnum.VT_UI1)]
out byte[] result
);
The VT_UI1 enum field tells the .NET Marshaller that the safe array contains bytes.
You can get the array data in C# with simple code like this:
byte[] data;
ProduceSafeArrayOfBytes(out data);
As you can see, in your C# code you deal with a simple byte[] array; all the proper data marshalling (including freeing memory) happens automatically under the hood.
You can modify the aforementioned skeleton code, adding the other function parameters, like your image width and height.
As an alternative, another option would be developing a tiny bridging layer using C++/CLI, to convert from raw C-style arrays to .NET managed arrays.
Anyway, the note on your DLL function interface to use a common memory allocator for allocating and releasing the array memory is still valid.
As a third option, if you can modify your DLL function interface, you can require the caller to allocate an array and pass it to the DLL function. The function will write the result data in this caller-allocated array.
This would simplify the memory management, as you give the DLL function a block of memory that is already allocated by the caller. The caller will have the responsibility for both allocating and releasing that memory.
So, your DLL function would look like this:
extern "C" void __cdecl RotateImage90(Byte* result, Byte* data, int width, int height);
The result array is allocated by the caller, who is also responsible to free it.
The function just writes its output in this caller-allocated array.
PInvoke would look like this:
[DllImport("NativeDll.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern void RotateImage90(byte[] result, byte[] data, int width, int height);
I need to pass a buffer containing audio stream from C# to native dll. Buffer resides in struct. It's desirable that buffer will pass via interface & not via disk path. I've seen this method:
// native
struct MyStruct
{
short* buffer
}
void Foo(MyStruct *myStruct);
// managed
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst=1000)]
public short[] buffer;
}
[DllImport("My.dll")]
public static extern void Foo(ref MyStruct myStruct);
First question is if this code is correct for short* buffer of size up to 1K shorts?
Second, size is unkown in advance: do I need to set maximum size in SizeConst (could be several MB)?
First of all, the two structures in your question don't match. The C# structure would match
struct MyStruct
{
short arr[1000];
};
That's what ByValArray means – an array allocated inline in the structure.
If the size is dynamic (that is only known at runtime), then you probably should not expect to get the marshaller to handle this for you. You certainly don't want to force marshalling to a constant sized buffer every time because that will be inefficient. In fact you really want to avoid copying the buffer around at all. And the p/invoke marshaler has an upper limit on the size of objects that it is prepared to marshal.
It is going to be far cleaner and efficient to pin the array manually, and pass its address. And you should also pass the length of the array so that the C++ code knows how much it is expected to read.
On the C++ side:
struct BufferStruct
{
int len;
short* arr;
};
void Foo(const BufferStruct buffer);
On the C# side:
[StructLayout(LayoutKind.Sequential)]
public struct BufferStruct
{
public int len;
public IntPtr arr;
}
[DllImport("My.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern void Foo(BufferStruct buffer);
Then you call the function like this:
short[] arr = ...;
GCHandle gch = GCHandle.Alloc(arr, GCHandleType.Pinned);
try
{
BufferStruct buffer;
buffer.len = buffer.Length;
buffer.arr = gch.AddrOfPinnedObject();
Foo(buffer);
}
finally
{
gch.Free();
}
All of this would be easier still if you did not force the array into a struct. If you passed the length and array as parameters, then the marshaller would pin the array for you and make the code even simpler.
I'm calling the IDiaSourceFile::get_checksum member function, which has the following auto-generated .NET signature:
void get_checksum(uint cbData, out uint pcbData, byte[] pbData);
Note that the parameter pbData is an out parameter. Do I need to worry about pinning the array passed in here when someone calls get_checksum?
(Background:
I inherited some code that looks like this:
unsafe
{
fixed (byte* p = scratch_hash)
{
sourceFile.get_checksum(c, out c, scratch_hash);
}
}
which pins scratch_hash as p and then never actually uses p. I've not seen anything like this before and given the state of the surrounding code I suspect the pin is completely unnecessary here
)
With the current definition:
void get_checksum(uint cbData, out uint pcbData, byte[] pbData);
You don't need to pin pbData. However, you'd need to pre-allocate an array for the returned the data, but you don't know the size in advance. The method will fail if the array size you pass via cbData is not large enough, and your current method signature doesn't allow to find out the buffer size.
The original C++ declaration does allow that:
HRESULT get_checksum (
DWORD cbData,
DWORD* pcbData,
BYTE data[]
);
data [in, out] A buffer that is filled with the checksum bytes. If
this parameter is NULL, then pcbData returns the number of bytes
required.
So, a more efficient approach might be to declare and use it like this:
void get_checksum(
uint cbData,
out uint pcbData,
IntPtr data);
// get size
uint size;
obj.get_checksum(0, out size, IntPtr.Zero);
// get data
var buff = new byte[size];
unsafe
{
fixed (byte* p = buff)
{
uint cbData;
obj.get_checksum(size, out cbData, (IntPtr)p);
if (size != cbData)
throw new InvalidOperationException("cbData");
}
}
If you don't want (or can't) use unsafe code, here's an alternative:
// get size
uint size;
obj.get_checksum(0, out size, IntPtr.Zero);
// get the data
byte[] buff;
var p = Marshal.AllocHGlobal((int)size);
try
{
uint cbData;
obj.get_checksum(size, out cbData, p);
if (size < cbData)
throw new InvalidOperationException("cbData");
buff = new byte[cbData];
Marshal.Copy(p, buff, 0, (int)cbData);
}
finally
{
Marshal.FreeHGlobal(p);
}
No, you don't have to do it in this case - and you can't, even.
However, you might want to change that signature if you run into performance issues, because the marshaller does a lot of work to pass the byte array to the native method - it has to allocate new memory, copy the managed byte array to that, and then deallocate the memory again.
This means that the code you posted is complete nonsense, because while it does fix the scratch_hash array, it doesn't use that fixed pointer anyway.
If you're only calling the method once in a while and the byte array is relatively small, this can be safely ignored. However, if you do find that this causes unnecessary strain on your application, fixing could help - but you'd have to change the signature:
unsafe void get_checksum(uint cbData, out uint pcbData, byte *pbData);
Then you can use this:
fixed (byte *p = &scratch_hash[0])
sourceFile.get_checksum(c, out c, p);
All this said, note that fixing a managed object might cause issues of its own (for example, it pretty much kills any chance of heap compaction if your object is on top of the heap). Fixing should be done sparingly, and only for extremely short times - the only other reasonable alternative is to allocate the objects early, and keeping them together (for example, when using asynchronous I/O, you need a fixed reference, usually for as long as the application runs - so just allocate the buffers as early as possible, and you'll be mostly fine).
Also, I'm still finding that signature a bit wild. Could you share the header file's definition of the signature?
The question in short words is :
How to free memory returned from Native DLL as ItrPtr in managed code?
Details :
Assume we have simple function takes two parameters as OUTPUT, The first one is Reference Pointer to byte array and the second one is Reference Int .
The function will allocate amount of bytes based on some rules and return the pointer of memory and the size of bytes and the return value (1 for success and 0 for fail) .
The code below works fine and I can get the byte array correctly and the count of bytes and the return value, but when I try to free the memory using the pointer (IntPtr) I get exception :
Windows has triggered a breakpoint in TestCppDllCall.exe.
This may be due to a corruption of the heap, which indicates a bug in TestCppDllCall.exe or any of the DLLs it has loaded.
This may also be due to the user pressing F12 while TestCppDllCall.exe has focus.
The output window may have more diagnostic information.
To make things clear :
The next C# code work correctly with other DLL function have the same signature and freeing the memory works without any problem .
Any modification in (C) code accepted if you need to change allocation memory method or adding any other code .
All the functionality I need is Native DLL function accept Two Parameter by reference (Byte array and int , In c# [IntPtr of byte array and int]) fill them with some values based on some rules and return the function result (Success or Fail) .
CppDll.h
#ifdef CPPDLL_EXPORTS
#define CPPDLL_API __declspec(dllexport)
#else
#define CPPDLL_API __declspec(dllimport)
#endif
extern "C" CPPDLL_API int writeToBuffer(unsigned char *&myBuffer, int& mySize);
CppDll.cpp
#include "stdafx.h"
#include "CppDll.h"
extern "C" CPPDLL_API int writeToBuffer(unsigned char*& myBuffer, int& mySize)
{
mySize = 26;
unsigned char* pTemp = new unsigned char[26];
for(int i = 0; i < 26; i++)
{
pTemp[i] = 65 + i;
}
myBuffer = pTemp;
return 1;
}
C# code :
using System;
using System.Text;
using System.Runtime.InteropServices;
namespace TestCppDllCall
{
class Program
{
const string KERNEL32 = #"kernel32.dll";
const string _dllLocation = #"D:\CppDll\Bin\CppDll.dll";
const string funEntryPoint = #"writeToBuffer";
[DllImport(KERNEL32, SetLastError = true)]
public static extern IntPtr GetProcessHeap();
[DllImport(KERNEL32, SetLastError = true)]
public static extern bool HeapFree(IntPtr hHeap, uint dwFlags, IntPtr lpMem);
[DllImport(_dllLocation, EntryPoint = funEntryPoint, CallingConvention = CallingConvention.Cdecl)]
public static extern int writeToBuffer(out IntPtr myBuffer, out int mySize);
static void Main(string[] args)
{
IntPtr byteArrayPointer = IntPtr.Zero;
int arraySize;
try
{
int retValue = writeToBuffer(out byteArrayPointer, out arraySize);
if (retValue == 1 && byteArrayPointer != IntPtr.Zero)
{
byte[] byteArrayBuffer = new byte[arraySize];
Marshal.Copy(byteArrayPointer, byteArrayBuffer, 0, byteArrayBuffer.Length);
string strMyBuffer = Encoding.Default.GetString(byteArrayBuffer);
Console.WriteLine("Return Value : {0}\r\nArray Size : {1}\r\nReturn String : {2}",
retValue, arraySize, strMyBuffer);
}
}
catch (Exception ex)
{
Console.WriteLine("Error calling DLL \r\n {0}", ex.Message);
}
finally
{
if (byteArrayPointer != IntPtr.Zero)
HeapFree(GetProcessHeap(), 0, byteArrayPointer);
}
Console.ReadKey();
}
}
}
When I debug this code i set break point in the line (return 1) and the value of the buffer was :
myBuffer = 0x031b4fc0 "ABCDEFGHIJKLMNOPQRSTUVWXYZ««««««««î"
And I got the same value in C# code when the function call return and the value was :
52121536
The result I Got the correct Memory pointer and i am able to get the byte array value , how to free these memory blocks with this pointer in C# ?
Please let me know if there anything is not clear or if there any typo, I am not native English speaker .
Short answer: you should add a separate method in the DLL that frees the memory for you.
Long answer: there are different ways in which the memory can be allocated inside your DLL implementation. The way you free the memory must match the way in which you have allocated the memory. For example, memory allocated with new[] (with square brackets) needs to be freed with delete[] (as opposed to delete or free). C# does not provide a mechanism for you to do it; you need to send the pointer back to C++.
extern "C" CPPDLL_API void freeBuffer(unsigned char* myBuffer) {
delete[] myBuffer;
}
If you are allocating your own memory in native code, use CoTaskMemAlloc and you can free the pointer in managed code with Marshal.FreeCoTaskMem. CoTaskMemAlloc is described as "the only way to share memory in a COM-based application" (see http://msdn.microsoft.com/en-us/library/windows/desktop/aa366533(v=vs.85).aspx )
if you need to use the memory allocated with CoTaskMemAlloc with a native C++ object, you can use placement new to initialize the memory as if the new operator was used. For example:
void * p = CoTaskMemAlloc(sizeof(MyType));
MyType * pMyType = new (p) MyType;
This doesn't allocate memory with new just calls the constructor on the pre-allocated memory.
Calling Marshal.FreeCoTaskMem does not call the destructor of the type (which isn't needed if you just need to free memory); if you need to do more than free memory by calling the destructor you'll have to provide a native method that does that and P/Invoke it. Passing native class instances to managed code is not supported anyway.
If you need to allocate memory with some other API, you'll need to expose it in managed code through P/Invoke in order to free it in managed code.
HeapFree(GetProcessHeap(), 0, byteArrayPointer);
No, that can't work. The heap handle is wrong, the CRT creates its own heap with HeapCreate(). It's buried in the CRT data, you can't get to it. You could technically find the handle back from GetProcessHeaps(), except you don't know which one it is.
The pointer can be wrong too, the CRT may have added some extra info from the pointer returned by HeapAlloc() to store debugging data.
You'll need to export a function that calls delete[] to release the buffer. Or write a C++/CLI wrapper so you can use delete[] in the wrapper. With the extra requirement that the C++ code and the wrapper use the exact same version of the CRT DLL (/MD required). This almost always requires that you can recompile the C++ code.
I have a C# class library that contains methods that need to be used with an external application. Unfortunately this external application only supports external APIs in C/C++.
Now I've managed to get a very simple COM example working between a C++ dll and a C# DLL, but I am stuck on how I can move around array data.
This is what I've got so far, just as a little example I found on the web of communicating via COM:
DLL_EXPORT(void) runAddTest(int add1,long *result) {
// Initialize COM.
HRESULT hr = CoInitialize(NULL);
// Create the interface pointer.
IUnitModelPtr pIUnit(__uuidof(UnitModel));
long lResult = 0;
// Call the Add method.
pIUnit->Add(5, 10, &lResult);
*result = lResult;
// Uninitialize COM.
CoUninitialize();
}
This works fine to call the add method in my C# class. How can I modify this to take and return an array of doubles? (ill also need to do it with strings down the line).
I need to take an unmanaged array , pass this array to a C# class for some calculations, and then pass it back the results to the array reference specified in the original function call (unmanaged) C++.
I'll need to expose a function like this:
*calcin - reference to array of doubles
*calcOut - reference to array of doubles
numIN - value of size of input array
DLL_EXPORT(void) doCalc(double *calcIn, int numIn, double *calcOut)
{
//pass the calcIn array to C# class for the calcuations
//get the values back from my C# class
//put the values from the C# class
//into the array ref specified by the *calcOut reference
}
I think I can use a C++\CLI DLL for the external application so if this is easier than straight COM then i'll be willing to look at that.
Please be gentle as I am primarily a C# developer but have been thrown in the deep end of Interop and C++ .
I experimented with this a while ago but have unfortunately forgotten how it all fitted together... for my purpose it turned out to be woefully slow so I cut out the C# and went back to all C++. When you say you're primarily a C# developer I hope you understand pointers because if you don't there's no way to be gentle.
Passing arrays basically came down to using CoTaskMemAlloc family of functions on the C++ side (http://msdn.microsoft.com/en-us/library/ms692727(VS.85).aspx) and the Marshal class on the C# side (http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.marshal.aspx - which has methods like AllocCoTaskMem). For C# I ended up with a utility class:
public class serviceUtils
{
unsafe public long stringToCoTaskPtr( ref str thestring )
{
return (long)Marshal.StringToCoTaskMemAnsi(thestring.theString).ToPointer();//TODO : what errors occur from here? handle them
}
unsafe public long bytesToCoTaskPtr( ref bytes thebytes, ref short byteCnt)
{
byteCnt = (short)thebytes.theArray.Length;
IntPtr tmpptr = new IntPtr();
tmpptr = Marshal.AllocCoTaskMem(byteCnt);
Marshal.Copy(thebytes.theArray, 0, tmpptr, byteCnt);
return (long)tmpptr.ToPointer();
}
public void freeCoTaskMemPtr(long ptr)
{
Marshal.FreeCoTaskMem(new IntPtr(ptr));//TODO : errors from here?
}
public string coTaskPtrToString(long theptr)
{
return Marshal.PtrToStringAnsi(new IntPtr(theptr));
}
public byte[] coTaskPtrToBytes(long theptr, short thelen)
{
byte[] tmpbytes = new byte[thelen];
Marshal.Copy(new IntPtr(theptr), tmpbytes, 0, thelen);
return tmpbytes;
}
}
Just to throw some more code at you:
this c++
#import "..\COMClient\bin\Debug\COMClient.tlb" named_guids raw_interfaces_only
int _tmain(int argc, _TCHAR* argv[])
{
CoInitialize(NULL); //Initialize all COM Components
COMClient::IComCalculatorPtr pCalc;
// CreateInstance parameters
HRESULT hRes = pCalc.CreateInstance(COMClient::CLSID_ComCalculator);
if (hRes == S_OK) {
long size = 5;
LPVOID ptr = CoTaskMemAlloc( size );
if( ptr != NULL )
{
memcpy( ptr, "12345", size );
short ans = 0;
pCalc->changeBytes( (__int64*)&ptr, &size, &ans );
CoTaskMemFree(ptr);
}
}
CoUninitialize (); //DeInitialize all COM Components
return 0;
}
called this c#
public short changeBytes(ref long ptr, ref int arraysize)
{
try
{
IntPtr interopPtr = new IntPtr(ptr);
testservice.ByteArray bytes = new testservice.ByteArray();
byte[] somebytes = new byte[arraysize];
Marshal.Copy(interopPtr, somebytes, 0, arraysize);
bytes.theArray = somebytes;
CalculatorClient client = generateClient();
client.takeArray(ref bytes);
client.Close();
if (arraysize < bytes.theArray.Length)
{
interopPtr = Marshal.ReAllocCoTaskMem(interopPtr, bytes.theArray.Length);//TODO : throws an exception if fails... deal with it
}
Marshal.Copy(bytes.theArray, 0, interopPtr, bytes.theArray.Length);
ptr = interopPtr.ToInt64();
arraysize = bytes.theArray.Length;
//TODO : do we need to free IntPtr? check all code for memory leaks... check for successful allocation
}
catch(Exception e)
{
return 3;
}
return 2;
}
Sorry, but I don't have the time to work all this out and explain it properly, hopefully this will give you pointers in the right direction, at the very least some things to google. Good Luck
PS : I got all the info to write this stuff off the net, so it is out there.
I think I can use a C++\CLI DLL for the external application so if this is easier than straight COM then i'll be willing to look at that.
If you don't have much COM experience (and arrays are significantly not simple in COM) then C++/CLI wrapper around the 3rd party will likely be easier.
It will also only involve a single marshalling stage (native <-> managed) rather than the extra step the necessary COM Callable Wrapper you will need for managed <-> COM interfacing).
Would this also work?
In C#,
1. Call Marshal.PtrToStructure
2. Modify the value
3. Call Marshal.StructureToPtr