I'm calling the IDiaSourceFile::get_checksum member function, which has the following auto-generated .NET signature:
void get_checksum(uint cbData, out uint pcbData, byte[] pbData);
Note that the parameter pbData is an out parameter. Do I need to worry about pinning the array passed in here when someone calls get_checksum?
(Background:
I inherited some code that looks like this:
unsafe
{
fixed (byte* p = scratch_hash)
{
sourceFile.get_checksum(c, out c, scratch_hash);
}
}
which pins scratch_hash as p and then never actually uses p. I've not seen anything like this before and given the state of the surrounding code I suspect the pin is completely unnecessary here
)
With the current definition:
void get_checksum(uint cbData, out uint pcbData, byte[] pbData);
You don't need to pin pbData. However, you'd need to pre-allocate an array for the returned the data, but you don't know the size in advance. The method will fail if the array size you pass via cbData is not large enough, and your current method signature doesn't allow to find out the buffer size.
The original C++ declaration does allow that:
HRESULT get_checksum (
DWORD cbData,
DWORD* pcbData,
BYTE data[]
);
data [in, out] A buffer that is filled with the checksum bytes. If
this parameter is NULL, then pcbData returns the number of bytes
required.
So, a more efficient approach might be to declare and use it like this:
void get_checksum(
uint cbData,
out uint pcbData,
IntPtr data);
// get size
uint size;
obj.get_checksum(0, out size, IntPtr.Zero);
// get data
var buff = new byte[size];
unsafe
{
fixed (byte* p = buff)
{
uint cbData;
obj.get_checksum(size, out cbData, (IntPtr)p);
if (size != cbData)
throw new InvalidOperationException("cbData");
}
}
If you don't want (or can't) use unsafe code, here's an alternative:
// get size
uint size;
obj.get_checksum(0, out size, IntPtr.Zero);
// get the data
byte[] buff;
var p = Marshal.AllocHGlobal((int)size);
try
{
uint cbData;
obj.get_checksum(size, out cbData, p);
if (size < cbData)
throw new InvalidOperationException("cbData");
buff = new byte[cbData];
Marshal.Copy(p, buff, 0, (int)cbData);
}
finally
{
Marshal.FreeHGlobal(p);
}
No, you don't have to do it in this case - and you can't, even.
However, you might want to change that signature if you run into performance issues, because the marshaller does a lot of work to pass the byte array to the native method - it has to allocate new memory, copy the managed byte array to that, and then deallocate the memory again.
This means that the code you posted is complete nonsense, because while it does fix the scratch_hash array, it doesn't use that fixed pointer anyway.
If you're only calling the method once in a while and the byte array is relatively small, this can be safely ignored. However, if you do find that this causes unnecessary strain on your application, fixing could help - but you'd have to change the signature:
unsafe void get_checksum(uint cbData, out uint pcbData, byte *pbData);
Then you can use this:
fixed (byte *p = &scratch_hash[0])
sourceFile.get_checksum(c, out c, p);
All this said, note that fixing a managed object might cause issues of its own (for example, it pretty much kills any chance of heap compaction if your object is on top of the heap). Fixing should be done sparingly, and only for extremely short times - the only other reasonable alternative is to allocate the objects early, and keeping them together (for example, when using asynchronous I/O, you need a fixed reference, usually for as long as the application runs - so just allocate the buffers as early as possible, and you'll be mostly fine).
Also, I'm still finding that signature a bit wild. Could you share the header file's definition of the signature?
Related
On the unmanaged side I have short* m_pLevels[4], I need to get this array of pointers to the C# side so I can then use each point to copy from the unmanaged side to a managed array.
How do I get an array of short* to C# and would that array be an array of IntPtr ?
I recomend using C++/CLI to communicate between C# (managed) and C++ (native).
In the examples below I used the simplest case where the native data is simply 1 binary buffer.
You can adapt it to pass any form of native data.
You can take one of these 2 approaches:
The most efficient way is to pass the unmanaged data to C# and use it as is.
You will have to use an unsafe methods to handle raw pointers.
It's efficient but more risky.
An example for an unsafe C# method getting a native buffer:
unsafe void HandleNativeData(sbyte* pData, int dataLen)
{
// ...
}
The safest way is to marshal the unmanaged memory to managed one.
If for example you have a C# or C++/CLI method that got a raw pointer from C++ (like in approach 1), you can do the following:
unsafe void HandleNativeData(sbyte* pData, int dataLen)
{
byte[] DataManagedBuf = new byte[dataLen];
Marshal.Copy((IntPtr)pData, DataManagedBuf, 0, dataLen);
// At this point DataManagedBuf is a proper managed buffer,
// containing the data that was passed from native C++.
}
PInvoke is also an option:
// this function pre-supposes that everyone knows there are always
// exactly 4 buffers of data. that's pretty rare for IRL interop.
// normally you'd use a struct with fixed length arrays, but the
// question explicitly states short*[].
[DllImport("UnmanagedCode.dll", EntryPoint = "NameOfUnmanagedCodeFunction"]
private static extern void GetUnmanagedBuffers(IntPtr[] shortPtrs, int[] lengths);
// this function will call the unmanaged function and then
// marshal over the pointers into managed arrays
public List<short[]> GetBuffers()
{
var managedArrays = new List<short[]>(4);
// the unmanaged DLL fills in the buffer addresses for us
var shortPtrs = new IntPtr[4];
// and the lengths of each buffer
var lengths = new int[4];
GetUnmanagedBuffers(shortPtrs, lengths);
// input validation/exception handling omitted for brevity
for (int i = 0; i < 4; i++)
{
var length = bufferLengths[i];
// create the array and add it to the return values
managedArrays.Add(new short[length]);
// the overload of Marshal.Copy that works with int16[]
Marshal.Copy(bufferPtrs[i], //source pointer
managedArrays[i], //destination array
0, //starting index into dest
length) //number of int16 to copy
}
// often you'd see another PInvoke here telling the unmanaged
// code that you're done with the buffers so they can be freed.
return managedArrays;
}
I have a library in C++ containing a library returning a Byte* :
typedef unsigned char Byte;
Byte* RotateImage90(Byte* data, int w, int h);
I'm using this library in a program in C# (Xamarin) :
[DllImport("libCpp", EntryPoint = "RotateImage90")]
public static extern IntPtr rotate90(byte[] data, int w, int h);
Byte[] test(Byte[] data, int w, int h)
{
IntPtr ptr = rotate90(data, w, h);
Byte[] img = ????;// <= function missing
return img;
}
It works good, but I don't know how to convert the pointer to a Byte array. Someone know a function to do that ?
A problem of your function interface is that the memory that the function dynamically allocates to return the rotated image byte array, must be allocated with the same memory allocator that the C# side (or whatever client code) will use to release the memory.
In other words, the module that allocates the memory and the module that frees it must use the same allocator.
When I needed to pass some array data between native code and C# code I successfully used Safe Arrays. On the C++ side, you can use ATL's CComSafeArray to simplify the safe array programming; on the other hand, C# and the CLR understand safe arrays well, so it's easy to get the array data in C# and consume it in managed code.
You can use a function like this to produce a safe array of bytes in C++ (in your case, the safe array will store the rotated image data):
extern "C" HRESULT __stdcall ProduceSafeArrayOfBytes(/* [out] */ SAFEARRAY** ppsa)
{
HRESULT hr = S_OK;
try
{
// Create the safe array to be returned to the caller
CComSafeArray<BYTE> sa( /* Element count */);
// Fill the safe array data.
// You can use a simple sa[i] syntax,
// where 'i' is a 0-based index
...
// Return the safe array to the caller (transfer ownership)
*ppsa = sa.Detach();
}
// Convert exceptions to HRESULT return codes
catch (const CAtlException& e)
{
hr = e;
}
catch (const std::exception& )
{
hr = E_FAIL;
}
return hr;
}
On the C# side, you can use this PInvoke signature:
[DllImport("NativeDll.dll", PreserveSig = false)]
public static extern void ProduceSafeArrayOfBytes(
[Out, MarshalAs(UnmanagedType.SafeArray, SafeArraySubType = VarEnum.VT_UI1)]
out byte[] result
);
The VT_UI1 enum field tells the .NET Marshaller that the safe array contains bytes.
You can get the array data in C# with simple code like this:
byte[] data;
ProduceSafeArrayOfBytes(out data);
As you can see, in your C# code you deal with a simple byte[] array; all the proper data marshalling (including freeing memory) happens automatically under the hood.
You can modify the aforementioned skeleton code, adding the other function parameters, like your image width and height.
As an alternative, another option would be developing a tiny bridging layer using C++/CLI, to convert from raw C-style arrays to .NET managed arrays.
Anyway, the note on your DLL function interface to use a common memory allocator for allocating and releasing the array memory is still valid.
As a third option, if you can modify your DLL function interface, you can require the caller to allocate an array and pass it to the DLL function. The function will write the result data in this caller-allocated array.
This would simplify the memory management, as you give the DLL function a block of memory that is already allocated by the caller. The caller will have the responsibility for both allocating and releasing that memory.
So, your DLL function would look like this:
extern "C" void __cdecl RotateImage90(Byte* result, Byte* data, int width, int height);
The result array is allocated by the caller, who is also responsible to free it.
The function just writes its output in this caller-allocated array.
PInvoke would look like this:
[DllImport("NativeDll.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern void RotateImage90(byte[] result, byte[] data, int width, int height);
I need to pass a buffer containing audio stream from C# to native dll. Buffer resides in struct. It's desirable that buffer will pass via interface & not via disk path. I've seen this method:
// native
struct MyStruct
{
short* buffer
}
void Foo(MyStruct *myStruct);
// managed
[StructLayout(LayoutKind.Sequential)]
public struct MyStruct
{
[MarshalAs(UnmanagedType.ByValArray, SizeConst=1000)]
public short[] buffer;
}
[DllImport("My.dll")]
public static extern void Foo(ref MyStruct myStruct);
First question is if this code is correct for short* buffer of size up to 1K shorts?
Second, size is unkown in advance: do I need to set maximum size in SizeConst (could be several MB)?
First of all, the two structures in your question don't match. The C# structure would match
struct MyStruct
{
short arr[1000];
};
That's what ByValArray means – an array allocated inline in the structure.
If the size is dynamic (that is only known at runtime), then you probably should not expect to get the marshaller to handle this for you. You certainly don't want to force marshalling to a constant sized buffer every time because that will be inefficient. In fact you really want to avoid copying the buffer around at all. And the p/invoke marshaler has an upper limit on the size of objects that it is prepared to marshal.
It is going to be far cleaner and efficient to pin the array manually, and pass its address. And you should also pass the length of the array so that the C++ code knows how much it is expected to read.
On the C++ side:
struct BufferStruct
{
int len;
short* arr;
};
void Foo(const BufferStruct buffer);
On the C# side:
[StructLayout(LayoutKind.Sequential)]
public struct BufferStruct
{
public int len;
public IntPtr arr;
}
[DllImport("My.dll", CallingConvention = CallingConvention.Cdecl)]
public static extern void Foo(BufferStruct buffer);
Then you call the function like this:
short[] arr = ...;
GCHandle gch = GCHandle.Alloc(arr, GCHandleType.Pinned);
try
{
BufferStruct buffer;
buffer.len = buffer.Length;
buffer.arr = gch.AddrOfPinnedObject();
Foo(buffer);
}
finally
{
gch.Free();
}
All of this would be easier still if you did not force the array into a struct. If you passed the length and array as parameters, then the marshaller would pin the array for you and make the code even simpler.
I am wrapping a c++ dll which does high quality sample rate conversion in c# and I am not sure what kind of type I should use for the op0 parameter. A C++ wrapper would call it like this:
int _cdecl process(double* const ip0, int l, double*& op0)
The documentation says about the parameter:
"#param[out] op0 This variable receives the pointer to the resampled data.
This pointer may point to the address within the "ip0" input buffer, or to
*this object's internal buffer. In real-time applications it is suggested
to pass this pointer to the next output audio block and consume any data
left from the previous output audio block first before calling the
process() function again. The buffer pointed to by the "op0" on return may
be owned by the resampler, so it should not be freed by the caller."
What I would like to do is the following:
[DllImport("r8bsrc.dll", EntryPoint="process", CallingConvention = CallingConvention.Cdecl)]
public static extern int Process([in] double[] ip0,
int length,
[out] double[] op0);
But I am pretty sure this would not work, since the marshaller cannot know how big the memory behind op1 is, or am I wrong?
So I guess I have to copy the values behind op1 back to a managed array myself. Maybe:
[DllImport("r8bsrc.dll", EntryPoint="process", CallingConvention = CallingConvention.Cdecl)]
public static extern int Process([in] double[] ip0,
int length,
out IntPtr op0); //or do i need out double* ?
And then wrap it again with:
private IntPtr FOutBufferPtr; //reuse it as recommeded
public int Process(double[] input, out double[] output)
{
var outSamples = R8BrainDLLWrapper.Process(input, input.Length, out FOutBufferPtr);
output = new double[outSamples];
Marshal.Copy(FOutBufferPtr, output, 0, outSamples);
}
What is the optimal way which involves the least number of copies?
EDIT2:
This is the current code, it works perfectly:
public int Process(double[] input, ref double[] output)
{
//pin the input during process
var pinnedHandle = GCHandle.Alloc(input, GCHandleType.Pinned);
//resample
var outSamples = R8BrainDLLWrapper.Process(FUnmanagedInstance, pinnedHandle.AddrOfPinnedObject(), input.Length, out FOutBufferPtr);
//copy to output array
if(output.Length < outSamples)
output = new double[outSamples];
Marshal.Copy(FOutBufferPtr, output, 0, outSamples);
//free pin
pinnedHandle.Free();
return outSamples;
}
The signature is now:
[DllImport("r8bsrc.dll", EntryPoint="r8b_process", CallingConvention = CallingConvention.Cdecl)]
public static extern int Process(IntPtr instance,
IntPtr ip0,
int length,
out IntPtr op0);
#param[out] op0
This variable receives the pointer to the resampled data. This pointer may point to the address within the "ip0" input buffer, or to *this object's internal buffer. In real-time applications it is suggested to pass this pointer to the next output audio block and consume any data left from the previous output audio block first before calling the process() function again. The buffer pointed to by the "op0" on return may be owned by the resampler, so it should not be freed by the caller.
This immediately presents a constraint on ip0. You must arrange that the buffer that ip0 points to is stable beyond the end of the call to the function. That implies that you must pin it before calling the function. Which in turn implies that it must be declared as IntPtr.
For op0, this points to either memory owned by the resampler, or to a location within the ip0 input buffer. So, again you are going to have to use an IntPtr, this time an out parameter.
So, the declaration must be:
[DllImport("r8bsrc.dll", EntryPoint="process",
CallingConvention = CallingConvention.Cdecl)]
public static extern int Process(IntPtr ip0, int length, out IntPtr op0);
And as discussed above, the pointer you pass in ip0 must be obtained using the GCHandle class so that you can pin the array.
I've got a C# program that needs to pass a char buffer to an unmanaged function. I've found two ways that seem to work reliably, but I'm not sure which I should choose.
Here's the unmanaged function's signature.
extern "C" __declspec(dllexport) int getNextResponse(char *buffer);
The first option is to define the buffer as a StringBuilder, as follows.
//at class level...
[DllImport("mydll.dll")]
static extern int getNextResponse(StringBuilder buffer);
//in main method body...
StringBuilder sb = new StringBuilder(" ", 65536);
int rc = getNextResponse(sb);
This is simple, and it works, and I think I basically understand why it works because the StringBuilder has a buffer behind the scenes, so (I assume) the interop layer is just marshalling the StringBuilder to a char *.
The other option is using unsafe code.
//at class level...
[DllImport("mydll.dll")]
static extern int getNextResponse(byte* buffer);
//separate method...
private static unsafe int runGetNextResponse(byte[] buffer)
{
fixed (byte* p = buffer)
{
int rc = getNextResponse(p);
return rc;
}
}
//in main method body...
byte[] b = new byte[65536];
int rc = runGetNextResponse(b);
The second approach is more code, but it's also more explicit about what's going on.
Are these two approaches doing basically the same thing? Is there any reason to choose one over the other?
I'd strongly prefer using the StringBuilder version.
There's not going to be a huge difference between the two, and using unsafe code is not nearly as clean.
In my opinion, since there is a way to solve the problem using a core library class, using unsafe code without a clear (and needed) benefit is a premature optimization.
While using a StringBuilder is preferred there's one caveat. Imagine for example that in your getNextResponse method you store the pointer to some static variable and use it in another method:
char* globalPointer;
int getNextResponse(char *buffer) {
globalPointer = buffer;
return 0;
}
void someOtherMethod() {
printf("%s\n", globalPointer);
}
Now let's look at the managed side:
var sb = new StringBuilder();
sb.Append("Hello World");
int result = getNextResponse(sb);
Console.WriteLine(result);
someOtherMethod(); // kaboom: The GC could have already destroyed the string builder.
The unsafe method guarantees you that the memory location won't be moved around:
byte[] buffer = Encoding.UTF8.GetBytes("Hello World");
fixed (byte* p = buffer)
{
int result = getNextResponse(p);
Console.WriteLine(result);
someOtherMethod(); // works fine as the buffer address is pinned down in memory
}
In this case the unsafe version will work better.
While I can't weigh in definitively, I can share my own experiences. I have used the StringBuilder method exclusively and have had no problems with it. I like the simpler code of it and the avoidance of unsafe.
It depends on the cost of marshalling. If you do a lot of marshalling or the data being marshalled is large, you may want to reuse the buffer instead of creating/destroying the string builder buffer every time.
It depends on what the data in the buffer actually is. If it is character data use the StringBuilder approach. If it is binary data use a byte array instead, but don't use the unsafe method. While the exaggerated fear of 'unsafe' that is getting common is a little silly and there is no reason not to use it when warranted, in this case it is unnecessary. Use:
//at class level...
[DllImport("mydll.dll")]
static extern int getNextResponse([In, Out] byte[] buffer);
//in main method body...
byte[] buffer = new byte[65536];
int rc = getNextResponse(buffer);