We Keep Coding

Do I need to call Marshal.DestroyStructure after copying a struct to a byte[]?

Actually I have many questions, so I try to describe them in detail, thanks for your patience.
I want to write a some generic methods for wrapping native APIs, just like ReadProcessMemory and WriteProcessMemory.
Take WriteProcessMemory for example, I found some ways to do them:
1.GCHandle
// Write memory: Non-Array
var lengthInBytes = (IntPtr)Marshal.SizeOf<T>();
var tHandle = GCHandle.Alloc(data, GCHandleType.Pinned);
try
{
// Call native api;
}
finally
{
tHandle.Free();
}
// Write memory: Array
var lengthInBytes = (IntPtr)(data.Length * Marshal.SizeOf<T>());
var tHandle = GCHandle.Alloc(data, GCHandleType.Pinned);
try
{
// Call native api;
}
finally
{
tHandle.Free();
}
This way is very simple, but the only problem is the object that to be pinned must be a built-in value type ( except fixed array, but the type of the element in array is subject to the same requirements ), . So I found another way: Marshal.AllocHGlobal method.
2. The members in Marshal class
//Write memory: Non-Array
var lengthInBytes = Marshal.SizeOf<T>();
var memPtr = Marshal.AllocHGlobal(lengthInBytes);
try
{
Marshal.StructureToPtr<T>(data, memPtr, false);
// Call native API
}
finally
{
Marshal.DestroyStructure<T>(memPtr);
Marshal.FreeHGlobal(memPtr);
}
// Write memory: Array
var perSize = Marshal.SizeOf<T>();
var arrayLength = data.Length;
var lengthInBytes = arrayLength * perSize;
var memPtr = Marshal.AllocHGlobal(lengthInBytes);
var baseAdr = memPtr;
try
{
foreach (var item in data)
{
Marshal.StructureToPtr<T>(item, baseAdr, false);
baseAdr += perSize;
}
// call native API.
}
finally
{
for (int i = 0; i < arrayLength; i++, baseAdr -= perSize)
{
Marshal.DestroyStructure<T>(baseAdr);
}
Marshal.FreeHGlobal(memPtr);
}
This way will alloc some memory, copy data, and even traverse array twice, But it supports the reference types, and I was satisfied with it, But then I got a idea: how about using Byte[] instead of IntPtr?
Specifically,The extern method is:
[DllImport("Kernel32.dll", SetLastError = true, CallingConvention = CallingConvention.Winapi)]
[return: MarshalAs(UnmanagedType.Bool)]
public static extern Boolean WriteProcessMemory([In] HandleRef hProcess, [In]IntPtr lpBaseAddress, [In, MarshalAs(UnmanagedType.LPArray)] Byte[] lpBuffer, [In, MarshalAs(UnmanagedType.SysInt)]IntPtr nSize, [Out, Optional, MarshalAs(UnmanagedType.SysInt)] out IntPtr lpNumberOfBytesWritten);
And the code in method is:
var perSize = Marshal.SizeOf<T>();
var buffer = new Byte[perSize];
var handle = GCHandle.Alloc(buffer, GCHandleType.Pinned);
var pBuffer = handle.AddrOfPinnedObject();
try
{
Marshal.StructureToPtr<T>(data, pBuffer, false);
//Call native API.
}
finally
{
// Question!!!!!!!!!!!!!!
Marshal.DestroyStructure<T>(pBuffer);
handle.Free();
}
So there is a question: do I need to call Marshal.DestroyStructure method?
MSDN says
StructureToPtr(T, IntPtr, Boolean) copies the contents of structure
to the pre-allocated block of memory that the ptr parameter points to.
If structure contains reference types that marshal to COM interface
pointers (interfaces, classes without layout, and System.Object), the
managed objects are kept alive with reference counts. All other
reference types (for example, strings and arrays) are marshaled to
copies. To release these managed or unmanaged objects, you must call
the DestroyStructure(IntPtr) method before you free the memory
block.
And this method is "Marshals data from a managed object of a specified type to an unmanaged block of memory."
But variable buffer is a Byte[](In managed memory), so if I wont call DestroyStructure, can GC release the reference types in T? ( By the way, in my test, this way is much faster than second way, even though Marshal.DestroyStructure method has also been called too... )
And more questions is, will there be more ways to get the pointer of generic T? And if there are any mistake in the solution above?

C# pass pointer to struct (containing non-blittable types) to unmanaged C++ DLL

I try to use a C++ DLL in my C# code.
All data that is read must be passed to the dll as a pointer to a struct. My first idea was to just reserve some unmanaged memory, pass the pointer to the function and extract the data afterwards. The problem is that the functions will only returns an error code that translates to "An argument is not valid".
From the DLLs Header (Result and BusPortId shortened)
typedef enum {
BUS_COM1 = 0x01, // 2 to 15 ommited
BUS_COM16 = 0x10,
BUS_USB1 = 0x101, // 2 to 15 ommited
BUS_USB16 = 0x110
} BusPortId;
typedef enum {
BUS_SUCCESS = 0, //!< The operation was completed successfully
BUS_ERROR = 0x100, //!< An error occured
BUS_INVALIDARG = 0x1000, //!< An argument is not valid
} Result
struct BusPortInfo
{
ULONG portInfoSize;
CHAR portText[64];
BOOL portLocked;
BusPortId portId;
};
Result BUSDRV_API busGetAvailablePortCount( ULONG *retCount );
Result BUSDRV_API busGetAvailablePort( ULONG index, BusPortInfo *portInfo );
The relevant parts of my C# program so far
enum BusPortId
{
BUS_COM1 = 0x01, // 2 to 15 ommited
BUS_COM16 = 0x10,
BUS_USB1 = 0x101, // 2 to 15 ommited
BUS_USB16 = 0x110
};
public enum Result
{
BUS_SUCCESS = 0, //!< The operation was completed successfully
BUS_ERROR = 0x100, //!< An error occured
BUS_INVALIDARG = 0x1000, //!< An argument is not valid
};
struct BusPortInfo
{
public ULONG portInfoSize;
unsafe public fixed char portText[64];
public BOOL portLocked;
public BusPortId portId;
}
[DllImport(DLL_Path)]
unsafe static extern Result busGetAvailablePortCount(ULONG* retCount);
[DllImport(DLL_Path)]
unsafe static extern Result busGetAvailablePort(ULONG index, BusPortInfo* portInfo);
ulong count= 0;
Result res = busGetAvailablePortCount(&count);
ulong index = 0;
BusPortInfo info = new BusPortInfo();
Result res = busGetAvailablePort(0, &info);
The call to busGetAvailablePortCount (and other similar functions) works without any problems. But when I call busGetAvailablePort I get the following in my console output
Cannot marshal 'parameter #2': Pointers cannot reference marshaled structures. Use ByRef instead.
The Problem is that I can change my struct in C# so that I can pass the pointer, but then the function from the DLL also returns "An argument is not valid"
What do I have to do to my struct so I can pass a pointer to it to a function while still getting accepted by the DLL?
P.S. Sorry for the bad english, I'm not a native speaker.

There are a lot of problems with these declarations, tends to happen when the programmer keeps hacking the code to try to make the pinvoke call work. The most likely correct declaration for the structure is:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Ansi)]
struct BusPortInfo {
public int portInfoSize;
[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 64)]
public string portText;
public bool portLocked;
public BusPortId portId;
}
Emphasizing that an ULONG in native code is a 32-bit type and fixed just isn't necessary and is pretty awkward. This struct is not blittable due to the bool and string member, nothing much to fret about.
The [DllImport] declaration needs to declare the 2nd argument correctly. The CallingConvention property always matters a great deal and we can't see what BUSDRV_API means. Punting:
[DllImport(DLL_Path, CallingConvention = CallingConvention.StdCall)]
static extern Result busGetAvailablePortCount(out int retCount );
[DllImport(DLL_Path, CallingConvention = CallingConvention.StdCall)]
static extern Result busGetAvailablePort(int index,
[In, Out] ref BusPortInfo portInfo);
And the call does not look correct. When a struct has a "size" member then the api contract normally demands that it is set before the call. It is a safety measure, it ensures that the api cannot corrupt memory when the caller uses the wrong structure declaration. And when set wrong, an "Invalid argument" error is the expected outcome. So write it similar to:
int count;
Result res = busGetAvailablePortCount(out count);
if (res != Result.BUS_SUCCESS) throw new Exception("Api failure " + res.ToString());
for (int ix = 0; ix < count; ++ix) {
BusPortInfo info;
info.portInfoSize = Marshal.SizeOf(typeof(BusPortInfo)); // Important!
res = busGetAvailablePort(ix, ref info);
if (res != Result.BUS_SUCCESS) throw new Exception("Api failure " + res.ToString());
// etc...
}
Not tested of course, should be in the ballpark. If you still have problems then verify that sizeof(BusPortInfo) in native code matches the value of Marshal.SizeOf(typeof(BusPortInfo)) in C#. And if all fails then use C++/CLI instead so you can use the native declarations directly. And talk to the owner of the DLL for proper usage instructions, preferably he'll write a pinvoke sample for you.

What is the correct P/Invoke signature of double*& which points to an unmanaged array?

I am wrapping a c++ dll which does high quality sample rate conversion in c# and I am not sure what kind of type I should use for the op0 parameter. A C++ wrapper would call it like this:
int _cdecl process(double* const ip0, int l, double*& op0)
The documentation says about the parameter:
"#param[out] op0 This variable receives the pointer to the resampled data.
This pointer may point to the address within the "ip0" input buffer, or to
*this object's internal buffer. In real-time applications it is suggested
to pass this pointer to the next output audio block and consume any data
left from the previous output audio block first before calling the
process() function again. The buffer pointed to by the "op0" on return may
be owned by the resampler, so it should not be freed by the caller."
What I would like to do is the following:
[DllImport("r8bsrc.dll", EntryPoint="process", CallingConvention = CallingConvention.Cdecl)]
public static extern int Process([in] double[] ip0,
int length,
[out] double[] op0);
But I am pretty sure this would not work, since the marshaller cannot know how big the memory behind op1 is, or am I wrong?
So I guess I have to copy the values behind op1 back to a managed array myself. Maybe:
[DllImport("r8bsrc.dll", EntryPoint="process", CallingConvention = CallingConvention.Cdecl)]
public static extern int Process([in] double[] ip0,
int length,
out IntPtr op0); //or do i need out double* ?
And then wrap it again with:
private IntPtr FOutBufferPtr; //reuse it as recommeded
public int Process(double[] input, out double[] output)
{
var outSamples = R8BrainDLLWrapper.Process(input, input.Length, out FOutBufferPtr);
output = new double[outSamples];
Marshal.Copy(FOutBufferPtr, output, 0, outSamples);
}
What is the optimal way which involves the least number of copies?
EDIT2:
This is the current code, it works perfectly:
public int Process(double[] input, ref double[] output)
{
//pin the input during process
var pinnedHandle = GCHandle.Alloc(input, GCHandleType.Pinned);
//resample
var outSamples = R8BrainDLLWrapper.Process(FUnmanagedInstance, pinnedHandle.AddrOfPinnedObject(), input.Length, out FOutBufferPtr);
//copy to output array
if(output.Length < outSamples)
output = new double[outSamples];
Marshal.Copy(FOutBufferPtr, output, 0, outSamples);
//free pin
pinnedHandle.Free();
return outSamples;
}
The signature is now:
[DllImport("r8bsrc.dll", EntryPoint="r8b_process", CallingConvention = CallingConvention.Cdecl)]
public static extern int Process(IntPtr instance,
IntPtr ip0,
int length,
out IntPtr op0);

#param[out] op0
This variable receives the pointer to the resampled data. This pointer may point to the address within the "ip0" input buffer, or to *this object's internal buffer. In real-time applications it is suggested to pass this pointer to the next output audio block and consume any data left from the previous output audio block first before calling the process() function again. The buffer pointed to by the "op0" on return may be owned by the resampler, so it should not be freed by the caller.
This immediately presents a constraint on ip0. You must arrange that the buffer that ip0 points to is stable beyond the end of the call to the function. That implies that you must pin it before calling the function. Which in turn implies that it must be declared as IntPtr.
For op0, this points to either memory owned by the resampler, or to a location within the ip0 input buffer. So, again you are going to have to use an IntPtr, this time an out parameter.
So, the declaration must be:
[DllImport("r8bsrc.dll", EntryPoint="process",
CallingConvention = CallingConvention.Cdecl)]
public static extern int Process(IntPtr ip0, int length, out IntPtr op0);
And as discussed above, the pointer you pass in ip0 must be obtained using the GCHandle class so that you can pin the array.

How do I marshal a pointer to an array of pointers to structures from managed to unmanaged code?

I'm trying to write plugin dll which has to implement the following function:
int GetFunctionTable(FuncDescStruct **ppFunctionTable);
So my plugin code in C# declares this:
public static unsafe int GetFunctionTable(IntPtr functionTablePtr);
This function will be called and expected to fill functionTablePtr with pointer to array of structs describing set of callback functions.
In plain C/C++ it looks something like:
// declare func table
// VExampleF1, VExampleF2 - are function pointers
FuncDescStruct funcTable[] = {
"ExampleF1", { VExampleF1, 0, 0, 0, 0, NULL }, //filling descriptions.
"ExampleF2", { VExampleF2, 1, 0, 1, 0, NULL }
};
int GetFunctionTable(FuncDescStruct **ppFunctionTable)
{
*ppFunctionTable = funcTable; // how to do this correctly in C#?
// must return the number of functions in the table
return funcTableSize;
}
I'm trying to do the following:
static unsafe FunctionTag[] funcTable;
static List<IntPtr> allocatedMemory;
public static unsafe int GetFunctionTable(IntPtr functionTablePtr)
{
//create just one test callback description
funcTable = new FunctionTag[1];
funcTable[0].Name = "VExampleF1";
funcTable[0].Description.Function = VExampleF1;
funcTable[0].Description.ArrayQty = 0;
funcTable[0].Description.FloatQty = 0;
funcTable[0].Description.StringQty = 0;
funcTable[0].Description.DefaultQty = 0;
funcTable[0].Description.DefaultValues = null;
// saving allocated memory for further cleanup
allocatedMemory = new List<IntPtr>();
int intPtrSize = Marshal.SizeOf(typeof(IntPtr));
IntPtr nativeArray = Marshal.AllocHGlobal(intPtrSize * funcTable.Length);
for (int i = 0; i < funcTable.Length; i++)
{
IntPtr nativeFD = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(FunctionTag)));
allocatedMemory.Add(nativeFD);
Marshal.StructureToPtr(funcTable[i], nativeFD, false);
Marshal.WriteIntPtr(nativeArray, i * intPtrSize, nativeFD);
}
Marshal.WriteIntPtr(functionTablePtr, nativeArray);
return funcTable.Length;
}
Such code doesn't work, and the question is how to send a pointer to array of managed structs for use by unmanaged code? In which direction should I go?

We've been doing quite a lot of this sort of thing over the last couple of years.
We use a mixed-mode 'bridge' assembly to manage mapping function calls and marshalling data between managed and unmanaged environments. We often use a 'mixed-mode' class to wrap a managed class, providing a native interface for calling it's functionality.
Let's consider your problem. You could write GetFunctionTable in managed c++. It can call some managed c# code to get the function information in a managed struct, and then 'marshall' it into the native struct.
In (c#) a managed version of GetFunctionTable function:
delegate void FunctionDelegate();
public struct FuncDescStructManager
{
public string Name;
public FunctionDelegate Function;
//...
}
public static List<FuncDescStructManager> GetFunctionTableManaged()
{
List<FuncDescStructManager> list = new List<FuncDescStructManager>();
list.Add(new FuncDescStructManaged () {"ExampleF1", VExampleF1});
return list;
}
In the mixed-mode bridge assembly you can implement the native function GetFunctionTable,
calling the managed function and marshalling the data:
int GetFunctionTable(FuncDescStruct **ppFunctionTable)
{
// Call the managed function
List<FuncDescStructManaged>^ managedList = GetFunctionTableManaged();
nativeArray = malloc(managedList.Length * sizeof(FuncDescStruct));
int i=0;
foreach (FuncDescStructManaged managedFunc in managedList)
{
// Marshall the managed string to native string could look like this:
stringPtr = Marshal::StringToHGlobalUni(managedFunc.Name);
nativeArray[i].Name = ((wchar_t*)stringPtr.ToPointer());
Marshal::FreeHGlobal(stringPtr);
// Marshall a delegate into a native function pointer using a
// wrapper class:
WrapDelegateAsPtr funcPtr = new WrapDelegateAsPtr(managedFunc.Function);
// funcPtr will need to be deleted by caller
nativeArray[i].Function = funcPtr.NativeFunction;
i++;
}
return i;
}
// Mixed mode wrapper class
// Member is a reference to a managed delegate.
// Because we need to reference this from native code, the wrapped
// delegate will be stored as a void*.
class WrapDelegateAsFPtr
{
public:
WrapDelegateAsNativeFunctionPointer(FunctionDelegate _delegate)
{
delegate = _delegate;
// Tell the garbage handler not to remove the delegate object yet
GCHandle gch = GCHandle::Alloc(svgContents);
managedDelegatePtr = GCHandle::ToIntPtr(gch).ToPointer();
}
~WrapDelegateAsNativeFunctionPointer
{
// Tell the garbage collector we are finished with the managed object
IntPtr temp(managedDelegatePtr;);
GCHandle gch = static_cast<GCHandle>(temp);
gch.Free();
}
void NativeFunction()
{
}
private:
void* managedDelegatePtr;
}
Hope this helps - Any questions just ask!

THis is a rather belatedd answer, but I have come up against exactly the same problem. I have implmented a DATA PLUGIN using Kostya's framework, and got it to work perfectly, then trying to implmement an AFL plugin, I ran into the problem as per above. I have managed to get itworking without using a C++ rapper class.
The problem is with the function pointer/reference/address that is passed from withing the FUnctionTable. AS these functions have been dclared as STATIC for EXPORT purposes, they are incompatible with the C++ delegate implmentation in the GETFUNCTIONTABLE method. IF you ddo the following , it should work:-
Add the 2 signatures:-
[DllImport("kernel32", CharSet = CharSet.Ansi, ExactSpelling = true, SetLastError = true)]
static extern IntPtr GetProcAddress(IntPtr hModule, string procName);
[DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
internal static extern IntPtr LoadLibrary(string lpFileName);
[DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
internal static extern bool SetDllDirectory(string lpPathName);
the function reference definition in the GetFunctionTable structure must be changed to:
IntPtr Function;
Add the following statements to get the exported function address:-
IntPtr dllHandle = LoadLibrary(fullPath);
IntPtr fptr = GetProcAddress(dllHandle, "VfmExample1");
and lastly initialize the function variable in the GetFunctionTable structure , e.g.
functable[0].Description.function = fptr;
and that should do it

You have to use the fixing construct to fix the buffer in place, because the C# GC reserves the right to move it around if you don't, invalidating the pointer. I don't know how to fix a buffer indefinitely. You're going to have to worry about how that memory is managed, too.

P/Invoke C# struct with strings to C void*

I'm having a problem creating a C# P/invoke wrapper around a third party C library. In particular, the library has a method with the signature
int command(SomeHandle *handle, int commandNum, void *data, int datasize);
It is a wildcard method that does different things depending on commandNum. data can be a pointer to anything, like a single integer, or a char[], or a struct of some kind (my problem).
I have declared the wrapper as follows:
[DllImport("LIBRARY.DLL", EntryPoint = "command")]
public static extern int Command(IntPtr Handle, int CommandNum, [In, Out] IntPtr Data, int DataSize);
Now, when i call it with an opcode to fill a byte[] it works:
//WORKS, Buffer contains "library 1.0" after the call
const int BUFFER_SIZE = 128;
byte[] Buffer = new byte[BUFFER_SIZE];
int BytesWritten = 0;
GCHandle BufferHandle = GCHandle.Alloc(Buffer, GCHandleType.Pinned);
try
{
BytesWritten = Command(MyHandle, GET_VERSION, BufferHandle.AddrOfPinnedObject(), BUFFER_SIZE);
}
finally
{
BufferHandle.Free();
}
However, when I try the same with a simple struct, I cannot make it work, no matter what I try. the struct looks like this:
public struct FormatInfoType
{
public int Format;
public IntPtr Name; //const char*
public IntPtr Extension; //const char*
}
Here I am supposed to fill "Format" with an int (say, 1) and then the call to "command(...)" is meant to give me back the name and extension fields
If I pass this struct, the code compiles and runs correctly, but the values in the struct are never modified. If I change the IntPtr's to Strings or StringBuilders (and I've tried a myriad of MarshalAs attributes), then I cannot get the IntPtr to the struct because the it becomes non-blittable and the GCHandle line throws an exception.
Any help on this would be greatly appreciated.
EDIT:
I've tried many ways to call command() with the structure, but currently it looks like this:
FormatInfoType f = new FormatInfoType();
f.Format = 1;
f.Name = IntPtr.Zero;
f.Extension = IntPtr.Zero;
GCHandle fHandle = GCHandle.Alloc(f, GCHandleType.Pinned);
try
{
Command(MyHandle, GET_FORMAT_INFO, fHandle.AddrOfPinnedObject(), Marshal.SizeOf(f));
}
finally
{
fHandle.Free();
}

You can overload p/invoke signatures, try:
[DllImport("LIBRARY.DLL", EntryPoint = "command")]
public static extern int Command(IntPtr Handle, int CommandNum, ref FormatInfoType Data, int DataSize);