I'm trying to write C# bindings for a native DLL. In the DLL are structs defined as
typedef struct Foo {
Bar bars[32];
} Foo;
and
typedef struct Bar {
uint32_t data;
} Bar;
and a function
void Baz(Foo* foo);
Ideally, I would like to define the C# structs as
public unsafe struct Foo {
public unsafe fixed Bar bars[32]
}
and
public struct Bar {
public uint data;
}
and the function as
public unsafe void BazDelegate(Foo* foo);
This would allow me to use pointers to manipulate the structs. However, this is not allowed because fixed arrays can only be used on primitive types, so I can't have a fixed array of the second type. The API requires that the first type be passed in via pointer to a certain function.
The only solution I've found is by changing the type of the field to a managed array, Bar[], and letting the marshaller take care of it. But then that means that I can' take a pointer the the struct. So I would have to change the function's signature from unsafe pointer to ref.
The bindings are not written using [DllImport]. They are loaded at runtime through Marshal.GetDelegateForFunctionPointer. Could I change the structs to use [MarshalAs(UnmanagedType.ByValArray)] and the function use ref Foo instead of Foo*?
Is using a function created from Marshal.GetDelegateForFunctionPointer considered P/Invoke? Do managed arrays still get marshalled if they aren't passed through [DllImport]? Do they get marshalled if passed by ref?
Related
I'm trying to get a C# class (not struct!) from a C++ function back (using out parameter). This is the C# side:
[StructLayout(LayoutKind.Sequential)]
public class OutClass
{
public int X;
}
// This interface function will be mapped to the C++ function
void MarshalOutClass(out OutClass outClass);
The C++ part looks like this
struct OutClass
{
int x = 0;
};
extern "C" MARSHAL_TESTS_API void MarshalOutClass(OutClass** out);
The function mapping works via a custom mechanism and is not part of this question. All marshaling attributes are treated normally.
In C#, we usually declare out arguments inline, like:
MarshalOutClass(out var outClass);
DoSomething(outClass);
Since C# reference types are marshaled by pointer, I figured I would have to add another pointer for ref or out parameters. So I use OutClass** on the C++ side.
I assume that C# translates the
MarshalOutClass(out var outClass);
part to roughly
OutClass outClass = default(OutClass);
MarshalOutClass(ref outClass);
which is a reference to null, or on C++ side: a pointer to a nullptr. This wouldn't be a problem for C# value types (aka struct) because their default is a default constructed instance.
This means, I'd have to manually create an instance of my object on the C++ side and marshal it back to C#.
void MarshalOutClass(OutClass** out)
{
auto ptr = static_cast<OutClass*>(CoTaskMemAlloc(sizeof(OutClass)));
*ptr = OutClass{};
*out = ptr;
}
The code seems to work, but I'm not sure if I'm leaking memory with this or if the marshaler is going to to take care of it properly. I don't do any cleanup on the C# side.
This brings me to the following questions:
is my assumption on how ref and out are translated to C++ correct?
is CoTaskMemAlloc the correct function here?
do I have to perform any additional memory management related tasks (on either side)?
is the overall approach correct here? What should I do differently?
I know that static_cast<OutClass*>(CoTaskMemAlloc(sizeof(OutClass))) is a bit sketchy here but lets assume OutClass is always a trivial type.
I'm trying to create a C# interface which receives a callback from an external C DLL.
The callback's parameters contain pointers to C structs, which themselves have a pointer to a different struct.
The callback signature:
typedef application_event_result (*application_event_ptr)(abuffertype* read_buffer, abuffertype* write_buffer);
The buffer struct definitions in C:
typedef struct {
uint16 size;
uint8* data;
} anotherbuffertype;
typedef struct {
anotherbuffertype *buffer;
uint16 position;
} abuffertype;
I know that the C# signature of the callback should use "ref" for the pointer type of the parameter. But how can the pointer inside the "abuffertype" struct be defined in C#?
So far I have this definition of the two structs in C#:
[StructLayout(LayoutKind.Sequential)]
public struct anotherbuffer
{
UInt16 size;
IntPtr data;
}
[StructLayout(LayoutKind.Sequential)]
public struct abuffer
{
anotherbuffer buffer;
UInt16 position;
}
But that doesn't work. The contents of "abuffer" in C# is not what was there before the callback in the C code.
Do I need to unmarshal the internal struct pointer manually, and if so, how?
You will not get help from the marshaller, this normally causes a major memory management problem. But can work in the specific case of a callback since it is the calling C program that manages the data.
You have to convert the data yourself. Declare the pointers as IntPtr and use Marshal.PtrToStructure() to retrieve the data.
The anotherbuffertype.data member looks like an array, use Marshal.Copy() to copy its content into your own byte[] array. If you don't mind the unsafe keyword then you can keep it declared as byte* and access the elements with data[index], avoids the cost of the copy. It is not very unsafe, pretty easy to keep index constrained to [0..size).
I'm trying to marshal from C++ to C# a struct that looks something like this:
typedef struct FooStruct {
Uint8 bytesPerThingie;
void *arrayOfThingies;
// other members ...
}
So, in this case there are two unknowns:
The number of elements in the array.
The size (in bytes) of each element.
I had successfully marshaled the struct itself previously, with a definition like this:
[StructLayout(LayoutKind.Sequential)]
public struct FooStruct {
public byte bytesPerThingie;
public IntPtr arrayOfThingies;
// other members...
}
but now I need to inspect and modify the embedded array.
I understand that
By itself, an array of blittable elements of a blittable type is
itself blittable, but not when it is used as a field within a
structure.
When marshaling from unmanaged code to the .NET Framework, the array length is
determined from the SizeConst argument, optionally followed by the unmanaged type
of the array elements, if they aren’t blittable.
Even assuming that the elements in the array in this case are of a blittable type, how can I set SizeConst, a compile-time argument, if I can't know the size of the array until runtime?
Long story short, you can't. The SizeConst field on the MarshalAsAttribute class is compiled into metadata on the field and cannot be changed at runtime (at least, not in a way that would benefit you).
That said, you have the following options:
Marshal the contents manually as you've been doing using the methods on the Marshal class.
Use unsafe to access the pointer directly (and change your type to use pointers). This requires the /unsafe compiler option which may or may not be an option for you.
Use C++/CLI to create a managed wrapper in C++ that will export .NET types, but handle the marshaling in C++ (which might be easier, depending on your comfort level and the complexity of the API you are trying to access).
Note that in all of the cases above, you still have to know the length of the array that is returned (it's probably in the structure along with the pointer and the type).
.NET Platform Invoke advocates declaring pointer types as IntPtr. For example, the following
[DllImport("mylib")]
static extern IntPtr get_foo();
[DllImport("mylib")]
static extern void do_something_foo(IntPtr foo);
However, I find when interfacing with interesting native interfaces that have many pointer types, flattening everything into IntPtr makes the code very hard to read and removes the typical typechecking that a compiler can do.
I've been using a pattern where I declare an unsafe struct to be an opaque pointer type. I can store this pointer type in a managed object, and the compiler can typecheck it for me. For example:
class Foo {
unsafe struct FOO {}; // opaque type
unsafe FOO *my_foo;
class if {
[DllImport("mydll")]
extern static unsafe FOO* get_foo();
[DllImport("mydll")]
extern static unsafe void do_something_foo(FOO *foo);
}
public unsafe Foo() {
this.my_foo = if.get_foo();
}
public unsafe do_something_foo() {
if.do_something_foo(this.my_foo);
}
NOTE: I'm not trying to marshal a structure. The DLL is providing an opaque pointer which I'm not supposed to touch, I merely need to provide it to future calls to the DLL.
I know that the published way to do this is IntPtr, but I don't like using an untyped pointer. When there are several pointer types moving between managed and native code, using IntPtrs is very dangerous. Using my opaque struct pointer types for typechecking is a godsend.
I have not run into any trouble using this technique in practice. However, I also have not seen an examples of anyone using this technique, and I wonder why. Is there any reason that the above code is invalid in the eyes of the .NET runtime?
My main question is about how the .NET GC system treats "unsafe FOO *my_foo". My hope is that because the underlying type is a struct, and it's declared unsafe, that the GC would ignore it.
Is this pointer something the GC system is going to try to trace, or is it simply going to ignore it? Is it safe to use my technique instead of IntPtr?
It appears that the answer is "yes"... "unsafe pointers are treated as value types", which means it's safe to use them to store opaque types. In a sense, they work just like IntPtr, but they come with additional type-checking, because different types of unsafe pointers are not considered the same, as they would be if you made them all IntPtr.
For a more detailed article I wrote on the topic, check out..
http://www.codeproject.com/script/Articles/ArticleVersion.aspx?waid=1210&aid=339290
I wouldn't use unsafe code and pointers. Why not simply define the structure and let the CLR do the mapping:
[StructLayout(LayoutKind.Sequential)]
public struct Foo
{
public int Field1;
public long Field2;
}
and then:
[DllImport("mylib")]
static extern void do_something_foo(ref Foo foo);
My gut reaction is no, because managed and unmanaged memory are distinct, but I'm not sure if the .NET Framework is doing something with Marshaling behind the scenes.
What I believe happens is:
When getting a struct from my unmanaged DLL, it is the same as making that call gets an IntPtr and then uses it and the Marshal class to copy the struct into managed memory (and changes made to the struct in managed memory do not bubble up).
I can't seem to find this documented anywhere on MSDN. Any links would be appreciated.
Here is what my code looks like:
[DllImport("mydll.dll", BestFitMapping=false, CharSet=CharSet.Ansi)]
private static extern int GetStruct(ref MyStruct s);
[StructLayout(LayoutKind.Sequential, Pack=0)]
struct MyStruct
{
public int Field1;
public IntPtr Field2;
}
public void DoSomething()
{
MyStruct s = new MyStruct();
GetStruct(ref s);
s.Field1 = 100; //does unmanaged memory now have 100 in Field1 as well?
s.Field2 = IntPtr.Zero; //does unmanaged memory now have a NULL pointer in field Field2 as well?
}
No, the P/Invoke marshaller copied the unmanaged structure member values into the managed version of the structure. In general, the managed version of a structure is not in any way compatible with the unmanaged version of it. The memory layout is not discoverable, something the CLR uses to reorder fields to make the structure smaller. Marshaling is essential, you have to create a copy.
Modifying the structure is not possible with the given function signature since you let fill in the memory that's passed to it. The function itself already copies the structure. You can however party on the Field2 value since it is a raw pointer. If that points to a structure then marshal it yourself with Marshal.PtrToStructure(). Modify the managed copy of it and copy it back to unmanaged memory with Marshal.StructureToPtr(). Or access it directly with Marshal.ReadXxx() and WriteXxx().
CSharp Language Specification.doc pg 26
Struct constructors are invoked with the new operator, but that does not imply that memory is being allocated. Instead of dynamically allocating an object and returning a reference to it, a struct constructor simply returns the struct value itself (typically in a temporary location on the stack), and this value is then copied as necessary.
Since, there is nothing special about a 'struct' backing store, so one would not expect there to be annonymous marshalling operations going on behind the member assignments.