I am passing an array of integers from C++ to C#, using a parameter like this in my C# method:
[MarshalAs(UnmanagedType.LPArray, SizeParamIndex = 0)]
UInt32[] myStuff,
When this data arrives in the CLR, I think that "LPArray" indicates that I am working with the pointer from the C++-world directly? So if I want to hold onto this array after the method call is over, should I make a copy of it?
You have to be careful that this is allocated using the same memory allocation mechanisms in both the managed and unmanaged worlds. And even if that's the case, it's just safer to make a copy and work with that.
Note: In your example, the pointer isn't passed by reference, and so the callee can only add to your previous array, not give you a new array. Is that really what you intended?
Calling from unmanaged code into managed code (C++ to C#), the array gets copied into a fully managed array. This array will be freed when the garbage collector notices that nobody has any more references to it, not when the function exits.
But the other direction is more dangerous: If you were going the other direction (C# to C++), it would either: pin the C# array so it won't move, or make a temporary copy of the array into unmanaged memory (which gets freed when the function returns). In either case, it would not be safe to hold onto the array on the C++ side after the function call completes, you'd want to copy the contents to somewhere else.
Related
I want to put a reference to a C# object into unmanaged memory (C), I guess as a pointer (int), and when the C code calls back into C# later on, I want to get the reference back from the unmanaged memory, so I can resolve it, and access the object. The reason is that the C code controls which object should be used, there's no real alternative. I have limited control over the C code and C++/CLI is not an option.
Question: Is that possible and safe, if so, how?
Well, it is possible. Primary concern is that your scheme is very incompatible with the garbage collector, it moves objects in memory when it compacts the heap. That's something you can stop, you can pin the object so the GC cannot move it. You use GCHandle.Alloc() to allocate a GCHandleType.Pinned handle and pass the return value of GCHandle.AddrOfPinnedObject() to your C code, presumably with a pinvoke call.
You have to fret about how long that object needs to stay pinned. A couple of seconds, tops, is okay, but it gets pretty detrimental to the GC if you keep it pinned for a long time. It is a rock in the road that the GC constantly has to drive around. And the heap segment can never be recycled, that single object can cost you a handful of megabytes.
In which case you should consider allocating unmanaged memory and copying the object into it. Use Marshal.AllocHGlobal() to allocate, Marshal.StructureToPtr() to copy the object into it. Possibly multiple times if you modify the object and the changes need to be visible to the C code as well.
Either way, the object must be blittable or you get a runtime error. An expensive word that just means that the object must have simple field types, the kind that a C program has a shot at reading correctly. Don't use bool. Be careful with the declaration in the C program, pretty easy to corrupt the heap when you get it wrong.
When you control the 'handing out' and the 'use after receiving back' phases you can simply use a List or array and pass around the index.
It's possible to consume C# objects via COM and proxies created by the CLR called COM-Callable Wrappers.
You just need assign a GUID assembly attribute to identify the COM type library, e.g.:
[assembly: Guid ("39ec755f-022e-497a-9ac8-70ba92cfdb7c")]
And then use the Type Library Exporter tool (tlbexp.exe) to genereate the COM type library (.tlb) file which can be consumed in the COM world:
tlbexp.exe YourLibrary.dll
If you mean safe in the C#'s sense of the word, then certainly unsafe, as you'll be using the objects in the unmanaged world, and lifetimes are controlled from the COM side via reference counting as opposed to CLR's GC.
If i have understood right, when using structs with non blitable values, the struct data from unmanaged memory is copied into the managed memory (basically having the same struct twice).
Also if im not wrong, IntPtr variables, are stored in managed memory, but the data they point to is in the unmanaged memory.
Now lets say i have a delegate method, which is being called by a c++ function and receives a struct as ref, and a handler would be something like:
private void _handler(ref MyStruct p){}
Now the api says I should not keep the reference to the struct as the memory might be recycled and used for further calls, but since the unmanaged struct is copied into managed memory, and after I assign it to a local variable it is copied again (because it is a struct!) i shouldnt have any issues at all if the unmanaged memory gets freed or re written.
However, if the struct contains an IntPtr variable, i might save a copy of the pointer but not of the data it points, so if i try to access the IntPtr data from my saved struct, i might get some memory fault, is this correct?
One last question is, doing changes to the struct in managed memory, will also affect the unmanaged memory struct, since it is passed by ref, and it implicitly means IN/OUT, so after the event handler call ends, changes in managed memory will be made in the unmanaged memory as well?
Your understanding is largely correct. Taking away the details of blittabiltiy, the root question is one of ownership of the memory being passed to your callback. Typically, the contract between the caller of the callback and the callback itself is that the memory is owned by the caller and will be valid only for the duration of the callback itself. (Other contracts are possible as well - for instance, the caller could transfer ownership of the memory to the callback, in which case the callback is responsible for freeing the memory when it is done).
In the documentation you reference, the API is telling you not to keep a reference to the memory, which means that you are very likely in the standard case where the caller owns the structure. What this means is that both the contents of the structure and everything pointed to by that structure should be valid during the call to your callback but cannot be relied upon after your callback completes.
Copying a structure to a local wouldn't be problematic because the native memory should not be freed until after your callback completes, so keeping as many copies as you need during the execution of the callback should be fine. What would be problematic is saving a copy of the structure to a location, such as a static field, which will be alive after your callback returns.
Since the structure is implicitly in/out, then modifications you make to it will be reflected on the native side of the call as well. This is true regardless of if the structure contained only blittable data. If the structure contained non-blittable data then the CLR would marshal your changes to the structure back out to the native copy when your code completes.
Note that just because you can modify the native structure does not mean that the API contract expects you to do so. Frequently API signatures contain pointers to structures in C++ in order to avoid copying the entire structure when making a call, not to enable modification of the structure.
I have a .NT class which has multiple delegates for callbacks from native code. Is it necessary to allocate all the delegates? I mean does GCHandle.Alloc() protects just the delegate or the entire class that owns the delegate from being collected?
A delegate has two relevant properties, Method and Target. The Target will be non-null if the delegate was created for an instance method. And that keeps the object alive, as long as the garbage collector can see the delegate instance.
Native code is relevant to having problems with callbacks. When you pass a delegate instance to a pinvoked native function then the P/Invoke marshaller will use Marshal.GetFunctionPointerForDelegate() to create a little stub that produces the required Target reference when the native code makes the callback. The garbage collector however can not see this stub and therefore won't find a reference to the delegate object. And collects it. The next callback from the native code produces a crash.
To avoid this, you must store the delegate object yourself so that it stays referenced for as long as the native code can make the callback. Storing it in a static variable is an obvious solution.
I am wrong.
You don't need to pinned the delegate (at the other hand, you cannot pinned the delegate, there will be a exception be thrown (System.ArgumentException: Object contains non-primitive or non-blittable data.)
reference:
http://social.msdn.microsoft.com/Forums/vstudio/en-US/bd662199-d150-4fbf-a5ee-7a06af0493bb/interop-pinning-and-delegates?forum=
Details about that in Chris Brumme's blog http://blogs.msdn.com/cbrumme/archive/2003/05/06/51385.aspx
Chris Brumme wrote:
Along the same lines, managed Delegates can be marshaled to unmanaged code, where they are exposed as unmanaged function pointers. Calls on those pointers will perform an unmanaged to managed transition; a change in calling convention; entry into the correct AppDomain; and any necessary argument marshaling. Clearly the unmanaged function pointer must refer to a fixed address. It would be a disaster if the GC were relocating that! This leads many applications to create a pinning handle for the delegate. This is completely unnecessary. The unmanaged function pointer actually refers to a native code stub that we dynamically generate to perform the transition & marshaling. This stub exists in fixed memory outside of the GC heap.
However, the application is responsible for somehow extending the lifetime of the delegate until no more calls will occur from unmanaged code. The lifetime of the native code stub is directly related to the lifetime of the delegate. Once the delegate is collected, subsequent calls via the unmanaged function pointer will crash or otherwise corrupt the process. In our recent release, we added a Customer Debug Probe which allows you to cleanly detect this – all too common – bug in your code. If you haven’t started using Customer Debug Probes during development, please take a look!
I think there will be something wrong happened (but not usually) when you just store the delegate object.
As we all know , the managed memory will be arranged by Garbage Collect. ( That means the physical memory address of a managed object will be changed. )
Imaging there is a long-time-life delegate to be called by native code, we set the delegate as the static member or class member . But sometime (we don't know when , we just know it will happen) , GC arranged memory, and the physical memory of the delegate may from 0x000000A to 0x0000010 . But the native code know nothing about it , for the native code , it only knows to call at 0x000000A forever.
So we should not only store the delegate object but also use GCHandle.Alloc to tell GC not move the physical memory of the delegate object. Then the native code will do well at callback time.
Well , because the GC do not arrange managed memory frequencly , so for a short-time-life delegate , even you do not call the GCHandle.Alloc ， your codes always "DO WELL" ， but sometimes it will mad.
Now , you know the reason.
reference:
http://dotnet.dzone.com/news/net-memory-control-use-gchandl
I have a unmanaged DLL that exposes a function that takes a pointer to a data structure. I have C# code that creates the data structure and calls the dll function without any problem. At the point of the function call to the dll the pointer is correct.
My problem is that the DLL keeps the pointer to the structure and uses the data structure pointer at a later point in time. When the DLL comes to use the pointer it has become invalid (I assume the .net runtime has moved the memory somewhere else).
What are the possible solutions to this problem?
The possible solutions I can think of are:
Fix the memory location of the data structure somehow? I don't know how you would do this in C# or even if you can.
Allocate memory manually so that I have control over it e.g. using Marshal.AllocHGlobal
Change the DLL function contract to copy the structure data (this is what I'm currently doing as a short term change, but I don't want to change the dll at all if I can help it as it's not my code to begin with).
Are there any other better solutions?
You can allocate the structure using AllocHGlobal, which puts it in unmanaged memory, where the GC won't move it around or release it. You could also use a helper class like this to have the GC pin the memory, so it won't be moved or released until un-pinned.
See the fixed C# statement: http://msdn.microsoft.com/en-us/library/f58wzh21(VS.80).aspx
Allocate memory manually so that I have control over it e.g. using Marshal.AllocHGlobal
Pretty close.
In this specific case I'd P/Invoke LocalAlloc to allocate the memory block and use StructureToPtr to initialize it.
UPDATE: Since you can edit the DLL I'd change the DLL to provide AllocXXX and FreeXXX functions.
Without having the DLL on hand to try this, it's hard to say if this would work. I would try making the object "fixed" in your C# class, that way the memory hangs out for the life of you application. Then just pass the static object to the DLL.
The GCHandle class was designed to handle this exact scenario. Essentially, you box a copy of your struct onto the heap, and then call GCHandle.Alloc with GCHandleType.Pinned. When the DLL is done with the structure, call GCHandle.Free. To give the DLL function the address of the object, pass it GCHandle.AddrOfPinnedObject. As long as it is pinned, the GC won't move it.
I have the following struct:
[StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
public struct WAVEHDR
{
internal IntPtr lpData; // pointer to locked data buffer
internal uint dwBufferLength; // length of data buffer
internal uint dwBytesRecorded; // used for input only
internal IntPtr dwUser; // for client's use
internal uint dwFlags; // assorted flags (see defines)
internal uint dwLoops; // loop control counter
internal IntPtr lpNext; // reserved for driver
internal IntPtr reserved; // reserved for driver
}
I need to allocate unmanaged memory to store an instance of above struct. A pointer to this struct will be passed to waveOut win32 api functions (waveOutPrepareHeader, waveOutWrite, waveOutUnprepareHeader).
Should I use Marshal.AllocHGlobal() or Marshal.AllocCoTaskMem()? What is the difference?
Should I pass sizeof(WAVEHDR) or Marshal.SizeOf(typeof(WAVEHDR)) to the memory allocation method? What is the difference?
NOTE that the allocated memory must be pinned.
A Windows program always has at least two heaps in which unmanaged memory is allocated. First is the default process heap, used by Windows when it needs to allocate memory on behalf of the program. The second is a heap used by the COM infrastructure to allocate. The .NET P/Invoke marshaller assumes this heap was used by any unmanaged code whose function signature requires de-allocating memory.
AllocHGlobal allocates from the process heap, AllocCoTaskMem allocates from the COM heap.
Whenever you write unmanaged interop code, you should always avoid a situation where code that allocates unmanaged memory is not the same as the code that frees it. There would be a good chance that the wrong de-allocator is used. This is especially true for any code that interops with a C/C++ program. Such programs have their own allocator that uses its own heap, created by the CRT at startup. De-allocating such memory in other code is impossible, you can't reliably get the heap handle. This is a very common source of P/Invoke trouble, especially because the HeapFree() function in XP and earlier silently ignore requests to free memory that wasn't allocated in the right heap (leaking the allocated memory) but Vista and Win7 crash the program with an exception.
No need to worry about this in your case, the mmsystem API functions you are using are clean. They were designed to ensure the same code that allocates also deallocates. This is one reason you have to call waveInPrepareHeader(), it allocates buffers with the same code that ultimately deallocates them. Probably with the default process heap.
You only need to allocate the WAVEHDR structure. And you are responsible for releasing it when you're done with it. The mmsystem APIs don't do it for you, most of all because they cannot do so reliably. Accordingly, you can use either allocator, you just need to make sure to call the corresponding free method. All Windows APIs work this way. I use CoTaskMemAlloc() but there really isn't a preference. Just that if I'm calling badly designed code, it is slightly likelier to use the COM heap.
You should never use sizeof() in an interop scenario. It returns the managed size of value type. That might not be the same after the P/Invoke marshaller has translated a structure type according to the [StructLayout] and [MarshalAs] directives. Only Marshal.SizeOf() gives you a guaranteed correct value.
UPDATE: there was a big change in VS2012. The C runtime library included with it now allocates from the default process heap instead of using its own heap. Long term, that makes AllocHGlobal the most likely avenue for success.
1) Marshal.AllocHGlobal will work for sure. Based on the docs for Marshal.AllocCoTaskMem, Marshal.AllocCoTaskMem should work too.
2) Use Marshal.SizeOf(typeof(WAVEHDR)). Although you can use the Marshal.SizeOf method, the value returned by this method is not always the same as the value returned by sizeof. Marshal.SizeOf returns the size after the type has been marshaled, whereas sizeof returns the size as it has been allocated by the common language runtime, including any padding.
2) As far as I know the sizeof can only be used with types that have a predefined size at the compile-time.
So use Marshal.SizeOf(typeof(WAVEHDR)).