I have a unmanaged DLL that exposes a function that takes a pointer to a data structure. I have C# code that creates the data structure and calls the dll function without any problem. At the point of the function call to the dll the pointer is correct.
My problem is that the DLL keeps the pointer to the structure and uses the data structure pointer at a later point in time. When the DLL comes to use the pointer it has become invalid (I assume the .net runtime has moved the memory somewhere else).
What are the possible solutions to this problem?
The possible solutions I can think of are:
Fix the memory location of the data structure somehow? I don't know how you would do this in C# or even if you can.
Allocate memory manually so that I have control over it e.g. using Marshal.AllocHGlobal
Change the DLL function contract to copy the structure data (this is what I'm currently doing as a short term change, but I don't want to change the dll at all if I can help it as it's not my code to begin with).
Are there any other better solutions?
You can allocate the structure using AllocHGlobal, which puts it in unmanaged memory, where the GC won't move it around or release it. You could also use a helper class like this to have the GC pin the memory, so it won't be moved or released until un-pinned.
See the fixed C# statement: http://msdn.microsoft.com/en-us/library/f58wzh21(VS.80).aspx
Allocate memory manually so that I have control over it e.g. using Marshal.AllocHGlobal
Pretty close.
In this specific case I'd P/Invoke LocalAlloc to allocate the memory block and use StructureToPtr to initialize it.
UPDATE: Since you can edit the DLL I'd change the DLL to provide AllocXXX and FreeXXX functions.
Without having the DLL on hand to try this, it's hard to say if this would work. I would try making the object "fixed" in your C# class, that way the memory hangs out for the life of you application. Then just pass the static object to the DLL.
The GCHandle class was designed to handle this exact scenario. Essentially, you box a copy of your struct onto the heap, and then call GCHandle.Alloc with GCHandleType.Pinned. When the DLL is done with the structure, call GCHandle.Free. To give the DLL function the address of the object, pass it GCHandle.AddrOfPinnedObject. As long as it is pinned, the GC won't move it.
Related
I want to put a reference to a C# object into unmanaged memory (C), I guess as a pointer (int), and when the C code calls back into C# later on, I want to get the reference back from the unmanaged memory, so I can resolve it, and access the object. The reason is that the C code controls which object should be used, there's no real alternative. I have limited control over the C code and C++/CLI is not an option.
Question: Is that possible and safe, if so, how?
Well, it is possible. Primary concern is that your scheme is very incompatible with the garbage collector, it moves objects in memory when it compacts the heap. That's something you can stop, you can pin the object so the GC cannot move it. You use GCHandle.Alloc() to allocate a GCHandleType.Pinned handle and pass the return value of GCHandle.AddrOfPinnedObject() to your C code, presumably with a pinvoke call.
You have to fret about how long that object needs to stay pinned. A couple of seconds, tops, is okay, but it gets pretty detrimental to the GC if you keep it pinned for a long time. It is a rock in the road that the GC constantly has to drive around. And the heap segment can never be recycled, that single object can cost you a handful of megabytes.
In which case you should consider allocating unmanaged memory and copying the object into it. Use Marshal.AllocHGlobal() to allocate, Marshal.StructureToPtr() to copy the object into it. Possibly multiple times if you modify the object and the changes need to be visible to the C code as well.
Either way, the object must be blittable or you get a runtime error. An expensive word that just means that the object must have simple field types, the kind that a C program has a shot at reading correctly. Don't use bool. Be careful with the declaration in the C program, pretty easy to corrupt the heap when you get it wrong.
When you control the 'handing out' and the 'use after receiving back' phases you can simply use a List or array and pass around the index.
It's possible to consume C# objects via COM and proxies created by the CLR called COM-Callable Wrappers.
You just need assign a GUID assembly attribute to identify the COM type library, e.g.:
[assembly: Guid ("39ec755f-022e-497a-9ac8-70ba92cfdb7c")]
And then use the Type Library Exporter tool (tlbexp.exe) to genereate the COM type library (.tlb) file which can be consumed in the COM world:
tlbexp.exe YourLibrary.dll
If you mean safe in the C#'s sense of the word, then certainly unsafe, as you'll be using the objects in the unmanaged world, and lifetimes are controlled from the COM side via reference counting as opposed to CLR's GC.
I'm attempting to marshal a forest of objects from C# .NET to native C++. That is: I have a graph of hundreds of millions of objects (if not more), that I wish to use in native C++. See it as a normal 'leaf'/'node' construction with pointers between leafs and nodes. I control both the C++ and the C# code, so I can make adjustments to the code.
The inner loop of the software is going to be implemented in native C++ for performance reasons. I basically want to tell the GC to stop for a while (to ensure objects aren't moved), then do the fancy C++ routine, and then continue the GC once it's done.
There are also things that I don't want to do:
Make my own mark & sweep algorithm to pin all objects in the graph. Not only will that be very time consuming, it'll also cost a lot of memory because I then have to keep track of all these GCHandle objects by myself.
Use native allocation methods like malloc. I've had a native C++ application in the past, and it suffered greatly from memory fragmentation, that .NET 'automatically' solves just fine... not to mention the benefit of GC.
So, any suggestions on how to do this?
I will look at using managed C++.
Maybe accessing the .NET objects from manage C++ will be fast enough.
Otherwise use managed C++ to “walk” the .net objects and create native C++ objects, delete them all once done.
Or create a class factory class in manage C++ that can be used to create the C++ object being callable from C#, once again delete them all once done.
Or do as Marc Gravel says and manually allocating a buffer of unmanaged memory, and dealing with structs inside that space, maybe using a code generator driven from attributes on your C# classes.
I am using a C# wrapper to an API which is providing image data from a camera.
I have a class which contains data for each frame received, and I would like to implement a dispose method to run once the frame data has been processed. The problem is the primary object I wish to dispose of (an image buffer) is only accessible via a pointer.
I don't know if this is even necessary in C#. I know that in C++ I would have to do something with it to prevent a memory leak, however I don't know much about how pointers work in C# in terms of garbage collection. As it is, the program works okay, but the memory usage does jump around a lot more than I'd like it to. I've found that proper disposal of certain objects has given much more consistent memory usage compared to relying on what I assume to be finalizers being automatically run, such as when disposing of an existing image on an image box before setting a new one. I'd like to try something similar here if possible.
Potentially important: the data the pointer points to is processed by unsafe code, which creates two new pointers to the object (one as UInt16* and another as byte*). Both of these pointers only exist within the scope of the unsafe routine but I'm not sure if this interferes with GC in some way.
Any advice, recommendations appreciated. Thanks
Edit: I am receiving the frame data via callback from an external API.
I have a C# application which is using a third party (closed source) native DLL through p/invoke.
During some of these calls, the DLL will allocate memory using operator new / operator new[] and return a pointer to use for reading the results.
This memory is never freed by the DLL after being allocated and returned. How can I perform the equivalent of a native operator delete / operator delete[] in C#?
Not possible. The DLL stores the handle returned by HeapCreate() internally. You'd have to know that handle to release the memory, you cannot get it out of the DLL. And you would have to know how many extra bytes were allocated by the DLL's malloc function to adjust the pointer.
You cannot even do this reliably if you write a wrapper for the DLL so that you can call free(). The DLL will have to use the DLL version of the CRT and you have to compile the wrapper with the exact same version of the compiler and the CRT so they'll share the same CRT DLL.
If you are really desperate, you could try to hack through GetProcessHeaps(). Although that's hard to get right, error prone and you really have to know what you're doing. DLLs returning pointers that need to be freed is a really bad practice even in C or C++. Just not in .NET, hail the garbage collector :)
If it does a global ::operator new you may be able to P/Invoke ::operator delete, although you'll have to use the mangled name for it (and the right CRT dll of course). If it does something like AClass *p = new AClass, I would not recommend trying to duplicate the effects of delete p with P/Invoke, it is going to be a little rough to get right (especially if they do anything interesting at all like overload new/delete). I would suggest using C++/CLI to wrap some of this stuff instead if it gets hairy (and frankly I think calling any delete via P/Invoke is probably hairy enough that it is worth breaking out the C++/CLI to save your sanity).
I don't know how to do this explicitly, but you could wrap the DLL in Managed C++/CLI wrapper that has this facility for you.
I do not know if you can do this with unmanaged code, but a separate app domain may be the answer. Load it up in its own memory space, and just dump the whole app domain when you're done.
I'm writing some native C++ code which needs to be called from C# (and I can't replace the C++ native code with C# code).
I found memory corruptions while allocating/deallocating some memory in the native C++ code using malloc/free. Then I used LocalAlloc/LocalFree and HeapAlloc/HeapFree and had the same problems.
The allocations/deallocations seem to be correct, and they happen in a separate thread, created by the native code.
I was wondering which is the best allocation strategy to use in a native C++ library called by C#
EDIT: found the problem: the problem wasn't in the allocation/deallocation code, but in some memory being written after being deallocated.
As long as the C# side of the code uses the compiler's /unsafe switch and the fixed keyword used for holding the buffer of data, I think you should be ok.
As to the question of your memory allocation, it may not be the C++ memory allocation code that is causing the problem, it could be the way how the C++ code is interacting with the driver...maybe using VirtualAlloc/VirtualFree pair as per the MSDN docs...
Edit: When you try to allocate the buffer to hold the data from the C++ side after interacting with the driver...possibly a race-condition or interrupt latency is causing a memory corruption...just a thought...
Hope this helps,
Best regards,
Tom.
Your question is missing essential details, it isn't at all clear whether the memory allocated by the C++ code needs to be released on the C# side. That's normally done automatically with, say, the P/Invoke marshaller or the COM interop layer in the CLR. Or can be done manually by declaring a method argument as IntPtr, then use of the Marshal class.
If it is done automatically you must use the COM memory allocator, CoTaskMemAlloc(). If you marshal yourself you could also use GlobalAlloc(), release on the C# side with Marshal.FreeHGlobal(). There isn't any advantage to using GlobalAlloc(), you might as well use CoTaskMemAlloc() and release with Marshal.FreeCoTaskMem().
But you should have noticed this yourself. Allocating with malloc() or HeapAlloc() on the C++ side causes leaks instead of corruption if the managed code releases the memory. Vista and Win7 have a much stricter heap manager, it terminates the program if it notices a bad release.
It sounds to me that you have simple heap corruption in your C++ code. That is the most common scourge of unmanaged C++ programming, over-running the end of a buffer, writing to memory that's been freed, bad pointer values. The way to get rid of bugs like these is a careful code review and use of a debug allocator, such as the one provided by <crtdbg.h>. Good luck with it.
The windows driver developpement kit recommend against the use of C++ for drivers.
Also the best strategy is to have the driver manage its own memory. When the c# needs to see the data then pass it a marshalled buffer and have the driver to fill it