Need some expert advice:
I have an unmanaged code from a 3rd party provider which is on C/C++ and I am writing a wrapper from C#.Net to get referred and use the respective methods. Since its unmanaged and there is a lot of places which has the memory leak and unmanaged code is not known to manage the memory what is the best solution for this kind of scenario?
I have the solution with PInvoke and Marshalling which bridges the Managed and Unmanaged with a knowledge of taking the memory management. But will GC takes care of this completely or is there still a gap which the memory should be released? Is there a solution from the .Net wrapper class i have a generic way to manage the memory for the unmanaged calls?
Related
I'm attempting to marshal a forest of objects from C# .NET to native C++. That is: I have a graph of hundreds of millions of objects (if not more), that I wish to use in native C++. See it as a normal 'leaf'/'node' construction with pointers between leafs and nodes. I control both the C++ and the C# code, so I can make adjustments to the code.
The inner loop of the software is going to be implemented in native C++ for performance reasons. I basically want to tell the GC to stop for a while (to ensure objects aren't moved), then do the fancy C++ routine, and then continue the GC once it's done.
There are also things that I don't want to do:
Make my own mark & sweep algorithm to pin all objects in the graph. Not only will that be very time consuming, it'll also cost a lot of memory because I then have to keep track of all these GCHandle objects by myself.
Use native allocation methods like malloc. I've had a native C++ application in the past, and it suffered greatly from memory fragmentation, that .NET 'automatically' solves just fine... not to mention the benefit of GC.
So, any suggestions on how to do this?
I will look at using managed C++.
Maybe accessing the .NET objects from manage C++ will be fast enough.
Otherwise use managed C++ to “walk” the .net objects and create native C++ objects, delete them all once done.
Or create a class factory class in manage C++ that can be used to create the C++ object being callable from C#, once again delete them all once done.
Or do as Marc Gravel says and manually allocating a buffer of unmanaged memory, and dealing with structs inside that space, maybe using a code generator driven from attributes on your C# classes.
I'm profiling memory use with ANTS Memory Profiler 7.0 and noticed that unmanaged memory use is ~193MB (~62%) for a console application that does little more than populate some DTOs from 10 million or so records.
The help text for unmanaged memory says:
The memory is assigned to the parts of the application that aren't running as pure .NET code. This includes the common language runtime itself, graphics buffers and any unmanaged data accessed through P/Invoke or COM+
Why might this figure be so high?
You will inevitable use unmanaged code when accessing a database. The interface to the engine is always code that's been around for a long time, predating .NET and wrapped by managed classes that provide the interop. True for, say, SQL Server and any provider that piggy-backs onto OleDb or ODBC.
These managed classes will always implement IDisposable so you can release the resources consumed by the native provider early. Forgetting to do so is very common and rarely noticed. Other than seeing the process running "heavy", seemingly consuming a lot of handles and unmanaged memory for no good reason. This will be especially the case when the garbage collector does not run frequently enough, something you can see with Perfmon.exe. So beyond not using Dispose, part of the problem can be that you don't do enough work with these DTO objects yet to get enough GC churn.
Review your code and ensure you use Dispose() and the using statement where required.
is there a way to share memory between managed and native code, just like memory mapped files (I'am using .Net 2.0, so MMF is not available in this version), I'am new to this .Net, so what is actually marshalling? I've done this for communication between managed and unmanaged code, but my question is during marshaling do we copy the memory portion of unmanaged to managed memory area, or we share the same address space, thus when we change a variable from unmanaged code, then the value is changed too in managed code?
This post is a useful start point to gain some ideas about Marshaling between Managed and Unmanaged Code
you can skip the part related to [InAttribute] and [OutAttribute]
What are unmanaged objects? Can you please explain it in terms of the CLR? I learned on the internet that they say unmanaged objects don't run under the CLR environment. Can you please give me an example of unmanaged objects?
Any memory not managed by the CLR memory management (i.e. garbage collector) is unmanaged memory.
An OS file handle is one example of unmanaged memory (under .NET and windows).
To properly dispose of unmanaged
resources, it is recommended that you
implement a public Dispose or Close
method that executes the necessary
cleanup code for the object. The
IDisposable interface provides the
Dispose method for resource classes
that implement the interface. Because
it is public, users of your
application can call the Dispose
method directly to free memory used by
unmanaged resources. When you properly
implement a Dispose method, the
Finalize method becomes a safeguard to
clean up resources in the event that
the Dispose method is not called.
Ref: Cleaning Up Unmanaged Resources
In simple words, the unmanaged objects are the objects that aren't managed by the .Net framework.
Best example is the database connection or file operation are handled by the OS at the end and need to be liberated explicitly (File.Close() or Connection close) and won't be handled automatically by the Garbage collector.
VC++6.0 samples or many of activeX and COM objects your using everyday for your application or website are unmanaged, for example Excel VBA is unmanaged and too many other samples.
I learned on the internet that they say unmanaged objects don't run under the CLR environment.
This is not right, the CLR is pretty much able to do everything whats possible within C. In C# you have got a keywoard called unsafe which allows you to access even pointers and pointer offsets. I have a project where I do heavy Interop with a game engine and the C wrapper is so small, because I can handle all the memory objects within the CLR/C#.
By doesn't run they probably wanted to make it explicitly clear that the unamanged objects are not handled by the virtual machine: you have to do the cleanup or create wrapper classes which do the clean up for you.
I'm writing some native C++ code which needs to be called from C# (and I can't replace the C++ native code with C# code).
I found memory corruptions while allocating/deallocating some memory in the native C++ code using malloc/free. Then I used LocalAlloc/LocalFree and HeapAlloc/HeapFree and had the same problems.
The allocations/deallocations seem to be correct, and they happen in a separate thread, created by the native code.
I was wondering which is the best allocation strategy to use in a native C++ library called by C#
EDIT: found the problem: the problem wasn't in the allocation/deallocation code, but in some memory being written after being deallocated.
As long as the C# side of the code uses the compiler's /unsafe switch and the fixed keyword used for holding the buffer of data, I think you should be ok.
As to the question of your memory allocation, it may not be the C++ memory allocation code that is causing the problem, it could be the way how the C++ code is interacting with the driver...maybe using VirtualAlloc/VirtualFree pair as per the MSDN docs...
Edit: When you try to allocate the buffer to hold the data from the C++ side after interacting with the driver...possibly a race-condition or interrupt latency is causing a memory corruption...just a thought...
Hope this helps,
Best regards,
Tom.
Your question is missing essential details, it isn't at all clear whether the memory allocated by the C++ code needs to be released on the C# side. That's normally done automatically with, say, the P/Invoke marshaller or the COM interop layer in the CLR. Or can be done manually by declaring a method argument as IntPtr, then use of the Marshal class.
If it is done automatically you must use the COM memory allocator, CoTaskMemAlloc(). If you marshal yourself you could also use GlobalAlloc(), release on the C# side with Marshal.FreeHGlobal(). There isn't any advantage to using GlobalAlloc(), you might as well use CoTaskMemAlloc() and release with Marshal.FreeCoTaskMem().
But you should have noticed this yourself. Allocating with malloc() or HeapAlloc() on the C++ side causes leaks instead of corruption if the managed code releases the memory. Vista and Win7 have a much stricter heap manager, it terminates the program if it notices a bad release.
It sounds to me that you have simple heap corruption in your C++ code. That is the most common scourge of unmanaged C++ programming, over-running the end of a buffer, writing to memory that's been freed, bad pointer values. The way to get rid of bugs like these is a careful code review and use of a debug allocator, such as the one provided by <crtdbg.h>. Good luck with it.
The windows driver developpement kit recommend against the use of C++ for drivers.
Also the best strategy is to have the driver manage its own memory. When the c# needs to see the data then pass it a marshalled buffer and have the driver to fill it