I try to figure out an Memory Leak in my mixed VC++/C# application. Interop is done by com. I figured out the garbage collector does not clean up the managed Object LcUI.Portal.PortalMainForm. It is created and released in the native part of the application. But each time this is done an instance gets created and not cleaned up.
Here a sample output from windbg:
!dumpheap -type LcUI.Portal.PortalMainForm
Address MT Size
234528ac 1dcf176c 652
!gcroot find the object referenced as (strong handle) on the HandleTable:
!gcroot -all 234528ac
HandleTable:
02fd530c (strong handle)
-> 234528ac LcUI.Portal.PortalMainForm
But I'm not able to get the handle info:
!handle 02fd530c
Could not duplicate handle 2fd530c, error 6
Because of the (strong handle) I believe a static self reference from LcUI.Portal.PortalMainForm to the instance of LcUI.Portal.PortalMainForm prevents cleanup by the gc. Or is this a misinterpretation?
Via !DumpObj /d 234528ac I did not find such a reference. How may I find who and where the handle is still held?
After a long time of debugging I found it out. It was the nativ windows handel that was hanging in the WinForms Framework which caused the memory leak.
This happened because in the OnClosing event the close was canceled but some method that was called fromt he event did also call close.
After I cleaned this state so there was no need to cancel the close the memory leak was gone.
Related
I'm facing with a .Net server application, which crashes on an almost weekly basis on a problem in a "GC Finalizer Thread", more exactly at line 798 of "mscorlib.dll ...~DestroyScout()", according to Visual Studio.
Visual Studio also tries to open the file "DynamicILGenerator.gs". I don't have this file, but I've found a version of that file, where line 798 indeed is inside the destructor or the DestroyScout (whatever this might mean).
I have the following information in my Visual Studio environment:
Threads :
Not Flagged > 5892 0 Worker Thread GC Finalizer Thread mscorlib.dll!System.Reflection.Emit.DynamicResolver.DestroyScout.~DestroyScout
Call stack:
[Managed to Native Transition]
> mscorlib.dll!System.Reflection.Emit.DynamicResolver.DestroyScout.~DestroyScout() Line 798 C#
[Native to Managed Transition]
kernel32.dll!#BaseThreadInitThunk#12() Unknown
ntdll.dll!__RtlUserThreadStart() Unknown
ntdll.dll!__RtlUserThreadStart#8() Unknown
Locals (no way to be sure if that $exception object is correct):
+ $exception {"Exception of type 'System.ExecutionEngineException' was thrown."} System.ExecutionEngineException
this Cannot obtain value of the local variable or argument because it is not available at this instruction pointer,
possibly because it has been optimized away. System.Reflection.Emit.DynamicResolver.DestroyScout
Stack objects No CLR objects were found in the stack memory range of the current frame.
Source code of "DynamicILGenerator.cs", mentioning the DestroyScout class (line 798 is mentioned in comment):
private class DestroyScout
{
internal RuntimeMethodHandleInternal m_methodHandle;
[System.Security.SecuritySafeCritical] // auto-generated
~DestroyScout()
{
if (m_methodHandle.IsNullHandle())
return;
// It is not safe to destroy the method if the managed resolver is alive.
if (RuntimeMethodHandle.GetResolver(m_methodHandle) != null)
{
if (!Environment.HasShutdownStarted &&
!AppDomain.CurrentDomain.IsFinalizingForUnload())
{
// Somebody might have been holding a reference on us via weak handle.
// We will keep trying. It will be hopefully released eventually.
GC.ReRegisterForFinalize(this);
}
return;
}
RuntimeMethodHandle.Destroy(m_methodHandle); // <===== line 798
}
}
Watch window (m_methodHandle):
m_methodHandle Cannot obtain value of the local variable or argument because
it is not available at this instruction pointer,
possibly because it has been optimized away.
System.RuntimeMethodHandleInternal
General dump module information:
Dump Summary
------------
Dump File: Application_Server2.0.exe.5296.dmp : C:\Temp_Folder\Application_Server2.0.exe.5296.dmp
Last Write Time: 14/06/2022 19:08:30
Process Name: Application_Server2.0.exe : C:\Runtime\Application_Server2.0.exe
Process Architecture: x86
Exception Code: 0xC0000005
Exception Information: The thread tried to read from or write to a virtual address
for which it does not have the appropriate access.
Heap Information: Present
System Information
------------------
OS Version: 10.0.14393
CLR Version(s): 4.7.3920.0
Modules
-------
Module Name Module Path Module Version
----------- ----------- --------------
...
clr.dll C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll 4.7.3920.0
...
Be aware: the dump arrived on a Windows-Server 2016 computer, I'm investigating the dump on my Windows-10 environment (don't be mistaking on OS Version in the dump summary)!
Edit
What might the destroyscout be trying to destroy? That might be very interesting.
I don't know what exactly is causing this crash, but I can tell you what DestroyScout does.
It's related to creating dynamic methods. The class DynamicResolver needs to clean up related unmanaged memory, which is not tracked by GC. But it cannot be cleaned up until there are definitely no references to the method anymore.
However, because malicious (or outright weird) code can use a long WeakReference which can survive a GC, and therefore resurrect the reference to the dynamic method after its finalizer has run. Hence DestroyScout comes along with its strange GC.ReRegisterForFinalize code in order to ensure that it's the last reference to be destroyed.
It's explained in a comment in the source code
// We can destroy the unmanaged part of dynamic method only after the managed part is definitely gone and thus
// nobody can call the dynamic method anymore. A call to finalizer alone does not guarantee that the managed
// part is gone. A malicious code can keep a reference to DynamicMethod in long weak reference that survives finalization,
// or we can be running during shutdown where everything is finalized.
//
// The unmanaged resolver keeps a reference to the managed resolver in long weak handle. If the long weak handle
// is null, we can be sure that the managed part of the dynamic method is definitely gone and that it is safe to
// destroy the unmanaged part. (Note that the managed finalizer has to be on the same object that the long weak handle
// points to in order for this to work.) Unfortunately, we can not perform the above check when out finalizer
// is called - the long weak handle won't be cleared yet. Instead, we create a helper scout object that will attempt
// to do the destruction after next GC.
As to your crash, this is happening in internal code, and is causing an ExecutionEngineException. This most likely happens when there is memory corruption, when memory is used in a way it wasn't supposed to be.
Memory corruption can happen for a number of reasons. In order of likelihood:
Incorrect use of PInvoke to native Win32 functions (DllImport and asscociated marshalling).
Incorrect use of unsafe (including library classes such as Unsafe and Buffer which do the same thing).
Multi-threaded race conditions on objects which the Runtime does not expect to be used multi-threaded. This can cause such problems as torn reads and memory-barrier violations.
A bug in .NET itself. This can be the easiest to exclude: just upgrade to the latest build.
Consider submitting the crash report to Microsoft for investigation.
Edit from the author:
In order to submit a crash report to Microsoft, the following URL can be used: https://www.microsoft.com/en-us/unifiedsupport. Take into account that this is a paying service and that you might need to deliver your entire source code Microsoft in order to get a full analysis of your crash dump.
Is it instantaneous after the GC runs?
I was monitoring Memory (Available MBytes) using perfmon while running an application I'm working on. As I loaded files I could see the Available MBytes going down as expected, but as I closed them it didn't go back up so I'm wondering if I am not disposing correctly or if there is something else that affects this.
When a file is loaded:
the contents are stored in some objects
a new instance of a UserControl is created
a new TabPage is created
the UserControl is added to the TabPage
the TabPage is added to the TabControl
Reading on SO I saw that one of the most common things that can cause Memory Leaks is not unsubscribing from event handlers and so I made sure I did that in my UserControl's dispose method. I subscribe to the 'RemovingTab' event of the TabControl but the TabPages themselves have no attached handlers.
When closing a file, I remove stored instances of the objects and call dispose on both the UserControl and the TabPage. I tried running the CLR profiler (which I haven't had any experience with) and it said there was no GC runs.
Another common source of memory not being released is holding onto unmanaged resources, open files being one example.
The memory is available after the GC cleans up whatever is using it. Not every GC run results in memory being released. Any object that survives at least one GC run can end up living a long time because it's no longer in generation 0 - those could end up being around until the app actually needs more memory than is currently available to it. Large objects (larger than 85kb, as far as I remember) are stored on the large object heap which also isn't part of every GC run.
Your best bet is to look at what is actually holding the memory. Use the Visual Studio performance tools or try PerfView. Once you know what is actually holding the memory you'll be in a better position to figure out how to get it released.
Is it instantaneous after the GC runs?
No. As a matter of fact, depending on the memory pressure of your particular application, the GC might not free up any memory during the whole lifespan of your application.
Reading on SO I saw that one of the most common things that can cause Memory Leaks is not unsubscribing from event handlers and so I made sure I did that in my UserControl's dispose method.
Yes that is a common cause.
When closing a file, I remove stored instances of the objects and call dispose on both the UserControl and the TabPage. I tried running the CLR profiler (which I haven't had any experience with) and it said there was no GC runs.
The GC is very good at doing what it's supposed to do and it has been fined tuned to extremes. If it doesn't run its probably because it has decided that it doesn't need to run. Therefore, when the memory is freed up by the GC is up to the GC to decide and, if I'm not mistaken, is implementation defined, so it is not something you can rely upon.
And last but not least, the IDisposable pattern is not, by any means, a deterministic pattern for reclaiming memory; that is most definitely not the pattern's goal.
I am working on a .NET application where there appears to be a memory leak. I know the text-book answers, that events should be unsubscribed, disposable objects should be disposed etc...
I have a test harness that can reproduce the error. In the finalizer of a certain class I write to console
public class Foo
{
// Ctor
public Foo()
{
}
~public Foo()
{
Console.WriteLine("Foo Finalized");
}
}
In the test harness, I am creating a single instance of Foo (which in turn creates and interacts with hundreds of other types) then removing it and invoking the Garbage collector.
I am finding the Foo Finalizer is never called. I have a similar class with this setup which is finalized as a control test.
So my question is this:
How can I determine using commercial or open source tools exactly what
is holding a reference to Foo?
I have a professional license to dotTrace Memory profiler but can't figure out from the help files how to use it.
Update: I am now using dotMemory 4.0, which is the successor to the (good, but unusable) dotTrace Memory 3.5.
Have a look at the SOS debugger extension (It's free, an can be used within Visual Studio).
You may find this and this helpful to get startet.
If you have succefully set up SOS (this can be tricky sometimes), knowing what holds a reference to what is as easy as
// load sos
.load sos
// list of all instances of YourTypeName in memory with their method tables
!DumpHeap -type YourTypeName
// put here the method table displayed by the previous command
// it will show you the memory address of the object
!DumpHeap -mt 07f66b44
// displays information about references the object at the specified address
!GCRoot 02d6ec94
Debugging memory leaks can be quite involved process and requires thorough understanding of your program logic and at least some .Net internals (especially garbage collector behaviour).
For more information see the following links:
Good introduction
http://msdn.microsoft.com/en-us/library/ee658248.aspx
Hands-on course:
http://www.dotnetfunda.com/articles/article508.aspx
http://www.dotnetfunda.com/articles/article524.aspx
GC and .Net internals
http://blogs.msdn.com/b/tess/archive/2008/04/17/how-does-the-gc-work-and-what-are-the-sizes-of-the-different-generations.aspx
http://msdn.microsoft.com/en-us/magazine/cc163491.aspx
http://blogs.msdn.com/b/maoni/archive/2004/06/03/148029.aspx
WinDbg with SOS extension
http://www.codeproject.com/Articles/19490/Memory-Leak-Detection-in-NET
http://www.simple-talk.com/dotnet/.net-framework/investigating-.net-memory-management-and-garbage-collection/
Good Luck!
The finalizer isn't deterministically called, so beware of using it to track things in a reliable way. If you remove the finalizer and instead use a WeakReference<Foo> you should be able to determine whether the object was collected.
All memory profilers should be able to find an issue such as this, but with varying degree of difficulty. I have personally used ANTS which is very easy yo use, but not free. It will help you show a reference diagram to the Foo instance, all the way from a GC root object. Seeing this diagram it is usually easy to spot who is holding the reference.
You can use memory profilers to identify the memory leaks. Here are some,
MemProfiler
ANTS Profiler
Firstly you shouldn't use a finalizer, because:
Finalize operations have the following limitations:
The exact time when the finalizer executes during garbage collection
is undefined. Resources are not guaranteed to be released at any
specific time, unless calling a Close method or a Dispose method.
The finalizers of two objects are not guaranteed to run in any
specific order, even if one object refers to the other. That is, if
Object A has a reference to Object B and both have finalizers, Object
B might have already finalized when the finalizer of Object A starts.
The thread on which the finalizer is run is unspecified.
Quote from: http://msdn.microsoft.com/en-us/library/system.object.finalize.aspx
I would suggest using Dispose method instead.
Secondly, any memory profiler should be able to find what holds those references. Personally I was using ANTS Profiler, it's a very nice tool and has quite rich documentation. You can try reading this doc: http://downloads.red-gate.com/HelpPDF/ANTS_Memory_Profiler/InstanceCategorizer.pdf
Instance categorizer displays chains of references from sets of objects to GC root.
I have an application which does a bunch of text parsing. After each pass it spits out some info into a database and clears out all the internal state.
My issue is the memeory allocated in Windows Task Mgr / Resource Monitor keeps growing and growing. I've done some profile using .Net Mem Profiler and it looks like it should be going down. Here is a screen shot from the profiler:
But in the Task Mgr after each pass the memory private working set increases. I'd like for the memory to grow as its used and then return to a normal level after each pass, that way I could keep this thing running.
Any advice on what to look for or any ideas what causes this?
A few things to check for are:
Event handlers keeping objects alive, make sure that if the object which subscribes to an event goes out of scope before the object publishing the event that it unsubscribes from the event to prevent the publishing object keep a reference to it.
Make sure that you call dispose on any object which implements IDisposable. Generally speaking objects which implement IDisposable contains resources which need special tidy up.
If you reference any com objects, make sure you release them properly.
You shouldn't ever have to call GC.Collect() within production code.
Task Mgr is not an accurate representation of the memory your application is actually using. Its more a representation of how much memory windows has allotted or planned for your application- if you app needs more, windows can expand this number, but if your app needs less, windows may not re-allocate this memory until another application actually needs it..
I think what you have above is a pretty accurate representation of what your application is actually doing (ie, its not leaking).
This forces .NET to collect all unused objects from memory, therefore reclaiming some of it:
GC.Collect();
GC.WaitForPendingFinalizers();
If you do have a memory leak, you can use the SOS debugging extension to try to find it. This article is also a very good example and a little more complete then what my answer will include.
You can use this in either VS or WinDbg, and the only difference is how you load the dll. For Visual Studio, first enable unmanaged debugging in the debug tab of your project's properties. When it comes time to load it, use .load SOS.dll in the Immediate Window. For WinDbg, either open the executable or attach to the process, and, to load it, use .loadby sos clr for .NET 4 or .loadby sos mscorwks for 2 or 3.5.
After letting the application run for a while, pause it (break all). Now you can load SOS. Upon success, enter !dumpheap -stat. This will list off how much memory each class is using. If this isn't enough to find the leak, the other article I linked goes into more depth on how to locate the memory leak.
One way to solve this is break up your work into separate executable programs. Once a program completes its work and ends, all its memory will be reclaimed by the system.
All,
I am creating a component that uses composite pattern. The core component uses an XMl Meta data to define the composites (parts). at run time, the core component would use reflection to load the part assembly into the memory and call methods (e.g IPart.execute method).
Now my question is
1) will the (dynamic) memory that is occupied by the assembly that is loaded using reflection will be unloaded when i dispose the object or not.
2) if it does not unload and free the memory, is there is any way i can remove it from memory.
The reason of this question is, the component which i am building will be the core of my Business layer of an enterprise application which can be customized heavily.
Thanks
Albert Arul Prakash
I've seen people load additional libraries via reflection (we could call these libraries "plugins") in another appdomain. For instance, see this article: http://adrianvintu.com/blogengine/post/Unloadable-plugins.aspx
This way you are protected from "evil" plugins as well as memory can be managed in such appdomain (when the appdomain unloads, the memory is freed up as well)
There are two things that can consume memory in a plugin/module. Grossly speaking, these are code (loading the assembly into your process space consumes memory), and objects (creating an instance of something consumes memory).
will the (dynamic) memory that is occupied by the assembly that is loaded using reflection will be unloaded when i dispose the object or not.
Calling Dispose on IDisposable doesn't do anything to the object as far as memory is concerned. It might free resources that the object uses (e.g. if you close a file, it will get rid of open file handles), but it won't free the object itself. IDisposable isn't a magic memory-freeing function - it is just a method on an interface that lets an object know that it should get rid of resources that it owns.
To free the object itself, you must get rid of all references to it (possibly set them to null, or let them fall off your program's stack), and the garbage collector must eventually run to reclaim that memory.
if it does not unload and free the memory, is there is any way i can remove it from memory.
If you are only concerned about resources like GUI and file handles, make sure you call Dispose. You should always do this :)
If you are concerned about object memory, just let the GC do its work. Don't pester it to, either. Let it run on its own.
If you are concerned about code memory, you must unload the AppDomain that the code is in. If this is your default AppDomain then you can't unload it without quitting your program. Instead, you should load that plugin in a sub AppDomain that you created at runtime. You can then get the code out of your process space by unloading the sub AppDomain.
See naivists' answer for information on how to use a sub AppDomain.