I have got a simple ConsoleProgram which is creating a list of 80 IDisposable objects. This object hold two System.Threading.ManualResetEvent which are closed in the Dispose() method. Please consider the code:
public class Program
{
private static void Main(string[] args)
{
Console.ReadLine();
Test(80);
Console.WriteLine("end.");
Console.ReadLine();
}
private static void Test(int c)
{
Console.WriteLine("Test start.");
List<TestObject> list = new List<TestObject>();
for (int i = 0; i < c; i++)
list.Add(new TestObject());
Console.WriteLine("End of adding. Added: {0} items.", c);
Console.ReadLine();
foreach (TestObject obj in list)
obj.Dispose();
list = null;
Console.WriteLine("Dispose end.");
}
public class TestObject : IDisposable
{
public ManualResetEvent mr1 = new ManualResetEvent(true);
public ManualResetEvent mr2 = new ManualResetEvent(false);
public void Dispose()
{
mr1.Close();
mr2.Close();
}
}
}
I have tested my program in case of consumed memory and memory leaks. I supposed that consumed memory will increase after creating all list objects, but it will decrease after calling the Dispose() method and setting null value to the list object. Unfortunately I have observer different behaviour. Please consider following result of my test:
Program starts (nothing created). WorkingSet = 6.700K
List of 80 objects was created. WorkingSet = 7.160K (memory grow: 460K)
Program called the Dispose() method an set null to list object. WorkingSet = 7.164K (memory grow form last point: 4K)
Program is hanging for more than 20 minutes. WorkingSet = 7.296K (memory grow from last point: 105K, memory grow from the beginning: 596K)
Program was closed.
I am really confused about the point 3 and 4. Why the memory was not released? I think that this is a memory leak, because the total memory grow was equal to 596K and it was never released.
Thank you very much for any answer.
PS. Changing the number of object, for example to 9000, cause a memory growth of more than 2MB.
PS2. At the end of my program I am calling GC.Collect() to force the clean-up. But after that amount of used memory is still the same and it is not decreasing - I am confused.
Calling Dispose() and setting a value to null does not free the memory up immediately. Your memory should be released when the Garbage Collector next runs, but this time is not deterministic. Typically, it will happen when the application is under memory pressure, usually as a result of object creation requesting more memory. Without that pressure, and with the application idle, the GC might never collect your memory.
In short, this is not a memory leak.
Related
In a standard dispose/finalise pattern such as Finalizers with Dispose() in C# the Dispose(bool) does not touch managed objects if the method is called from the finalizer, it is considered unsafe as they may have already been collected by the garbage collector.
What is special about IntPtr etc that makes them safe?
As some background, to keep the cleanup code near the allocate code I'm adding the cleanup action to an event as soon as I allocate, then calling the event from the dispose method:
class TestClass : IDisposable
{
private IntPtr buffer;
public void test()
{
buffer = Marshal.AllocHGlobal(1024);
OnFreeUnmanagedResource += (() => Marshal.FreeHGlobal(buffer));
}
private List<IDisposable> managedObjectsToBeDisposed = new List<IDisposable>();
private event Action OnFreeUnmanagedResource = delegate { };
private bool _isDisposed = false;
private void Dispose(bool itIsSafeToAlsoFreeManagedObjects)
{
if (_isDisposed) return;
OnFreeUnmanagedResource();
if (itIsSafeToAlsoFreeManagedObjects)
for (var i = managedObjectsToBeDisposed.Count - 1; i >= 0; i--)
{
var managedObjectToBeDisposed = managedObjectsToBeDisposed[i];
managedObjectToBeDisposed.Dispose();
managedObjectsToBeDisposed.Remove(managedObjectToBeDisposed);
}
_isDisposed = true;
}
public void Dispose()
{
Dispose(true);
GC.SuppressFinalize(this);
}
~TestClass()
{
Dispose(false);
}
}
I'm uncertain of this code because OnFreeUnmanagedResource may be collected before the class, but why would this not be the case for buffer?
With that anti-pattern (really you're better off either only having managed fields or only an unmanaged field rather than mixing them and then having to be clever about how that mixing is dealt with, but alas the pattern is still with us and has to be dealt with sometimes) the danger is not that disposable managed objects may have been collected (they won't be, they're being kept alive by the field in the class in question, and now at least rooted from that object in the finalisation queue, if not elsewhere) but that they may have been finalised by their own finaliser. Or conversely, that they might get finalised here and then finalised again because they are already in the finalisation queue.
If you'd arrived at this code via Dispose() then they would not (assuming no bugs, obviously) have been cleaned up because they only path prior to a collection attempt that can clean them is that very method.
If you'd arrived at this code via the finaliser then this object has had collection attempted on it, and been put in the finalisation queue, which means that likely the objects accessible only through it also had collection attempted on it, and if finalisable had been put in the queue, and there's no guarantee of which was put there first.
If the object was disposable but not finalisable, it quite likely had fields in turn that were finalisable and likewise are likely to be on that queue.
And if the object was disposable but not finalisable, and had no finalisable fields then it doesn't matter that you don't do anything with it.
This would be OK:
private void Dispose(bool itIsSafeToAlsoFreeManagedObjects)
{
if (_isDisposed) return;
Marshal.FreeHGlobal(buffer));
if (itIsSafeToAlsoFreeManagedObjects)
for (var i = managedObjectsToBeDisposed.Count - 1; i >= 0; i--)
{
var managedObjectToBeDisposed = managedObjectsToBeDisposed[i];
managedObjectToBeDisposed.Dispose();
managedObjectsToBeDisposed.Remove(managedObjectToBeDisposed);
}
_isDisposed = true;
}
An IntPtr is a struct, that essentially contains a native handle. The resource refered to by the handle is not touched by the garbage collector. The struct itself is also still valid.
I'm not so sure about your code, where you use a managed object of reference type (the delegate attached to OnFreeUnmanagedResource) in the finalizer.
Edit: After reading the other answer, I think, your code is also OK, as the delegate doesn't have a finalizer.
Here's my code:
using System;
using System.Collections.Generic;
using System.Threading;
namespace testt
{
class MainClass
{
public static List<TestObject> testjobs = new List<TestObject> ();
public static void Main (string[] args)
{
Console.WriteLine ("Hello World!");
addTask (); //put a breakpoint here
Thread.Sleep (5000);
deleteObj ();
while (true) {
Console.WriteLine ("MAIN STILL EXISTS!");
Thread.Sleep (1500);
GC.Collect ();
}
}
public static void addTask()
{
for (int i = 0; i < 10; i++)
{
testjobs.Add (new TestObject ());
testjobs [i].Start ();
}
}
public static void deleteObj()
{
for(int i=0; i<10;i++)
{
testjobs [0].dispose ();
testjobs.RemoveAt (0);
}
Console.WriteLine (testjobs.Count);
}
}
public class TestObject
{
private bool _isStopRequested;
private Thread _thread;
public void Start()
{
_thread = new Thread(ThreadRoutine);
_thread.Start();
}
public void Stop()
{
_isStopRequested = true;
if(!_thread.Join(5000))
{
_thread.Abort();
}
}
public void dispose(){
this.Stop ();
this._thread.Abort ();
this._thread=null;
}
private void ThreadRoutine()
{
//while(!_isStopRequested) //THIS CAUSES THE MEMORY LEAK!!!
{
Thread.Sleep (1);
}
Console.WriteLine ("THREAD FINISHED");
}
~TestObject(){
Console.WriteLine ("===================TESTOBJECT DESTROYED!!===============");
}
}
}
If you run it with the //while(!_isStopRequested) uncommented, the TestObject instances will not be destroyed, i.e their destructor methods will not be called.
If you run it as is, then only 4-8 objects will be destroyed, not all 10 of them.
Why does this happen when the threads have fully exited? I checked with the Xamarin debugger and the threads were definitely stopped. If you put a breakpoint in Xamarin at addTask(); then you can see that 10 threads
My only explanation for this is that the thread somehow holds a reference back to their parent object TestObject instance even after they have finished. How can a thread hold a reference to their parent object when the thread has already finished?
Also, if I change Thread.Sleep(1) to Thread.Sleep(5000), the TestObjects also stop being collected.
Also, as it is, only some TestObjects get collected whilst others don't.
Why do these things happen? How can I ensure that ALL the TestObjects get garbage collected by the time the deleteObj() function returns?
EDIT: I just tested the exact same code in Visual Studio (.NET) and all of the objects were garbage collected regardless of whether if that line was commented out or not.
Therefore I now consider this issue to be a Mono-specific problem and there was no memory leak to begin with.
Finalizers are not deterministic. You cannot rely on them being called.
If it is vitally important for your program to clean up the resource in question then you should be explicitly disposing of it, and not relying on a finalizer.
If cleaning up the resource would be nice, but you don't really care all that much if the finializer gets to it or not, then you can choose to not explicitly dispose of the unmanaged resources.
Also note that making a managed object eligible for garbage collection doesn't necessarily mean that it will be garbage collected. It means it can be collected whenever the collector feels like it.
Finally, recognize that aborting a thread is another unreliable thing to do. The thread do. There are a number of ways for a thread that has been requested to abort will not successfully do so, or where it will cause any number of different types of problems when it does. You should avoid using Thread.Abort unless the thread in question was designed to be aborted, and you have a strong understanding of all of the many possible pitfalls of trying to reason about a program that could throw an exception between any two operations.
class A
{
public int m_a;
}
void fun(ref int a)
{
...
}
fun(ref new A().m_a);
in fun,how do "ref int a" keep the object(new A()) from being reclaimed before returning from fun?
<example 0>
using System;
class A
{
public int m_a;
~A()
{
Console.WriteLine("~A()");
}
}
class Program
{
static void fun(ref int a)
{
Console.WriteLine("Begin<<<<<<<<<<<<<<");
a++;
GC.Collect();
GC.WaitForPendingFinalizers();
Console.WriteLine("End>>>>>>>>>>>>>>>>>");
}
static void Main(string[] args)
{
fun(ref new A().m_a);
Console.WriteLine("Over");
}
}
output:
Begin<<<<<<<<<<<<<<
~A()
End>>>>>>>>>>>>>>>>>
Over
<example 1>
using System;
class A
{
public int m_a;
~A()
{
Console.WriteLine("~A()");
}
}
class Program
{
static void fun(ref int a)
{
Console.WriteLine("Begin<<<<<<<<<<<<<<");
a++;
GC.Collect();
GC.WaitForPendingFinalizers();
//add a code
a++;
Console.WriteLine("End>>>>>>>>>>>>>>>>>");
}
static void Main(string[] args)
{
fun(ref new A().m_a);
Console.WriteLine("Over");
}
}
output:
Begin<<<<<<<<<<<<<<
End>>>>>>>>>>>>>>>>>
Over
~A()
please build by release mode in VS.
I view the ASM code, only two lines are added:
a++;
0000002f mov eax,dword ptr [ebp-4]
00000032 inc dword ptr [eax]
other parts between two examples are identical. How do GC make sure the variable a is no longer usefull in machine code?
It depends how a is used in fun. The GC is able to determine if any given object is rooted. In this case a is an alias to the field m_a in an instance of A and the object is considered rooted. However, if the JIT compiler determines a is not used in the remainder of fun then from that point onward in the method, the instance of A will no longer be rooted and is eligible for garbage collection.
Some examples:
void fun(ref int a)
{
// forgot to use a. our object is already eligible for GC!
for(int i = 0; i < 10; i++)
{
Console.WriteLine(i);
}
}
void fun2(ref int a)
{
for(int i = 0; i < 10; i++)
{
Console.WriteLine(a);
}
// GC has to wait until a is no longer in use. now our object is eligible for GC.
}
void fun3(ref int a)
{
// can't gc yet. a is still being used.
int b = a;
// b has a copy of the value in a so now our object is eligible for GC.
for(int i = 0; i < 10; i++)
{
Console.WriteLine(b);
}
}
Update:
I'm not an expert in how this is implemented in the clr, but my understanding is that using ref causes a managed pointer to the field m_a to be passed into fun. When the GC runs, roots are determined from references to heap objects in statics, call stacks of all threads, and registers. I'm guessing here, but maybe that managed pointer to the field m_a stores a reference to the container object. Or maybe the GC can determine which object a given managed pointer is in. Either way the object is marked as rooted from that managed reference.
I'd thought this old presentation (Slide 30 onwards) to be sufficient, but it developed into a bit of back-and-forth in the comments section, so I thought I'd put an answer up.
Whenever the JIT prepares any method, it also constructs a "table", that maps which local variable slots are "live" at any particular point in the method. So, when the GC is examining each thread, it takes the Instruction Pointer for that thread, consults the table, and uses that to determine live references within the current method.
There is nothing written into the machine code for a particular method that has to notify the GC of anything - the JITs analysis covers all paths through the code, and only has to be done once for each method.
Under Debug, the JIT marks all variables as used for the entire body of the method - this keeps references alive longer than strictly necessary, but does mean that you can examine the state of variables after their last use in the method (via e.g. Locals or Autos windows, or any other way you may suddenly wish to reference a variable)
You would have to create the A instance beforehand and keep a reference to it, along the lines of:
A a = new A();
fun(ref a.m_a);
Otherwise, when fun returns, the new instance of A goes out of scope, and is therefore up for garbage collection.
I'm creating app which uses very large amount of memory. When I finish computation the program remains with high memory as I can notice in Task Manager.
How to reset RAM memory usage of my program?
I'm creating a large array in which I store MB worth of random characters. Then print all characters into a variable and then write the content of the variable to the file.
namespace Fake_File_Creator
{
public partial class MainWindow : Form
{
private string text;
private List<char> stringChars;
public MainWindow()
{
InitializeComponent();
}
private void btnNewFile_Click(object sender, EventArgs e)
{
sfdNewFile.Filter = "All Files|*.*";
if (sfdNewFile.ShowDialog() == DialogResult.OK)
{
lblMessage.Text = "Generating data...";
bwCreateData.RunWorkerAsync((int)nudSize.Value * 1024000);
}
}
private void bwCreateData_DoWork(object sender, DoWorkEventArgs e)
{
var random = new Random();
var chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz 0123456789 ~!##$%^&*()_+ /.,'[];{}|:<>?";
stringChars = new List<char>();
for (int i = 0; i < (int)e.Argument; i++)
{
stringChars.Add(chars[random.Next(chars.Length)]);
}
text = new string(stringChars.ToArray());
}
void bwCreateData_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
lblMessage.Text = "Saving file...";
bwCreateFile.RunWorkerAsync();
}
private void bwCreateFile_DoWork(object sender, DoWorkEventArgs e)
{
using (StreamWriter outfile = new StreamWriter(sfdNewFile.FileName))
{
outfile.Write(text);
}
}
void bwCreateFile_RunWorkerCompleted(object sender, RunWorkerCompletedEventArgs e)
{
lblMessage.Text = "File succesfully created...";
text = string.Empty;
stringChars.Clear();
stringChars.TrimExcess();
}
}
}
There is little you can do about this. The memory consumed by unused object gets freed when garbage collection happens. And you can not force garbage collection. You can read more about it on MSDN also check GC Class
Example from MSDN
private const long maxGarbage = 1000;
static void Main()
{
MyGCCollectClass myGCCol = new MyGCCollectClass();
// Determine the maximum number of generations the system
// garbage collector currently supports.
Console.WriteLine("The highest generation is {0}", GC.MaxGeneration);
myGCCol.MakeSomeGarbage();
// Determine which generation myGCCol object is stored in.
Console.WriteLine("Generation: {0}", GC.GetGeneration(myGCCol));
// Determine the best available approximation of the number
// of bytes currently allocated in managed memory.
Console.WriteLine("Total Memory: {0}", GC.GetTotalMemory(false));
// Perform a collection of generation 0 only.
GC.Collect(0);
// Determine which generation myGCCol object is stored in.
Console.WriteLine("Generation: {0}", GC.GetGeneration(myGCCol));
Console.WriteLine("Total Memory: {0}", GC.GetTotalMemory(false));
// Perform a collection of all generations up to and including 2.
GC.Collect(2);
// Determine which generation myGCCol object is stored in.
Console.WriteLine("Generation: {0}", GC.GetGeneration(myGCCol));
Console.WriteLine("Total Memory: {0}", GC.GetTotalMemory(false));
Console.Read();
}
void MakeSomeGarbage()
{
Version vt;
for(int i = 0; i < maxGarbage; i++)
{
// Create objects and release them to fill up memory
// with unused objects.
vt = new Version();
}
}
Types with resources should implement IDisposable, and freeing these resources is done by calling Dispose (or wrapping them in a using statement). GC.Collect is only necessary for objects that have resources but don't implement IDisposable.
You can't force GC
But if you want to set the memory free as soon as you are done with your array.
//I assume stringChars is the array
stringChars.Clear();
stringChars = null;
GC.Collect();
It depends on what type of memory you want to be freed.
Really calling GC.Collect(); may not work. Why?
It collects objects but you don't know which objects. Your object is large. Usually garbage collectors don't collect large objects or they do it rarely (but rarely for machine time measurement not for human).
It is very likely that virtual memory allocated for your array wouldn't be released to the OS. Because virtual machine memory managers usually reuse freed memory.
Also they perform preallocation of memory. They allocate large piece of memory and then use it. It allows not to perform a lot of system calls to get memory for new objects because .NET languages usually create a lot of them (in C++ copying is used a lot and heap is "dummy" in compare with .NET memory manager).
i have a problem disposing class in Dictionary
this is my code
private Dictionary<string, MyProcessor> Processors = new Dictionary<string, MyProcessor>();
private void button1_Click(object sender, EventArgs e)
{
if (!Processors.ContainsKey(textBox1.Text))
{
Processors.Add(textBox1.Text, new MyProcessor());
}
}
private void button2_Click(object sender, EventArgs e)
{
MyProcessor currnt_processor = Processors[textBox2.Text];
Processors.Remove(textBox2.Text);
currnt_processor.Dispose();
currnt_processor = null;
GC.Collect();
GC.WaitForFullGCComplete();
}
public class MyProcessor: IDisposable
{
private bool isDisposed = false;
string x = "";
public MyProcessor()
{
for (int i = 0; i < 20000; i++)
{
//this line only to increase the memory usage to know if the class is dispose or not
x = x + "gggggggggggg";
}
}
public void Dispose()
{
x=null;
this.Dispose(true);
GC.SuppressFinalize(this);
}
public void Dispose(bool disposing)
{
if (!this.isDisposed)
{
isDisposed = true;
this.Dispose();
}
}
~MyProcessor()
{
Dispose(false);
}
}
i use "ANTS Memory Profiler" to monitor heap memory
the disposing work only when i remove all keys from dictionary
how can i destroy the class from heap memory ?
this a video link for the problem
http://www.youtube.com/watch?v=ePorlksv2QY
thanks in advance
I think you are seeing ghosts - keep in mind that the .NET garbage collection is a generational garbage collection based on memory pressure. If there is no memory pressure your resources will not be garbage collected. Also calling GC.Collect() is just a bad idea, I hope you are only doing this for your profiling tests.
As an aside what resources exactly are your disposing in your Dispose method? Doesn't look like you need one.
In the implementation you provided not a single Dispose() method call nor the finalizer ~MyProcessor() is needed at all.
I think you might want to call a GC.WaitForFullGCComplete(); as GC.Collect(); just kicks off the GC.
It is possible to have a reference leak (ie you inadvertently keep a whole load of references to objects in a list somewhere, preventing their collection). I doubt that the dictionary has such a "leak" as it is such a widely used class that it would be a known problem. Most likely if you have problems with your code most likely the problem is somewhere else in your code.
If this is not causing you actual problems, lay off it, grab your towel and don't panic, the GC does work :)
You are removing item with one name but disposing item with another name from dictionary. So item that disposed is still referenced from dictionary (Processors.Remove(textBox2.Text); while current = dict[textBox1.Text]).
Take a note of Eric's comment - Dispose does not make object ready for garbage collection, if it has references it still will be in the memory.