We Keep Coding

What mechanisms of the C# language are used in order to pass an instance of an object to the `GC.AddMemoryPressure` method?

What mechanisms of the C# language are used in order to pass an instance of an object to the GC.AddMemoryPressure method?
I met the following code sample in the CLR via C# book:
private sealed class BigNativeResource {
private readonly Int32 m_size;
public BigNativeResource(Int32 size) {
m_size = size;
// Make the GC think the object is physically bigger
if (m_size > 0) GC.AddMemoryPressure(m_size);
Console.WriteLine("BigNativeResource create.");
}
~BigNativeResource() {
// Make the GC think the object released more memory
if (m_size > 0) GC.RemoveMemoryPressure(m_size);
Console.WriteLine("BigNativeResource destroy.");
}
}
I can not understand how do we associate an instance of an object with the pressure it adds. I do not see object reference being passed to the GC.AddMemoryPressure. Do we associate the added memory pressure (amp) with an object at all?
Also, I do not see any reasons in calling the GC.RemoveMemoryPressure(m_size);. Literally it should be of no use. Let me explain myself. There are two possibilities: there is an association between the object instance or there is no such association.
In the former case, the GC should now the m_size in order to prioritize and decide when to undertake a collection. So, it definitely should remove the memory pressure by itself (otherwise what would it mean for a GC to remove an object while taking into an account the amp?).
In the later case it is not clear what the use of the adding and removing the amp at all. The GC can only work with the roots which are by definitions instances of classes. I.e. GC only can collect the objects. So, in case there is no association between objects and the amp I see no way how the amp could affect the GC (so I assume there is an association).

I can not understand how do we associate an instance of an object with the pressure it adds.
The instance of the object associates the pressure it adds with a reference to itself by calling AddMemoryPressure. The object already has identity with itself! The code which adds and removes the pressure knows what this is.
I do not see object reference being passed to the GC.AddMemoryPressure.
Correct. There is not necessarily an association between added pressure and any object, and regardless of whether there is or not, the GC does not need to know that information to act appropriately.
Do we associate the added memory pressure (amp) with an object at all?
The GC does not. If your code does, that's the responsibility of your code.
Also, I do not see any reasons in calling the GC.RemoveMemoryPressure(m_size)
That's so that the GC knows that the additional pressure has gone away.
I see no way how the amp could affect the GC
It affects the GC by adding pressure!
I think there is a fundamental misunderstanding of what's going on here.
Adding memory pressure is just telling the GC that there are facts about memory allocation that you know, and that the GC does not know, but are relevant to the action of the GC. There is no requirement that the added memory pressure be associated with any instance of any object or tied to the lifetime of any object.
The code you've posted is a common pattern: an object has additional memory associated with each instance, and it adds a corresponding amount of pressure upon allocation of the additional memory and removes it upon deallocation of the additional memory. But there is no requirement that additional pressure be associated with a specific object or objects. If you added a bunch of unmanaged memory allocations in your static void Main() method, you might decide to add memory pressure corresponding to it, but there is no object associated with that additional pressure.

These methods exist to let GC know about memory usage outside of managed heap. There is no object to pass to these methods because memory is not directly related to any particular managed object. It's responsibility of author of the code to notify GC about change in memory usage correctly.
GC.AddMemoryPressure(Int64)
… the runtime takes into account only the managed memory, and thus underestimates the urgency of scheduling garbage collection.
Extreme example would be you have 32 bit app and GC thinks it can easily allocate almost 2GB of managed (C#) objects. As part of the code you use native interop to allocate 1GB. Without the AddMemoryPressure call GC will still think it's free to wait till you allocate/deallocate a lot of managed objects... but around the time you allocated 1GB of managed objects GC runs into strange state - it should have whole extra GB to play with but there is nothing left so it has to scramble to collect memory at that point. If AddMemoryPressure was properly used GC would had chance to adjust and more aggressively collect earlier in background or at points that allowed shorter/smaller impact.

AddMemoryPressure is used to declare (emphasis here) that you have sensible sized unmanaged data allocated somewhere. This method is a courtesy that the runtime gives to you.
The purpose of the method is to declare under your own responsibility that somewhere you have unmanaged data that is logically bound to some managed object instance. The garbage collector has a simple counter and tracks your request by simply adding the amount you specify to the counter.
The documentation is is clear about that: when the unmanaged memory goes away, you must tell the garbage collector that it has gone away.
You need to use this method to inform the garbage collector that the unmanaged memory is there but could be freed if the associated object is disposed. Then, the garbage collector is able to schedule better its collection tasks.

Should garbage collection have run by now?

I have had a C# application running for the past few weeks. When I first started it, it had ~10000K memory usage. Since then, I've checked and it's at a ~20000K memory footprint.
I don't know .Net garbage collection very well, but is this a sign that I definitely have a memory leak? Shouldn't the GC have run by now?
I've read into garbage collection here:
https://msdn.microsoft.com/en-us/library/ee787088(v=vs.110).aspx
And they have the following conditions for when GC should run:
The system has low physical memory. (I have well over 70000K of memory left to use, so this shouldn't be triggered)
The memory that is used by allocated objects on the managed heap
surpasses an acceptable threshold. This threshold is continuously
adjusted as the process runs. (how do you define "acceptable" threshold?)
Shouldn't I expect GC to have run in a few weeks timespan?

Shouldn't I expect GC to have run in a few weeks timespan?
Nope, you shouldn't expect GC.Collect to occur in timely manner.
Garbage collection occurs when one of the following conditions is
true:
The system has low physical memory.
The memory that is used by allocated objects on the managed heap surpasses an acceptable threshold. This threshold is continuously
adjusted as the process runs.
The GC.Collect method is called. In almost all cases, you do not have to call this method, because the garbage collector runs
continuously. This method is primarily used for unique situations
and testing.
Read this for more fundamental details.

C# memory leak?

I have a C# application that loops through a datatable, and pushes these into some locations such as Sage and a SQL table.
While it used to work fine, I'm inexplicably now getting Out of Memory exceptions after an hour or so of running it. I've noticed in the task manager, the memory usage rises by anbout 1mb every second, and keeps on going!
I was under the impression garbage collection would take of anything, but to be sure I ensure I dispose any objects after using them. I know without code it's hard to diagnose, but there's a lot of it and I'm looking more for general advice.

but to be sure I ensure I dispose any objects after using them
Dispose() is not directly related to memory management or leaks.
You'll have to look for unused objects that are still 'reachable'. Use a memory-profiler to find out.
You can start with the free CLR-Profiler.

There are a couple of potential problems that spring to mind:
There is a large pool of objects that are left inelegible for garbage collection (i.e. they are still "reachable"). For example if you add an object to an list in every loop then the list will grown unboundedly and each element in the list will remain inelegible for garbage collection as long as that list is still reachable. I'm not claiming that this is what is happening, this is just an example of how memory might be allocated and then left without being collected.
For some reason the garbage collector isn't doing a collection.
The high memory use is actually due to an unmanaged component that you are using in your application (e.g. via P/Invoke or COM interop).
Without seeing any code its tricky to give specific advice on how to fix your problem however reading through Investigating Memory Issues should give you some pointers on how to diagnose the memory problem yourself. In particular my first step would probably be to examine performance counters to see if the garbage collector is actually running, and to check the various heap sizes.
Note that Dispose and the IDisposable interface is unrelated to memory use - its important to dispose of objects like database connections once you are done with them as it frees up any associated resources (e.g. handles) however disposing of objects that implement IDisposable is very unlikely to have an impact on memory use.

Garbage collection can only get rid of objects that are no longer referenced from anything else. In addition it can only get rid of managed objects - it has no control about memory created from native code you may be interfacing with. These therefore are the two root causes for memory leaks in C# code.
The first thing to look at is perfmon. Get the counters for the private bytes and the .net heap size for the process. If the heap size remains flat (or rises and drops) but private bytes keeps increasing you've got some native code allocating memory and not releasing it.
If the heap size just keeps growing then the leak is in your managed code and you'll need a profiler like ANTS, DotTrace or even WinDbg (with SOS extension) to inspect the heap and see what objects are lying about.

The most popular "memory leak" on .Net platform is forgotten collection that repeatetly added in some infinite loop.

When you new something for temporary memory use.
Always use following way, it ensures calling dispose.
using (Someclass A = new Someclass())
{
....something about A
}
Someclass is a class implemented interface IDisposable
GC won't save you if there some part of unsafe code is involved(P/Invoke, Com etc..), and if there still a reference some where exists.
If you find memory leaking, use WinDbg will see what is in the heap.
This article may give you some help.
http://www.codeproject.com/KB/dotnet/Memory_Leak_Detection.aspx

What happens if I don't call Dispose on the pen object?

What happens if I don't call Dispose on the pen object in this code snippet?
private void panel_Paint(object sender, PaintEventArgs e)
{
var pen = Pen(Color.White, 1);
//Do some drawing
}

A couple of corrections should be made here:
Regarding the answer from Phil Devaney:
"...Calling Dispose allows you to do deterministic cleanup and is highly recommended."
Actually, calling Dispose() does not deterministically cause a GC collection in .NET - i.e. it does NOT trigger a GC immediately just because you called Dispose(). It only indirectly signals to the GC that the object can be cleaned up during the next GC (for the Generation that the object lives in). In other words, if the object lives in Gen 1 then it wouldn't be disposed of until a Gen 1 collection takes place. One of the only ways (though not the only) that you can programmatically and deterministically cause the GC to perform a collection is by calling GC.Collect(). However, doing so is not recommended since the GC "tunes" itself during runtime by collecting metrics about your memory allocations during runtime for your app. Calling GC.Collect() dumps those metrics and causes the GC to start its "tuning" all over again.
Regarding the answer:
IDisposable is for disposing unmanaged resources. This is the pattern in .NET.
This is incomplete. As the GC is non-deterministic, the Dispose Pattern, (How to properly implement the Dispose pattern), is available so that you can release the resources you are using - managed or unmanaged. It has nothing to do with what kind of resources you are releasing. The need for implementing a Finalizer does have to do with what kind of resources you are using - i.e. ONLY implement one if you have non-finalizable (i.e. native) resources. Maybe you are confusing the two. BTW, you should avoid implementing a Finalizer by using the SafeHandle class instead which wraps native resources which are marshaled via P/Invoke or COM Interop. If you do end up implementing a Finalizer, you should always implement the Dispose Pattern.
One critical note which I haven't seen anyone mention yet is that if disposable object is created and it has a Finalizer (and you never really know whether they do - and you certainly shouldn't make any assumptions about that), then it will get sent directly to the Finalization Queue and live for at least 1 extra GC collection.
If GC.SuppressFinalize() is not ultimately called, then the finalizer for the object will be called on the next GC. Note that a proper implementation of the Dispose pattern should call GC.SuppressFinalize(). Thus, if you call Dispose() on the object, and it has implemented the pattern properly, you will avoid execution of the Finalizer. If you don't call Dispose() on an object which has a finalizer, the object will have its Finalizer executed by the GC on the next collection. Why is this bad? The Finalizer thread in the CLR up to and including .NET 4.6 is single-threaded. Imagine what happens if you increase the burden on this thread - your app performance goes to you know where.
Calling Dispose on an object provides for the following:
reduce strain on the GC for the process;
reduce the app's memory pressure;
reduce the chance of an OutOfMemoryException (OOM) if the LOH (Large Object Heap) gets fragmented and the object is on the LOH;
Keep the object out of the Finalizable and f-reachable Queues if it has a Finalizer;
Make sure your resources (managed and unmanaged) are cleaned up.
Edit:
I just noticed that the "all knowing and always correct" MSDN documentation on IDisposable (extreme sarcasm here) actually does say
The primary use of this interface is
to release unmanaged resources
As anyone should know, MSDN is far from correct, never mentions or shows 'best practices', sometimes provides examples that don't compile, etc. It is unfortunate that this is documented in those words. However, I know what they were trying to say: in a perfect world the GC will cleanup all managed resources for you (how idealistic); it will not, however cleanup unmanaged resources. This is absolutely true. That being said, life is not perfect and neither is any application. The GC will only cleanup resources that have no rooted-references. This is mostly where the problem lies.
Among about 15-20 different ways that .NET can "leak" (or not free) memory, the one that would most likely bite you if you don't call Dispose() is the failure to unregister/unhook/unwire/detach event handlers/delegates. If you create an object that has delegates wired to it and you don't call Dispose() (and don't detach the delegates yourself) on it, the GC will still see the object as having rooted references - i.e. the delegates. Thus, the GC will never collect it.
#joren's comment/question below (my reply is too long for a comment):
I have a blog post about the Dispose pattern I recommend to use - (How to properly implement the Dispose pattern). There are times when you should null out references and it never hurts to do so. Actually, doing so does do something before GC runs - it removes the rooted reference to that object. The GC later scans its collection of rooted references and collects those that do not have a rooted reference. Think of this example when it is good to do so: you have an instance of type "ClassA" - let's call it 'X'. X contains an object of type "ClassB" - let's call this 'Y'. Y implements IDisposable, thus, X should do the same to dispose of Y. Let's assume that X is in Generation 2 or the LOH and Y is in Generation 0 or 1. When Dispose() is called on X and that implementation nulls out the reference to Y, the rooted reference to Y is immediately removed. If a GC happens for Gen 0 or Gen 1, the memory/resources for Y is cleaned up but the memory/resources for X is not since X lives in Gen 2 or the LOH.

The Pen will be collected by the GC at some indeterminate point in the future, whether or not you call Dispose.
However, any unmanaged resources held by the pen (e.g., a GDI+ handle) will not be cleaned up by the GC. The GC only cleans up managed resources. Calling Pen.Dispose allows you to ensure that these unmanaged resources are cleaned up in a timely manner and that you aren't leaking resources.
Now, if the Pen has a finalizer and that finalizer cleans up the unmanaged resources then those said unmanaged resources will be cleaned up when the Pen is garbage collected. But the point is that:
You should call Dispose explicitly so that you release your unmanaged resources, and
You shouldn't need to worry about the implementation detail of if there is a finalizer and it cleans up the unmanaged resources.
Pen implements IDisposable. IDisposable is for disposing unmanaged resources. This is the pattern in .NET.
For previous comments on the this topic, please see this answer.

The underlying GDI+ pen handle will not be released until some indeterminate time in the future i.e. when the Pen object is garbage collected and the object's finalizer is called. This might not be until the process terminates, or it might be earlier, but the point is its non-deterministic. Calling Dispose allows you to do deterministic cleanup and is highly recommended.

The total amount of .Net memory in use is the .Net part + all 'external' data in use. OS objects, open files, database and network connections all take some resources that are not purely .Net objects.
Graphics uses Pens and other objects wich are actually OS objects that are 'quite' expensive to keep around. (You can swap your Pen for a 1000x1000 bitmap file). These OS objects only get removed from the OS memory once you call a specific cleanup function. The Pen and Bitmap Dispose functions do this for you immediatly when you call them.
If you don't call Dispose the garbage collector will come to clean them up 'somewhere in the future*'.
(It will actually call the destructor/finalize code that probably calls Dispose())
*on a machine with infinite memory (or more then 1GB) somewhere in the future can be very far into the future. On a machine doing nothing it can be easily longer then 30 minutes to clean up that huge bitmap or very small pen.

If you really want to know how bad it is when you don't call Dispose on graphics objects you can use the CLR Profiler, available free for the download here. In the installation folder (defaults to C:\CLRProfiler ) is CLRProfiler.doc which has a nice example of what happens when you don't call Dispose on a Brush object. It is very enlightening. The short version is that graphics objects take up a larger chunk of memory than you might expect and they can hang around for a long time unless you call Dispose on them. Once the objects are no longer in use the system will, eventually, clean them up, but that process takes up more CPU time than if you had just called Dispose when you were finished with the objects.
You may also want to read up on using IDisposable here and here.

It will keep the resources until the garbage collector cleans it up

Depends if it implements finalizer and it calls the Dispose on its finalize method. If so, handle will be released at GC.
if not, handle will stay around until process is terminated.

With graphic stuff it can be very bad.
Open the Windows Task Manager. Click "choose columns" and choose column called "GDI Objects".
If you don't dispose certain graphic objects, this number will keep raising and raising.
In older versions of Windows this can crash the whole application (limit was 10000 as far as I remember), not sure about Vista/7 though but it's still a bad thing.

the garbage collector will collect it anyway BUT it matters WHEN:
if you dont call dispose on an object that you dont use it will live longer in memory and gets promoted to higher generations meaning that collecting it has a higher cost.

in back of my mind first idea came to the surface is that this object will be disposed as soon as the method finishes execution!, i dont know where did i got this info!, is it right?

Question about the garbage collector in .NET (memory leak)

I guess this is very basic but since I'm learning .NET by myself, I have to ask this question.
I'm used to coding in C, where you have to free() everything. In C++/.NET, I have read about the garbage collector. From what I understand, when an instance is no longer used (in the scope of the object), it is freed by the garbage collector.
So, having that in mind, I built a little testing application. But, it seems I didn't get something because when doing the same things a few times (like, opening a form, closing it, reopening it, etc), memory leaks. And big time.
I tried to look this up on Google but I didn't find anything good for a beginner.
Is the garbage collector really freeing every objects when no longer used or there are exceptions that I have to handle? What am I missing?
Are there free tools to look up for memory leaks?

Yeah, the garbage collector is freeing your objects when they're not used anymore.
What we usually call a memory leak in .NET is more like:
You're using external resources (that are not garbage collected) and forgetting to free them. This is solved usually by implementing the IDisposable interface.
You think there aren't references to a given object but indeed there are, somewhere and you're not using them any more but the garbage collector does not know about them.
In addition, memory is only reclaimed when needed, meaning the garbage collector activates at given times and performs a collection determining them what memory can be freed and freeing it. Until it does, the memory isn't claimed so it might look like memory is being lost.
Here, I think I'll provide a more complex answer just to clarify.
First, the garbage collector runs in its own thread. You have to understand that, in order do reclaim memory the garbage collector needs to stop all other threads so that he can follow up the reference chain an determine what depends on what. That's the reason memory isn't freed right away, a garbage collector imply certain costs in performance.
Internally the garbage collector manage memory in generations. In a very simplified way there are several generations for long lived, short lived and big size objects. The garbage collector moves the object from one generation to another each time its performs a collection which happens more often for short lived generation that for long lived one. It also reallocates objects to get you the most possible contiguous space so again, performing a collection is a costly process.
If you really want to see the effects of you freeing the form (when getting out of scope and having no more reference to it) you can call GC.Collect(). Do that just for testing, it's highly unwise to call Collect except for a very few cases where you know exactly what you're doing and the implications it will have.
A little more explaining about the Dispose method of the IDispose interface.
Dispose isn't a destructor in the usual C++ way, it isn't a destructor at all. Dispose is a way to deterministically get rid of unmanaged objects. An example: Suppose you call an external COM library that happens to allocate 1GB of memory due to what it is doing. If you have no Dispose that memory will sit there, wasting space until the GC inits a collection and reclaims the unmanaged memory by calling the actual object destructor. So if you want to free the memory right away you have to call the Dispose method but you're not "forced" to do so.
If you don't use IDisposable interface then you have to free you're unmanaged resources in the Finalize method. Finalize is automatically called from the object destructor when the GC attempts to reclaim the object. So if you have a proper finalize method the unmanaged memory will get freed either way. Calling Dispose will only make it deterministic.

What leads you to conclude that there are memory leaks? Under garbage collection, there is no guarantee that memory is freed immediately, and in general the GC doesn't kick in until your process memory reaches some threshold, or the available heap has been exhausted. (The exact heuristic is complicated and not important.) So the fact that your process's memory goes up and doesn't go down doesn't necessarily mean that there's a bug. It might just be that the GC didn't get around to cleaning up your process yet.
Additionally, are you sure that there are no references to your objects? It's possible that you have references that you aren't aware of. Most memory leaks in .NET applications are because people don't realize that their memory is still being referenced somewhere.

Task Manager is a terrible way to examine your memory usage. If you want to study how the garbage collector works, install the CLR Profiler and use it to analyze your application. It will tell you exactly what the garbage collector is doing.
See How To: Use CLR Profiler.

I'm adding this as an answer rather than a comment on the question, but this follows-up a question asked by the OP in a comment:
using statement on MSDN.
IDisposable interface on MSDN.
Here is the crux of the issue: what you're used to as far as object destructors is gone. Everything you've been told about how to code correctly is screaming up from your subconscious, saying this can't be true, and if it is true, it's wrong and terrible. It's very different; it's hard to remember how much I really despised it at first (I was a proud C++ developer).
I personally promise you: it's going to be OK!
Here's another good thing to read:
Destructors and Finalizers in Visual C++.

The GC will not necessarily actually free things at the moment the object is no longer referenced. The GC will collect it at some time in the future - you don't know exactly when, but if memory is needed, the GC will perform a collection if necessary.

If you just want to figure out if you have a memory leak or not, have a look at perfmon which ships with your copy of windows:
In particular the .NET CLR Memory counter bytes in all heaps, if this number is steadily growing you have a leak.
You can even dig deeper by comparing the Gen 2 heap size to the Large Object Heap Size. If the former is growing steadily you have a large blobs of data leaking.
Once you confirm there is a leak, you can attach with windbg and use the sos extensions to determine what is leaking.
If you can afford to spend a few bucks have a look at the .NET Memory Profiler.

There are free tools available to look at the managed heap in .Net, using the SOSEX extensions to WinDBG and SOS, it is possible to run a program, pause it, then look at which objects exist on the stack and (more importantly) which other objects are holding references to them (roots) which will be preventing the the GC from collecting them.