We Keep Coding

.NET Free memory usage (how to prevent overallocation / release memory to the OS)

I'm currently working on a website that makes large use of cached data to avoid roundtrips.
At startup we get a "large" graph (hundreds of thouthands of different kinds of objects).
Those objects are retrieved over WCF and deserialized (we use protocol buffers for serialization)
I'm using redgate's memory profiler to debug memory issues (the memory didn't seem to fit with how much memory we should need "after" we're done initializing and end up with this report
Now what we can gather from this report is that:
1) Most of the memory .NET allocated is free (it may have been rightfully allocated during deserialisation, but now that it's free, i'd like for it to return to the OS)
2) Memory is fragmented (which is bad, as everytime i refresh the cash i need to redo the memory hungry deserialisation process and this, in turn creates large object that may throw an OutOfMemoryException due to fragmentation)
3) I have no clue why the space is fragmented, because when i look at the large object heap, there are only 30 instances, 15 object[] are directly attached to the GC and totally unrelated to me, 1 is a char array also attached directly to the GC Heap, the remaining 15 are mine but are not the cause of this as i get the same report if i comment them out in code.
So my question is, what can i do to go further with this? I'm not really sure what to look for in debugging / tools as it seems my memory is fragmented, but not by me, and huge amounts of free spaces are allocated by .net , which i can't release.
Also please make sure you understand the question well before answering, i'm not looking for a way to free memory within .net (GC.Collect), but to free memory that is already free in .net , to the system as well as to defragment said memory.
Note that a slow solution is fine, if it's possible to manually defragment the large heap i'd be all for it as i can call it at the end of RefreshCache and it's ok if it takes 1 or 2 second to run.
Thanks for your help!
A few notes i forgot:
1) The project is a .net 2.0 website, i get the same results running it in a .net 4 pool, idem if i run it in a .net 4 pool and convert it to .net 4 and recompile.
2) These are results of a release build, so debug build can not be the issue.
3) And this is probably quite important, i do not get these issues at all in the webdev server, only in IIS, in the webdev i get memory consumption rather close to my actual consumption (well more, but not 5-10X more!)

Objects allocated on the large object heap (objects >= 85,000 bytes, normally arrays) are not compacted by the garbage collector. Microsoft decided that the cost of moving those objects around would be too high.
The recommendation is to reuse large objects if possible to avoid
fragmentation on the managed heap and the VM space.
http://msdn.microsoft.com/en-us/magazine/cc534993.aspx
I'm assuming that your large objects are temporary byte arrays created by your deserialization library. If the library allows you to supply your own byte arrays, you could preallocate them at the start of the program and then reuse them.

I know this isn't the answer you'd like to hear, but you can't forcefully release the memory back to the OS. However, for what reason do you want to do so? .NET will free its heap back to the OS once you're running low on physical memory. But if there's an ample amount of free physical memory, .NET will keep its heap to make future allocation of objects faster. If you really wanted to force .NET to release its heap back to the OS, I suppose you could write a C program which just mallocs until it runs out of memory. This should cause the OS to signal .NET to free its unused portion of the heap.
It's better that unused memory be reseved for .NET so that your application will have better allocation performance (since the runtime knows what memory is free and what isn't, allocation can just use the free memory without having to syscall into the OS to get more memory).
The garbage collector is in charge of defragmenting the heap. Every so often (usually during collection runs), it will move objects around the heap if it determines this needs to be done. (This is why C++/CLI has the pin_ptr construct for "pinning" objects).
Fragmentation usually isn't a big issue though with memory, since it provides fast random access.
As for your OutOfMemoryException, I don't have a good answer for. Ordinarily I'd suspect that your old object graph isn't being collected (some object somewhere is holding a reference onto it, a "memory leak"). But since you're using a profiler, I don't know then.

As of .NET 4.5.1 you can set a one-time flag to compact LOH before issuing a call to GC collect, i.e.
Runtime.GCSettings.LargeObjectHeapCompactionMode = System.Runtime.GCLargeObjectHeapCompactionMode.CompactOnce;
GC.Collect(); // This will cause the LOH to be compacted (once).

Some testing and some C++ later, i've found the reason why i get so much free memory, it's because of IIS instancing the CLR via VM Hoarding (providing a dll to instantiate it without VM Hoarding takes up as much initial memory, but does release most of it as time goes which is the behavior i expect).
So this does fix my reported memory issue, however i still get about 100mb free memory no matter what, and i still think this is due to fragmentation and fragments only being released at once, because the profiler still reports memory fragmentation. So not marking my own answer as an answer in hope someone can shed some light on this or direct me to tools that can either fix this or help me debug the root cause.

It's intriguing that it works differently on the WebDevServer as to IIS...
Is it possible that IIS is using the server garbage-collector, and the WebDev server the workstation garbage collector? The method of garbage collection can affect fragmentation. It'll probably be set in your aspnet.config file. See: http://support.microsoft.com/kb/911716

If you havent found your answer I think the following clues can help you :
Back to the basics : we sometimes forget that the objects can be explicitly set free, call explicitly the Dispose method of the objects (because you didnt mention it, I suppose you do an "object = null" instruction instead).
Use the inherited method, you dont need to implement one, unless your class doesnt have it, which I doubt it.
MSDN Help states about this method :
... There is no performance benefit in implementing the Dispose
method on types that use only managed resources (such as arrays)
because they are automatically reclaimed by the garbage collector. Use
the Dispose method primarily on managed objects that use native
resources and on COM objects that are exposed to the .NET
Framework. ...
Because it says that "they are automatically reclaimed by garbage collector" we can infer that when the method is called does the "releasing thing" (Again Im trying only to give you clues).
Besides I found this interesting article (I suppose ... I didn read it ...completely) : Garbage Collection: Automatic Memory Management in the Microsoft .NET Framework (http://msdn.microsoft.com/en-us/magazine/bb985010.aspx) which states the following in the "Forcing an Object to Clean Up" section :
..., it is also recommended that you add an additional method to
the type that allows a user of the type to explicitly clean up the
object when they want. By convention, this method should be called
Close or Dispose ....
Maybe the answer lies in this article if you read it carefully or just keep investigating in this direction.

How can I tell if the .Net 3.5 garbage collector has run?

I have an application that creates trees of nodes, then tosses them and makes new trees.
The application allocates about 20 MB upon startup. But I tried loading a large file of nodes many times, and the allocated memory went above 700 MB. I thought I would see memory being freed by the garbage collector on occasion.
The machine I'm on has 12 GB of RAM, so maybe it's just that such a "small" amount of memory being allocated doesn't matter to the GC.
I've found a lot of good info on how the GC works, and that it's best not to tell it what to do. But I wanted to verify that it's actually doing anything, and that I'm not somehow doing something wrong in the code that prevents my objects from being cleaned up.

The GC generally runs when either of the scenarios below occur:
You call GC.Collect (which you shouldn't)
Gen0's budget is exhausted
There are some other scenarios as well, but I'll skip those for now.
You didn't tell us how you measured the memory usage, but if you're looking at the memory usage of process itself (e.g. through task manager), then you may not see the numbers you expect. Remember that the .NET runtime essentially has its own memory manager that handles memory usage on behalf of you managed application. The runtime tries to be smart about it so it doesn't allocate and free memory to the OS all the time (those are expensive operations). This question may be relevant as well.
If you're concerned about memory leaks have a look at some of the answers here.

When does the .Net 3.5 garbage collector run?
I thought I would see memory being freed by the garbage collector on occasion.
Since the GC is non-deterministic, you won't be able to necessarily determine when it is going to issue a collection. Short answer: It will run when needed. Trying to analyze your code and predict or assume it should be running at a certain time usually ends up down a rabbit hole.

Answer to: do I leak objects or GC have not need to run yet?
Use memory profiler to see what objects are allocated. As basic step - force garbage collection (GC.Collect) and check out if allocated memory (GC.GetTotalMemory) seems to be reasonable.

If you want to make sure that you're not leaving any unwanted object behind you can use dotTrace memory profiler. It allows you to take two snapshots of objects in memory (taken some time apart) and compare them. You will can clearly see if any old nodes are still hanging around and what is keeping a reference to them and stops them from being collected.

You may have the managed equivalent of a memory leak. Are you maintaining stale references to these objects (i.e., do you have a List<T> or some other object which tracks these nodes)?
Are the subscribing to an event of an object that is not going out of scope? A reference to the subscribee of an event is maintained, so if you don't detach it will keep your objects alive.
You may also be forgetting to Dispose of objects that implement IDisposable. Can't say without seeing your code.
The exact behavior of the GC is implementation defined. You should design your application such that it does not matter. If you need deterministic memory management then you are using the wrong language. Use a tool (RedGate's ANTS profiler will do) to see if you are leaking references somewhere.

This comic is the best way to explain this topic :)

You can monitor the garbage collector activity in .NET 4.0 and beyond with GC.RegisterForFullGCNotification as described in this link: http://www.abhisheksur.com/2010/08/garbage-collection-notifications-in-net.html

Want to understand how a managed language like C# has memory leaks

Since C# is a managed language that performs garbage collection automatically to clean up objects etc, ...
what are the ways one can introduce a memory leak?
Are there some non-obvious ways that one should look out for?
How can you detect or look for memory leaks (once you understand how they are generated etc.)

Usually leaks show up in the form of a developer writing code that "holds on" to objects when they shouldn't be, which subsequently disallows the garbage collector from collecting on those objects.
The garbage collector is pretty good at what it does, but if you don't understand what it's doing, the likelihood of you introducing memory issues into your program is pretty high.
I would suggest reading up on the GC and understanding how it works.
Here's something to get you started:
http://www.simple-talk.com/dotnet/.net-framework/understanding-garbage-collection-in-.net/

Two.
First, building up referenced objects. Like creating objects that subscribe to an event on a form. The form is active, so can not be collected, and the event subscriptions... keep the objects alive.
Second, block the garbage colelctor in native code. Like the oracle ODP.NET driever does under certain conditions. Stops the finalizer, so any object requiring finalization WILL NOT GET IT - and thus never be released.

The obvious "memory leak" one could cause in a GC-ed language would be caused simply be retaining a reference to an object after it's needed - this is especially likely if you roll your own caching or keep other global state.
Another way would be leaking memory in unmanaged resources that weren't disposed of, although most of the standard library classes will probably dispose of those in destructors so the memory will be reclaimed sooner or later.
(I'm marking the post as CW because of the open-ended nature of the question.)

A memory leak means keeping in memory objects you don't need anymore. In C#, considering the GC collects unreferenced objects, it's equivalent to say keeping references to objects you don't need.
Think about uncorrectly scoped declarations, infinite recursion or iteration...

Question about the garbage collector in .NET (memory leak)

I guess this is very basic but since I'm learning .NET by myself, I have to ask this question.
I'm used to coding in C, where you have to free() everything. In C++/.NET, I have read about the garbage collector. From what I understand, when an instance is no longer used (in the scope of the object), it is freed by the garbage collector.
So, having that in mind, I built a little testing application. But, it seems I didn't get something because when doing the same things a few times (like, opening a form, closing it, reopening it, etc), memory leaks. And big time.
I tried to look this up on Google but I didn't find anything good for a beginner.
Is the garbage collector really freeing every objects when no longer used or there are exceptions that I have to handle? What am I missing?
Are there free tools to look up for memory leaks?

Yeah, the garbage collector is freeing your objects when they're not used anymore.
What we usually call a memory leak in .NET is more like:
You're using external resources (that are not garbage collected) and forgetting to free them. This is solved usually by implementing the IDisposable interface.
You think there aren't references to a given object but indeed there are, somewhere and you're not using them any more but the garbage collector does not know about them.
In addition, memory is only reclaimed when needed, meaning the garbage collector activates at given times and performs a collection determining them what memory can be freed and freeing it. Until it does, the memory isn't claimed so it might look like memory is being lost.
Here, I think I'll provide a more complex answer just to clarify.
First, the garbage collector runs in its own thread. You have to understand that, in order do reclaim memory the garbage collector needs to stop all other threads so that he can follow up the reference chain an determine what depends on what. That's the reason memory isn't freed right away, a garbage collector imply certain costs in performance.
Internally the garbage collector manage memory in generations. In a very simplified way there are several generations for long lived, short lived and big size objects. The garbage collector moves the object from one generation to another each time its performs a collection which happens more often for short lived generation that for long lived one. It also reallocates objects to get you the most possible contiguous space so again, performing a collection is a costly process.
If you really want to see the effects of you freeing the form (when getting out of scope and having no more reference to it) you can call GC.Collect(). Do that just for testing, it's highly unwise to call Collect except for a very few cases where you know exactly what you're doing and the implications it will have.
A little more explaining about the Dispose method of the IDispose interface.
Dispose isn't a destructor in the usual C++ way, it isn't a destructor at all. Dispose is a way to deterministically get rid of unmanaged objects. An example: Suppose you call an external COM library that happens to allocate 1GB of memory due to what it is doing. If you have no Dispose that memory will sit there, wasting space until the GC inits a collection and reclaims the unmanaged memory by calling the actual object destructor. So if you want to free the memory right away you have to call the Dispose method but you're not "forced" to do so.
If you don't use IDisposable interface then you have to free you're unmanaged resources in the Finalize method. Finalize is automatically called from the object destructor when the GC attempts to reclaim the object. So if you have a proper finalize method the unmanaged memory will get freed either way. Calling Dispose will only make it deterministic.

What leads you to conclude that there are memory leaks? Under garbage collection, there is no guarantee that memory is freed immediately, and in general the GC doesn't kick in until your process memory reaches some threshold, or the available heap has been exhausted. (The exact heuristic is complicated and not important.) So the fact that your process's memory goes up and doesn't go down doesn't necessarily mean that there's a bug. It might just be that the GC didn't get around to cleaning up your process yet.
Additionally, are you sure that there are no references to your objects? It's possible that you have references that you aren't aware of. Most memory leaks in .NET applications are because people don't realize that their memory is still being referenced somewhere.

Task Manager is a terrible way to examine your memory usage. If you want to study how the garbage collector works, install the CLR Profiler and use it to analyze your application. It will tell you exactly what the garbage collector is doing.
See How To: Use CLR Profiler.

I'm adding this as an answer rather than a comment on the question, but this follows-up a question asked by the OP in a comment:
using statement on MSDN.
IDisposable interface on MSDN.
Here is the crux of the issue: what you're used to as far as object destructors is gone. Everything you've been told about how to code correctly is screaming up from your subconscious, saying this can't be true, and if it is true, it's wrong and terrible. It's very different; it's hard to remember how much I really despised it at first (I was a proud C++ developer).
I personally promise you: it's going to be OK!
Here's another good thing to read:
Destructors and Finalizers in Visual C++.

The GC will not necessarily actually free things at the moment the object is no longer referenced. The GC will collect it at some time in the future - you don't know exactly when, but if memory is needed, the GC will perform a collection if necessary.

If you just want to figure out if you have a memory leak or not, have a look at perfmon which ships with your copy of windows:
In particular the .NET CLR Memory counter bytes in all heaps, if this number is steadily growing you have a leak.
You can even dig deeper by comparing the Gen 2 heap size to the Large Object Heap Size. If the former is growing steadily you have a large blobs of data leaking.
Once you confirm there is a leak, you can attach with windbg and use the sos extensions to determine what is leaking.
If you can afford to spend a few bucks have a look at the .NET Memory Profiler.

There are free tools available to look at the managed heap in .Net, using the SOSEX extensions to WinDBG and SOS, it is possible to run a program, pause it, then look at which objects exist on the stack and (more importantly) which other objects are holding references to them (roots) which will be preventing the the GC from collecting them.

What are ways to solve Memory Leaks in C#

I'm learning C#. From what I know, you have to set things up correctly to have the garbage collector actually delete everything as it should be. I'm looking for wisdom learned over the years from you, the intelligent.
I'm coming from a C++ background and am VERY used to code-smells and development patterns. I want to learn what code-smells are like in C#. Give me advice!
What are the best ways to get things deleted?
How can you figure out when you have "memory leaks"?
Edit: I am trying to develop a punch-list of "stuff to always do for memory management"
Thanks, so much.

C#, the .NET Framework uses Managed Memory and everything (but allocated unmanaged resources) is garbage collected.
It is safe to assume that managed types are always garbage collected. That includes arrays, classes and structures. Feel free to do int[] stuff = new int[32]; and forget about it.
If you open a file, database connection, or any other unmanaged resource in a class, implement the IDisposable interface and in your Dispose method de-allocate the unmanaged resource.
Any class which implements IDisposable should be explicitly closed, or used in a (I think cool) Using block like;
using (StreamReader reader = new StreamReader("myfile.txt"))
{
... your code here
}
Here .NET will dispose reader when out of the { } scope.

The first thing with GC is that it is non-deterministic; if you want a resource cleaned up promptly, implement IDisposable and use using; that doesn't collect the managed memory, but can help a lot with unmanaged resources and onward chains.
In particular, things to watch out for:
lots of pinning (places a lot of restrictions on what the GC can do)
lots of finalizers (you don't usually need them; slows down GC)
static events - easy way to keep a lot of large object graphs alive ;-p
events on an inexpensive long-life object, that can see an expensive object that should have been cleaned up
"captured variables" accidentally keeping graphs alive
For investigating memory leaks... "SOS" is one of the easiest routes; you can use SOS to find all instances of a type, and what can see it, etc.

In general, the less you worry about memory allocation in C#, the better off you are. I would leave it to a profiler to tell me when I'm having issues with collection.
You can't create memory leaks in C# in the same way as you do in C++. The garbage collector will always "have your back". What you can do is create objects and hold references to them even though you never use them. That's a code smell to look out for.
Other than that:
Have some notion of how frequently collection will occur (for performance reasons)
Don't hold references to objects longer than you need
Dispose of objects that implement IDisposable as soon as you're done with them (use the using syntax)
Properly implement the IDisposable interface

The main sources of memory leaks I can think of are:
keeping references to objects you don't need any more (usually in some sort of collection) So here you need to remember that all things that you add to a collection that you have reference too will stay in memory.
Having circular references, e.g. having delegates registered with an event. So even though you explicitly don't reference an object, it can't get garbage collected because one of its methods is registered as a delegate with an event. In these cases you need to remember to remove the delegate before discarding the reference.
Interoperating with native code and failing to free it. Even if you use managed wrappers that implement finalizers, often the CLR doesn't clean them fast enough, because it doesn't understand the memory footprint. You should use the using(IDisposable ){} pattern

One other thing to consider for memory management is if you are implementing any Observer patterns and not disposing of the references correctly.
For instance:
Object A watches Object B
Object B is disposed if the reference from A to B is not disposed of property the GC will not properyly dispose of the object. Becuase the event handler is still assigned the GC doesn't see it as a non utilized resource.
If you have a small set of objects you're working with this may me irrelevant. However, if your working with thousands of objects this can cause a gradual increase in memory over the life of the application.
There are some great memory management software applications to monitor what's going on with the heap of your application. I found great benefit from utilizing .Net Memory Profiler.
HTH

I recommend using .NET Memory Profiler
.NET Memory Profiler is a powerful tool for finding memory leaks and optimizing the memory usage in programs written in C#, VB.NET or any other .NET Language.
.NET Memory Profiler will help you to:
View real-time memory and resource information
Easily identify memory leaks by collecting and comparing snapshots of .NET memory
Find instances that are not properly disposed
Get detailed information about unmanaged resource usage
Optimize memory usage
Investigate memory problems in production code
Perform automated memory testing
Retrieve information about native memory
Take a look at their video tutorials:
http://memprofiler.com/tutorials/

Others have already mentioned the importance of IDisposable, and some of the things to watch out for in your code.
I wanted to suggest some additional resources; I found the following invaluable when learning the details of .NET GC and how to trouble-shoot memory issues in .NET applications.
CLR via C# by Jeffrey Richter is an excellent book. Worth the purchase price just for the chapter on GC and memory.
This blog (by a Microsoft "ASP.NET Escalation Engineer") is often my go-to source for tips and tricks for using WinDbg, SOS, and for spotting certain types of memory leaks. Tess even designed .NET debugging demos/labs which will walk you through common memory issues and how to recognize and solve them.
Debugging Tools for Windows (WinDbg, SOS, etc)

You can use tools like CLR profiler it takes some time to learn how to use it correctly, but after all it is free. (It helped me several times to find my memory leakage)

The best way to ensure that objects get deleted, or in .NET lingo, garbage-collected, is to ensure that all root references (references that can be traced through methods and objects to the first method on a thread's call stack) to an object are set to null.
The GC cannot, and will not, collect an object if there are any rooted references to it, no matter whether it implements IDisposable or not.
Circular references impose no penalty or possibility of memory leaks, as the GC marks which objects it has visited in the object graph. In the case of delegates or eventhandlers it may be common to forget to remove the reference in an event to a target method, so that the object that contains the target method can't be collected if the event is rooted.

What are the best ways to get things deleted?
NOTE: the following works only for types containing unmanaged resources. It doesn't help with purely managed types.
Probably the best method is to implement and follow the IDisposable pattern; and call the dispose method on all objects implementing it.
The 'using' statement is your best friend. Loosely put, it will call dispose for you on objects implementing IDisposable.