We Keep Coding

C# Memory Issues

I've got an application that:
Targets C# 6
Targets .net 4.5.2
Is a Windows Forms application
Builds in AnyCPU Mode beacuse it...
Utilizes old 32 bit libraries that cannot be upgraded to 64 bit, unmanaged memory
Uses DevExpress, a third party control vendor
Processes many gigabytes of data daily to produce reports
After a few hours of use in jobs that have many plots, the application eventually runs out of memory. I've spend quite a long time cleaning up many leaks found in the code and have gotten the project to a state where at the worst case it may be using upwards 400,000K of memory at any given time, according to performance counters. Processing this data has not yielded any issues at this point since data is processed in Jagged arrays, preventing any issues with the Large Object Heap.
Last time this happened the user was using ~305,000K of memory. The application is so "out of memory" that the error dialog cannot even draw the error icon in the MessageBox that comes up, the space where the icon would usually be is all black.
So far I've done the following to clean this up:
Windows forms utilize the Disposed event to ensure that resources are cleaned up, dispose is called manually when required
Business objects utilize IDisposable to remove references
Verified cleanup using ANTS memory profiler and SciTech memory profiler.
The low memory usage suggests this is not the case but I wanted to see if I saw anything that could be helpful, I could not
Utilized the GCSettings.LargeObjectHeapCompactionMode property to remove any fragmentation from processing data that may be fragmented in the Large Object Heap (LoH)
Nearly every article that I've used to get to this point suggests that out of memory actually means out of contiguous address space and given the amount that's in use, I agree with this. I'm not sure what to do at this point since from what I understand (and am probably very wrong about) is that the garbage collector clears this up to make room as the process moves along, with the exception of the LoH, which is cleaned up manually now using the new LargeObejctHeapCompactionMode property introduced in .net 4.5.1.
What am I missing here? I cannot build to 64 bit due to the old 32 bit libraries that contain proprietary algorithms that we do not have access to even dream of producing a 64 bit version of. Are there any modes in these profiles I should be using to identify exactly what is growing out of control here?
If this address space cannot be cleared up does this mean that all c# applications will eventually run "out of memory" because of this?

Nearly every article that I've used to get to this point suggests that out of memory actually means out of contiguous address space and given the amount that's in use, I agree with this.
This is a reasonable hypothesis, but even reasonable hypotheses can be wrong. Yours probably is wrong. What should you do?
Test it with science. That is, look for evidence that falsifies your hypothesis. You want to assume that it is anything else, and be forced by the evidence you've gathered that your hypothesis is not false.
So:
at the point where your application runs out of memory, is it actually out of contiguous free pages of the necessary size? It sure sounds like your observations do not indicate that this is true, so the hypothesis is probably false.
What is other evidence that the hypothesis might be false?
"After a few hours of use in jobs that have many plots, the application eventually runs out of memory."
"Uses DevExpress, a third party control vendor"
"the error dialog cannot even draw the error icon in the MessageBox"
None of this sounds like an out of memory problem. This sounds like a third party control library leaking OS handles for graphics objects. Unfortunately, such leaks usually surface as "out of memory" errors and not "out of handles" errors.
So, that's a new hypothesis. Look for evidence for and against this hypothesis too. You're doing a good job by using a memory profiler. Use a handle profiler next.
If this address space cannot be cleared up does this mean that all c# applications will eventually run "out of memory" because of this?
Nope. The GC does a good job of cleaning up managed memory; lots of applications have no problem running forever without leaking.

UWP application is getting slow after usage of some time?

I have developed one UWP application.In that nearly 20 windows are there.
Every window contains lot of Xaml controls. For some time it's working fine.
But after usage of some time application is getting very slow.
After doing some R&D I came to know that it's called Memory Leakage.
As per my knowledge in .Net Garbage Collector has to take care of this if I am not wrong.It seems like In UWP Application it is not happening. So I thought I should use GC.Collect() in Page Unload Event.
Is that correct approach or anything else I need to do to release memory which are used by the window controls?

The performance optimization is a vast subject and may not be possible to answer on an open ended question (without knowledge of your environment and architecture etc).
However you can use the Visual Studio profiler to measure and track performance and to find out the area where you need to take action, these can be;
Data binding
UI Virtualization
Image rendering
Visual Tree Size
Further reading these urls may also help you.
ms docs and this blog

The GC takes care of orphaned objects or objects that are no longer referenced by any other classes. when the GC finds objects like these it removes them from memory. memory leaks happen when an object is referenced by another class even after it is done with it. this means you need to look at your code and find where this is happening. you need to help GC in doing its job by making sure you no longer reference objects you don't need.
I also wouldn't recommend using GC.Collect in page unload event since GC has to freeze threads in order to collect. this might impact performance.

Tools for Memory leaks in .Net executable [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
What Are Some Good .NET Profilers?
I am trying to test an application in Windows for memory leaks. I have looked at Linux alternatives (eg. Valgrind) and I am looking for a similar tool for Windows. The problem with .Net is that the memory is not released straight away like in Linux.
My knowledge of .Net is limited so my problem is I would like to use a tool that would find memory leaks for a Windows .Net executable. Ideally the program should be able to detect any memory leaks while I use the application. Does anyone know of such a tool or have any suggestions to solve this issue?
Thanks for any help in advance.
Thanks for the responses. Can anyone tell me if any of these program's allow you to execute from the command line and output memory leak reports into a text file.
Thanks

Redgate has a nice memory profiler you can download here. It even comes with a 14-day trial. I have used it and it is very good.

Some others have already posted good links to profilers -- redgate's profiler is particularly good.
Here's a link to a good article on finding leaks in .Net:
http://blogs.msdn.com/b/ricom/archive/2004/12/10/279612.aspx
Here's a great blog -- the author is a support engineer at MS (these people are really good programmers that work in windebug all day to find problems). A great many of her articles are about trackign down memory leaks. Most of her examples are asp.net, but most of the techniques should apply to Windows apps as well:
http://blogs.msdn.com/b/tess/archive/2006/01/23/516139.aspx
Also, be aware that you might not have a real leak. By design, the .Net garbage collector does not immediately release memory. It does periodic collections that are triggered by various events (I don't think that MS has published a complete list of the things that will cause the garbage collector to fire). I do know that one of the triggers is low memory conditions, and another trigger can be a memory allocation. If nothing triggers it, the GC will not collect.
That means that you can start a Winform .net app, let it eat a bunch of memory (and then release it), and then leave it sitting all night long. If the computer has been idle for the whole time, the GC may not have fired, and the app still might own a lot of memory. When something happens to trigger the collection, memory will be freed. It's very common that people report that .net apps appear to be slow to release memory as a result of this behavior.
Go ahead and profile -- you might have a leak, but also be aware that this might be behavior by design. Either way, good luck!

We have tried a lot of the memoryprofilers for .NET and have come to the conclusion that the .NET memory profiler is the best one currently on the market with a good combination to ease of use, amount of availabele data and perfomance.

How to detect where a Memory Leak is?

I have a large website that seems to be sucking up all the memory that is being allocated. There is nothing else on the server beside this site. Within a week it eats away the 2 gigs and requires a restart. Currently this is server 2008 32 bit using IIS 7. We are reinstalling to use 64bit and add more memory. It would be nice to be able to track down where the leaks are occurring.
So what is the best practice to tracking memory leaks?

Memory leaks in .NET are not that common, but when they happen it is most commonly due to unattached event handlers. Make sure you detach handlers, before the listeners go out of scope.
Another option is if you forget to call Dispose() on IDisposable resources. This may prevent cleanup of unmanaged resources (which are not handled by GC).
And yet another possible reason is a deadlocked finalizer. That will prevent all remaining objects in the finalizer queue from being collected.
I use WinDbg + Sos to track down leaks. The steps are as follows
Dump the heap and look for suspects
Use !gcroot to find out what is keeping the suspects alive
Repeat as necessary
Be aware that large memory usage may also be due to heap fragmentation. The regular heaps are compacted, but pinned objects can cause fragmentation. Also, the LOH is not compacted so fragmentation is not uncommon for LOH.
Excellent tutorials on WinDbg + sos here: http://blogs.msdn.com/tess/

Run a profiler on your code.
Here are two good options:
RedGate's memory profiler
Jetbrains dotTrace
I believe both products have a free trial.

Run, don't walk, over to Tess
Ferrandez's blog, If broken it is,
fix it you should, which has well
scripted labs dedicated to learning
how to diagnose and debug crash, hang
and memory issues with .NET code. She
has some of the best material I've
found to date to help get you started.
Commercial memory profilers such as ANTS and SciTech are excellent resources that will show what objects are in the heap, and how they are rooted. Most commercial memory profilers have the ability to load a memory 'snap' of a process (say from your production environment).
You can capture a memory 'snap' (see Snap v. Dump) using adplus.vbs or DebugDiag. Adplus is available as part of the Debugging Tools for Windows. DebugDiag will also have some rudimentary analysis (but seems to be more reliable on unmanaged code) automagically.
Monitor the Application
For an idea on what to monitor, see Improving .NET Performance and Scalability, specifically Chapter 15.
As to how to monitor, there are commercial tools available for that as well, however, every Windows machine also comes with Perfmon.exe, which can be used to record relevant performance counters.
Test the Application
For an idea on how to perform load, or stress, tests, check out the Patterns and Practices Performance Testing Guidance for Web Applications.
Debug the Application
Once you've identified you've got a problem (monitoring) and your able to reproduce the problem (testing) you can get down to debugging the problem. See the links for Tess - that information will carry you a long way.
Then rinse and repeat! :)
Good luck!
Z

In performance monitor, add counters for Process/Private Bytes and .NET CLR Memory/# Bytes in All Heaps. Private bytes is all memory and CLR memory is just managed. So if the CLR memory remains fairly even but the private bytes continues to grow over time, this means that the leak is in unmanaged resource. That usually means that you are not disposing of native resources properly. A good thing to look at is stuff like COM or IO (streams and files). Make sure all of that stuff gets disposed when you are done with it.

You can try using profilers such as dotTrace -- set it into a memory trace and run your application.
This should give you clues in which assemblies and areas of the application that eat up too much memory on the get go.

This is probably prevention rather then detection, but at the c# code level, you should check that any classes that use large resources such as images and other files are correctly implementing the dispose pattern. If needed you may also need to override the finalizer.
MSDN has good guidance on this subject.
If you have any classes in your application that you know make use of large resources, these are the first places to look for memory issues.

I've found that the EQATEC Profiler is pretty good, plus it's free!

Check out the Memory and Memory Leak labs on this blog post:
.NET Debugging Demos
They may be of some help. Basically, you can use WinDBG to analyze a memory dump and help determine what is eating up all of your memory.
We used a similar approach to determine that Regex was chewing up all of our memory, but only when the product was run on 64-bit machines. The learning curve is kind of steep, but WinDBG is a very powerful tool.

This new article of mine maybe useful: How to detect and avoid memory and resources leaks in .NET applications

Do you do any Office interop? If so, make sure you clean up your application objects. That's one possible culprit.
Another is global objects, such as anything that's static.

Do you have lots of dynamic pages on your site?
You can also try IBM's Purify
I suggest you try with a small set of dynamic pages disabling all others for the meantime. I hate to say this but it's very much possible that IIS 7 may also have leaks.

Look at this article on detecting .NET application memory leaks and related articles mentioned at the bottom of the page and hopefully you will find a solution or at least an idea to resolve it.
Thanks

What strategies and tools are useful for finding memory leaks in .NET?

I wrote C++ for 10 years. I encountered memory problems, but they could be fixed with a reasonable amount of effort.
For the last couple of years I've been writing C#. I find I still get lots of memory problems. They're difficult to diagnose and fix due to the non-determinancy, and because the C# philosophy is that you shouldn't have to worry about such things when you very definitely do.
One particular problem I find is that I have to explicitly dispose and cleanup everything in code. If I don't, then the memory profilers don't really help because there is so much chaff floating about you can't find a leak within all the data they're trying to show you. I wonder if I've got the wrong idea, or if the tool I've got isn't the best.
What kind of strategies and tools are useful for tackling memory leaks in .NET?

I use Scitech's MemProfiler when I suspect a memory leak.
So far, I have found it to be very reliable and powerful. It has saved my bacon on at least one occasion.
The GC works very well in .NET IMO, but just like any other language or platform, if you write bad code, bad things happen.

Just for the forgetting-to-dispose problem, try the solution described in this blog post. Here's the essence:
public void Dispose ()
{
// Dispose logic here ...
// It's a bad error if someone forgets to call Dispose,
// so in Debug builds, we put a finalizer in to detect
// the error. If Dispose is called, we suppress the
// finalizer.
#if DEBUG
GC.SuppressFinalize(this);
#endif
}
#if DEBUG
~TimedLock()
{
// If this finalizer runs, someone somewhere failed to
// call Dispose, which means we've failed to leave
// a monitor!
System.Diagnostics.Debug.Fail("Undisposed lock");
}
#endif

We've used Ants Profiler Pro by Red Gate software in our project. It works really well for all .NET language-based applications.
We found that the .NET Garbage Collector is very "safe" in its cleaning up of in-memory objects (as it should be). It would keep objects around just because we might be using it sometime in the future. This meant we needed to be more careful about the number of objects that we inflated in memory. In the end, we converted all of our data objects over to an "inflate on-demand" (just before a field is requested) in order to reduce memory overhead and increase performance.
EDIT: Here's a further explanation of what I mean by "inflate on demand." In our object model of our database we use Properties of a parent object to expose the child object(s). For example if we had some record that referenced some other "detail" or "lookup" record on a one-to-one basis we would structure it like this:
class ParentObject
Private mRelatedObject as New CRelatedObject
public Readonly property RelatedObject() as CRelatedObject
get
mRelatedObject.getWithID(RelatedObjectID)
return mRelatedObject
end get
end property
End class
We found that the above system created some real memory and performance problems when there were a lot of records in memory. So we switched over to a system where objects were inflated only when they were requested, and database calls were done only when necessary:
class ParentObject
Private mRelatedObject as CRelatedObject
Public ReadOnly Property RelatedObject() as CRelatedObject
Get
If mRelatedObject is Nothing
mRelatedObject = New CRelatedObject
End If
If mRelatedObject.isEmptyObject
mRelatedObject.getWithID(RelatedObjectID)
End If
return mRelatedObject
end get
end Property
end class
This turned out to be much more efficient because objects were kept out of memory until they were needed (the Get method was accessed). It provided a very large performance boost in limiting database hits and a huge gain on memory space.

You still need to worry about memory when you are writing managed code unless your application is trivial. I will suggest two things: first, read CLR via C# because it will help you understand memory management in .NET. Second, learn to use a tool like CLRProfiler (Microsoft). This can give you an idea of what is causing your memory leak (e.g. you can take a look at your large object heap fragmentation)

Are you using unmanaged code? If you are not using unmanaged code, according to Microsoft, memory leaks in the traditional sense are not possible.
Memory used by an application may not be released however, so an application's memory allocation may grow throughout the life of the application.
From How to identify memory leaks in the common language runtime at Microsoft.com
A memory leak can occur in a .NET
Framework application when you use
unmanaged code as part of the
application. This unmanaged code can
leak memory, and the .NET Framework
runtime cannot address that problem.
Additionally, a project may only
appear to have a memory leak. This
condition can occur if many large
objects (such as DataTable objects)
are declared and then added to a
collection (such as a DataSet). The
resources that these objects own may
never be released, and the resources
are left alive for the whole run of
the program. This appears to be a
leak, but actually it is just a
symptom of the way that memory is
being allocated in the program.
For dealing with this type of issue, you can implement IDisposable. If you want to see some of the strategies for dealing with memory management, I would suggest searching for IDisposable, XNA, memory management as game developers need to have more predictable garbage collection and so must force the GC to do its thing.
One common mistake is to not remove event handlers that subscribe to an object. An event handler subscription will prevent an object from being recycled. Also, take a look at the using statement which allows you to create a limited scope for a resource's lifetime.

This blog has some really wonderful walkthroughs using windbg and other tools to track down memory leaks of all types. Excellent reading to develop your skills.

I just had a memory leak in a windows service, that I fixed.
First, I tried MemProfiler. I found it really hard to use and not at all user friendly.
Then, I used JustTrace which is easier to use and gives you more details about the objects that are not disposed correctly.
It allowed me to solve the memory leak really easily.

If the leaks you are observing are due to a runaway cache implementation, this is a scenario where you might want to consider the use of WeakReference. This could help to ensure that memory is released when necessary.
However, IMHO it would be better to consider a bespoke solution - only you really know how long you need to keep the objects around, so designing appropriate housekeeping code for your situation is usually the best approach.

I prefer dotmemory from Jetbrains

Big guns - Debugging Tools for Windows
This is an amazing collection of tools. You can analyze both managed and unmanaged heaps with it and you can do it offline. This was very handy for debugging one of our ASP.NET applications that kept recycling due to memory overuse. I only had to create a full memory dump of living process running on production server, all analysis was done offline in WinDbg. (It turned out some developer was overusing in-memory Session storage.)
"If broken it is..." blog has very useful articles on the subject.

After one of my fixes for managed application I had the same thing, like how to verify that my application will not have the same memory leak after my next change, so I've wrote something like Object Release Verification framework, please take a look on the NuGet package ObjectReleaseVerification. You can find a sample here https://github.com/outcoldman/OutcoldSolutions-ObjectReleaseVerification-Sample, and information about this sample http://outcoldman.com/en/blog/show/322

The best thing to keep in mind is to keep track of the references to your objects. It is very easy to end up with hanging references to objects that you don't care about anymore.
If you are not going to use something anymore, get rid of it.
Get used to using a cache provider with sliding expirations, so that if something isn't referenced for a desired time window it is dereferenced and cleaned up. But if it is being accessed a lot it will say in memory.

One of the best tools is using the Debugging Tools for Windows, and taking a memory dump of the process using adplus, then use windbg and the sos plugin to analyze the process memory, threads, and call stacks.
You can use this method for identifying problems on servers too, after installing the tools, share the directory, then connect to the share from the server using (net use) and either take a crash or hang dump of the process.
Then analyze offline.

From Visual Studio 2015 consider to use out of the box Memory Usage diagnostic tool to collect and analyze memory usage data.
The Memory Usage tool lets you take one or more snapshots of the managed and native memory heap to help understand the memory usage impact of object types.

one of the best tools I used its DotMemory.you can use this tool as an extension in VS.after run your app you can analyze every part of memory(by Object, NameSpace, etc) that your app use and take some snapshot of that, Compare it with other SnapShots.
DotMemory