We Keep Coding

Weird out of memory exceptions [duplicate]

If you application is such that it has to do lot of allocation/de-allocation of large size objects (>85000 Bytes), its eventually will cause memory fragmentation and you application will throw an Out of memory exception.
Is there any solution to this problem or is it a limitation of CLR memory management?

Unfortunately, all the info I've ever seen only suggests managing risk factors yourself: reuse large objects, allocate them at the beginning, make sure they're of sizes that are multiples of each other, use alternative data structures (lists, trees) instead of arrays. That just gave me an another idea of creating a non-fragmenting List that instead of one large array, splits into smaller ones. Arrays / Lists seem to be the most frequent culprits IME.
Here's an MSDN magazine article about it:
http://msdn.microsoft.com/en-us/magazine/cc534993.aspx, but there isn't that much useful in it.

The thing about large objects in the CLR's Garbage Collector is that they are managed in a different heap.
The garbage collector uses a mechanism called "Compacting", which is basically fragmentation and re-linkage of objects in the regular heap.
The thing is, since "compacting" large objects (copying and re-linking them) is an expensive procedure, the GC provides a different heap for them, which is never being compacted.
Note also that memory allocation is contiguous. Meaning if you allocate Object #1 and then Object #2, Object #2 will always be placed after Object #1.
This is probably what's causing you to get OutOfMemoryExceptions.
I would suggest having a look at design patterns like Flyweight, Lazy Initialization and Object Pool.
You could also force GC collection, if you're suspecting that some of those large objects are already dead and have not been collected due to flaws in your flow of control, causing them to reach higher generations just before being ready for collection.

A program always bombs on OOM because it is asking for a chunk of memory that's too large, never because it completely exhausted all virtual memory address space. You could argue that's a problem with the LOH getting fragmented, it is just as easy to argue that the program is using too much virtual memory.
Once a program goes beyond allocating half the addressable virtual memory (a gigabyte), it is really time to either consider making its code smarter so it doesn't gobble so much memory. Or making a 64-bit operating system a prerequisite. The latter is always cheaper. It doesn't come out of your pocket either.

Is there any solution to this problem or is it a limitation of CLR memory management?
There is no solution besides reconsidering your design. And it is not a problem of the CLR. Note, the problem is the same for unmanaged applications. It is given by the fact, that too much memory is used by the application at the same time and in segments laying 'disadvantageous' out in memory. If some external culprit has to be pointed at nevertheless, I would rather point at the OS memory manager, which (of course) does not compact its vm address space.
The CLR manages free regions of the LOH in a free list. This in most cases is the best what can be done against fragmentation. But since for really large objects, the number of objects per LOH segment decreases - we eventually end up having only one object per segment. And where those objects are positioned in the vm space is completely up to the memory manager of the OS. This means, the fragmentation mostly happens on the OS level - not on the CLR. This is an often overseen aspect of heap fragmentation and it is not .NET to blame for it. (But it is also true, fragmentation can also occour on the managed side like nicely demonstrated in that article.)
Common solutions have been named already: reuse your large objects. I up to now was not confronted with any situation, where this could not be done by proper design. However, it can be tricky sometimes and therefore may be expensive though.

We were precessing images in multiple threads. With images being large enough, this also caused OutOfMemory exceptions due to memory fragmentation. We tried to solve the problem by using unsafe memory and pre-allocating heap for every thread. Unfortunately, this didn't help completely since we relied on several libraries: we were able to solve the problem in our code, but not 3rd party.
Eventually we replaced threads with processes and let operating system do the hard work. Operating systems have long ago built a solution for memory fragmentation, so it's unwise to ignore it.

I have seen in a different answer that the LOH can shrink in size:
Large Arrays, and LOH Fragmentation. What is the accepted convention?
"
...
Now, having said that, the LOH can shrink in size if the area at its end is completely free of live objects, so the only problem is if you leave objects in there for a long time (e.g. the duration of the application).
...
"
Other then that you can make your program run with extended memory up to 3GB on 32bit system and up to 4 GB on 64bit system.
Just add the flag /LARGEADDRESSAWARE in your linker or this post build event:
call "$(DevEnvDir)..\tools\vsvars32.bat"
editbin /LARGEADDRESSAWARE "$(TargetPath)"
In the end if you are planning to run the program for a long time with lots of large objects you will have to optimize the memory usage and you might even have to reuse allocated objects to avoid garbage collector which is similar in concept, to working with real time systems.

Does C# allows pointers? [duplicate]

I know C# gives the programmer the ability to access, use pointers in an unsafe context. But When is this needed?
At what circumstances, using pointers becomes inevitable?
Is it only for performance reasons?
Also why does C# expose this functionality through an unsafe context, and remove all of the managed advantages from it? Is it possible to have use pointers without losing any advantages of managed environment, theoretically?

When is this needed? Under what circumstances does using pointers becomes inevitable?
When the net cost of a managed, safe solution is unacceptable but the net cost of an unsafe solution is acceptable. You can determine the net cost or net benefit by subtracting the total benefits from the total costs. The benefits of an unsafe solution are things like "no time wasted on unnecessary runtime checks to ensure correctness"; the costs are (1) having to write code that is safe even with the managed safety system turned off, and (2) having to deal with potentially making the garbage collector less efficient, because it cannot move around memory that has an unmanaged pointer into it.
Or, if you are the person writing the marshalling layer.
Is it only for performance reasons?
It seems perverse to use pointers in a managed language for reasons other than performance.
You can use the methods in the Marshal class to deal with interoperating with unmanaged code in the vast majority of cases. (There might be a few cases in which it is difficult or impossible to use the marshalling gear to solve an interop problem, but I don't know of any.)
Of course, as I said, if you are the person writing the Marshal class then obviously you don't get to use the marshalling layer to solve your problem. In that case you'd need to implement it using pointers.
Why does C# expose this functionality through an unsafe context, and remove all of the managed advantages from it?
Those managed advantages come with performance costs. For example, every time you ask an array for its tenth element, the runtime needs to do a check to see if there is a tenth element, and throw an exception if there isn't. With pointers that runtime cost is eliminated.
The corresponding developer cost is that if you do it wrong then you get to deal with memory corruption bugs that formats your hard disk and crashes your process an hour later rather than dealing with a nice clean exception at the point of the error.
Is it possible to use pointers without losing any advantages of managed environment, theoretically?
By "advantages" I assume you mean advantages like garbage collection, type safety and referential integrity. Thus your question is essentially "is it in theory possible to turn off the safety system but still get the benefits of the safety system being turned on?" No, clearly it is not. If you turn off that safety system because you don't like how expensive it is then you don't get the benefits of it being on!

Pointers are an inherent contradiction to the managed, garbage-collected, environment.
Once you start messing with raw pointers, the GC has no clue what's going on.
Specifically, it cannot tell whether objects are reachable, since it doesn't know where your pointers are.
It also cannot move objects around in memory, since that would break your pointers.
All of this would be solved by GC-tracked pointers; that's what references are.
You should only use pointers in messy advanced interop scenarios or for highly sophisticated optimization.
If you have to ask, you probably shouldn't.

The GC can move references around; using unsafe keeps an object outside of the GC's control, and avoids this. "Fixed" pins an object, but lets the GC manage the memory.
By definition, if you have a pointer to the address of an object, and the GC moves it, your pointer is no longer valid.
As to why you need pointers: Primary reason is to work with unmanaged DLLs, e.g. those written in C++
Also note, when you pin variables and use pointers, you're more susceptible to heap fragmentation.
Edit
You've touched on the core issue of managed vs. unmanaged code... how does the memory get released?
You can mix code for performance as you describe, you just can't cross managed/unmanaged boundaries with pointers (i.e. you can't use pointers outside of the 'unsafe' context).
As for how they get cleaned... You have to manage your own memory; objects that your pointers point to were created/allocated (usually within the C++ DLL) using (hopefully) CoTaskMemAlloc(), and you have to release that memory in the same manner, calling CoTaskMemFree(), or you'll have a memory leak. Note that only memory allocated with CoTaskMemAlloc() can be freed with CoTaskMemFree().
The other alternative is to expose a method from your native C++ dll that takes a pointer and frees it... this lets the DLL decide how to free the memory, which works best if it used some other method to allocate memory. Most native dlls you work with are third-party dlls that you can't modify, and they don't usually have (that I've seen) such functions to call.
An example of freeing memory, taken from here:
string[] array = new string[2];
array[0] = "hello";
array[1] = "world";
IntPtr ptr = test(array);
string result = Marshal.PtrToStringAuto(ptr);
Marshal.FreeCoTaskMem(ptr);
System.Console.WriteLine(result);
Some more reading material:
C# deallocate memory referenced by IntPtr
The second answer down explains the different allocation/deallocation methods
How to free IntPtr in C#?
Reinforces the need to deallocate in the same manner the memory was allocated
http://msdn.microsoft.com/en-us/library/aa366533%28VS.85%29.aspx
Official MSDN documentation on the various ways to allocate and deallocate memory.
In short... you need to know how the memory was allocated in order to free it.
Edit
If I understand your question correctly, the short answer is yes, you can hand the data off to unmanaged pointers, work with it in an unsafe context, and have the data available once you exit the unsafe context.
The key is that you have to pin the managed object you're referencing with a fixed block. This prevents the memory you're referencing from being moved by the GC while in the unsafe block. There are a number of subtleties involved here, e.g. you can't reassign a pointer initialized in a fixed block... you should read up on unsafe and fixed statements if you're really set on managing your own code.
All that said, the benefits of managing your own objects and using pointers in the manner you describe may not buy you as much of a performance increase as you might think. Reasons why not:
C# is very optimized and very fast
Your pointer code is still generated as IL, which has to be jitted (at which point further optimizations come into play)
You're not turning the Garbage Collector off... you're just keeping the objects you're working with out of the GC's purview. So every 100ms or so, the GC still interrupts your code and executes its functions for all the other variables in your managed code.
HTH,
James

The most common reasons to use pointers explicitly in C#:
doing low-level work (like string manipulation) that is very performance sensitive,
interfacing with unmanaged APIs.
The reason why the syntax associated with pointers was removed from C# (according to my knowledge and viewpoint — Jon Skeet would answer better B-)) was it turned out to be superfluous in most situations.
From the language design perspective, once you manage memory by a garbage collector you have to introduce severe constraints on what is and what is not possible to do with pointers. For example, using a pointer to point into the middle of an object can cause severe problems to the GC. Hence, once the restrictions are in place, you can just omit the extra syntax and end up with “automatic” references.
Also, the ultra-benevolent approach found in C/C++ is a common source of errors. For most situations, where micro-performance doesn't matter at all, it is better to offer tighter rules and constrain the developer in favor of less bugs that would be very hard to discover. Thus for common business applications the so-called “managed” environments like .NET and Java are better suited than languages that presume to work against the bare-metal machine.

Say you want to communicate between 2 application using IPC (shared memory) then you can marshal the data to memory and pass this data pointer to the other application via windows messaging or something. At receiving application you can fetch data back.
Useful also in case of transferring data from .NET to legacy VB6 apps wherein you will marshal the data to memory, pass pointer to VB6 app using win msging, use VB6 copymemory() to fetch data from the managed memory space to VB6 apps unmanaged memory space..

C# Garbage Collection -> to C++ delete

I'm converting a C# project to C++ and have a question about deleting objects after use. In C# the GC of course takes care of deleting objects, but in C++ it has to be done explicitly using the delete keyword.
My question is, is it ok to just follow each object's usage throughout a method and then delete it as soon as it goes out of scope (ie method end/re-assignment)?
I know though that the GC waits for a certain size of garbage (~1MB) before deleting; does it do this because there is an overhead when using delete?
As this is a game I am creating there will potentially be lots of objects being created and deleted every second, so would it be better to keep track of pointers that go out of scope, and once that size reachs 1MB to then delete the pointers?
(as a side note: later when the game is optimised, objects will be loaded once at startup so there is not much to delete during gameplay)

Your problem is that you are using pointers in C++.
This is a fundamental problem that you must fix, then all your problems go away. As chance would have it, I got so fed up with this general trend that I created a set of presentation slides on this issue. – (CC BY, so feel free to use them).
Have a look at the slides. While they are certainly not entirely serious, the fundamental message is still true: Don’t use pointers. But more accurately, the message should read: Don’t use delete.
In your particular situation you might find yourself with a lot of long-lived small objects. This is indeed a situation which a modern GC handles quite well, and which reference-counting smart pointers (shared_ptr) handle less efficiently. If (and only if!) this becomes a performance problem, consider switching to a small object allocator library.

You should be using RAII as much as possible in C++ so you do not have to explicitly deleteanything anytime.
Once you use RAII through smart pointers and your own resource managing classes every dynamic allocation you make will exist only till there are any possible references to it, You do not have to manage any resources explicitly.

Memory management in C# and C++ is completely different. You shouldn't try to mimic the behavior of .NET's GC in C++. In .NET allocating memory is super fast (basically moving a pointer) whereas freeing it is the heavy task. In C++ allocating memory isn't that lightweight for several reasons, mainly because a large enough chunk of memory has to be found. When memory chunks of different sizes are allocated and freed many times during the execution of the program the heap can get fragmented, containing many small "holes" of free memory. In .NET this won't happen because the GC will compact the heap. Freeing memory in C++ is quite fast, though.
Best practices in .NET don't necessarily work in C++. For example, pooling and reusing objects in .NET isn't recommended most of the time, because the objects get promoted to higher generations by the GC. The GC works best for short lived objects. On the other hand, pooling objects in C++ can be very useful to avoid heap fragmentation. Also, allocating a larger chunk of memory and using placement new can work great for many smaller objects that need to be allocated and freed frequently, as it can occur in games. Read up on general memory management techniques in C++ such as RAII or placement new.
Also, I'd recommend getting the books "Effective C++" and "More effective C++".

Well, the simplest solution might be to just use garbage collection in
C++. The Boehm collector works well, for example. Still, there are
pros and cons (but porting code originally written in C# would be a
likely candidate for a case where the pros largely outweigh the cons.)
Otherwise, if you convert the code to idiomatic C++, there shouldn't be
that many dynamically allocated objects to worry about. Unlike C#, C++
has value semantics by default, and most of your short lived objects
should be simply local variables, possibly copied if they are returned,
but not allocated dynamically. In C++, dynamic allocation is normally
only used for entity objects, whose lifetime depends on external events;
e.g. a Monster is created at some random time, with a probability
depending on the game state, and is deleted at some later time, in
reaction to events which change the game state. In this case, you
delete the object when the monster ceases to be part of the game. In
C#, you probably have a dispose function, or something similar, for
such objects, since they typically have concrete actions which must be
carried out when they cease to exist—things like deregistering as
an Observer, if that's one of the patterns you're using. In C++, this
sort of thing is typically handled by the destructor, and instead of
calling dispose, you call delete the object.

Substituting a shared_ptr in every instance that you use a reference in C# would get you the closest approximation at probably the lowest effort input when converting the code.
However you specifically mention following an objects use through a method and deleteing at the end - a better approach is not to new up the object at all but simply instantiate it inline/on the stack. In fact if you take this approach even for returned objects with the new copy semantics being introduced this becomes an efficient way to deal with returned objects also - so there is no need to use pointers in almost every scenario.

There are a lot more things to take into considerations when deallocating objects than just calling delete whenever it goes out of scope. You have to make sure that you only call delete once and only call it once all pointers to that object have gone out of scope. The garbage collector in .NET handles all of that for you.
The construct that is mostly corresponding to that in C++ is tr1::shared_ptr<> which keeps a reference counter to the object and deallocates when it drops to zero. A first approach to get things running would be to make all C# references in to C++ tr1::shared_ptr<>. Then you can go into those places where it is a performance bottleneck (only after you've verified with a profile that it is an actual bottleneck) and change to more efficient memory handling.

GC feature of c++ has been discussed a lot in SO.
Try Reading through this!!
Garbage Collection in C++

Does allocating objects of the same size improve GC or "new" performance?

Suppose we have to create many small objects of byte array type. The size varies but it always below 1024 bytes , say 780,256,953....
Will it improve operator new or GC efficiency over time if we always allocate only bytes[1024], and use only space needed?
UPD: This is short living objects, created for parsing binary protocol messages.
UPD: The number of the objects is the same in both cases, it just the size of allocation which changes (random vs. always 1024).
In C++ it would matter because of fragmentation and C++ new performance. But in C#....

Will it improve operator new or GC efficiency over time if we always allocate only bytes[1024], and use only space needed?
Maybe. You're going to have to profile it and see.
The way we allocate syntax tree nodes inside the Roslyn compiler is quite interesting, and I'm eventually going to do a blog post about it. Until then, the relevant bit to your question is this interesting bit of trivia. Our allocation pattern typically involves allocating an "underlying" immutable node (which we call the "green" node) and a "facade" mutable node that wraps it (which we call the "red" node). As you might imagine, it is frequently the case that we end up allocating these in pairs: green, red, green, red, green, red.
The green nodes are persistent and therefore long-lived; the facades are short-lived, because they are discarded on every edit. Therefore it is frequently the case that the garbage collector has green / hole / green / hole / green / hole, and then the green nodes move up a generation.
Our assumption had always been that making data structures smaller will always improve GC performance. Smaller structures equals less memory allocated, equals less collection pressure, equals fewer collections, equals more performance, right? But we discovered through profiling that making the red nodes smaller in this scenario actually decreases GC performance. Something about the particular size of the holes affects the GC in some odd way; not being an expert on the internals of the garbage collector, it is opaque to me why that should be.
So is it possible that changing the size of your allocations can affect the GC in some unforseen way? Yes, it is possible. But, first off, it is unlikely, and second it is impossible to know whether you are in that situation until you actually try it in real-world scenarios and carefully measure GC performance.
And of course, you might not be gated on GC performance. Roslyn does so many small allocations that it is crucial that we tune our GC-impacting behaviour, but we do an insane number of small allocations. The vast majority of .NET programs do not stress the GC the way we do. If you are in the minority of programs that stress the GC in interesting ways then there is no way around it; you're going to have to profile and gather empirical data, just like we do on the Roslyn team.
If you are not in that minority, then don't worry about GC performance; you probably have a bigger problem somewhere else that you should be dealing with first.

new is fast, it is the GC that causes problems. So, it depends on how long your arrays live for.
If they only live a short time, I don't think there will be any improvement from allocating 1024 byte arrays. In fact this will put more pressure on the GC because of the wasted space and will probably degrade performance.
If they live for the life of your application, I would consider allocating one large array and using chunks of it for each small array. You would need to profile this to see if it helps.

Not really, Allocating or clearing a bytes array requires only one instruction, regardless of its size. (I speak about your case. there are exceptions)
You shouldn't worry about the performance aspect of garbage collection, unless you are sure that it's a bottleneck for your application (ie you create a lot of references with complex relationship, and throw it shortly afterward... And the garbage collection is noticeable.)
To read an excellent story about a well known (and quite useful) site having performance issues with the .NET GC (in an impressive use case) see this blog. http://samsaffron.com/archive/2011/10/28/in-managed-code-we-trust-our-recent-battles-with-the-net-garbage-collector ;)
But the most important thing about GC is: Never, ever do optimisations before being sure that you have a problem. Because if you do, you will probably have one. Applications are complex, and the GC interacts with every parts of it, at runtime. Apart from simple cases, predicting its behavior and bottlenecks beforehand seems (in my opinion) difficult.

I also don't think only allocating 1024 byte arrays will improve the GC.
Since the GC is not determistic, I also think it will not be your problem.
You can influence the GC by using using {} statements arround your arrays to free memory (maybe) sooner.

I do not think GC will be the problem and/ or bottleneck.
Allocating objects of different sizes and freeing them in a different sequence can lead to fragmented heap memory. So this is where allocating objects of the same size might come in handy.
If you really do allocate/ free a lot and really do see this as your bottleneck, try to re-use the objects with a local object cache. This can lead to performance increase, especially if they are data-only objects that do not implement a lot of logic. If the objects do implement a lot of logic and require mor complex initialisation (RAII-pattern) I would forego the performance increase for robustness of the program...
hth
Mario

C# memory management: unsafe keyword and pointers

What are the consequences (positive/negative) of using the unsafe keyword in C# to use pointers? For example, what becomes of garbage collection, what are the performance gains/losses, what are the performance gains/losses compared to other languages manual memory management, what are the dangers, in which situation is it really justifiable to make use of this language feature, is it longer to compile... ?

As already mentioned by Conrad, there are some situations where unsafe access to memory in C# is useful. There are not as many of them, but there are some:
Manipulating with Bitmap is almost a typical example where you need some additional performance that you can get by using unsafe.
Interoperability with older API (such as WinAPI or native C/C++ DLLs) is another area where unsafe can be quite useful - for example you may want to call a function that takes/returns an unmanaged pointer.
On the other hand, you can write most of the things using Marshall class, which hides many of the unsafe operations inside method calls. This will be a bit slower, but it is an option if you want to avoid using unsafe (or if you're using VB.NET which doesn't have unsafe)
Positive consequences:
So, the main positive consequences of the existence of unsafe in C# is that you can write some code more easily (interoperability) and you can write some code more efficiently (manipulating with bitmap or maybe some heavy numeric calculations using arrays - although, I'm not so sure about the second one).
Negative consequences: Of course, there is some price that you have to pay for using unsafe:
Non-verifiable code: C# code that is written using the unsafe features becomes non-verifiable, which means that the your code could compromise the runtime in any way. This isn't a big problem in a full-trust scenario (e.g. unrestricted desktop app) - you just don't have all the nice .NET CLR guarantees. However, you cannot run the application in a restricted enviornment such as public web hosting, Silverlight or partial trust (e.g. application running from network).
Garbage collector also needs to be careful when you use unsafe. GC is usually allowed to relocate objects on the managed heap (to keep the memory defragmented). When you take a pointer to some object, you need to use the fixed keyword to tell the GC that it cannot move the object until you finish (which could probably affect the performance of garbage collection - but of course, depending on the exact scenario).
My guess that if C# didn't have to interoperate with older code, it probably wouldn't support unsafe (and research projects like Singularity that attempt to create more verifiable operating system based on managed languages definitely disallow usnsafe code). However, in the real-world, unsafe is useful in some (rare) cases.

I can give you a situation where it was worth using:
I have to generate a bitmap pixel by pixel. Drawing.Bitmap.SetPixel() is way too slow. So I build my own managed Array of bitmap data, and use unsafe to get the IntPtr for Bitmap.Bitmap( Int32, Int32, Int32, PixelFormat, IntPtr).

To quote Professional C# 2008:
"The two main reasons for using
pointers are:
Backward compability - Despite all of the facilities provided by the
.NET-runtime it is still possible to
call native Windows API functions, and
for some operations this may be the
only way to accompling your task.
These API functions are generally
written in C and often require
pointers as parameters. However, in
many cases it is possible to write the
DllImport declaration in a way that
avoids use of pointers; for example,
by using the System.IntPtr class.
Performance - On those occasions where speed is of the utmost
importance, pointer can provide a
route to optimized perfomance. If you
know what you are doing, you can
ensure that data is accessed or
manipulated in the most efficient way.
However, be aware that, more often
than not, there are other areas of
your code where you can make necessary
performance improvemens without
resoirting to pointers. Try using a
code profiler to look for bottlenecks
in your code - one comes with Visual
Studio 2008."
And if you use pointer your code will require higher lever of trust to execute and if the user does not grant that your code will not run.
And wrap it up with a last quote:
"We strongly advice against using
pointers unnecessarily because it will
not only be harder to write and debug,
but it will also fail the memory
type-safety checks imposed by the
CLR."

Garbage Collection is inefficient with long-lived objects. .Net's garbage collector works best when most objects are released rather quickly, and some objects "live forever." The problem is that longer-living objects are only released during full garbage collections, which incurs a significant performance penalty. In essence, long-living objects quickly move into generation 2.
(For more information, you might want to read up on .Net's generational garbage collector: http://msdn.microsoft.com/en-us/library/ms973837.aspx)
In situations where objects, or memory use in general, is going to be long-lived, manual memory management will yield better performance because it can be released to the system without requiring a full garbage collection.
Implementing some kind of a memory management system based around a single large byte array, structs, and lots of pointer arithmetic, could theoretically increase performance in situations where data will be stored in RAM for a long time.
Unfortunately, I'm not aware of a good way to do manual memory management in .Net for objects that are going to be long-lived. This basically means that applications that have long-lived data in RAM will periodically become unresponsive when they run a full garbage collection of all of the memory.