I have the following piece of code that runs in a loop.
public void Test(Bitmap bmp)
{
FormatConvertedBitmap fBitmapSource = new FormatConvertedBitmap();
PngBitmapEncoder pngBitmapEncoder = new PngBitmapEncoder();
BitmapImage bi = new BitmapImage();
using (MemoryStream ms = new MemoryStream())
{
bmp.Save(ms, ImageFormat.Png);
bmp.Dispose();
bmp = null;
bi.BeginInit();
bi.StreamSource = ms;
bi.EndInit();
BitmapPalette pallete = new BitmapPalette(bi, 256);
...
Last line
BitmapPalette pallete = new BitmapPalette(bi, 256);
Sometimes throws the following exception
Insufficient memory to continue the execution of the program.at System.Windows.Media.Imaging.BitmapPalette..ctor(BitmapSource bitmapSource, Int32 maxColorCount)
Any ideas ? I clearly have enough memory to continue execution.
There are other sources of OutOfMemoryException in a managed program that don't have anything to do with running out of managed memory. The exception is also raised when it translates error codes returned by legacy native code. Like the E_OUTOFMEMORY error that can be returned by COM method calls. And relevant in your case, by GDI+. Which has only 20 distinct error codes to indicate failure, you'll find them documented in this answer. One of them is OutOfMemory.
Which can mean more than one thing. Running out of unmanaged memory, the kind used by GDI+ to store bitmap pixels is certainly a possibility. It can also mean that your process has run out of available GDI object handles, Windows imposes a handle quota of 10,000 GDI handles. Which is an enormous number btw, exceeding that quota almost always indicates a bug in the code. A handle leak. Which in the case of a managed program is almost always caused by forgetting to use the Image.Dispose() method and not having the garbage collector run often enough to allow the finalizer to release handles.
Sadly it can even be triggered by corrupted bitmap data, not likely in your case since you bomb on allocating the palette. Which indicates a handle leak, which ought to be readily visible in Taskmgr.exe, Processes tab. View + Select columns and tick GDI Objects. Keep an eye on the displayed value for your process while you test it. A steadily increasing number spells trouble, the show is over when it reaches 10,000. Also look at the "Commit size" column, that can show you trouble with consuming too much unmanaged memory.
Related
This question already has answers here:
What strategies and tools are useful for finding memory leaks in .NET?
(15 answers)
Understanding garbage collection in .NET
(2 answers)
Memory Leak in C#
(21 answers)
Memory Leak in C# WPF
(4 answers)
Closed 3 years ago.
Hi everyone I'm making a C# WinForms application that searches for duplicate images in a directory. It starts by calling a constructor with every image in the directory.
There is a lot of files in the directory and memory was quickly rising to 2gb and then the program would throw an out of memory exception.
Now I've added a check in my for loop to check if the memory has exceeded 800 Megabits and I force a garbage collection. But I've noticed after the first forced collection the memory no longer rises.
(The forced garbage collection occurs at loop ~180 out of ~800 then it never occurs again)
(It looks a little like a shark fin swimming through the water leaving waves in its wake.)
I'm stumped as to why this is happening and have come here in search of help.
private void GeneratePrints()
{
for (int i = 0; i < files.Count; i++)
{
if (imageFileExtensions.Contains(Path.GetExtension(files[i])))
prints.Add(new FilePrint(directory + "/" + files[i]));
//800 Megabits
const long MAX_GARBAGE = 800 * 125000;
if (GC.GetTotalMemory(false) > MAX_GARBAGE)
{
GC.Collect();
GC.WaitForPendingFinalizers();
}
}
Console.WriteLine("File Prints Complete.");
}
GeneratePrints() is called 1 time, once a directory is selected.
I will also show you the constructor for the FilePrint class.
I'm pretty sure this all has something to do with the MemoryStream object.
public FilePrint(string filename)
{
Bitmap img;
using (var fs = new FileStream(filename, FileMode.Open))
{
using (var ms = new MemoryStream())
{
fs.CopyTo(ms);
ms.Position = 0;
img = (Bitmap)Bitmap.FromStream(ms);
ms.Close();
}
fs.Close();
}
this.size = img.Size;
img = ResizeImage(img, 8, 8);
img = MakeGrayscale(img);
//I do some basic for-loop arithmetic here
//calculating the average color of the image, not worth posting.
img.Dispose();
}
So basically I'm wondering how can I make it so that the 'shark-fin-like' memory usage spike at the start never happens so that I do not have to force a garbage collection.
Here is a memory snapshot I have taken when the forced garbage collection occurs (Am I not disposing of the MemoryStreams properly?):
Thank you for your replies, ideas and answers in advance!
You don't show the methods ResizeImage(img, 8, 8) and MakeGrayscale(img), but most likely they simply create and return a new image based on the old. If that's true, your code constructs two Bitmap objects that it never explicitly disposes, so try disposing them e.g. as follows:
using (var old = img)
img = ResizeImage(old, 8, 8);
using (var old = img)
img = MakeGrayscale(old);
You might also want to guarantee that the final img is disposed using a try/finally:
Bitmap img = null;
try
{
img = new Bitmap(filename); // Here I simplified the code, but this will leave the file locked until `img` is disposed after resizing.
this.size = img.Size;
using (var old = img)
img = ResizeImage(old, 8, 8);
using (var old = img)
img = MakeGrayscale(old);
//I do some basic for-loop arithmetic here
//calculating the average color of the image, not worth posting.
}
finally
{
if (img != null)
img.Dispose();
}
The possible reason you get the long buildup in memory use then a precipitous drop is that eventually the unmanaged resources of the undisposed images will get finalized, however because the GC is unaware of unmanaged memory owned by the undisposed Bitmap objects, it doesn't necessarily kick in, identify the bitmaps as unreferenced and pass them on to the finalizer thread for quite a while. It's not always easy to predict when or even if the finalizer thread will spin up and start working; see Are .net finalizers always executed? to which the answer is not necessarily. But by calling GC.WaitForPendingFinalizers(); you may be kickstarting that process.
Incidentally, a MemoryStream doesn't actually need to be disposed in the current implementation as it lacks unmanaged resources. See this answer by Jon Skeet to Is a memory leak created if a MemoryStream in .NET is not closed? for confirmation. (It's still good practice to do so, though in the case of Bitmap there's that pesky file/stream lock that makes it impossible.)
Edit:I am trying to use Constrained Execution Regions as a new weapon against Abort(). I am still running test, I hope it work. Abort() is really a bad thing. I will report later.. If anyone has any argument against CER please note that.
I am having a problem about .NET memory leaking.
I have a project, quite complex, threads in threads in threads, makes it hard to debug.
I know Abort() is depreciated, but I have my reasons to use them:
My method is a long method, I/O, network related. There's no explicit time-consuming operation that I can put a flag inside. If I put flags everywhere the code will be a mess.
When needed, the thread has to be terminated at once, sooner is better. All the work inside is no longer needed.
When I run the program normally, by which threads finish their work and die naturally one by one, everything is ok (I have run my program for 2 years, no memory leaking).
But if I try to create new thread, Start() and Abort() them frequently (1-2 threads/second, just a bit faster than the case I don't abort()), the memory leaks.
And the more wired thing is that, after the operation is finished, the memory occupied will stay high for a few time, like minutes or 10 minutes, but it will finally return to normal level, like nothing happened.
In debug mode, I can see no active thread, but memory leaked.
So I used .NET memory profiler to trace. Most instances occupied the memory are many byte[] referenced by MemoryStream.
Yes, I do use MemoryStream, but ALL of my MemoryStreams are inside using blocks, no exception. Everything should be in using block correctly.
As far as I know, Thread.Abort() is throwing an exception to force it to close, but as I confirmed in debug mode, all the threads are closed expectedly. Why there are still reference? Why the memory is released after some time? (But still much longer than the situation when I don't abort the thread and let it finish the work.)
IMO using blocks can guarantee that even the ThreadAbortException is thrown inside the Dispose() can be executed correctly.
Edit:
public static byte[] Recover(byte[] png)
{
using (MemoryStream pngStream = new MemoryStream(png))
{
using (System.Drawing.Image image = System.Drawing.Image.FromStream(pngStream))
{
using (MemoryStream bmpStream = new MemoryStream())
{
image.Save(bmpStream, System.Drawing.Imaging.ImageFormat.Bmp);
bmpStream.Seek(0, System.IO.SeekOrigin.Begin);
byte[] BMP = new byte[bmpStream.Length];
bmpStream.Read(BMP, 0, (int)bmpStream.Length);
return BMP;
}
}
}
}
public xxxxMission(byte[] png, Server server, int no) //constructor
{
byte[] bmp = xxxBMP.Recover(png); //png is generated by getBlankBMP();
//....
}
I'm trying to record a video from a certain window of an application (the window is showing some graphics build with DirectX) anyway
What I'm actually doing is, taking snapshots from that window and passing it to the AVIWriter from Aforge.net, I'm repeating the process 20 time per-second to get a video with 20fps.
Everything is working fine for 30 to 40 seconds, anything longer I get an out of memory exception.
Upon profiling with JetBrains dotMemory, I found that the unmanaged memory is increasing rapidly. Although the call to Dispose() is made to remove allocations. Still memory consumption is high.
I tried also using
GC.Collect();
GC.WaitForPendingFinalizers();
but without success
here my code below, it's running in a Backgroundworker which repeat every 50 ms until the button for CancellationPending is hit.
DisplayMode dm = device.DisplayMode;
Bitmap bmp = null;
Surface renderTarget = device.GetRenderTarget(0);
Surface destTarget = device.CreateOffscreenPlainSurface(ClientRectangle.Width, ClientRectangle.Height, dm.Format, Pool.SystemMemory);
device.GetRenderTargetData(renderTarget, destTarget);
GraphicsStream gs = SurfaceLoader.SaveToStream(ImageFileFormat.Bmp, destTarget);
renderTarget.Dispose();
destTarget.Dispose();
bmp = new Bitmap(gs);
gs.Dispose();
image = new Bitmap(bmp, size);
bmp.Dispose();
writer.AddFrame(image);
image.Dispose();
GC.Collect();
GC.WaitForPendingFinalizers();
I'm writing a simple image resizing program. By dragging multiple files onto the .exe, it will go through and resize each file. It works up to a certain point where an OOM (out of memory) exception is being thrown. I've tried calling Dispose on the bitmap and setting it to Null, but neither seems to do anything.
Bitmap current_image;
for (int i = 0; i < imagesfilepath.Count; ++i)
{
// Load the image.
if ( current_image != Null )
{
current_image.Dispose();
current_image = Null;
}
current_image = (Bitmap)Image.FromFile(imagesfilepath[i], true);
// Resize it.
// Save it.
}
The exception is generally thrown after 1.5 GB has been used. I can get around this issue by limiting the amount of images a user can resize at one time, but shouldn't I be able to just allocate memory for 1 Bitmap, and reuse it every iteration?
Image.FromFile() throws OutOfMemoryException when the file is not a valid image:
Exception Condition
OutOfMemoryException
The file does not have a valid image format.
-or-
GDI+ does not support the pixel format of the file.
Yes, this makes no sense and is confusing, but it is what it is.
MSDN: Image.FromFile
As long as you dispose of the images you should not receive the OutOfMemoryException. Tested with the following snippet where disposing allowed the program to finish successfully while not disposing caused the exception.
var path = #"C:\Users\mdearing\Desktop\Untitled.bmp";
for (var i = 0; i < 1000; i++)
{
var image = Bitmap.FromFile(path);
//image.Dispose(); //commenting this line out causes the OOM Exception
}
The out of memory is caused by memory segmentation, lack of a contiguous memory block of required size. you should rather use the same buffer to avoid it.
I write a code that read a png image from file and show with control.
I want read image from stream and set
control.BackgroundImage = Image.FromStream(memStream);
but when use this code , occur "out of memory" exception. but when use
control.Image = Image.FromStream(memStream);
or
control.BackgroundImage = Image.FromFile(fileSource);
, that is work.
image file size is 5KB.
if (System.IO.File.Exists(imgSource))
{
using (FileStream localFileStream = new FileStream(imgSource, FileMode.Open))
{
using (MemoryStream memStream = new MemoryStream())
{
int bytesRead;
byte[] buffer = new byte[1024];
while ((bytesRead = localFileStream.Read(buffer, 0, buffer.Length)) > 0)
{
memStream.Write(buffer, 0, bytesRead);
}
retIMG = Image.FromStream(memStream);
pictureBox1.Image = retIMG; // is work
label1.Image = retIMG; // is work
button1.Image = retIMG; // is work
button1.BackgroundImage = retIMG; // don't work
groupBox1.BackgroundImage = retIMG; // don't work
panel1.BackgroundImage = retIMG; // don't work
}
}
}
I think a bug in .net framework.
please you help me?
Read the remarks on Image.FromStream on MSDN:
You must keep the stream open for the lifetime of the Image.
So if you remove the using around the creation of your MemoryStream your code works fine.
Of course you should preferrably dispose the MemoryStream once you no longer need the Image you created, although there is likely no harm in this case in not calling Dispose() and leaving it up to the GC to collect it once unused.
The fact that it seems to work with some of your code is likely pure luck and should not be considered a working solution. Always read the documentation to find out about quirks like this.
Giving some background to add to DeCaf's correct answer. GDI+ tries very hard to avoid copying the pixels of a bitmap. That's expensive, bitmaps taking dozens of megabytes is not unusual. When you load a bitmap from a file with the Bitmap constructor or Image.FromFile() then GDI+ creates a memory-mapped file. The pixels are paged-in on demand, only when needed. Very efficient but it puts a lock on the file. Clearly you were trying to avoid that in lock in this code.
You indeed avoid that lock by loading the bytes into memory yourself with a MemoryStream. But the same principle still applies, GDI+ still doesn't copy the pixels and only reads from the stream when it needs to. This goes wrong when you Dispose() the stream. Very hard to diagnose because the exception occurs later, typically when the bitmap needs to be drawn. It bombs in the painting code, you don't have any code to look at but Application.Run(). With a crappy exception message, GDI+ only has a handful of error codes. You are not out of memory, it only looks that way to GDI+, it cannot otherwise figure out why the stream suddenly isn't readable anymore.
At least part of the problem is caused by the very awkward implementation of MemoryStream.Dispose(). Dispose is meant to release unmanaged resources. A memory stream doesn't have any, it only owns memory. That's already taken care of by the garbage collector. Unfortunately they implemented it anyway. Not by actually disposing anything, since there's nothing to dispose, but by marking the MemoryStream unreadable. Which triggers the error in GDI+ when it tries to read while drawing the bitmap.
So simply remove the using statement to avoid disposing the MemoryStream to solve your problem. And don't fret about disposing it later when the bitmap is no longer in use. There's nothing to dispose, the garbage collector automatically frees the memory.
Two things which together resolved this intermittent issue, which has nothing to do with image size.
First, ensure that the image is in RGB mode and definitely not CMYK mode. In our experience, the RGB rendering is actually larger.
Second, erase (and dispose of if possible) any previous image in the image container before loading the new image, such as
Control.Image = Nothing