I am working with a 32-bit Console Application that operates as a background processor. The part I am working on uses GhostScript to Perform OCR on PDFs. Each page of a PDF is rendered to a PNG image in a temp folder which the OCR Reader then reads. The OCR text is saved to a database and the files in the temp folder are then deleted.
The problem is with the GhostScriptRasterizer object eating all of the memory the processor has available. When I call the GhostScriptRasterizer.GetPage(dpi, dpi, pageNumber) method I get either get an OutOfMemory Exception or a System.ArgumentException with Message "Parameter is not valid". My research on the second exception tells me it is really a symptom of the first. The method call eats all of the avialable memory.
The GetPage method is creating a System.Drawing.Bitmap image which requires contiguous unfragmented memory. The problem code begins here.
try
{
img = rasterizer.GetPage(dpi, dpi, pageNumber);
}
catch (OutOfMemoryException ex)
{
img = GetImage(rasterizer, dpi, pageNumber, ms);
}
catch (System.ArgumentException ex)
{
img = GetImage(rasterizer, dpi, pageNumber, ms);
}
The GetImage method I wrote looks like this.
public Image GetImage(GhostscriptRasterizer rasterizer, int dpi, int pageNumber, MemoryStream ms)
{
rasterizer.Close();
rasterizer.Dispose();
rasterizer = new GhostscriptRasterizer();
rasterizer.Open(ms);
dpi = dpi - 50;
Image image = null;
if (dpi > 0)
{
try
{
image = rasterizer.GetPage(dpi, dpi, pageNumber);
}
catch (OutOfMemoryException ex)
{
image = GetImage(rasterizer, dpi, pageNumber, ms);
}
catch (System.ArgumentException ex)
{
image = GetImage(rasterizer, dpi, pageNumber, ms);
}
}
return image;
}
The dpi I start with is 300 and it has worked for 95% of the documents we have run through our first test of this system. However for certain pages 300 dpi is clearly too high as I get the Outofmemory exception. It looks like some of the pages are about 35 X 59 inches. I have no control over this. The solution for me is to keep trying at a lower and lower dpi until I have something that doesn't eat all of the memory. However, all of that memory remains in the rasterizer object so I need to dispose of it somehow. Calling rasterizer.Close() gives me the following error.
Managed Debugging Assistant 'FatalExecutionEngineError' has detected a problem in 'F:\Development\bin\Debug\Processor.Run.vshost.exe'.
Additional information: The runtime has encountered a fatal error. The address of the error was at 0x7331e8c6, on thread 0x3e90. The error code is 0xc0000005. This error may be a bug in the CLR or in the unsafe or non-verifiable portions of user code. Common sources of this bug include user marshaling errors for COM-interop or PInvoke, which may corrupt the stack.
Removing the Close() call and calling rasterizer.Dispose() gives me:
An unhandled exception of type 'System.AccessViolationException' occurred in Ghostscript.NET.dll
Additional information: Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
I even just tried to just break if I hit an exception and return the file list and this still required me to not use a using declaration for the rasterizer because I got the same exception at the end of the using because of course it is trying to Dispose of the object. It appears the garbage collector picked up that memory later down the line but that does not in any way solve my problem. I still have no way of rasterizing the page within the same job.
The only solution I can think of is somehow resizing the pdf ahead of time but I'm hoping someone knows a way of disposing that memory and re-rasterizing at a new lower dpi.
You can write PostScript that alters the media size when a PDF requests a large media. But that would require some PostScript programming knowledge.
I believe the actual problem is not Ghostscript, however, because when exceeding memory limits Ghostscript will switch to a display list model where it outputs the page to disk in bands (running the display list as many times as there are bands to output). Provided you actually have a disk, which you clearly do, and there's enough memory for one raster line, then it will (eventually in the case of one band per line) output the whole thing.
Which suggests to me the actual problem is with the C++ or C# wrapper you are using, not Ghostscript tself.
I suspect that your wrapper is trying to create a huge bitmap in memory to hold the rendered output before writing it to disk. That isn't required.
Try running Ghostscript directly from the command line with one of your failing files, if that works then you can simply use Ghostscript, its perfectly capable of producing a PNG file as output. For what its worth I have used Ghostscript to output media of that size, and larger, at 600 dpi.
I have a similar issue, I get the "Attempted to read or write protected memory" when disposing memory after an exception occurs. This happens when I am trying to convert a password-protected PDF - even after catching the exception, the above access violation occurs and crashes the program.
The solution I used:
I am also using iTextSharp in my program. So I wrote a method using iTextSharp to check if the PDF file is password protected first, using help from this thread: https://stackoverflow.com/questions/11298651/checking-if-pdf-is-password-protected-using-itextsharp#=
So now I am checking for the problem before I run into it. It's the only way I've found around this problem - I don't think the Ghostscript.NET wrapper is being updated or maintained any more.
I used HandleProcessCorruptedStateExceptionsAttribute and SecurityCritical attributes on top of my method which is calling GhostScript method.
This got issue fixed for me. I no more get this exception.
Related
Having a code that works for ages when loading and storing images, I discovered that I have one single image that breaks this code:
const string i1Path = #"c:\my\i1.jpg";
const string i2Path = #"c:\my\i2.jpg";
var i = Image.FromFile(i1Path);
i.Save(i2Path, ImageFormat.Jpeg);
The exception is:
System.Runtime.InteropServices.ExternalException occurred
A generic error occurred in GDI+.
at System.Drawing.Image.Save(String filename, ImageCodecInfo encoder, EncoderParameters encoderParams)
at System.Drawing.Image.Save(String filename, ImageFormat format)
at ...
As far as I can see, there is nothing special about the image. It is approx 250 pixels in size and can be opened in e.g. Windows Image Viewer or Paint.NET:
(Since the image above, after being uploaded to Stack Overflow does not produce the error anymore, I've put the original image here)
What I discovered is that upon calling the Save method, the destination image file is being created with zero bytes.
I am really clueless on what causes the error.
My questions:
Can you think of any special thing that would hinder .NET from saving the image?
Is there any way (beside panicing) to narrow down these kind of errors?
While I still did not find out the reason what exactly caused the error when saving the image, I found a workaround to apply:
const string i1Path = #"c:\my\i1.jpg";
const string i2Path = #"c:\my\i2.jpg";
var i = Image.FromFile(i1Path);
var i2 = new Bitmap(i);
i2.Save(i2Path, ImageFormat.Jpeg);
I.e. by copying the image internally into a Bitmap instance and saving this image instead of the original image, the error disappeared.
I'm assuming that by copying it, the erroneous parts the caused the original Save call to fail are being removed an/or normalized, thus enabling the save operation to succeed.
Interestingly, the so stored image has a smaller file on disk (16 kB) than its original source (26 kB).
First of all make sure, that the desired folder has Read/Write permissions. Changing the permissions solved this problem for me.
Solution is here, you must dispose image object to release the memory on the server.
Try use using statement. Make sure destination directory on server exists too.
The reason may be that the image is loaded lazily and the loading process is not yet finished when you try to save it.
Following what's said in this blog post (assuming you're German by the picture you linked in your question) provides a possible solution. Also this SO question's accepted answer indicates this is due to the fact the image file you're trying to save to is locked.
EDIT
For Ulysses Alves, from the linked blog entry: If you load an image using Image.FromFile() it remains locked until it is disposed of. This prevents calls to Save().
pictureBox1.Image = Image.FromFile("C:\\test\\test1.jpg");
pictureBox1.Image.Save("C:\\test\\test2.jpg");
The above code throws an error.
To make it work, you need to copy the image. The following code works:
pictureBox1.Image = Image.FromFile("C:\\test\\test1.jpg");
Image copy = pictureBox1.Image;
copy.Save("C:\\test\\test2.jpg")
I found this question because I also faced the similar error and the file was actually created with zero length (if you don't see any file, first check the permissions to write into folder as other answers suggest). Although my code was slightly different (I use stream to read the image from memory, not from file), I think my answer may be helpful to anyone facing similar problem.
It may looks counter-intuitive, but you can't really dispose memory stream until you finish with image.
NOT WORKING:
Image patternImage;
using (var ms = new MemoryStream(patternBytes)) {
patternImage = new Bitmap(ms);
}
patternImage.Save(patternFile, ImageFormat.Jpeg);
Just don't dispose the stream until you done with image.
WORKS:
using (var ms = new MemoryStream(patternBytes)) {
patternImage = new Bitmap(ms);
patternImage.Save(patternFile, ImageFormat.Jpeg);
}
What is misleading:
Error message doesn't really tell you anything
You can see the image properties, like width and height, but can't
save it
my solution was to make, write temp content (File.WriteAllText) just before saving the file
Here is the code:
var i = Image.FromFile(i1Path);
File.WriteAllText(i2Path, "empty"); // <---- magic goes here
i.Save(i2Path, ImageFormat.Jpeg);
Please try and let me know
In my case I have accidentally deleted the directory where image was getting stored.
Key Information:
// Using System.Drawing.Imaging:
new Bitmap(image).Save(memoryStream, ImageFormat.Jpeg);
You MUST Cast the Image to a Bitmap to Save it.
Using:
// Using System.Drawing.Imaging:
image.Save(memoryStream, ImageFormat.Jpeg);
WILL throw the Error:
Generic GDI+ error when saving an image
Just use the visual studio as administrator or run the application created by the code as administrator it should work smoothly.
It is user access rights issue.
I faced the same and resolved it by running visual studio as administrator.
In my case, I set validateImageData to false:
Image.FromStream(stream, validateImageData: false);
solution:
Image.FromStream(stream, validateImageData: true);
Open in the program
const string i1Path = #"c:\my\i1.jpg";
const string i2Path = #"c:\my\i2.jpg";
var i = Image.FromFile(i1Path);
i.Save(i2Path, ImageFormat.Jpeg);
i.Dispose();
I am saving images on the server and I am covering the scenario where the disk has no available free space. To simulate that, I created an almost-no-space virtual hard drive and I am trying to save the images there.
I was expecting to receive an exception when trying to save the image, with:
using (var ms = new MemoryStream(myPictureStream))
using (Image image = Image.FromStream(ms))
{
image.Save(fileName, ImageFormat.Jpeg);
}
But that is not the case, there is no exception thrown. Instead, an empty file is saved, so I can not be aware of the error.
I would like to receive some error when trying to save the image, or being able to anticipate the error before an exception is thrown.
I already considered some possible solutions:
I know I can check the available free space on a drive with DriveInfo, but I will not have the drive letter to check that on production, so this option is discarded.
After saving the image, I try to retrieve it with Image.FromFile and then an OutOfMemoryException is thrown, but I know that this can happen in other scenarios, and I am not sure if this is a legitimate way of checking that the disk has no free space.
So... Any idea why Image.Save() does not throw errors when no free space available? Any suggestion on how to handle this scenario?
EDIT
GetLastError does not help (it is code 0, success), so it seems that the best way to handle this scenario is saving the image to a temporal MemoryStream and then writing it into a file, as pointed out by answers given by #Eldar and #nvoigt.
Doing it this way, a IOException is thrown, and you can check the HResult of the exception to see if it is a not-enough-free-space-in-disk easy (see this answer: https://stackoverflow.com/a/9294382/4067893).
Image.Save() internally calls a GDI function GdipSaveImageToFile() that doesn't fail if the drive space is insufficient. You may give a try to GetLastError() to get the last Win32 error.
If GetLastError() doesn't help, the only solution that comes up in my mind is saving the image to a temporal MemoryStream, then writing the resulting stream into a file.
You will need to do the stream handling yourself:
using (var ms = new MemoryStream(myPictureStream))
using (Image image = Image.FromStream(ms))
using(var jpg = new MemoryStream())
using (var file = new FileStream(fileName, FileMode.Create, FileAccess.Write))
{
image.Save(jpg, ImageFormat.Jpeg);
jpg.Seek(0, SeekOrigin.Begin);
jpg.WriteTo(file);
}
I'm trying to use this class but I'm getting a Generic error occured in GDI+ in Image.Save() method. From what I read it's some stream I need to close but I don't know which one.
I'm calling using this:
Image image = Image.FromFile(#"C:\a.jpg");
using (var resized = ImageUtilities.ResizeImage(image, 50, 100))
{
//save the resized image as a jpeg with a quality of 90
ImageUtilities.SaveJpeg(#"C:\myimage.jpeg", resized, 90);
}
Why is that error and how do I solve this?
Unless your program is running as administrator you can not save directly to the root of C: make a folder and save it inside there instead.
Have you tested saving the images in different locations?
If it is still failing then without knowing exactly what is going on in your code I would hazard a guess to say that the original image is getting disposed somewhere before it should be. That's usually the most common cause of the error.
I've written a library that handles many different imaging operations whilst ensuring that memory is correctly handled. It's well tested and very simple to use.
You can get it here. http://imageprocessor.org/
Example code using the library:
using (ImageFactory imageFactory = new ImageFactory())
{
// Load, resize, set the quality and save an image.
imageFactory.Load(#"C:\a.jpg")
.Resize(new Size(50, 100))
.Quality(90)
.Save(#"C:\myimage.jpeg);
}
I am trying to write a windows application to convert .avi videos to bitmap frames. I am able to get the bitmaps but I am having problems saving them.
Frames are saved perfectly up to 1649th frame. After that I get this exception:
Attempted to read or write protected memory. This is often an indication that other memory is corrupt
I ran the code several times, the code always throw above exception when processing 1649th frame. The output folder is empty at the beginning and its size is 389 MBs when the program stops.
I am guessing that windows do not allow a program to write this amount of data in a short interval but I am not sure and I don't know how to fix it. Can anyone help?
for(counter = reader.Start; counter<(reader.Start + reader.Length); counter++)
{
DummyBitmap = reader.GetNextFrame();
DummyBitmap.Save(folderBrowserDialog2.SelectedPath + "\\" + counter.ToString() + ".bmp");
reader.Position++;
}
Bitmap class implements IDisposable interface so it would be wise to use it as:
using (var b = new Bitmap(...))
{
}
Also, maybe this post can give you some answers: Bitmap memory leak.
Is it possible to use the FromStream method of System.Drawing.Image without having to keep the stream open for the lifetime of the image?
I have an application which loads a bunch of toolbar graphics from resource files, using a combination of Image.FromStream and Assembly.GetManifestResourceStream.
The problem I'm having is while this works fine on Windows 7, on Windows XP the application crashes if a user interface element linked to one of these images is disabled. On Windows 7, the image is rendered in grayscale. On XP, it crashes with an out of memory exception.
After a load of hairpulling I have finally traced it to the initial loading of the image. As a matter of course, if I create any object implementing IDisposable that is also destroyed in the same method, I wrap it in a using statement, for example
using (Stream resourceStream = assembly.GetManifestResourceStream(resourceName))
{
image = Image.FromStream(resourceStream);
}
If I remove the using statement so that the stream isn't disposed, then the application no longer crashes on XP. But I now have a bunch of "orphan" streams hanging about - the images are stored in command classes and these correctly dispose of the images when they themselves are disposed, but the original stream isn't.
I checked the documentation for FromStream and it confirms the stream needs to remain open. Why this hasn't crashed and burned on the Windows 7 development system is a mystery however!
I really don't want this stream hanging around, and I certainly don't want to have to store a reference to this stream as well as the image so I can dispose of it later. I only have need of that stream once so I want to get rid of it :)
Is it possible to create the image and then kill of the stream there and then?
The reason the stream needs to be open is the following:
GDI+, and therefore the System.Drawing namespace, may defer the decoding of raw image bits until the bits are required by the image. Additionally, even after the image has been decoded, GDI+ may determine that it is more efficient to discard the memory for a large Bitmap and to re-decode later. Therefore, GDI+ must have access to the source bits for the image for the life of the Bitmap or the Image object.
The documented workaround is to create either a non-indexed image using Graphics.DrawImage or to create an indexed Bitmap from the original image as described here:
Bitmap and Image constructor dependencies
According to the documentation of Image.FromStream, the stream must be kept open while the image is in use. Therefore, even if closing worked (and there's nothing to say you can't close a stream before it's disposed, as far as the stream object itself goes) it may not be a very reliable approach.
You could copy the image to another image object, and use that. However, this is likely to be more memory intensive than just keeping the stream open.
You could save the stream to a temporary file and use the Image.FromFile method. Or simply don't embed the image, keep it as a file and load it from this file at runtime.
I'am sure this will help someone :)
I used it for my dataGridView_SelectionChanged:
private void dataGridViewAnzeige_SelectionChanged(object sender, EventArgs e)
{
var imageAsByteArray = File.ReadAllBytes(path);
pictureBox1.Image = byteArrayToImage(imageAsByteArray);
}
public Image byteArrayToImage(byte[] byteArrayIn)
{
MemoryStream ms = new MemoryStream(byteArrayIn);
Image returnImage = Image.FromStream(ms);
return returnImage;
}