I am using EmguCV library to stitch images. Its working fine for small images but getting exception when processing high resolution images or images above 20MB size or even if I try to process more than 30 images it fails.
Libraries I am using
Emgu.CV.UI
Emgu.CV.UI.GL
Emgu.CV.World
opencv_core2410
opencv_imgproc2410
Code
List<Image<Bgr, Byte>> sourceImages = new List<Image<Bgr,byte>>();
foreach (string path in ImgPath)
sourceImages.Add(new Image<Bgr, Byte>(path));
using (Stitcher stitcher = new Stitcher(false))
{
using (VectorOfMat vm = new VectorOfMat())
{
Mat result = new Mat();
vm.Push(sourceImages.ToArray());
stitcher.Stitch(vm, result);
if (result.Bitmap != null)
{
result.Bitmap.Save(Application.StartupPath + "\\imgs\\StitchedImage.png");
}
else
{
MessageBox.Show("Some thing went wrong"); return null;
}
}
}
Exception
((Emgu.CV.MatDataAllocator)(result))._dataHandle.Target' threw an exception of type 'System.InvalidOperationException
Image
I was fairly certain that you are running into a memory issue and so I went ahead and made a simple console app targeting .Net 4.7.2, using the latest EmguCV package from NuGet, which is version 3.4.3.3016, and using the code below on sample data from Adobe, which can be downloaded here. If I compile as "AnyCPU" with "prefer 32 bit" and run this code against the rio image set (I loaded up the png's for the test purposes) and let it run the memory will slowly jump up until it hits close to 3 GB and then shortly thereafter crashes giving the exception about the refcount. Pretty clearly a memory issue. I then recompiled targeting 64 bit and was able to successfully run the code. The memory usage peaked out around 6 GB. So, with that in mind, I would be fairly certain that your issue is also memory related. You have yet to answer the question on whether you are building a 64 bit app or not, but based on what you are seeing, I would guess that you are not. So, the solution to your problem is to compile as 64 bit and then be sure you have enough memory. With the rio test set it jumped up to close to 6 GB. Without having your images I can't tell you how large it might grow, but these type of operations are pretty memory intensive - so more is better. This would explain both the issue with large image files and the issue with a large number of small image files. I was able to successfully process sets of images that were between 10 and 20 images using a 32 bit build, but as soon as I moved to the 50+ image sets it would only work with a 64 bit build due to the memory requirements.
var images = Directory.EnumerateFiles(#"C:\test\adobe\rio", "*.png", SearchOption.TopDirectoryOnly).Select(x => new Mat(x)).ToArray();
using(var stitcher = new Stitcher(false))
{
using (var vm = new VectorOfMat(images))
{
var result = new Mat();
stitcher.Stitch(vm, result);
result.Bitmap.Save(#"C:\test\adobe\rio_stitched.jpg", System.Drawing.Imaging.ImageFormat.Jpeg);
result.Dispose();
}
}
foreach (var image in images) { image.Dispose(); }
Related
I'm working on a small personal application that should read some text (2 sentences at most) from a really simple Android screenshot. The text is always the same size, same font, and in approx. the same location. The background is very plain, usually a few shades of 1 color (think like bright orange fading into a little darker orange). I'm trying to figure out what would be the best way (and most importantly, the fastest way) to do this.
My first attempt involved the IronOcr C# library, and to be fair, it worked quite well! But I've noticed a few issues with it:
It's not 100% accurate
Despite having a community/trial version, it sometimes throws exceptions telling you to get a license
It takes ~400ms to read a ~600x300 pixel image, which in the case of my simple image, I consider to be rather long
As strange as it sounds, I have a feeling that libraries like IronOcr and Tesseract may just be too advanced for my needs. To improve speeds I have even written a piece of code to "treshold" my image first, making it completely black and white.
My current IronOcr settings look like this:
ImageReader = new AdvancedOcr()
{
CleanBackgroundNoise = false,
EnhanceContrast = false,
EnhanceResolution = false,
Strategy = AdvancedOcr.OcrStrategy.Fast,
ColorSpace = AdvancedOcr.OcrColorSpace.GrayScale,
DetectWhiteTextOnDarkBackgrounds = true,
InputImageType = AdvancedOcr.InputTypes.Snippet,
RotateAndStraighten = false,
ReadBarCodes = false,
ColorDepth = 1
};
And I could totally live with the results I've been getting using IronOcr, but the licensing exceptions ruin it. I also don't have $399 USD to spend on a private hobby project that won't even leave my own PC :(
But my main goal with this question is to find a better, faster or more efficient way to do this. It doesn't necessarily have to be an existing library, I'd be more than willing to make my own kind of letter-detection code that would work (only?) for screenshots like mine if someone can point me in the right direction.
I have researched about this topic and the best solution which I could find is Azure cognitive services. You can use Computer vision API to read text from an image. Here is the complete document.
How fast does it have to be?
If you are using C# I recommend the Google Cloud Vision API. You pay per request but the first 1000 per month are free (check pricing here). However, it does require a web request but I find it to be very quick
using Google.Cloud.Vision.V1;
using System;
namespace GoogleCloudSamples
{
public class QuickStart
{
public static void Main(string[] args)
{
// Instantiates a client
var client = ImageAnnotatorClient.Create();
// Load the image file into memory
var image = Image.FromFile("wakeupcat.jpg");
// Performs label detection on the image file
var response = client.DetectText(image);
foreach (var annotation in response)
{
if (annotation.Description != null)
Console.WriteLine(annotation.Description);
}
}
}
}
I find it works well for pictures and scanned documents so it should work perfectly for your situation. The SDK is also available in other languages too like Java, Python, and Node
I am generating a large Report pdf file using PDFClown using data from a database.
The process takes a very long time and eventually runs out of memory when the number of pages nears the 150 mark taking up more than 1.5GB of ram and with the error:
A first chance exception of type 'System.OutOfMemoryException' occurred in PDFClown.dll
Since I will need to regularly generate reports of over 1500 pages this is a major problem.
Is there anything I can do to:
Not run out of memory (Necessary)
Speed up the file creation (Ideally)
Please note: the reports generated (with smaller data sets) are accurate, although the file size is rather large.
Here is a sample of my code:
protected void PopulateReport()
{
foreach (Page page in _lstPages)
{
if (page != _Titlepage)
{
PrimitiveComposer composer = new PrimitiveComposer(page);
BlockComposer blockComposer = new BlockComposer(composer);
DataRow drInspection;
if (_mapPage1Rows.TryGetValue(page, out dataRow))
{
GeneratePage1(page, composer, blockComposer, dataRow);
}
else if (_mapPage2Rows.TryGetValue(page, out dataRow))
{
GeneratePage2(page, composer, blockComposer, dataRow);
}
}
}
}
protected void GeneratePage1()
{
composer.BeginLocalState();
composer.SetFont(ReportFonts.GetFont(GetPDFDocument(), bIsFieldName, false), nFontSize);
blockComposer.Begin(GetRectangle(fMarginX, fMarginY, fWidth, nFontSize), AlignX, AlignY);
int nIndex = blockComposer.ShowText(strText, false);
blockComposer.End();
....
composer.End();
composer.Flush();
}
Screenshot Sample Report Page (redacted for client-privacy reasons):
The Function: ReportFonts.GetFont(...) was creating a new font every single time it was called.
This font was then stored in the dll's memory space and the final file which was what was taking up so much space.
Used a Dictionary<> to solve the issue, not only is the memory space clean and the file size acceptable, but the execution time is also much improved.
Moving to 64-bit also helped somewhat.
I am writing a SharpDX (v4.0.1) application that displays assemblies composed of many (possibly very big) separate parts. Each part has its own vertex and index buffer. When I try to load a very big assembly, I eventually run into E_OUTOFMEMORY exception when trying to create a vertex buffer for a part:
int size = _meshVertices.Length * Utilities.SizeOf<MeshVertex>();
BufferDescription descr = new BufferDescription(size, ResourceUsage.Default, BindFlags.VertexBuffer, CpuAccessFlags.None, ResourceOptionFlags.None, 0);
_vertexBuffer = Buffer.Create(_device.Device, _meshVertices, descr);
$exception {"HRESULT: [0x8007000E], Module: [General], ApiCode: [E_OUTOFMEMORY/Out of memory], Message: Not enough storage is available to complete this operation.\r\n"} SharpDX.SharpDXException
When I was googling a possible solution, I found out that there is a size limit on a resource - but this doesn't seem to be the case as it happens even for smaller parts.
Another solution I tried was using the MemoryFailPoint class to try to reserve the memory before creating the buffer. But I suppose this checks for available RAM, while I believe the exception is caused by having not enough GPU memory. Anyway, this seems to alleviate the problem a bit but it doesn't fix it completely. It just takes longer before it fails.
int size = _meshVertices.Length * Utilities.SizeOf<MeshVertex>();
using (System.Runtime.MemoryFailPoint memFailPoint = new System.Runtime.MemoryFailPoint(size / 1024 / 0124))
{
BufferDescription descr = new BufferDescription(size, ResourceUsage.Default, BindFlags.VertexBuffer, CpuAccessFlags.None, ResourceOptionFlags.None, 0);
_vertexBuffer = Buffer.Create(_device.Device, _meshVertices, descr);
}
My problem is that after I get the exception, the device is lost. I assume that the resources such as vertex buffers are bound to the device, so to recreate the device after it is lost would mean to recreate all the resources which would be very time consuming.
So preferably, I am looking for a way to know that the device does not have enough memory before I try to create the buffer, so that I could just discard the single part if needed. Is it possible to do something like that?
Any help would be very much appreciated. I am new to SharpDX so if you think I misunderstood the problem and something else is causing it, please let me know as well. Thanks in advance for any input on this.
We have started facing an issue in the past months where the method System.Drawing.Bitmap.Save(string filename) stops working after the server has been running for several hours. The hours after which it starts failing are proportional to the server load.
This code involved with that call has been working perfectly well for several years, it gets and image from the user, resizes it and saves it to disk. The application is an ASP.Net application developed .Net Framework 3.5.
When the call to the Bitmap.Save fails, it raises the infamous exception:
ERROR MESSAGE: Error genérico en GDI+. STACK TRACE:
en System.Drawing.Image.Save(String filename, ImageCodecInfo encoder, EncoderParameters encoderParams)
en System.Drawing.Image.Save(String filename, ImageFormat format)
en System.Drawing.Image.Save(String filename)
I've been reading a lot of posts about that exception, and I am fully aware that this exception is raised in many different situations. It is a generic exception that is raised for many different reasons (i.e when the application has no permisison to write to a folder, when the code is trying to save the image in the same file it is in reading from, etc) but none of those scenarios is our case.
When we realied the method was failing, we had just released a new version of our application, so we initially thought the error could be in our code (despite that part had not been changed, we thought we had changed something else that was interfering and producing the error), but the more we have investigated the less we understand when the error happens.
It is also important to mention that at the same time the error appeared, we also updated our servers with the latest windows updates, where I see also two suspicious ones:
KB2929755 http://support.microsoft.com/kb/2929755/en-us
Out of memory when you load some image resources in a Windows application
KB2957503 http://support.microsoft.com/kb/2957503/en-us
MS14-036: Description of the security update for Windows 7, Windows Server 2008 R2, Windows Server 2008, Windows Vista, and Windows Server 2003: June 10, 2014
This ones updates the GDIplus.dll on the server to avoid some security issues.
My feeling is that something on the server is not being released, like if some piece of code were leaking handlers or memory, and at some point the server runs out of them and can no longer save the Bitmap. When the error appears, any call to Bitmap.Save from any part of the application raises that exception. Rebooting the IIS solves the problem until the next time, however, that's not the case if we only restart the application pool.
If it can help in any way, we are using the HiQPdf library (version 6.8.0.0) to convert some HTML documents to PDF. This tool creates a new process for every conversion and killing it when finished, and supposedly releasing all the resources used in a proper way.
Is someone exeperiencing any similar issues?
Has someone had any issues with the Windows updates described?
I forgot to mention also that we tried resizing the images using WPF APIs, instead of GDI+ but we were facing the same problem, after several hours the server started raising an error (although this time the message was Excepción de HRESULT: 0x88982F8A), so we reverted it back to our original code.
Any help would be appreciated!
EDIT:
Actually, the code that peforms the image resizing and saving is the following. All variables are within using blocks, but for the casted image to a Bitmap, which I assume gets disposed anyway:
using (Image originalLogo = Image.FromStream(fuLogo.PostedFile.InputStream))
{
using (Image resizedLogo = ImageUtil.ResizeImage(originalLogo, maxWidth, maxHeight))
{
((System.Drawing.Bitmap)resizedLogo).Save(newFilePath);
}
}
Could the casting to Bitmap require creating an intermediate variable to force its dispose too, or that is not needed?
using (Image originalLogo = Image.FromStream(fuLogo.PostedFile.InputStream))
{
using (Image resizedLogo = ImageUtil.ResizeImage(originalLogo, maxWidth, maxHeight))
{
using (Bitmap bitmap=((System.Drawing.Bitmap)resizedLogo))
{
bitmap.Save(newFilePath);
}
}
}
Thanks,
Enric
EDIT:
I'm pasting below the code for the ResizeImage method:
public static System.Drawing.Image ResizeImage(System.Drawing.Image originalImage, int maxWidth, int maxHeight)
{
int imgWidth = originalImage.Width;
int imgHeight = originalImage.Height;
if (originalImage.Height<maxHeight && originalImage.Width<maxWidth) return originalImage;
double widhtRatio = ((double)maxWidth) / ((double)imgWidth);
double heightRatio = ((double)maxHeight) / ((double)imgHeight);
int targetWidth, targetHeight;
if (widhtRatio < heightRatio)
{
targetWidth = (int)(imgWidth * widhtRatio);
targetHeight = (int)(imgHeight * widhtRatio);
}
else
{
targetWidth = (int)(imgWidth * heightRatio);
targetHeight = (int)(imgHeight * heightRatio);
}
//Not all pixel formats are supported, therefore, using the
//originalImage pixel format raises an exception if the pixel format is not supported.
//By not specifying it, the Bitmap constructor will use one compatible with the FromImage method.
//For more info see: http://forums.asp.net/p/1195630/2066153.aspx
// Image resizedImage = new Bitmap(targetWidth, targetHeight, originalImage.PixelFormat);
System.Drawing.Image resizedImage = new Bitmap(targetWidth, targetHeight);
using (Graphics riGraphics = Graphics.FromImage(resizedImage))
{
riGraphics.CompositingQuality = CompositingQuality.HighQuality;
riGraphics.SmoothingMode = SmoothingMode.HighQuality;
Rectangle rectangle = new Rectangle(0, 0, targetWidth, targetHeight);
riGraphics.DrawImage(originalImage, rectangle);
riGraphics.Flush();
}
return resizedImage;
}
I've got an absolute stab in the dark for you:
In your application code, is the bitmap object wrapped in a using or explicitly disposed of after the Save() is called? I've seen hard to trace exceptions thrown from memory leaks in the past when dealing with IDisposible objects when they're used in repetition and don't get properly disposed of. These circumstances always seemed to be intermittent like what you're describing and took tons of digging before realizing the solution was as simple as calling Bitmap.Dispose() (or putting it back after having it accidentally deleted during a re-factor or code update).
However unlikely, because it sounds like your code worked before having windows updates applied...I thought I'd throw that out there.
After several days of tracking down bizarre GDI+ errors, I've stumbled across this little gem on MSDN:
Classes within the System.Drawing namespace are not supported for use within a Windows or ASP.NET service. Attempting to use these classes from within one of these application types may produce unexpected problems, such as diminished service performance and run-time exceptions.
I don't know whether "ASP.NET service" means "web application" in this context, but "diminished service performance" certainly seems to cover the random assortment of "A generic error occurred in GDI+" and "Out of memory" errors that my app is throwing - intermittent, non-reproducible errors reading and writing JPEG images that - in many cases - were actually created by System.Drawing.Imaging in the first place.
So - if GDI+ can't read and write JPEG files reliably in a Web app, what should I be using instead?
I want users to be able to upload images (JPEG required, other formats nice-to-have), resample them reliably, and display useful error messages if anything goes wrong. Any ideas? Are the System.Media namespaces from WPF worth considering?
EDIT: Yeah, I know GDI+ works "most of the time". That's not good enough, because when it fails, it does so in a way that's impossible to isolate or recover from gracefully. I am not interested in examples of GDI+ code that works for you: I am looking for alternative libraries to use for image processing.
There is an excellent blog post including C# code about using the ImageMagick graphics library through Interop over at TopTen Software Blog. This post deals specifically with running ASP.net on linux under mono; however, the C# code should be perfectly copy-paste-able, the only thing you'll need to change is the Interop attributes if you are running under windows referencing a window binary (DLL).
ImageMagick® is a software suite to create, edit, compose, or convert
bitmap images. It can read and write images in a variety of formats
(over 100) including DPX, EXR, GIF, JPEG, JPEG-2000, PDF, PhotoCD,
PNG, Postscript, SVG, and TIFF. Use ImageMagick to resize, flip,
mirror, rotate, distort, shear and transform images, adjust image
colors, apply various special effects, or draw text, lines, polygons,
ellipses and Bézier curves.
There is also an ImageMagick .Net development project on codeplex that wraps up everything for you. But it doesn't show active development since 2009, so it may be lagging behind the current ImageMagick library version. For a small trivial resizing routine, I'd probably stick with the interop. You just need to watch your implementation carefully for your own memory leak or unreleased resources (the library itself is well tested and vetted by the community).
The library is free and open source. The Apache 2 license appears to be compatible with both personal and commercial purposes. See ImageMagick License Page.
The library is totally cross platform and implements many powerful image handling and transformation routines that are not found in GDI+ (or not implemented under mono) and has a good reputation as an alternative for ASP.net image processing.
Update: Looks like there is an updated version of a .NET wrapper here: http://magick.codeplex.com/
Yes, use the WPF System.Windows.Media classes. Being fully managed they don't suffer the same problems as the GDI stuff.
Here's an excerpt from some MVC code I use to render gradients, to give you an idea how to get from a WPF Visual to a PNG:
using System;
using System.IO;
using System.Web.Mvc;
using System.Windows;
using System.Windows.Media;
using System.Windows.Media.Imaging;
namespace MyMvcWebApp.Controllers
{
public class ImageGenController : Controller
{
// GET: ~/ImageGen/Gradient?color1=red&color2=pink
[OutputCache(CacheProfile = "Image")]
public ActionResult Gradient(Color color1, Color color2, int width = 1, int height = 30, double angle = 90)
{
var visual = new DrawingVisual();
using (DrawingContext dc = visual.RenderOpen())
{
Brush brush = new LinearGradientBrush(color1, color2, angle);
dc.DrawRectangle(brush, null, new Rect(0, 0, width, height));
}
return new FileStreamResult(renderPng(visual, width, height), "image/png");
}
static Stream renderPng(Visual visual, int width, int height)
{
var rtb = new RenderTargetBitmap(width, height, 96, 96, PixelFormats.Default);
rtb.Render(visual);
var frame = BitmapFrame.Create(rtb);
var encoder = new PngBitmapEncoder();
encoder.Frames.Add(frame);
var stream = new MemoryStream();
encoder.Save(stream);
stream.Position = 0;
return stream;
}
}
}
You can find a very good article from a Microsoft Employee here: Resizing images from the server using WPF/WIC instead of GDI+ that proposes to use WPF instead of GDI+. It's more about thumbnailing but it's overall the same issues.
Anyway, at the end it states this:
I contacted the WPF team to have the final word on whether this is
supported. Unfortunately, it's not, and the documentation is being
updated accordingly. I apologize about any confusion this may have
caused. We're looking at ways to make that story more acceptable in
the future.
So WPF is also unsupported in web apps and still is I believe :-S
ImageSharp
ImageSharp is an open-source cross-platform 2D graphics library. It's written in C# on top of the new .NET Standard, with no dependency on any OS-specific API.
It's currently still in pre-release on MyGet (you'll have to add the package source in the VS options or a NuGet.config file), but we are already using it with some very positive results.
Most of the issues I have read about pertain to resources not being disposed properly.
I have used variants of this code time and time again with no issues from web applications:
public void GenerateThumbNail(HttpPostedFile fil, string sPhysicalPath,
string sOrgFileName,string sThumbNailFileName,
System.Drawing.Imaging.ImageFormat oFormat, int rez)
{
try
{
System.Drawing.Image oImg = System.Drawing.Image.FromStream(fil.InputStream);
decimal pixtosubstract = 0;
decimal percentage;
//default
Size ThumbNailSizeToUse = new Size();
if (ThumbNailSize.Width < oImg.Size.Width || ThumbNailSize.Height < oImg.Size.Height)
{
if (oImg.Size.Width > oImg.Size.Height)
{
percentage = (((decimal)oImg.Size.Width - (decimal)ThumbNailSize.Width) / (decimal)oImg.Size.Width);
pixtosubstract = percentage * oImg.Size.Height;
ThumbNailSizeToUse.Width = ThumbNailSize.Width;
ThumbNailSizeToUse.Height = oImg.Size.Height - (int)pixtosubstract;
}
else
{
percentage = (((decimal)oImg.Size.Height - (decimal)ThumbNailSize.Height) / (decimal)oImg.Size.Height);
pixtosubstract = percentage * (decimal)oImg.Size.Width;
ThumbNailSizeToUse.Height = ThumbNailSize.Height;
ThumbNailSizeToUse.Width = oImg.Size.Width - (int)pixtosubstract;
}
}
else
{
ThumbNailSizeToUse.Width = oImg.Size.Width;
ThumbNailSizeToUse.Height = oImg.Size.Height;
}
Bitmap bmp = new Bitmap(ThumbNailSizeToUse.Width, ThumbNailSizeToUse.Height);
bmp.SetResolution(rez, rez);
System.Drawing.Image oThumbNail = bmp;
bmp = null;
Graphics oGraphic = Graphics.FromImage(oThumbNail);
oGraphic.CompositingQuality = CompositingQuality.HighQuality;
oGraphic.SmoothingMode = SmoothingMode.HighQuality;
oGraphic.InterpolationMode = InterpolationMode.HighQualityBicubic;
Rectangle oRectangle = new Rectangle(0, 0, ThumbNailSizeToUse.Width, ThumbNailSizeToUse.Height);
oGraphic.DrawImage(oImg, oRectangle);
oThumbNail.Save(sPhysicalPath + sThumbNailFileName, oFormat);
oImg.Dispose();
}
catch (Exception ex)
{
Response.Write(ex.Message);
}
}
You may have a look at http://gd-sharp.sourceforge.net/ which is a wrapper for the GD library. I haven't tested it but it seems promising.
I've had good behavior from the Cairo library (http://www.cairographics.org) in an ASP.Net webserver environment. I actually moved to cairo from WPF due to WPF's poor memory usage model for web-based stuff.
WPF actually tends to run your worker process out of memory. None of the WPF objects implement IDisposable, and many of them reference unmanaged memory that's only freed via a finalizer. Heavy use of WPF (especially if your server is significantly CPU-taxed) will eventually run you out of memory because your finalizer queue gets saturated. When I was profiling my app, for instance, the finalization queue had upwards of 50,000 objects on it, many of them holding references to unmanaged memory. Cairo has behaved much better for me, and its memory usage pattern has been much more predictable than WPF's.
If you're interested in using cairo, grab the libs from GTK+'s website. They have an x86 as well as an x64 set of binaries.
The only downside is that cairo can't read/write JPG natively; however, you could easily adapt WPF's stuff for reading/writing JPG and do the resampling/scaling/drawing/whatever else using Cairo.
Aspose.Drawing is a drop-in replacement for System.Drawing that is fully managed and can be safely used in web applications. (I'm one of the developers.)