We Keep Coding

Converting OpenCVSharp4 Rectangle to IronOCR CropRectangle(System.Drawing.Rectangle)

I have a project in which I'm using IronOCR to read an area define by OpenCVSharp4 but the problem I'm encountering is IronOCrs CropRectangle method, it uses System.drawing.rectangle and for some reason my OpenCvSharp.Rect cannot be converted to it, by this I mean when I Finally uses IronOCRs Input.Add(Image, ContentArea) the results I get are not what is expected.
Below the code I have attached a picture of what the code currently produces.
Don't worry about IronOCR not getting the correct letters I believe it has to do with it creating a weird box and some letters getting cut off, it works if I made the area larger for crop rectangle width and height
var Ocr = new IronTesseract();
String[] splitText;
using (var Input = new OcrInput())
{
//OpenCv
OpenCvSharp.Rect rect = new OpenCvSharp.Rect(55, 107, 219, 264);
//IronOCR
Rectangle ContentArea = new Rectangle() { X = rect.TopLeft.X, Y = rect.TopLeft.Y, Height = rect.Height, Width = rect.Width };
CropRectangle r = new CropRectangle(ContentArea);
CordBox.Text = r.Rectangle.ToString();
//OpenCv
resizedMat.Rectangle(rect.TopLeft, rect.BottomRight, Scalar.Blue, 3);
resizedMat.Rectangle(new OpenCvSharp.Point(55, 107), new OpenCvSharp.Point(219, 264), Scalar.Brown, 3);
Cv2.ImShow("resizedMat", resizedMat);
//IronOCR
Input.Add(#"C:\Projects\AnExperiment\WpfApp1\Images\TestSave.PNG", r);
Input.EnhanceResolution();
var Result = Ocr.Read(Input);
ResultBox.Text = Result.Text;
splitText = ResultBox.Text.Split('\n');
}

SO here is the solution I came up with.
This problem is a OpenCvSharp4 one where OpenCvSharp4.Rectangle for some reason does have matching coordinates to System.Drawing.Rectangle. I have posted this on the gitHub for OpenCvSHarp4 and he says its fine, but its not.
So I switched over to Emgu NuGet package its better for C# applications and is a OpenCv Wrapper made for C# (I was just scared of giving it a try before because i never really understood it.)
Emgu uses System.Drawing.Rectangle by default instead of something like OpenCvSharp4.Rectangle so everything matches up nicely.
Mat testMat = new Mat();
System.Drawing.Rectangle roi = CvInvoke.SelectROI("main", testMat );
After finding this out the rest was pretty easy so the final code is below on how it was transformed. (For reference Emgu.CV.CVInvoke is how its called and Emgu.CV.BitmapExtension is its own separate NuGet package)
// Get the original Image
fullPage = CvInvoke.Imread(#"C:\Projects\AnExperiment\WpfApp1\Images\TestImageFinalFilled.png");
// Resize it so it works with the cordinates stored previously in a json file
CvInvoke.Resize(fullPage, resizedMat, EmguSetResolution(fullPage, dpi));
// Save the small version so iron ocr doesnt mess up
var bitmap = Emgu.CV.BitmapExtension.ToBitmap(resizedMat);
bitmap.Save(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG");
// Let user select box
System.Drawing.Rectangle roi = CvInvoke.SelectROI("main", resizedMat);
CvInvoke.DestroyWindow("main");
// Draw Rect for debugging
CvInvoke.Rectangle(resizedMat, roi, new MCvScalar(0, 0, 255), 2);
// Read section we highlighted by pulling the saved resuze imag as a reference
var Ocr = new IronTesseract();
IronOcr.OcrResult ocrResult;
Ocr.UseCustomTesseractLanguageFile(#"C:\Projects\AnExperiment\WpfApp1\tessdata_best-main\eng.traineddata");
using (var Input = new OcrInput())
{
CvInvoke.Rectangle(resizedMat, roi, new MCvScalar(0, 0, 255), 2);
IronOcr.CropRectangle contentArea = new CropRectangle(roi);
Input.AddImage(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG", contentArea);
Input.EnhanceResolution();
Input.Sharpen();
Input.Contrast();
ocrResult = Ocr.Read(Input);
}
File.Delete(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG");
CvInvoke.Imshow("m", resizedMat);
After all this I have some functions that spit the ocrResult.Text into the textbox and separate certain things I needed from it.

Pdfnet Print sample not working

While testing in the samples from PDFPrintTest, we noticed that Example 2 coupled with Event Handler's example 1 is not behaving properly.
Example 1 of PrintPage Event Handler:
void PrintPage(object sender, PrintPageEventArgs ev)
{
Graphics gr = ev.Graphics;
gr.PageUnit = GraphicsUnit.Inch;
Rectangle rectPage = ev.PageBounds; //print without margins
//Rectangle rectPage = ev.MarginBounds; //print using margins
float dpi = gr.DpiX;
if (dpi > 300) dpi = 300;
int example = 1;
bool use_hard_margins = false;
// Example 1) Print the Bitmap.
if (example == 1)
{
pdfdraw.SetDPI(dpi);
Bitmap bmp = pdfdraw.GetBitmap(pageitr.Current());
//bmp.Save("tiger.jpg");
gr.DrawImage(bmp, rectPage, 0, 0, bmp.Width, bmp.Height, GraphicsUnit.Pixel);
}
Full sample code here: https://www.pdftron.com/pdfnet/samplecode/PDFPrintTest.cs.html
You'll notice the bmp.Save("tiger.jpg"); in comment, that's the point where it goes wrong. If we run the code and save the bmp, we get exactly what we need in the jpg file. However, gr.DrawImage(bmp, rectPage, 0, 0, bmp.Width, bmp.Height, GraphicsUnit.Pixel); prints a plain empty pdf page. Why is that ?
Our goal: We need to force a printjob with 40% grayscale in certain circumstances. Winforms does not support this, we can only set grayscale, not specify a percentage, so we are looking to intercept the print and change the output to 40% grayscale, which lead us to the PdfNet Print samples. From these samples, only example 2 in the handler has Graphics gr which accepts a color matrix to set the wanted grayscale to the page.
Any non-PdfNet solution is welcome aswell, but it's still odd that the sample code isn't working out of the box.

Thank you for pointing this out. Just as you are, I am unclear why bmp.Save works fine, but Graphics.DrawImage(bmp,... is only showing the background color. I suspect it has something to do with the other parameters passed into Graphics.DrawImage
Since the Bitmap object is correct, then this particular issue is really a .Net question and not a PDFNet question, which I am currently unable to answer.
The other part of the sample runs fine, the one using PDFDraw.DrawInRect. Does this not work for you?

We got it working, apparently it was only giving a white page when printing to pdf. The exact same code rendered a much too small image but actually printed.
We're still not entirely sure what the issue was, but worked out new code that properly prints to pdf, and prints full-scale to a printer.
void PrintPage(object sender, PrintPageEventArgs ev)
{
Graphics gr = ev.Graphics;
gr.PageUnit = GraphicsUnit.Pixel; //this has been changed to Pixel, from Inch.
float dpi = gr.DpiX;
//if (dpi > 300) dpi = 300;
Rectangle rectPage = ev.PageBounds; //print without margins
//Rectangle rectPage = ev.MarginBounds; //print using margins
float dpi = gr.DpiX;
int example = 1;
bool use_hard_margins = false;
// Example 1) Print the Bitmap.
if (example == 1)
{
pdfdraw.SetDPI(dpi);
pdfdraw.SetDrawAnnotations(false);
Bitmap bmp = pdfdraw.GetBitmap(pageitr.Current());
gr.DrawImage(bmp, new Rectangle(0, 0, bmp.Width, bmp.Height), 0, 0, bmp.Width, bmp.Height, GraphicsUnit.Pixel);
}
`
if (dpi > 300) dpi = 300; This was the main culprit in rendering a too small image being sent to the printer. It also fixed the 'white pdf' issue.
Second, we didn't pass rectPage to DrawImage, and replaced it with: new Rectangle(0, 0, bmp.Width, bmp.Height).
I can understand the smaller size being sent to the printer, but why it didn't pick up anything to print to pdf is still unclear.
While the ultimate goal is still printing, it's much easier to debug and test with a properly working 'print to pdf'. The above code works in 2 separate projects, so I'm going to assume this indeed fixes the issue.

A generic error occurred in GDI+, for random images

The questions in SO, for example this one or this one have discussed many reason for this exception, but none could help me to find out my problem, and I still cannot figure out why this is happening.
I'm having this error, when trying to save (code is below). And the strangest thing I've got is that - this error is occurring for random images (let's say "random" for now, because I don't know exact problem yet). Only for particular images, out of thousands, I got this problem.
Here is the one successful and one unsuccessful image when trying.
1. 2009-05-23_00.51.39.jpg
Image dimensions: 768x1024
Size: 87KB
Bit depth: 24
Color representation: sRGB
2. 2009-05-23_00.52.50.jpg
Image dimensions: 768x1024
Size: 335KB
Bit depth: 24
Color representation: sRGB
For image #1 the process is going smoothly, with no errors. However, for image #2, I keep getting this error. As you see the images are almost the same. Both are .jpg, same dimensions, and most of the properties are just same.
Here is the method.
public static Stream GetImageBytes(Stream fileStream, int maxWidth, int maxHeight, bool showCopyRight, string link, int fillColor = 0xFFFFFF)
{
var img = Image.FromStream(fileStream);
if (maxHeight != 0 && img.Height > maxHeight)
ResizeByHeight(ref img, maxHeight, fillColor.ToString("X6"));
if (maxWidth != 0 && img.Width > maxWidth)
ResizeByWidth(ref img, maxWidth, fillColor.ToString("X6"));
if (showCopyRight)
img = ApplyWatermark(ref img, "watermark.png", link);
var ms = new MemoryStream();
//problem is here
img.Save(ms, ImageFormat.Jpeg);
img.Dispose();
return ms;
}
Edit after comments.
Ok, here is the Watermark method
private static Bitmap ApplyWatermark(ref Image img, string watermarkPathPng, string link) {
const int x = 0;
var y = img.Height - 20;
string watermarkText;
if (img.Width <= 220) {
watermarkText = "shorter copy";
} else {
if (img.Width < 500)
watermarkText = "longer copy";
else
watermarkText = "full copy " + link;
}
var bp = new Bitmap(img);
img.Dispose();
using (var gr = Graphics.FromImage(bp)) {
var watermark = Image.FromFile(watermarkPathPng);
gr.DrawImage(watermark, x, y, watermark.Width, watermark.Height);
watermark.Dispose();
gr.DrawString(watermarkText, new Font(new FontFamily("Batang"), 13, FontStyle.Regular, GraphicsUnit.Pixel), Brushes.Black, 5, y + 5);
}
return bp;
}
The reason why I've used Bitmap inside this method is that I had Indexed Pixels problem before and that solved the issue.

Try copying the image rather than operating directly on the one you loaded. Often gdi errors are caused by you trying to change something that you think you "own" when the loader thinks it owns it.
As it only happens for some images, check the logic in your three processing methods - perhaps the failure only occurs if you need to do a vertical resize, in which case look for a bug in that method. Our maybe it's images that have to be processed by more than one of your methods.
It is also suspicious that you return img and pass it by reference in ApplyWatermark(). Check your logic here.

As I've not a great knowledge of GDI+, I'm not fully sure what has happened, but changing the following code line:
var ms = new MemoryStream();
//problem is here
img.Save(ms, ImageFormat.Jpeg);
img.Dispose();
return ms;
to this,
var ms = new MemoryStream();
var bit = new Bitmap(img);
img.Dispose();
bit.Save(ms, ImageFormat.Jpeg);
bit.Dispose();
return ms;
Worked well, so I just create a Bitmap from the image, then save the Bitmap to the Memory. Now this function is working well for all the images.

How to implement pan/zoom on gigapixel bitmaps?

In my project, I'm using (uncompressed 16-bit grayscale) gigapixel images which come from a high resolution scanner for measurement purposes. Since these bitmaps can not be loaded in memory (mainly due to memory fragmentation) I'm using tiles (and tiled TIFF on disc). (see StackOverflow topic on this)
I need to implement panning/zooming in a way like Google Maps or DeepZoom. I have to apply image processing on the fly before presenting it on screen, so I can not use a precooked library which directly accesses an image file. For zooming I intend to keep a multi-resolution image in my file (pyramid storage). The most useful steps seem to be +200%, 50% and show all.
My code base is currently C# and .NET 3.5. Currently I assume Forms type, unless WPF gives me great advantage in this area. I have got a method which can return any (processed) part of the underlying image.
Specific issues:
hints or references on how to implement this pan/zoom with on-demand generation of image parts
any code which could be used as a basis (preferably commercial or LGPL/BSD like licenses)
can DeepZoom be used for this (i.e. is there a way that I can provide a function to provide a tile at the right resulution for the current zoom level?) ( I need to have pixel accurate addressing still)

This CodeProject article: Generate...DeepZoom Image Collection might be a useful read since it talks about generating a DeepZoom image source.
This MSDN article has a section Dynamic Deep Zoom: Supplying Image Pixels at Run Time and links to this Mandelbrot Explorer which 'kinda' sounds similar to what you're trying to do (ie. he is generating specific parts of the mandelbrot set on-demand; you want to retrieve specific parts of your gigapixel image on-demand).
I think the answer to "can DeepZoom be used for this?" is probably "Yes", however as it is only available in Silverlight you will have to do some tricks with an embedded web browser control if you need a WinForms/WPF client app.
Sorry I can't provide more specific answers - hope those links help.
p.s. I'm not sure if Silverlight supports TIFF images - that might be an issue unless you convert to another format.

I decided to try something myself. I came up with a straightforward GDI+ code, which uses the tiles I've already got. I just filter out the parts which are relevant for current clipping region. It works like magic! Please find my code below.
(Form settings double buffering for the best results)
protected override void OnPaint(PaintEventArgs e)
{
base.OnPaint(e);
Graphics dc = e.Graphics;
dc.ScaleTransform(1.0F, 1.0F);
Size scrollOffset = new Size(AutoScrollPosition);
int start_x = Math.Min(matrix_x_size,
(e.ClipRectangle.Left - scrollOffset.Width) / 256);
int start_y = Math.Min(matrix_y_size,
(e.ClipRectangle.Top - scrollOffset.Height) / 256);
int end_x = Math.Min(matrix_x_size,
(e.ClipRectangle.Right - scrollOffset.Width + 255) / 256);
int end_y = Math.Min(matrix_y_size,
(e.ClipRectangle.Bottom - scrollOffset.Height + 255) / 256);
// start * contain the first and last tile x/y which are on screen
// and which need to be redrawn.
// now iterate trough all tiles which need an update
for (int y = start_y; y < end_y; y++)
for (int x = start_x; x < end_x; x++)
{ // draw bitmap with gdi+ at calculated position.
dc.DrawImage(BmpMatrix[y, x],
new Point(x * 256 + scrollOffset.Width,
y * 256 + scrollOffset.Height));
}
}
To test it, I've created a matrix of 80x80 of 256 tiles (420 MPixel). Of course I'll have to add some deferred loading in real life. I can leave tiles out (empty) if they are not yet loaded. In fact, I've asked my client to stick 8 GByte in his machine so I don't have to bother about performance too much. Once loaded tiles can stay in memory.
public partial class Form1 : Form
{
bool dragging = false;
float Zoom = 1.0F;
Point lastMouse;
PointF viewPortCenter;
private readonly Brush solidYellowBrush = new SolidBrush(Color.Yellow);
private readonly Brush solidBlueBrush = new SolidBrush(Color.LightBlue);
const int matrix_x_size = 80;
const int matrix_y_size = 80;
private Bitmap[,] BmpMatrix = new Bitmap[matrix_x_size, matrix_y_size];
public Form1()
{
InitializeComponent();
Font font = new Font("Times New Roman", 10, FontStyle.Regular);
StringFormat strFormat = new StringFormat();
strFormat.Alignment = StringAlignment.Center;
strFormat.LineAlignment = StringAlignment.Center;
for (int y = 0; y < matrix_y_size; y++)
for (int x = 0; x < matrix_x_size; x++)
{
BmpMatrix[y, x] = new Bitmap(256, 256, PixelFormat.Format24bppRgb);
// BmpMatrix[y, x].Palette.Entries[0] = (x+y)%1==0?Color.Blue:Color.White;
using (Graphics g = Graphics.FromImage(BmpMatrix[y, x]))
{
g.FillRectangle(((x + y) % 2 == 0) ? solidBlueBrush : solidYellowBrush, new Rectangle(new Point(0, 0), new Size(256, 256)));
g.DrawString("hello world\n[" + x.ToString() + "," + y.ToString() + "]", new Font("Tahoma", 8), Brushes.Black,
new RectangleF(0, 0, 256, 256), strFormat);
g.DrawImage(BmpMatrix[y, x], Point.Empty);
}
}
BackColor = Color.White;
Size = new Size(300, 300);
Text = "Scroll Shapes Correct";
AutoScrollMinSize = new Size(256 * matrix_x_size, 256 * matrix_y_size);
}
Turned out this was the easy part. Getting async multithreaded i/o done in the background was a lot harder to acchieve. Still, I've got it working in the way described here. The issues to resolve were more .NET/Form multithreading related than to this topic.
In pseudo code it works like this:
after onPaint (and on Tick)
check if tiles on display need to be retrieved from disc
if so: post them to an async io queue
if not: check if tiles close to display area are already loaded
if not: post them to an async io/queue
check if bitmaps have arrived from io thread
if so: updat them on screen, and force repaint if visible
Result: I now have my own Custom control which uses roughly 50 MByte for very fast access to arbitrary size (tiled) TIFF files.

I guess you can address this issue following the steps below:
Image generation:
segment your image in multiple subimages (tiles) of a small resolution, for instace, 500x500. These images are depth 0
combine a series of tiles with depth 0 (4x4 or 6x6), resize the combination generating a new tile with 500x500 pixels in depth 1.
continue with this approach until get the entire image using only a few tiles.
Image visualization
Start from the highest depth
When user drags the image, load the tiles dynamically
When the user zoom a region of the image, decrease the depth, loading the tiles for that region in a higher resolution.
The final result is similar to Google Maps.

C# Image.Clone Out of Memory Exception

Why am I getting an out of memory exception?
So this dies in C# on the first time through:
splitBitmaps.Add(neededImage.Clone(rectDimensions, neededImage.PixelFormat));
Where splitBitmaps is a List<BitMap> BUT this works in VB for at least 4 iterations:
arlSplitBitmaps.Add(Image.Clone(rectDimensions, Image.PixelFormat))
Where arlSplitBitmaps is a simple array list. (And yes I've tried arraylist in c#)
This is the fullsection:
for (Int32 splitIndex = 0; splitIndex <= numberOfResultingImages - 1; splitIndex++)
{
Rectangle rectDimensions;
if (splitIndex < numberOfResultingImages - 1)
{
rectDimensions = new Rectangle(splitImageWidth * splitIndex, 0,
splitImageWidth, splitImageHeight);
}
else
{
rectDimensions = new Rectangle(splitImageWidth * splitIndex, 0,
sourceImageWidth - (splitImageWidth * splitIndex), splitImageHeight);
}
splitBitmaps.Add(neededImage.Clone(rectDimensions, neededImage.PixelFormat));
}
neededImage is a Bitmap by the way.
I can't find any useful answers on the intarweb, especially not why it works just fine in VB.
Update:
I actually found a reason (sort of) for this working but forgot to post it. It has to do with converting the image to a bitmap instead of just trying to clone the raw image if I remember.

Clone() may also throw an Out of memory exception when the coordinates specified in the Rectangle are outside the bounds of the bitmap. It will not clip them automatically for you.

I found that I was using Image.Clone to crop a bitmap and the width took the crop outside the bounds of the original image. This causes an Out of Memory error. Seems a bit strange but can beworth knowing.

I got this too when I tried to use the Clone() method to change the pixel format of a bitmap. If memory serves, I was trying to convert a 24 bpp bitmap to an 8 bit indexed format, naively hoping that the Bitmap class would magically handle the palette creation and so on. Obviously not :-/

This is a reach, but I've often found that if pulling images directly from disk that it's better to copy them to a new bitmap and dispose of the disk-bound image. I've seen great improvement in memory consumption when doing so.
Dave M. is on the money too... make sure to dispose when finished.

I struggled to figure this out recently - the answers above are correct. Key to solving this issue is to ensure the rectangle is actually within the boundaries of the image. See example of how I solved this.
In a nutshell, checked to if the area that was being cloned was outside the area of the image.
int totalWidth = rect.Left + rect.Width; //think -the same as Right property
int allowableWidth = localImage.Width - rect.Left;
int finalWidth = 0;
if (totalWidth > allowableWidth){
finalWidth = allowableWidth;
} else {
finalWidth = totalWidth;
}
rect.Width = finalWidth;
int totalHeight = rect.Top + rect.Height; //think same as Bottom property
int allowableHeight = localImage.Height - rect.Top;
int finalHeight = 0;
if (totalHeight > allowableHeight){
finalHeight = allowableHeight;
} else {
finalHeight = totalHeight;
}
rect.Height = finalHeight;
cropped = ((Bitmap)localImage).Clone(rect, System.Drawing.Imaging.PixelFormat.DontCare);

Make sure that you're calling .Dispose() properly on your images, otherwise unmanaged resources won't be freed up. I wonder how many images are you actually creating here -- hundreds? Thousands?