Converting OpenCVSharp4 Rectangle to IronOCR CropRectangle(System.Drawing.Rectangle) - c#

I have a project in which I'm using IronOCR to read an area define by OpenCVSharp4 but the problem I'm encountering is IronOCrs CropRectangle method, it uses System.drawing.rectangle and for some reason my OpenCvSharp.Rect cannot be converted to it, by this I mean when I Finally uses IronOCRs Input.Add(Image, ContentArea) the results I get are not what is expected.
Below the code I have attached a picture of what the code currently produces.
Don't worry about IronOCR not getting the correct letters I believe it has to do with it creating a weird box and some letters getting cut off, it works if I made the area larger for crop rectangle width and height
var Ocr = new IronTesseract();
String[] splitText;
using (var Input = new OcrInput())
{
//OpenCv
OpenCvSharp.Rect rect = new OpenCvSharp.Rect(55, 107, 219, 264);
//IronOCR
Rectangle ContentArea = new Rectangle() { X = rect.TopLeft.X, Y = rect.TopLeft.Y, Height = rect.Height, Width = rect.Width };
CropRectangle r = new CropRectangle(ContentArea);
CordBox.Text = r.Rectangle.ToString();
//OpenCv
resizedMat.Rectangle(rect.TopLeft, rect.BottomRight, Scalar.Blue, 3);
resizedMat.Rectangle(new OpenCvSharp.Point(55, 107), new OpenCvSharp.Point(219, 264), Scalar.Brown, 3);
Cv2.ImShow("resizedMat", resizedMat);
//IronOCR
Input.Add(#"C:\Projects\AnExperiment\WpfApp1\Images\TestSave.PNG", r);
Input.EnhanceResolution();
var Result = Ocr.Read(Input);
ResultBox.Text = Result.Text;
splitText = ResultBox.Text.Split('\n');
}

SO here is the solution I came up with.
This problem is a OpenCvSharp4 one where OpenCvSharp4.Rectangle for some reason does have matching coordinates to System.Drawing.Rectangle. I have posted this on the gitHub for OpenCvSHarp4 and he says its fine, but its not.
So I switched over to Emgu NuGet package its better for C# applications and is a OpenCv Wrapper made for C# (I was just scared of giving it a try before because i never really understood it.)
Emgu uses System.Drawing.Rectangle by default instead of something like OpenCvSharp4.Rectangle so everything matches up nicely.
Mat testMat = new Mat();
System.Drawing.Rectangle roi = CvInvoke.SelectROI("main", testMat );
After finding this out the rest was pretty easy so the final code is below on how it was transformed. (For reference Emgu.CV.CVInvoke is how its called and Emgu.CV.BitmapExtension is its own separate NuGet package)
// Get the original Image
fullPage = CvInvoke.Imread(#"C:\Projects\AnExperiment\WpfApp1\Images\TestImageFinalFilled.png");
// Resize it so it works with the cordinates stored previously in a json file
CvInvoke.Resize(fullPage, resizedMat, EmguSetResolution(fullPage, dpi));
// Save the small version so iron ocr doesnt mess up
var bitmap = Emgu.CV.BitmapExtension.ToBitmap(resizedMat);
bitmap.Save(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG");
// Let user select box
System.Drawing.Rectangle roi = CvInvoke.SelectROI("main", resizedMat);
CvInvoke.DestroyWindow("main");
// Draw Rect for debugging
CvInvoke.Rectangle(resizedMat, roi, new MCvScalar(0, 0, 255), 2);
// Read section we highlighted by pulling the saved resuze imag as a reference
var Ocr = new IronTesseract();
IronOcr.OcrResult ocrResult;
Ocr.UseCustomTesseractLanguageFile(#"C:\Projects\AnExperiment\WpfApp1\tessdata_best-main\eng.traineddata");
using (var Input = new OcrInput())
{
CvInvoke.Rectangle(resizedMat, roi, new MCvScalar(0, 0, 255), 2);
IronOcr.CropRectangle contentArea = new CropRectangle(roi);
Input.AddImage(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG", contentArea);
Input.EnhanceResolution();
Input.Sharpen();
Input.Contrast();
ocrResult = Ocr.Read(Input);
}
File.Delete(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG");
CvInvoke.Imshow("m", resizedMat);
After all this I have some functions that spit the ocrResult.Text into the textbox and separate certain things I needed from it.

Related

ImageSharp: Drawing a clipped Polygon out of bounds

I'm currently in the process of migrating a project that uses System.Drawing.Bitmap over to use ImageSharp. As part of this migration, I am migrating logic that would draw circles onto the bitmap using the Graphics.FromImage function.
The problem I am faced with is, if the circle previously went outside the bounds of the Bitmap, this was previously fine and it would draw the parts of the circle it could, simply clipping the drawn circle. With ImageSharp, this would, understandably throw an out of bounds exception.
A simplified implementation:
void Main()
{
var width = 70;
var height = 70;
SystemDrawingImpl(width, height);
ImageSharpImpl(width, height);
}
private void SystemDrawingImpl(int width, int height)
{
using var bitmap = new Bitmap(64, 64);
using var graphics = Graphics.FromImage(bitmap);
var xLocation = ((bitmap.Width / 2) - (width / 2)) - 1;
var yLocation = ((bitmap.Height / 2) - (height / 2)) - 1;
graphics.DrawEllipse(new System.Drawing.Pen(System.Drawing.Color.Green, 1.1f), xLocation, yLocation, width, height);
var memoryStream = new MemoryStream();
bitmap.Save(memoryStream, ImageFormat.Jpeg);
Util.Image(memoryStream.ToArray()).Dump();
}
private void ImageSharpImpl(int width, int height)
{
using var image = new Image<Rgba32>(64, 64);
var brush = SixLabors.ImageSharp.Drawing.Processing.Brushes.Solid(SixLabors.ImageSharp.Color.Green);
var pen = SixLabors.ImageSharp.Drawing.Processing.Pens.Solid(SixLabors.ImageSharp.Color.Green, 0.1f);
var ellipse = new EllipsePolygon(32, 32, width, height);
image.Mutate(ctx => ctx.Draw(pen, ellipse));
var memoryStream = new MemoryStream();
image.Save(memoryStream, new JpegEncoder());
Util.Image(memoryStream.ToArray()).Dump();
}
For which the output for System.Drawing would be:
The output/exception from ImageSharp is:
ImageProcessingException: An error occurred when processing the image
using FillRegionProcessor`1. See the inner exception for more detail.
ArgumentOutOfRangeException: Specified argument was out of the range
of valid values. (Parameter 'edgeIdx')
I was wondering if there was a way to get a similar output. Is there a way in which I'd be able to still draw the parts of the ellipse that is possible?
I have attempted to remove the points that are "invalid" via a simple where:
var points = ellipse.Points.ToArray();
var validPoints = points.Where(x => (x.X <= image.Width && x.X >= 0) && (x.Y <= image.Height && x.Y >= 0)).ToArray();
image.Mutate(ctx => ctx.DrawPolygon(pen, validPoints));
However this will still try to create a fully joined path which is not the desired effect:
Any advice on how I might achieve this would be appreciated
After a lot of digging I managed to find somewhat of an answer.
I decided to pull down the latest version of ImageSharp.Drawing and executed the following simple piece of code which I knew threw an exception:
using var image = new Image<Rgba32>(64, 64);
var ellipse = new EllipsePolygon(32, 32, 70, 70);
image.Mutate(x => x.Draw(Color.Green, 2f, ellipse));
This code which when I pulled the repository, works just fine! This code being the most up to date code for the repository as of this answer being posted, I reverted back to a previous commit which was around the time of the NuGet package for ImageSharp.Drawing was published, ~8th October, version 1.0.0-beta13. This code now threw the exception I had before.
Debugging this code, I found the file where this was thrown and found where it was executed from, it was PolygonScanner.cs. The file history showed this had one change this year which referenced an issue:
Issue 108
This issue fixed a problem which arose around this area which seemed to fix my issue.
Therefore the issue is fixed... In the latest version of the code. This package is somewhat out of date with the latest beta however there looks to be an alpha I can use until this package is updated and a RC is released, which sounds like it will be soon!
Release Candidate Discussion
Curse of using a beta I suppose!

Having issues when attempting to load images into a bitmap/Picture Box Visual Studio 2019

I am attempting to load images into an Picture Box using the following code:
private void button1_Click(object sender, EventArgs e)
{
//Clear the Image Area
if (templateArea.Image != null)
{ templateArea.Image = null; }
//Set Base Variables
Bitmap img = new Bitmap(Resources.BlankBackground);
Graphics gpx = Graphics.FromImage(img);
Bitmap newBitmap = null;
string bitmapToLoad;
ResourceManager rm;
//Loop Lists
for (int i = 0; i < comps.Count; i++)
{
bitmapToLoad = Convert.ToString(comps[i][3]);
rm = Resources.ResourceManager;
newBitmap = (Bitmap)rm.GetObject(bitmapToLoad);
gpx.DrawImage(newBitmap,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap.Width,
newBitmap.Height);
gpx.Dispose();
}
//Set Image Area to newBitmap
templateArea.Image = newBitmap;
}
when trying to run the code I get errors on the second loop where i = 1 in this portion:
gpx.DrawImage(newBitmap,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap.Width,
newBitmap.Height);
//given error: System.ArgumentException: 'Parameter is not valid.' //Does not point to anything specific.
the comps list contains lists formatted as such:
//These listed items are defaulted at program start but more are added while running
// p elementName type sub type x y sX sY R G B O i
eleComps = new List<Object> {0,"Background","Static","BlankBackground",0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
eleComps = new List<Object> {0,"Border", "Static","CardBorder_02", 0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
eleComps = new List<Object> {0,"Cut Line", "Static","00_Card_CutLine",0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
Each possible "sub type" is a bitmap file in resources, and there are more than just those listed above.
What I have tried:
I have consulted the miracle that is Google and YouTube and haven't been able to solve this one. Although I feel like I'm missing something basic I have been looking at it all day and may need some fresh eyes on it to correct something that may be obvious but for some reason I seem to be missing.
What I am looking for:
I am trying to create an image viewer that will overlap images or parts of images (.png) at specific coordinates in an Picture Box. The comps list will be added to and removed from actively during application use and as such using a variable to reference the Resources is necessary and that portion of the code has given me the most difficult time out of the supplied code samples.
UPDATE 1: changed image box to picture box to reduce confusion.
UPDATE 2: NOTE: This sample achieves the desired end result, but does not allow me to use the variables as needed to change positions, scale, color, and opacity.
//Clear Img Area
templateArea.InitialImage = null;
//Find Bitmap(s)
string p = #"C:\Users\david\Desktop\Test Fields\Default\";
Bitmap img1 = new Bitmap(p + "00_Empty.png", true);
Bitmap img2 = new Bitmap(p + "CardBorder_02.png", true);
Bitmap img3 = new Bitmap(p + "00_Card_CutLine.png", true);
//Set initial image for image area
Graphics gpx = Graphics.FromImage(img1);
//place an image at x, y, Width, Height
gpx.DrawImage(img2, 0, 0, img2.Width, img2.Height);
gpx.DrawImage(img3, 0, 0, img3.Width, img3.Height);
//clear gpx cache
gpx.Dispose();
//set image area to modified img1
templateArea.Image = img1;
Also please let me know if I'm just plain going about this in the wrong way, and what I need to do to get it right.
UPDATE 3: I seem to have gotten it to work properly with this variation, using the originally supplied list of listed parameters. Thanks to Idle_Mind's pointers I was able to figure it out.
Bitmap newBitmap_01 = new Bitmap(templateArea.Width, templateArea.Height);
Graphics gpx = Graphics.FromImage(newBitmap_01);
string bitmapToLoad;
ResourceManager rm = Resources.ResourceManager;
Bitmap newBitmap_02;
for (int i = 0; i < comps.Count; i++)
{
bitmapToLoad = Convert.ToString(comps[i][3]);
newBitmap_02 = (Bitmap)rm.GetObject(bitmapToLoad);
gpx.DrawImage(newBitmap_02,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap_02.Width,
newBitmap_02.Height);
}
gpx.Dispose();
templateArea.Image = newBitmap_01;

How to reset MatrixBox in EmguCv using winforms

I've been unable to find much documentation for properly using the MatrixBox from the Emgu.CV.UI.
I'm using EmguCV version 3.4.1 and I would like to use the MatrixBox to show a live update of pixel values from a video I'm streaming.
Winforms Matrixbox Control
private void BufferReceiver_ProgressChanged(object sender, ProgressChangedEventArgs e)
{
Matrix<UInt16> matrix = new Matrix<UInt16>(mat.Rows, mat.Cols, mat.NumberOfChannels);
Matrix<UInt16> subMatrix = matrix.GetSubRect(new Rectangle(0, 0, 10, 10));
matrixBox1.Matrix = subMatrix;
matrixBox1.Refresh();
}
I'm able to display a 10x10 matrix of the pixel values, but when the above method executes again, the matrix grows to 20x20, but replaces the 10x10 matrix with the updated values.
First Run
Second Run
I can see from the OpenCv source that the MatrixBox is just updating a DataGridView, but I can't find a way to clear the MatrixBox before updating the values.
You may try
Size sz = matrixBox1.Size;
Point lc = matrixBox1.Location;
int ti = matrixBox1.TabIndex;
matrixBox1.Dispose();
matrixBox1 = New Emgu.CV.UI.MatrixBox;
matrixBox1.Parent = this;
matrixBox1.Location = lc;
matrixBox1.Matrix = Nothing;
matrixBox1.Name = "MatrixBox1";
matrixBox1.Size = sz;
matrixBox1.TabIndex = ti;
matrixBox1.Matrix = subMatrix;

OpenNI show RGB camera feed

Have spent the last week trying to have my C# program show both the depth feed and the RGB feed (similar to how /Samples/Bin64/Release/NiViewer64.exe shows both feeds in a window).
Project specs: C# - VS2013 Express OpenNI - Using a modified
SimpleViewer.net (has two feeds of depth). Asus Xtion Pro Live
I would like one of the feeds to become a normal camera feed instead of the depth feed.
I'm guessing it has something to do with this:
MapOutputMode mapMode = this.depth.MapOutputMode;
this.bitmap = new Bitmap((int)mapMode.XRes, (int)mapMode.YRes,System.Drawing.Imaging.PixelFormat.Format24bppRgb);
Any ideas?
Finally figured it out, thanks to another programmer.
image = context.FindExistingNode(NodeType.Image) as ImageGenerator;
ImageMetaData imd = image.GetMetaData();
lock (this)
{
//**************************************//
//***********RGB Camera Feed************//
//**************************************//
Rectangle rect = new Rectangle(0, 0, this.bitmap.Width, this.bitmap.Height);
BitmapData data = this.camera_feed.LockBits(rect, ImageLockMode.WriteOnly, System.Drawing.Imaging.PixelFormat.Format24bppRgb);
byte* pDest = (byte*)data.Scan0.ToPointer();
byte* imstp = (byte*)image.ImageMapPtr.ToPointer();
// set pixels
for (int i = 0; i < imd.DataSize; i += 3, pDest += 3, imstp += 3)
{
pDest[0] = imstp[2];
pDest[1] = imstp[1];
pDest[2] = imstp[0];
}
and declare this somewhere:
public ImageGenerator image;

Huge Whitespace exists to right after drawing image. Want to get rid of it

I'm using the following codeproject to build an asp.net website and so far everything is good. My only problem is after the barcode is generated, a huge whitespace exist to the right of the barcode. I've been playing with this and am unable to resolve it.
Details below:
Link to Code Project Article: http://www.codeproject.com/KB/aspnet/AspBarCodes.aspx?msg=3543809
Copy of the Font is here: http://trussvillemethodist.web01.appliedi-labs.net/IDAutomationHC39M.ttf
//Working Path
string sWorkPath = "";
sWorkPath = this.Context.Server.MapPath("");
//Fonts
PrivateFontCollection fnts = new PrivateFontCollection();
fnts.AddFontFile(sWorkPath + #"\IDAutomationHC39M.ttf");
FontFamily fntfam = new FontFamily("IDAutomationHC39M", fnts);
Font oFont = new Font(fntfam, 18);
// Get the Requested code sent from the previous page.
string strCode = Request["code"].ToString();
//Graphics
//I don't know what to set the width to as I can't call the MeasureString without creating the Graphics object.
Bitmap oBitmaptemp = new Bitmap(40, 100);
Graphics oGraphicstemp = Graphics.FromImage(oBitmaptemp);
int w = (int)oGraphicstemp.MeasureString(strCode, oFont).Width + 4;
// Create a bitmap object of the width that we calculated and height of 100
Bitmap oBitmap = new Bitmap(w, 100);
// then create a Graphic object for the bitmap we just created.
Graphics oGraphics = Graphics.FromImage(oBitmap);
// Let's create the Point and Brushes for the barcode
PointF oPoint = new PointF(2f, 2f);
SolidBrush oBrushWrite = new SolidBrush(Color.Black);
SolidBrush oBrush = new SolidBrush(Color.White);
// Now lets create the actual barcode image
// with a rectangle filled with white color
oGraphics.FillRectangle(oBrush, 0, 0, w, 100);
// We have to put prefix and sufix of an asterisk (*),
// in order to be a valid barcode
oGraphics.DrawString("*" + strCode + "*", oFont, oBrushWrite, oPoint);
// Then we send the Graphics with the actual barcode
Response.ContentType = "image/gif";
oBitmap.Save(Response.OutputStream, ImageFormat.Gif);
oBitmap.Dispose();
oGraphics.Dispose();
oBrush.Dispose();
oFont.Dispose();
The code just assumes 40 pixels per character, which is why you get a lot of image left on the right of the text. You can use the MeasureString method to measure the size of the text, and use that to create an image of the correct size:
int w = (int)oGraphics.MeasureString("*123$10.00*", oFont).Width + 4;
I noticed that you don't dispose any of the objects that you are using. The Graphics, Bitmap, SolidBrush and Font objects need to be disposed.
You might also want to consider using a GIF image instead of JPEG, it's more suited for this kind of graphics.

Categories