We Keep Coding

Converting OpenCVSharp4 Rectangle to IronOCR CropRectangle(System.Drawing.Rectangle)

I have a project in which I'm using IronOCR to read an area define by OpenCVSharp4 but the problem I'm encountering is IronOCrs CropRectangle method, it uses System.drawing.rectangle and for some reason my OpenCvSharp.Rect cannot be converted to it, by this I mean when I Finally uses IronOCRs Input.Add(Image, ContentArea) the results I get are not what is expected.
Below the code I have attached a picture of what the code currently produces.
Don't worry about IronOCR not getting the correct letters I believe it has to do with it creating a weird box and some letters getting cut off, it works if I made the area larger for crop rectangle width and height
var Ocr = new IronTesseract();
String[] splitText;
using (var Input = new OcrInput())
{
//OpenCv
OpenCvSharp.Rect rect = new OpenCvSharp.Rect(55, 107, 219, 264);
//IronOCR
Rectangle ContentArea = new Rectangle() { X = rect.TopLeft.X, Y = rect.TopLeft.Y, Height = rect.Height, Width = rect.Width };
CropRectangle r = new CropRectangle(ContentArea);
CordBox.Text = r.Rectangle.ToString();
//OpenCv
resizedMat.Rectangle(rect.TopLeft, rect.BottomRight, Scalar.Blue, 3);
resizedMat.Rectangle(new OpenCvSharp.Point(55, 107), new OpenCvSharp.Point(219, 264), Scalar.Brown, 3);
Cv2.ImShow("resizedMat", resizedMat);
//IronOCR
Input.Add(#"C:\Projects\AnExperiment\WpfApp1\Images\TestSave.PNG", r);
Input.EnhanceResolution();
var Result = Ocr.Read(Input);
ResultBox.Text = Result.Text;
splitText = ResultBox.Text.Split('\n');
}

SO here is the solution I came up with.
This problem is a OpenCvSharp4 one where OpenCvSharp4.Rectangle for some reason does have matching coordinates to System.Drawing.Rectangle. I have posted this on the gitHub for OpenCvSHarp4 and he says its fine, but its not.
So I switched over to Emgu NuGet package its better for C# applications and is a OpenCv Wrapper made for C# (I was just scared of giving it a try before because i never really understood it.)
Emgu uses System.Drawing.Rectangle by default instead of something like OpenCvSharp4.Rectangle so everything matches up nicely.
Mat testMat = new Mat();
System.Drawing.Rectangle roi = CvInvoke.SelectROI("main", testMat );
After finding this out the rest was pretty easy so the final code is below on how it was transformed. (For reference Emgu.CV.CVInvoke is how its called and Emgu.CV.BitmapExtension is its own separate NuGet package)
// Get the original Image
fullPage = CvInvoke.Imread(#"C:\Projects\AnExperiment\WpfApp1\Images\TestImageFinalFilled.png");
// Resize it so it works with the cordinates stored previously in a json file
CvInvoke.Resize(fullPage, resizedMat, EmguSetResolution(fullPage, dpi));
// Save the small version so iron ocr doesnt mess up
var bitmap = Emgu.CV.BitmapExtension.ToBitmap(resizedMat);
bitmap.Save(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG");
// Let user select box
System.Drawing.Rectangle roi = CvInvoke.SelectROI("main", resizedMat);
CvInvoke.DestroyWindow("main");
// Draw Rect for debugging
CvInvoke.Rectangle(resizedMat, roi, new MCvScalar(0, 0, 255), 2);
// Read section we highlighted by pulling the saved resuze imag as a reference
var Ocr = new IronTesseract();
IronOcr.OcrResult ocrResult;
Ocr.UseCustomTesseractLanguageFile(#"C:\Projects\AnExperiment\WpfApp1\tessdata_best-main\eng.traineddata");
using (var Input = new OcrInput())
{
CvInvoke.Rectangle(resizedMat, roi, new MCvScalar(0, 0, 255), 2);
IronOcr.CropRectangle contentArea = new CropRectangle(roi);
Input.AddImage(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG", contentArea);
Input.EnhanceResolution();
Input.Sharpen();
Input.Contrast();
ocrResult = Ocr.Read(Input);
}
File.Delete(#"C:\Projects\AnExperiment\WpfApp1\Images\Test.PNG");
CvInvoke.Imshow("m", resizedMat);
After all this I have some functions that spit the ocrResult.Text into the textbox and separate certain things I needed from it.

Copy smaller Mat to larger Mat OpenCv For Unity

I'm trying to take a smaller image mat and copy it into larger mat so I can resize it while keeping the aspect ratio of the image. So, basically this:
So far, this is the code I've written:
private Mat MakeMatFrame(Texture2D image)
{
// Texture must be of right input size
Mat img_mat = new Mat(image.height, image.width, CvType.CV_8UC4, new Scalar(0, 0, 0, 255));
texture2DToMat(image, img_mat);
return img_mat;
}
private void letterBoxImage(Texture2D image)
{
// Get input image as a mat
Mat source = MakeMatFrame(image);
// Create the mat that the source will be put in
int col = source.cols();
int row = source.rows();
int _max = Math.Max(col, row);
Mat resized = Mat.zeros(_max, _max, CvType.CV_8UC4);
// Fill resized
Mat roi = new Mat(resized, new Rect(0, 0, col, row));
source.copyTo(roi);
Texture2D tex2d = new Texture2D(resized.cols(), resized.rows());
matToTexture2D(resized, tex2d);
rawImage.texture = tex2d;
}
Everything I've looked at tells me this is the right approach to take (get a region of interest, fill it in). But instead of getting that third image with the children above the gray region, I just have a gray region.
In other words, the image isn't copying over properly. I've trying using a submat as well, but it failed miserably.
I've been looking for C# code on how to do this sort of thing with OpenCv For Unity, but I can only find C++ code. Which tells me to do exactly this.
Is there some sort of "apply changes" function I'm unaware of for Mats? Am I selecting the region of interest incorrectly? Or is it something else?

sorry for my english,but ur code has a bug.
Mat roi = new Mat(resized, new Rect(0, 0, col, row));
image copied to roi,but this mat not related with resized Mat.so u have to do like this:
Rect roi=new Rect(0,0,width,height);
source.copyto(resized.submat(roi));

how to crop image with polygon in c#

I'm making a labeling tool.
Goal :By drawing a polygon on the picture, you have to export the image ​​inside the polygon to the outside.
example
extract
This is what I drew in the my program.
But I don't know how to extract this region. I want to know how to extract this area.
I have saved the vertices of the picture above in an object. But I don't know how to extract data from the image through these vertices
========================================
So I found this.
https://www.codeproject.com/Articles/703519/Cropping-Particular-Region-In-Image-Using-Csharp
but it is not work
Can't convert Bitmap to IplImage
It doesn't work for the same reason.
In the post, I am going to use opencvsharp 4.x, but the program I am fixing now is .netframework 3.5, so it does not support opencvsharp 4.x.
What should I do?
============================
I made a function referring to the answer, but it doesn't work...
I want to know why.
void CropImage(Bitmap bitmap, Point[] points)
{
Rectangle rect = PaddingImage(points, bitmap);
TextureBrush textureBrush = new TextureBrush(bitmap);
Bitmap bmp1 = new Bitmap(rect.Width, rect.Height);
using (Graphics g = Graphics.FromImage(bmp1))
{
g.FillPolygon(textureBrush, points);
}
string ima_path = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments);
bmp1.Save(ima_path + "\\Image.png", ImageFormat.Png);
}
extract Image
original
If you use a small polygon, there is no output at all.
You will notice that the two images are slightly different.
It seems to me that the part where the center point is cut and extracted is different. I don't know if what I was thinking is correct.

You would create a new bitmap, at least as large as the bounding box of your polygon. Create a graphics object from this new bitmap. You can then draw the polygon to this bitmap, using the original image as a texture brush. Note that you might need to apply transform matrix to translate from the full image coordinates to the cropped image coordinates.
Note that it looks like you have radiological images. These are typically 16 bit images, so they will need to be converted to 8bit mono, or 24bit RGB before they can be used. This should already be done in the drawing code if you have access to the source. Or you can do it yourself.

this works for me
private Bitmap CropImage(Bitmap bitmap, List<Point> points)
{
int pminx = 9999, pminy = 9999, pmaxx = 0, pmaxy = 0; System.Drawing.Point[] pcol = new System.Drawing.Point[points.Count]; int i = 0;
foreach (Point pc in points)
{
if (pc.X > pmaxx) pmaxx = (int)pc.X;
if (pc.Y > pmaxy) pmaxy = (int)pc.Y;
if (pc.X < pminx) pminx = (int)pc.X;
if (pc.Y < pminy) pminy = (int)pc.Y;
pcol[i] = new System.Drawing.Point((int)pc.X, (int)pc.Y);
i++;
}
TextureBrush textureBrush = new TextureBrush(bitmap);
Bitmap bmpWrk = new Bitmap(bitmap.Width, bitmap.Height);
using (Graphics g = Graphics.FromImage(bmpWrk))
{
g.FillPolygon(textureBrush, pcol);
}
System.Drawing.Rectangle CropRect = new System.Drawing.Rectangle(pminx, pminy, pmaxx - pminx, pmaxy - pminy);
return bmpWrk.Clone(CropRect, bmpWrk.PixelFormat);
}

Morphological Operations On Image

I am currently doing a project in which I am trying to identify humans based on hands vascular pattern in C# using Emgu CV.
The gray-scale image of the hand was first processed using the Adaptive Threshold function.
Now I want to create a mask of the image using the morphological operations.
The purpose is to remove the noise from the image.
This is the adaptive-thresholded image:
Kindly guide me which function should I use and how to use.

The code here is in C++. It shouldn't be difficult to port to C#, since it's mostly OpenCV functions calls. You can use that as guideline. Sorry about that.
You can apply a open operation with a small kernel to remove most of the noise:
Mat1b opened;
Mat kernel = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
morphologyEx(thresholded, opened, MORPH_OPEN, kernel);
As you can see, some noise is still present, and you can't remove it with other morphological operations. You can simply consider the largest blob as the correct one (in green here):
Then you can floodfill the inside of the hand (in gray here):
And set to 0 all values in original image where the corresponding mask in not the same color of the inside of the image:
This is the full code (again, it's C++):
#include <opencv2/opencv.hpp>
using namespace cv;
int main(int, char**)
{
// Load grayscale image
Mat1b thresholded = imread("path_to_image", IMREAD_GRAYSCALE);
// Get rid of JPEG compression artifacts
thresholded = thresholded > 100;
// Needed so findContours handles borders contours correctly
Mat1b bin;
copyMakeBorder(thresholded, bin, 1,1,1,1, BORDER_CONSTANT, 0);
// Apply morphological operation "close"
Mat1b closed;
Mat kernel = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
morphologyEx(bin, closed, MORPH_OPEN, kernel);
// Find contours
vector<vector<Point>> contours;
findContours(bin.clone(), contours, RETR_EXTERNAL, CHAIN_APPROX_NONE, Point(-1,-1)); // Point(-1,-1) accounts for previous copyMakeBorder
// Keep largest contour
int size_largest = 0;
int idx_largest = -1;
for (int i = 0; i < contours.size(); ++i)
{
if (contours[i].size() > size_largest)
{
size_largest = contours[i].size();
idx_largest = i;
}
}
Mat3b dbg;
cvtColor(closed, dbg, COLOR_GRAY2BGR);
// Black initialized mask
Mat1b mask(thresholded.rows, thresholded.cols, uchar(0));
if (idx_largest >= 0)
{
drawContours(dbg, contours, idx_largest, Scalar(0, 255, 0), CV_FILLED);
// Draw filled polygin on mask
drawContours(mask, contours, idx_largest, Scalar(255), 1);
}
// Get a point inside the contour
Moments m = moments(contours[idx_largest]);
Point2f inside(m.m10 / m.m00, m.m01 / m.m00);
floodFill(mask, inside, Scalar(127));
Mat3b result;
cvtColor(thresholded, result, COLOR_GRAY2BGR);
result.setTo(Scalar(0), mask != 127);
imshow("Closed", closed);
imshow("Contour", dbg);
imshow("Result", result);
waitKey();
return 0;
}

OCR Line detecting

I am trying to detectect the bounding box of sentences in an Image. I am using Emgu OpenCV in C# using HougLinesP method to extract lines, but I am obviously doing something wrong. I have looked at many examples and estimate the skew level with houghLines is pretty much what I am trying to do.
Using that sample image I do some pre-processing (Thresholding, canny, ect) and end up with http://snag.gy/sWCuO.jpg, but then when I do HoughLines and draw the lines on the original image, I get http://snag.gy/ESKmR.jpg .
Here is an extract of my code:
using (MemStorage stor = new MemStorage())
{
Image<Hsv, byte> imgHSV = new Image<Hsv, byte>(bitmap);
Image<Gray, Byte> gray = imgHSV.Convert<Gray, Byte>().PyrDown().PyrUp();
CvInvoke.cvCanny(gray, EdgeMap, 100, 400, 3);
IntPtr lines = CvInvoke.cvHoughLines2(EdgeMap, stor,
Emgu.CV.CvEnum.HOUGH_TYPE.CV_HOUGH_PROBABILISTIC, 1, Math.PI / 360, 10,
gray.Width / 4, 20);
Seq<LineSegment2D> segments = new Seq<LineSegment2D>(lines, stor);
ar = segments.ToArray();
}
Graphics g = Graphics.FromImage(OriginalImage);
foreach (LineSegment2D line in ar)
{
g.DrawLine(new Pen(Color.Blue),
new Point(line.P1.X, line.P1.Y),
new Point(line.P2.X, line.P2.Y));
}
g.Save();
Any help would be appreciated.

You can try two approaches:
1- Utilize the frequency domain. Example here
2- After pre-processing, extract the contours, collect all the points (or at least collect all points that are not black); find the minimum bounding rectangle with its angle. Example here