We Keep Coding

Copy smaller Mat to larger Mat OpenCv For Unity

I'm trying to take a smaller image mat and copy it into larger mat so I can resize it while keeping the aspect ratio of the image. So, basically this:
So far, this is the code I've written:
private Mat MakeMatFrame(Texture2D image)
{
// Texture must be of right input size
Mat img_mat = new Mat(image.height, image.width, CvType.CV_8UC4, new Scalar(0, 0, 0, 255));
texture2DToMat(image, img_mat);
return img_mat;
}
private void letterBoxImage(Texture2D image)
{
// Get input image as a mat
Mat source = MakeMatFrame(image);
// Create the mat that the source will be put in
int col = source.cols();
int row = source.rows();
int _max = Math.Max(col, row);
Mat resized = Mat.zeros(_max, _max, CvType.CV_8UC4);
// Fill resized
Mat roi = new Mat(resized, new Rect(0, 0, col, row));
source.copyTo(roi);
Texture2D tex2d = new Texture2D(resized.cols(), resized.rows());
matToTexture2D(resized, tex2d);
rawImage.texture = tex2d;
}
Everything I've looked at tells me this is the right approach to take (get a region of interest, fill it in). But instead of getting that third image with the children above the gray region, I just have a gray region.
In other words, the image isn't copying over properly. I've trying using a submat as well, but it failed miserably.
I've been looking for C# code on how to do this sort of thing with OpenCv For Unity, but I can only find C++ code. Which tells me to do exactly this.
Is there some sort of "apply changes" function I'm unaware of for Mats? Am I selecting the region of interest incorrectly? Or is it something else?

sorry for my english,but ur code has a bug.
Mat roi = new Mat(resized, new Rect(0, 0, col, row));
image copied to roi,but this mat not related with resized Mat.so u have to do like this:
Rect roi=new Rect(0,0,width,height);
source.copyto(resized.submat(roi));

Emgucv crop detected shape automatically

I have an application which is going to be used to crop blank spaces from scanned documents for example this image. What I want to do is extract only the card and remove all the white/blank area. I'm using Emgucv FindContours to do this and at the moment I'm able to find the card contour and some noise captured by the scanner in the image as you can see below.
My question is how can I crop the largest contour found or how to extract it by removing other contours and blanks/whitespaces? Or maybe it is possible with the contour index?
Edit: Maybe another possible solution is if is possible to draw the contour to another pictureBox.
Here is the code that I'm using:
Image<Bgr, byte> imgInput;
Image<Bgr, byte> imgCrop;
private void abrirToolStripMenuItem_Click(object sender, EventArgs e)
{
try
{
OpenFileDialog dialog = new OpenFileDialog();
if (dialog.ShowDialog() ==DialogResult.OK)
{
imgInput = new Image<Bgr, byte>(dialog.FileName);
pictureBox1.Image = imgInput.Bitmap;
imgCrop = imgInput;
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
private void shapeToolStripMenuItem_Click(object sender, EventArgs e)
{
if (imgCrop == null)
{
return;
}
try
{
var temp = imgCrop.SmoothGaussian(5).Convert<Gray, byte>().ThresholdBinaryInv(new Gray(230), new Gray(255));
VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint();
Mat m = new Mat();
CvInvoke.FindContours(temp, contours, m, Emgu.CV.CvEnum.RetrType.External, Emgu.CV.CvEnum.ChainApproxMethod.ChainApproxSimple);
for (int i = 0; i < contours.Size; i++)
{
double perimeter = CvInvoke.ArcLength(contours[i], true);
VectorOfPoint approx = new VectorOfPoint();
CvInvoke.ApproxPolyDP(contours[i], approx, 0.04 * perimeter, true);
CvInvoke.DrawContours(imgCrop, contours, i, new MCvScalar(0, 0, 255), 2);
pictureBox2.Image = imgCrop.Bitmap;
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}

I'll give you my answer in C++, but the same operations should be available in Emgu CV.
I propose the following approach: Segment (that is – separate) the target object using the HSV color space. Calculate a binary mask for the object of interest. Get the biggest blob in the binary mask, this should be the card. Compute the bounding box of the card. Crop the card out of the input image
Ok, first get (or read) the input image. Apply a median blur filter, it will help in getting rid of that high-frequency noise (the little grey blobs) that you see on the input. The main parameter to adjust is the size of the kernel (or filter aperture) be careful, though – a high value will result in an aggressive effect and will likely destroy your image:
//read input image:
std::string imageName = "C://opencvImages//yoshiButNotYoshi.png";
cv::Mat imageInput = cv::imread( imageName );
//apply a median blur filter, the size of the kernel is 5 x 5:
cv::Mat blurredImage;
cv::medianBlur ( imageInput, blurredImage, 5 );
This is the result of the blur filter (The embedded image is resized):
Next, segment the image. Exploit the fact that the background is white, and everything else (the object of interest, mainly) has some color information. You can use the HSV color space. First, convert the BGR image into HSV:
//BGR to HSV conversion:
cv::Mat hsvImg;
cv::cvtColor( blurredImage, hsvImg, CV_RGB2HSV );
The HSV color space encodes color information differently than the typical BGR/RGB color space. Its advantage over other color models pretty much depends on the application, but in general, it is more robust while working with hue gradients. I'll try to get an HSV-based binary mask for the object of interest.
In a binary mask, everything you are interested on the input image is colored in white, everything else in black (or vice versa). You can obtain this mask using the inRange function. However, you must specify the color ranges that will be rendered in white (or black) in the output mask. For your image, and using the HSV color model those values are:
cv::Scalar minColor( 0, 0, 100 ); //the lower range of colors
cv::Scalar maxColor( 0, 0, 255 ); //the upper range of colors
Now, get the binary mask:
//prepare the binary mask:
cv::Mat binaryMask;
//create the binary mask using the specified range of color
cv::inRange( hsvImg, minColor, maxColor, binaryMask );
//invert the mask:
binaryMask = 255 - binaryMask;
You get this image:
Now, you can get rid of some of the noise (that survived the blur filter) via morphological filtering. Morphological filters are, essentially, logical rules applied on binary (or gray) images. They take a "neighborhood" of pixels in the input and apply logical functions to get an output. They are quite handy while cleaning up binary images. I'll apply a series of logical filters to achieve just that.
I'll first erode the image and then dilate it using 3 iterations. The structuring element is a rectangle of size 3 x 3:
//apply some morphology the clean the binary mask a little bit:
cv::Mat SE = cv::getStructuringElement( cv::MORPH_RECT, cv::Size(3, 3) );
int morphIterations = 3;
cv::morphologyEx( binaryMask, binaryMask, cv::MORPH_ERODE, SE, cv::Point(-1,-1), morphIterations );
cv::morphologyEx( binaryMask, binaryMask, cv::MORPH_DILATE, SE, cv::Point(-1,-1), morphIterations );
You get this output. Check out how the noisy blobs are mostly gone:
Now, comes the cool part. You can loop through all the contours in this image and get the biggest of them all. That's a typical operation that I constantly perform, so, I've written a function that does that. It is called findBiggestBlob. I'll present the function later. Check out the result you get after finding and extracting the biggest blob:
//find the biggest blob in the binary image:
cv::Mat biggestBlob = findBiggestBlob( binaryMask );
You get this:
Now, you can get the bounding box of the biggest blob using boundingRect:
//Get the bounding box of the biggest blob:
cv::Rect bBox = cv::boundingRect( biggestBlob );
Let's draw the bounding box on the input image:
cv::Mat imageClone = imageInput.clone();
cv::rectangle( imageClone, bBox, cv::Scalar(255,0,0), 2 );
Finally, let's crop the card out of the input image:
cv::Mat croppedImage = imageInput( bBox );
This is the cropped output:
This is the code for the findBiggestBlob function. The idea is just to compute all the contours in the binary input, calculate their area and store the contour with the largest area of the bunch:
//Function to get the largest blob in a binary image:
cv::Mat findBiggestBlob( cv::Mat &inputImage ){
cv::Mat biggestBlob = inputImage.clone();
int largest_area = 0;
int largest_contour_index = 0;
std::vector< std::vector<cv::Point> > contours; // Vector for storing contour
std::vector< cv::Vec4i > hierarchy;
// Find the contours in the image
cv::findContours( biggestBlob, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE );
for( int i = 0; i < (int)contours.size(); i++ ) {
//Find the area of the contour
double a = cv::contourArea( contours[i], false);
//Store the index of largest contour:
if( a > largest_area ){
largest_area = a;
largest_contour_index = i;
}
}
//Once you get the biggest blob, paint it black:
cv::Mat tempMat = biggestBlob.clone();
cv::drawContours( tempMat, contours, largest_contour_index, cv::Scalar(0),
CV_FILLED, 8, hierarchy );
//Erase the smaller blobs:
biggestBlob = biggestBlob - tempMat;
tempMat.release();
return biggestBlob;
}

How to increase accuracy of OCR text detection in tesseract with emgucv?

Well I am not able to get good accuracy of text detection in tesseract. Please check code and image below.
Mat imgInput = CvInvoke.Imread(#"D:\workspace\raw2\IMG_20200625_194541.jpg", ImreadModes.AnyColor);
int kernel_size = 11;
//Dilation
Mat imgDilatedEdges = new Mat();
CvInvoke.Dilate(
imgInput,
imgDilatedEdges,
CvInvoke.GetStructuringElement(
ElementShape.Rectangle,
new Size(kernel_size, kernel_size),
new Point(1, 1)),
new Point(1, 1),
1,
BorderType.Default,
new MCvScalar(0));
//Blur
Mat imgBlur = new Mat();
CvInvoke.MedianBlur(imgDilatedEdges, imgBlur, kernel_size);
//Abs diff
Mat imgAbsDiff = new Mat();
CvInvoke.AbsDiff(imgInput, imgBlur, imgAbsDiff);
Mat imgNorm = imgAbsDiff;
//Normalize
CvInvoke.Normalize(imgAbsDiff, imgNorm, 0, 255, NormType.MinMax, DepthType.Default);
Mat imgThreshhold = new Mat();
//getting threshhold value
double thresholdval = CvInvoke.Threshold(imgAbsDiff, imgThreshhold, 230, 0, ThresholdType.Trunc);
//Normalize
CvInvoke.Normalize(imgThreshhold, imgThreshhold, 0, 255, NormType.MinMax, DepthType.Default);
imgThreshhold.Save(#"D:\workspace\ocr_images\IMG_20200625_194541.jpg");
//contrast correction
Mat lab = new Mat();
CvInvoke.CvtColor(imgThreshhold, lab, ColorConversion.Bgr2Lab);
VectorOfMat colorChannelB = new VectorOfMat();
CvInvoke.Split(lab, colorChannelB);
CvInvoke.CLAHE(colorChannelB[0], 3.0, new Size(12, 12), colorChannelB[0]);
Mat clahe = new Mat();
//merge
CvInvoke.Merge(colorChannelB, clahe);
Image<Bgr, byte> output = new Image<Bgr, byte>(#"D:\workspace\ocr_images\IMG_20200625_194541.jpg");
Bitmap bmp = output.ToBitmap();
//setting image to 300 dpi since tesseract likes that
bmp.SetResolution(300, 300);
bmp.Save(#"D:\workspace\ocr_images\IMG_20200625_194541.jpg");
I am not getting expected accuracy. Please check how image is converted.
source image
converted image
I have posted few images above that you can refer. For first image i am getting garbage data. For last two images i am getting partial data.
Converting image to gray scale and playing with threshold gives better output.
I want to understand that if in case threshold is the key part then how i will be able to get dynamic threshhold value for each new image? It is going to work as service so user will simply pass the image and get the result. My app should be intelligent enough to process and understand image.
Do i have to adjust contrast, threshold more accurately? If yes how i will do that? or image itself is faulty I mean noise causing problem.
Please let me know what i am doing wrong in the algorithm or anything which will help me to understand issue. Any one who is aware of please tell me what should be ideal steps for image preprocessing for OCR?
I am using csharp, emucv and tesseract.
Any suggestion will be highly appreciated.

EmguCV - Fisheye Calibration

For a school project, I am trying to calibrate a Dahua IP Camera, which has Fisheye distortion.
We need to calibrate the camera to undistort the image, because we need a flat image to do image processing.
So far, we have managed to do the calibration in Python with OpenCV, but the rest of the script is written in C#, so we would like to convert the code to C# using the EmguCV wrapper (OpenCV for .NET)
Correct me if I'm wrong, but so far I have done these steps:
I took 50 pictures of a Chessboard Grid, which is used to find the Corners
I let OpenCV calculate the correct matrices, one called CameraMatrix and the other is the Distortion Coëfficiënts.
With the 2 calculated matrices, we then Undistort an image, and the result is a flat image with no distortion.
In Python, this code works. I get 2 matrices that works to undistort.
I tried to copy these matrices in C#, without doing the rest of the Calibration. Since it's the same camera setup, the Distortion should be the same.
However, when I try to hardcode the matrices, the result is not what I want.
Could it be an issue related to EmguCV (The wrapper of OpenCV for .NET), or is it something related to my code?
Python Code
img = cv2.imread(filename)
K = np.array(...) # removed for brevity
D = np.array(...) # removed for brevity
DIM = (width, height) # image resolution
map1, map2 = cv2.fisheye.initUndistortRectifyMap(K, D, np.eye(3), K, DIM, cv2.CV_16SC2)
undistorted_img = cv2.remap(img, map1, map2, interpolation=cv2.INTER_LINEAR, borderMode=cv2.BORDER_CONSTANT)
C# code
Matrix<double> cameraMatrix = new Matrix<double>(3, 3);
Matrix<double> distortionCoeffs = new Matrix<double>(4, 1);
Mat outputMap1 = new Mat();
Mat outputMap2 = new Mat();
Mat r = new Mat(); // Can be empty, according to the documentation
Mat p = new Mat(); // Can be empty, according to the documentation
Fisheye.InitUndistorRectifyMap(cameraMatrix, distortionCoeffs, r, p, image.Size, DepthType.Cv32F, outputMap1, outputMap2);
CvInvoke.Remap(image, undistorted, outputMap1, outputMap2, Inter.Linear, BorderType.Constant);

Color tracking using EMGUcv

I am trying to make an colored object tracker which uses a binary image and blob detector to follow the target sort of like this: https://www.youtube.com/watch?v=9qky6g8NRmI . However I can not figure out how the ThresholdBinary() method work and if it is even the right one.
Here is a relevant bit of the code:
cam._SmoothGaussian(3);
blobDetector.Update(cam);
Image<Bgr,byte> binaryImage = cam.ThresholdBinary(new Bgr(145,0,145),new Bgr(0,0,0));
Image<Gray,byte> binaryImageGray = binaryImage.Conver<Gray,byte>();
blobTracker.Process(cam, binaryImageGray);
foreach (MCvBlob blob in blobTracker)
{
cam.Draw((Rectangle)blob, new Bgr(0,0,255),2);
}
When I display the binaryImage I do not even get blobs. I just get a black image.

Typically, the colored blob detection part of such an application works along the lines of:
Convert the image to HSV (hue, saturation, value) color space.
Filter the hue channel for all pixels with a hue value near the target value. Thresholding will typically give you all pixels with a value above or below the threshold. You are interested in the pixels near some target value.
Filter the obtained mask some more, possibly using the saturation/value channels or by removing small blobs. Ideally only the target blob remains.
Some sample code that aims to find a green object (hue ~50) such as the green ball shown in the video:
// 1. Convert the image to HSV
using (Image<Hsv, byte> hsv = original.Convert<Hsv, byte>())
{
// 2. Obtain the 3 channels (hue, saturation and value) that compose the HSV image
Image<Gray, byte>[] channels = hsv.Split();
try
{
// 3. Remove all pixels from the hue channel that are not in the range [40, 60]
CvInvoke.cvInRangeS(channels[0], new Gray(40).MCvScalar, new Gray(60).MCvScalar, channels[0]);
// 4. Display the result
imageBox1.Image = channels[0];
}
finally
{
channels[1].Dispose();
channels[2].Dispose();
}
}