We Keep Coding

EmguCV PCACompute InputData - Convert VectorOfPoint to Mat

The CvInvoke.PCACompute method expects a IInputArray of data, to do the analysis.
I tried using the source image as the input Mat, but the eigenvectors computed are abnormal, as per my understanding. And I am not able to convert my Contour VectorOfPoint to Mat, which can me fed.
I could also not find a good literature online about implementing PCA Analysis in EmguCV / C#.
Can someone please point me in the right direction.
Below is my code -
public static void getOrientation(Image<Gray,byte> inputImage)
{
Image<Gray, Byte> cannyGray = inputImage.Canny(85, 255);
VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint();
Mat eigen_vectors = new Mat(inputImage.Size,DepthType.Cv8U,1);
Mat mean_mat = new Mat(inputImage.Size, DepthType.Cv8U, 1);
CvInvoke.FindContours(cannyGray, contours, null, RetrType.External, ChainApproxMethod.ChainApproxSimple);
Point[][] cont_points = contours.ToArrayOfArray();
Mat contour_mat = new Mat();
contour_mat.SetTo(cont_points[0]);
//CvInvoke.PCACompute(cannyGray.Mat, mean_mat, eigen_vectors,2);
CvInvoke.PCACompute(contours, mean_mat, eigen_vectors);
}

You have to convert each of your contour to a Mat containing your coordinates.
Here is an example of how you can do it:
// points are the point of one contour
var pointList = points.ToArray();
// use DepthType.Cv64F to allow numbers > 255
Mat dataPoints = new Mat(pointList.Length, 2, DepthType.Cv64F, 1);
double[] pointsData = new double[((int)dataPoints.Total * dataPoints.NumberOfChannels)];
// store the points coordinates in the Mat
for (int i = 0; i < dataPoints.Rows; i++)
{
pointsData[i * dataPoints.Cols] = pointList[i].X;
pointsData[i * dataPoints.Cols + 1] = pointList[i].Y;
}
// set the Mat to dataPointsData values
dataPoints.SetTo(pointsData);
// compute PCA
Mat mean = new Mat();
Mat eigenvectors = new Mat();
Mat eigenvalues = new Mat();
CvInvoke.PCACompute(dataPoints, mean, eigenvectors);

emgucv: pan card improper skew detection in C#

I am having three image of pan card for testing skew of image using emgucv and c#.
1st image which is on top Detected 180 degree working properly.
2nd image which is in middle Detected 90 dgree should detected as 180 degree.
3rd image Detected 180 degree should detected as 90 degree.
One observation I am having that i wanted to share here is when i crop unwanted part of image from up and down side of pan card using paint brush, it gives me expected result using below mention code.
Now i wanted to understand how i can remove the unwanted part using programming.
I have played with contour and roi but I am not able to figure out how to fit the same. I am not able to understand whether emgucv itself selects contour or I have to do something.
Please suggest any suitable code example.
Please check code below for angle detection and please help me. Thanks in advance.
imgInput = new Image<Bgr, byte>(impath);
Image<Gray, Byte> img2 = imgInput.Convert<Gray, Byte>();
Bitmap imgs;
Image<Gray, byte> imgout = imgInput.Convert<Gray, byte>().Not().ThresholdBinary(new Gray(50), new Gray(125));
VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint();
Emgu.CV.Mat hier = new Emgu.CV.Mat();
var blurredImage = imgInput.SmoothGaussian(5, 5, 0 , 0);
CvInvoke.AdaptiveThreshold(imgout, imgout, 255, Emgu.CV.CvEnum.AdaptiveThresholdType.GaussianC, Emgu.CV.CvEnum.ThresholdType.Binary, 5, 45);
CvInvoke.FindContours(imgout, contours, hier, Emgu.CV.CvEnum.RetrType.External, Emgu.CV.CvEnum.ChainApproxMethod.ChainApproxSimple);
if (contours.Size >= 1)
{
for (int i = 0; i <= contours.Size; i++)
{
Rectangle rect = CvInvoke.BoundingRectangle(contours[i]);
RotatedRect box = CvInvoke.MinAreaRect(contours[i]);
PointF[] Vertices = box.GetVertices();
PointF point = box.Center;
PointF edge1 = new PointF(Vertices[1].X - Vertices[0].X, Vertices[1].Y - Vertices[0].Y);
PointF edge2 = new PointF(Vertices[2].X - Vertices[1].X, Vertices[2].Y - Vertices[1].Y);
double r = edge1.X + edge1.Y;
double edge1Magnitude = Math.Sqrt(Math.Pow(edge1.X, 2) + Math.Pow(edge1.Y, 2));
double edge2Magnitude = Math.Sqrt(Math.Pow(edge2.X, 2) + Math.Pow(edge2.Y, 2));
PointF primaryEdge = edge1Magnitude > edge2Magnitude ? edge1 : edge2;
double primaryMagnitude = edge1Magnitude > edge2Magnitude ? edge1Magnitude : edge2Magnitude;
PointF reference = new PointF(1, 0);
double refMagnitude = 1;
double thetaRads = Math.Acos(((primaryEdge.X * reference.X) + (primaryEdge.Y * reference.Y)) / (primaryMagnitude * refMagnitude));
double thetaDeg = thetaRads * 180 / Math.PI;
imgInput = imgInput.Rotate(thetaDeg, new Bgr());
imgout = imgout.Rotate(box.Angle, new Gray());
Bitmap bmp = imgout.Bitmap;
break;
}
}

The Problem
Let us start with the problem before the solution:
Your Code
When you submit code, asking for help, at least make some effort to "clean" it. Help people help you! There's so many lines of code here that do nothing. You declare variables that are never used. Add some comments that let people know what it is that you think your code should do.
Bitmap imgs;
var blurredImage = imgInput.SmoothGaussian(5, 5, 0, 0);
Rectangle rect = CvInvoke.BoundingRectangle(contours[i]);
PointF point = box.Center;
double r = edge1.X + edge1.Y;
// Etc
Adaptive Thresholding
The following line of code produces the following images:
CvInvoke.AdaptiveThreshold(imgout, imgout, 255, Emgu.CV.CvEnum.AdaptiveThresholdType.GaussianC, Emgu.CV.CvEnum.ThresholdType.Binary, 5, 45);
Image 1
Image 2
Image 3
Clearly this is not what you're aiming for since the primary contour, the card edge, is completely lost. As a tip, you can always use the following code to display images at runtime to help you with debugging.
CvInvoke.NamedWindow("Output");
CvInvoke.Imshow("Output", imgout);
CvInvoke.WaitKey();
The Soltuion
Since your in example images the card is primarily a similar Value (in the HSV sense) to the background. I do not think simple gray scale thresholding is the correct approach in this case. I purpose the following:
Algorithm
Use Canny Edge Detection to extract the edges in the image.
Dilate the edges so as the card content combines.
Use Contour Detection to filter for the combined edges with the largest bounding.
Fit this primary contour with a rotated rectangle in order to extract the corner points.
Use the corner points to define a transformation matrix to be applied using WarpAffine.
Warp and crop the image.
The Code
You may wish to experiment with the parameters of the Canny Detection and Dilation.
// Working Images
Image<Bgr, byte> imgInput = new Image<Bgr, byte>("Test1.jpg");
Image<Gray, byte> imgEdges = new Image<Gray, byte>(imgInput.Size);
Image<Gray, byte> imgDilatedEdges = new Image<Gray, byte>(imgInput.Size);
Image<Bgr, byte> imgOutput;
// 1. Edge Detection
CvInvoke.Canny(imgInput, imgEdges, 25, 80);
// 2. Dilation
CvInvoke.Dilate(
imgEdges,
imgDilatedEdges,
CvInvoke.GetStructuringElement(
ElementShape.Rectangle,
new Size(3, 3),
new Point(-1, -1)),
new Point(-1, -1),
5,
BorderType.Default,
new MCvScalar(0));
// 3. Contours Detection
VectorOfVectorOfPoint inputContours = new VectorOfVectorOfPoint();
Mat hierarchy = new Mat();
CvInvoke.FindContours(
imgDilatedEdges,
inputContours,
hierarchy,
RetrType.External,
ChainApproxMethod.ChainApproxSimple);
VectorOfPoint primaryContour = (from contour in inputContours.ToList()
orderby contour.GetArea() descending
select contour).FirstOrDefault();
// 4. Corner Point Extraction
RotatedRect bounding = CvInvoke.MinAreaRect(primaryContour);
PointF topLeft = (from point in bounding.GetVertices()
orderby Math.Sqrt(Math.Pow(point.X, 2) + Math.Pow(point.Y, 2))
select point).FirstOrDefault();
PointF topRight = (from point in bounding.GetVertices()
orderby Math.Sqrt(Math.Pow(imgInput.Width - point.X, 2) + Math.Pow(point.Y, 2))
select point).FirstOrDefault();
PointF botLeft = (from point in bounding.GetVertices()
orderby Math.Sqrt(Math.Pow(point.X, 2) + Math.Pow(imgInput.Height - point.Y, 2))
select point).FirstOrDefault();
PointF botRight = (from point in bounding.GetVertices()
orderby Math.Sqrt(Math.Pow(imgInput.Width - point.X, 2) + Math.Pow(imgInput.Height - point.Y, 2))
select point).FirstOrDefault();
double boundingWidth = Math.Sqrt(Math.Pow(topRight.X - topLeft.X, 2) + Math.Pow(topRight.Y - topLeft.Y, 2));
double boundingHeight = Math.Sqrt(Math.Pow(botLeft.X - topLeft.X, 2) + Math.Pow(botLeft.Y - topLeft.Y, 2));
bool isLandscape = boundingWidth > boundingHeight;
// 5. Define warp crieria as triangles
PointF[] srcTriangle = new PointF[3];
PointF[] dstTriangle = new PointF[3];
Rectangle ROI;
if (isLandscape)
{
srcTriangle[0] = botLeft;
srcTriangle[1] = topLeft;
srcTriangle[2] = topRight;
dstTriangle[0] = new PointF(0, (float)boundingHeight);
dstTriangle[1] = new PointF(0, 0);
dstTriangle[2] = new PointF((float)boundingWidth, 0);
ROI = new Rectangle(0, 0, (int)boundingWidth, (int)boundingHeight);
}
else
{
srcTriangle[0] = topLeft;
srcTriangle[1] = topRight;
srcTriangle[2] = botRight;
dstTriangle[0] = new PointF(0, (float)boundingWidth);
dstTriangle[1] = new PointF(0, 0);
dstTriangle[2] = new PointF((float)boundingHeight, 0);
ROI = new Rectangle(0, 0, (int)boundingHeight, (int)boundingWidth);
}
Mat warpMat = new Mat(2, 3, DepthType.Cv32F, 1);
warpMat = CvInvoke.GetAffineTransform(srcTriangle, dstTriangle);
// 6. Apply the warp and crop
CvInvoke.WarpAffine(imgInput, imgInput, warpMat, imgInput.Size);
imgOutput = imgInput.Copy(ROI);
imgOutput.Save("Output1.bmp");
Two extension methods are used:
static List<VectorOfPoint> ToList(this VectorOfVectorOfPoint vectorOfVectorOfPoint)
{
List<VectorOfPoint> result = new List<VectorOfPoint>();
for (int contour = 0; contour < vectorOfVectorOfPoint.Size; contour++)
{
result.Add(vectorOfVectorOfPoint[contour]);
}
return result;
}
static double GetArea(this VectorOfPoint contour)
{
RotatedRect bounding = CvInvoke.MinAreaRect(contour);
return bounding.Size.Width * bounding.Size.Height;
}
Outputs
Meta Example

Imgproc.FindContours return empty contours

I use OpenCV 2.4.11 for Xamarin.Android with OpenCvBinding. I'm trying to find the largest color area in image.
static public Tuple<Bitmap,double> GetArea(Bitmap srcBitmap)
{
Mat mat = new Mat();
Mat gray = new Mat();
Mat mat2 = new Mat();
double max = 0;
Mat Hierarchy = new Mat();
List<MatOfPoint> contours = new List<MatOfPoint>();
Utils.BitmapToMat(srcBitmap, mat);
Imgproc.CvtColor(mat, gray, Imgproc.ColorRgba2gray);
Imgproc.AdaptiveThreshold(gray, mat2, 255, Imgproc.AdaptiveThreshGaussianC, Imgproc.ThreshBinaryInv,1111,0);
Imgproc.FindContours(mat2, contours, Hierarchy, Imgproc.RetrTree, Imgproc.ChainApproxSimple);
foreach (MatOfPoint contour in contours)
{ // never goes here
if (max < Imgproc.ContourArea(contour)) max = Imgproc.ContourArea(contour);
}
Utils.MatToBitmap(mat2,srcBitmap);
return new Tuple<Bitmap, double>(srcBitmap,max);
}
Input Image
If I comment the line with FindContours, I'll get an excellent picture for searching contours.
Threshholded image
FindContours returns correct image(Reputation doesn't allow to add another link), but(!!) list of contours standing empty. So i can't get the area of these contures.
I would be glad of any help. Thanks!

use IList contours = new JavaList();

How to crop an image with a rotated rectangle?

I have an image in the form of a System.Drawing.Bitmap and a rectangle in the form of 4 points (Vector2s which are trivially converted to PointFs).
I want to use those points to crop out a section of the image. I found this answer which is pretty close to what I want, but I'm not sure how to get the right matrix out of it.
Here's what I've got so far:
protected static Bitmap CropImage(Bitmap src, Vector2[] rect)
{
var width = (rect[1] - rect[0]).Length;
var height = (rect[3] - rect[0]).Length;
var result = new Bitmap(M2.Round(width), M2.Round(height));
using (Graphics g = Graphics.FromImage(result))
{
g.InterpolationMode = InterpolationMode.HighQualityBicubic;
using (Matrix mat = new Matrix())
{
// ????
}
}
return result;
}
How can I get the proper transform matrix out of my rect?

It would be the same as in the linked answer, but instead of:
mat.Translate(-rect.Location.X, -rect.Location.Y);
mat.RotateAt(angle, rect.Location);
You would use:
double angle = Math.Atan2(rect[1].Y - rect[0].Y, rect[1].X - rect[0].X);
mat.Translate(-rect[0].X, -rect[0].Y);
mat.RotateAt((float)angle, rect[0]);
(Or something along those lines. It may be -angle, or rect[0] instead of rect[1] and vice-versa in Atan2. I can’t check immediately…)

Figured it out:
protected static Bitmap CropImage(Bitmap src, Vector2[] rect)
{
var width = (rect[1] - rect[0]).Length;
var height = (rect[3] - rect[0]).Length;
var result = new Bitmap(M2.Round(width), M2.Round(height));
using (Graphics g = Graphics.FromImage(result))
{
g.InterpolationMode = InterpolationMode.HighQualityBicubic;
using (Matrix mat = new Matrix())
{
var rot = -Math.Atan2(rect[1].Y - rect[0].Y, rect[1].X - rect[0].X) * M2.RadToDeg;
mat.Translate(-rect[0].X, -rect[0].Y);
mat.RotateAt((float)rot, rect[0].ToPointF());
g.Transform = mat;
g.DrawImage(src, new Rectangle(0, 0, src.Width, src.Height));
}
}
return result;
}

Emgu CV - How i can get all occurrence of pattern in Image

Hi have already function solution but one issue:
// The screenshot will be stored in this bitmap.
Bitmap capture = new Bitmap(rec.Width, rec.Height, PixelFormat.Format24bppRgb);
using (Graphics g = Graphics.FromImage(capture))
{
g.CopyFromScreen(rec.Location, new System.Drawing.Point(0, 0), rec.Size);
}
MCvSURFParams surfParam = new MCvSURFParams(500, false);
SURFDetector surfDetector = new SURFDetector(surfParam);
// Template image
Image<Gray, Byte> modelImage = new Image<Gray, byte>("template.jpg");
// Extract features from the object image
ImageFeature[] modelFeatures = surfDetector.DetectFeatures(modelImage, null);
// Prepare current frame
Image<Gray, Byte> observedImage = new Image<Gray, byte>(capture);
ImageFeature[] imageFeatures = surfDetector.DetectFeatures(observedImage, null);
// Create a SURF Tracker using k-d Tree
Features2DTracker tracker = new Features2DTracker(modelFeatures);
Features2DTracker.MatchedImageFeature[] matchedFeatures = tracker.MatchFeature(imageFeatures, 2);
matchedFeatures = Features2DTracker.VoteForUniqueness(matchedFeatures, 0.8);
matchedFeatures = Features2DTracker.VoteForSizeAndOrientation(matchedFeatures, 1.5, 20);
HomographyMatrix homography = Features2DTracker.GetHomographyMatrixFromMatchedFeatures(matchedFeatures);
// Merge the object image and the observed image into one image for display
Image<Gray, Byte> res = modelImage.ConcateVertical(observedImage);
#region draw lines between the matched features
foreach (Features2DTracker.MatchedImageFeature matchedFeature in matchedFeatures)
{
PointF p = matchedFeature.ObservedFeature.KeyPoint.Point;
p.Y += modelImage.Height;
res.Draw(new LineSegment2DF(matchedFeature.SimilarFeatures[0].Feature.KeyPoint.Point, p), new Gray(0), 1);
}
#endregion
#region draw the project region on the image
if (homography != null)
{
// draw a rectangle along the projected model
Rectangle rect = modelImage.ROI;
PointF[] pts = new PointF[] {
new PointF(rect.Left, rect.Bottom),
new PointF(rect.Right, rect.Bottom),
new PointF(rect.Right, rect.Top),
new PointF(rect.Left, rect.Top)
};
homography.ProjectPoints(pts);
for (int i = 0; i < pts.Length; i++)
pts[i].Y += modelImage.Height;
res.DrawPolyline(Array.ConvertAll<PointF, Point>(pts, Point.Round), true, new Gray(255.0), 2);
}
#endregion
pictureBoxScreen.Image = res.ToBitmap();
the result is:
And my problem is that, function homography.ProjectPoints(pts);
Get only first occurrence of pattern (white rectangle in pic above)
How i can Project all occurrence of template, respectively how I can get occurrence of template rectangle in image

I face a problem similar to yours in my master thesis. Basically you have two options:
Use a clustering such as Hierarchical k-means or a point density one such as DBSCAN (it depends on two parameters but you can make it threshold free in bidimensional R^2 space)
Use a multiple robust model fitting estimation techniques such as JLinkage. In this more advanced technique you clusters points that share an homography instead of cluster points that close to each other in euclidean space.
Once you partition your matches in "clusters" you can estimate homographies between matches belonging to correspondant clusters.