Emgu CV Image sharpening and controur detection - c#

I am working on a project where I need to identify dots from IR lasers on a surface. I use for that a camera with IR filter
Some input images:
There can be several dots, too. So I tried to sharpen this image from webcam and then use FindContours method of Emgu CV.
There is my code:
public static Image<Gray, byte> Sharpen(Image<Gray, byte> image, int w, int h, double sigma1, double sigma2, int k)
{
w = (w % 2 == 0) ? w - 1 : w;
h = (h % 2 == 0) ? h - 1 : h;
//apply gaussian smoothing using w, h and sigma
var gaussianSmooth = image.SmoothGaussian(w, h, sigma1, sigma2);
//obtain the mask by subtracting the gaussian smoothed image from the original one
var mask = image - gaussianSmooth;
//add a weighted value k to the obtained mask
mask *= k;
//sum with the original image
image += mask;
return image;
}
private void ProcessFrame(object sender, EventArgs arg)
{
Mat frame = new Mat();
if (_capture.Retrieve(frame, CameraDevice))
{
Image<Bgr, byte> original = frame.ToImage<Bgr, byte>();
Image<Gray, byte> img = Sharpen(frame.ToImage<Gray, byte>(), 100, 100, 100, 100, 30);
Image<Gray, byte> thresh = new Image<Gray, byte>(img.Size);
CvInvoke.PyrDown(img, thresh);
CvInvoke.PyrUp(thresh, thresh);
Image<Gray, byte> mask = new Image<Gray, byte>(thresh.Size);
Image<Gray, byte> cannyImg = thresh.Canny(10, 50);
VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint();
Mat hierarchy = new Mat();
CvInvoke.FindContours(
cannyImg,
contours,
hierarchy,
RetrType.External,
ChainApproxMethod.ChainApproxSimple
);
Image<Bgr, byte> resultImage = img.Copy().Convert<Bgr, byte>();
int contCount = contours.Size;
for (int i = 0; i < contCount; i++)
{
using (VectorOfPoint contour = contours[i])
{
resultImage.Draw(CvInvoke.BoundingRectangle(contour), new Bgr(255, 0, 0), 5);
}
}
captureBox.Image = original.Bitmap;
cvBox.Image = resultImage.Bitmap;
}
}
Example of result image:
So it almost all the time works as I expect it to, but framerate is very low. I'm getting like 10-15 fps with resolution of 640x480. I need to be able to do the same thing for 1920x1080 with at least 30 fps. It's my first time with OpenCV and Emgu.CV. What can I do to make it perform better?

I solved this just setting the threshold, so that image turns black and white only. By adjusting the threshold I was able to achieve the same results if not better in terms of clarity, but also performance drastically improved since there is not heavy processing going on
Here is a snippet with ARCore library instead on EmguCV
var bitmap = eventArgs.Frame;
var filter = new Grayscale(0.2125, 0.7154, 0.0721);
var grayImage = filter.Apply(bitmap);
var thresholdFilter = new Threshold(CurrentThreshold);
thresholdFilter.ApplyInPlace(grayImage);
var blobCounter = new BlobCounter();
blobCounter.ProcessImage(grayImage);
var rectangles = blobCounter.GetObjectsRectangles();

Related

How to use Facemarker in EMGUCV?

I'm trying to follow this OpenCV tutorial but I have not managed to create the FaceInvoke.FaceDetectNative function, I tried to use this function but the application stops working.
static bool MyDetector(IntPtr input, IntPtr output)
{
CascadeClassifier faceDetector = new CascadeClassifier(#"..\..\Resource\EMGUCV\haarcascade_frontalface_default.xml");
Image<Gray, byte> grayImage = (new Image<Bgr, byte>(CvInvoke.cvGetSize(input))).Convert<Gray, byte>();
grayImage._EqualizeHist();
Rectangle[] faces = faceDetector.DetectMultiScale(grayImage, 1.1, 10, Size.Empty);
VectorOfRect rects = new VectorOfRect(faces);
CvInvoke.cvCopy(rects.Ptr, output, IntPtr.Zero);
return true;
}
On the other hand I tried calling the GetFaces method by passing a Mat object = new Mat (); as IOutputArray which also has not worked (Crash error).
FacemarkLBFParams fParams = new FacemarkLBFParams();
fParams.ModelFile = #"..\..\Resource\EMGUCV\facemarkmodel.yaml";
FacemarkLBF facemark = new FacemarkLBF(fParams);
facemark.SetFaceDetector(MyDetector);
VectorOfRect result = new VectorOfRect();
Image<Bgr, Byte> image = new Image<Bgr, byte>(#"C:\Users\matias\Documents\Proyectos\100-20.bmp");
bool success = facemark.GetFaces(image, result);
Rectangle[] faces = result.ToArray();
Thank's
After several hours I have managed to detect the points of a face, for that use the Fit method, which receives the image, the faces (such as VectorOfRect) and a VectorOfVectorOfPointF for the output
public Image<Bgr, Byte> GetFacePoints()
{
CascadeClassifier faceDetector = new CascadeClassifier(#"..\..\Resource\EMGUCV\haarcascade_frontalface_default.xml");
FacemarkLBFParams fParams = new FacemarkLBFParams();
fParams.ModelFile = #"..\..\Resource\EMGUCV\lbfmodel.yaml";
fParams.NLandmarks = 68; // number of landmark points
fParams.InitShapeN = 10; // number of multiplier for make data augmentation
fParams.StagesN = 5; // amount of refinement stages
fParams.TreeN = 6; // number of tree in the model for each landmark point
fParams.TreeDepth = 5; //he depth of decision tree
FacemarkLBF facemark = new FacemarkLBF(fParams);
//facemark.SetFaceDetector(MyDetector);
Image<Bgr, Byte> image = new Image<Bgr, byte>(#"C:\Users\matias\Downloads\personas-buena-vibra-caracteristicas-1200x600.jpg");
Image<Gray, byte> grayImage = image.Convert<Gray, byte>();
grayImage._EqualizeHist();
VectorOfRect faces = new VectorOfRect(faceDetector.DetectMultiScale(grayImage));
VectorOfVectorOfPointF landmarks = new VectorOfVectorOfPointF();
facemark.LoadModel(fParams.ModelFile);
bool success = facemark.Fit(grayImage, faces, landmarks);
if (success)
{
Rectangle[] facesRect = faces.ToArray();
for (int i = 0; i < facesRect.Length; i++)
{
image.Draw(facesRect[i], new Bgr(Color.Blue), 2);
FaceInvoke.DrawFacemarks(image, landmarks[i], new Bgr(Color.Blue).MCvScalar);
}
return image;
}
return null;
}
Now all that remains is to optimize the code and continue with my project

Using Earth's Mover Distance with Emgu CV

I am working on a project in which I try to compare two images in C#. I'm using EmguCV (a C# wrapper for OpenCV). I've tested some funcitons which work (compareHist for example).
I am now trying to use the implementation of Earth's Mover Distance. As I use color images, I build a 2d Histogram based on the HSV image. I then build the corresponding signature (as described in the docs, and explained here).
The problem is that I always obtain a NaN as the output of my code. Since I'm new to C# and EmguCV, I've tried to make the same steps, but this time in Python, and it works, EMD returns a number without any error.
I've spent a lot of time on this problem trying to change the type of the histogram (between the OpenCV Mat and EmguCV image), looking at the histograms values to verify if they are right,... But I don't find what I'm doing wrong.
About the code:
I have a "Comparator" class which just contain 2 images:
class Comparator
{
public Image<Bgr, Byte> RefImage;
public Image<Bgr, Byte> TestImage;
public Comparator(string TestPath, string RefPath)
{
Image<Bgr, Byte> TestImagetemp = new Image<Bgr, Byte>(TestPath);
Image<Bgr, Byte> RefImagetemp = new Image<Bgr, Byte>(RefPath);
int newCols = Math.Min(TestImagetemp.Cols, RefImagetemp.Cols);
int newRows = Math.Min(RefImagetemp.Rows, TestImagetemp.Rows);
Rectangle roi = new Rectangle(0, 0, newCols, newRows);
this.RefImage = crop(RefImagetemp, roi);
this.TestImage = crop(TestImagetemp, roi);
string DiffPath = "C:\\Users\\EPIERSO\\Docs\\testdiff";
this.TestImage.Save(DiffPath + "testavant.png");
}
Here is the method used for computing the histogram:
public static Mat CalcHistHSV(Image<Bgr,Byte> image)
{
int[] histbins = new int[] { 30, 32 };
float[] ranges = new float[] { 0.0f, 180.0f, 0.0f, 256.0f };
Mat hist = new Mat();
VectorOfMat vm = new VectorOfMat();
Image<Hsv,float> imghsv = image.Convert<Hsv, float>();
vm.Push(imghsv);
CvInvoke.CalcHist(vm, new int[] { 0, 1 }, null, hist, histbins, ranges, false);
return hist;
}
And this is the method used for comparing with EMD:
public bool EMDCompare()
{
int hbins = 30;
int sbins = 32;
Mat histref = CalcHistHSV(RefImage);
Mat histtest = CalcHistHSV(TestImage);
//Computing the signatures
Mat sigref = new Mat(hbins*sbins,3,Emgu.CV.CvEnum.DepthType.Cv32F,1);
Mat sigtest = new Mat(hbins*sbins,3, Emgu.CV.CvEnum.DepthType.Cv32F, 1);
for (int h = 0; h<hbins; h++)
{
for (int s = 0; s < sbins; s++)
{
var bin = MatExtension.GetValue(histref,h,s);
MatExtension.SetValue(sigref, h * sbins + s, 0, bin);
MatExtension.SetValue(sigref, h * sbins + s, 1, h);
MatExtension.SetValue(sigref, h * sbins + s, 2, s);
var bin2 = MatExtension.GetValue(histtest, h, s);
MatExtension.SetValue(sigtest, h * sbins + s, 0, bin2);
MatExtension.SetValue(sigtest, h * sbins + s, 1, h);
MatExtension.SetValue(sigtest, h * sbins + s, 2, s);
}
}
float emd = CvInvoke.EMD(sigref, sigtest, DistType.L2);
return ((1 - emd) > 0.7);
}
For modifying Mat values, I use an extension named MatExtension, found here: How can I get and set pixel values of an EmguCV Mat image?
This is the equivalent Python code: https://pastebin.com/drhvNMNs

EmguCV: Draw contour on object in Motion using Optical Flow?

I would like to do motion detection in C# (using EmguCV 3.0) to remove object in motion or in foreground to draw an overlay.
Here is a sample test I done with a Kinect (because It's a depth camera)
How can I get started with EmguCV 3.0 ?
I tried many background removal code that do not work
It seems OpticalFlow is a good start but there si no example in EmguCV 3.0
If I find the largest blob how can I find its contours ?
Can someone help me to get started ?
EDIT: 17/06/2015
In EmguCV3.0.0 RC I don't see OpticalFlow in the package and documentation:
http://www.emgu.com/wiki/files/3.0.0-rc1/document/html/b72c032d-59ae-c36f-5e00-12f8d621dfb8.htm
There is only : DenseOpticalFlow, OpticalFlowDualTVL1 ???
This is a AbsDiff Code:
var grayFrame = frame.Convert<Gray, Byte>();
var motionFrame = grayFrame.AbsDiff(backFrame)
.ThresholdBinary(new Gray(20), new Gray(255))
.Erode(2)
.Dilate(2);
Result:
I don't know how to get the motion in white ?
This is the Blob Code:
Image<Bgr, Byte> smoothedFrame = new Image<Bgr, byte>(frame.Size);
CvInvoke.GaussianBlur(frame, smoothedFrame, new Size(3, 3), 1); //filter out noises
Mat forgroundMask = new Mat();
fgDetector.Apply(smoothedFrame, forgroundMask);
CvBlobs blobs = new CvBlobs();
blobDetector.Detect(forgroundMask.ToImage<Gray, byte>(), blobs);
blobs.FilterByArea(400, int.MaxValue);
blobTracker.Update(blobs, 1.0, 0, 1);
foreach (var pair in blobs) {
CvBlob b = pair.Value;
CvInvoke.Rectangle(frame, b.BoundingBox, new MCvScalar(255.0, 255.0, 255.0), 2);
}
Result:
Why so much false positive ?
This is a MOG2 Code:
forgroundDetector.Apply(frame, forgroundMask);
motionHistory.Update(forgroundMask);
var motionMask = GetMotionMask();
Image<Bgr, Byte> motionImage = new Image<Bgr, byte>(motionMask.Size);
CvInvoke.InsertChannel(motionMask, motionImage, 0);
Rectangle[] rects;
using (VectorOfRect boundingRect = new VectorOfRect()) {
motionHistory.GetMotionComponents(segMask, boundingRect);
rects = boundingRect.ToArray();
}
foreach (Rectangle comp in rects) { ...
Result:
If I select the biggest Area how can I get the contour of the object ?
First, I can give you some example Optical Flow code.
Let oldImage and newImage be variables that hold the previous and current frame. In my code, it's of type Image<Gray, Byte>.
// prep containers for x and y vectors
Image<Gray, float> velx = new Image<Gray, float>(newImage.Size);
Image<Gray, float> vely = new Image<Gray, float>(newImage.Size);
// use the Horn and Schunck dense optical flow algorithm.
OpticalFlow.HS(oldImage, newImage, true, velx, vely, 0.1d, new MCvTermCriteria(100));
// color each pixel
Image<Hsv, Byte> coloredMotion = new Image<Hsv, Byte>(newImage.Size);
for (int i = 0; i < coloredMotion.Width; i++)
{
for (int j = 0; j < coloredMotion.Height; j++)
{
// Pull the relevant intensities from the velx and vely matrices
double velxHere = velx[j, i].Intensity;
double velyHere = vely[j, i].Intensity;
// Determine the color (i.e, the angle)
double degrees = Math.Atan(velyHere / velxHere) / Math.PI * 90 + 45;
if (velxHere < 0)
{
degrees += 90;
}
coloredMotion.Data[j, i, 0] = (Byte) degrees;
coloredMotion.Data[j, i, 1] = 255;
// Determine the intensity (i.e, the distance)
double intensity = Math.Sqrt(velxHere * velxHere + velyHere * velyHere) * 10;
coloredMotion.Data[j, i, 2] = (intensity > 255) ? 255 : intensity;
}
}
// coloredMotion is now an image that shows intensity of motion by lightness
// and direction by color.
Regarding the larger question of how to remove the foreground:
If I had a way to get a static background image, that's the best way to start. Then, the foreground would be detected by the AbsDiff method and using Erode and Dilate or Gaussian to smooth the image, then use blob detection.
For simple foreground detection, I found Optical Flow to be way too much processing (8fps max), whereas the AbsDiff method was just as accurate but had no effect on framerate.
Regarding contours, if you're merely looking to find the size, position, and other moments, then the blob detection in the AbsDiff tutorial above seems to be sufficient, which uses Image.FindContours(...).
If not, I would start looking at the CvBlobDetector class as used in this tutorial. There's a built-in DrawBlob function that might come in handy.

C# EmguCV - Circle line thickness calculation

I want to Circle line thickness calculation like this below:
Which method can help me to do so?
Thanks for your reply David. I'm new to emgucv. So I do not know where I'll start. I can do the following image using canny edge. But I can not calculate distance, because I do not know what I would use the code. Which can I use code?
private void button1_Click(object sender, EventArgs e)
{
string strFileName = string.Empty;
OpenFileDialog ofd = new OpenFileDialog();
if (ofd.ShowDialog() == DialogResult.OK)
{
//Load image
Image<Bgr, Byte> img1 = new Image<Bgr, Byte>(ofd.FileName);
//Convert the img1 to grayscale and then filter out the noise
Image<Gray, Byte> gray1 = img1.Convert<Gray, Byte>().PyrDown().PyrUp();
//Canny Edge Detector
Image<Gray, Byte> cannyGray = gray1.Canny(120, 180);
pictureBox1.Image = cannyGray.ToBitmap();
}
}
Let me Guide you a little bit further.
//load image
Image<Gray, Byte> loaded_img = new Image<Gray, byte>(Filename);
Image<Gray, Byte> Thresh_img = loaded_img.CopyBlank();
//threshold to make it binary if necessaray
CvInvoke.cvThreshold(loaded_img.Ptr, Thresh_img.Ptr, 0, 255, Emgu.CV.CvEnum.THRESH.CV_THRESH_OTSU | Emgu.CV.CvEnum.THRESH.CV_THRESH_BINARY);
//get contours of circle
Contour<Point> Circle_cont = Thresh_img.FindContours();
//get height & width of the bounding rectangle
int height_Circle_1 = Circle_cont.BoundingRectangle.Height;
int width_circle_1 = Circle_cont.BoundingRectangle.Width;
Circle_cont = Circle_cont.HNext;
int height_Circle_2 = Circle_cont.BoundingRectangle.Height;
int width_Circle_2 = Circle_cont.BoundingRectangle.Width;
//get ring thicknes in Px
double ring_thickness_1 = Math.Abs(height_Circle_1 - height_Circle_2) / 2;
double ring_thickness_2 = Math.Abs(width_circle_1 - width_Circle_2) / 2;
This should get you the thickness of your ring in Pixels. If you want it in cm you need to scale the Px value with the real Pixel length. I applied this code snippet to your example image and got 56 Pixel for both Thickness values.
I hope this will help you.

Finding contour points in emgucv

I am working with emguCV for finding contours essential points then saving this point in a file and user redraw this shape in future. so, my goal is this image:
example
my solution is this:
1. import image to picturebox
2. edge detection with canny algorithm
3. finding contours and save points
I found a lot of points with below codes but i can't drawing first shape with this point!
using Emgu.CV;
using Emgu.Util;
private void button1_Click(object sender, EventArgs e)
{
Bitmap bmp = new Bitmap(pictureBox1.Image);
Image<Bgr, Byte> img = new Image<Bgr, byte>(bmp);
Image<Gray, Byte> gray = img.Convert<Gray, Byte>().PyrDown().PyrUp();
Gray cannyThreshold = new Gray(80);
Gray cannyThresholdLinking = new Gray(120);
Gray circleAccumulatorThreshold = new Gray(120);
Image<Gray, Byte> cannyEdges = gray.Canny(cannyThreshold, cannyThresholdLinking).Not();
Bitmap color;
Bitmap bgray;
IdentifyContours(cannyEdges.Bitmap, 50, true, out bgray, out color);
pictureBox1.Image = color;
}
public void IdentifyContours(Bitmap colorImage, int thresholdValue, bool invert, out Bitmap processedGray, out Bitmap processedColor)
{
Image<Gray, byte> grayImage = new Image<Gray, byte>(colorImage);
Image<Bgr, byte> color = new Image<Bgr, byte>(colorImage);
grayImage = grayImage.ThresholdBinary(new Gray(thresholdValue), new Gray(255));
if (invert)
{
grayImage._Not();
}
using (MemStorage storage = new MemStorage())
{
for (Contour<Point> contours = grayImage.FindContours(Emgu.CV.CvEnum.CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE, Emgu.CV.CvEnum.RETR_TYPE.CV_RETR_LIST, storage); contours != null; contours = contours.HNext)
{
Contour<Point> currentContour = contours.ApproxPoly(contours.Perimeter * 0.015, storage);
if (currentContour.BoundingRectangle.Width > 20)
{
CvInvoke.cvDrawContours(color, contours, new MCvScalar(255), new MCvScalar(255), -1, 1, Emgu.CV.CvEnum.LINE_TYPE.EIGHT_CONNECTED, new Point(0, 0));
color.Draw(currentContour.BoundingRectangle, new Bgr(0, 255, 0), 1);
}
Point[] pts = currentContour.ToArray();
foreach (Point p in pts)
{
//add points to listbox
listBox1.Items.Add(p);
}
}
}
processedColor = color.ToBitmap();
processedGray = grayImage.ToBitmap();
}
In your code you have added contour approximation operation
Contour<Point> currentContour = contours.ApproxPoly(contours.Perimeter * 0.015, storage);
This contour approximation will approximate your Contour to a nearest polygon & so your actual points got shifted. If you want to reproduce the same image you need not to do any approximation.
Refer this thread.

Categories