I am working on a project in which I try to compare two images in C#. I'm using EmguCV (a C# wrapper for OpenCV). I've tested some funcitons which work (compareHist for example).
I am now trying to use the implementation of Earth's Mover Distance. As I use color images, I build a 2d Histogram based on the HSV image. I then build the corresponding signature (as described in the docs, and explained here).
The problem is that I always obtain a NaN as the output of my code. Since I'm new to C# and EmguCV, I've tried to make the same steps, but this time in Python, and it works, EMD returns a number without any error.
I've spent a lot of time on this problem trying to change the type of the histogram (between the OpenCV Mat and EmguCV image), looking at the histograms values to verify if they are right,... But I don't find what I'm doing wrong.
About the code:
I have a "Comparator" class which just contain 2 images:
class Comparator
{
public Image<Bgr, Byte> RefImage;
public Image<Bgr, Byte> TestImage;
public Comparator(string TestPath, string RefPath)
{
Image<Bgr, Byte> TestImagetemp = new Image<Bgr, Byte>(TestPath);
Image<Bgr, Byte> RefImagetemp = new Image<Bgr, Byte>(RefPath);
int newCols = Math.Min(TestImagetemp.Cols, RefImagetemp.Cols);
int newRows = Math.Min(RefImagetemp.Rows, TestImagetemp.Rows);
Rectangle roi = new Rectangle(0, 0, newCols, newRows);
this.RefImage = crop(RefImagetemp, roi);
this.TestImage = crop(TestImagetemp, roi);
string DiffPath = "C:\\Users\\EPIERSO\\Docs\\testdiff";
this.TestImage.Save(DiffPath + "testavant.png");
}
Here is the method used for computing the histogram:
public static Mat CalcHistHSV(Image<Bgr,Byte> image)
{
int[] histbins = new int[] { 30, 32 };
float[] ranges = new float[] { 0.0f, 180.0f, 0.0f, 256.0f };
Mat hist = new Mat();
VectorOfMat vm = new VectorOfMat();
Image<Hsv,float> imghsv = image.Convert<Hsv, float>();
vm.Push(imghsv);
CvInvoke.CalcHist(vm, new int[] { 0, 1 }, null, hist, histbins, ranges, false);
return hist;
}
And this is the method used for comparing with EMD:
public bool EMDCompare()
{
int hbins = 30;
int sbins = 32;
Mat histref = CalcHistHSV(RefImage);
Mat histtest = CalcHistHSV(TestImage);
//Computing the signatures
Mat sigref = new Mat(hbins*sbins,3,Emgu.CV.CvEnum.DepthType.Cv32F,1);
Mat sigtest = new Mat(hbins*sbins,3, Emgu.CV.CvEnum.DepthType.Cv32F, 1);
for (int h = 0; h<hbins; h++)
{
for (int s = 0; s < sbins; s++)
{
var bin = MatExtension.GetValue(histref,h,s);
MatExtension.SetValue(sigref, h * sbins + s, 0, bin);
MatExtension.SetValue(sigref, h * sbins + s, 1, h);
MatExtension.SetValue(sigref, h * sbins + s, 2, s);
var bin2 = MatExtension.GetValue(histtest, h, s);
MatExtension.SetValue(sigtest, h * sbins + s, 0, bin2);
MatExtension.SetValue(sigtest, h * sbins + s, 1, h);
MatExtension.SetValue(sigtest, h * sbins + s, 2, s);
}
}
float emd = CvInvoke.EMD(sigref, sigtest, DistType.L2);
return ((1 - emd) > 0.7);
}
For modifying Mat values, I use an extension named MatExtension, found here: How can I get and set pixel values of an EmguCV Mat image?
This is the equivalent Python code: https://pastebin.com/drhvNMNs
Related
The CvInvoke.PCACompute method expects a IInputArray of data, to do the analysis.
I tried using the source image as the input Mat, but the eigenvectors computed are abnormal, as per my understanding. And I am not able to convert my Contour VectorOfPoint to Mat, which can me fed.
I could also not find a good literature online about implementing PCA Analysis in EmguCV / C#.
Can someone please point me in the right direction.
Below is my code -
public static void getOrientation(Image<Gray,byte> inputImage)
{
Image<Gray, Byte> cannyGray = inputImage.Canny(85, 255);
VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint();
Mat eigen_vectors = new Mat(inputImage.Size,DepthType.Cv8U,1);
Mat mean_mat = new Mat(inputImage.Size, DepthType.Cv8U, 1);
CvInvoke.FindContours(cannyGray, contours, null, RetrType.External, ChainApproxMethod.ChainApproxSimple);
Point[][] cont_points = contours.ToArrayOfArray();
Mat contour_mat = new Mat();
contour_mat.SetTo(cont_points[0]);
//CvInvoke.PCACompute(cannyGray.Mat, mean_mat, eigen_vectors,2);
CvInvoke.PCACompute(contours, mean_mat, eigen_vectors);
}
You have to convert each of your contour to a Mat containing your coordinates.
Here is an example of how you can do it:
// points are the point of one contour
var pointList = points.ToArray();
// use DepthType.Cv64F to allow numbers > 255
Mat dataPoints = new Mat(pointList.Length, 2, DepthType.Cv64F, 1);
double[] pointsData = new double[((int)dataPoints.Total * dataPoints.NumberOfChannels)];
// store the points coordinates in the Mat
for (int i = 0; i < dataPoints.Rows; i++)
{
pointsData[i * dataPoints.Cols] = pointList[i].X;
pointsData[i * dataPoints.Cols + 1] = pointList[i].Y;
}
// set the Mat to dataPointsData values
dataPoints.SetTo(pointsData);
// compute PCA
Mat mean = new Mat();
Mat eigenvectors = new Mat();
Mat eigenvalues = new Mat();
CvInvoke.PCACompute(dataPoints, mean, eigenvectors);
I am trying to implement the unsharp masking method on emgucv using c#.
The python code I have now is (ref):
def unsharp_mask(image, kernel_size=(5, 5), sigma=1.0, amount=1.0, threshold=0):
"""Return a sharpened version of the image, using an unsharp mask."""
# For details on unsharp masking, see:
# https://en.wikipedia.org/wiki/Unsharp_masking
# https://homepages.inf.ed.ac.uk/rbf/HIPR2/unsharp.htm
blurred = cv.GaussianBlur(image, kernel_size, sigma)
sharpened = float(amount + 1) * image - float(amount) * blurred
sharpened = np.maximum(sharpened, np.zeros(sharpened.shape))
sharpened = np.minimum(sharpened, 255 * np.ones(sharpened.shape))
sharpened = sharpened.round().astype(np.uint8)
if threshold > 0:
low_contrast_mask = np.absolute(image - blurred) < threshold
np.copyto(sharpened, image, where=low_contrast_mask)
return sharpened
The c# code I have now cannot do the work as the above code does. Does anyone know how to implement it emgu cv using c#?
public static void GetMat(Image<Gray, byte> srcimg, Image<Gray, byte> imgBlurred, ref Mat dst, int nAmount = 200)
{
float amount = nAmount / 100f;
using (Image<Gray, byte> dst_temp = new Image<Gray, byte>(srcimg.Width, srcimg.Height))
{
for (int v = 0; v < srcimg.Height; v++)
{
for (int u = 0; u < srcimg.Width; u++)
{
byte a = srcimg.Data[v, u, 0]; //Get Pixel Color | fast way
byte b = imgBlurred.Data[v, u, 0];
int c = (int)(a * (1 + amount) - (amount * b));
if (c < 0) c = 0;
if (c > 255) c = 255;
dst_temp.Data[v, u, 0] = (byte)c;
}
}
dst = dst_temp.Mat.Clone();
}
}
public static void getSharpenImage(Mat src, ref Mat dst, int nAmount = 200, double sigma = 3, int threshold = 0)
{
float amount = nAmount / 100.0F;
using (Mat imgBlurred = new Mat())
{
CvInvoke.GaussianBlur(src, imgBlurred, new System.Drawing.Size(0, 0), sigma, sigma);
using (Mat mask_temp = new Mat())
{
CvInvoke.AbsDiff(src, imgBlurred, mask_temp);
using (Mat lowcontrastmask = new Mat())
{
CvInvoke.Threshold(mask_temp, lowcontrastmask, threshold, 255, ThresholdType.BinaryInv);
GetMat(src.ToImage<Gray, byte>(), imgBlurred.ToImage<Gray, byte>(), ref dst);
src.CopyTo(dst, lowcontrastmask);
}
}
}
}
https://www.idtools.com.au/unsharp-masking-python-opencv/ has a python solution.
the following works in C#:
Mat blurredImage = new Mat();
Mat lapImage = new Mat();
CvInvoke.MedianBlur(grayImage, blurredImage, 1);
CvInvoke.Laplacian(blurredImage, lapImage, blurredImage.Depth);
blurredImage -= (0.9*lapImage);
I'm currently trying to use the Fisheye.Calibrate method and the Fisheye.UndistorImage method from the Emgu.CV library. As far as I've understood, the Calibrate method is used to calculate a camera matrix (K) and a distortion vector (D), which are to be used to undistort fisheye-images using the UndistorImage method. However, when I use these two methods the results are not convincing. This is the input image I'm testing on: fisheye input image and this is the result: fisheye output image.
When I tried to look at the values of K and D by looking at the data-variable of the objects, it said 'null' for both K and D. Therefore I'm unsure if I'm using the Calibrate() metod correctly. My code is as follow:
private void EmguCVUndistortFisheye()
{
string[] fileNames = Directory.GetFiles(#"C:\Users\Test\Desktop\Jakob\ImageAnalysis\Images\Calibration", "*.png");
Size patternSize = new Size(6, 8);
VectorOfVectorOfPoint3D32F objPoints = new VectorOfVectorOfPoint3D32F();
VectorOfVectorOfPointF imagePoints = new VectorOfVectorOfPointF();
foreach (string file in fileNames)
{
Mat img = CvInvoke.Imread(file, ImreadModes.Grayscale);
CvInvoke.Imshow("input", img);
VectorOfPointF corners = new VectorOfPointF(patternSize.Width * patternSize.Height);
bool find = CvInvoke.FindChessboardCorners(img, patternSize, corners);
if (find)
{
MCvPoint3D32f[] points = new MCvPoint3D32f[patternSize.Width * patternSize.Height];
int loopIndex = 0;
for (int i = 0; i < patternSize.Height; i++)
{
for (int j = 0; j < patternSize.Width; j++)
points[loopIndex++] = new MCvPoint3D32f(j, i, 0);
}
objPoints.Push(new VectorOfPoint3D32F(points));
imagePoints.Push(corners);
}
}
Size imageSize = new Size(1280, 1024);
Mat K = new Mat();
Mat D = new Mat();
Mat rotation = new Mat();
Mat translation = new Mat();
Fisheye.Calibrate(
objPoints,
imagePoints,
imageSize,
K,
D,
rotation,
translation,
Fisheye.CalibrationFlag.CheckCond,
new MCvTermCriteria(30, 0.1)
);
foreach (string file in fileNames)
{
Mat img = CvInvoke.Imread(file, ImreadModes.Grayscale);
Mat output = img.Clone();
Fisheye.UndistorImage(img, output, K, D);
CvInvoke.Imshow("output", output);
}
}
Is the reason for my strange results a consequence of wrong parameters to the Calibrate method or is it simply the case of not using enough input images?
This looks like a similar problem to one I had recently when trying to pass a Mat into the calibration function when it needed a Matrix and as you've found it just doesn't work without reporting any errors. I think you'll need the following:
var K = new Matrix<double>(3, 3);
var D = new Matrix<double>(4, 1);
Also note that if you want to retrieve the rotation and translation vectors passing a Mat in is fine but you'll probably want to convert back to a Matrix if you want to perform calculations on them. I was just using a normal camera calibration rather than fish-eye but the following working code fragment might be useful to get the idea:
var cameraMatrix = new Matrix<double>(3, 3);
var distortionCoeffs = new Matrix<double>(4, 1);
var termCriteria = new MCvTermCriteria(30, 0.1);
System.Drawing.PointF[][] imagePoints = imagePointsList.Select(p => p.ToArray()).ToArray();
MCvPoint3D32f[][] worldPoints = worldPointsList.Select(p => p.ToArray()).ToArray();
double error = CvInvoke.CalibrateCamera(worldPoints, imagePoints, imageSize, cameraMatrix, distortionCoeffs, CalibType.RationalModel, termCriteria, out Mat[] rotationVectors, out Mat[] translationVectors);
var rotation = new Matrix<double>(rotationVectors[0].Rows, rotationVectors[0].Cols, rotationVectors[0].DataPointer);
var translation = new Matrix<double>(translationVectors[0].Rows, translationVectors[0].Cols, translationVectors[0].DataPointer);
I am working on a project where I need to identify dots from IR lasers on a surface. I use for that a camera with IR filter
Some input images:
There can be several dots, too. So I tried to sharpen this image from webcam and then use FindContours method of Emgu CV.
There is my code:
public static Image<Gray, byte> Sharpen(Image<Gray, byte> image, int w, int h, double sigma1, double sigma2, int k)
{
w = (w % 2 == 0) ? w - 1 : w;
h = (h % 2 == 0) ? h - 1 : h;
//apply gaussian smoothing using w, h and sigma
var gaussianSmooth = image.SmoothGaussian(w, h, sigma1, sigma2);
//obtain the mask by subtracting the gaussian smoothed image from the original one
var mask = image - gaussianSmooth;
//add a weighted value k to the obtained mask
mask *= k;
//sum with the original image
image += mask;
return image;
}
private void ProcessFrame(object sender, EventArgs arg)
{
Mat frame = new Mat();
if (_capture.Retrieve(frame, CameraDevice))
{
Image<Bgr, byte> original = frame.ToImage<Bgr, byte>();
Image<Gray, byte> img = Sharpen(frame.ToImage<Gray, byte>(), 100, 100, 100, 100, 30);
Image<Gray, byte> thresh = new Image<Gray, byte>(img.Size);
CvInvoke.PyrDown(img, thresh);
CvInvoke.PyrUp(thresh, thresh);
Image<Gray, byte> mask = new Image<Gray, byte>(thresh.Size);
Image<Gray, byte> cannyImg = thresh.Canny(10, 50);
VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint();
Mat hierarchy = new Mat();
CvInvoke.FindContours(
cannyImg,
contours,
hierarchy,
RetrType.External,
ChainApproxMethod.ChainApproxSimple
);
Image<Bgr, byte> resultImage = img.Copy().Convert<Bgr, byte>();
int contCount = contours.Size;
for (int i = 0; i < contCount; i++)
{
using (VectorOfPoint contour = contours[i])
{
resultImage.Draw(CvInvoke.BoundingRectangle(contour), new Bgr(255, 0, 0), 5);
}
}
captureBox.Image = original.Bitmap;
cvBox.Image = resultImage.Bitmap;
}
}
Example of result image:
So it almost all the time works as I expect it to, but framerate is very low. I'm getting like 10-15 fps with resolution of 640x480. I need to be able to do the same thing for 1920x1080 with at least 30 fps. It's my first time with OpenCV and Emgu.CV. What can I do to make it perform better?
I solved this just setting the threshold, so that image turns black and white only. By adjusting the threshold I was able to achieve the same results if not better in terms of clarity, but also performance drastically improved since there is not heavy processing going on
Here is a snippet with ARCore library instead on EmguCV
var bitmap = eventArgs.Frame;
var filter = new Grayscale(0.2125, 0.7154, 0.0721);
var grayImage = filter.Apply(bitmap);
var thresholdFilter = new Threshold(CurrentThreshold);
thresholdFilter.ApplyInPlace(grayImage);
var blobCounter = new BlobCounter();
blobCounter.ProcessImage(grayImage);
var rectangles = blobCounter.GetObjectsRectangles();
I'm trying to create a histogram for Back Projection of an image using the Emgu C# wrapper for OpenCV. I have the following OpenCV C++ code which I am trying to convert to Emgu C#:
char* filename = (char*)"C:\\Images\\items.jpg";
Mat im = imread(filename);
if (im.empty())
return -1;
const int channels[] = { 0, 1, 2 };
const int histSize[] = { 32, 32, 32 };
const float rgbRange[] = { 0, 256 };
const float* ranges[] = { rgbRange, rgbRange, rgbRange };
Mat hist;
Mat im32fc3, backpr32f;
im.convertTo(im32fc3, CV_32FC3);
calcHist(&im32fc3, 1, channels, Mat(), hist, 3, histSize, ranges, true, false);
calcBackProject(&im32fc3, 1, channels, hist, backpr32f, ranges);
Emgu C# conversion:
string filename = #"C:\Images\items.jpg";
Mat im = CvInvoke.Imread(filename);
if (im.IsEmpty)
{
return -1;
}
int[] channels = { 0, 1, 2 };
int[] histSize = { 32, 32, 32 };
float[] ranges = { 0.0f, 256.0f, 0.0f, 256.0f, 0.0f, 256.0f };
Mat im32fc3 = new Mat();
Mat hist = new Mat();
Mat mask = new Mat();
Mat backpr32f = new Mat();
im.ConvertTo(im32fc3, Emgu.CV.CvEnum.DepthType.Cv32F);
CvInvoke.CalcHist(im32fc3, channels, mask, hist, histSize, ranges, false);
CvInvoke.CalcBackProject(im32fc3, channels, hist, backpr32f, ranges);
However, I get the following error at CvInvoke.CalcHist:
OpenCV: 0 <= _rowRange.start && _rowRange.start <= _rowRange.end && _rowRange.end <= m.rows
Any help is appreciated.
I had the same problem. found this piece of code in emgu VS solution.
using (Util.VectorOfMat vm = new Util.VectorOfMat())
{
vm.Push(channels[i]);
float[] ranges = new float[] { minVal, maxVal };
CvInvoke.CalcHist(vm, new int[] { 0 }, null, hist, new int[] { numberOfBins }, ranges, false);
}
use VectorOfMat for 1st arugment instead of Mat.
Just wrap the Mat into a vector(array in c/c++) then pass it the first argument of CalcHist():
using var im32fc3Vector = new VectorOfMat(im32fc3);
CvInvoke.CalcHist(im32fc3Vector, ...);