We Keep Coding

C# Emgu BackgroundSubtractorMOG2 returns black image

Hey and thanks for your time.
im working on some opencv in C#(Emgu) and trying to diplay the forground from BackgroundSubtractorMOG2
but with no luck here is my code
internal Bitmap AdaptableBackgroundSubtraction(Bitmap org, int history = 2, bool detectShadows = true, int threshold = 16)
{
Image<Bgr, byte> imageorg = org.ToImage<Bgr, byte>();
var forgroundmask = imageorg;
var mDetector = new BackgroundSubtractorMOG2(history, threshold, detectShadows);
mDetector.Apply(imageorg, forgroundmask, 0.5);
return forgroundmask.ToBitmap();
}
i have tried a few things and changing parameters but i dont get any results. any help will be appreciated
Picture of program running
Link to the full code if needed
https://gitlab.com/Clipcometx/semesterportefolje/-/tree/master/Machinelearning/ComputerVision

According to attached picture, you set the max hue value to 255, while the hue range is 0-179.

Emgucv crop detected shape automatically

I have an application which is going to be used to crop blank spaces from scanned documents for example this image. What I want to do is extract only the card and remove all the white/blank area. I'm using Emgucv FindContours to do this and at the moment I'm able to find the card contour and some noise captured by the scanner in the image as you can see below.
My question is how can I crop the largest contour found or how to extract it by removing other contours and blanks/whitespaces? Or maybe it is possible with the contour index?
Edit: Maybe another possible solution is if is possible to draw the contour to another pictureBox.
Here is the code that I'm using:
Image<Bgr, byte> imgInput;
Image<Bgr, byte> imgCrop;
private void abrirToolStripMenuItem_Click(object sender, EventArgs e)
{
try
{
OpenFileDialog dialog = new OpenFileDialog();
if (dialog.ShowDialog() ==DialogResult.OK)
{
imgInput = new Image<Bgr, byte>(dialog.FileName);
pictureBox1.Image = imgInput.Bitmap;
imgCrop = imgInput;
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
private void shapeToolStripMenuItem_Click(object sender, EventArgs e)
{
if (imgCrop == null)
{
return;
}
try
{
var temp = imgCrop.SmoothGaussian(5).Convert<Gray, byte>().ThresholdBinaryInv(new Gray(230), new Gray(255));
VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint();
Mat m = new Mat();
CvInvoke.FindContours(temp, contours, m, Emgu.CV.CvEnum.RetrType.External, Emgu.CV.CvEnum.ChainApproxMethod.ChainApproxSimple);
for (int i = 0; i < contours.Size; i++)
{
double perimeter = CvInvoke.ArcLength(contours[i], true);
VectorOfPoint approx = new VectorOfPoint();
CvInvoke.ApproxPolyDP(contours[i], approx, 0.04 * perimeter, true);
CvInvoke.DrawContours(imgCrop, contours, i, new MCvScalar(0, 0, 255), 2);
pictureBox2.Image = imgCrop.Bitmap;
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}

I'll give you my answer in C++, but the same operations should be available in Emgu CV.
I propose the following approach: Segment (that is – separate) the target object using the HSV color space. Calculate a binary mask for the object of interest. Get the biggest blob in the binary mask, this should be the card. Compute the bounding box of the card. Crop the card out of the input image
Ok, first get (or read) the input image. Apply a median blur filter, it will help in getting rid of that high-frequency noise (the little grey blobs) that you see on the input. The main parameter to adjust is the size of the kernel (or filter aperture) be careful, though – a high value will result in an aggressive effect and will likely destroy your image:
//read input image:
std::string imageName = "C://opencvImages//yoshiButNotYoshi.png";
cv::Mat imageInput = cv::imread( imageName );
//apply a median blur filter, the size of the kernel is 5 x 5:
cv::Mat blurredImage;
cv::medianBlur ( imageInput, blurredImage, 5 );
This is the result of the blur filter (The embedded image is resized):
Next, segment the image. Exploit the fact that the background is white, and everything else (the object of interest, mainly) has some color information. You can use the HSV color space. First, convert the BGR image into HSV:
//BGR to HSV conversion:
cv::Mat hsvImg;
cv::cvtColor( blurredImage, hsvImg, CV_RGB2HSV );
The HSV color space encodes color information differently than the typical BGR/RGB color space. Its advantage over other color models pretty much depends on the application, but in general, it is more robust while working with hue gradients. I'll try to get an HSV-based binary mask for the object of interest.
In a binary mask, everything you are interested on the input image is colored in white, everything else in black (or vice versa). You can obtain this mask using the inRange function. However, you must specify the color ranges that will be rendered in white (or black) in the output mask. For your image, and using the HSV color model those values are:
cv::Scalar minColor( 0, 0, 100 ); //the lower range of colors
cv::Scalar maxColor( 0, 0, 255 ); //the upper range of colors
Now, get the binary mask:
//prepare the binary mask:
cv::Mat binaryMask;
//create the binary mask using the specified range of color
cv::inRange( hsvImg, minColor, maxColor, binaryMask );
//invert the mask:
binaryMask = 255 - binaryMask;
You get this image:
Now, you can get rid of some of the noise (that survived the blur filter) via morphological filtering. Morphological filters are, essentially, logical rules applied on binary (or gray) images. They take a "neighborhood" of pixels in the input and apply logical functions to get an output. They are quite handy while cleaning up binary images. I'll apply a series of logical filters to achieve just that.
I'll first erode the image and then dilate it using 3 iterations. The structuring element is a rectangle of size 3 x 3:
//apply some morphology the clean the binary mask a little bit:
cv::Mat SE = cv::getStructuringElement( cv::MORPH_RECT, cv::Size(3, 3) );
int morphIterations = 3;
cv::morphologyEx( binaryMask, binaryMask, cv::MORPH_ERODE, SE, cv::Point(-1,-1), morphIterations );
cv::morphologyEx( binaryMask, binaryMask, cv::MORPH_DILATE, SE, cv::Point(-1,-1), morphIterations );
You get this output. Check out how the noisy blobs are mostly gone:
Now, comes the cool part. You can loop through all the contours in this image and get the biggest of them all. That's a typical operation that I constantly perform, so, I've written a function that does that. It is called findBiggestBlob. I'll present the function later. Check out the result you get after finding and extracting the biggest blob:
//find the biggest blob in the binary image:
cv::Mat biggestBlob = findBiggestBlob( binaryMask );
You get this:
Now, you can get the bounding box of the biggest blob using boundingRect:
//Get the bounding box of the biggest blob:
cv::Rect bBox = cv::boundingRect( biggestBlob );
Let's draw the bounding box on the input image:
cv::Mat imageClone = imageInput.clone();
cv::rectangle( imageClone, bBox, cv::Scalar(255,0,0), 2 );
Finally, let's crop the card out of the input image:
cv::Mat croppedImage = imageInput( bBox );
This is the cropped output:
This is the code for the findBiggestBlob function. The idea is just to compute all the contours in the binary input, calculate their area and store the contour with the largest area of the bunch:
//Function to get the largest blob in a binary image:
cv::Mat findBiggestBlob( cv::Mat &inputImage ){
cv::Mat biggestBlob = inputImage.clone();
int largest_area = 0;
int largest_contour_index = 0;
std::vector< std::vector<cv::Point> > contours; // Vector for storing contour
std::vector< cv::Vec4i > hierarchy;
// Find the contours in the image
cv::findContours( biggestBlob, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE );
for( int i = 0; i < (int)contours.size(); i++ ) {
//Find the area of the contour
double a = cv::contourArea( contours[i], false);
//Store the index of largest contour:
if( a > largest_area ){
largest_area = a;
largest_contour_index = i;
}
}
//Once you get the biggest blob, paint it black:
cv::Mat tempMat = biggestBlob.clone();
cv::drawContours( tempMat, contours, largest_contour_index, cv::Scalar(0),
CV_FILLED, 8, hierarchy );
//Erase the smaller blobs:
biggestBlob = biggestBlob - tempMat;
tempMat.release();
return biggestBlob;
}

Emgu CV to Detect Circular Cookies on a Baking Sheet

I'm fairly new to image processing and that being said, I was hoping someone could tell me if I am on the right track and if not, point me in the right direction and/or provide some code samples.
The requirements I am working on:
Detect the number of cookies on a baking sheet.
The cookies can be any color.
The cookies may be covered in chocolate (white or black) in which case they will have a mess of chocolate around each cookies meaning doing a simple contrast check probably won't work.
The cookies will not overlap but they may touch one another.
I am trying to use the Emgu CV library with HoughCirlces but I am getting mixed results. Here is my code using winforms and C# in which I load an image of cookies on a baking sheet and run it on (I am not confident in my values).
Am I on the right track? Any ideas? Code samples?
Below are some test images:
http://imgur.com/a/dJmU6, followed by my code
private int GetHoughCircles(Image image)
{
Bitmap bitmap = new Bitmap(image);
Image<Bgr, Byte> img = new Image<Bgr, byte>(bitmap).Resize(466, 345, Emgu.CV.CvEnum.INTER.CV_INTER_CUBIC);
//Get and sharpen gray image (don't remember where I found this code; prob here on SO)
Image<Gray, Byte> graySoft = img.Convert<Gray, Byte>().PyrDown().PyrUp();
Image<Gray, Byte> gray = graySoft.SmoothGaussian(3);
gray = gray.AddWeighted(graySoft, 1.5, -0.5, 0);
Image<Gray, Byte> bin = gray.ThresholdBinary(new Gray(149), new Gray(255));
Gray cannyThreshold = new Gray(150);
Gray cannyThresholdLinking = new Gray(120);
Gray circleAccumulatorThreshold = new Gray(50);
Image<Gray, Byte> cannyEdges = bin.Canny(cannyThreshold.Intensity, cannyThresholdLinking.Intensity);
//Image<Gray, Byte> cannyEdges = bin.Canny(cannyThreshold, cannyThresholdLinking);
//Circles
CircleF[] circles = cannyEdges.HoughCircles(
cannyThreshold,
circleAccumulatorThreshold,
3.0, //Resolution of the accumulator used to detect centers of the circles
50.0, //min distance
20, //min radius
30 //max radius
)[0]; //Get the circles from the first channel
//draw circles (on original image)
foreach (CircleF circle in circles)
{
img.Draw(circle, new Bgr(Color.Brown), 2);
}
pictureBox1.Image = new Bitmap(img.ToBitmap());
return circles.Count();
}

EmguCV 2.3.0 in C# QueryFrame returns the previous queried frame

I am using EmguCV 2.3.0 and am querying frames at random intervals from a Capture to be saved to hard drive. The problem is when I call the Capture.QueryFrame() or Capture.QuerySmallFrame() it is delayed by a frame.
To make this more clear: I start the program and query a frame pointed at my face. My face appears in the .jpeg. I then point the camera away from my face and query another frame, and my face appears in the .jpeg again. I then point it back at my face once more, query a frame and the .jpeg contains the image pointed away from my face.
There appears to be a 1 frame delay in a query. Is there some underlying buffer? What is causing this? And most importantly: how can I solve this problem without querying multiple frames for a single captured image?
Another question I have is that when I set the webcam resolution to 1600x1200, the program and computer begins to lag - even without making use of the images or querying frames. Is that caused only because I create a Capture and keep it in memory? Is there a way to lessen the affects of this?
Note: This solution is not sufficient for quick acquisition of frames, the question continues here:
System.TypeInitializationException using Emgu.CV in C#

Well the problem you have is that your not disposing of your old capture so when you send of for another it will always return the old one and then get another. The following code adjusted from the Web camera example should do the trick.
_capture = new Capture();
Image<Bgr, Byte> frame = _capture.QueryFrame();
Image<Gray, Byte> grayFrame = frame.Convert<Gray, Byte>();
Image<Gray, Byte> smallGrayFrame = grayFrame.PyrDown();
Image<Gray, Byte> smoothedGrayFrame = smallGrayFrame.PyrUp();
Image<Gray, Byte> cannyFrame = smoothedGrayFrame.Canny(new Gray(100), new Gray(60));
captureImageBox.Image = frame;
grayscaleImageBox.Image = grayFrame;
smoothedGrayscaleImageBox.Image = smoothedGrayFrame;
cannyImageBox.Image = cannyFrame;
_capture.Dispose();
It is the _capture.Dispose(); that is the important bit.
As for your 1600x1200 yes your right it is because you have a large amount of data in memory. First of start by efficiently 'using' the memory stream and disposing of it when where done with it. This is done with the 'using' statement which automatically creates the object at the start and calls its .Dispose function at the end, just so you don't have to. Note the copy procedure else a pointer is passed and when you exit the using code you will dispose of frame as well. You should also practice the 'using' statement when using images. But the above code will now look like this:
Image<Bgr, Byte> frame;
using (Capture capture = new Capture())
{
frame = capture1.QueryFrame().Copy(); //You must copy else frame will be disposed off
}
Image<Gray, Byte> grayFrame = frame.Convert<Gray, Byte>();
Image<Gray, Byte> smallGrayFrame = grayFrame.PyrDown();
Image<Gray, Byte> smoothedGrayFrame = smallGrayFrame.PyrUp();
Image<Gray, Byte> cannyFrame = smoothedGrayFrame.Canny(new Gray(100), new Gray(60));
grayscaleImageBox.Image = grayFrame;
smoothedGrayscaleImageBox.Image = smoothedGrayFrame;
cannyImageBox.Image = cannyFrame;
Second of all you can resize the captured image using .Resize(scale, Interpolation Method). Large images are usually passed from a camera using a dedicated frame grabber so system dependency is avoided obviously this is no longer the case with HD USB web cams.
You can resize your input image like so and with your 'using' statements efficiently your final code will look like this:
Image<Bgr, Byte> frame;
using (Capture capture1 = new Capture())
{
frame = capture1.QueryFrame().Resize(0.5, Emgu.CV.CvEnum.INTER.CV_INTER_AREA).Copy();
captureImageBox.Image = frame;
}
using (Image<Gray, Byte> grayFrame = frame.Convert<Gray, Byte>())
{
grayscaleImageBox.Image = grayFrame;
using (Image<Gray, Byte> smallGrayFrame = grayFrame.PyrDown())
{
using (Image<Gray, Byte> smoothedGrayFrame = smallGrayFrame.PyrUp())
{
smoothedGrayscaleImageBox.Image = smoothedGrayFrame;
using (Image<Gray, Byte> cannyFrame = smoothedGrayFrame.Canny(new Gray(100), new Gray(60)))
{
cannyImageBox.Image = cannyFrame;
}
}
}
}
Cheers
Chris

I mean as stupid as it sounds, just query the frame twice. I know that is not the "state-of-the-art"-way to solve Problems but i had the same issue and i could solve it that way.
Here is my Capture-Method
public void Capture_Image()
{
try
{
capture.Start();
capture.Pause();
tempImg = capture.QueryFrame().Bitmap;//Capture twice here
tempImg = capture.QueryFrame().Bitmap;
tempImg.Save("C:/FHT59N3/Bildanalyse_Projekt/image.jpg", System.Drawing.Imaging.ImageFormat.Jpeg);
}
catch (NullReferenceException)
{
string message = "No Camera found";
string title = "Please connect Camera";
MessageBoxButtons buttons = MessageBoxButtons.OK;
MessageBox.Show(message, title, buttons, MessageBoxIcon.Warning);
}
}

How to make white blob tracking for video or camera capture on Emgu?

I want to make program using C# with Emgu that can detect white blobs on images from camera and also track it. Also, the program can return IDs of tracked blobs
Frame1: http://www.freeimagehosting.net/uploads/ff2ac19054.jpg
Frame2: http://www.freeimagehosting.net/uploads/09e20e5dd6.jpg

The Emgu sample project "VideoSurveilance" in the Emgu.CV.Example solution (Emgu.CV.Example.sln) demonstrates blob tracking and assigns ID's to them.
I'm a newbie to OpenCV but it seems to me that the tracking of only "white" blobs may be harder than it sounds. For example, the blobs in your sample picture aren't really "white" are they? What I think you are really trying to do is "get the blobs that are brighter than the background by a certain amount" i.e. find a gray blob on a black background or a white blob on a gray background.

It depends what's your background like. If it is constantly dark like on those images you attached, then you should be able to extract those "white" blobs with some threshold. For any smarter segmentation you'll need to use some other features as well (e.g. like correlation if your object is color consistent).

I cannot say the code will work because I haven't tested it.
The general idea is to take the captured frame (assuming you're capturing frames) and filter out the noise by modifying the saturation and value(brightness). This modified HSV image is then processed as greyscale. Blobs can be labeled by looping through the blob collection generated by the tracker and assigned id's and bounding boxes.
Also, you may be interested in AForge.net and the related article: Hands Gesture Recognition on the mechanics and implementation of using the histogram for computer vision.
This is a modified version of custom tracker code found on the nui forums:
static void Main(){
Capture capture = new Capture(); //create a camera captue
Image<Bgr, Byte> img = capture.QuerySmallFrame();
OptimizeBlobs(img);
BackgroundStatisticsModel bsm = new BackgroundStatisticsModel(img, Emgu.CV.CvEnum.BG_STAT_TYPE.FGD_STAT_MODEL);
bsm.Update(img);
BlobSeq oldBlobs = new BlobSeq();
BlobSeq newBlobs = new BlobSeq();
ForgroundDetector fd = new ForgroundDetector(Emgu.CV.CvEnum.FORGROUND_DETECTOR_TYPE.FGD);
BlobDetector bd = new BlobDetector(Emgu.CV.CvEnum.BLOB_DETECTOR_TYPE.CC);
BlobTracker bt = new BlobTracker(Emgu.CV.CvEnum.BLOBTRACKER_TYPE.CC);
BlobTrackerAutoParam btap = new BlobTrackerAutoParam();
btap.BlobDetector = bd;
btap.ForgroundDetector = fd;
btap.BlobTracker = bt;
btap.FGTrainFrames = 5;
BlobTrackerAuto bta = new BlobTrackerAuto(btap);
Application.Idle += new EventHandler(delegate(object sender, EventArgs e)
{ //run this until application closed (close button click on image viewer)
//******* capture image *******
img = capture.QuerySmallFrame();
OptimizeBlobs(img);
bd.DetectNewBlob(img, bsm.Foreground, newBlobs, oldBlobs);
List<MCvBlob> blobs = new List<MCvBlob>(bta);
MCvFont font = new MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_SIMPLEX, 1.0, 1.0);
foreach (MCvBlob blob in blobs)
{
img.Draw(Rectangle.Round(blob), new Gray(255.0), 2);
img.Draw(blob.ID.ToString(), ref font, Point.Round(blob.Center), new Gray(255.0));
}
Image<Gray, Byte> fg = bta.GetForgroundMask();
});
}
public Image<Gray, Byte> OptimizeBlobs(Image<Gray, Byte img)
{
// can improve image quality, but expensive if real-time capture
img._EqualizeHist();
// convert img to temporary HSV object
Image<Hsv, Byte> imgHSV = img.Convert<Hsv, Byte>();
// break down HSV
Image<Gray, Byte>[] channels = imgHSV.Split();
Image<Gray, Byte> imgHSV_saturation = channels[1]; // saturation channel
Image<Gray, Byte> imgHSV_value = channels[2]; // value channel
//use the saturation and value channel to filter noise. [you will need to tweak these values]
Image<Gray, Byte> saturationFilter = imgHSV_saturation.InRange(new Gray(0), new Gray(80));
Image<Gray, Byte> valueFilter = imgHSV_value.InRange(new Gray(200), new Gray(255));
// combine the filters to get the final image to process.
Image<Gray, byte> imgTarget = huefilter.And(saturationFilter);
return imgTarget;
}