Tracking Blobs with Microsoft Kinect - c#

I am working on a people counter. For this I have the Microsoft Kinect installed over the door.
I am working with C# and EmguCV. I have extracted the heads of the people, so that they appear as white blobs on a black image. Then I have created a bounding box around the heads. That works fine. So I now how many blobs I have per frame and I also now their position. This works fine. But now I want to track the blobs because I want to count how much people come in and go out, but I don't know how to do this. Can anyone help me? The problem is that every frame, new blobs can appear and old blobs can disappear. Can anyone give me an algorithm or maybe some code? or a paper.
Thanks a lot!
Sure. This is the code for the blobs:
using (MemStorage stor = new MemStorage())
{
Contour<System.Drawing.Point> contours = head_image.FindContours(Emgu.CV.CvEnum.CHAIN_APPROX_METHOD.CV_CHAIN_APPROX_SIMPLE, Emgu.CV.CvEnum.RETR_TYPE.CV_RETR_EXTERNAL, stor);
for (int i = 0; contours != null; contours = contours.HNext)
{
i++;
//if ((contours.Area > Math.Pow(sliderMinSize.Value, 2)) && (contours.Area < Math.Pow(sliderMaxSize.Value, 2)))
{
MCvBox2D box = contours.GetMinAreaRect();
blobCount++;
contour_image.Draw(box, new Bgr(System.Drawing.Color.Red), 1);
new_position = new System.Drawing.Point((int)(box.center.X), (int)(box.center.Y));
new_x = box.center.X;
new_y = box.center.Y;
}
}
}

Please see Emgu CV Blob Detection for more information. Assuming you are using Emgu CV 2.1 or higher, then the answer will work. If you are using version 1.5 or higher, see this thread on how to easily detect blobs. Or look at the code below
Capture capture = new Capture();
ImageViewer viewer = new ImageViewer();
BlobTrackerAutoParam param = new BlobTrackerAutoParam();
param.ForgroundDetector = new ForgroundDetector(Emgu.CV.CvEnum.FORGROUND_DETECTOR_TYPE.FGD);
param.FGTrainFrames = 10;
BlobTrackerAuto tracker = new BlobTrackerAuto(param);
Application.Idle += new EventHandler(delegate(object sender, EventArgs e)
{
tracker.Process(capture.QuerySmallFrame().PyrUp());
Image<Gray, Byte> img = tracker.GetForgroundMask();
//viewer.Image = tracker.GetForgroundMask();
MCvFont font = new MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_SIMPLEX, 1.0, 1.0);
foreach (MCvBlob blob in tracker)
{
img.Draw(Rectangle.Round(blob), new Gray(255.0), 2);
img.Draw(blob.ID.ToString(), ref font, Point.Round(blob.Center), new Gray(255.0));
}
viewer.Image = img;
});
viewer.ShowDialog();
Hope this helps!
EDIT
I think you should use this code every ten frames or so (~3 times a second) and do something like this:
Capture capture = new Capture();
ImageViewer viewer = new ImageViewer();
BlobTrackerAutoParam param = new BlobTrackerAutoParam();
param.ForgroundDetector = new ForgroundDetector(Emgu.CV.CvEnum.FORGROUND_DETECTOR_TYPE.FGD);
param.FGTrainFrames = 10;
BlobTrackerAuto tracker = new BlobTrackerAuto(param);
int frames = 0;
Application.Idle += new EventHandler(delegate(object sender, EventArgs e)
{
frames++;//Add to number of frames
if (frames == 10)
{
frames = 0;//if it is after 10 frames, do processing and reset frames to 0
tracker.Process(capture.QuerySmallFrame().PyrUp());
Image<Gray, Byte> img = tracker.GetForgroundMask();
//viewer.Image = tracker.GetForgroundMask();
int blobs = 0;
MCvFont font = new MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_SIMPLEX, 1.0, 1.0);
foreach (MCvBlob blob in tracker)
{
//img.Draw(Rectangle.Round(blob), new Gray(255.0), 2);
//img.Draw(blob.ID.ToString(), ref font, Point.Round(blob.Center), new Gray(255.0));
//Only uncomment these if you want to draw a rectangle around the blob and add text
blobs++;//count each blob
}
blobs = /*your counter here*/;
blobs = 0; //reset
viewer.Image = img;//get next frame
});
viewer.ShowDialog();
EDIT 2
It sounds like you just want to identify the blobs, it sounds like you want McvBlob.ID. This is the ID of the blob and you can check which ID's are still there and which are not. I would still do this every ten frames to not slow it down as much. You just need a simple algorithm that can observe what the ID's are, and if they have changed. I would store the IDs in a List<string> and check that list for changes every few frames. Example:
List<string> LastFrameIDs, CurrentFrameIDs;
Capture capture = new Capture();
ImageViewer viewer = new ImageViewer();
BlobTrackerAutoParam param = new BlobTrackerAutoParam();
param.ForgroundDetector = new ForgroundDetector(Emgu.CV.CvEnum.FORGROUND_DETECTOR_TYPE.FGD);
param.FGTrainFrames = 10;
BlobTrackerAuto tracker = new BlobTrackerAuto(param);
int frames = 0;
Application.Idle += new EventHandler(delegate(object sender, EventArgs e)
{
frames++;//Add to number of frames
if (frames == 10)
{
frames = 0;//if it is after 10 frames, do processing and reset frames to 0
tracker.Process(capture.QuerySmallFrame().PyrUp());
Image<Gray, Byte> img = tracker.GetForgroundMask();
//viewer.Image = tracker.GetForgroundMask();
int blobs = 0, i = 0;
MCvFont font = new MCvFont(Emgu.CV.CvEnum.FONT.CV_FONT_HERSHEY_SIMPLEX, 1.0, 1.0);
foreach (MCvBlob blob in tracker)
{
i++;
//img.Draw(Rectangle.Round(blob), new Gray(255.0), 2);
//img.Draw(blob.ID.ToString(), ref font, Point.Round(blob.Center), new Gray(255.0));
//Only uncomment these if you want to draw a rectangle around the blob and add text
CurrentFrameIDs.Add(blob.ID.ToString());
if (CurrentFrameIDs[i] == LastFrameIDs[i])
img.Draw(Rectangle.Round(blob), new Gray(0,0), 2);//mark the new/changed blob
blobs++;//count each blob
}
blobs = /*your counter here*/;
blobs = 0; //reset
i = 0;
LastFrameIDs = CurrentFrameIDs;
CurrentFrameIDs = null;
viewer.Image = img;//get next frame
});
viewer.ShowDialog();

Related

How can I convert extracted Mat Contours to Bitmap in Opencvsharp?

After I applied the Threshold on the Destination Image
I get a Mat as Reference
then I called the function FindContours()
to extract all existed Contours on the target Image
finally I tried to convert the extracted contours one at a time to Bitmap and here at this point am getting an Error:
System.ArgumentException: "Number of channels must be 1, 3 or 4. Parametername: src" by converting Mat to Bitmap
private void button1_Click(object sender, EventArgs e)
{
OpenFileDialog getImag = new OpenFileDialog();
getImag.Filter = "PNG,JPG|*.png;*.jpg";
DialogResult result = getImag.ShowDialog();
string Source_Logo_Link = string.Empty;
if (result == DialogResult.OK)
{
Source_Logo_Link = getImag.FileName;
System.Drawing.Image image = System.Drawing.Image.FromFile(Source_Logo_Link);
Mat src = new OpenCvSharp.Mat(Source_Logo_Link);
OpenCvSharp.Mat gray = src.CvtColor(ColorConversionCodes.BGR2GRAY);
OpenCvSharp.Mat binary = gray.Threshold(0, 255, ThresholdTypes.Otsu);
OpenCvSharp.Mat[] contoursQuery;
OutputArray hierarchyQ = InputOutputArray.Create(new List<Vec4i>());
binary.FindContours(out contoursQuery, hierarchyQ, RetrievalModes.CComp, ContourApproximationModes.ApproxTC89KCOS);
List<Bitmap> images = new List<Bitmap>();
for (int i = 0; i <= contoursQuery.Length; i++)
images.add(contoursQuery[i].toBitmap());
}
}
sorry this comes a bit late, but I hope this still helps though. Apart from a few other issues (like the counter of the for loop), I guess that your major problem is, that you miss-understood the result of FindContours.
It really just gives you a list of all contour points, which it found in the image, not complete images.
Have a look at the documentation FindContours, it clearly states for the contour parameter:
Each contour is stored as a vector of points
So that means if you wand them "one at a time to Bitmap" you'll need to create these bitmaps first, and then draw the contours into it. To make drawing a bit more easy (and faster) there is the DrawContours command, which you could use for the drawing part.
Over all, in the end, it should look somehow like this:
private void button1_Click(object sender, EventArgs e)
{
OpenFileDialog getImag = new OpenFileDialog();
getImag.Filter = "PNG,JPG|*.png;*.jpg";
DialogResult result = getImag.ShowDialog();
string Source_Logo_Link = string.Empty;
if (result == DialogResult.OK)
{
Source_Logo_Link = getImag.FileName;
System.Drawing.Image image = System.Drawing.Image.FromFile(Source_Logo_Link);
Mat src = new OpenCvSharp.Mat(Source_Logo_Link);
OpenCvSharp.Mat gray = src.CvtColor(ColorConversionCodes.BGR2GRAY);
OpenCvSharp.Mat binary = gray.Threshold(0, 255, ThresholdTypes.Otsu);
// I'd really prefer to work with point lists and HierarchyIndex because
// it's much more descriptive that way
OpenCvSharp.Point[][] contours;
OpenCvSharp.HierarchyIndex[] hierarchyQ;
binary.FindContours(out contours,out hierarchyQ, RetrievalModes.List, ContourApproximationModes.ApproxTC89KCOS);
List<Bitmap> images = new List<Bitmap>();
for (int i = 0; i < contoursQuery.Length; i++){
var singleContour = new OpenCvSharp.Mat(src.Size(), MatType.CV_8U,0);
singleContour.DrawContours(contours,i,Scalar.White);
images.Add(singleContour.ToBitmap());
}
}
}
I also changed the RetrievalMode to "List", since you don't really seem to care about the hierarchical relationships.
Hope that helps ;-)

linear image stitching using emgu.cv in c#

I have a device that rotates an object and takes a picture of a portion of the object at regular intervals. Currently, I have 30 pictures. To stitch the images into a flat image, I am taking a slice right out of the center of each picture of a fixed width (between 50 and 75 pixels). I am trying to stitch these slices together into a flat image of the original picture using the EMGU CV Stitching library with the sample stitching code that comes with EMGU. I am testing with between 5 and 10 slices at a time. Sometimes, I am getting an error that says "Error, need more images". When I do get a result, it looks terrible with weird curvatures. I don't need any spatial adjustments. I just want to stitch them in a linear fashion from left to right. Any ideas, either using EMGU or other?
Here are a few slices and the result:
Why is the resulting image not the same height as the 4 slices? What must be done just to stitch these together in a linear fashion so that the text is continuous?
Here is the code I am using:
private void selectImagesButton_Click(object sender, EventArgs e)
{
OpenFileDialog dlg = new OpenFileDialog();
dlg.CheckFileExists = true;
dlg.Multiselect = true;
if (dlg.ShowDialog() == System.Windows.Forms.DialogResult.OK)
{
sourceImageDataGridView.Rows.Clear();
Image<Bgr, byte>[] sourceImages = new Image<Bgr, byte>[dlg.FileNames.Length];
for (int i = 0; i < sourceImages.Length; i++)
{
sourceImages[i] = new Image<Bgr, byte>(dlg.FileNames[i]);
using (Image<Bgr, byte> thumbnail = sourceImages[i].Resize(200, 200, Emgu.CV.CvEnum.Inter.Cubic, true))
{
DataGridViewRow row = sourceImageDataGridView.Rows[sourceImageDataGridView.Rows.Add()];
row.Cells["FileNameColumn"].Value = dlg.FileNames[i];
row.Cells["ThumbnailColumn"].Value = thumbnail.ToBitmap();
row.Height = 200;
}
}
try
{
//only use GPU if you have build the native binary from code and enabled "NON_FREE"
using (Stitcher stitcher = new Stitcher(false))
{
using (AKAZEFeaturesFinder finder = new AKAZEFeaturesFinder())
{
stitcher.SetFeaturesFinder(finder);
using (VectorOfMat vm = new VectorOfMat())
{
Mat result = new Mat();
vm.Push(sourceImages);
Stopwatch watch = Stopwatch.StartNew();
this.Text = "Stitching";
Stitcher.Status stitchStatus = stitcher.Stitch(vm, result);
watch.Stop();
if (stitchStatus == Stitcher.Status.Ok)
{
resultImageBox.Image = result;
this.Text = String.Format("Stitched in {0} milliseconds.", watch.ElapsedMilliseconds);
}
else
{
MessageBox.Show(this, String.Format("Stiching Error: {0}", stitchStatus));
resultImageBox.Image = null;
}
}
}
}
}
finally
{
foreach (Image<Bgr, Byte> img in sourceImages)
{
img.Dispose();
}
}
}
}

C# Draw a horizontal line on each stacked column chart

Currently I need to develop a tool with chart as the major component. Chart control is also new for me. I have do a lot of reading, researching to learn and understand the whole picture of the chart control.
After all, I had stuck and question on how to draw a horizontal line (blue & red horizontal line) on stacked column chart as image shown below:
Here what I have done so far:
This is my code so far:
// X-Axis labels settings
chart.ChartAreas[0].AxisX.LabelStyle.Angle = -45;
chart.ChartAreas[0].AxisX.Interval = 1;
// Y-Axis labels settings
//chart.ChartAreas[0].AxisY.Minimum = 100;
chart.ChartAreas[0].AxisY.Minimum = 95;
// Plotting chart
using (YieldEntities context = new YieldEntities())
{
// Extract yield loss list
var yeilds = (
from yeild in context.YeildDatas
group yeild by new { yeild.Loss } into newyeild
select new
{
Loss = newyeild.Key.Loss,
Percentage = newyeild.Sum(p => p.Percentage)
}).OrderByDescending(p => p.Percentage);
//context.YeildDatas.Select(p => new { p.Loss, Percentage = p }).Distinct();
// Create new series
foreach (var yield in yeilds)
{
chart.Series.Add(yield.Loss);
chart.Series[yield.Loss].ChartType = SeriesChartType.StackedColumn100;
}
// Label settings for first series
chart.Series[0].SmartLabelStyle.Enabled = false;
chart.Series[0].LabelAngle = -90;
chart.Series[0].Font = new Font(Font.FontFamily, 15, FontStyle.Bold);
chart.Series[0].IsValueShownAsLabel = true;
var query = context.YeildDatas.ToList();
foreach (var item in query)
{
DataPoint dp = new DataPoint();
dp.SetValueXY(item.DateString, item.Percentage);
chart.Series[item.Loss].Points.Add(dp);
}
// Set empty datapoint for each series
foreach (var yield in yeilds)
{
DataPoint nulldp = new DataPoint();
nulldp.SetValueXY("", 0);
chart.Series[yield.Loss].Points.Insert(1, nulldp);
chart.Series[yield.Loss].Points.Insert(6, nulldp);
chart.Series[yield.Loss].Points.Insert(11, nulldp);
}
chart.Legends["Legend"].IsEquallySpacedItems = true;
chart.Legends["Legend"].IsTextAutoFit = true;
}
I hope to get any expert to guide me to solve this problem.
This is only sample, you can start from there:
// Data point to pixel
var pixelX = this.chart1.ChartAreas[0].AxisX.ValueToPixelPosition(dataPointX);
var pixelY = this.chart1.ChartAreas[0].AxisY.ValueToPixelPosition(dataPointY);
// Pixel to data point
var dataPointX = this.chart1.ChartAreas[0].AxisX.PixelPositionToValue(pixelX);
var dataPointY = this.chart1.ChartAreas[0].AxisY.PixelPositionToValue(pixelY);
// Use event Paint to draw your line
private void chart1_Paint(object sender, PaintEventArgs e)
{
// Convert dataPoint to pixel
var dataPointX = this.chart1.Series[0].Points[0].XValue;
var dataPointY = this.chart1.Series[0].Points[0].YValues[0];
var pixelX = this.chart1.ChartAreas[0].AxisX.ValueToPixelPosition(dataPointX);
var pixelY = this.chart1.ChartAreas[0].AxisY.ValueToPixelPosition(dataPointY);
// Only sample, pen should be initialized outside Paint method
Pen pen = new Pen(Brushes.Red);
// Example of drawing line with width=100 pixel
e.Graphics.DrawLine(pen, pixelX, pixelY, pixelX + 100, pixelY);
}

AForge.NET -> AVIWriter Adding Images as Frames

I want to make a video from images, and each image should stay for one second.
The AVIWriter has 25 frames rate, so i have to add one image 25 times to make it stay for one second.
I tried changing the frame-rate, but it is not working.
Can anyone suggest a workaround?
The following is the code for writing frames into video:
private void writeVideo()
{
// instantiate AVI writer, use WMV3 codec
AVIWriter writer = new AVIWriter("wmv3");
// create new AVI file and open it
writer.Open(fileName, 320, 240);
// create frame image
Bitmap image = new Bitmap(320, 240);
var cubit = new AForge.Imaging.Filters.ResizeBilinear(320, 240);
string[] files = Directory.GetFiles(imagesFolder);
writer.FrameRate = 25;
int index = 0;
int failed = 0;
foreach (var item in files)
{
index++;
try
{
image = Image.FromFile(item) as Bitmap;
//image = cubit.Apply(image);
for (int i = 0; i < 25; i++)
{
writer.AddFrame(image);
}
}
catch
{
failed++;
}
this.Text = index + " of " + files.Length + ". Failed: " + failed;
}
writer.Close();
}
Hello I had the same problem and saw this topic read it all and no comment for
http://www.aforgenet.com/framework/docs/html/bc8345f8-8e09-c1a4-4834-8330e5e85605.htm
There is a note like that "The property should be set befor opening new file to take effect."
The solution of Hakan is working, if you use this code:
AVIWriter videoWriter;
videoWriter = new AVIWriter("wmv3");
videoWriter.FrameRate = 1;
videoWriter.Open("test.avi", 320, 240);
videoWriter.AddFrame(bitmap1);
videoWriter.AddFrame(bitmap2);
videoWriter.AddFrame(bitmap3);
videoWriter.Close();
There is well one bitmap displayed per second.
(just for giving a directly working piece of code).
The default frame rate for AVIWriter is 25. As you have no option to specify dropped or otherwise skipped frames, why wouldn't you set AVIWriter::FrameRate property to 1 before you start populating your writer with frames?

Creating numerous PictureBoxes by code - only one is visible

I am trying to draw lots of instances of an image using the following code:
PictureBox[] sprites = new PictureBox[100];
private void Game_Load(object sender, EventArgs e)
{
PictureBox mainSprite = new PictureBox();
Bitmap img = new Bitmap(SpriteTest.Properties.Resources.Image); //Load a png image
mainSprite.Size = new Size(16, 16);
mainSprite.Image = img;
for(var i = 0; i < sprites.Length; i++)
{
sprites[i] = mainSprite;
//Keeping it simple for now with a single row of sprites
sprites[i].Location = new Point(i * 16, 8);
}
Game.ActiveForm.Controls.AddRange(sprites);
}
When it comes to running the code, only the last image is shown. While debugging the code, everything seems to be working as expected. I can also verify that the location is in fact being updated.
I have also tried adding the controls differently using the following code in the for loop (with no luck);
this.Controls.Add(sprites[i]);
I have had this problem many times, especially when I tried to create many GroupBoxes in a similar fashion. For the hours that I searched online as I tried to find a solution, nothing has ever fixed it.
You're only actually creating one instance of PictureBox:
PictureBox mainSprite = new PictureBox();
...
for(var i = 0; i < sprites.Length; i++)
{
sprites[i] = mainSprite;
Your array will have lots of reference to the same object. You should create a new PictureBox on each iteration of the loop:
for(var i = 0; i < sprites.Length; i++)
{
PictureBox mainSprite = new PictureBox();
mainSprite.Size = new Size(16, 16);
mainSprite.Image = img;
sprites[i] = mainSprite;
...
}

Categories