We Keep Coding

Set Specified region for Motion Detection

I need to specify a region on where motion detection will occur. What I am trying to do is count the number of vehicles that passes a certain region. Below is my code:
private static void ProcessFrame(Mat backgroundFrame, int threshold, int erodeIterations, int dilateIterations)
{
// Find difference between background (first) frame and current frame
CvInvoke.AbsDiff(backgroundFrame, rawFrame, diffFrame);
// Apply binary threshold to grayscale image (white pixel will mark difference)
CvInvoke.CvtColor(diffFrame, grayscaleDiffFrame, ColorConversion.Bgr2Gray);
CvInvoke.Threshold(grayscaleDiffFrame, binaryDiffFrame, threshold, 255, ThresholdType.Binary);
// Remove noise with opening operation (erosion followed by dilation)
CvInvoke.Erode(binaryDiffFrame, denoisedDiffFrame, null, new Point(-1, -1), erodeIterations, BorderType.Default, new MCvScalar(1));
CvInvoke.Dilate(denoisedDiffFrame, denoisedDiffFrame, null, new Point(-1, -1), dilateIterations, BorderType.Default, new MCvScalar(1));
rawFrame.CopyTo(finalFrame);
//Rectangle rec = new Rectangle(100, 100, 100, 100);
//finalFrame = crop_color_frame(rawFrame, rec);
var img = crop_color_frame(denoisedDiffFrame, rec);
DetectObject(denoisedDiffFrame, finalFrame);
}
static int vnum = 0;
private static void DetectObject(Mat detectionFrame, Mat displayFrame)
{
using (VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint())
{
// Build list of contours
CvInvoke.FindContours(detectionFrame, contours, null, RetrType.List, ChainApproxMethod.ChainApproxSimple);
// Selecting largest contour
if (contours.Size > 0)
{
double maxArea = 0;
int chosen = 0;
for (int i = 0; i < contours.Size; i++)
{
VectorOfPoint contour = contours[i];
double area = CvInvoke.ContourArea(contour);
if (area > maxArea)
{
maxArea = area;
chosen = i;
}
}
// Draw on a frame
MarkDetectedObject(displayFrame, contours[chosen], maxArea, contours.Size, maxArea);
}
}
}
private static void MarkDetectedObject(Mat frame, VectorOfPoint contour, double area, double contourSize, double maxArea)
{
// Getting minimal rectangle which contains the contour
Rectangle box = CvInvoke.BoundingRectangle(contour);
// Drawing contour and box around it
CvInvoke.Polylines(frame, contour, true, drawingColor);
CvInvoke.Rectangle(frame, box, drawingColor);
// Write information next to marked object
Point center = new Point(box.X + box.Width / 2, box.Y + box.Height / 2);
Point center2 = new Point(box.Width, box.Height);
var info = new string[] {
$"Area: {area}",
$"Position: {center.X}, {center.Y}"
};
Console.WriteLine($"X: {center.X} | Y: {center.Y} | Area: {area} | Count: {vnum} | Status: {vehicleState} | contour: {contour.Size}");
switch (vehicleState)
{
case VehicleState.Entering:
if(_startCount)
{
//if(((maxArea > 15000 && maxArea <= 20000) && center.Y <= 120) || ((maxArea >= 5000 && maxArea < 10000) && center.Y >= 150))
if(center.Y >= 100 && maxArea > 20000)
{
CountVehicle();
vehicleState = VehicleState.Exiting;
_startCount = false;
}
}
break;
case VehicleState.Exiting:
if (!_startCount)
{
//if(maxArea < 12000 && center.Y <= 120)
if(center.Y <= 130 && center.X <= 100 && maxArea <= 15000)
{
vehicleState = VehicleState.Entering;
_startCount = true;
}
}
break;
}
WriteMultilineText(frame, info, new Point(box.Right + 5, center.Y));
}
As of the moment, this code works on detecting vehicles but I am just using the
if(center.Y >= 100 && maxArea > 20000) condition to start counting the vehicles
the problem with that approach is, all movements in the frame are being monitored. That is why I need to set an specific region only.
Could you please show me how to do this?

You can set ROI for the input image
public static Mat crop_roi(Mat input_img)
{
Image<Gray, byte> img = input_img.ToImage<Gray, byte>();
double w = input_img.Width;
double h = input_img.Height;
Rectangle r = new Rectangle((int)(w * 0.2), (int)(h * 0.4), (int)(w * 0.6), (int)(h * 0.6));
Image<Gray, byte> output = img.Copy(r);
return output.Mat;
}
//USE
private static void DetectObject(Mat detectionFrame, Mat displayFrame)
{
using (VectorOfVectorOfPoint contours = new VectorOfVectorOfPoint())
{
//set roi to the frame
Mat roi = new Mat()
roi = set_roi(detectionFrame);
// Build list of contours
CvInvoke.FindContours(roi , contours, null, RetrType.List, ChainApproxMethod.ChainApproxSimple);
// Selecting largest contour
...
MarkDetectedObject(roi , contours[chosen], maxArea, contours.Size, maxArea);
}
Below is the image I draw the ROI in an image, you can adjust the ROI by changing the parameter in this line Rectangle r = new Rectangle((int)(w * 0.2), (int)(h * 0.4), (int)(w * 0.6), (int)(h * 0.6));

Detect passing of rectangle over yellow pixel

I have a query regarding the best approach to detect when a moving and potentially rotated rectangle passes over a yellow pixel of a Panel's background image.
I have a method which accepts an Image and a Point, and returns true if that point is that of a yellow pixel. I require this colour detection for the function of my game, which resets the car (player) if it drives over the yellow borders of the track. This method is shown below:
private Boolean isYellow(Image image, Point point)
{
Bitmap bitmap = new Bitmap(image);
Color color = bitmap.GetPixel(point.X, point.Y);
return (color.R > 220 && color.G > 220 && color.B < 200);
}
Previously, to detect if the player rectangle passes over yellow, I checked against the location of the rectangle, as provided by the X and Y values of the object. The issue with this is that the location is the top left corner of a horizontal rectangle, meaning the car can drive almost entirely off the track without detection occurring.
I'd like to fix this by checking all points covered by the rectangle. This is not as simple as it may seem as the rectangle is likely to be rotated. My drawing and movement logic is shown below:
public void draw(Graphics g)
{
int dx = rectangle.X + (rectangle.Height / 2);
int dy = rectangle.Y + (rectangle.Width / 2);
g.ScaleTransform(xScale, yScale);
g.TranslateTransform(dx, dy);
g.RotateTransform((float) ((180 * angle) / Math.PI));
g.TranslateTransform(-dx, -dy);
g.DrawImage(image, rectangle.X, rectangle.Y);
g.ResetTransform();
}
public void move(uRaceGame game, Panel panel)
{
double cos = Math.Cos(angle), sin = Math.Sin(angle);
int xLocation = 200;
int yLocation = 200;
xLocation = (int) Math.Floor(rectangle.X + (cos * game.moveDir * 60));
yLocation = (int) Math.Floor(rectangle.Y + (sin * game.moveDir * 60));
angle = (angle + (game.rotateDir * (Math.PI / 128))) % (Math.PI * 2);
if (xLocation * xScale > panel.Width - (rectangle.Width * cos) || yLocation * yScale > panel.Height - (rectangle.Width * sin) - 5 || xLocation * xScale < 0 || yLocation * yScale < 5) return;
rectangle.Location = new Point(xLocation, yLocation);
}
I tried but failed to create a method which translates the coords of the corner and figures out the middle of the rectangle, but this does not work, and the yellow detection fires in very obscure places:
public Point getCentre()
{
int cX = (int) (rectangle.X + ((rectangle.Width / 2) / xScale)), cY = (int) (rectangle.Y + ((rectangle.Height / 2) / yScale));
float tempX = (rectangle.X - cX), tempY = (rectangle.Y - cY);
double rX = (tempX * Math.Cos(angle)) - (tempY * Math.Sin(angle));
double rY = (tempX * Math.Sin(angle)) - (tempY * Math.Cos(angle));
return new Point((int) ((rX + cX) * xScale), (int) ((rY + cY) * yScale));
}
I'd really appreciate any suggestions on how to tackle this. I included the translation and yellow detection code in case I'm miles off in my attempt and someone else has a better idea.
Thank you very much.

There are two approaches that come to my mind:
You can create loops that go along the tilted sides of the car rectangle
Or you can copy the car to an untilted bitmap and loop over it normally.
Here is an example of the second approach.
It uses a LockBits method that detects Yellow with your code in a Bitmap.
And it prepares that bitmap by copying it from the original BackgroundImage un-rotated.
Here is the result, including a control Panel that shows the untilted Rectangle:
Here is the yellow finder function. It uses Lockbits for speed:
using System.Runtime.InteropServices;
using System.Drawing.Imaging;
public bool testForYellowBitmap(Bitmap bmp)
{
Size s1 = bmp.Size;
PixelFormat fmt = new PixelFormat();
fmt = bmp.PixelFormat;
Rectangle rect = new Rectangle(0, 0, s1.Width, s1.Height);
BitmapData bmp1Data = bmp.LockBits(rect, ImageLockMode.ReadOnly, fmt);
byte bpp1 = 4;
if (fmt == PixelFormat.Format24bppRgb) bpp1 = 3;
else if (fmt == PixelFormat.Format32bppArgb) bpp1 = 4; else return false; // throw!!
int size1 = bmp1Data.Stride * bmp1Data.Height;
byte[] data1 = new byte[size1];
System.Runtime.InteropServices.Marshal.Copy(bmp1Data.Scan0, data1, 0, size1);
for (int y = 0; y < s1.Height; y++)
{
for (int x = 0; x < s1.Width; x++)
{
Color c1;
int index1 = y * bmp1Data.Stride + x * bpp1;
if (bpp1 == 4)
c1 = Color.FromArgb(data1[index1 + 3], data1[index1 + 2],
data1[index1 + 1], data1[index1 + 0]);
else c1 = Color.FromArgb(255, data1[index1 + 2],
data1[index1 + 1], data1[index1 + 0]);
if (c1.R > 220 && c1.G > 220 && c1.B < 200)
{ bmp.UnlockBits(bmp1Data); return true; }
}
}
bmp.UnlockBits(bmp1Data);
return false;
}
I prepare the Bitmap to compare in the MouseMove. The variables w, h, w2, h2 hold the width, height and halves of that of the car's size. The source bitmap is in drawPanel1.BackgroundImage. The current angle is in a TrackBar tr_a.Value. For further control I also display the rotated car rectangle in White.
private void drawPanel1_MouseMove(object sender, MouseEventArgs e)
{
if (e.Button.HasFlag(MouseButtons.Left))
{
Size sz = drawPanel1.BackgroundImage.Size;
Rectangle rectSrc = new Rectangle(e.X - w2, e.Y - h2, w, h);
Rectangle rectTgt = new Rectangle(e.X - w, e.Y - h, 2 * w, 2 * h);
using (Graphics g = drawPanel1.CreateGraphics()) // start optional
{
g.TranslateTransform(e.X, e.Y);
g.RotateTransform(trb_a.Value);
g.TranslateTransform(-e.X, -e.Y);
drawPanel1.Refresh();
g.DrawRectangle(Pens.White, rectSrc);
}
using (Graphics g = drawPanel2.CreateGraphics())
{ // end optional
using (Bitmap bmp = new Bitmap(sz.Width, sz.Height))
using (Graphics g2 = Graphics.FromImage(bmp))
{
g2.TranslateTransform(e.X, e.Y);
g2.RotateTransform(-trb_a.Value);
g2.TranslateTransform(-e.X, -e.Y);
g2.DrawImage(drawPanel1.BackgroundImage, rectTgt, rectTgt,
GraphicsUnit.Pixel);
drawPanel2.Refresh();
g.DrawImage(bmp, rectSrc, rectSrc, GraphicsUnit.Pixel);
Text = testForYellowBitmap(bmp) ? "!!YELLOW!!" : "";
}
}
}
The first approach would use a similar LockBits method, but with loops inside that go along the rotated sides of the car rectangle, using floats wth the loop variables to calculate the x-coordinates. Those data should be prepared on each change of car size or angle. The code is a little longer but should be a bit faster, too.
The advantage if the second approach is that by using a ClippingRegion on the Graphics object one could check an arbitrary shape while the first method can be easily modified for concave polygons but not for curved shapes.
Here is the adapted version of the checking code for the first version:
public bool testForYellowBitmapTilt(Bitmap bmp, List<int> leftPts,
List<int> rightPts, Point topLeft)
{
Size s1 = bmp.Size;
PixelFormat fmt = new PixelFormat();
fmt = bmp.PixelFormat;
Rectangle rect = new Rectangle(0, 0, s1.Width, s1.Height);
BitmapData bmp1Data = bmp.LockBits(rect, ImageLockMode.ReadOnly, fmt);
byte bpp1 = 4;
if (fmt == PixelFormat.Format24bppRgb) bpp1 = 3;
else if (fmt == PixelFormat.Format32bppArgb) bpp1 = 4;
else return false; // or throw!!
if (leftPts.Count != rightPts.Count) return false; // or throw!!
int size1 = bmp1Data.Stride * bmp1Data.Height;
byte[] data1 = new byte[size1];
System.Runtime.InteropServices.Marshal.Copy(bmp1Data.Scan0, data1, 0, size1);
for (int y = 0; y < (leftPts.Count); y++)
{
for (int x = leftPts[y] + topLeft.X; x < rightPts[y] + topLeft.X; x++)
{
Color c1;
int index1 = (y + topLeft.Y) * bmp1Data.Stride + x * bpp1;
if (index1 > 0)
{
if (bpp1 == 4)
c1 = Color.FromArgb(data1[index1 + 3], data1[index1 + 2],
data1[index1 + 1], data1[index1 + 0]);
else c1 = Color.FromArgb(255, data1[index1 + 2],
data1[index1 + 1], data1[index1 + 0]);
if (c1.R > 220 && c1.G > 220 && c1.B < 200)
{ bmp.UnlockBits(bmp1Data); return true; }
}
}
}
bmp.UnlockBits(bmp1Data);
return false;
}
The left- and rightside coordinates are stored here:
List<int> leftPts = new List<int>();
List<int> rightPts = new List<int>();
Point top = Point.Empty;
void getOuterPoints(List<PointF> corners, out List<int> leftPts,
out List<int> rightPts, out Point top)
{
leftPts = new List<int>();
rightPts = new List<int>();
PointF left = corners.Select(x => x).OrderBy(x => x.X).First();
PointF right = corners.Select(x => x).OrderByDescending(x => x.X).First();
top = Point.Round(corners.Select(x => x).OrderBy(x => x.Y).First());
PointF bottom = corners.Select(x => x).OrderByDescending(x => x.Y).First();
int w1 = -(int)(top.X - left.X);
int w2 = -(int)(left.X - bottom.X );
int h1 = (int)(left.Y - top.Y);
int h2 = (int)(bottom.Y - left.Y);
float d1 = 1f * w1 / h1;
float d2 = 1f * w2 / h2;
for (int y = 0; y < h1; y++) leftPts.Add( (int)(y * d1) );
for (int y = 0; y < h2; y++) leftPts.Add( (int)(y * d2 + w1));
for (int y = 0; y < h2; y++) rightPts.Add( (int)(y * d2));
for (int y = 0; y < h1; y++) rightPts.Add( (int)(y * d1 + w2));
}
You need to feed in the four corners as a List<PointF> in any order; the top can be anything, it will be set in the method. The coodinates are relative to the car, so they don't change when the car moves..

Get coordinates points between x and y

I have a jpeg image in a picturebox, on page load i am drawing the rectangle based on the x(150) and y(440) coordinates. now when i mousemove on the picturebox i need to identify the rectangle by their coordinates and highlight the image. for example see the below image ..
lets take the first rectangle, on mouse move any points inside the rectangle i need to perform some actions.. how to find the coordinates between these x and y for the rectangle?..

A rectangle has 4 Points (edges):
Left
Top
Right
Bottom
If your mouse coordinates (MouseEventArgs properties) are between them, the mouse pointer is in the rectangle.
Are the mouse coordinates greater than right or bottom or lower than left / top, your mouse is outside the rectangle.
Taking #samgak`s Comment:
if(
(point.x >= rectangle.min_x) && (point.x <= rectangle.max_x) &&
(point.y >= rectangle.min_y) && (point.y <= rectangle.max_y)) {
//do something
}
and replacing point with e is exactly what you want.
Maybe the following link will help to understand:
How to check if a point is inside a rectangle

Provided images are the same size and assuming it is stored in variable imageSize (of type System.Drawing.Size) then:
Size imageSize = new Size(...) // define the size here
...
int row = point.y / imageSize.height;
int col = point.x / imageSize.width;
var rect = new Rectangle(col * imageSize.Width, row * imageSize.Height, imageSize.Width, imageSize.Height);
You can then use rect to draw you frame around the image (you may want to inflate the rectangle by a couple of pixels)

Hi Samgak /Clijsters,
I have completed my functionality
// page level declaration
private Rectangle SelectedRect;
public List<Rectangle> listRec = new List<Rectangle>();
// on page load add all the rectangle in the rectanglelist.
private void Highlightimage_Load(object sender, EventArgs e)
{
for (int i = 0; i < table.Rows.Count; i++)
{
int x = Convert.ToInt32(table.Rows[i][0]);
int y = Convert.ToInt32(table.Rows[i][1]);
int width = Convert.ToInt32(table.Rows[i][2]);
int height = Convert.ToInt32(table.Rows[i][3]);
SelectedRect.Size = new Size(width, height);
SelectedRect.X = x;
SelectedRect.Y = y;
listRec.Add(SelectedRect);
}
}
// draw the rectangle
private void pictureBox1_Paint(object sender, PaintEventArgs e)
{ Graphics g = e.Graphics;
foreach (Rectangle rec in listRec)
{
Pen p = new Pen(Color.Red);
g.DrawRectangle(p, rec);
}
}
private Rectangle MakeRectangle(int x0, int y0, int x1, int y1)
{
return new Rectangle(
Math.Min(x0, x1),
Math.Min(y0, y1),
Math.Abs(x0 - x1),
Math.Abs(y0 - y1));
}
//finally on mouse move checking the condition
private void pictureBox1_MouseMove(object sender, MouseEventArgs e)
{
foreach (Rectangle rec in listRec)
{
SelectedRect = MakeRectangle(rec.Left, rec.Top, rec.Right, rec.Bottom);
if (
(e.X >= SelectedRect.Left) && (e.X <= SelectedRect.Right) &&
(e.Y >= SelectedRect.Top) && (e.Y <= SelectedRect.Bottom))
{
MessageBox.Show("test");
}
}
}
Also refered this link
Drawing Multiple Rectangles c#
I thought this will help some one.
Thanks
Dev

How do I fill everything over a straight line and under a curve?

I am using the Charts component in Windows Forms.
I create a straight line using
chart1.Series["Grenzwert"].Points.Add(new DataPoint(0, y));
chart1.Series["Grenzwert"].Points.Add(new DataPoint(maxwidth, y));
Also I plot a a series of points connected by a line, let's call it curve.
How do I show everything over straight line and under curve filled?
Column fills the whole area, not just above straight line.
Example:

This is late and not really short but imo it is the best way to color areas in a chart.
The Lines and also the Spline charttypes can be very precisely colored by coding the Paint event with the right data. The necessary pixel values can be obtained by the axis function ValueToPixelPosition. See here for another example!
The following code is a little longer because we need to add certain points at the start and end of both the chart and each colored area. Other than that it is very straight forward: Create GraphicsPaths by adding the pixel coordinates with AddLines and fill the GraphicsPaths in the Paint event.
For testing and for fun I have added a movable HorizontalLineAnnotation, so I can see how the areas vary when I drag it up and down..:
The Paint event is rather simple; it refers to a HorizontalLineAnnotation hl :
private void chart1_Paint(object sender, PaintEventArgs e)
{
double limit = hl.Y; // get the limit value
hl.X = 0; // reset the x value of the annotation
List<GraphicsPath> paths = getPaths(chart1.ChartAreas[0], chart1.Series[0], limit);
using (SolidBrush brush = new SolidBrush(Color.FromArgb(127, Color.Red)))
foreach (GraphicsPath gp in paths)
{ e.Graphics.FillPath(brush, gp); gp.Dispose(); }
}
The code to get the paths is obviously way too long for comfort..:
List<GraphicsPath> getPaths(ChartArea ca, Series ser, double limit)
{
List<GraphicsPath> paths = new List<GraphicsPath>();
List<PointF> points = new List<PointF>();
int first = 0;
float limitPix = (float)ca.AxisY.ValueToPixelPosition(limit);
for (int i = 0; i < ser.Points.Count; i++)
{
if ((ser.Points[i].YValues[0] > limit) && (i < ser.Points.Count - 1))
{
if (points.Count == 0) first = i; // remember group start
// insert very first point:
if (i == 0) points.Insert(0, new PointF(
(float)ca.AxisX.ValueToPixelPosition(ser.Points[0].XValue), limitPix));
points.Add( pointfFromDataPoint(ser.Points[i], ca)); // the regular points
}
else
{
if (points.Count > 0)
{
if (first > 0) points.Insert(0, median(
pointfFromDataPoint(ser.Points[first - 1], ca),
pointfFromDataPoint(ser.Points[first], ca), limitPix));
if (i == ser.Points.Count - 1)
{
if ((ser.Points[i].YValues[0] > limit))
points.Add(pointfFromDataPoint(ser.Points[i], ca));
points.Add(new PointF(
(float)ca.AxisX.ValueToPixelPosition(ser.Points[i].XValue), limitPix));
}
else
points.Add(median(pointfFromDataPoint(ser.Points[i - 1], ca),
pointfFromDataPoint(ser.Points[i], ca), limitPix));
GraphicsPath gp = new GraphicsPath();
gp.FillMode = FillMode.Winding;
gp.AddLines(points.ToArray());
gp.CloseFigure();
paths.Add(gp);
points.Clear();
}
}
}
return paths;
}
It uses two helper functions:
PointF pointfFromDataPoint(DataPoint dp, ChartArea ca)
{
return new PointF( (float)ca.AxisX.ValueToPixelPosition(dp.XValue),
(float)ca.AxisY.ValueToPixelPosition(dp.YValues[0]));
}
PointF median(PointF p1, PointF p2, float y0)
{
float x0 = p2.X - (p2.X - p1.X) * (p2.Y - y0) / (p2.Y - p1.Y);
return new PointF(x0, y0);
}
The HorizontalLineAnnotation is set up like this:
hl = new HorizontalLineAnnotation();
hl.AllowMoving = true;
hl.LineColor = Color.OrangeRed;
hl.LineWidth = 1;
hl.AnchorDataPoint = S1.Points[1];
hl.X = 0;
hl.Y = 0; // or some other starting value..
hl.Width = 100; // percent of chart..
hl.ClipToChartArea = chart1.ChartAreas[0].Name; // ..but clipped
chart1.Annotations.Add(hl);

I have an idea that use SeriesChartType.Range as follow.
private void UpdateChart(float straight_line, List<DataPoint> curve)
{
float y = straight_line; // YValue of the straight line
var list = curve.ToList(); // Clone the curve
int count = list.Count - 2;
for (int i = 0; i < count; i++) // Calculate intersection point between the straight line and a line between (x0,y0) and (x1,y1)
{
double x0 = list[i + 0].XValue;
double y0 = list[i + 0].YValues[0];
double x1 = list[i + 1].XValue;
double y1 = list[i + 1].YValues[0];
if ((y0 > y && y1 < y) || (y0 < y && y1 > y))
{
double x = (y - y0) * (x1 - x0) / (y1 - y0) + x0;
list.Add(new DataPoint(x, y));
}
}
list.Sort((a, b) => Math.Sign(a.XValue - b.XValue));
chart1.Series[0].Points.Clear();
chart1.Series[0].ChartType = SeriesChartType.Range;
chart1.Series[0].Color = Color.Red;
chart1.Series[0].BorderColor = Color.Cyan;
chart1.ChartAreas[0].AxisX.Minimum = 0;
chart1.ChartAreas[0].AxisX.Interval = 1;
for (int i = 0; i < list.Count; i++)
{
double xx = list[i].XValue;
double yy = list[i].YValues[0];
if (yy > y)
{
chart1.Series[0].Points.AddXY(xx, y, yy);
}
else
{
chart1.Series[0].Points.AddXY(xx, yy, yy);
}
}
chart1.ChartAreas[0].AxisY.StripLines.Add(new StripLine { IntervalOffset = y, Interval = 0, BorderColor = Color.Orange, BorderWidth = 2 });
}
As in the below drawing to judge whether the straight line and a line between (x0,y0) and (x1,y1) intersect, case 1 is (y0 < y && y1 > y) and case 2 is (y0 > y && y1 < y) . In case 1 and case 2, they intersect each other. In case 3 and case 4, they don't intersect each other.

You can do this as follows.
Set the column fill like you did before. Everything will be red.
Create a new column graph on the same chart.
Set its values to the same as your jagged line, but capped at the y value of the straight line you already have.
Set the fill colour for the columns to white. This will block out the red fill for any areas not between the lines.

How to find rectangle of difference between two images

I have two images the same size. What is the best way to find the rectangle in which they differ. Obviously I could go through the image 4 times in different directions, but i'm wondering if there's an easier way.
Example:

A naive approach would be to start at the origin, and work line by line, column by column. Compare each pixel, keeping note of the topmost, leftmost, rightmost, and bottommost, from which you can calculate your rectangle. There will be cases where this single pass approach would be faster (i.e. where there is a very small differing area)

If you want a single rectangle, use int.MaxValue for the threshold.
var diff = new ImageDiffUtil(filename1, filename2);
var diffRectangles = diff.GetDiffRectangles(int.MaxValue);
If you want multiple rectangles, use a smaller threshold.
var diff = new ImageDiffUtil(filename1, filename2);
var diffRectangles = diff.GetDiffRectangles(8);
ImageDiffUtil.cs
using System;
using System.Collections.Generic;
using System.Drawing;
using System.Linq;
namespace diff_images
{
public class ImageDiffUtil
{
Bitmap image1;
Bitmap image2;
public ImageDiffUtil(string filename1, string filename2)
{
image1 = Image.FromFile(filename1) as Bitmap;
image2 = Image.FromFile(filename2) as Bitmap;
}
public IList<Point> GetDiffPixels()
{
var widthRange = Enumerable.Range(0, image1.Width);
var heightRange = Enumerable.Range(0, image1.Height);
var result = widthRange
.SelectMany(x => heightRange, (x, y) => new Point(x, y))
.Select(point => new
{
Point = point,
Pixel1 = image1.GetPixel(point.X, point.Y),
Pixel2 = image2.GetPixel(point.X, point.Y)
})
.Where(pair => pair.Pixel1 != pair.Pixel2)
.Select(pair => pair.Point)
.ToList();
return result;
}
public IEnumerable<Rectangle> GetDiffRectangles(double distanceThreshold)
{
var result = new List<Rectangle>();
var differentPixels = GetDiffPixels();
while (differentPixels.Count > 0)
{
var cluster = new List<Point>()
{
differentPixels[0]
};
differentPixels.RemoveAt(0);
while (true)
{
var left = cluster.Min(p => p.X);
var right = cluster.Max(p => p.X);
var top = cluster.Min(p => p.Y);
var bottom = cluster.Max(p => p.Y);
var width = Math.Max(right - left, 1);
var height = Math.Max(bottom - top, 1);
var clusterBox = new Rectangle(left, top, width, height);
var proximal = differentPixels
.Where(point => GetDistance(clusterBox, point) <= distanceThreshold)
.ToList();
proximal.ForEach(point => differentPixels.Remove(point));
if (proximal.Count == 0)
{
result.Add(clusterBox);
break;
}
else
{
cluster.AddRange(proximal);
}
};
}
return result;
}
static double GetDistance(Rectangle rect, Point p)
{
var dx = Math.Max(rect.Left - p.X, 0);
dx = Math.Max(dx, p.X - rect.Right);
var dy = Math.Max(rect.Top - p.Y, 0);
dy = Math.Max(dy, p.Y - rect.Bottom);
return Math.Sqrt(dx * dx + dy * dy);
}
}
}
Form1.cs
using System.Drawing;
using System.Linq;
using System.Windows.Forms;
namespace diff_images
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
var filename1 = #"Gelatin1.PNG";
var filename2 = #"Gelatin2.PNG";
var diff = new ImageDiffUtil(filename1, filename2);
var diffRectangles = diff.GetDiffRectangles(8);
var img3 = Image.FromFile(filename2);
Pen redPen = new Pen(Color.Red, 1);
var padding = 3;
using (var graphics = Graphics.FromImage(img3))
{
diffRectangles
.ToList()
.ForEach(rect =>
{
var largerRect = new Rectangle(rect.X - padding, rect.Y - padding, rect.Width + padding * 2, rect.Height + padding * 2);
graphics.DrawRectangle(redPen, largerRect);
});
}
var pb1 = new PictureBox()
{
Image = Image.FromFile(filename1),
Left = 8,
Top = 8,
SizeMode = PictureBoxSizeMode.AutoSize
};
var pb2 = new PictureBox()
{
Image = Image.FromFile(filename2),
Left = pb1.Left + pb1.Width + 16,
Top = 8,
SizeMode = PictureBoxSizeMode.AutoSize
};
var pb3 = new PictureBox()
{
Image = img3,
Left = pb2.Left + pb2.Width + 16,
Top = 8,
SizeMode = PictureBoxSizeMode.AutoSize
};
Controls.Add(pb1);
Controls.Add(pb2);
Controls.Add(pb3);
}
}
}

Image processing like this is expensive, there are a lot of bits to look at. In real applications, you almost always need to filter the image to get rid of artifacts induced by imperfect image captures.
A common library used for this kind of bit whacking is OpenCV, it takes advantage of dedicated CPU instructions available to make this fast. There are several .NET wrappers available for it, Emgu is one of them.

I don't think there is an easier way.
In fact doing this will just be a (very) few lines of code, so unless you find a library that does that for you directly you won't find a shorter way.

Idea:
Consider an image as a 2D Array with each Array element as a pixel of the image. Hence, I would say Image Differencing is nothing but 2D Array Differencing.
Idea is to just scan through the array elements width-wise and find the place where there is a difference in pixel values. If example [x, y] co-ordinates of both 2D Array are different then our rectangle finding logic starts. Later on the rectangles would be used to patch the last updated Frame Buffer.
We need to scan through the boundaries of the rectangles for differences and if any difference is found in the boundary of rectangle, then the boundary will be increased width-wise or height-wise depending upon the type of scan made.
Consider I scanned width-wise of 2D Array and I found a location where there exist a co-ordinate which is different in both the 2D Arrays, I will create a rectangle with the starting position as [x-1, y-1] and with the width and height as 2 and 2 respectively. Please note that width and height refers to the number of pixels.
eg: Rect Info:
X = 20
Y = 35
W = 26
H = 23
i.e width of the rectangle starts from co-ordinate [20, 35] -> [20, 35 + 26 - 1]. Maybe when you find the code you may be able to understand it better.
Also there are possibilities that there are smaller rectangles inside a bigger rectangle you have found, thus we need to remove the smaller rectangles from our reference because they mean nothing to us except that they occupu my precious space !!
The above logic would be helpful in the case of VNC Server Implementation where there would be a need of rectangles that denotes differences in the image that is currently taken. Those rectangles could be sent in the network to the VNC Client which can patch the rectangles in the local copy of Frame Buffer it possesses thereby displaying it on the VNC Client Display Board.
P.S.:
I will be attaching the code in which I implemented my own algorithm. I would request viewers to comment for any mistakes or performance tuning. I would also request viewers to comment about any better algorithm that would make life simpler.
Code:
Class Rect:
public class Rect {
public int x; // Array Index
public int y; // Array Index
public int w; // Number of hops along the Horizontal
public int h; // Number of hops along the Vertical
#Override
public boolean equals(Object obj) {
Rect rect = (Rect) obj;
if(rect.x == this.x && rect.y == this.y && rect.w == this.w && rect.h == this.h) {
return true;
}
return false;
}
}
Class Image Difference:
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.util.LinkedList;
import javax.imageio.ImageIO;
public class ImageDifference {
long start = 0, end = 0;
public LinkedList<Rect> differenceImage(int[][] baseFrame, int[][] screenShot, int xOffset, int yOffset, int width, int height) {
// Code starts here
int xRover = 0;
int yRover = 0;
int index = 0;
int limit = 0;
int rover = 0;
boolean isRectChanged = false;
boolean shouldSkip = false;
LinkedList<Rect> rectangles = new LinkedList<Rect>();
Rect rect = null;
start = System.nanoTime();
// xRover - Rovers over the height of 2D Array
// yRover - Rovers over the width of 2D Array
int verticalLimit = xOffset + height;
int horizontalLimit = yOffset + width;
for(xRover = xOffset; xRover < verticalLimit; xRover += 1) {
for(yRover = yOffset; yRover < horizontalLimit; yRover += 1) {
if(baseFrame[xRover][yRover] != screenShot[xRover][yRover]) {
// Skip over the already processed Rectangles
for(Rect itrRect : rectangles) {
if(( (xRover < itrRect.x + itrRect.h) && (xRover >= itrRect.x) ) && ( (yRover < itrRect.y + itrRect.w) && (yRover >= itrRect.y) )) {
shouldSkip = true;
yRover = itrRect.y + itrRect.w - 1;
break;
} // End if(( (xRover < itrRect.x + itrRect.h) && (xRover >= itrRect.x) ) && ( (yRover < itrRect.y + itrRect.w) && (yRover >= itrRect.y) ))
} // End for(Rect itrRect : rectangles)
if(shouldSkip) {
shouldSkip = false;
// Need to come out of the if condition as below that is why "continue" has been provided
// if(( (xRover <= (itrRect.x + itrRect.h)) && (xRover >= itrRect.x) ) && ( (yRover <= (itrRect.y + itrRect.w)) && (yRover >= itrRect.y) ))
continue;
} // End if(shouldSkip)
rect = new Rect();
rect.x = ((xRover - 1) < xOffset) ? xOffset : (xRover - 1);
rect.y = ((yRover - 1) < yOffset) ? yOffset : (yRover - 1);
rect.w = 2;
rect.h = 2;
/* Boolean variable used to re-scan the currently found rectangle
for any change due to previous scanning of boundaries */
isRectChanged = true;
while(isRectChanged) {
isRectChanged = false;
index = 0;
/* I */
/* Scanning of left-side boundary of rectangle */
index = rect.x;
limit = rect.x + rect.h;
while(index < limit && rect.y != yOffset) {
if(baseFrame[index][rect.y] != screenShot[index][rect.y]) {
isRectChanged = true;
rect.y = rect.y - 1;
rect.w = rect.w + 1;
index = rect.x;
continue;
} // End if(baseFrame[index][rect.y] != screenShot[index][rect.y])
index = index + 1;;
} // End while(index < limit && rect.y != yOffset)
/* II */
/* Scanning of bottom boundary of rectangle */
index = rect.y;
limit = rect.y + rect.w;
while( (index < limit) && (rect.x + rect.h != verticalLimit) ) {
rover = rect.x + rect.h - 1;
if(baseFrame[rover][index] != screenShot[rover][index]) {
isRectChanged = true;
rect.h = rect.h + 1;
index = rect.y;
continue;
} // End if(baseFrame[rover][index] != screenShot[rover][index])
index = index + 1;
} // End while( (index < limit) && (rect.x + rect.h != verticalLimit) )
/* III */
/* Scanning of right-side boundary of rectangle */
index = rect.x;
limit = rect.x + rect.h;
while( (index < limit) && (rect.y + rect.w != horizontalLimit) ) {
rover = rect.y + rect.w - 1;
if(baseFrame[index][rover] != screenShot[index][rover]) {
isRectChanged = true;
rect.w = rect.w + 1;
index = rect.x;
continue;
} // End if(baseFrame[index][rover] != screenShot[index][rover])
index = index + 1;
} // End while( (index < limit) && (rect.y + rect.w != horizontalLimit) )
} // while(isRectChanged)
// Remove those rectangles that come inside "rect" rectangle.
int idx = 0;
while(idx < rectangles.size()) {
Rect r = rectangles.get(idx);
if( ( (rect.x <= r.x) && (rect.x + rect.h >= r.x + r.h) ) && ( (rect.y <= r.y) && (rect.y + rect.w >= r.y + r.w) ) ) {
rectangles.remove(r);
} else {
idx += 1;
} // End if( ( (rect.x <= r.x) && (rect.x + rect.h >= r.x + r.h) ) && ( (rect.y <= r.y) && (rect.y + rect.w >= r.y + r.w) ) )
} // End while(idx < rectangles.size())
// Giving a head start to the yRover when a rectangle is found
rectangles.addFirst(rect);
yRover = rect.y + rect.w - 1;
rect = null;
} // End if(baseFrame[xRover][yRover] != screenShot[xRover][yRover])
} // End for(yRover = yOffset; yRover < horizontalLimit; yRover += 1)
} // End for(xRover = xOffset; xRover < verticalLimit; xRover += 1)
end = System.nanoTime();
return rectangles;
}
public static void main(String[] args) throws IOException {
LinkedList<Rect> rectangles = null;
// Buffering the Base image and Screen Shot Image
BufferedImage screenShotImg = ImageIO.read(new File("screenShotImg.png"));
BufferedImage baseImg = ImageIO.read(new File("baseImg.png"));
int width = baseImg.getWidth();
int height = baseImg.getHeight();
int xOffset = 0;
int yOffset = 0;
int length = baseImg.getWidth() * baseImg.getHeight();
// Creating 2 Two Dimensional Arrays for Image Processing
int[][] baseFrame = new int[height][width];
int[][] screenShot = new int[height][width];
// Creating 2 Single Dimensional Arrays to retrieve the Pixel Values
int[] baseImgPix = new int[length];
int[] screenShotImgPix = new int[length];
// Reading the Pixels from the Buffered Image
baseImg.getRGB(0, 0, baseImg.getWidth(), baseImg.getHeight(), baseImgPix, 0, baseImg.getWidth());
screenShotImg.getRGB(0, 0, screenShotImg.getWidth(), screenShotImg.getHeight(), screenShotImgPix, 0, screenShotImg.getWidth());
// Transporting the Single Dimensional Arrays to Two Dimensional Array
long start = System.nanoTime();
for(int row = 0; row < height; row++) {
System.arraycopy(baseImgPix, (row * width), baseFrame[row], 0, width);
System.arraycopy(screenShotImgPix, (row * width), screenShot[row], 0, width);
}
long end = System.nanoTime();
System.out.println("Array Copy : " + ((double)(end - start) / 1000000));
// Finding Differences between the Base Image and ScreenShot Image
ImageDifference imDiff = new ImageDifference();
rectangles = imDiff.differenceImage(baseFrame, screenShot, xOffset, yOffset, width, height);
// Displaying the rectangles found
int index = 0;
for(Rect rect : rectangles) {
System.out.println("\nRect info : " + (++index));
System.out.println("X : " + rect.x);
System.out.println("Y : " + rect.y);
System.out.println("W : " + rect.w);
System.out.println("H : " + rect.h);
// Creating Bounding Box
for(int i = rect.y; i < rect.y + rect.w; i++) {
screenShotImgPix[ ( rect.x * width) + i ] = 0xFFFF0000;
screenShotImgPix[ ((rect.x + rect.h - 1) * width) + i ] = 0xFFFF0000;
}
for(int j = rect.x; j < rect.x + rect.h; j++) {
screenShotImgPix[ (j * width) + rect.y ] = 0xFFFF0000;
screenShotImgPix[ (j * width) + (rect.y + rect.w - 1) ] = 0xFFFF0000;
}
}
// Creating the Resultant Image
screenShotImg.setRGB(0, 0, width, height, screenShotImgPix, 0, width);
ImageIO.write(screenShotImg, "PNG", new File("result.png"));
double d = ((double)(imDiff.end - imDiff.start) / 1000000);
System.out.println("\nTotal Time : " + d + " ms" + " Array Copy : " + ((double)(end - start) / 1000000) + " ms");
}
}
Description:
There would be a function named
public LinkedList<Rect> differenceImage(int[][] baseFrame, int[][] screenShot, int width, int height)
which does the job of finding differences in the images and return a linkedlist of objects. The objects are nothing but the rectangles.
There is main function which does the job of testing the algorithm.
There are 2 sample images passed into the code in main function, they are nothing but the "baseFrame" and "screenShot" thereby creating the resultant image named "result".
I don't possess the desired reputation to post the resultant image which would be very interesting.
There is a blog which would provide the output
Image Difference

I don't think there can be anything better than exhaustively searching from each side in turn for the first point of difference in that direction. Unless, that is, you know a fact that in some way constrains the set of points of difference.

So here comes the easy way if you know how to use Lockbit :)
Bitmap originalBMP = new Bitmap(pictureBox1.ImageLocation);
Bitmap changedBMP = new Bitmap(pictureBox2.ImageLocation);
int width = Math.Min(originalBMP.Width, changedBMP.Width),
height = Math.Min(originalBMP.Height, changedBMP.Height),
xMin = int.MaxValue,
xMax = int.MinValue,
yMin = int.MaxValue,
yMax = int.MinValue;
var originalLock = originalBMP.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadWrite, originalBMP.PixelFormat);
var changedLock = changedBMP.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadWrite, changedBMP.PixelFormat);
for (int y = 0; y < height; y++)
{
for (int x = 0; x < width; x++)
{
//generate the address of the colour pixel
int pixelIdxOrg = y * originalLock.Stride + (x * 4);
int pixelIdxCh = y * changedLock.Stride + (x * 4);
if (( Marshal.ReadByte(originalLock.Scan0, pixelIdxOrg + 2)!= Marshal.ReadByte(changedLock.Scan0, pixelIdxCh + 2))
|| (Marshal.ReadByte(originalLock.Scan0, pixelIdxOrg + 1) != Marshal.ReadByte(changedLock.Scan0, pixelIdxCh + 1))
|| (Marshal.ReadByte(originalLock.Scan0, pixelIdxOrg) != Marshal.ReadByte(changedLock.Scan0, pixelIdxCh))
)
{
xMin = Math.Min(xMin, x);
xMax = Math.Max(xMax, x);
yMin = Math.Min(yMin, y);
yMax = Math.Max(yMax, y);
}
}
}
originalBMP.UnlockBits(originalLock);
changedBMP.UnlockBits(changedLock);
var result = changedBMP.Clone(new Rectangle(xMin, yMin, xMax - xMin, yMax - yMin), changedBMP.PixelFormat);
pictureBox3.Image = result;
disclaim it looks like your 2 pictures contains more differences than we can see with the naked eye so the result will be wider than you expect but you can add a tolerance so it wil fit even if the rest isn't 100% identical
to speed things up you will maybe able to us Parallel.For but do it only for the outer loop