I have two images the same size. What is the best way to find the rectangle in which they differ. Obviously I could go through the image 4 times in different directions, but i'm wondering if there's an easier way.
Example:
A naive approach would be to start at the origin, and work line by line, column by column. Compare each pixel, keeping note of the topmost, leftmost, rightmost, and bottommost, from which you can calculate your rectangle. There will be cases where this single pass approach would be faster (i.e. where there is a very small differing area)
If you want a single rectangle, use int.MaxValue for the threshold.
var diff = new ImageDiffUtil(filename1, filename2);
var diffRectangles = diff.GetDiffRectangles(int.MaxValue);
If you want multiple rectangles, use a smaller threshold.
var diff = new ImageDiffUtil(filename1, filename2);
var diffRectangles = diff.GetDiffRectangles(8);
ImageDiffUtil.cs
using System;
using System.Collections.Generic;
using System.Drawing;
using System.Linq;
namespace diff_images
{
public class ImageDiffUtil
{
Bitmap image1;
Bitmap image2;
public ImageDiffUtil(string filename1, string filename2)
{
image1 = Image.FromFile(filename1) as Bitmap;
image2 = Image.FromFile(filename2) as Bitmap;
}
public IList<Point> GetDiffPixels()
{
var widthRange = Enumerable.Range(0, image1.Width);
var heightRange = Enumerable.Range(0, image1.Height);
var result = widthRange
.SelectMany(x => heightRange, (x, y) => new Point(x, y))
.Select(point => new
{
Point = point,
Pixel1 = image1.GetPixel(point.X, point.Y),
Pixel2 = image2.GetPixel(point.X, point.Y)
})
.Where(pair => pair.Pixel1 != pair.Pixel2)
.Select(pair => pair.Point)
.ToList();
return result;
}
public IEnumerable<Rectangle> GetDiffRectangles(double distanceThreshold)
{
var result = new List<Rectangle>();
var differentPixels = GetDiffPixels();
while (differentPixels.Count > 0)
{
var cluster = new List<Point>()
{
differentPixels[0]
};
differentPixels.RemoveAt(0);
while (true)
{
var left = cluster.Min(p => p.X);
var right = cluster.Max(p => p.X);
var top = cluster.Min(p => p.Y);
var bottom = cluster.Max(p => p.Y);
var width = Math.Max(right - left, 1);
var height = Math.Max(bottom - top, 1);
var clusterBox = new Rectangle(left, top, width, height);
var proximal = differentPixels
.Where(point => GetDistance(clusterBox, point) <= distanceThreshold)
.ToList();
proximal.ForEach(point => differentPixels.Remove(point));
if (proximal.Count == 0)
{
result.Add(clusterBox);
break;
}
else
{
cluster.AddRange(proximal);
}
};
}
return result;
}
static double GetDistance(Rectangle rect, Point p)
{
var dx = Math.Max(rect.Left - p.X, 0);
dx = Math.Max(dx, p.X - rect.Right);
var dy = Math.Max(rect.Top - p.Y, 0);
dy = Math.Max(dy, p.Y - rect.Bottom);
return Math.Sqrt(dx * dx + dy * dy);
}
}
}
Form1.cs
using System.Drawing;
using System.Linq;
using System.Windows.Forms;
namespace diff_images
{
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
var filename1 = #"Gelatin1.PNG";
var filename2 = #"Gelatin2.PNG";
var diff = new ImageDiffUtil(filename1, filename2);
var diffRectangles = diff.GetDiffRectangles(8);
var img3 = Image.FromFile(filename2);
Pen redPen = new Pen(Color.Red, 1);
var padding = 3;
using (var graphics = Graphics.FromImage(img3))
{
diffRectangles
.ToList()
.ForEach(rect =>
{
var largerRect = new Rectangle(rect.X - padding, rect.Y - padding, rect.Width + padding * 2, rect.Height + padding * 2);
graphics.DrawRectangle(redPen, largerRect);
});
}
var pb1 = new PictureBox()
{
Image = Image.FromFile(filename1),
Left = 8,
Top = 8,
SizeMode = PictureBoxSizeMode.AutoSize
};
var pb2 = new PictureBox()
{
Image = Image.FromFile(filename2),
Left = pb1.Left + pb1.Width + 16,
Top = 8,
SizeMode = PictureBoxSizeMode.AutoSize
};
var pb3 = new PictureBox()
{
Image = img3,
Left = pb2.Left + pb2.Width + 16,
Top = 8,
SizeMode = PictureBoxSizeMode.AutoSize
};
Controls.Add(pb1);
Controls.Add(pb2);
Controls.Add(pb3);
}
}
}
Image processing like this is expensive, there are a lot of bits to look at. In real applications, you almost always need to filter the image to get rid of artifacts induced by imperfect image captures.
A common library used for this kind of bit whacking is OpenCV, it takes advantage of dedicated CPU instructions available to make this fast. There are several .NET wrappers available for it, Emgu is one of them.
I don't think there is an easier way.
In fact doing this will just be a (very) few lines of code, so unless you find a library that does that for you directly you won't find a shorter way.
Idea:
Consider an image as a 2D Array with each Array element as a pixel of the image. Hence, I would say Image Differencing is nothing but 2D Array Differencing.
Idea is to just scan through the array elements width-wise and find the place where there is a difference in pixel values. If example [x, y] co-ordinates of both 2D Array are different then our rectangle finding logic starts. Later on the rectangles would be used to patch the last updated Frame Buffer.
We need to scan through the boundaries of the rectangles for differences and if any difference is found in the boundary of rectangle, then the boundary will be increased width-wise or height-wise depending upon the type of scan made.
Consider I scanned width-wise of 2D Array and I found a location where there exist a co-ordinate which is different in both the 2D Arrays, I will create a rectangle with the starting position as [x-1, y-1] and with the width and height as 2 and 2 respectively. Please note that width and height refers to the number of pixels.
eg: Rect Info:
X = 20
Y = 35
W = 26
H = 23
i.e width of the rectangle starts from co-ordinate [20, 35] -> [20, 35 + 26 - 1]. Maybe when you find the code you may be able to understand it better.
Also there are possibilities that there are smaller rectangles inside a bigger rectangle you have found, thus we need to remove the smaller rectangles from our reference because they mean nothing to us except that they occupu my precious space !!
The above logic would be helpful in the case of VNC Server Implementation where there would be a need of rectangles that denotes differences in the image that is currently taken. Those rectangles could be sent in the network to the VNC Client which can patch the rectangles in the local copy of Frame Buffer it possesses thereby displaying it on the VNC Client Display Board.
P.S.:
I will be attaching the code in which I implemented my own algorithm. I would request viewers to comment for any mistakes or performance tuning. I would also request viewers to comment about any better algorithm that would make life simpler.
Code:
Class Rect:
public class Rect {
public int x; // Array Index
public int y; // Array Index
public int w; // Number of hops along the Horizontal
public int h; // Number of hops along the Vertical
#Override
public boolean equals(Object obj) {
Rect rect = (Rect) obj;
if(rect.x == this.x && rect.y == this.y && rect.w == this.w && rect.h == this.h) {
return true;
}
return false;
}
}
Class Image Difference:
import java.awt.image.BufferedImage;
import java.io.File;
import java.io.IOException;
import java.util.LinkedList;
import javax.imageio.ImageIO;
public class ImageDifference {
long start = 0, end = 0;
public LinkedList<Rect> differenceImage(int[][] baseFrame, int[][] screenShot, int xOffset, int yOffset, int width, int height) {
// Code starts here
int xRover = 0;
int yRover = 0;
int index = 0;
int limit = 0;
int rover = 0;
boolean isRectChanged = false;
boolean shouldSkip = false;
LinkedList<Rect> rectangles = new LinkedList<Rect>();
Rect rect = null;
start = System.nanoTime();
// xRover - Rovers over the height of 2D Array
// yRover - Rovers over the width of 2D Array
int verticalLimit = xOffset + height;
int horizontalLimit = yOffset + width;
for(xRover = xOffset; xRover < verticalLimit; xRover += 1) {
for(yRover = yOffset; yRover < horizontalLimit; yRover += 1) {
if(baseFrame[xRover][yRover] != screenShot[xRover][yRover]) {
// Skip over the already processed Rectangles
for(Rect itrRect : rectangles) {
if(( (xRover < itrRect.x + itrRect.h) && (xRover >= itrRect.x) ) && ( (yRover < itrRect.y + itrRect.w) && (yRover >= itrRect.y) )) {
shouldSkip = true;
yRover = itrRect.y + itrRect.w - 1;
break;
} // End if(( (xRover < itrRect.x + itrRect.h) && (xRover >= itrRect.x) ) && ( (yRover < itrRect.y + itrRect.w) && (yRover >= itrRect.y) ))
} // End for(Rect itrRect : rectangles)
if(shouldSkip) {
shouldSkip = false;
// Need to come out of the if condition as below that is why "continue" has been provided
// if(( (xRover <= (itrRect.x + itrRect.h)) && (xRover >= itrRect.x) ) && ( (yRover <= (itrRect.y + itrRect.w)) && (yRover >= itrRect.y) ))
continue;
} // End if(shouldSkip)
rect = new Rect();
rect.x = ((xRover - 1) < xOffset) ? xOffset : (xRover - 1);
rect.y = ((yRover - 1) < yOffset) ? yOffset : (yRover - 1);
rect.w = 2;
rect.h = 2;
/* Boolean variable used to re-scan the currently found rectangle
for any change due to previous scanning of boundaries */
isRectChanged = true;
while(isRectChanged) {
isRectChanged = false;
index = 0;
/* I */
/* Scanning of left-side boundary of rectangle */
index = rect.x;
limit = rect.x + rect.h;
while(index < limit && rect.y != yOffset) {
if(baseFrame[index][rect.y] != screenShot[index][rect.y]) {
isRectChanged = true;
rect.y = rect.y - 1;
rect.w = rect.w + 1;
index = rect.x;
continue;
} // End if(baseFrame[index][rect.y] != screenShot[index][rect.y])
index = index + 1;;
} // End while(index < limit && rect.y != yOffset)
/* II */
/* Scanning of bottom boundary of rectangle */
index = rect.y;
limit = rect.y + rect.w;
while( (index < limit) && (rect.x + rect.h != verticalLimit) ) {
rover = rect.x + rect.h - 1;
if(baseFrame[rover][index] != screenShot[rover][index]) {
isRectChanged = true;
rect.h = rect.h + 1;
index = rect.y;
continue;
} // End if(baseFrame[rover][index] != screenShot[rover][index])
index = index + 1;
} // End while( (index < limit) && (rect.x + rect.h != verticalLimit) )
/* III */
/* Scanning of right-side boundary of rectangle */
index = rect.x;
limit = rect.x + rect.h;
while( (index < limit) && (rect.y + rect.w != horizontalLimit) ) {
rover = rect.y + rect.w - 1;
if(baseFrame[index][rover] != screenShot[index][rover]) {
isRectChanged = true;
rect.w = rect.w + 1;
index = rect.x;
continue;
} // End if(baseFrame[index][rover] != screenShot[index][rover])
index = index + 1;
} // End while( (index < limit) && (rect.y + rect.w != horizontalLimit) )
} // while(isRectChanged)
// Remove those rectangles that come inside "rect" rectangle.
int idx = 0;
while(idx < rectangles.size()) {
Rect r = rectangles.get(idx);
if( ( (rect.x <= r.x) && (rect.x + rect.h >= r.x + r.h) ) && ( (rect.y <= r.y) && (rect.y + rect.w >= r.y + r.w) ) ) {
rectangles.remove(r);
} else {
idx += 1;
} // End if( ( (rect.x <= r.x) && (rect.x + rect.h >= r.x + r.h) ) && ( (rect.y <= r.y) && (rect.y + rect.w >= r.y + r.w) ) )
} // End while(idx < rectangles.size())
// Giving a head start to the yRover when a rectangle is found
rectangles.addFirst(rect);
yRover = rect.y + rect.w - 1;
rect = null;
} // End if(baseFrame[xRover][yRover] != screenShot[xRover][yRover])
} // End for(yRover = yOffset; yRover < horizontalLimit; yRover += 1)
} // End for(xRover = xOffset; xRover < verticalLimit; xRover += 1)
end = System.nanoTime();
return rectangles;
}
public static void main(String[] args) throws IOException {
LinkedList<Rect> rectangles = null;
// Buffering the Base image and Screen Shot Image
BufferedImage screenShotImg = ImageIO.read(new File("screenShotImg.png"));
BufferedImage baseImg = ImageIO.read(new File("baseImg.png"));
int width = baseImg.getWidth();
int height = baseImg.getHeight();
int xOffset = 0;
int yOffset = 0;
int length = baseImg.getWidth() * baseImg.getHeight();
// Creating 2 Two Dimensional Arrays for Image Processing
int[][] baseFrame = new int[height][width];
int[][] screenShot = new int[height][width];
// Creating 2 Single Dimensional Arrays to retrieve the Pixel Values
int[] baseImgPix = new int[length];
int[] screenShotImgPix = new int[length];
// Reading the Pixels from the Buffered Image
baseImg.getRGB(0, 0, baseImg.getWidth(), baseImg.getHeight(), baseImgPix, 0, baseImg.getWidth());
screenShotImg.getRGB(0, 0, screenShotImg.getWidth(), screenShotImg.getHeight(), screenShotImgPix, 0, screenShotImg.getWidth());
// Transporting the Single Dimensional Arrays to Two Dimensional Array
long start = System.nanoTime();
for(int row = 0; row < height; row++) {
System.arraycopy(baseImgPix, (row * width), baseFrame[row], 0, width);
System.arraycopy(screenShotImgPix, (row * width), screenShot[row], 0, width);
}
long end = System.nanoTime();
System.out.println("Array Copy : " + ((double)(end - start) / 1000000));
// Finding Differences between the Base Image and ScreenShot Image
ImageDifference imDiff = new ImageDifference();
rectangles = imDiff.differenceImage(baseFrame, screenShot, xOffset, yOffset, width, height);
// Displaying the rectangles found
int index = 0;
for(Rect rect : rectangles) {
System.out.println("\nRect info : " + (++index));
System.out.println("X : " + rect.x);
System.out.println("Y : " + rect.y);
System.out.println("W : " + rect.w);
System.out.println("H : " + rect.h);
// Creating Bounding Box
for(int i = rect.y; i < rect.y + rect.w; i++) {
screenShotImgPix[ ( rect.x * width) + i ] = 0xFFFF0000;
screenShotImgPix[ ((rect.x + rect.h - 1) * width) + i ] = 0xFFFF0000;
}
for(int j = rect.x; j < rect.x + rect.h; j++) {
screenShotImgPix[ (j * width) + rect.y ] = 0xFFFF0000;
screenShotImgPix[ (j * width) + (rect.y + rect.w - 1) ] = 0xFFFF0000;
}
}
// Creating the Resultant Image
screenShotImg.setRGB(0, 0, width, height, screenShotImgPix, 0, width);
ImageIO.write(screenShotImg, "PNG", new File("result.png"));
double d = ((double)(imDiff.end - imDiff.start) / 1000000);
System.out.println("\nTotal Time : " + d + " ms" + " Array Copy : " + ((double)(end - start) / 1000000) + " ms");
}
}
Description:
There would be a function named
public LinkedList<Rect> differenceImage(int[][] baseFrame, int[][] screenShot, int width, int height)
which does the job of finding differences in the images and return a linkedlist of objects. The objects are nothing but the rectangles.
There is main function which does the job of testing the algorithm.
There are 2 sample images passed into the code in main function, they are nothing but the "baseFrame" and "screenShot" thereby creating the resultant image named "result".
I don't possess the desired reputation to post the resultant image which would be very interesting.
There is a blog which would provide the output
Image Difference
I don't think there can be anything better than exhaustively searching from each side in turn for the first point of difference in that direction. Unless, that is, you know a fact that in some way constrains the set of points of difference.
So here comes the easy way if you know how to use Lockbit :)
Bitmap originalBMP = new Bitmap(pictureBox1.ImageLocation);
Bitmap changedBMP = new Bitmap(pictureBox2.ImageLocation);
int width = Math.Min(originalBMP.Width, changedBMP.Width),
height = Math.Min(originalBMP.Height, changedBMP.Height),
xMin = int.MaxValue,
xMax = int.MinValue,
yMin = int.MaxValue,
yMax = int.MinValue;
var originalLock = originalBMP.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadWrite, originalBMP.PixelFormat);
var changedLock = changedBMP.LockBits(new Rectangle(0, 0, width, height), ImageLockMode.ReadWrite, changedBMP.PixelFormat);
for (int y = 0; y < height; y++)
{
for (int x = 0; x < width; x++)
{
//generate the address of the colour pixel
int pixelIdxOrg = y * originalLock.Stride + (x * 4);
int pixelIdxCh = y * changedLock.Stride + (x * 4);
if (( Marshal.ReadByte(originalLock.Scan0, pixelIdxOrg + 2)!= Marshal.ReadByte(changedLock.Scan0, pixelIdxCh + 2))
|| (Marshal.ReadByte(originalLock.Scan0, pixelIdxOrg + 1) != Marshal.ReadByte(changedLock.Scan0, pixelIdxCh + 1))
|| (Marshal.ReadByte(originalLock.Scan0, pixelIdxOrg) != Marshal.ReadByte(changedLock.Scan0, pixelIdxCh))
)
{
xMin = Math.Min(xMin, x);
xMax = Math.Max(xMax, x);
yMin = Math.Min(yMin, y);
yMax = Math.Max(yMax, y);
}
}
}
originalBMP.UnlockBits(originalLock);
changedBMP.UnlockBits(changedLock);
var result = changedBMP.Clone(new Rectangle(xMin, yMin, xMax - xMin, yMax - yMin), changedBMP.PixelFormat);
pictureBox3.Image = result;
disclaim it looks like your 2 pictures contains more differences than we can see with the naked eye so the result will be wider than you expect but you can add a tolerance so it wil fit even if the rest isn't 100% identical
to speed things up you will maybe able to us Parallel.For but do it only for the outer loop
Related
I am implementing error diffusion algorithm (color reduction) in C# (WPF) and I am using writeableBitmap.
Following function opens new window with computed image:
private void OpenNewWindow()
{
// TODO: Bug with image resizes
const int margin = 50;
currAlgWindow = new Window { Owner = this };
var w = algorithmBitmap.Width;
var h = algorithmBitmap.Height;
var canvas = new Canvas { Width = w + margin * 2, Height = h + margin * 2 };
var img = new Image { Source = algorithmBitmap };
Canvas.SetLeft(img, (canvas.Width - w) / 2);
Canvas.SetTop(img, (canvas.Height - h) / 2);
canvas.Children.Add(img);
currAlgWindow.SizeToContent = SizeToContent.WidthAndHeight;
currAlgWindow.Content = canvas;
currAlgWindow.WindowStartupLocation = WindowStartupLocation.CenterOwner;
currAlgWindow.Show();
}
And I am getting weirdest issue I have ever encountered, because on resize of the new Window, assigned image changes even though there no onSizeChanged events attached anywhere.
Example:
1. Image right after currAlgWindow.Show()
2. Image after resizing the window.
3. Image after further resizing
Moving the window also changes the image a bit (refresh is visible).
All the issues does not occur on the original image, only on the computed one.
I am also providing the code of algorithm:
public abstract class Algorithm
{
protected WriteableBitmap bitmap;
protected byte[] originalCopy;
protected int bytesPerPixel;
protected int width;
protected int height;
protected Algorithm()
{
}
public virtual WriteableBitmap Bitmap
{
get => bitmap;
set
{
bitmap = value;
bytesPerPixel = (bitmap.Format.BitsPerPixel + 7) / 8;
width = bitmap.PixelWidth;
height = bitmap.PixelHeight;
originalCopy = new byte[height * bitmap.BackBufferStride];
bitmap.CopyPixels(originalCopy, bitmap.BackBufferStride, 0);
}
}
public abstract void Apply(int Kr, int Kg, int Kb);
protected static int RoundToNeareastMultiple(int num, int multiple)
{
return (int)(((num + multiple / 2) / multiple) * multiple);
}
}
public class ErrorDiffusionDithering : Algorithm
{
public ErrorDiffusionDithering(WriteableBitmap imageBitmap)
{
this.Bitmap = imageBitmap;
}
public override void Apply(int Kr, int Kg, int Kb)
{
int itR = 255 / (Kr - 1);
int itG = 255 / (Kg - 1);
int itB = 255 / (Kb - 1);
var bmpRect = new System.Windows.Int32Rect(0, 0, width, height);
bitmap.WritePixels(bmpRect, originalCopy, bitmap.BackBufferStride, 0); // Copy cached original image
bitmap.Lock();
unsafe
{
byte* bmpArray = (byte*)bitmap.BackBuffer.ToPointer();
for (int i = 0; i < height; ++i)
{
byte* currPos = bmpArray + i * bitmap.BackBufferStride;
Position row = GetPos(i, height);
for (int j = 0; j < width; ++j)
{
Position col = GetPos(j, width);
byte newVal = RoundToNeareastMultiple(currPos[0], itB).ToByte();
PropagateError(currPos[0] - newVal, 0, currPos, row, col);
currPos[0] = newVal;
newVal = RoundToNeareastMultiple(currPos[1], itG).ToByte();
PropagateError(currPos[1] - newVal, 1, currPos, row, col);
currPos[1] = newVal;
newVal = RoundToNeareastMultiple(currPos[2], itR).ToByte();
PropagateError(currPos[2] - newVal, 2, currPos, row, col);
currPos[2] = newVal;
currPos += bytesPerPixel;
}
}
}
bitmap.AddDirtyRect(bmpRect);
bitmap.Unlock();
}
private unsafe void PropagateError(int error, int colorNum, byte* currPos, Position row, Position col)
{
// x - from left to right
// y - from top to bottom
int ind;
if (col != Position.Last)
{
ind = bytesPerPixel + colorNum;
// pixel[x + 1][y] := pixel[x + 1][y] + quant_error * 7 / 16
currPos[ind] = (currPos[ind] + ((error * 7) >> 4)).ToByte();
}
if (row != Position.Last)
{
if (col != Position.First)
{
ind = bitmap.BackBufferStride - bytesPerPixel + colorNum;
// pixel[x - 1][y + 1] := pixel[x - 1][y + 1] + quant_error * 3 / 16
currPos[ind] = (currPos[ind] + ((error * 3) >> 4)).ToByte();
}
ind = bitmap.BackBufferStride + colorNum;
// pixel[x][y + 1] := pixel[x][y + 1] + quant_error * 5 / 16
currPos[ind] = (currPos[ind] + ((error * 5) >> 4)).ToByte();
if (col != Position.Last)
{
ind = bitmap.BackBufferStride + bytesPerPixel + colorNum;
//pixel[x + 1][y + 1] := pixel[x + 1][y + 1] + quant_error * 1 / 16
currPos[ind] = (currPos[ind] + ((error * 1) >> 4)).ToByte();
}
}
}
private enum Position { First, Last, Other };
private Position GetPos(int r, int dim)
{
return r == dim - 1 ? Position.Last : r == 0 ? Position.First : Position.Other;
}
}
Also the artifacts looks differently on the pictures that you can see here than on my computer - like after saving it to different format it is displayed in other way.
I also tested app on a different computer and there the issue does not occur.
I am completely confused about what is really the cause of my problem - screen, system, software, code?
I had tested it on other computer and artifacts were gone.
It was the issue with my laptop matrix - it had been replaced and wasn't fully compatible with my laptop.
Well, I'm trying to optimize what I did here (Smoothing noises with different amplitudes (Part 2)).
By this reason, I did a new implementation from scratch (https://youtu.be/o7pVEXhh3TI) to draw the path:
private void Start()
{
Polygon pol = File.ReadAllText(PolyPath).Deserialize<Polygon>();
// Create tex object
var list = pol.Vertices.AsEnumerable();
tex = list.CreateTextureObject(pol.Position, offset);
exampleTexture = new Texture2D(tex.Width, tex.Height);
exampleTexture.SetPixels32(new Color32[tex.Width * tex.Height]);
exampleTexture.Apply();
vertices = pol.Vertices.Select(v => (v - pol.Position) + offset).Clone().ToList();
_ss = new List<Segment>(pol.Segments.Select(s => new Segment((s.start + pol.Center - pol.Position) + offset, (s.end + pol.Center - pol.Position) + offset)));
foreach (Segment curSeg in _ss)
for (int i = -effectDistance; i < effectDistance; ++i)
{
Vector2 perp = Vector2.Perpendicular(((Vector2)curSeg.start - (Vector2)curSeg.end)).normalized;
segments.Add((Vector2)curSeg.start + perp * i);
F.DrawLine((Vector2)curSeg.start + perp * i, (Vector2)curSeg.end + perp * i, (x, y) => layers.Add(new Point(x, y)));
}
Debug.Log("Layer Count: " + layers.Count);
drawPath = true;
}
private void OnGUI()
{
if (exampleTexture == null)
return;
GUI.DrawTexture(new Rect((Screen.width - tex.Width) / 2, (Screen.height - tex.Height) / 2, tex.Width, tex.Height), exampleTexture);
if (drawPath)
{
{
Point? cur = layers.Count > 0 ? (Point?)layers.First() : null;
if (cur.HasValue)
{
exampleTexture.SetPixel(cur.Value.x, cur.Value.y, new Color32(170, 0, 0, 255));
exampleTexture.Apply();
layers.Remove(cur.Value);
}
}
{
Point? cur = segments.Count > 0 ? (Point?)segments.First() : null;
if (cur.HasValue)
{
exampleTexture.SetPixel(cur.Value.x, cur.Value.y, new Color32(0, 170, 0, 255));
exampleTexture.Apply();
segments.Remove(cur.Value);
}
}
{
Point? cur = vertices.Count > 0 ? (Point?)vertices.First() : null;
//Debug.Log(cur);
if (cur.HasValue)
{
exampleTexture.SetPixel(cur.Value.x, cur.Value.y, new Color32(255, 128, 0, 255));
exampleTexture.Apply();
vertices.Remove(cur.Value);
}
}
if (vertices.Count == 0 && segments.Count == 0 && layers.Count == 0)
drawPath = false;
}
}
This is what DrawLines actually do:
public static class F
{
public static void DrawLine(Point p1, Point p2, Action<int, int> action)
{
DrawLine(p1.x, p1.y, p2.x, p2.y, action);
}
public static void DrawLine(int x0, int y0, int x1, int y1, Action<int, int> action)
{
int sx = 0,
sy = 0;
int dx = Mathf.Abs(x1 - x0),
dy = Mathf.Abs(y1 - y0);
if (x0 < x1) { sx = 1; } else { sx = -1; }
if (y0 < y1) { sy = 1; } else { sy = -1; }
int err = dx - dy,
e2 = 0;
while (true)
{
action?.Invoke(x0, y0);
if ((x0 == x1) && (y0 == y1))
break;
e2 = 2 * err;
if (e2 > -dy)
{
err = err - dy;
x0 = x0 + sx;
}
if (e2 < dx)
{
err = err + dx;
y0 = y0 + sy;
}
}
}
}
This is an implemenentation of Bresenham algorithm.
This implementation is better because I have lowered iterations from 280k to 6k, but there is an issue as you can see this is innacurate...
The way this works first is getting the perpendicular of each segment on the shape (green pixels) and then drawing lines between the start and the end point of that segment. Segmenents are obtained using Ramer-Douglas-Peucker algorithm.
So I was thinking on draw the "orange" path spirally. I don't know how to explain this, basically, obtaining the same path but, with an scale (Translating/transforming? list of points from its center with an offset/distance) but I think I will have the same innacuracy.
Any guide will be appreciated. What algorithm could I use to draw the path with "layers"?
Following some of the information here, you might be able to use "inward/outward polygon offsetting" (aka "polygon buffering") to get the result you are interested in.
A tool such as Clipper can help.
Once you have a way to outwardly offset your shape, do the following:
First, draw the outer shape (black region below), then offset the inner shape outwards as far as you need it to go, and draw it on top of the outer shape (brown region below) using an appropriate noise/color scheme:
Then, apply a smaller offset, then draw that shape on top using a different noise/colorscheme (orange region below).
Repeat until you have as many gradients as you need:
Finally, draw the inner shape without any offsetting with its noise/color scheme:
In my previous question, I transformed this image:
into this:
which Tesseract OCR interprets as this:
1O351
Putting a frame around the image
actually improves the OCR result.
1CB51
However, I need all 5 characters to OCR correctly, so as an experiment I used Paint.NET to rotate and align each individual letter into its proper orientation:
Resulting in the correct answer:
1CB52
How would I go about performing this correction in C#?
I've done a bit of research on various text alignment algorithms, but they all assume the existence of lines of text in the source image, lines from which you can derive a rotation angle, but which already contain the proper spacing and orientation relationships between the letters.
You can use the code in the following code project article to segment each individual character. However, when trying to deskew these characters individually any result you get is not going to be very good because there isn't very much information to go off of.
I tried using AForge.NETs HoughLineTransformation class and I got angles in the range of 80 - 90 degrees. So I tried using the following code to deskew them:
private static Bitmap DeskewImageByIndividualChars(Bitmap targetBitmap)
{
IDictionary<Rectangle, Bitmap> characters = new CCL().Process(targetBitmap);
using (Graphics g = Graphics.FromImage(targetBitmap))
{
foreach (var character in characters)
{
double angle;
BitmapData bitmapData = character.Value.LockBits(new Rectangle(Point.Empty, character.Value.Size), ImageLockMode.ReadWrite, PixelFormat.Format8bppIndexed);
try
{
HoughLineTransformation hlt = new HoughLineTransformation();
hlt.ProcessImage(bitmapData);
angle = hlt.GetLinesByRelativeIntensity(0.5).Average(l => l.Theta);
}
finally
{
character.Value.UnlockBits(bitmapData);
}
using (Bitmap bitmap = RotateImage(character.Value, 90 - angle, Color.White))
{
g.DrawImage(bitmap, character.Key.Location);
}
}
}
return targetBitmap;
}
With the RotateImage method taken from here. However, the results didn't seem to be the best. Maybe you can try and make them better.
Here is the code from the code project article for your reference. I have made a few changes to it so that it behaves a bit safer, such as adding try-finally around the LockBits and disposing of objects properly using the using statement etc.
using System.Collections.Generic;
using System.Drawing;
using System.Drawing.Imaging;
using System.Linq;
namespace ConnectedComponentLabeling
{
public class CCL
{
private Bitmap _input;
private int[,] _board;
public IDictionary<Rectangle, Bitmap> Process(Bitmap input)
{
_input = input;
_board = new int[_input.Width, _input.Height];
Dictionary<int, List<Pixel>> patterns = Find();
var images = new Dictionary<Rectangle, Bitmap>();
foreach (KeyValuePair<int, List<Pixel>> pattern in patterns)
{
using (Bitmap bmp = CreateBitmap(pattern.Value))
{
images.Add(GetBounds(pattern.Value), (Bitmap)bmp.Clone());
}
}
return images;
}
protected virtual bool CheckIsBackGround(Pixel currentPixel)
{
return currentPixel.color.A == 255 && currentPixel.color.R == 255 && currentPixel.color.G == 255 && currentPixel.color.B == 255;
}
private unsafe Dictionary<int, List<Pixel>> Find()
{
int labelCount = 1;
var allLabels = new Dictionary<int, Label>();
BitmapData imageData = _input.LockBits(new Rectangle(0, 0, _input.Width, _input.Height), ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
try
{
int bytesPerPixel = 3;
byte* scan0 = (byte*)imageData.Scan0.ToPointer();
int stride = imageData.Stride;
for (int i = 0; i < _input.Height; i++)
{
byte* row = scan0 + (i * stride);
for (int j = 0; j < _input.Width; j++)
{
int bIndex = j * bytesPerPixel;
int gIndex = bIndex + 1;
int rIndex = bIndex + 2;
byte pixelR = row[rIndex];
byte pixelG = row[gIndex];
byte pixelB = row[bIndex];
Pixel currentPixel = new Pixel(new Point(j, i), Color.FromArgb(pixelR, pixelG, pixelB));
if (CheckIsBackGround(currentPixel))
{
continue;
}
IEnumerable<int> neighboringLabels = GetNeighboringLabels(currentPixel);
int currentLabel;
if (!neighboringLabels.Any())
{
currentLabel = labelCount;
allLabels.Add(currentLabel, new Label(currentLabel));
labelCount++;
}
else
{
currentLabel = neighboringLabels.Min(n => allLabels[n].GetRoot().Name);
Label root = allLabels[currentLabel].GetRoot();
foreach (var neighbor in neighboringLabels)
{
if (root.Name != allLabels[neighbor].GetRoot().Name)
{
allLabels[neighbor].Join(allLabels[currentLabel]);
}
}
}
_board[j, i] = currentLabel;
}
}
}
finally
{
_input.UnlockBits(imageData);
}
Dictionary<int, List<Pixel>> patterns = AggregatePatterns(allLabels);
patterns = RemoveIntrusions(patterns, _input.Width, _input.Height);
return patterns;
}
private Dictionary<int, List<Pixel>> RemoveIntrusions(Dictionary<int, List<Pixel>> patterns, int width, int height)
{
var patternsCleaned = new Dictionary<int, List<Pixel>>();
foreach (var pattern in patterns)
{
bool bad = false;
foreach (Pixel item in pattern.Value)
{
//Horiz
if (item.Position.X == 0)
bad = true;
else if (item.Position.Y == width - 1)
bad = true;
//Vert
else if (item.Position.Y == 0)
bad = true;
else if (item.Position.Y == height - 1)
bad = true;
}
if (!bad)
patternsCleaned.Add(pattern.Key, pattern.Value);
}
return patternsCleaned;
}
private IEnumerable<int> GetNeighboringLabels(Pixel pix)
{
var neighboringLabels = new List<int>();
for (int i = pix.Position.Y - 1; i <= pix.Position.Y + 2 && i < _input.Height - 1; i++)
{
for (int j = pix.Position.X - 1; j <= pix.Position.X + 2 && j < _input.Width - 1; j++)
{
if (i > -1 && j > -1 && _board[j, i] != 0)
{
neighboringLabels.Add(_board[j, i]);
}
}
}
return neighboringLabels;
}
private Dictionary<int, List<Pixel>> AggregatePatterns(Dictionary<int, Label> allLabels)
{
var patterns = new Dictionary<int, List<Pixel>>();
for (int i = 0; i < _input.Height; i++)
{
for (int j = 0; j < _input.Width; j++)
{
int patternNumber = _board[j, i];
if (patternNumber != 0)
{
patternNumber = allLabels[patternNumber].GetRoot().Name;
if (!patterns.ContainsKey(patternNumber))
{
patterns[patternNumber] = new List<Pixel>();
}
patterns[patternNumber].Add(new Pixel(new Point(j, i), Color.Black));
}
}
}
return patterns;
}
private unsafe Bitmap CreateBitmap(List<Pixel> pattern)
{
int minX = pattern.Min(p => p.Position.X);
int maxX = pattern.Max(p => p.Position.X);
int minY = pattern.Min(p => p.Position.Y);
int maxY = pattern.Max(p => p.Position.Y);
int width = maxX + 1 - minX;
int height = maxY + 1 - minY;
Bitmap bmp = DrawFilledRectangle(width, height);
BitmapData imageData = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, PixelFormat.Format24bppRgb);
try
{
byte* scan0 = (byte*)imageData.Scan0.ToPointer();
int stride = imageData.Stride;
foreach (Pixel pix in pattern)
{
scan0[((pix.Position.X - minX) * 3) + (pix.Position.Y - minY) * stride] = pix.color.B;
scan0[((pix.Position.X - minX) * 3) + (pix.Position.Y - minY) * stride + 1] = pix.color.G;
scan0[((pix.Position.X - minX) * 3) + (pix.Position.Y - minY) * stride + 2] = pix.color.R;
}
}
finally
{
bmp.UnlockBits(imageData);
}
return bmp;
}
private Bitmap DrawFilledRectangle(int x, int y)
{
Bitmap bmp = new Bitmap(x, y);
using (Graphics graph = Graphics.FromImage(bmp))
{
Rectangle ImageSize = new Rectangle(0, 0, x, y);
graph.FillRectangle(Brushes.White, ImageSize);
}
return bmp;
}
private Rectangle GetBounds(List<Pixel> pattern)
{
var points = pattern.Select(x => x.Position);
var x_query = points.Select(p => p.X);
int xmin = x_query.Min();
int xmax = x_query.Max();
var y_query = points.Select(p => p.Y);
int ymin = y_query.Min();
int ymax = y_query.Max();
return new Rectangle(xmin, ymin, xmax - xmin, ymax - ymin);
}
}
}
With the above code I got the following input/output:
As you can see the B has rotated quite well but the others aren't as good.
An alternative to trying to deskew the individual characters is to find there location using the segmentation routine above. Then passing each individual character through to your recognition engine separately and seeing if this improves your results.
I have used the following method to find the angle of the character using the List<Pixel> from inside the CCL class. It works by finding the angle between the "bottom left" and "bottom right" points. I haven't tested if it works if the character is rotated the other way around.
private double GetAngle(List<Pixel> pattern)
{
var pixels = pattern.Select(p => p.Position).ToArray();
Point bottomLeft = pixels.OrderByDescending(p => p.Y).ThenBy(p => p.X).First();
Point rightBottom = pixels.OrderByDescending(p => p.X).ThenByDescending(p => p.Y).First();
int xDiff = rightBottom.X - bottomLeft.X;
int yDiff = rightBottom.Y - bottomLeft.Y;
double angle = Math.Atan2(yDiff, xDiff) * 180 / Math.PI;
return -angle;
}
Note my drawing code is a bit broken so that is why the 5 is cut off on the right but this code produces the following output:
Note that the B and the 5 are rotated further than you'd expect because of their curvature.
Using the following code by getting the angle from the left and right edges and then choosing the best one, the rotations seems to be better. Note I have only tested it with letters that need rotating clockwise so if they need to go the opposite way it might not work too well.
This also "quadrants" the pixels so that each pixel is chosen from it's own quadrant as not to get two that are too nearby.
The idea in selecting the best angle is if they are similar, at the moment within 1.5 degrees of each other but can easily be updated, average them. Else we pick the one that is closest to zero.
private double GetAngle(List<Pixel> pattern, Rectangle bounds)
{
int halfWidth = bounds.X + (bounds.Width / 2);
int halfHeight = bounds.Y + (bounds.Height / 2);
double leftEdgeAngle = GetAngleLeftEdge(pattern, halfWidth, halfHeight);
double rightEdgeAngle = GetAngleRightEdge(pattern, halfWidth, halfHeight);
if (Math.Abs(leftEdgeAngle - rightEdgeAngle) <= 1.5)
{
return (leftEdgeAngle + rightEdgeAngle) / 2d;
}
if (Math.Abs(leftEdgeAngle) > Math.Abs(rightEdgeAngle))
{
return rightEdgeAngle;
}
else
{
return leftEdgeAngle;
}
}
private double GetAngleLeftEdge(List<Pixel> pattern, double halfWidth, double halfHeight)
{
var topLeftPixels = pattern.Select(p => p.Position).Where(p => p.Y < halfHeight && p.X < halfWidth).ToArray();
var bottomLeftPixels = pattern.Select(p => p.Position).Where(p => p.Y > halfHeight && p.X < halfWidth).ToArray();
Point topLeft = topLeftPixels.OrderBy(p => p.X).ThenBy(p => p.Y).First();
Point bottomLeft = bottomLeftPixels.OrderByDescending(p => p.Y).ThenBy(p => p.X).First();
int xDiff = bottomLeft.X - topLeft.X;
int yDiff = bottomLeft.Y - topLeft.Y;
double angle = Math.Atan2(yDiff, xDiff) * 180 / Math.PI;
return 90 - angle;
}
private double GetAngleRightEdge(List<Pixel> pattern, double halfWidth, double halfHeight)
{
var topRightPixels = pattern.Select(p => p.Position).Where(p => p.Y < halfHeight && p.X > halfWidth).ToArray();
var bottomRightPixels = pattern.Select(p => p.Position).Where(p => p.Y > halfHeight && p.X > halfWidth).ToArray();
Point topRight = topRightPixels.OrderBy(p => p.Y).ThenByDescending(p => p.X).First();
Point bottomRight = bottomRightPixels.OrderByDescending(p => p.X).ThenByDescending(p => p.Y).First();
int xDiff = bottomRight.X - topRight.X;
int yDiff = bottomRight.Y - topRight.Y;
double angle = Math.Atan2(xDiff, yDiff) * 180 / Math.PI;
return Math.Abs(angle);
}
This now produces the following output, again my drawing code is slightly broken. Note that the C looks to not have deskewed very well but looking closely it is just the shape of it that has caused this to happen.
I improved the drawing code and also attempted to get the characters onto the same baseline:
private static Bitmap DeskewImageByIndividualChars(Bitmap bitmap)
{
IDictionary<Rectangle, Tuple<Bitmap, double>> characters = new CCL().Process(bitmap);
Bitmap deskewedBitmap = new Bitmap(bitmap.Width, bitmap.Height, bitmap.PixelFormat);
deskewedBitmap.SetResolution(bitmap.HorizontalResolution, bitmap.VerticalResolution);
using (Graphics g = Graphics.FromImage(deskewedBitmap))
{
g.FillRectangle(Brushes.White, new Rectangle(Point.Empty, deskewedBitmap.Size));
int baseLine = characters.Max(c => c.Key.Bottom);
foreach (var character in characters)
{
int y = character.Key.Y;
if (character.Key.Bottom != baseLine)
{
y += (baseLine - character.Key.Bottom - 1);
}
using (Bitmap characterBitmap = RotateImage(character.Value.Item1, character.Value.Item2, Color.White))
{
g.DrawImage(characterBitmap, new Point(character.Key.X, y));
}
}
}
return deskewedBitmap;
}
This then produces the following output. Note each character isn't on the exact same baseline due to the pre rotation bottom being taken to work it out. To improve the code using the baseline from post rotation would be needed. Also thresholding the image before doing the baseline would help.
Another improvement would be to calculate the Right of each of the rotated characters locations so when drawing the next one it doesn't overlap the previous and cut bits off. Because as you can see in the output the 2 is slightly cutting into the 5.
The output is now very similar to the manually created one in the OP.
I am developing a system just like Camera mouse or other face control mouse, I have implemented all the functionality, the mouse pointer is also moving well, but I want to create the movement smooth just like the mouse control the pointer. the code I am using is:
if (startButton == true)
{
try
{
cap = new Capture();
pictureBox1.Image = cap.QueryFrame().ToImage<Bgr, Byte>().Bitmap;
}
catch (Exception exp)
{
MessageBox.Show("Error:" + exp);
}
_cascadeClassifier = new CascadeClassifier(Application.StartupPath + "/haarcascade_frontalface_default.xml");
eye_cascadeClassifier = new CascadeClassifier(Application.StartupPath + "/haarcascade_eye.xml");
timer1.Start();
}
private void timer1_Tick(object sender, EventArgs e)
{
using (var imageFrame = cap.QueryFrame().ToImage<Bgr, Byte>().Flip(FlipType.Horizontal))
{
if (imageFrame != null)
{
var grayframe = imageFrame.Convert<Gray, byte>();
var faces = _cascadeClassifier.DetectMultiScale(grayframe, 1.1, 10, Size.Empty); //the actual face detection happens here
foreach (var face in faces)
{
if(Configure.FaceBoxCheck==true)
imageFrame.Draw(face, new Bgr(Color.LightGreen), 2); //the detected face(s) is highlighted here using a box that is drawn around it/them
Int32 yCoordStartSearchEyes = face.Top + (face.Height * 3 / 11);
Point startingPointSearchEyes = new Point(face.X, yCoordStartSearchEyes);
Size searchEyesAreaSize = new Size(face.Width, (face.Height * 3 / 11));
Rectangle possibleROI_eyes = new Rectangle(startingPointSearchEyes, searchEyesAreaSize);
int widthNav = (imageFrame.Width / 11 * 3);
int heightNav = (imageFrame.Height / 11 * 3);
Rectangle nav = new Rectangle(new Point(imageFrame.Width / 2 - widthNav / 2, imageFrame.Height / 2 - heightNav / 2), new Size(widthNav, heightNav));
imageFrame.Draw(nav, new Bgr(Color.Lavender), 3);
Point cursor = new Point(face.X + searchEyesAreaSize.Width / 2, yCoordStartSearchEyes + searchEyesAreaSize.Height / 2);
grayframe.ROI = possibleROI_eyes;
var eyes = eye_cascadeClassifier.DetectMultiScale(grayframe, 2.15, 3, Size.Empty);
foreach (var eye in eyes)
{
//imageFrame.Draw(eye, new Bgr(Color.Red), 2);
if(Configure.EyeBoxCheck==true)
imageFrame.Draw(possibleROI_eyes, new Bgr(Color.DarkGreen), 2);
if (nav.Left < cursor.X && cursor.X < (nav.Left + nav.Width) && nav.Top < cursor.Y && cursor.Y < nav.Top + nav.Height)
{
LineSegment2D CursorDraw = new LineSegment2D(cursor, new Point(cursor.X, cursor.Y + 1));
imageFrame.Draw(CursorDraw, new Bgr(Color.White), 3);
//we compute new cursor coordinate using a simple scale based on frame width and height
int xCoord = (imageFrame.Width * (cursor.X - nav.Left)) / nav.Width;
int yCoord = (imageFrame.Height * (cursor.Y - nav.Top)) / nav.Height;
//We set our new cursor position
Cursor.Position = new Point(xCoord * 2, yCoord *2);
}
}
}
Ok, I'm sure there are a lot of other better ways, but this is a quick&dirty way of moving cursor position in a "Smooth" way from point a to point b. Of course this implementation can and should be optimized using a different thread instead of using Application.DoEvents() to avoid blocking the UI thread, but i hope this gets you on the track. First, how you should use it. Instead of:
Cursor.Position = new Point(xCoord * 2, yCoord *2);
You should do this:
MoveCursorSmooth(Cursor.Position, new Point(xCoord * 2, yCoord *2));
Now, the implementation of MoveCursorSmooth:
private void MoveCursorSmooth(Point a, Point b)
{
var step = 5;
var left = Math.Min(a.X, b.X);
var right = Math.Max(a.X, b.X);
int width = right - left;
var top = a.Y;
var bottom = b.Y;
int height = bottom - top;
if (width > height)
{
double slope = (double)height / (double)width;
if (a.X <= b.X)
for (int x = 1; x < width; ++x)
{
Cursor.Position = new Point((left + x), (a.Y + ((int)(slope * x + 0.5))));
System.Threading.Thread.Sleep(step);
Application.DoEvents();
}
else
for (int x = 1; x < width; ++x) // xOffset
{
Cursor.Position = new Point((right - x), (a.Y + ((int)(slope * x + 0.5))));
System.Threading.Thread.Sleep(step);
Application.DoEvents();
}
}
else
{
double slope = (double)width / (double)height;
if (a.X <= b.X)
{
for (int y = 1; y < height; ++y)
{
Cursor.Position = new Point((a.X + ((int)(slope * y + 0.5))), (top + y));
System.Threading.Thread.Sleep(step);
Application.DoEvents();
}
}
else
{
for (int y = 1; y < height; ++y)
{
Cursor.Position = new Point((b.X + ((int)(slope * y + 0.5))), (bottom - y));
System.Threading.Thread.Sleep(step);
Application.DoEvents();
}
}
}
}
This method is based on this answer
I'm working on a screen sharing app, which runs a loop and grab fast screenshots using GDI methods . example here
Of course I also use a flood fill algorithm to find the changes areas between 2 images (previous screenshot and current).
I use another small trick - I downscale the snapshot resolution in 10, because processing 1920*1080=2073600 pixels very constantly is not very efficient.
However when I find the rectangle bounds - I apply it on the original full size bitmap and I just multiply by 10 the dimension (including top, left, width, height).
This is the scanning code:
unsafe bool ArePixelsEqual(byte* p1, byte* p2, int bytesPerPixel)
{
for (int i = 0; i < bytesPerPixel; ++i)
if (p1[i] != p2[i])
return false;
return true;
}
private unsafe List<Rectangle> CodeImage(Bitmap bmp, Bitmap bmp2)
{
List<Rectangle> rec = new List<Rectangle>();
var bmData1 = bmp.LockBits(new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, bmp.PixelFormat);
var bmData2 = bmp2.LockBits(new System.Drawing.Rectangle(0, 0, bmp.Width, bmp.Height), System.Drawing.Imaging.ImageLockMode.ReadOnly, bmp2.PixelFormat);
int bytesPerPixel = 4;
IntPtr scan01 = bmData1.Scan0;
IntPtr scan02 = bmData2.Scan0;
int stride1 = bmData1.Stride;
int stride2 = bmData2.Stride;
int nWidth = bmp.Width;
int nHeight = bmp.Height;
bool[] visited = new bool[nWidth * nHeight];
byte* base1 = (byte*)scan01.ToPointer();
byte* base2 = (byte*)scan02.ToPointer();
for (int y = 0; y < nHeight; y ++)
{
byte* p1 = base1;
byte* p2 = base2;
for (int x = 0; x < nWidth; ++x)
{
if (!ArePixelsEqual(p1, p2, bytesPerPixel) && !(visited[x + nWidth * y]))
{
// fill the different area
int minX = x;
int maxX = x;
int minY = y;
int maxY = y;
var pt = new Point(x, y);
Stack<Point> toBeProcessed = new Stack<Point>();
visited[x + nWidth * y] = true;
toBeProcessed.Push(pt);
while (toBeProcessed.Count > 0)
{
var process = toBeProcessed.Pop();
var ptr1 = (byte*)scan01.ToPointer() + process.Y * stride1 + process.X * bytesPerPixel;
var ptr2 = (byte*)scan02.ToPointer() + process.Y * stride2 + process.X * bytesPerPixel;
//Check pixel equality
if (ArePixelsEqual(ptr1, ptr2, bytesPerPixel))
continue;
//This pixel is different
//Update the rectangle
if (process.X < minX) minX = process.X;
if (process.X > maxX) maxX = process.X;
if (process.Y < minY) minY = process.Y;
if (process.Y > maxY) maxY = process.Y;
Point n; int idx;
//Put neighbors in stack
if (process.X - 1 >= 0)
{
n = new Point(process.X - 1, process.Y); idx = n.X + nWidth * n.Y;
if (!visited[idx]) { visited[idx] = true; toBeProcessed.Push(n); }
}
if (process.X + 1 < nWidth)
{
n = new Point(process.X + 1, process.Y); idx = n.X + nWidth * n.Y;
if (!visited[idx]) { visited[idx] = true; toBeProcessed.Push(n); }
}
if (process.Y - 1 >= 0)
{
n = new Point(process.X, process.Y - 1); idx = n.X + nWidth * n.Y;
if (!visited[idx]) { visited[idx] = true; toBeProcessed.Push(n); }
}
if (process.Y + 1 < nHeight)
{
n = new Point(process.X, process.Y + 1); idx = n.X + nWidth * n.Y;
if (!visited[idx]) { visited[idx] = true; toBeProcessed.Push(n); }
}
}
//finaly set a rectangle.
Rectangle r = new Rectangle(minX * 10, minY * 10, (maxX - minX + 1) * 10, (maxY - minY + 1) * 10);
rec.Add(r);
//got the rectangle now i'll do whatever i want with that.
//notify i scaled everything by x10 becuse i want to apply the changes on the originl 1920x1080 image.
}
p1 += bytesPerPixel;
p2 += bytesPerPixel;
}
base1 += stride1;
base2 += stride2;
}
bmp.UnlockBits(bmData1);
bmp2.UnlockBits(bmData2);
return rec;
}
This is my call:
private void Start()
{
full1 = GetDesktopImage();//the first,intial screen.
while (true)
{
full2 = GetDesktopImage();
a = new Bitmap(full1, 192, 108);//resizing for faster processing the images.
b = new Bitmap(full2, 192, 108); // resizing for faster processing the images.
CodeImage(a, b);
count++; // counter for the performance.
full1 = full2; // assign old to current bitmap.
}
}
However, after all the tricks and techniques I used, the algorithm runs quite slow... on my machine - Intel i5 4670k 3.4ghz - it runs only 20 times (at the maximum! It might get lower)! It maybe sounds fast (don't forget I have to send each changed area over the network after), but I'm looking to achieve more processed image per second. I think the main bottleneck is in the resizing of the 2 images - but I just thought it would be even faster after resizing - because it would have to loop through less pixels... 192*108=200,000 only..
I would appreciate any help, any improvement. Thanks.