We Keep Coding

Scan shoting target paper for score

I am trying to code a desktop app for calculating score of shooting target range paper.
After researching, find some article can help, but still the problem how to work with openCv or emguCv, I am good at C# but C++ need time to learn it.
Another questio, what is the best approach to detect overlapping bullet holes in a shooting target?
like this image
Image is above. In the rings 7 and 8 there are two bullet holes overlapping. In this case it would be easy to solve it by simply performing an erosion.
However, in cases where the circles are almost completely overlapped i don't see how i can identify them.
Some links can help:
Detecting circles and shots from paper target
http://www.emgu.com/wiki/index.php/Shape_(Triangle,_Rectangle,_Circle,_Line)_Detection_in_CSharp

You can isolate overlapping bullets by following these steps:
Isolate your bullets from the rest of the image
Apply opening on the bullets (erode then dilate)
Compute the distance for each white pixel to the closest black pixel
Apply thresholding
The C++ code:
cv::Mat preprocess(const cv::Mat image) {
display(image, "Original");
// Color thresholds
cv::Scalar minColor(141, 0, 0);
cv::Scalar maxColor(255, 255, 124);
cv::Mat filtered;
// Isolate the interesting range of colors
cv::inRange(image, minColor, maxColor, filtered);
filtered.convertTo(filtered, CV_8U);
// Apply opening (erode then dilate)
cv::Mat opening;
cv::Mat kernel = cv::getStructuringElement(cv::MORPH_RECT, cv::Size(3, 3));
cv::morphologyEx(filtered, opening, cv::MORPH_OPEN, kernel, cv::Point(-1,-1), 2);
// Compute the distance to the closest zero pixel (euclidian)
cv::Mat distance;
cv::distanceTransform(opening, distance, CV_DIST_L2, 5);
cv::normalize(distance, distance, 0, 1.0, cv::NORM_MINMAX);
display(distance, "Distance");
// Thresholding using the longest distance
double min, max;
cv::minMaxLoc(distance, &min, &max);
cv::Mat thresholded;
cv::threshold(distance, thresholded, 0.7 * max, 255, CV_THRESH_BINARY);
thresholded.convertTo(thresholded, CV_8U);
// Find connected components
cv::Mat labels;
int nbLabels = cv::connectedComponents(thresholded, labels);
// Assign a random color to each label
vector<int> colors(nbLabels, 0);
for (int label = 1; label < nbLabels; ++label) {
colors[label] = rand() & 255;
}
cv::Mat result(distance.size(), CV_8U);
for (int r = 0; r < result.rows; ++r) {
for (int c = 0; c < result.cols; ++c) {
int label = labels.at<int>(r, c);
result.at<uchar>(r, c) = colors[label];
}
}
display(result, "Labels");
return result;
}

there are two ways in which you can complete your task.
The simpler method is to subtract the images. Take an ideal Target image and subtract it with the target image after each hit or at last after completing the entire shoot.
The other method would be to separate colors. If the Color of bullet is blue, then you can us inRange function to filter out the color. You can even make a library of different color bullets so the user can choose it from the option. I have recently done similar project in C#. For more details contact be on my email.(rajkumarm704#gmail.com)

How to find a path from source to target with obstacle avoidance

Actually my robot wants to move from source to target with obstacle avoidance. I find out the obstacle(rectangle shape) and Target(circle shape) in pixels. But i don't know how to find the path from source to target... Please help me.
Here is the code for finding obstacle and target.
for (int i = 0, n = blobs.Length; i < n; i++)
{
List<IntPoint> edgePoints = blobCounter.GetBlobsEdgePoints(blobs[i]);
AForge.Point center;
float radius;
// is circle ?
if (shapeChecker.IsCircle(edgePoints, out center, out radius))
{
g.DrawEllipse(whitePen, (float)(center.X - radius), (float)(center.Y - radius),
(float)(radius * 2), (float)(radius * 2));
target.Add(center.ToString());
}
else
{
List<IntPoint> corners;
// is triangle or quadrilateral
if (shapeChecker.IsConvexPolygon(edgePoints, out corners))
{
// get sub-type
PolygonSubType subType = shapeChecker.CheckPolygonSubType(corners);
Pen pen;
if (subType == PolygonSubType.Unknown)
{
pen = (corners.Count == 4) ? redPen : bluePen;
}
else
{
pen = (corners.Count == 4) ? greenPen : brownPen;
}
g.DrawPolygon(pen, ToPointsArray(corners));
}
}
}
This above coding will detect obstacle and target position pixel values and store it in a seperate array. But from these pixel values how to calculate the path? Waiting for ur suggestions.....

Trying looking up the A* search algorithm.

I have not looked into your code but it is a classic path finding problem. One suggestion could be to map the entire area the robot moves onto a grid. The grid can have discrete cells. And then you can use any graph search algorithm to find a path from start cell to goal cell.
You can use few of the algorithms, like Dijkistra, Best-first and A-Star search algorithms. It turns out that A-Star is efficient and easy to implement. Check this, contains a nice explanation about A-Star.

Kinect v2 Alignment of Infrared Sensor & RGB Image always slightly off

I'm using the official Kinect SDK 2.0 and Emgu CV in order to recognize the colors of a Rubik's Cube.
At first I use Canny Edge Extraction on the Infrared Camera since it handles different lightning conditions better than the RGB Camera and is much better to detect contours.
Then I use this code to convert the coordinates of the infrared sensor to the ones of the RGB camera.
As you can see the in the picture they are still off from what I am looking for. Since I already use the official KinectSensor.CoordinateMapper.MapDepthFrameToColorSpace I don't know how else I can improve the situation.
using (var colorFrame = reference.ColorFrameReference.AcquireFrame())
using (var irFrame = reference.InfraredFrameReference.AcquireFrame())
{
if (colorFrame == null || irFrame == null)
return;
// initialize depth frame data
FrameDescription depthDesc = irFrame.FrameDescription;
if (_depthData == null)
{
uint depthSize = depthDesc.LengthInPixels;
_depthData = new ushort[depthSize];
_colorSpacePoints = new ColorSpacePoint[depthSize];
// fill Array with max value so all pixels can be mapped
for (int i = 0; i < _depthData.Length; i++)
{
_depthData[i] = UInt16.MaxValue;
}
// didn't work so well with the actual depth-data
//depthFrame.CopyFrameDataToArray(_depthData);
_sensor.CoordinateMapper.MapDepthFrameToColorSpace(_depthData, _colorSpacePoints);
}
}
This is a helper-function I created in order to convert Point-Arrays in Infrared-Space to Color-Space
public static System.Drawing.Point[] DepthPointsToColorSpace(System.Drawing.Point[] depthPoints, ColorSpacePoint[] colorSpace){
for (int i = 0; i < depthPoints.Length; i++)
{
// 512 is the width of the depth/infrared image
int index = 512 * depthPoints[i].Y + depthPoints[i].X;
depthPoints[i].X = (int)Math.Floor(colorSpace[index].X + 0.5);
depthPoints[i].Y = (int)Math.Floor(colorSpace[index].Y + 0.5);
}
return depthPoints;
}

We can solve this problem by transforming infrared image coordinates to color image coordinates with 2 quadrilateral mapping.
A quadrilateral Q(x1,y1,x2,y2,x3,y3,x4,y4) in an infrared image, similarly,
it's mapping quadrilateral Q'(x1',y1',x2',y2',x3',y3',x4',y4') in the corresponding color image.
We can write the above mapping in form of equation as follows:
Q'= Q*A
where, A is a 3 X 3 matrix with coefficients a11, a12, a13, a21,.., a33;
The formula to obtain the coefficients are listed as follows:
x1=173; y1=98; x2=387; y2=93; x3=395; y3=262; x4=172; y4=264;
x1p=787; y1p=235; x2p=1407; y2p=215; x3p=1435; y3p=705; x4p=795; y4p=715;
tx=(x1p-x2p+x3p-x4p)*(y4p-y3p)-(y1p-y2p+y3p-y4p)*(x4p-x3p);
ty=(x2p-x3p)*(y4p-y3p)-(x4p-x3p)*(y2p-y3p);
a31=tx/ty;
tx=(y1p-y2p+y3p-y4p)*(x2p-x3p)-(x1p-x2p+x3p-x4p)*(y2p-y3p);
ty=(x2p-x3p)*(y4p-y3p)-(x4p-x3p)*(y2p-y3p);
a32=tx/ty;
a11=x2p-x1p+a31*x2p;
a12=x4p-x1p+a32*x4p;
a13=x1p;
a21=y2p-y1p+a31*y2p;
a22=y4p-y1p+a32*y4p;
a23=y1p;
a33=1.0;

Its because its not the same camera the camera that retrieves the depth data and the one that retrieves color data.
So you should apply a correction factor to displace the depth data.
Its a factor that is almost constant but its related to the distance.
I've got no code for you, but its something you can calculate yourself.

Automatically Merging Ajacent Rectangles

I've been making a top-down shooter game in XNA that requires rectangular collision for the map.
The collision walls for a map is stored in a text file in the format of:rect[0,0,1024,8]
The values correspond to defining a rectangle (x, y, width, height).
I've been thinking that I could write a separate application that can illiterate through the data of the map image, find out the pixels that are black (or any color of the wall) and make rectangles there. Basically, this program will generate the rectangles required for the collision. Ideally, it would be pixel perfect, which would require something like a thousand rectangles each 1 pixel wide that covers all the walls.
Is there a possible way to detect which of these rectangles (or squares I should say) are adjacent to one another, then connect them into the a bigger (but still covering the same area) rectangle?
EG. Lets say I have a wall that is 10 by 2. The program would generate 20 different rectangles, each 1 pixel high. How would I efficiently detect that these rectangles are adjacent and automatically make a 10 by 2 rectangle covering the whole wall instead of having 20 different little pixel rectangles?
EDIT: I've worked out a solution that fits my purposes, for future reference, my code is below:
//map is a bitmap, horizontalCollisions and collisions are List<Rectangle>s
for (int y = 0; y < map.Height; y++) //loop through pixels
{
for (int x = 0; x < map.Width; x++)
{
if (map.GetPixel(x, y).Name == "ff000000") //wall color
{
int i = 1;
while (map.GetPixel(x + i, y).Name == "ff000000")
{
if (i != map.Width - x)
{
i++;
}
if (i == map.Width - x)
{
break;
}
}
Rectangle r = new Rectangle(x, y, i, 1);//create and add
x += i - 1;
horizontalCollisions.Add(r);
}
}
}
for (int j = 0; j < horizontalCollisions.Count; j++)
{
int i = 1;
Rectangle current = horizontalCollisions[j];
Rectangle r = new Rectangle(current.X, current.Y + 1, current.Width, 1);
while(horizontalCollisions.Contains(r))
{
i++;
horizontalCollisions.Remove(r);
r = new Rectangle(current.X, current.Y + i, current.Width, 1);
}
Rectangle add = new Rectangle(current.X, current.Y, current.Width, i);
collisions.Add(add);
}
//collisions now has all the rectangles
Basically, it will loop through the pixel data horizontally. When it encounters a wall pixel, it will stop the counter and (using a while loop) move the counter towards the right, one by one until it hits a non-wall pixel. Then, it will create a rectangle of that width, and continue on. After this process, there will be a big list of rectangles, each 1px tall. Basically, a bunch of horizontal lines. The next loop will run through the horizontal lines, and using the same process as above, it will find out of there are any rectangles with the same X value and the same Width value under it (y+1). This will keep incrementing until there are none, in which one big rectangle will be created, and the used rectangles are deleted from the List. The final resulting list contains all the rectangles that will make up all the black pixels on the image (pretty efficiently, I think).

Etiquette may suggest that I should comment this instead of add it as an answer, but I do not yet have that capability, so bear with me.
I'm afraid I am not able to translate this into code for you, but I can send you towards some academic papers that discuss algorithms that can do some of the things that you're asking.
Other time this questions has appeared:
Find the set of largest contiguous rectangles to cover multiple areas
Puzzle: Find largest rectangle (maximal rectangle problem)
Papers linked in those questions:
Fast Algorithms To Partition Simple Rectilinear Polygons
Polygon Decomposition
The Maximal Rectangle Problem
Hopefully these questions and papers can lead help you find the answer you're looking for, or at least scare you off towards finding another solution.

Efficient ways to determine tilt of an image

I'm trying to write a program to programmatically determine the tilt or angle of rotation in an arbitrary image.
Images have the following properties:
Consist of dark text on a light background
Occasionally contain horizontal or vertical lines which only intersect at 90 degree angles.
Skewed between -45 and 45 degrees.
See this image as a reference (its been skewed 2.8 degrees).
So far, I've come up with this strategy: Draw a route from left to right, always selecting the nearest white pixel. Presumably, the route from left to right will prefer to follow the path between lines of text along the tilt of the image.
Here's my code:
private bool IsWhite(Color c) { return c.GetBrightness() >= 0.5 || c == Color.Transparent; }
private bool IsBlack(Color c) { return !IsWhite(c); }
private double ToDegrees(decimal slope) { return (180.0 / Math.PI) * Math.Atan(Convert.ToDouble(slope)); }
private void GetSkew(Bitmap image, out double minSkew, out double maxSkew)
{
decimal minSlope = 0.0M;
decimal maxSlope = 0.0M;
for (int start_y = 0; start_y < image.Height; start_y++)
{
int end_y = start_y;
for (int x = 1; x < image.Width; x++)
{
int above_y = Math.Max(end_y - 1, 0);
int below_y = Math.Min(end_y + 1, image.Height - 1);
Color center = image.GetPixel(x, end_y);
Color above = image.GetPixel(x, above_y);
Color below = image.GetPixel(x, below_y);
if (IsWhite(center)) { /* no change to end_y */ }
else if (IsWhite(above) && IsBlack(below)) { end_y = above_y; }
else if (IsBlack(above) && IsWhite(below)) { end_y = below_y; }
}
decimal slope = (Convert.ToDecimal(start_y) - Convert.ToDecimal(end_y)) / Convert.ToDecimal(image.Width);
minSlope = Math.Min(minSlope, slope);
maxSlope = Math.Max(maxSlope, slope);
}
minSkew = ToDegrees(minSlope);
maxSkew = ToDegrees(maxSlope);
}
This works well on some images, not so well on others, and its slow.
Is there a more efficient, more reliable way to determine the tilt of an image?

I've made some modifications to my code, and it certainly runs a lot faster, but its not very accurate.
I've made the following improvements:
Using Vinko's suggestion, I avoid GetPixel in favor of working with bytes directly, now the code runs at the speed I needed.
My original code simply used "IsBlack" and "IsWhite", but this isn't granular enough. The original code traces the following paths through the image:
http://img43.imageshack.us/img43/1545/tilted3degtextoriginalw.gif
Note that a number of paths pass through the text. By comparing my center, above, and below paths to the actual brightness value and selecting the brightest pixel. Basically I'm treating the bitmap as a heightmap, and the path from left to right follows the contours of the image, resulting a better path:
http://img10.imageshack.us/img10/5807/tilted3degtextbrightnes.gif
As suggested by Toaomalkster, a Gaussian blur smooths out the height map, I get even better results:
http://img197.imageshack.us/img197/742/tilted3degtextblurredwi.gif
Since this is just prototype code, I blurred the image using GIMP, I did not write my own blur function.
The selected path is pretty good for a greedy algorithm.
As Toaomalkster suggested, choosing the min/max slope is naive. A simple linear regression provides a better approximation of the slope of a path. Additionally, I should cut a path short once I run off the edge of the image, otherwise the path will hug the top of the image and give an incorrect slope.
Code
private double ToDegrees(double slope) { return (180.0 / Math.PI) * Math.Atan(slope); }
private double GetSkew(Bitmap image)
{
BrightnessWrapper wrapper = new BrightnessWrapper(image);
LinkedList<double> slopes = new LinkedList<double>();
for (int y = 0; y < wrapper.Height; y++)
{
int endY = y;
long sumOfX = 0;
long sumOfY = y;
long sumOfXY = 0;
long sumOfXX = 0;
int itemsInSet = 1;
for (int x = 1; x < wrapper.Width; x++)
{
int aboveY = endY - 1;
int belowY = endY + 1;
if (aboveY < 0 || belowY >= wrapper.Height)
{
break;
}
int center = wrapper.GetBrightness(x, endY);
int above = wrapper.GetBrightness(x, aboveY);
int below = wrapper.GetBrightness(x, belowY);
if (center >= above && center >= below) { /* no change to endY */ }
else if (above >= center && above >= below) { endY = aboveY; }
else if (below >= center && below >= above) { endY = belowY; }
itemsInSet++;
sumOfX += x;
sumOfY += endY;
sumOfXX += (x * x);
sumOfXY += (x * endY);
}
// least squares slope = (NΣ(XY) - (ΣX)(ΣY)) / (NΣ(X^2) - (ΣX)^2), where N = elements in set
if (itemsInSet > image.Width / 2) // path covers at least half of the image
{
decimal sumOfX_d = Convert.ToDecimal(sumOfX);
decimal sumOfY_d = Convert.ToDecimal(sumOfY);
decimal sumOfXY_d = Convert.ToDecimal(sumOfXY);
decimal sumOfXX_d = Convert.ToDecimal(sumOfXX);
decimal itemsInSet_d = Convert.ToDecimal(itemsInSet);
decimal slope =
((itemsInSet_d * sumOfXY) - (sumOfX_d * sumOfY_d))
/
((itemsInSet_d * sumOfXX_d) - (sumOfX_d * sumOfX_d));
slopes.AddLast(Convert.ToDouble(slope));
}
}
double mean = slopes.Average();
double sumOfSquares = slopes.Sum(d => Math.Pow(d - mean, 2));
double stddev = Math.Sqrt(sumOfSquares / (slopes.Count - 1));
// select items within 1 standard deviation of the mean
var testSample = slopes.Where(x => Math.Abs(x - mean) <= stddev);
return ToDegrees(testSample.Average());
}
class BrightnessWrapper
{
byte[] rgbValues;
int stride;
public int Height { get; private set; }
public int Width { get; private set; }
public BrightnessWrapper(Bitmap bmp)
{
Rectangle rect = new Rectangle(0, 0, bmp.Width, bmp.Height);
System.Drawing.Imaging.BitmapData bmpData =
bmp.LockBits(rect,
System.Drawing.Imaging.ImageLockMode.ReadOnly,
bmp.PixelFormat);
IntPtr ptr = bmpData.Scan0;
int bytes = bmpData.Stride * bmp.Height;
this.rgbValues = new byte[bytes];
System.Runtime.InteropServices.Marshal.Copy(ptr,
rgbValues, 0, bytes);
this.Height = bmp.Height;
this.Width = bmp.Width;
this.stride = bmpData.Stride;
}
public int GetBrightness(int x, int y)
{
int position = (y * this.stride) + (x * 3);
int b = rgbValues[position];
int g = rgbValues[position + 1];
int r = rgbValues[position + 2];
return (r + r + b + g + g + g) / 6;
}
}
The code is good, but not great. Large amounts of whitespace cause the program to draw relatively flat line, resulting in a slope near 0, causing the code to underestimate the actual tilt of the image.
There is no appreciable difference in the accuracy of the tilt by selecting random sample points vs sampling all points, because the ratio of "flat" paths selected by random sampling is the same as the ratio of "flat" paths in the entire image.

GetPixel is slow. You can get an order of magnitude speed up using the approach listed here.

If text is left (right) aligned you can determine the slope by measuring the distance between the left (right) edge of the image and the first dark pixel in two random places and calculate the slope from that. Additional measurements would lower the error while taking additional time.

First I must say I like the idea. But I've never had to do this before and I'm not sure what all to suggest to improve reliability. The first thing I can think of this is this idea of throwing out statistical anomalies. If the slope suddenly changes sharply then you know you've found a white section of the image that dips into the edge skewing (no pun intended) your results. So you'd want to throw that stuff out somehow.
But from a performance standpoint there are a number of optimizations you could make which may add up.
Namely, I'd change this snippet from your inner loop from this:
Color center = image.GetPixel(x, end_y);
Color above = image.GetPixel(x, above_y);
Color below = image.GetPixel(x, below_y);
if (IsWhite(center)) { /* no change to end_y */ }
else if (IsWhite(above) && IsBlack(below)) { end_y = above_y; }
else if (IsBlack(above) && IsWhite(below)) { end_y = below_y; }
To this:
Color center = image.GetPixel(x, end_y);
if (IsWhite(center)) { /* no change to end_y */ }
else
{
Color above = image.GetPixel(x, above_y);
Color below = image.GetPixel(x, below_y);
if (IsWhite(above) && IsBlack(below)) { end_y = above_y; }
else if (IsBlack(above) && IsWhite(below)) { end_y = below_y; }
}
It's the same effect but should drastically reduce the number of calls to GetPixel.
Also consider putting the values that don't change into variables before the madness begins. Things like image.Height and image.Width have a slight overhead every time you call them. So store those values in your own variables before the loops begin. The thing I always tell myself when dealing with nested loops is to optimize everything inside the most inner loop at the expense of everything else.
Also... as Vinko Vrsalovic suggested, you may look at his GetPixel alternative for yet another boost in speed.

At first glance, your code looks overly naive.
Which explains why it doesn't always work.
I like the approach Steve Wortham suggested,
but it might run into problems if you have background images.
Another approach that often helps with images is to blur them first.
If you blur your example image enough, each line of text will end up
as a blurry smooth line. You then apply some sort of algorithm to
basically do a regression analisys. There's lots of ways to do
that, and lots of examples on the net.
Edge detection might be useful, or it might cause more problems that its worth.
By the way, a gaussian blur can be implemented very efficiently if you search hard enough for the code. Otherwise, I'm sure there's lots of libraries available.
Haven't done much of that lately so don't have any links on hand.
But a search for Image Processing library will get you good results.
I'm assuming you're enjoying the fun of solving this, so not much in actual implementation detalis here.

Measuring the angle of every line seems like overkill, especially given the performance of GetPixel.
I wonder if you would have better performance luck by looking for a white triangle in the upper-left or upper-right corner (depending on the slant direction) and measuring the angle of the hypotenuse. All text should follow the same angle on the page, and the upper-left corner of a page won't get tricked by the descenders or whitespace of content above it.
Another tip to consider: rather than blurring, work within a greatly-reduced resolution. That will give you both the smoother data you need, and fewer GetPixel calls.
For example, I made a blank page detection routine once in .NET for faxed TIFF files that simply resampled the entire page to a single pixel and tested the value for a threshold value of white.

What are your constraints in terms of time?
The Hough transform is a very effective mechanism for determining the skew angle of an image. It can be costly in time, but if you're going to use Gaussian blur, you're already burning a pile of CPU time. There are also other ways to accelerate the Hough transform that involve creative image sampling.

Your latest output is confusing me a little.
When you superimposed the blue lines on the source image, did you offset it a bit? It looks like the blue lines are about 5 pixels above the centre of the text.
Not sure about that offset, but you definitely have a problem with the derived line "drifting" away at the wrong angle. It seems to have too strong a bias towards producing a horizontal line.
I wonder if increasing your mask window from 3 pixels (centre, one above, one below) to 5 might improve this (two above, two below). You'll also get this effect if you follow richardtallent's suggestion and resample the image smaller.

Very cool path finding application.
I wonder if this other approach would help or hurt with your particular data set.
Assume a black and white image:
Project all black pixels to the right (EAST). This should give a result of a one dimensional array with a size of IMAGE_HEIGHT. Call the array CANVAS.
As you project all the pixels EAST, keep track numerically of how many pixels project into each bin of CANVAS.
Rotate the image an arbitrary number of degrees and re-project.
Pick the result that gives the highest peaks and lowest valleys for values in CANVAS.
I imagine this will not work well if in fact you have to account for a real -45 -> +45 degrees of tilt. If the actual number is smaller(?+/- 10 degrees), this might be a pretty good strategy. Once you have an intial result, you could consider re-running with a smaller increment of degrees to fine tune the answer. I might therefore try to write this with a function that accepted a float degree_tick as a parm so I could run both a coarse and fine pass (or a spectrum of coarseness or fineness) with the same code.
This might be computationally expensive. To optimize, you might consider selecting just a portion of the image to project-test-rotate-repeat on.