We Keep Coding

How can I speed up this image matching code in EmguCV C# code?

Here's the code. See the commented line for where possible optimization could be (at the foreach). Anyone have suggestions on improving the speed here?
public void FindMatches(Matrix<float> dbDescriptors, Matrix<float> queryDescriptors, ref IList<IndecesMapping> imap)
{
var indices = new Matrix<int>(queryDescriptors.Rows, 2); // matrix that will contain indices of the 2-nearest neighbors found
var dists = new Matrix<float>(queryDescriptors.Rows, 2); // matrix that will contain distances to the 2-nearest neighbors found
// create FLANN index with 4 kd-trees and perform KNN search over it look for 2 nearest neighbours
var flannIndex = new Index(dbDescriptors, 4);
flannIndex.KnnSearch(queryDescriptors, indices, dists, 2, 24);
for (int i = 0; i < indices.Rows; i++)
{
// filter out all inadequate pairs based on distance between pairs
if (dists.Data[i, 0] < (0.5 * dists.Data[i, 1]))
{
// find image from the db to which current descriptor range belongs and increment similarity value.
// in the actual implementation this should be done differently as it's not very efficient for large image collections.
foreach (var img in imap)
{
if (img.IndexStart <= indices[i, 0] && img.IndexEnd >= indices[i, 0])
{
img.Similarity++;
break;
}
}
}
}
}

You can improve your work by changing foreach loop as following:
foreach (var img in imap)
{
if (img.IndexStart <= indices[i, 1] && img.IndexEnd >= indices[i, 1])
{
img.Similarity++;
break;
}
}
Then it is working fine

Shortest way between two points by going through all other points from point collection

Given a point collection defined by x and y coordinates.
In this collection I get the start point, the end point and all the other n-2 points.
I have to find the shortest way between the start point and end point by going through all the other points. The shortest way is defined by its value and if possible the crossing point order.
At a first look this seems to be a graph problem, but i am not so sure about that right now, any way i am trying to find this shortest way by using only geometric relations since currently all the information that i have is only the x and y coordinates of the points, and which point is the start point and which is the end point.
My question is, can this way be found by using only geometric relations?
I am trying to implement this in C#, so if some helpful packages are available please let me know.

The simplest heuristic with reasonable performance is 2-opt. Put the points in an array, with the start point first and the end point last, and repeatedly attempt to improve the solution as follows. Choose a starting index i and an ending index j and reverse the subarray from i to j. If the total cost is less, then keep this change, otherwise undo it. Note that the total cost will be less if and only if d(p[i - 1], p[i]) + d(p[j], p[j + 1]) > d(p[i - 1], p[j]) + d(p[i], p[j + 1]), so you can avoid performing the swap unless it's an improvement.
There are a possible number of improvements to this method. 3-opt and k-opt consider more possible moves, resulting in better solution quality. Data structures for geometric search, kd-trees for example, decrease the time to find improving moves. As far as I know, the state of the art in local search algorithms for TSP is Keld Helsgaun's LKH.
Another family of algorithms is branch and bound. These return optimal solutions. Concorde (as far as I know) is the state of the art here.
Here's a Java implementation of the O(n^2 2^n) DP that Niklas described. There are many possible improvements, e.g., cache the distances between points, switch to floats (maybe), reorganize the iteration so that subsets are enumerating in increasing order of size (to allow only the most recent layer of minTable to be retained, resulting in a significant space saving).
class Point {
private final double x, y;
Point(double x, double y) {
this.x = x;
this.y = y;
}
double distanceTo(Point that) {
return Math.hypot(x - that.x, y - that.y);
}
public String toString() {
return x + " " + y;
}
}
public class TSP {
public static int[] minLengthPath(Point[] points) {
if (points.length < 2) {
throw new IllegalArgumentException();
}
int n = points.length - 2;
if ((1 << n) <= 0) {
throw new IllegalArgumentException();
}
byte[][] argMinTable = new byte[1 << n][n];
double[][] minTable = new double[1 << n][n];
for (int s = 0; s < (1 << n); s++) {
for (int i = 0; i < n; i++) {
int sMinusI = s & ~(1 << i);
if (sMinusI == s) {
continue;
}
int argMin = -1;
double min = points[0].distanceTo(points[1 + i]);
for (int j = 0; j < n; j++) {
if ((sMinusI & (1 << j)) == 0) {
continue;
}
double cost =
minTable[sMinusI][j] +
points[1 + j].distanceTo(points[1 + i]);
if (argMin < 0 || cost < min) {
argMin = j;
min = cost;
}
}
argMinTable[s][i] = (byte)argMin;
minTable[s][i] = min;
}
}
int s = (1 << n) - 1;
int argMin = -1;
double min = points[0].distanceTo(points[1 + n]);
for (int i = 0; i < n; i++) {
double cost =
minTable[s][i] +
points[1 + i].distanceTo(points[1 + n]);
if (argMin < 0 || cost < min) {
argMin = i;
min = cost;
}
}
int[] path = new int[1 + n + 1];
path[1 + n] = 1 + n;
int k = n;
while (argMin >= 0) {
path[k] = 1 + argMin;
k--;
int temp = s;
s &= ~(1 << argMin);
argMin = argMinTable[temp][argMin];
}
path[0] = 0;
return path;
}
public static void main(String[] args) {
Point[] points = new Point[20];
for (int i = 0; i < points.length; i++) {
points[i] = new Point(Math.random(), Math.random());
}
int[] path = minLengthPath(points);
for (int i = 0; i < points.length; i++) {
System.out.println(points[path[i]]);
System.err.println(points[i]);
}
}
}

The Euclidean travelling salesman problem can be reduced to this and it's NP-hard. So unless your point set is small or you have a very particular structure, you should probably look out for an approximation. Note that the Wikipedia article mentions the existence of a PTAS for the problem, which could turn out to be quite effective in practice.
UPDATE: Since your instances seem to have only few nodes, you can use a simple exponential-time dynamic programming approach. Let f(S, p) be the minimum cost to connect all the points in the set S, ending at the points p. We have f({start}, start) = 0 and we are looking for f(P, end), where P is the set of all points. To compute f(S, p), we can check all potential predecessors of p in the tour, so we have
f(S, p) = MIN(q in S \ {p}, f(S \ {p}, q) + distance(p, q))
You can represent S as a bitvector to save space (just use an single-word integer for maximum simplicity). Also use memoization to avoid recomputing subproblem results.
The runtime will be O(2^n * n^2) and the algorithm can be implemented with a rather low constant factor, so I predict it to be able to solve instance with n = 25 within seconds a reasonable amount of time.

This can be solved using an evolutionary algorithm.
Look at this: http://johnnewcombe.net/blog/post/23

You might want to look at TPL (Task Parallel Library) to speed up the application.
EDIT
I found this Link which has a Traveling Salesman algorithm:
http://msdn.microsoft.com/en-us/magazine/gg983491.aspx
The Source Code is said to be at:
http://archive.msdn.microsoft.com/mag201104BeeColony

Find minimum distance between 2 points in 2 arrays

I have 2 lists containing points (x,y and z) and would like to find the closest points.
I assume I need to do something like this:
for (int i = 0; i < myarray1.Count; i++)
{
for (int j = 0; j < myarray2.Count; j++)
{
// Calculate the quadratic distance between 2 points (point in index i and j)
// Then store the minimum distance I guess?
}
}

another option is to use Kd-tree
using the Nearest neighbour search will give you a O(log n) complexity to find the nearest point to a given set of points, and your code will have O( n log n), instead of O (n^2).
see here for implementation and example of how to use it.

double min_dist = DOUBLE_MAX;
for (i = 0; i < myarray1.Count; i++)
{
for (j = 0; j < myarray2.Count; j++)
{
curr_dist = dist(a[i],a[j]);
if( min_dist > curr_dist)
{
min_dist = curr_dist;
}
}
}
where
double dist(Point a, Point b) {
return sqrt(pow(a.x-b.x,2)+pow(a.y-b.y,2)+pow(a.z-b.z,2);
}

To compute the distance:
double sqr(double x) {return x*x;}
double distance(MyPoint a, MyPoint b) {
return sqrt(sqr(a.x-b.x)+sqr(a.y-b.y)+sqr(a.z-b.z);
}
Then in the second loop you store the minimum distance found so far:
double d = distance(myarray1[i],myarray2[j]);
if (d<min_d) min_d = d;
where min_d is defined at the beginning:
double min_d = Float.MAX_VALUE;

In C# I would do this using Linq.
First I would define the function that calculates the distance between two points as in Emanuelle Paolini's answer:
public double Distance(Point p1, Point p2)
{
return Math.Sqrt((p1.x - p2.x) * (p1.x - p2.x) + (p1.y - p1.y) * (p1.y - p1.y) + (p1.z - p1.z) * (p1.z - p1.z));
}
Then I would query the two lists as follows:
var distanceQuery = from p1 in myarray1
from p2 in myarray2
select Dist(p1, p2);
And finally I would retrieve the minimum distance:
var minimumDistance = distanceQuery.Min();

Formula is
Math.sqrt(Math.pow(Math.abs(x1-x2),2) + Math.pow(Math.abs(y1-y2),2)+ Math.pow(Math.abs(z1-z2),2))

How can I convert this divide and conquer code to compare one point to a list of points?

I found this code on the website http://rosettacode.org/wiki/Closest-pair_problem and I adopted the C# version of the divide and conquer method of finding the closest pair of points but what I am trying to do is adapt it for use to only find the closest point to one specific point. I have googled quite a bit and searched this website to find examples but none quite like this. I am not entirely sure what to change so that it only checks the list against one point rather than checking the list to find the two closest. I'd like to make my program operate as fast as possible because it could be searching a list of several thousand Points to find the closest to my current coordinate Point.
public class Segment
{
public Segment(PointF p1, PointF p2)
{
P1 = p1;
P2 = p2;
}
public readonly PointF P1;
public readonly PointF P2;
public float Length()
{
return (float)Math.Sqrt(LengthSquared());
}
public float LengthSquared()
{
return (P1.X - P2.X) * (P1.X - P2.X)
+ (P1.Y - P2.Y) * (P1.Y - P2.Y);
}
}
public static Segment Closest_BruteForce(List<PointF> points)
{
int n = points.Count;
var result = Enumerable.Range(0, n - 1)
.SelectMany(i => Enumerable.Range(i + 1, n - (i + 1))
.Select(j => new Segment(points[i], points[j])))
.OrderBy(seg => seg.LengthSquared())
.First();
return result;
}
public static Segment MyClosestDivide(List<PointF> points)
{
return MyClosestRec(points.OrderBy(p => p.X).ToList());
}
private static Segment MyClosestRec(List<PointF> pointsByX)
{
int count = pointsByX.Count;
if (count <= 4)
return Closest_BruteForce(pointsByX);
// left and right lists sorted by X, as order retained from full list
var leftByX = pointsByX.Take(count / 2).ToList();
var leftResult = MyClosestRec(leftByX);
var rightByX = pointsByX.Skip(count / 2).ToList();
var rightResult = MyClosestRec(rightByX);
var result = rightResult.Length() < leftResult.Length() ? rightResult : leftResult;
// There may be a shorter distance that crosses the divider
// Thus, extract all the points within result.Length either side
var midX = leftByX.Last().X;
var bandWidth = result.Length();
var inBandByX = pointsByX.Where(p => Math.Abs(midX - p.X) <= bandWidth);
// Sort by Y, so we can efficiently check for closer pairs
var inBandByY = inBandByX.OrderBy(p => p.Y).ToArray();
int iLast = inBandByY.Length - 1;
for (int i = 0; i < iLast; i++)
{
var pLower = inBandByY[i];
for (int j = i + 1; j <= iLast; j++)
{
var pUpper = inBandByY[j];
// Comparing each point to successivly increasing Y values
// Thus, can terminate as soon as deltaY is greater than best result
if ((pUpper.Y - pLower.Y) >= result.Length())
break;
Segment segment = new Segment(pLower, pUpper);
if (segment.Length() < result.Length())
result = segment;// new Segment(pLower, pUpper);
}
}
return result;
}
I used this code in my program to see the actual difference in speed and divide and conquer easily wins.
var randomizer = new Random(10);
var points = Enumerable.Range(0, 10000).Select(i => new PointF((float)randomizer.NextDouble(), (float)randomizer.NextDouble())).ToList();
Stopwatch sw = Stopwatch.StartNew();
var r1 = Closest_BruteForce(points);
sw.Stop();
//Debugger.Log(1, "", string.Format("Time used (Brute force) (float): {0} ms", sw.Elapsed.TotalMilliseconds));
richTextBox.AppendText(string.Format("Time used (Brute force) (float): {0} ms", sw.Elapsed.TotalMilliseconds));
Stopwatch sw2 = Stopwatch.StartNew();
var result2 = MyClosestDivide(points);
sw2.Stop();
//Debugger.Log(1, "", string.Format("Time used (Divide & Conquer): {0} ms", sw2.Elapsed.TotalMilliseconds));
richTextBox.AppendText(string.Format("Time used (Divide & Conquer): {0} ms", sw2.Elapsed.TotalMilliseconds));
//Assert.Equal(r1.Length(), result2.Length());

You can store the points in a better data structure that takes advantage of their position. Something like a quadtree.
The divide and conquer algorithm that you are trying to use doesn't really apply to this problem.

Don't use this algorithm at all, just go through the list one at a time comparing the distance to your reference point and at the end return the point that was the closest. This will be O(n).
You can probably add some extra speed ups but this should be good enough.
I can write some example code if you want.

You're mixing up two different problems. The only reason divide and conquer for the closest pair problem is faster than brute force is that it avoids comparing every point to every other point, so that it gets O(n log n) instead of O(n * n). But finding the closest point to just one point is just O(n). How can you find the closest point in a list of n points, while examining less than n points? What you're trying to do doesn't even make sense.
I can't say why your divide and conquer runs in less time than your brute force; maybe the linq implementation runs slower. But I think you'll find two things: 1) Even if, in absolute terms, your implementation of divide and conquer for 1 point runs in less time than your implementation of brute force for 1 point, they still have the same O(n). 2) If you just try a simple foreach loop and record the lowest distance squared, you'll get even better absolute time than your divide and conquer - and, it will still be O(n).

public static float LengthSquared(PointF P1, PointF P2)
{
return (P1.X - P2.X) * (P1.X - P2.X)
+ (P1.Y - P2.Y) * (P1.Y - P2.Y);
}
If, as your question states, you want to compare 1 (known) point to a list of points to find the closest then use this code.
public static Segment Closest_BruteForce(PointF P1, List<PointF> points)
{
PointF closest = null;
float minDist = float.MaxValue;
foreach(PointF P2 in points)
{
if(P1 != P2)
{
float temp = LengthSquared(P1, P2);
if(temp < minDist)
{
minDist = temp;
closest = P2;
}
}
}
return new Segment(P1, closest);
}
However, if as your example shows, you want to find the closest 2 points from a list of points try the below.
public static Segment Closest_BruteForce(List<PointF> points)
{
PointF closest1;
PointF closest2;
float minDist = float.MaxValue;
for(int x=0; x<points.Count; x++)
{
PointF P1 = points[x];
for(int y = x + 1; y<points.Count; y++)
{
PointF P2 = points[y];
float temp = LengthSquared(P1, P2);
if(temp < minDist)
{
minDist = temp;
closest1 = P1;
closest2 = P2;
}
}
}
return new Segment(closest1, closest2);
}
note the code above was written in the browser and may have some syntax errors.
EDIT Odd... is this an acceptable answer or not? Down-votes without explanation, oh well.

Graham Scan algorithm for finding convex hull

Well this is not exactly a programming related question. But see if you people can help me on it.
I have to implement the graham scan algorithm for convex hull but the problem is I'm not able to find a pseudo code that gives all the info. I found some but they leave some points.
Thanks.

#include <iostream>
#include <stack>
#include <stdlib.h>
using namespace std;
struct Point
{
int x, y;
};
// A global point needed for sorting points with reference
// to the first point Used in compare function of qsort()
Point p0;
// A utility function to find next to top in a stack
Point nextToTop(stack<Point> &S)
{
Point p = S.top();
S.pop();
Point res = S.top();
S.push(p);
return res;
}
// A utility function to swap two points
int swap(Point &p1, Point &p2)
{
Point temp = p1;
p1 = p2;
p2 = temp;
}
// A utility function to return square of distance
// between p1 and p2
int distSq(Point p1, Point p2)
{
return (p1.x - p2.x)*(p1.x - p2.x) +
(p1.y - p2.y)*(p1.y - p2.y);
}
// To find orientation of ordered triplet (p, q, r).
// The function returns following values
// 0 --> p, q and r are colinear
// 1 --> Clockwise
// 2 --> Counterclockwise
int orientation(Point p, Point q, Point r)
{
int val = (q.y - p.y) * (r.x - q.x) -
(q.x - p.x) * (r.y - q.y);
if (val == 0) return 0; // colinear
return (val > 0)? 1: 2; // clock or counterclock wise
}
// A function used by library function qsort() to sort an array of
// points with respect to the first point
int compare(const void *vp1, const void *vp2)
{
Point *p1 = (Point *)vp1;
Point *p2 = (Point *)vp2;
// Find orientation
int o = orientation(p0, *p1, *p2);
if (o == 0)
return (distSq(p0, *p2) >= distSq(p0, *p1))? -1 : 1;
return (o == 2)? -1: 1;
}
// Prints convex hull of a set of n points.
void convexHull(Point points[], int n)
{
// Find the bottommost point
int ymin = points[0].y, min = 0;
for (int i = 1; i < n; i++)
{
int y = points[i].y;
// Pick the bottom-most or chose the left
// most point in case of tie
if ((y < ymin) || (ymin == y &&
points[i].x < points[min].x))
ymin = points[i].y, min = i;
}
// Place the bottom-most point at first position
swap(points[0], points[min]);
// Sort n-1 points with respect to the first point.
// A point p1 comes before p2 in sorted output if p2
// has larger polar angle (in counterclockwise
// direction) than p1
p0 = points[0];
qsort(&points[1], n-1, sizeof(Point), compare);
// If two or more points make same angle with p0,
// Remove all but the one that is farthest from p0
// Remember that, in above sorting, our criteria was
// to keep the farthest point at the end when more than
// one points have same angle.
int m = 1; // Initialize size of modified array
for (int i=1; i<n; i++)
{
// Keep removing i while angle of i and i+1 is same
// with respect to p0
while (i < n-1 && orientation(p0, points[i],
points[i+1]) == 0)
i++;
points[m] = points[i];
m++; // Update size of modified array
}
// If modified array of points has less than 3 points,
// convex hull is not possible
if (m < 3) return;
// Create an empty stack and push first three points
// to it.
stack<Point> S;
S.push(points[0]);
S.push(points[1]);
S.push(points[2]);
// Process remaining n-3 points
for (int i = 3; i < m; i++)
{
// Keep removing top while the angle formed by
// points next-to-top, top, and points[i] makes
// a non-left turn
while (orientation(nextToTop(S), S.top(), points[i]) != 2)
S.pop();
S.push(points[i]);
}
// Now stack has the output points, print contents of stack
while (!S.empty())
{
Point p = S.top();
cout << "(" << p.x << ", " << p.y <<")" << endl;
S.pop();
}
}
// Driver program to test above functions
int main()
{
Point points[] = {{0, 3}, {1, 1}, {2, 2}, {4, 4},
{0, 0}, {1, 2}, {3, 1}, {3, 3}};
int n = sizeof(points)/sizeof(points[0]);
convexHull(points, n);
return 0;
}

Here you have my implementation of graham algorithm in c++. Check it out: graham algorithm

I think you need this.
http://aduni.org/courses/algorithms/courseware/psets/Problem_Set_04.doc