How to extract a rectangular and non-rectangular area from a video? - c#

I am developing a face tracking application using Kinect, and I have output like the following.
What I want to do is the extract the exact face area, either the yellow-lined area or the red rectangular, it would be great if you can tell me how to do both. Basically, I am expecting an output with only the interested area with black otherwise.
Right now I have all the point coordinates that I need, but I am not sure which class and method to use.
Please note that I am working with video frames, if it makes any difference.
Thank you,

I haven't done this in C#, but in c++, running something like this generates an array of RGB values:
const XnRGB24Pixel *pImage = imageMD->RGB24Data(); // generate array
XnRGB24Pixel pixelRGB = *pImage; // get the first element of array
byte red = pixelRGB.nRed // read the RED value
The array is 1-dimensional and elements are stored in it row wise from the bottom (last row, then the row before last row, ...). Going through them with a nested for loop like this would give you the rectangular area you want:
for(int y = RECTANGLE_Y1; y < RECTANGLE_Y2; y++)
{
for(int x = RECTANGLE_X1; x < RECTANGLE_x2; x++)
{
pixelRGB = pImage[y*RESOLUTION_WIDTH + x]; // get the element
// work with pixelRGB
}
}
Replace RECTANGLE_X1 and RECTANGLE_Y1 with the coordinates of the lower left edge coordinates of your rectangle and RECTANGLE_X2 and RECTANGLE_Y2 with the upper right edge coordinates.
For getting the values of pixels inside the non-rectangular area, a similar approach would work if you can figure out the geometric calculations necessary to recognize when a given [x,y] are inside the area; but even after that, I don't have any clue on how to store it in conventional data structures.
Hope this helps.

Related

C# MS Charts get Y value from a generic X

I have a chart with 3 lines, all of them dynamics (the points series varies any time).
For two lines I have few points, up to 20, and for the other one, that is a Spline, up to 500. (always in the same x range)
I have to verify if the Spline is between the two other lines.
In other words, for each point of the Spline get the Y value of all the lines and verify if it's in the range.
How I could achieve that?
I've already tried the follow code, but when I call an X point that is not defined in the other two Line I'll have only the Y value for the Spline:
var a = chart1.Series.Select(series => series.Points.Where(point => point.XValue == 7).ToList()).ToList();
Follow an image of the graph-like:
The blue and the yellow line are the ones "less defined", and I have to verify if the red one stays between them
It's rather a math problem than a chart problem.
To determine the mid line is in between the upper and lower bounds, it comes down to the linear interpolation of the bounds.
Suppose your upper bound (ub) is defined on five points: 1,2,5,6,9.
The easy way to do the linear interpolation is to find the two closest neighbors and do the weighted average. e.g. ub(5.5) = ub(5)/2 + ub(6)/2
There are many ways to do the 1D linear interpolation and if you search it on stackoverflow you should be able to find existing solutions.

Adding single dimension float array values

I have two questions regarding the Evil Dicom library.
I know with the floats function, all of the pixel data is contained as float elements in a one dimensional array. My question here is how can I add up the individual elements to get one value?
After I have multiplied a black and white mask with the original image, how can I then add up the non-zero values in the image? Do I also use the floats function to get the data as an array and then add up the array elements? If not, how can I add up the pixels in the image from top left corner to bottom right corner?
As far as I know there is no special functionality in the Evil DICOM library for these operations, but there is always LINQ.
If you are looking for the sum of all pixel elements:
var imgMtx = new ImageMatrix("image.dcm");
var sum = imgMtx.Image.Sum();
If you are looking for the sum of the non-zero values:
var nonZeroSum = imgMtx.Image.Where(val => val != 0.0f).Sum();
If you are looking for the average value, simply replace Sum with Average.

Creating a equilateral triangular grid over geometry

I need to create a equilateral triangular grid that fits a given geometry.
I have an image containing the geometry, it might include holes or thin paths. and i need to create a grid similar to this image:
The circles are variable in diameter, and need to cover the entire geometry. the points does not have to be on the geometry.
You can think of the triangular grid as being an oblique rectangular grid
This enables you to store the state of each circle in a 2-dimensional matrix, for instance, and to use simple nested loops for processing. Of cause then you will have to translate these logical coordinates to the geometry plane coordinates for drawing.
const double Sin30 = 0.5;
static readonly double Cos30 = Math.Cos(30*Math.PI/180);
for (int xLogical = 0; xLogical < NX; xLogical++) {
for (int yLogical = 0; yLogical < NY; yLogical++) {
double xGeo = GridDistance * xLogical * Cos30;
double yGeo = GridDistance * (yLogical + xLogical * Sin30);
...
}
}
I am assuming this is to create a 2D meshing tool. If it is, and it is homework, I suggest doing it yourself as you will get alot out of it. If it is not a meshing problem what I will have to say should help you regardless...
To do this, use the grid node centres to generate your equilaterals. If you don't have the centre points to start with you will need to look at first selecting an orientation for your object and then creating these (rectangular based) grid nodes (you will have to work out a way of testing whether these points actually lie inside your object boundaries). You can then construct your equilateral triangles using these points. Note. You again will have to deal with edge detection to get half decent accuracy.
To go a bit further that just equilaterals, and get a more accurate mesh, you will have to look into anisotropic mesh adaptation (AMA) using triangulation. This will be a lot harder than the basic approach outlined above - but fun!
Check out this link to a 2D tet-mesh generator using AMA. The paper this code is based on is:
V. Dolejsi: Anisotropic mesh adaptation for finite volume and finite element methods on triangular meshes
Computing and Visualisation in Science, 1:165-178, 1998.

I have a n x n grid filled with photo urls. How can I make sure photos do not appear together in c#

I basically have a grid, lets say 100 x 100 which is filled with url's of a photo collection. Some of these are duplicates as I may only have 50 photos but I want to duplicate them to make sure the 100 x 100 grid is filled.
I randomly fill the grid with the URL's and then display them which is fine. The problem I have is that obviously sometimes photos with the same URL are randomly places together either on the x axis or y axis or sometimes both.
How can I make sure that I fill the grid so that these images with the same URL are as far apart as possible thus preventing 2 of the same photos appearing next to each other.
Any help appreciated
Mike
If you really want "as far apart as possible" then (1) I bet you're out of luck and (2) if that were achievable it would probably produce not-very-random-looking results. But if all you want is "somewhat far apart", it's not so bad. Here are a few things you can do.
(1) Classify grid positions according to the parity of their x,y coordinates: that is, whether they're odd and even. Divide the photos into four roughly-equal-sized batches. Now select from different batches according to the parity of the coordinates. The following code (which is a bit too "clever"; sorry) does this, modulo bugs and typos.
System.Random rng = new System.Random();
for (int x=0; x<nx; ++x) {
for (int y=0; y<ny; ++y) {
k = ((x&1)<<1) + (y&1); // 0..3
int n_photos_in_batch = (n_photos+3-k) >> 2;
int photo_number = (rng.Next(0,n_photos_in_batch-1) << 2) + k;
// use this photo
}
}
Downsides: doesn't do anything to move copies of a photo any further away from one another than one step. Reduces randomness somewhat since all copies of any given photo will be in a fixed subset of positions; in some contexts this may be visible and look rather silly.
Variations: we're basically covering the grid with 2x2 tiles, and restricting the range of photos allowed to occur in each tile. You could use larger tiles, or differently-shaped tiles, or arrange them differently. For instance, if you say k = ((x&1)<<1) ^ (y&3) you get 2x2 tiles arranged in a kinda-hexagonal pattern, which is actually probably better than the version above.
(2) Loop over positions in your grid (raster order will do, though there might be better alternatives) and for each one choose a photo that (a) doesn't already occur too near to the position you're looking at and (b) is otherwise random. The following code (again, modulo bugs and typos) does something like this, though for large grids you might want to make it more efficient.
System.Random rng = new System.Random();
radius = MAX_RADIUS; // preferably not too big, so that the search isn't too slow
while ((2*radius+1)*(2*radius+1) >= n_photos) --radius; // gratuitously inefficient!
for (int x=0; x<nx; ++x) {
for (int y=0; y<ny; ++y) {
// which photos already appear too near to here?
System.Collections.BitArray unseen = new System.Collections.BitArray(n_photos,True);
for (x1=x-radius; x1<=x+radius; ++x1) {
for (int y1=y-radius; y1<=y+radius; ++y1) {
if (0 <= x1 && x1 < nx && 0 <= y1 && y1 < nx && (y1<y || (y1==y && x1<x))) {
unseen[photos[x1,y1]] = False;
}
}
}
// now choose a random one of them
int n_unseen = 0;
for (int i=0; i<n_photos; ++i) if (unseen[i]) ++n_unseen;
System.Debug.Assert(n_unseen>0, "no photos available");
int j = rng.Next(0,n_unseen-1);
for (int i=0; i<n_photos; ++i) {
if (unseen[i]) {
if (j==0) { photos[x,y] = i; break; }
--j;
}
}
}
}
Notes: This is much more expensive than option 1. The validity check on x1,y1 is gratuitously inefficient here, of course. So is the choice of radius. The obvious more-efficient versions of these, however, may break down if you adopt some of the variations I'm about to list. This code, as it stands, won't do anything to keep photos apart if there are fewer than 9. The choice of radius is actually completely bogus, for the grid-traversal order I've used, because there are never more than 2r^2+2r "excluded" positions; again, that may change if you traverse the grid in a different order. Etc.
Variations: there's no real reason why the region you search over should be square. Circular might well be better, for instance. You could, with some extra work, construct a region that always has exactly as many points in it as you have photos (though if you do that you'll get a mostly-periodic pattern of photos, so better to be a bit less aggressive ). It might be better to process the grid entries in a different position -- e.g., spiralling out from the centre.
(3) Option 2 above will keep photos unique within a certain range (about as large as it can be given how many different photos you have) but not care about keeping copies further away apart from that. You could, instead, decide how bad it is having two identical photos at any given distance and then choose photos to minimize total badness. This will be even more expensive than option 2. I shan't bother giving sample code; you can probably work out how you might do it.
[EDITED to add ...]
(4) Here's a cute variation on the theme of (1). It will work best when the grid is square and its size is a power of 2, but you can adapt it to work more generally. It takes time only proportional to the size of your grid, however many photos you have. For each position (x,y): Throw away all but the bottom k bits of the coordinates, for some k. Bit-reverse them and interleave the bits, giving a number m from 0 to 2^(2k)-1. Choose k so that this is somewhere on the order of, say, n_photos/4. Now, at position (x,y) you'll put photo number round(n_photos*m/2^(2k) + smallish_random_number). There are a few details I'll leave for you to fill in :-).
Fastest way is somthing like this:
You have array of n imgs URL & grid x*y
Find a central cell of the grid.
Randomly extract imgs URL from array and put each URL around central cell (first URL put to the center)
Do it until you don't fill all grid cells or while you have URLs in array.
If every URL is used then you should take URLs from concentric circles that you are made. Folow from the central cell to the circle with the bigest radius.
URLs taken by this method you should randomly put around biggest circle.
This algorithm will work if you have enough URLs for drawing less then 2 disks to the grid.
You can successfully modify it if you will follow the rule that URLs from one set must fill as big circle as it can.
What you want is a space-filling-curve for example a hilbert curve. It fills your grid with a continous line separating each square by only 1 bit. Because the nature of a sfc is to recursivley fill the space and maintain a neighborhood you can exploit this and place the picture along the line. If you don't want to place the same picture in the direct neighboorhood you can use a depth-seach on the sfc on each node eliminates copies.

How to fill a square with smaller squares/rectangles?

In my office at work, we are not allowed to paint the walls, so I have decided to frame out squares and rectangles, attach some nice fabric to them, and arrange them on the wall.
I am trying to write a method which will take my input dimensions (9' x 8' 8") and min/max size (1' x 3', 2', 4', etc..) and generate a random pattern of squares and rectangles to fill the wall. I tried doing this by hand, but I'm just not happy with the layout that I got, and it takes about 35 minutes each time I want to 'randomize' the layout.
One solution is to start with x*y squares and randomly merge squares together to form rectangles. You'll want to give differing weights to different size squares to keep the algorithm from just ending up with loads of tiny rectangles (i.e. large rectangles should probably have a higher chance of being picked for merging until they get too big).
Sounds like a Treemap
Another idea:
1. Randomly generate points on the wall
Use as many points as the number of rectangles you want
Introduce sampling bias to get cooler patterns
2. Build the kd-tree of these points
The kd-tree will split the space in a number of rectangles. There might be too much structure for what you want, but its still a neat geeky algorithm.
(see: http://en.wikipedia.org/wiki/Kd-tree)
Edit: Just looked at JTreeMap, looks a bit like this is what its doing.
If you're talking on a pure programing problem ;) There is a technique called Bin Packing that tries to pack a number of bins into the smallest area possible. There's loads of material out there:
http://en.wikipedia.org/wiki/Bin_packing_problem
http://mathworld.wolfram.com/Bin-PackingProblem.html
http://www.cs.sunysb.edu/~algorith/files/bin-packing.shtml
So you 'could' create a load of random squares and run it through a bin packer to generate your pattern.
I've not implemented a bin packing algorithm myself but I've seen it done by a colleague for a Nike website. Best of luck
Since you can pick the size of the rectangles, this is not a hard problem.
I'd say you can do something as simple as:
Pick an (x,y) coordinate that is not currently inside a rectangle.
Pick a second (x,y) coordinate so that when you draw a rectangle between
the two coordinates, it won't overlap anything. The bounding box of
valid points is just bounded by the nearest rectangles' walls.
Draw that rectangle.
Repeat until, say, you have 90% of the area covered. At that point you
can either stop, or fill in the remaining holes with as big rectangles
as possible.
It might be interesting to parametrize the generation of points, and then make a genetic algorithm. The fitness function will be how much you like the arrangement - it would draw hundreds of arrangements for you, and you would rate them on a scale of 1-10. It would then take the best ones and tweak those, and repeat until you get an arrangement you really like.
Bin packing or square packing?
Bin packing:
http://www.cs.sunysb.edu/~algorith/files/bin-packing.shtml
Square packing:
http://www.maa.org/editorial/mathgames/mathgames_12_01_03.html
This actually sounds more like an old school random square painting demo, circa 8-bit computing days, especially if you don't mind overlaps. But if you want to be especially geeky, create random squares and solve for the packing problem.
Building off Philippe Beaudoin answer.
There are treemap implementations in other languages that you can also use. In Ruby with RubyTreeMap you could do
require 'Treemap'
require 'Treemap/image_output.rb'
root = Treemap::Node.new 0.upto(100){|i| root.new_child(:size => rand) }
output = Treemap::ImageOutput.new do |o|
o.width = 800
o.height = 600
end
output.to_png(root, "C:/output/test.png")
However it sorts the rectangles, so it doesn't look very random, but it could be a start. See rubytreemap.rubyforge.org/docs/index.html for more info
I would generate everything in a spiral slowly going in. If at any point you reach a point where your solution is proven to be 'unsolvable' (IE, can't put any squares in the remaining middle to satisfy the constraints), go to an earlier draft and change some square until you find a happy solution.
Pseudocode would look something like:
public Board GenerateSquares(direction, board, prevSquare)
{
Rectangle[] rs = generateAllPossibleNextRectangles(direction, prevSquare, board);
for(/*all possible next rectangles in some random order*/)){
if(board.add(rs[x]){
//see if you need to change direction)
Board nBoard = GenerateSquares(direction, board, rs[x]);
if(nBoard != null) return nBoard; //done
else board.remove(rs[x]);
}
}
//all possibilities tried, none worked
return null;
}
}
I suggest:
Start by setting up a polygon with four vertices to be eaten in varying size (up to maxside) rectangle lumps:
public double[] fillBoard(double width, double height, double maxside) {
double[] dest = new int[0];
double[] poly = new int[10];
poly[0] = 0; poly[1] = 0; poly[2] = width; poly[3] = 0;
poly[4] = width; poly[5] = height; poly[6] = 0; poly[7] = height;
poly[8] = 0; poly[9] = 0;
...
return dest; /* x,y pairs */
}
Then choose a random vertex, find polygon lines within (inclusive) 2 X maxside of the line.
Find x values of all vertical lines and y values of all horizontal lines. Create ratings for the "goodness" of choosing each x and y value, and equations to generate ratings for values in between the values. Goodness is measured as reducing number of lines in remaining polygon. Generate three options for each range of values between two x coordinates or two y coordinates, using pseudo-random generator. Rate and choose pairs of x and pair of y values on weighted average basis leaning towards good options. Apply new rectangle to list by cutting its shape from the poly array and adding rectangle coordinates to the dest array.
Question does not state a minimum side parameter. But if one is needed, algorithm should (upon hitting a hitch with a gap being too small) not include too small candidates in selection lists (whic will occasionally make them empty) and deselect a number of the surrounding rectangles in a certain radius of the problem with size and perform new regeneration attempts of that area, and hopefully the problem area, until the criteria are met. Recursion can remove progressively larger areas if a smaller relaying of tiles fails.
EDIT
Do some hit testing to eliminate potential overlaps. And eat some spinach before starting the typing. ;)
Define input area;
Draw vertical lines at several random horizontal locations through the entire height;
Draw horizontal lines at several vertical positions through the entire width;
Shift some "columns" up or down by arbitrary amounts;
Shift some "rows" left or right by arbitrary amounts (it may be required to subdivide some cells to obtain full horizontal seams;
Remove seams as aesthetically required.
This graphical method has similarities to Brian's answer.

Categories