We Keep Coding

Plot values of a line from a grayscale image

I am trying to take a grayscale bitmap and extract a single line from it and then graph the gray values. I got something to work, but I'm not really happy with it. It just seems slow and tedious. I am sure someone has a better idea
WriteableBitmap someImg; //camera image
int imgWidth = someImg.PixelWidth;
int imgHeight = someImg.PixelHeight;
Int32Rect rectLine = new Int32Rect(0, imgHeight / 2, imgWidth, 1); //horizontal line half way down the image as a rectangle with height 1
//calculate stride and buffer size
int imgStride = (imgWidth * someImg.Format.BitsPerPixel + 7) / 8; // not sure I understand this part
byte[] buffer = new byte[imgStride * rectLine.Height];
//copy pixels to buffer
someImg.CopyPixels(rectLine, buffer, imgStride, 0);
const int xGraphHeight = 256;
WriteableBitmap xgraph = new WriteableBitmap(imgWidth, xGraphHeight, someImg.DpiX, someImg.DpiY, PixelFormats.Gray8, null);
//loop through pixels
for (int i = 0; i < imgWidth; i++)
{
Int32Rect dot = new Int32Rect(i, buffer[i], 1, 1); //1x1 rectangle
byte[] WhiteDotByte = { 255 }; //white
xgraph.WritePixels(dot, WhiteDotByte, imgStride, 0);//write pixel
}
You can see the image and the plot below the green line. I guess I am having some WPF issues that make it look funny but that's a problem for another post.

I assume the goal is to create a plot of the pixel value intensities of the selected line.
The first approach to consider it to use an actual plotting library. I have used oxyplot, it works fine, but is lacking in some aspects. Unless you have specific performance requirements this will likely be the most flexible approach to take.
If you actually want to render to an image you might be better of using unsafe code to access the pixel values directly. For example:
xgraph.Lock();
for (int y = 0; y < imgHeight; y++){
var rowPtr = (byte*)(xgraph.BackBuffer + y * xgraph.BackBufferStride);
for(int x = 0; x < imgWidth; x++){
rowPtr[x] = (byte)(y < buffer[i] ? 0 : 255);
}
}
self.Unlock(); // this should be placed in a finally statement
This should be faster than writing 1x1 rectangles. It should also write columns instead of single pixels, and that should help making the graph more visible. You might also consider allowing arbitrary image height and scale the comparison value.
If you want to plot the pixel values along an arbitrary line, and not just a horizontal one. You can take equidistant samples along the line, and use bilinear interpolation to sample the image.

How do I get a partial image from a raw byte array of colors?

So I want to grab a partial image from a byte array of colors. The image is a unity logo that is 64x64 pixels. I want to grab a third of the image (Unity Logo). How would I traverse the byte array to get this image?
Unity Byte Array

assuming each byte is a single pixel (which is only true for 8-bit depth images), the bytes 0-63 are the first row, 64-127 are the second row, etc etc.
meaning that to find out the position of a pixel in the one-dimensional array, based on its two-dimensional coordinates in the image itself, you do
int oneDimPos = (y*64) + x;
if each pixel were 3 bytes (24-bit color depth), the conversion from 2dimensional to 1dimensional coordinates would be:
int oneDimPos = (y * 64 * 3) + (x * 3);
(so the most generic equation is:
int oneDimPos = (y * imageWidth * colorDepth) + (x * colorDepth);
and you need to keep this in mind and adjust the code accordingly. or even better, use this most generic version, and actually read the image width and its color depth from the asset you're using as source.
BEWARE: if the image is anything else than 8bits per pixel, this equation will, naturally, only give you the first, starting bit belonging to that pixel, and you still need to take care to actually also read the other ones that belong to that pixel
i'm gonna finish the answer assuming 8bit color depth, for simplicity, as well as so that you can't just copypaste the answer, but also have to understand it and re-shape it according to your specific needs ;)
)
meaning you can now do classic two nested loops for x and y:
List<byte> result = new List(); //i'm going to use list so i can just .Add each byte instead of having to calculate and allocate the final size in advance, and having to mess around with recalculating the index from the source array into the destination one, because i'm lazy
for(int x=0; x < 22; x++){ //no way for you to grab precise third since that boundary is in the middle of a pixel for an image 64pixels wide
for(int y = 0; y < 64; y++){ //we go all the way to the bottom
result.Add(sourceAsset.bytes[(y*64) + x]);
}
}
//now just convert the list to actual byte array
byte[] resultBytes = result.ToArray();

The original issue that I was having was not exactly the same as the question. I wanted to simplify it by having a byte array that everyone could take a look at. The byte array from Unity's website wasn't exactly what I was getting.
So I have 3 x 1080p portrait screen (1080 x 1920 pixels) with RGBA channels. I grabbed a screenshot from this and got a 24,883,200 size byte array.
Note, 3 * width(1080) * height(1920) * channels(4) = 24,883,200.
byte[] colors = new byte[24883200]; // Screenshot of 3x1080p screen.
byte[] leftThird = new byte[colors.Length / 3];
Array.Copy(colors, 0, leftThird, 0, colors.Length / 3); // Grab the first third of array
This is an issue because the colors array is read from top to bottom, left to right. So instead, you should read a portion of the 3 x 1080 x 4 channels.
int width = 1080 * 4; // 4 channels of colors (RGBA)
int fullWidth = width * 3; // Three screens
int height = 1920;
byte[] leftScreen = new byte[screenShotByteArray.Length / 3];
for(int i = 0; i < height; i++)
{
Array.Copy(screenShotByteArray, (i * fullWidth) + (offset * 4), leftScreen, i * width, width);
}

How to split an image into smaller pieces in C#?

How can I split an image into multiple sub images? Like, would I have to read the pixels and convert it to an image somehow?
For example:
If an image's dimension is 100px (width) and 180px (height) and I wanted to, say, split it as a 4x4, would I read the first 25px for the width and the first 45 px for the height and then just increment it correctly?
If so, what would I store the pixels to? More specifically, would it be saved as an array of bytes, images, etc?

You may try the following code sample (taken from https://stackoverflow.com/a/4118195/);
for (int i = 0; i < 4; i++)
{
for (int y = 0; y < 4; y++)
{
Rectangle r = new Rectangle(i*(pictureBox1.Image.Width / 4),
y*(pictureBox1.Image.Height / 4),
pictureBox1.Image.Width / 4,
pictureBox1.Image.Height / 4);
g.DrawRectangle(pen,r );
list.Add(cropImage(pictureBox1.Image, r));
}
}
The other alternative is using BitMap.Clone, you may find an example in the following link.

Use the Bitmap class to hold the image and its Clone method to cut out your rectangles of arbitrary size. As a Bitmap it comes with several convenience methods such as Save this overload will save to a stream another allows you to save it to a file.

Rasterizing image with colored fixed-width chars

uI am making a program to turn an image into coloured 0's, the problem is that the 0's are not colouring properly. To get anything near resembling the image I have to start my for loop at 2 and increase by 3 each time. The following is my current code:
public partial class MainWindow : Window
{
public MainWindow()
{
TextSelection textRange;
TextPointer start;
TextPointer startPos;
TextPointer endPos;
System.Drawing.Color x;
int pixelX = 3;
int pixelY = 8;
InitializeComponent();
Bitmap b = new Bitmap(#"E:\Documents\Visual Studio 2015\Projects\RichTextBox Image to ASCII\RichTextBox Image to ASCII\Akarin.jpg");
for (int i = 2; i < 8000; i += 3)
{
textRange = richTextBox1.Selection;
start = richTextBox1.Document.ContentStart;
startPos = start.GetPositionAtOffset(i);
endPos = start.GetPositionAtOffset(i + 1);
textRange.Select(startPos, endPos);
x = b.GetPixel(pixelX, pixelY);
textRange.ApplyPropertyValue(TextElement.ForegroundProperty, new SolidColorBrush(System.Windows.Media.Color.FromArgb(x.A, x.R, x.G, x.B)));
pixelX += 6;
if (pixelX > 1267)
{
pixelX = 3;
pixelY += 16;
}
i += 3;
textRange = richTextBox1.Selection;
start = richTextBox1.Document.ContentStart;
startPos = start.GetPositionAtOffset(i);
endPos = start.GetPositionAtOffset(i + 1);
textRange.Select(startPos, endPos);
x = b.GetPixel(pixelX, pixelY);
textRange.ApplyPropertyValue(TextElement.ForegroundProperty, new SolidColorBrush(System.Windows.Media.Color.FromArgb(x.A, x.R, x.G, x.B)));
pixelX += 7;
if (pixelX > 1267)
{
pixelX = 3;
pixelY += 16;
}
}
}
}
The reason that I am putting the code in the for loop twice is because when you take the amount of 0's that fit horizontally and find out how many pixels each 0 takes up, it comes to about 6.5 because of the space between each 0.
EDIT: Something else that is also strange, if you look in the top left corner where it starts colouring the 0's, 4 in a row are properly coloured, but then the rest are coloured every other.

A few serious problems I'm seeing here. Normally when rasterizing you either loop through the source pixels or through the target pixels. You however... you loop by a fixed value of roughly 2666 ((8000 - 2) / 3). It's also a very bad idea to do things twice in a loop and even change the loop variable (i). Furthermore since you're having only one loop you have to care about both axes in one run. This is very error prone.
How about this approach?:
Your source image is 1280 × 720 square pixels
Since your zeros are not square you have to know their aspect ratio. If you know that you can calculate how many rows and columns you need. You probably don't want to match them 1:1 as this would give you a huge and stretched image.
Once you know how many rows and columns you need, do two loops, one inside the other and call the loop variables targetX and targetY
If your target image is supposed to be let's say 400 zeroes long in the x-axis, make the first loop go from 1 to 400
Inside the loop pick one pixel (color) from the source at 1280/400 * targetX. Your first target pixel would be at x position 1280/400 * 1 = 3,2 which is roughly 3 (round the number after calculating it). The second would be 1280/400 * 2 = 6 and so on. I think this is the biggest pain in your algorithm since you're trying to get around the 6,5px width. Just round it after calculating! If the first is 6,5, make it 7, the second is 13... you get the idea.
Same logic goes for Y axis, but you handle this with targetY.

Is there an efficient algorithm for segmentation of handwritten text?

I want to automatically divide an image of ancient handwritten text by lines (and by words in future).
The first obvious part is preprocessing the image...
I'm just using a simple digitization (based on brightness of pixel). After that I store data into two-dimensional array.
The next obvious part is analyzing the binary array.
My first algorithm was pretty simple - if there are more black pixels in a row of the array than the root-mean-square of Maximum and Minimum value, then this row is part of line.
After forming the list of lines I cut off lines with height that is less than average.
Finally it turned out into some kind of linear regression, trying to minimize the difference between the blank rows and text rows. (I assumed that fact)
My second attempt - I tried to use GA with several fitness functions.
The chromosome contained 3 values - xo, x1, x2. xo [-1;0] x1 [0;0.5] x2 [0;0.5]
Function, that determines identity the row to line is (xo + α1 x1 + α2 x2) > 0, where α1 is scaled sum of black pixels in row, α2 is median value of ranges between the extreme black pixels in row. (a1,a2 [0,1])
Another functions, that I tried is (x1 < α1 OR x2 > α2) and (1/xo + [a1 x1] / [a2 x2] ) > 0
The last function is the most efficient.
The fitness function is
(1 / (HeigthRange + SpacesRange)
Where range is difference between maximum and minimum. It represents the homogeneity of text. The global optimum of this function - the most smooth way to divide the image into lines.
I am using C# with my self-coded GA (classical, with 2-point crossover, gray-code chromosomes, maximum population is 40, mutation rate is 0.05)
Now I ran out of ideas how to divide this image into lines with ~100% accuracy.
What is the efficient algorithm to do this?
UPDATE:
Original BMP (1.3 MB)
UPDATE2:
Improved results on this text to 100%
How I did it:
fixed minor bug in range count
changed fitness function to 1/(distancesRange+1)*(heightsRange+1))
minimized classifying function to (1/xo + x2/range) > 0 (points in row now don't affect classification)
(i.e. optimized input data and made fitness function optimizations more explicit)
Problem:
GA surprisingly failed to recognize this line. I looked at debug data of 'find rages' function and found, that there is too much noise in 'unrecognized' place.
The function code is below:
public double[] Ranges()
{
var ranges = new double[_original.Height];
for (int y = 0; y < _original.Height; y++ )
{
ranges[y] = 0;
var dx = new List<int>();
int last = 0;
int x = 0;
while (last == 0 && x<_original.Width)
{
if (_bit[x, y])
last = x;
x++;
}
if (last == 0)
{
ranges[y] = 0;
continue;
}
for (x = last; x<_original.Width; x++)
{
if (!_bit[x, y]) continue;
if (last != x - 1)
{
dx.Add((x-last)+1);
}
last = x;
}
if (dx.Count > 2)
{
dx.Sort();
ranges[y] = dx[dx.Count / 2];
//ranges[y] = dx.Average();
}
else
ranges[y] = 0;
}
var maximum = ranges.Max();
for (int i = 0; i < ranges.Length; i++)
{
if (Math.Abs(ranges[i] - 0) < 0.9)
ranges[i] = maximum;
}
return ranges;
}
I'm using some hacks in this code. The main reason - I want to minimize the range between nearest black pixels, but if there are no pixels, the value becomes '0', and it becomes impossible to solve this problem with finding optimas. The second reason - this code is changing too frequently.
I'll try to fully change this code, but I have no idea how to do it.
Q:
If there is more efficient fitness function?
How to find more versatile determination function?

Although I'm not sure how to translate the following algorithm into GA (and I'm not sure why you need to use GA for this problem), and I could be off base in proposing it, here goes.
The simple technique I would propose is to count the number of black pixels per row. (Actually it's the dark pixel density per row.) This requires very few operations, and with a few additional calculations it's not difficult to find peaks in the pixel-sum histogram.
A raw histogram will look something like this, where the profile along the left side shows the number of dark pixels in a row. For visibility, the actual count is normalized to stretch out to x = 200.
After some additional, simple processing is added (described below), we can generate a histogram like this that can be clipped at some threshold value. What remains are peaks indicating the center of lines of text.
From there it's a simple matter to find the lines: just clip (threshold) the histogram at some value such as 1/2 or 2/3 the maximum, and optionally check that the width of the peak at your clipping threshold is some minimum value w.
One implementation of the full (yet still simple!) algorithm to find the nicer histogram is as follows:
Binarize the image using a "moving average" threshold or similar local thresholding technique in case a standard Otsu threshold operating on pixels near edges isn't satisfactory. Or, if you have a nice black-on-white image, just use 128 as your binarization threshold.
Create an array to store your histogram. This array's length will be the height of the image.
For each pixel (x,y) in the binarized image, find the number of dark pixels above and below (x,y) at some radius R. That is, count the number of dark pixels from (x, y - R) to x (y + R), inclusive.
If the number of dark pixels within a vertical radius R is equal or greater to R--that is, at least half the pixels are dark--then pixel (x,y) has sufficient vertical dark neighbors. Increment your bin count for row y.
As you march along each row, track the leftmost and rightmost x-values for pixels with sufficient neighbors. As long as the width (right - left + 1) exceeds some minimum value, divide the total count of dark pixels by this width. This normalizes the count to ensure the short lines like the very last line of text are included.
(Optional) Smooth the resulting histogram. I just used the mean over 3 rows.
The "vertical count" (step 3) eliminates horizontal strokes that happen to be located above or below the center line of text. A more sophisticated algorithm would just check directly above and below (x,y), but also to the upper left, upper right, lower left, and lower right.
With my rather crude implementation in C# I was able to process the image in less than 75 milliseconds. In C++, and with some basic optimization, I've little doubt the time could be cut down considerably.
This histogram method assumes the text is horizontal. Since the algorithm is reasonably fast, you may have enough time to calculate pixel count histograms at increments of every 5 degrees from the horizontal. The scan orientation with the greatest peak/valley differences would indicate the rotation.
I'm not familiar with GA terminology, but if what I've suggested is of some value I'm sure you can translate it into GA terms. In any case, I was interested in this problem anyway, so I might as well share.
EDIT: maybe for use GA, it's better to think in terms of "distance since previous dark pixel in X" (or along angle theta) and "distance since previous dark pixel in Y" (or along angle [theta - pi/2]). You might also check distance from white pixel to dark pixel in all radial directions (to find loops).
byte[,] arr = get2DArrayFromBitamp(); //source array from originalBitmap
int w = arr.GetLength(0); //width of 2D array
int h = arr.GetLength(1); //height of 2D array
//we can use a second 2D array of dark pixels that belong to vertical strokes
byte[,] bytes = new byte[w, h]; //dark pixels in vertical strokes
//initial morph
int r = 4; //radius to check for dark pixels
int count = 0; //number of dark pixels within radius
//fill the bytes[,] array only with pixels belonging to vertical strokes
for (int x = 0; x < w; x++)
{
//for the first r rows, just set pixels to white
for (int y = 0; y < r; y++)
{
bytes[x, y] = 255;
}
//assume pixels of value < 128 are dark pixels in text
for (int y = r; y < h - r - 1; y++)
{
count = 0;
//count the dark pixels above and below (x,y)
//total range of check is 2r, from -r to +r
for (int j = -r; j <= r; j++)
{
if (arr[x, y + j] < 128) count++;
}
//if half the pixels are dark, [x,y] is part of vertical stroke
bytes[x, y] = count >= r ? (byte)0 : (byte)255;
}
//for the last r rows, just set pixels to white
for (int y = h - r - 1; y < h; y++)
{
bytes[x, y] = 255;
}
}
//count the number of valid dark pixels in each row
float max = 0;
float[] bins = new float[h]; //normalized "dark pixel strength" for all h rows
int left, right, width; //leftmost and rightmost dark pixels in row
bool dark = false; //tracking variable
for (int y = 0; y < h; y++)
{
//initialize values at beginning of loop iteration
left = 0;
right = 0;
width = 100;
for (int x = 0; x < w; x++)
{
//use value of 128 as threshold between light and dark
dark = bytes[x, y] < 128;
//increment bin if pixel is dark
bins[y] += dark ? 1 : 0;
//update leftmost and rightmost dark pixels
if (dark)
{
if (left == 0) left = x;
if (x > right) right = x;
}
}
width = right - left + 1;
//for bins with few pixels, treat them as empty
if (bins[y] < 10) bins[y] = 0;
//normalize value according to width
//divide bin count by width (leftmost to rightmost)
bins[y] /= width;
//calculate the maximum bin value so that bins can be scaled when drawn
if (bins[y] > max) max = bins[y];
}
//calculated the smoothed value of each bin i by averaging bin i-1, i, and i+1
float[] smooth = new float[bins.Length];
smooth[0] = bins[0];
smooth[smooth.Length - 1] = bins[bins.Length - 1];
for (int i = 1; i < bins.Length - 1; i++)
{
smooth[i] = (bins[i - 1] + bins[i] + bins[i + 1])/3;
}
//create a new bitmap based on the original bitmap, then draw bins on top
Bitmap bmp = new Bitmap(originalBitmap);
using (Graphics gr = Graphics.FromImage(bmp))
{
for (int y = 0; y < bins.Length; y++)
{
//scale each bin so that it is drawn 200 pixels wide from the left edge
float value = 200 * (float)smooth[y] / max;
gr.DrawLine(Pens.Red, new PointF(0, y), new PointF(value, y));
}
}
pictureBox1.Image = bmp;

After fiddling around this for a while I found that I simply need to count the number of crossings for each line, that is, a switch from white to black would count as one, and a switch from black to white would increment by one again. By highlighting each line with a count > 66 I got close to 100% accuracy, except for the bottom most line.
Of course, would not be robust to slightly rotated scanned documents. And there is this disadvantage of needing to determine the correct threshold.

IMHO with the image shown that would be so hard to do 100% perfectly.
My answer is to give you alternate idea's.
Idea 1:
Make your own version of ReCaptcha (to put on your very own pron site) - and make it a fun game.. "Like cut out a word (edges should all be white space - with some tolerance for overlapping chars on above and below lines)."
Idea 2:
This was a game we played as kids, the wire of a coat hanger was all bent in waves and connected to a buzzer and you had to navigate a wand with a ring in the end with the wire through it, across one side to the other without making the buzzer go off. Perhaps you could adapt this idea and make a mobile game where people trace out the lines without touching black text (with tolerance for overlapping chars)... when they can do a line they get points and get to new levels where you give them harder images..
Idea 3:
Research how google/recaptcha got around it
Idea 4:
Get the SDK for photoshop and master the functionality of it Extract Edges tool
Idea 5:
Stretch the image heaps on the Y Axis which should help, apply the algorithm, then reduce the location measurements and apply them on the normal sized image.