"OCR" characters/short sequences of text/numbers - c#

here is my problem: I have to identify numbers (such as 853 / 52) and some text (containing around 8 letters of the alphabet) from a bitmap and i have to do that really fast.
Tesseract does the trick, but its execution time is a bit too slow for my liking. Since i have such a limited amount of characters that are always of the same font size and same font, i was thinking i could just extract them all and build a lookup table for certain characteristics of one character.
Yet to achieve this i would have to be able to "split" up a bitmap containing lets say 853 into its individual characters (kinda box them as some of those OCR trainers do).
Unfortunately i have no idea, how to start boxing/seperating them.. Any suggestions would be appreciated.

Thank you for the arictle.
I kinda solved half my problem.. If i use Aforge i can run them through a set of filters, in my case i increase contrast before i grayscale and binarize, and then run blob extraction on them, which allows me to chop up the picture. Now i have a clean set of character images that i will "only" have to match against comparitve ones.

Related

Downsample a double array to another length

I have a data source that provides many (4096) double values in an array. These are measured with high resolution and are the result of a FFT. For visualisation purposes, they need to be reduced. (Reapplying the FFT on the raw signal is not possible here.) I could simply average each n samples and have a resulting array of length / n values. To allow more flexible selection of the number of resulting values, I need interpolation, though.
I've looked up some basic information about this on Wikipedia. I am already familiar with 2D downsampling/interpolation from a user prespective in raster image editors. Now I need this in 1D in C# code. Think of it as changing (reducing) the raster image size of a 1D barcode image, or resampling an audio wave file.
One library I've found recommended is Math.NET Numerics. This is already used for other tasks in my application, so I could easily use that. There's the CubicSpline class in there but I have no idea how to use it.
Q: What would be an approach to reduce the number of samples in a double[] to an arbitrary number using interpolation?
I'm not interested in finding a single double value between two others. I need to combine multiple source values into each a single output value, while not losing any information (single frequency bins with an extreme level) at the boundaries of the groups, and without aliasing effects or rounding because of different group sizes if the numbers aren't divisible.
Maybe the use of bitmap image functions and a 1*n bitmap size is a good solution instead of dealing with the math directly? This would involve a lot of data conversion, though, which reduces performance and probably also precision. Or some library from the autio processing field?

Why don't some images show proper Fourier and Inverse spectrums?

I am trying to develop an application for image processing.
Here is my complete code in DotNetFiddle.
I have tested my application with different images from the Internet:
Cameraman is GIF.
Baboon is PNG.
Butterfly is PNG.
Pheasant is JPG.
Butterfly and Pheasant are re-sized to 300x300.
The following two images show correct Fourier and Inverse Fourier spectrum:
The following two images do not show the expected outcome:
What could be the reason?
Are there any problem with the later two images?
Do we need to use images of specific quality to test Image-processing applications?
The code you linked to is a radix-2 FFT implementation which would work for any image with sizes that are exact powers of 2.
Incidentally, the Cameraman image is 256 x 256 (powers of 2) and the Baboon image is 512 x 512 (again powers of 2). The other two images, being resized to 300 x 300 are not powers of 2. After resizing those images to an exact power of 2 (for example 256 or 512), the output of FrequencyPlot for the brightness component of the last two images should look somewhat like the following:
butterfly
pheasant
A common workaround for images of other sizes is to pad the image to sizes that are exact powers of 2. Otherwise, if you must process arbitrary sized images, you should consider other 2D discrete Fourier transform (DFT) algorithms or libraries which will often support sizes that are the product of small primes.
Note that for the purpose of validating your output, you also have option to use the direct DFT formula (though you should not expect the same performance).
I got not time to dig through your code. Like I said in my comments you should focus on the difference between those images.
There is no reason why you should not be able to calculate the FFT of one image and fail for another. Unless you have some problem in your code that can't handle some difference between those images. If you can display them you should be able to process them.
So the first thing that catches my eye is that both images you succeed with have even dimensions while the images your algorithm produces garbage for have at least one odd dimension. I won't look into it any further as from experience I'm pretty confident that this causes your issue.
So befor you do anything else:
Take one of those images that work fine, remove one line or row and see if you get a good result. Then fix your code.

16bit greyscale image to heatmap

I'm working on a scientific imaging software for my university, and I've encountered a major problem. Scientific camera (Apogee Alta U57) at my lab provides images as 16bpp array - it's 0-65535 values per pixel! We want to keep this range, but in fact we can't display them on monitor (0-255 grayscale range). So I found a way to resolve this problem - simply to make use of colors, and to display whole image as a heatmap (from black, blue, through green and red, to pure white).
I mean something like this - Example heatmap image I want to achieve
My only question is: How to efficiently convert 16bpp array of pixel values to complete heatmap bitmap in c#? Are there any libraries for doing that? If not, how do I achieve that using .NET resources?
My idea was to create function that maps 65536 values into (255 R, 255G, 255B), but it's a tough job - especially without using HSV model.
I would be much obliged for any help provided!
Your question consist of several parts:
reading in the 16 bit pixel data values
mapping them to 24 bit rgb colors
writing them out to an image file
I'll skip part one and three and give you a few ideas about part 2.
It is in fact harder than it seems. A unique mapping that doesn't lose any information is simple, in fact trivial, just a little bit shifting will do.
But you also want the result to work visually, meaning not so much is should be visually appealing but should make sense to a human eye. so we need a mapping that has a credible yet large enough gradient.
For this you should experiment a little. I suggest to make use of the LinearGradientBrush, as I show here. Have a look at the interpolateColors function! It uses only 6 colors in the example, way to few for your case!
You should pick many more; you may need to go through the color space in a spiral..
The trick for you will be to choose both nice and enough stop colors to create a 64k large set of unique colors, best going from blueish to reddish..
You will need to test the result for uniqueness; in fact you may want to create a pair of Dictionary and Dictionary for the mappings..

C# Scanning an image for a certain block

I have a picture:
size of 1000x1000 white with random black dots. (It may contain a black square (size 50x50))
Is there an easy way to know if the picture contains a black square (size 50x50)? I thought of scanning every pixel of the picture and if a black pixel was found, scan the one next to him till I get a 50x50 square or till I get a white pixel and keep scanning. but it will have to scan over a million pixel (if he hasn't found the square).
Thats the basically the code (no need to complete it, as I said before, it will take way too much time to scan it. million times if the whole picture is white and a lot more according to the number of black pixels.)
for (int i = 0; i < pic.Width; i++)
{
for (int j = 0; j < pic.Height; j++)
{
if (pic.GetPixel(i, j) == Color.Black)
{
//Search for the entire square at that area
}
}
}
And yes, time is important (thats why I don't want to get pixel over a million times). Any ideas?
Like the Boyer-Moore string searching algorithm, if the item you are looking at is not part of what you are looking for, you can skip the whole size of the what you are looking for. In your case, you can check to see if a given pixel is black. If it's not, you can skip forward 50 pixels. If it is, you have a small box to look for your black square in.
In this case, though, you may not need anything so complicated. I'm guessing that if your algorithm is too slow it's because you're calling the GetPixel function a million times, and that is the slow part. If you can get your pixels into a 2D array, your algorithm will probably go fast enough to not need to be rewritten.
Assuming that you're using a System.Drawing.Bitmap check out the LockBits documentation to see a small sample that includes copying a bitmap to a 1D array for super-fast access.
If you are only looking for a square of a specific size, then you can optimise this by only scanning every 50th row (or column) of pixels, thereby cutting your workload dramatically.
Theoretically, you only need to check for a black/white in 1 pixel from every 50x50 block. If it's black, then you try spreading from there, if it's white then just skip to the next block: clearly there is a white pixel in this block so there isn't a black box here. Following this, you've already cut your work to 1 / 2500th of what it was originally, you are now only checking 400 pixels initially.
Usually the best optimisations, especially when first designing an algorithm, are around reducing work done rather than doing it more efficiently. Try to think of creative ways to reducing the input to a more manageable size.
check out AForge.net framework suite. It has imaging library with Blob and pattern searching. You can search shapes too. Its free.
You can find it here http://www.aforgenet.com/framework/
Here is the link that lists features
http://www.aforgenet.com/framework/features
Edit
Here is the sample for shape checking. I have used Aforge in a prototype and it worked for me.
http://www.aforgenet.com/articles/shape_checker/

How to check if an image is a scaled version of another image

I am looking for an EASY way to check if an image is a scaled version of another image. It does not have to be very fast, it just should be "fairly" accurate. And written in .NET. And for free.
I know, wishful thinking :-)
I am pretty sure, even without having tried it, that converting the bigger image to the smaller scale and comparing checksums is not working (especially if the smaller version was done with another software then .NET).
The next approach would be to scale down and compare pixels. But first of all, it seems like a really bad idea running a loop over all pixels with a bool comparison results, I am sure there will be some pixels off by a bit or so...
Any library coming to mind? Way back in the university we had some MPEG7 classes, so I am thinking about using a combination of "statistics" like tone distribution, brightness, etc..
Any ideas or links for that topic?
Thanks,
Chris
I think this is going to be your best solution. First check the aspect ratio. Then scale the images to the smaller of the 2 if they're not the same size. Finally, do a hash comparison of the 2 images. This is a lot faster than doing a pixel compare. I found the hash compare method in a post from someone else and just adapted the answer here to fit. I was trying to think of the best way to do this myself for a project where I'm going to have to compare over 5200 images. After I read a few of the posts here I realized I already had everything I needed for it and figured I'd share.
public class CompareImages2
{
public enum CompareResult
{
ciCompareOk,
ciPixelMismatch,
ciAspectMismatch
};
public static CompareResult Compare(Bitmap bmp1, Bitmap bmp2)
{
CompareResult cr = CompareResult.ciCompareOk;
//Test to see if we have the same size of image
if (bmp1.Size.Height / bmp1.Size.Width == bmp2.Size.Height / bmp2.Size.Width)
{
if (bmp1.Size != bmp2.Size)
{
if (bmp1.Size.Height > bmp2.Size.Height)
{
bmp1 = (new Bitmap(bmp1, bmp2.Size));
}
else if (bmp1.Size.Height < bmp2.Size.Height)
{
bmp2 = (new Bitmap(bmp2, bmp1.Size));
}
}
//Convert each image to a byte array
System.Drawing.ImageConverter ic = new System.Drawing.ImageConverter();
byte[] btImage1 = new byte[1];
btImage1 = (byte[])ic.ConvertTo(bmp1, btImage1.GetType());
byte[] btImage2 = new byte[1];
btImage2 = (byte[])ic.ConvertTo(bmp2, btImage2.GetType());
//Compute a hash for each image
SHA256Managed shaM = new SHA256Managed();
byte[] hash1 = shaM.ComputeHash(btImage1);
byte[] hash2 = shaM.ComputeHash(btImage2);
//Compare the hash values
for (int i = 0; i < hash1.Length && i < hash2.Length && cr == CompareResult.ciCompareOk; i++)
{
if (hash1[i] != hash2[i])
cr = CompareResult.ciPixelMismatch;
}
}
else cr = CompareResult.ciAspectMismatch;
return cr;
}
}
One idea to achieve this:
If the image is 10x10, and your original is 40x40
Loop each pixel in the 10x10, then retrieve the 4 pixels representative of that looped pixel.
So for each pixel in the smaller image, find the corresponding scaled amount of pixels in the larger image.
You can then take the average colour of the 4 pixels, and compare with the pixel in the smaller image. You can specify error bounds, IE -10% or +10% bounds are considered a match, others are considered a failure.
Build up a count of matches and failures and use the bounds to determine if it is considered a match or not.
I think this might perform better than scaling the image to the same size and doing a 1pixel:1pixel comparison as I'm not sure how resizing algorithms necesserially work and you might lose some detail which will give less accurate results. Or if there might be different ways and methods of resizing images. But, again I don't know how the resize might work depends on how you go about doing it.
Just scale the larger image back to the size of the smaller one, then compare each pixel by taking the absolute value of the difference in each of the red, green and blue components.
You can then set a threshold for deciding how close you need to be to count it as a match, e.g. if 95%+ of the pixels are within 5% of the colour value, you have a match.
The fuzzy match is necessary because you may have scaling artefacts / anti-aliasing effects.
You'll have to loop over the pixels at some point or another.
Something that is easy to implement yet quite powerful is to calculate the difference between individual color components (RGB) for each pixel, find the average, and see if it crosses a certain threshold. It's certainly not the best method, but for a quick check it should do.
I'd have said roughly what Tom Gullen except I'd just scale down the bigger image to the smaller before comparing (otherwise you're just going to have hard maths if you are comparing a 25x25 with a 30x30 or something).
The other thing I might consider depending on image sizes is to scale them both down to a smaller image. ie if you have one that is 4000x4000 and another that is 3000x3000 then you can scale them both down to 200x200 and compare them at that size.
As others have said you would then need to do a check with a threshold (preferably on colour components) and decide what tolerances work best. I'd suggest this is probably best done by trial and error.
The easiest way is just to scale the biggest image to the smaller images size and compare color difference. Since you don't know if the scaling is cubic or linear (or something else) you have to accept a small difference.
Don't forget to take the absolute value of each pixel difference. ;)
Having absolutely no authority or experience in this area I'm going to make a stab at helping you.
I'd start with the aspect ratio matching by some tolerance, unless you're comparing cropped sections of images, which will makes things a bit harder.
I'd then scan the pixels for regions of similarity, no exactness, again a tolerance level is needed. Then when an area is similar, run along in a straight line comparing one to the other, and find another similarly coloured area. Black & white's gonna be harder.
If you get a hit, you'll have two areas in a line with patches of likeness. With two points you have a reference of length between them and so now you can see what the scaling might be. You could also scale the images first, but this doesn't account for cropped sections where aspects don't match.
Now choose a random point in the source image and get the colour info. Then using the scale factor, find that same random point on the other image and see if the colour checks out. Do it a few times with random points. If many turn up similar it's likely a copy.
You might then want to mark it for further, more CPU intensive, inspection. Either a pixel by pixel comparison or something else.
I know Microsoft (Photosynth) use filters like "outline" (the sort of stuff in Photoshop) to remove the image colours and leave just squrly lines which leave just the 'components' of the picture for matching (they match boundaries and overlap).
For speed, I'd break the problem down into chunks and really think about how humans decide two photos are similar. For non-speed, exhaustively comparing colour will probably get you there.
The process in short:
If you hole punched a sheet of paper randomly 4 times, then put it over two photos, just by seeing the colours coming through you could tell if they were likely a copy and need further inspection.

Categories