As a pet project/learning experience (no this is not homework) I'm working on software to recognize barcodes from a photograph. I'm not looking for software or a library that does it - instead I'm using this as a learning exercise that I'm blogging about and will post up on Codeplex.
I have code that successfully recognizes EAN13 barcodes (which I published on CodePlex) and UPC version A/E should follow shortly. I have two areas that I'm concerned about, though. First is in decoding barcodes that are in a picture that is bit blurry or with poor contrast, etc. Second is in simply finding the actual barcode in a larger picture (right now you have to give me a photo of just the barcode).
I have the gut feeling that some form of AI is going to help me out here. I played a bit in the past with genetic algorithms and I took a course ages ago on AI so it's not totally foreign to me, but I'm not quite sure where to start.
What type of algorithm is best suited to this type of problem? Any recommended reading or code for the AI grunt work? Yes, I want to understand what's happening, but I don't necessarily want to go down to the level of coding the sorts, etc myself.
I would suggest to search for properties that a barcode has. Some that I have in mind are:
Histogram of colors shows two distinct colors in about even distribution
Doing a hough transformation finds many parallel lines
The thickness of the lines have two distinct dimensions.
Some other?
Having this I would split the image into pieces and do a classification with these features then cobine the results to calculate a liklyhood if the piece contains an barcode or not.
For your second problem (blurry image) I would suggest to calculate the 1st order derivative of the grayvalues and then detect the edges of the lines in this space. The maximum of the derivative is lower if the image is blurred but it should be detectable to a certain blurring factor.
Does this help you?
As mp already noted you don't necessary need any real AI technique for it. Have a look at chapter 12 of Real World Haskell. It implements an almost complete barcode recognizer. Sample code is in Haskell, but there is plenty of explanation, so you can probably understand the ideas and tricks even without Haskell experience.
If you want to solve it with AI then the best bet is probably using ANNs. For the given problem I would recommend to use a quite advanced technique called HyperNEAT. See my explanation (and links) as the first answer to the SO question Neural Network Size...
I would probably use two or three different networks,
The first one to find the barcode on the bigger picture. One output neuron per pixel/set of pixels, output value is the confidence if that pixel seems to be a part of a barcode. Based on the result I would use some image transformation to convert it to a "standard" format (x*y rectangle)
If you have difficulties with finding the location of the barcode use a second one. Feed the result of the first one, and ask it to give the coordinates of two corners. However, I'm not quite sure that it will be very easy to evolve this one.
Last one would work on the standardized format, output neurons for each line (or square, if you work with a possibly 2D barcode), saying if the given area should be considered black or white.
Probably it would also help to do some pre-processing of the image, e.g. those that are described in RWH.
You don't need any specific AI or softcomputing technique. You need to apply image processing technique to improve the quality of the image or to isolate the barcode from a larger image.
You could use Matlab for prototyping and learnig more about image processing.
Related
Searching for advice: We are rewriting (in c#) the graphical user interface for the Watershed Risk Analysis Management Framework model, and are using the DotSpatial libraries for our map operations. We need to perform some simple tabulations on raster data, and I'm having trouble finding examples. We need to calculate land use (using national land cover dataset) percentages within polygons, calculate average slope and aspect within polygons. Pretty standard stuff for hydrologic analysis. Does anyone know of tutorials or available code sources for DotSpatial raster analysis? Thanks for your time.
did you find a way to do it? I am in the same position. For the moment, my current workaround is this. I have converted my raster into a List<GeoAPI.Geometries.IPoint> listPts using the center coordinates of the pixels, with the Z value as the corresponding raster pixel value. Then, with my PolygonShapefile, I loop over each feature, and use the feature.Geometry.Covers(listPts[i]) methode to build a list of the points failing in each polygons. After that, I simply cross the two lists together to calculates the corresponding statistics that I need.
I would like a better suggestion, but for the moment, it fits my needs.
I try to identify changes on an object. Therefore I take a picture before and after using the object. At the moment I'm working with the absolute Difference of the two pictures and taking the contours of the resulting difference image. That works fine as long as the object is positioned perfectly and captured like in the image before. Only small differences in its position make my method useless.
Has anybody a different solution approach with OpenCV oder EmguCV? I was thinking about checking if one of the neighbor pixels is identical then there should be no change detected, but I don't know of an existing performant algorithm.
Example Images (Pictures don't match my usecase, but they should be helpful to illustrate my problem):
Before
After
Yes there are many way to do this. I like the following:
Histogram match. Get a histogram before and after and check for differences. Is sensitive to changes in lighting. Very good method if you are in a controlled lighting setting
Correlation match. If you use MatchTemplate you can get the “quality” of the match. This can be made to be less sensitive to light. But is sensitive to rotation changes between the two images.
Try to implement some and let’s see your code.
I'm looking for a good way to isolate an air bubble from the following image. I'm using Visual Studio 2015 and C#.
I've heard of the watershed method and believe it may be a good solution.
I tried implementing the code solution found here: watershed image segmentation
I haven't had much success. The solution has trouble finding functions, for example: FilterGrayToGray.
Does anyone know of a good way to do this?
You should just train a Neural network to recognize parts of image when there are no bubbles (in example groups of 16x16 pixels). Then when recognizing a square is not successfull you do a burst of horizontal scanlines and you register where the edge starts and ends. You can determine pretty precisely the section of a bubble (however determine its volume needs to keep into account surface curvature, wich is possible but more hard) on the image. If you have the possibility to use more cameras you can triangulate more sections of a bubble and get a precise idea of real volume. As another euristic to know bubble size you can also use the known volume throughput, so you know that if in a time interval you emitted X liters of air, and the bubbles have sections given in a certain proportion you can redistribute total volume across bubbles and further increase precision (of course you have to keep in mind pressure since bubbles on bottom of the pool will be more little).
As you see you can play with simple algorithms like gaussian difference and contrast to achieve different quality results.
In the left picture you can easily remove all background noise, however you have lost now part of the bubbles. It is possible you can re-gain the missed bubbles edge by using a different illumination on the pool
In the right picture you have the whole bubbles edges, but now you also have more areas that you need to manually discard from picture.
As for edge detections algorithm you should use an algorithm that do not add a fixed offset to edges (like convolution matrix or laplace), for this I think gaussian difference would work best.
Keep all intermediate data so one can easily verify and tweak the algorithm and increase its precision.
EDIT:
The code depends on wich library you use, you can easily implement Gaussian Blur and Horizontal Scanline, for Neural Networks there are already c# solutions out there.
// Do gaussian difference
Image ComputeGaussianDifference (Image img, float r1, float r2){
Image img = img.GaussianBlur( r1);
Image img2 = img.GaussianBlur( r2);
return (img-img2).Normalize(); // make values more noticeable
}
more edits pending.. try do document yourself in the meantime, I already given enough trace to let you do the job, you just need basic understanding of simple image processing algorithms and usage of ready neural networks.
Just in case if you are looking for some fun - you could investigate Application Example: Photo OCR. Basically you train one NN to detect bubble, and try it on a sliding window across the image. When you capture one - you use another NN, which is trained to estimate bubble size or volume (you probably can measure your air stream to train the NN). It is not so difficult as it sounds, and provides very high precision and adaptability.
P.S. Azure ML looks good as a free source of all the bells and whistles without need to go deep.
To solutions come to mind:
Solution 1:
Use the Hough transform for circles.
Solution 2:
In the past I also had a lot of trouble with similar image segmentation tasks. Basically I ended up with a flood fill, which is similar to the watershed algorithm you programmed.
A few hat tricks that I would try here:
Shrink the image.
Use colors. I notice you're just making everything gray; that makes little sense if you have a dark-blue background and black boundaries.
Do you wish to isolate the air bubble in a single image, or track the same air bubble from an image stream?
To isolate a 'bubble' try using a convolution matrix on the image to detect the edges. You should pick the edge detection convolution based on the nature of the image. Here is an example of a laplace edge detection done in gimp, however it is faily straight forward to implement in code.
This can help in isolating the edges of the bubbles.
If you are tracking the same bubble from a stream, this is more difficult due to as the way bubbles distort when flowing through liquid. If the frame rate is high enough it would be easy to see difference from frame to frame and you can judge which bubble it is likely to be (based on positional difference). i.e you would have to compare current frame to previous frame and use some intelligence to attempt to work out which bubble is the same from frame to frame. Using a fiducial to help give a point of reference would be useful too. The nozzle at the bottom of the image might make a good one, as you can generate a signature for it (nozzle won't change shape!) and check that each time. Signatures for the bubbles aren't going to help much since they could change drastically from one image to the next, so instead you would be processing blobs and their likely location in the image from one frame to the next.
For more information on how convolution matrices work see here.
For more information on edge detection see here.
Hope this helps, good luck.
I want to recognize handwriting shape and figure out which shape it probably is in the set. Simply saying, if I draw a triangle, the application should recognize it as an triangle.
How can I do this using C# or java, any help is appreciated.
Thanks in advance.
These are some of the shapes I need to identify
You can try to use OpenCV for that. EmguCV is a good wrapper to OpenCV for .net. Watch for ShapeDetection demo (included in OpenCV)
If you want to "roll your own" I would suggest the following steps:
First, skeletonize (thin out the image till all the lines are one pixel thick). There are many ways to do this, and it is a well studied problem. Google for more information.
Now, starting at a black pixel, go through and trace out the outline of the image, one pixel at a time. You add each of these segments to a list of segments outlining the shape (each segment will be a simple line from one pixel to its adjacent pixel). Now you have the outline of your shape as a many-sided polygon.
(Possible step at this point: smooth the outline by pulling each vertex closer to the average of its neighbors)
Now, you use a corner detection algorithm to find the corners (take a look here:http://visual.ipan.sztaki.hu/corner/node7.html).
This should be enough to identify the shapes you have listed.
If you want to get smarter, you can also identify the types of edges that exist between corners. If the segment between two corners stays within some threshold of the straight line between them, you treat it as a "straight line" edge. If it doesn't, you treat it as a curving edge.
With corners +straight/curving edge, you probably could detect any of the shapes you are looking for pretty well.
I'd suggest using a neural network.
You could teach it what the shapes look like.
This is one library for example:
Neural Networks on C#
If you are looking for particular shapes inside a larger image then OpenCV is a great alternative. Emgu.CV is a good .Net wrapper for it. See my picture of a SURF implementation for this. Also see other options in OpenCV, it has plenty to offer. Note that this approach requires a lot of processing power.
If you can easily identify the shape you want as a BLOB (that is, give the algorithm a picture of only this shape) you can do a search for "ANN OCR" ("Artificial Neural Networks" and "Optical Character Recognition"). Many (most?) ANN-implementations come with sample code for feeding it shapes (letters) and recognizing closest shape (hand written letters). For example Neural Network OCR. I believe this approach would solve your problem. (Sidenote: I've encountered and tested numerous libs that can do this. It's Neural Networks 101.)
If you need BLOB algorithms for the ANN-OCR OpenCV can provide this.
Both these approaches are farily easy to implement.
There is indeed a vast tree of research in shape recognition.
If your shapes are indeed some what predictable and are basic geometry,
the most straightforward way is to find the edges and apply hough transform.
Some managable reading materials for you to start with,
[1] Google Scholar for Hough Transform Shape Detection
http://scholar.google.com/scholar?q=hough+transform+shape+recognition&hl=en&as_sdt=0&as_vis=1&oi=scholart
[2] Hough Transform # Wiki http://en.wikipedia.org/wiki/Hough_transform
image http://prod.triplesign.com/map.jpg
How can I produce a similar output in C# window forms in the easiest way?
Is there a good library for this purpose?
I just needs to be pointed in the direction of which graphic library is best for this.
You should just roll your own in a 3d graphics library. You could use directx. If using WPF it is built-in, you can lookup viewport3d. http://msdn.microsoft.com/en-us/magazine/cc163449.aspx
In graphics programming what you are building is a very simple version of a heightmap. I think building your own would give your greater flexibility in the long run.
So a best library doesn't exist. There are plenty of them and some are just for different purposes. Here a small list of possibilities:
Tao: Make anything yourself with OpenGL
OpenTK: The successor of the Tao framework
Dundas: One of the best but quite expensive (lacks in real time performance)
Nevron: Quite good, but much cheaper (also has problems with real time data)
National Instruments: Expensive, not the best looking ones, but damn good in real time data.
... Probably someone else made some other experiences.
Checkout Microsoft Chart Controls library.
Here's how I'd implement this using OpenGL.
First up, you will need a wrapper to import the OpenGL API into C#. A bit of Googling led me to this:
CsGL - OpenGL .NET
There a few example programs available to demonstrate how the OpenGL interface works. Play around with them to get an idea of how the system works.
To implement the 3D map:
Create an array of vectors (that's not the std::vector/List type but x,y,z triplets) where x and y are along the horizontal plane and z is the up amount.
Set the Z compare to less-than-or-equal (so the overlaid line segments are visible).
Create a list of quads where the vertices of the quads are taken from the array in (1)
Calculate the colour of the quad. Use a dot-product of the quad's normal and a light source direction to get a value to shade value, i.e. normal.light of 1 is black and -1 is white.
Create a list of line segments, again from the array in (1).
Calculate the screen position of the various projected axes points.
Set up your camera and world->view transform (use the example programs to get an idea of how to do this).
Render the quads and lines, OpenGL will do the transformation from world co-ordinates (the list in (1)) to screen space. Draw the labels, you might not want to do this using OpenGL as the labels shouldn't scale with distance from camera, otherwise they could get too small to read.
Since the above is quite a lot of stuff, there isn't really the space (and time on my part) to post working code (but someone else might add something if you're lucky). You could break the task down and ask questions on the parts you don't quite understand.
Have you tried this... gigasoft data visualization tools (Its not free)
And you can checkout the online wireframe demo here