We Keep Coding

Recognize small objects with OpenCV

I would like to be able to recognize the position (center) and the angle of some small components with openCV with C#. To achieve that, I am grabbing pictures from a webcam and try to process them with the Canny algorithm. Unfortunately, the results are not that good as expected. Sometimes it is ok sometimes it is not.
I have attached an example image from the cam and the corresponding output of OpenCV.
I hope that someone could give me hints or maybe some code snippets, how to achieve my desired results. Is this something that is usually done with AI?
Example images:
Input:
Output 1:
Output 2:
Expected:
Thanks.
Actual code:
Mat src;
src = BitmapConverter.ToMat(lastFrame);
Mat dst = new Mat();
Mat dst2 = new Mat();
Cv2.Canny(src, dst, hScrollBar1.Value, hScrollBar2.Value);
// Find contours
OpenCvSharp.Point[][] contours; //vector<vector<Point>> contours;
HierarchyIndex[] hierarchyIndexes; //vector<Vec4i> hierarchy;
Cv2.FindContours(dst, out contours, out hierarchyIndexes, RetrievalModes.External, ContourApproximationModes.ApproxTC89L1);
foreach (OpenCvSharp.Point[] element in contours)
{
var biggestContourRect = Cv2.BoundingRect(element);
Cv2.Rectangle(dst,
new OpenCvSharp.Point(biggestContourRect.X, biggestContourRect.Y),
new OpenCvSharp.Point(biggestContourRect.X + biggestContourRect.Width, biggestContourRect.Y + biggestContourRect.Height),
new Scalar(255, 0, 0), 3);
}
using (new Window("dst image", dst)) ;
using (new Window("src image", src)) ;

If you already have a ROI (the box) and you just want to compute the actual orientation of it, you could use the contour inside the right box and compute its moments. A tutorial on how to do this is here (Sorry only C++).
Once you have the moments you can compute the orientation easily. To do this follow the solution here.
If you have trouble figuring out the right box itself, you are actually half way with canny boxes. You could then further try:
Equalize source image:
Posterize next (to 2 levels):
Threshold (255):
Then you can use all the canny boxes you found in the centre and use them as masks to get the right contour in the thresholded image. You can then find the biggest contour here and compute its orientation with image moments. Hope this helps!

C# - Working with high resolution screenshots

I need to capture an area within my desktop. But I need this area to be very high resolution (like, at least few thousand's pixels horizontal, same goes for vertical). Is it possible to get a screen capture that has high density of pixels? How can I do this? I tried capturing the screen with some AutoIt script, and got some very good results (images that were 350MB big), now I would like to do the same using C#.
Edit:
I am doing my read/write of a .tif file like that, and it already loses most of the data:
using (Bitmap bitmap = (Bitmap)Image.FromFile(#"ScreenShot.tif")) //this file has 350MB
{
using (Bitmap newBitmap = new Bitmap(bitmap))
{
newBitmap.Save("TESTRES.TIF", ImageFormat.Tiff); //now this file has about 60MB, Why?
}
}
I am trying to capture my screen like that, but the best I can get from this is few megabytes (nowhere near 350MB):
using (var bmpScreenCapture = new Bitmap(window[2], window[3], PixelFormat.Format32bppArgb))
{
using (var i = Graphics.FromImage(bmpScreenCapture))
{
i.InterpolationMode = InterpolationMode.High;
i.CopyFromScreen(window[0], window[1], 0, 0, bmpScreenCapture.Size, CopyPixelOperation.SourceCopy);
}
bmpScreenCapture.Save("test2.tif", ImageFormat.Tiff);
}

You can't gather more information than the source has.
This is a basic truth and it does apply here, too.
So you can't capture more the your 1920x1080 pixels at their color depth.
OTOH, since you want to feed the captured image into OCR, there a few more things to consider and in fact to do..
OCR is very happy if you help it by optimizing the image. This should involve
reducing colors and adding contrast
enlarging to the recommended dpi resolution
adding even more contrast
Funnily, this will help OCR although the real information cannot increase above the original source. But a good resizing algorithm will add invented data and these often will be just what the OCR software needs.
You should also take care to use a good i.e. non lossy format when you store the image to a file like png or tif and never jpg.
The best way will have to be adjusted by trial and error until the OCR results are good enough.
Hint: Due to font antialiasing most text on screenshots is surrounded by a halo of colorful pixels. Getting rid of it by the reducing or even removing saturation is one way; maybe you want to turn it off in your display properties? (Check out ClearType!)

How can I get the color of a Panel pixel at the mouse location (RFC)

I love SO. In 8 out of 10 questions it produces a straightforward and immediately usable answer. Or at least explains why my question is somehow wrong.
So I found it strange that I couldn't find an answer to my liking for this simple and, I had asumed, rather common question.
After searching high and low I patched together the below solution. It works alright but I wonder:
Are there flaws in it? (e.g.: do I need the dispose?)
Is there a better solution, maybe with less copying going on?
I'd like to see a soultion with CopyFromScreen , which potentially uses only a 1x1 bitmap size, but needs a Graphics.. So an alternative solution would be appreciated.
Or one that accesses the control's pixels directly.
Note 1: I want to grab the colors from a panel! Not from a picturebox and not from the screen..
Note 2: For my project speed is not important, as I want to create an eyedropper tool. But speedy ways are welcome, too; who knows where I'll go next ..
My solution:
public Color getColor(Control ctl, Point location)
{
Bitmap bmp = new Bitmap(ctl.Width, ctl.Height);
ctl.DrawToBitmap(bmp, new Rectangle(0, 0, ctl.Width, ctl.Height));
Color col = bmp.GetPixel(location.X, location.Y);
bmp.Dispose();
return col;
}
I use it like this in the colorsPanel_MouseClick event:
myPen = new Pen(getColor(colorsPanel, e.Location), myStrokeWidth);

I have come up with a version that uses CopyFromScreen like this:
public Color getScrColor(Control ctl, Point location)
{
Bitmap bmp = new Bitmap(1, 1);
Graphics g = Graphics.FromImage(bmp);
Point screenP = ctl.PointToScreen(location);
g.CopyFromScreen(screenP.X, screenP.Y, 0, 0, new Size(1, 1));
Color col = bmp.GetPixel(0, 0);
bmp.Dispose();
return col;
}
It works fine, too but seems to be a lot slower (by a factor of 10) than the one that uses DrawToBitmap. I doubt that the PointToScreen is the reason and a test version, that hands over a Graphics without creating it each time was just as slow.
So I guess the CopyFromScreen call is so much slower, that the number of pixels is not important. Well, within reasonable limits, probably. My color palette Control is about 60x400 pixels.
So for the time being I guess the original GetColor solution is OK to use.

How to get coordinates of cropped image from original [duplicate]

I have 2 bmp images.
ImageA is a screenshot (example)
ImageB is a subset of that. Say for example, an icon.
I want to find the X,Y coordinates of ImageB within ImageA (if it exists).
Any idea how I would do that?

Here's a quick sample but it is slow take around 4-6 seconds, but it does exactly what you looking for and i know this post is old but if anyone else visiting this post recently
you can look this thing
you need .NET AForge namespace or framework google it and install it
include AForge name space in your project and that's it
it finds the pictiure with another and gives out the coordinates.
System.Drawing.Bitmap sourceImage = (Bitmap)Bitmap.FromFile(#"C:\SavedBMPs\1.jpg");
System.Drawing.Bitmap template = (Bitmap)Bitmap.FromFile(#"C:\SavedBMPs\2.jpg");
// create template matching algorithm's instance
// (set similarity threshold to 92.1%)
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0.921f);
// find all matchings with specified above similarity
TemplateMatch[] matchings = tm.ProcessImage(sourceImage, template);
// highlight found matchings
BitmapData data = sourceImage.LockBits(
new Rectangle(0, 0, sourceImage.Width, sourceImage.Height),
ImageLockMode.ReadWrite, sourceImage.PixelFormat);
foreach (TemplateMatch m in matchings)
{
Drawing.Rectangle(data, m.Rectangle, Color.White);
MessageBox.Show(m.Rectangle.Location.ToString());
// do something else with matching
}
sourceImage.UnlockBits(data);

So is there any warping of ImageB in ImageA?
How "exact" are the images, as in, pixel-for-pixel they will be the same?
How much computational power do you have for this?
If the answers to the first two questions are No and Yes, then you have a simple problem. It also helps to know the answer to Q3.
Update:
The basic idea's this: instead of matching a window around every pixel in imageB with every pixel in imageA and checking the correlation, let's identify points of interest (or features) in both images which will be trackable. So it looks like corners are really trackable since the area around it is kinda similar (not going into details) - hence, let's find some really strong corners in both images and search for corners which look most similar.
This reduces the problem of searching every pixel in B with A to searching for, say, 500 corners in B with a 1000 corners in A (or something like that) - much faster.
And the awesome thing is you have several such corner detectors at your disposal in OpenCV. If you don't feel using emguCV (C# varriant), then use the FAST detector to find matching corners and thus locate multiple features between your images. Once you have that, you can find the location of the top-left corner of the image.

If image B is an exact subset of image A (meaning, the pixel values are exactly the same), this is not an image processing problem, it's just string matching in 2D. In 99% of the cases, taking a line form the middle of B and matching it against each line of A will do what you want, and super fast &mdhas; I guess C# has a function for that. After you get your matches (normally, a few of them), just check the whole of B against the appropriate part of A.
The only problem I can see with this is that in some cases you can get too many matches. E.g. if A is your desktop, B is an icon, and you are unlucky enough to pick a line in B consisting of background only. This problem is easy to solve (you have to choose lines from B a bit more carefully), but this depends on the specifics of your problem.

Finding sub images in an imageFind an image in an ImageCheck if an image exists within another image

How do I compare two images & recognize the pattern in an image?

How do I compare two images & recognize the pattern in an image irrespective of its size and pattern size, and using .Net C#? Also, which algorithms are used for doing so from Image Processing?

See Scale-invariant feature transform, template matching, and Hough transform. A quick and inaccurate guess may be to make a histogram of color and compare it. If the image is complicated enough, you might be able to distinguish between several sets of images.
To make the matter simple, assume we have three buckets for R, G, and B. A completely white image would have (100%, 100%, 100%) for (R, G, B). A completely red image would have (100%, 0%, 0%). A complicated image might have something like (23%, 53%, 34%). If you take the distance between the points in that (R, G, B) space, you can compare which one is "closer".

look up pattern recognition. I known very little about it other than the name.
Warning: If that is what you want, it is one of the hardest "real world" programming problems known.

I am no expert in image recognition by I once stummbeled upon the AForge library which is written in C# and does image recognition. Maybe it can help...

Techniques for image matching and image recognition can be very different. For the first task, you may make use of SIFT or hand craft your own distance function, based on RGB or otherwise. For recognition, there a vast amount of machine learning techniques that you can use, more popular techniques involves Adaboost, SVM and other hybrid neural networks method. There are no lack of related research papers in this field. Google is your friend.

Jinmala, you've asked a question here that is extremely broad. There are literally thousands of papers in the literature about these topics. There is no correct answer, and there are many unsolved issues in the comparison of images, so you really probably can't hope for a simple solution that just works (unless your situation is quite simple and constrained)
If you narrow things down, I might be able to help.

You might be looking for this
System.Drawing.Bitmap sourceImage = (Bitmap)Bitmap.FromFile(#"C:\SavedBMPs\1.jpg");
System.Drawing.Bitmap template = (Bitmap)Bitmap.FromFile(#"C:\SavedBMPs\2.jpg");
// create template matching algorithm's instance
// (set similarity threshold to 92.5%)
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0.921f);
// find all matchings with specified above similarity
TemplateMatch[] matchings = tm.ProcessImage(sourceImage, template);
// highlight found matchings
BitmapData data = sourceImage.LockBits(
new Rectangle(0, 0, sourceImage.Width, sourceImage.Height),
ImageLockMode.ReadWrite, sourceImage.PixelFormat);
foreach (TemplateMatch m in matchings)
{
Drawing.Rectangle(data, m.Rectangle, Color.White);
MessageBox.Show(m.Rectangle.Location.ToString());
// do something else with matching
}
sourceImage.UnlockBits(data);
I warn you it is quite slow takes around 6 seconds to process image of 1024x768 finding in it pciture with the size of 50x50.enter code here

template matching, you can do this with EmguCV ,OpendotnetCV,Aforge.net

Scale-invariant feature transform (SIFT) might be what you're looking for. It's not simple to understand or implement, however.