I am using NetTopologySuite with C# to filter points inside the precise boundaries of a country with a pretty simple way:
var path = "fr.shp"; // "big" country and boundaries need to be precise in my case
var reader = new ShapeDataReader(path);
var mbr = reader.ShapefileBounds;
var result = reader.ReadByMBRFilter(mbr);
var polygons = new List<Geometry>();
using (var coll = result.GetEnumerator())
{
while (coll.MoveNext())
{
var item = coll.Current;
if (item == null)
{
continue;
}
polygons.Add(item.Geometry);
}
}
var polygon = new GeometryCombiner(polygons).Combine();
var points = new List<Point>();
List<Point> pointsToFilterWithBorders; // loaded from DB, not visible here but we have 1,350,000 points to filter
Parallel.ForEach(pointsToFilterWithBorders, point =>
{
if (polygon.Contains(point))
points.Add(point);
});
It's working fine (filtering works great!) but it's pretty slow... like one day to do the filtering for only 1,350,000 points!
Any idea on how to improve that?
I tried to use Parallel.ForEach but still very long and I tried to find something like a "batch" compare in NetTopologySuite but couldn't find a quicker solution to filter my points in this big shapefile...
Presumably the polygon that defines the border is quite large and detailed, otherwise it should not take such a long time. There are a few approaches I would consider
Do an initial Bounding box check
Create an axis aligned bounding box for the polygon. Start by testing if the point is inside this box before continuing with any more complex check. This should be very easy to implement.
Render the polygon
create a large bitmap and render your polygon to the bitmap, you will need so use some kind of transform to translate between GIS coordinate and pixel coordinates. Then you can simply check the pixel value for each point in the bitmap. Just make sure to disable anti aliasing. Note that this would be an approximate solution.
You could also do something like rendering the polygon in one color and then rendering the border in another color using a pen more than one pixel wide. Any points on the border would be uncertain and may need a more accurate test, while any points outside the border should be guaranteed to be either inside or outside your polygon.
Use a tree structure to speed up the check
The typical approach to speed up any kind of repeated search is to build some kind of search structure to speedup the search, this is often a tree. In this specific case I believe a Binary Space Partition tree might be suitable. The advantage here is that the number of checks needed would be O(log n) where n is the number of lines in your polygons, instead of the O(n) that I suspect the .Contains() method is.
But I have never implemented a BSP tree, so I refer to other sources on how to implement one.
You could also consider simplifying your border polygon, but it might be more difficult to ensure the simplified polygon is either fully contained by the true polygon, or fully contains the true polygon.
Note that most methods assume that you are using a planar coordinate system, so you might need to do some conversions if you are using anything else.
If possible use spatial capabilities of your database and only load points into your set that possibly intersect with your MultiPolygon, i.e. they are withing the bounding box of it.
Then for 1:n geometry checks use NetTopologySuite's PreparedGeometry predicates:
// polygon and pointsToFilterWithBorders from sample above
var prep = NetTopologySuite.Geometries.Prepared.PreparedGeometryFactory.Prepare(polygon);
var points = new System.Collections.Generic<Point>();
foreach(var pt in pointsToFilterWithBorders)
if (prep.Contains(pt)) points.Add(pt);
Thanks everyone for the help! At the end, I found a solution inspired by your ideas :-)
I can't share the code itself because it's inside a closed source code but here is the idea:
My code was already grouping all the points by squares of 1 km² for other purposes: I just had to check if these polygons where not fully inside the boundaries. When that was not the case, I just had to do a check on all points inside these squares!
Still a little bit long (but less than one hour for France and it's mixed with other pieces of code which were already a little bit slow) but clearly quicker!
I am using a console app and very basic Tesseract to perform digit recognition. I have copied an image from google and tried to find the digits only.
Bitmap image = new Bitmap("1.png");
TesseractEngine t = new TesseractEngine("./tessdata", "eng", EngineMode.Default);
t.SetVariable("tessedit_char_whitelist", "01234567890");
var r = t.Process(image, PageSegMode.SingleBlock);
Console.WriteLine("Result: " + r.GetText());
Console.ReadLine();
The image is
The result are different depending on the pageSegMode, but none of them are close to the image.
What is the best way to use Tesseract to identify digits from such pics?
Tesseract won't work well with an image like that unless you somehow train it specifically for that case, but I don't think you have to if you can transform the image right
Your goal should be feeding it a black and white picture with black digits and white background, you should do this processing before OCRing the image, there are many libraries for this, most people use OpenCV.
Tesseract already does some image processing but it's not great and probably doesn't help much with an image like that. You can view the auto processed result with tessedit_write_images to see what's actually being OCRed.
Here's some useful links:
https://github.com/tesseract-ocr/tesseract/wiki/ImproveQuality#image-processing
Using tesseract to recognize license plates
I have 2 bmp images.
ImageA is a screenshot (example)
ImageB is a subset of that. Say for example, an icon.
I want to find the X,Y coordinates of ImageB within ImageA (if it exists).
Any idea how I would do that?
Here's a quick sample but it is slow take around 4-6 seconds, but it does exactly what you looking for and i know this post is old but if anyone else visiting this post recently
you can look this thing
you need .NET AForge namespace or framework google it and install it
include AForge name space in your project and that's it
it finds the pictiure with another and gives out the coordinates.
System.Drawing.Bitmap sourceImage = (Bitmap)Bitmap.FromFile(#"C:\SavedBMPs\1.jpg");
System.Drawing.Bitmap template = (Bitmap)Bitmap.FromFile(#"C:\SavedBMPs\2.jpg");
// create template matching algorithm's instance
// (set similarity threshold to 92.1%)
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0.921f);
// find all matchings with specified above similarity
TemplateMatch[] matchings = tm.ProcessImage(sourceImage, template);
// highlight found matchings
BitmapData data = sourceImage.LockBits(
new Rectangle(0, 0, sourceImage.Width, sourceImage.Height),
ImageLockMode.ReadWrite, sourceImage.PixelFormat);
foreach (TemplateMatch m in matchings)
{
Drawing.Rectangle(data, m.Rectangle, Color.White);
MessageBox.Show(m.Rectangle.Location.ToString());
// do something else with matching
}
sourceImage.UnlockBits(data);
So is there any warping of ImageB in ImageA?
How "exact" are the images, as in, pixel-for-pixel they will be the same?
How much computational power do you have for this?
If the answers to the first two questions are No and Yes, then you have a simple problem. It also helps to know the answer to Q3.
Update:
The basic idea's this: instead of matching a window around every pixel in imageB with every pixel in imageA and checking the correlation, let's identify points of interest (or features) in both images which will be trackable. So it looks like corners are really trackable since the area around it is kinda similar (not going into details) - hence, let's find some really strong corners in both images and search for corners which look most similar.
This reduces the problem of searching every pixel in B with A to searching for, say, 500 corners in B with a 1000 corners in A (or something like that) - much faster.
And the awesome thing is you have several such corner detectors at your disposal in OpenCV. If you don't feel using emguCV (C# varriant), then use the FAST detector to find matching corners and thus locate multiple features between your images. Once you have that, you can find the location of the top-left corner of the image.
If image B is an exact subset of image A (meaning, the pixel values are exactly the same), this is not an image processing problem, it's just string matching in 2D. In 99% of the cases, taking a line form the middle of B and matching it against each line of A will do what you want, and super fast &mdhas; I guess C# has a function for that. After you get your matches (normally, a few of them), just check the whole of B against the appropriate part of A.
The only problem I can see with this is that in some cases you can get too many matches. E.g. if A is your desktop, B is an icon, and you are unlucky enough to pick a line in B consisting of background only. This problem is easy to solve (you have to choose lines from B a bit more carefully), but this depends on the specifics of your problem.
Finding sub images in an imageFind an image in an ImageCheck if an image exists within another image
Problem
Problem shaping
Image sequence position and size are fixed and known beforehand (it's not scaled). It will be quite short, maximum of 20 frames and in a closed loop. I want to verify (event driven by button click), that I have seen it before.
Lets say I have some image sequence, like:
http://img514.imageshack.us/img514/5440/60372aeba8595eda.gif
If seen, I want to see the ID associated with it, if not - it will be analyzed and added as new instance of image sequence, that has been seen. I have though about this quite a while, and I admit, this might be a hard problem. I seem to be having hard time of putting this all together, can someone assist (in C#)?
Limitations and uses
I am not trying to recreate copyright detection system, like content id system Youtube has implemented (Margaret Gould Stewart at TED ( link )). The image sequence can be thought about like a (.gif) file, but it is not and there is no direct way to get binary. Similar method could be used, to avoid duplicates in "image sharing database", but it is not what I am trying to do.
My effort
Gaussian blur
Mathematica function to generate Gaussian blur kernels:
getKernel[L_] := Transpose[{L}].{L}/(Total[Total[Transpose[{L}].{L}]])
getVKernel[L_] := L/Total[L]
Turns out, that it is much more efficient to use 2 passes of vector kernel, then matrix kernel. Thy are based on Pascal triangle uneven rows:
{1d/4, 1d/2, 1d/4}
{1d/16, 1d/4, 3d/8, 1d/4, 1d/16}
{1d/64, 3d/32, 15d/64, 5d/16, 15d/64, 3d/32, 1d/64}
Data input, hashing, grayscaleing and lightboxing
Example of source bits, that might be useful:
Lightbox around the known rectangle: FrameX
Using MD5CryptoServiceProvider to get md5 hash of the content inside known rectangle atm.
Using ColorMatrix to grayscale image
Source example
Source example (GUI; code):
Get current content inside defined rectangle.
private Bitmap getContentBitmap() {
Rectangle r = f.r;
Bitmap hc = new Bitmap(r.Width, r.Height);
using (Graphics gf = Graphics.FromImage(hc)) {
gf.CopyFromScreen(r.Left, r.Top, 0, 0, //
new Size(r.Width, r.Height), CopyPixelOperation.SourceCopy);
}
return hc;
}
Get md5 hash of bitmap.
private byte[] getBitmapHash(Bitmap hc) {
return md5.ComputeHash(c.ConvertTo(hc, typeof(byte[])) as byte[]);
}
Get grayscale of the image.
public static Bitmap getGrayscale(Bitmap hc){
Bitmap result = new Bitmap(hc.Width, hc.Height);
ColorMatrix colorMatrix = new ColorMatrix(new float[][]{
new float[]{0.5f,0.5f,0.5f,0,0}, new float[]{0.5f,0.5f,0.5f,0,0},
new float[]{0.5f,0.5f,0.5f,0,0}, new float[]{0,0,0,1,0,0},
new float[]{0,0,0,0,1,0}, new float[]{0,0,0,0,0,1}});
using (Graphics g = Graphics.FromImage(result)) {
ImageAttributes attributes = new ImageAttributes();
attributes.SetColorMatrix(colorMatrix);
g.DrawImage(hc, new Rectangle(0, 0, hc.Width, hc.Height),
0, 0, hc.Width, hc.Height, GraphicsUnit.Pixel, attributes);
}
return result;
}
I think you have a few issues with this:
Not all image sequences [videos] are equal [but many are similar]
Where is your data coming from?
How will you repesent the data related to your viewings?
Size of the data
Issue #1:
Many images can differ slightly by compression, water marking, missing frames, and adding clips. I would suggest sampling the video. For example you may want to consider sub-sampling small sections of the images in the video. Additionally, to avoid noisy images and issues with lossely compression algorithms. You may want to consider grayscaling the frames sampled, and doing a gaussian blur. [Guassian because its "more natural" (short answer)] Once you have enough sub samples to where you have a good confidence of similarity to the video then store it in a database. With the samples you can hash them, or store them to do a % similarity later.
Issue #2
Your datasource is going to influence the tool kits, and libraries that you use.
I would suggest keeping this simple [keep it with gifs and create a custom viewer, dont' try to write a browser plugin while developing your logic]
Issue #3
Using something like Postgres [if there are a lot of large sized objects] or SQLLite is highly suggested for indexing, storing, and recalling past meta data.
Issue #4
The size of the data will have a huge determination on recall, sampling, querying the database, etc.
Overall advice: Don't bite off more than you can handle at this stage. Start small and then grow.
Also take a look at Computer Vision algorithms for more help on the object representation/recall.
The question itself is sure very interesting and challenging, however there are many practical issues as stated by #monksy.
The opportunist pragmatic in me would take a step back, look at the big picture and see if there is another way to solve the problem. For example, if you are building some kind of "image sharing community" and want to avoid duplicates in the database, you could do a simple md5 on the file (animated gifs on the web are usually always the same, it's rare that people modify them).
Another example: if you are analyzing scientific samples (like meteo sequences) it may be easier to directly embed some kind of hash in every file when generating them.
This depends on wether you only want to know wether you've seen an absolutely identical movie again, or you also want to identify movies that are very similar but have been changed a bit (made lighter, have a watermark added, compression changed, etc.)
In the first case, just take any type of hash of the file and use that (because the file will be identical on the binary level.
In the second case (which I think is what you want) you have an interesting image processing problem on your hands. You could find yourself at the front-lines of image processing science with this if you'd want. If that is the case I suggest you start reading about SURF and OpenCV, and continue on from that.
If you want to match very similar, but not identical videos, and don't want to go the ultra-robus scientific route then I'd suggest the following process:
Do the gaussian blur you already do.
Divide each image into a few equally sized rectangles (you'd have to test for the best number, but I'd suggest you start with 9.
For each rectangle in each frame compute the full-colour histogram, then find the most occurring colour in that rectangle. This gives you 9*20 = 180 numbers. This is the "fingerprint" of this movie.
Find the most similar fingerprint in your database, if it is similar enough you already know about it, otherwise you don't.
Step 4 is a bit vague because I'm not really into this field. You are currently using an MD5 hash as a sort of fingerprint, but this is unsuitable in this case because slight differences in the input of a good cryptographic hashing function produce very large differences in the hash. This will mean that two very similar frames will have a totally different MD5 hash, so from the hash you'd never know they were similar.
As long as speed of database lookups is not an issue I'd just go for the sum of square differences as a measure of fingerprint similarity, and set a threshold on that to identify equal movies. However, this is not very fast for huge datasets, and in those cases you'd probably need to transform your fingerprint to something that will allow you to find similar fingerprints faster. One thing you could do here is start by selecting all known movies with very similar average colour for the entire video, then from that select the movies that have very similar average colour in each frame, and in the ones that remain at that point do the full rectangle-by-rectangle fingerprint match. But I'm sure there are even faster options for matching 180 numbers.
Perhaps you can find a way to get a binary copy of the image data of each frame in a variable. Hash that data (md5?) and store each of the hashes. Then you can see if you've ever seen that hash before. If you haven't, it's a new frame.
I am simulating a thermal camera effect. I have a webcam at a party pointed at people in front of a wall. I went with background subtraction technique and using Aforge blobcounter I get blobs that I want to fill with gradient coloring. My problem = GetBlobsEdgePoints doesn't return sorted point cloud so I can't use it with, for example, PathGradientBrush from GDI+ to simply draw gradients.
I'm looking for simple,fast, algorithm to trace blobs into path (can make mistakes).
A way to track blobs received by blobcounter.
A suggestion for some other way to simulate the effect.
I took a quick look at Emgu.CV.VideoSurveillance but didn't get it to work (examples are for v1.5 and I went with v2+) but I gave up because people say it's slow on forums.
thanks for reading.
sample code of aforge background removal
Bitmap bmp =(Bitmap)e.VideoFrame.Clone();
if (backGroundFrame == null)
{
backGroundFrame = (Bitmap)e.VideoFrame.Clone();
difference.OverlayImage = backGroundFrame;
}
difference.ApplyInPlace(bmp);
bmp = grayscale.Apply(bmp);
threshold.ApplyInPlace(bmp);
Well, could you post some sample image of the result of GetBlobsEdgePoints, then it might be easier to understand what types if image processing algorithms are needed.
1) You may try a greedy algorithm, first pick a point at random, mark that point as "taken", pick the closest point not marked as "taken" and so on.
You need to find suitable termination conditions. If there can be several disjunct paths you need to find out a definition of how far away points need to be to be part of disjunct paths.
3) If you have a static background you can try to create a difference between two time shifted images, like 200ms apart. Just do a pixel by pixel difference and use abs(diff) as index in your heat color map. That will give more like an edge glow effect of moving objects.
This is the direction i'm going to take (looks best for now):
Define a set of points on the blob by my own logic (color of skin blobs should be warmer etc..)
draw gradients around those points
GraphicsPath gp=new GraphicsPath();
var rect = new Rectangle(CircumferencePoint.X - radius, CircumferencePoint.Y - radius, radius*2, radius*2);
gp.AddEllipse(rect);
GradientShaper = new PathGradientBrush(gp);
GradientShaper.CenterColor = Color.White;
GradientShaper.SurroundColors = surroundingColors;
drawBmp.FillPath(GradientShaper,gp);
mask those gradients with blob shape
blobCounter.ExtractBlobsImage(bmp,blob,true);
mask.OverlayImage = blob.Image;
mask.ApplyInPlace(rslt);
colorize with color remapping
tnx for the help #Albin