Hi i'm struggling with an algorithm that can effectively extract image bounding box and arrows from a rasterized document. The arrows and images can change is size shape and color. The arrows may not always be arrows but rather lines. The images may be just an outline or a full color picture. This is the code i wrote so far and it kinda works but not always. I wrote this algorithm based on the excellent paper Image to CAD: Feature Extraction and Translation of Raster Image of CAD Drawing to DXF CAD Format by Aditya Intwala.
This is the original image:
For detecting the arrowheads
using var kernel1 = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(2, 2));
using var binary = grayImage.Threshold(0, 255, ThresholdTypes.Binary | ThresholdTypes.Otsu);
using var invertedBinary = grayImage.Threshold(0, 255, ThresholdTypes.BinaryInv | ThresholdTypes.Otsu);
using var blackHatResult = binary.MorphologyEx(MorphTypes.BlackHat, kernel1);
using var solidArrowHeads = invertedBinary - blackHatResult;
using var foundArrowHeads = solidArrowHeads.ToMat();
using var steKernel = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(3, 3));
using var eroded = foundArrowHeads.Erode(steKernel);
using var dialated = eroded.Dilate(steKernel);
dialated.FindContours(out var arrowHeadContours, out var hierarchy, RetrievalModes.External,
ContourApproximationModes.ApproxSimple);
After finding arrow heads I;m doing the following for finding the lines that intersect the bounding box of arroheads. These intersecting lines is what im classifying as an arrow. It does work but i get a lot more false positives and would like to know how to improve my algorithm or if theres a better way. Next i'm erasing the arrows by masking it and then using this image for the next step.
To find the boundaries of the images I've written the following code
using var se1 = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(70, 1));
using var closedImg = grayImage.MorphologyEx(MorphTypes.Close, se1);
Cv2.BitwiseAnd(grayImage, closedImg, grayImage);
using var structuringElement = Cv2.GetStructuringElement(MorphShapes.Rect, new Size(15, 15));
using var blurred = grayImage.MorphologyEx(MorphTypes.Gradient, structuringElement);
using var inverted = blurred.Threshold(197, 255, ThresholdTypes.Binary);
//Lets get bounding boxes for all large contours
Cv2.FindContours(inverted, out var contours, out var hierarchyIndices, RetrievalModes.External,
ContourApproximationModes.ApproxSimple);
for (var i = 0; (i >= 0) && (i < hierarchyIndices.Length); i = hierarchyIndices[i].Next)
{
CT_Assert.True(hierarchyIndices[i].Parent == -1, "Must be a top level contour ?");
var rect = Cv2.BoundingRect(contours[i]);
// save this rect as possible bounding box
}
This again kinda works but not always.
Final output image, black bounding box is detected arrowhead, red lines are arrows and blue boxes are detected image bounding boxes.
Final Output image:
Related
I have a MVC C# application that includes a .Net wrapper for tesseract-ocr nuget. The current version I am using is v4.1.0-beta1. The image that I am try to scan is shown below
My aim is to extract the player name and the number just above them to the left.
I tried making the OCR scan the field/pitch area but the results are way off base. So, I decided to section off all player names and all numbers as seen in the image below. Ratings area marked in blue and player names marked in red. As you can see the name and rating are always the same distance apart.
My current code setup is shown below.
public void Get(HttpPostedFileBase file)
{
using (var engine = new TesseractEngine(Path.Combine(HttpRuntime.AppDomainAppPath, "tessdata"), "eng+deu", EngineMode.Default))
{
var bitmap = (Bitmap)Image.FromStream(file.InputStream, true, true);
using (var img = PixConverter.ToPix(bitmap))
{
SetPlayerRatings(engine, img);
}
}
}
private void SetPlayerRatings(TesseractEngine engine, Pix img)
{
var width = 285;
var height = 76;
var textPositions = Service.Get<Formation>(this.FormationId).TextPositions.ToList();
foreach (var textPosition in textPositions)
{
var playerRating = GetPlayerData(engine, img, new Rect(textPosition.X, textPosition.Y, width, height));
}
}
private static PlayerRating GetPlayerData(TesseractEngine engine, Pix img, Rect region)
{
using (var page = engine.Process(img, region, PageSegMode.Auto))
{
var playerName = page.GetText();
}
var ratingRegion = new Rect(region.X1, region.Y1 - 52, 80, 50);
using (var page = engine.Process(img, ratingRegion, PageSegMode.Auto))
{
var playerRating = page.GetText();
}
}
This code is producing the correct results for the 1st image.
Is there any way to train OCR so that I dont have to workout the X and Y co-ordinates for each player position? I would like to just specify the area of the pitch and have OCR read in the rating followed by the player name.
With specifying coordinates you solved several problems regarding image processing. So if you do not want to specify coordinates, you have to deal with them: e.g. removing graphics component from OCR area like T-shirt, lines.
Next idea: Tesseract API has option GetComponentImages (I expect C# wrapper should provide it too - I am not familiar with C#), so you can iterate over found components.
I have a list of region borders inside SQL database and i am using sharpmap to render thumbnail images for every country i need. It works really well.
But I would like to go one step further and add a small globe around it and position country on it's place in globe, but I do not know where to start.
Here's the code i'm using so far to render country thumbs. Any ideas?
var map = new Map(new Size(command.Width, command.Height));
map.BackColor = Color.Transparent;
var countryGeometry = GeometryFromWKT.Parse(command.CountryLevelWkt);
IProvider countryProvider = new GeometryFeatureProvider(countryGeometry);
var countryLayer = new VectorLayer("country", countryProvider);
var borderColor = System.Drawing.ColorTranslator.FromHtml(command.BorderColor);
countryLayer.Style.EnableOutline = true;
countryLayer.Style.Outline = new Pen(borderColor);
countryLayer.Style.Outline.Width = command.BorderWidth;
countryLayer.Style.Fill = Brushes.Transparent;
var transformationFactory = new CoordinateTransformationFactory();
countryLayer.CoordinateTransformation = transformationFactory.CreateFromCoordinateSystems(
GeographicCoordinateSystem.WGS84,
ProjectedCoordinateSystem.WebMercator);
map.Layers.Add(countryLayer);
var bottomLeft = new Coordinate(command.Extents.BottomLeft.Longitude, command.Extents.BottomLeft.Latitude);
var topRight = new Coordinate(command.Extents.TopRight.Longitude, command.Extents.TopRight.Latitude);
// transformations
var bottomLeftLongLat = countryLayer.CoordinateTransformation.MathTransform.Transform(bottomLeft);
var topRightLongLat = countryLayer.CoordinateTransformation.MathTransform.Transform(topRight);
map.ZoomToBox(new Envelope(bottomLeftLongLat, topRightLongLat));
var img = map.GetMap();
return img;
Start by drawing all countries on a new map, each on its own layer.
Draw the country in which you're interested on its own layer.
Set map center to the Envelope.Center of the layer from Step 2. Eg, if drawing Australia, the map will move to the left.
Render map as image. Draw image on a drawing sufrace (System.Drawing.Graphics).
Re-center the map, to cover the empty space. Eg, if drawing Australia, move map almost all the way to the right. You will need to work out these offsets programmatically.
Render map from step 5 as image. Add image to the same drawing sufrace (see step 4).
Repeat steps 5-6 covering empty space below/above the render made at step 3.
Here is an example:
Note that:
Australia is in the center
There is a gap between map layers near the mouse pointer (gap in screenshot is intentional to demonstrate the logic)
Some countries are very large (eg Russia) and obtaining Envelope.Center will not work very well - consider centering based on the largest polygon only
Here's a sample Windows Forms project. In the sample, I used maps from http://thematicmapping.org/downloads/world_borders.php.
I looking for a way to calculate a rectangle (x,y,width & height) which can be used for cropping an image around the coordinates of a selected face.
I have an image 995x1000 (https://tourspider.blob.core.windows.net/img/artists/original/947a0903-9b64-42a1-8179-108bab2a9e46.jpg) by which the center of the face is located at 492x325. I can find this information using various services so even for multiple faces in an image I'm ableto find the most prominent - hence a single coordinate.
Now i need to make various sized cropped images from the source image (200x150, 200x200 & 750x250). Now I can't seem to solve how to best calculate a rectangle around the center coordinates while taking into account the edges of the images. The face should be as central as possible in the image.
Even after experimenting with various services (https://www.microsoft.com/cognitive-services/en-us/computer-vision-api) the result are pretty poor as the face, mainly in the 750x250, is sometimes not even present.
I'm also experimenting with the ImageProcessor (http://imageprocessor.org/) library with which you can use anchors for resizing but can't get the desired result.
Does anybody has an idea on how best crop around predefined coordinates?
Using Imageprocessor I created the following solution. It is not yet perfect but goes a long way ;)
public static void StoreImage(byte[] image, int destinationWidth, int destinationHeight, Point anchor)
{
using (var inStream = new MemoryStream(image))
using (var imageFactory = new ImageFactory())
{
// Load the image in the image factory
imageFactory.Load(inStream);
var originalSourceWidth = imageFactory.Image.Width;
var originalSourceHeight = imageFactory.Image.Height;
// Resizes the image until the shortest side reaches the set given dimension.
// This will maintain the aspect ratio of the original image.
imageFactory.Resize(new ResizeLayer(new Size(destinationWidth, destinationHeight), ResizeMode.Min));
var resizedSourceWidth = imageFactory.Image.Width;
var resizedSourceHeight = imageFactory.Image.Height;
//Adjust anchor position
var resizedAnchorX = anchor.X/(originalSourceWidth / resizedSourceWidth);
var resizedAnchorY = anchor.Y/(originalSourceHeight/resizedSourceHeight);
if (anchor.X > originalSourceWidth || anchor.Y > originalSourceHeight)
{
throw new Exception($"Invalid anchor point. Image: {originalSourceWidth}x{originalSourceHeight}. Anchor: {anchor.X}x{anchor.Y}.");
}
var cropX = resizedAnchorX - destinationWidth/2;
if (cropX < 0)
cropX = 0;
var cropY = resizedAnchorY - destinationHeight/2;
if (cropY < 0)
cropY = 0;
if (cropY > resizedSourceHeight)
cropY = resizedSourceHeight;
imageFactory
.Crop(new Rectangle(cropX, cropY, destinationWidth, destinationHeight))
.Save($#"{Guid.NewGuid()}.jpg");
}
}
i am using BarcodeInter25 class to make barcode. I am able to make it but its just blur how can it become more sharp ??
also its background white colour is not completely white
My Code:
BarcodeInter25 code25 = new BarcodeInter25();
Rectangle r = new iTextSharp.text.Rectangle(38, 152);
code25.ChecksumText = false;
code25.Code = "some digits";
code25.BarHeight = 2
System.Drawing.Image i = code25.CreateDrawingImage(System.Drawing.Color.Black, System.Drawing.Color.White);
MemoryStream ms = new MemoryStream();
i.Save(ms, System.Drawing.Imaging.ImageFormat.Jpeg);
Image img = Image.GetInstance(ms.ToArray());
ms.Dispose();
Looking at your code, it should be obvious why the barcode is blurry. You convert it to a System.Drawing.Image (making it a raster image) and then you convert it to an iTextSharp.text.Image (but by then the image is already blurry).
The correct way to achieve what you want, is to create an iTextSharp.text.Image straight from the barcode (do not pass through System.Drawing.Image). This can be done like this:
BarcodeInter25 code25 = new BarcodeInter25();
Rectangle r = new iTextSharp.text.Rectangle(38, 152);
code25.ChecksumText = false;
code25.Code = "some digits";
code25.BarHeight = 2;
PdfContentByte cb = writer.DirectContent;
Image img = code25.CreateImageWithBarcode(cb, null, null);
Now the Image object won't be a raster image (with pixels that make the lines blurry), but it will be a true vector image (no pixels, but instructions such as moveTo(), lineTo() and stroke()). Vector data has the advantage that it is resolution independent: you can zoom in and zoom out as much as you want, it will always be sharp.
This is explained in Chapter 10 of my book where you'll find the Barcodes example. In that chapter, you'll also discover the setFont() (or in iTextSharp the Font property). I quote from the API documentation:
public void setFont(BaseFont font)
Sets the text font.
Parameters:
font - the text font. Set to null to suppress any text
So if you don't want to see any text, you can add the following line to the above code snippet:
code25.Font = null;
You should avoid re-sizing by any means. The output is most likely pixel-perfect but when you scale it up/down a bilinear filter will smooth it rendering it blurry.
I had the same exact problem by embedding a QRC code in a PDF and solved it by avoiding a resize.
If you really need a different size apply it programmatically in code by using the correct interpolation algorithm.
I'm looking to connect or glue together two shapes or objects with a Line. These shapes will be generated dynamically, meaning I'll be calling a Web service on the backend to determine how many objects/shapes need to be created. Once this is determined, I'll need to have the objects/shapes connected together.
The method signature may look like this (similar to Visio's drawing capabilities):
GlueTogether(objButton1, objButton2);
I may need to get the position of each Rectangle shape or Button to determine where the starting Line point is. Then determine the second shape/objects position to draw the line.
Any help or suggestions would be great!
Use a Path or a Line below the shapes in stacking order or z index
Use instance.TransformToVisual() to get the transform of each shape
Use the transform to transform the centerpoint of each shape
Draw a line between the two centerpoints.
var transform1 = shape1.TransformToVisual(shape1.Parent as UIElement);
var transform2 = shape2.TransformToVisual(shape2.Parent as UIElement);
var lineGeometry = new LineGeometry()
{
StartPoint = transform1.Transform(new Point(shape1.ActualWidth / 2, shape1.ActualHeight / 2.0)),
EndPoint = transform2.Transform(new Point(shape2.ActualWidth / 2.0, shape2.ActualHeight / 2.0))
};
var path = new Path()
{
Data = lineGeometry
};
I am trying much the same, but instead of the line going from one centre to the other I want the lines to stop at the edge of the two shapes.
In particular I have arrows at the end of the lines, and the arrows need to stop at the bounds of the shapes instead of going inside/behind the shape to its centre.
My shape is a usercontrol with a grid and rectangle, and some labels and other stuff.
I can't find any methods that provide me with a geometry for the edge of the shape (which is a rounded rectangle).
I figured out a solution that uses the bounding box and intersection points to connect my elements by lines at their approximate edges, and it works well for me using arrow ended lines.
See Connecting two WPF canvas elements by a line, without using anchors?
Check this out: http://www.graphspe.com/Main.aspx#/Solution/graphviz-xaml-renderer
All you have to do is printf to a string and you get your Silverlight[2|3] diagram.
Ceyhun
In addition... Instead of connecting to the center point of your objects, I've modified the same code from Michael S. to:
var lineGeometry = new LineGeometry()
{
StartPoint = transform1.Transform(new Point(1 , b1.ActualHeight / 2.0)),
EndPoint = transform2.Transform(new Point(b2.ActualWidth , b2.ActualHeight / 2.0))
};
This will connect at the outer portions of each object.
I am using the above code to draw two buttons, I want a line between those two buttons, but all i get are two buttons that look like tiny circles and no line.
code:
Button b1 = new Button();
Button b2 = new Button();
canvas1.Children.Add(b1);
canvas1.Children.Add(b2);
Canvas.SetLeft(b1, 300);
var transform1 = b1.TransformToVisual(b1.Parent as UIElement);
var transform2 = b2.TransformToVisual(b2.Parent as UIElement);
var lineGeometry = new LineGeometry()
{
StartPoint = transform1.Transform(new Point(1, b1.ActualHeight / 2.0)),
EndPoint = transform2.Transform(new Point(b2.ActualWidth, b2.ActualHeight / 2.0))
};
var path = new Path()
{
Data = lineGeometry
};
canvas1.Children.Add(path);