Tesseract OCR - How to Train For an Image Like This - c#

I have a MVC C# application that includes a .Net wrapper for tesseract-ocr nuget. The current version I am using is v4.1.0-beta1. The image that I am try to scan is shown below
My aim is to extract the player name and the number just above them to the left.
I tried making the OCR scan the field/pitch area but the results are way off base. So, I decided to section off all player names and all numbers as seen in the image below. Ratings area marked in blue and player names marked in red. As you can see the name and rating are always the same distance apart.
My current code setup is shown below.
public void Get(HttpPostedFileBase file)
{
using (var engine = new TesseractEngine(Path.Combine(HttpRuntime.AppDomainAppPath, "tessdata"), "eng+deu", EngineMode.Default))
{
var bitmap = (Bitmap)Image.FromStream(file.InputStream, true, true);
using (var img = PixConverter.ToPix(bitmap))
{
SetPlayerRatings(engine, img);
}
}
}
private void SetPlayerRatings(TesseractEngine engine, Pix img)
{
var width = 285;
var height = 76;
var textPositions = Service.Get<Formation>(this.FormationId).TextPositions.ToList();
foreach (var textPosition in textPositions)
{
var playerRating = GetPlayerData(engine, img, new Rect(textPosition.X, textPosition.Y, width, height));
}
}
private static PlayerRating GetPlayerData(TesseractEngine engine, Pix img, Rect region)
{
using (var page = engine.Process(img, region, PageSegMode.Auto))
{
var playerName = page.GetText();
}
var ratingRegion = new Rect(region.X1, region.Y1 - 52, 80, 50);
using (var page = engine.Process(img, ratingRegion, PageSegMode.Auto))
{
var playerRating = page.GetText();
}
}
This code is producing the correct results for the 1st image.
Is there any way to train OCR so that I dont have to workout the X and Y co-ordinates for each player position? I would like to just specify the area of the pitch and have OCR read in the rating followed by the player name.

With specifying coordinates you solved several problems regarding image processing. So if you do not want to specify coordinates, you have to deal with them: e.g. removing graphics component from OCR area like T-shirt, lines.
Next idea: Tesseract API has option GetComponentImages (I expect C# wrapper should provide it too - I am not familiar with C#), so you can iterate over found components.

Related

Having issues when attempting to load images into a bitmap/Picture Box Visual Studio 2019

I am attempting to load images into an Picture Box using the following code:
private void button1_Click(object sender, EventArgs e)
{
//Clear the Image Area
if (templateArea.Image != null)
{ templateArea.Image = null; }
//Set Base Variables
Bitmap img = new Bitmap(Resources.BlankBackground);
Graphics gpx = Graphics.FromImage(img);
Bitmap newBitmap = null;
string bitmapToLoad;
ResourceManager rm;
//Loop Lists
for (int i = 0; i < comps.Count; i++)
{
bitmapToLoad = Convert.ToString(comps[i][3]);
rm = Resources.ResourceManager;
newBitmap = (Bitmap)rm.GetObject(bitmapToLoad);
gpx.DrawImage(newBitmap,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap.Width,
newBitmap.Height);
gpx.Dispose();
}
//Set Image Area to newBitmap
templateArea.Image = newBitmap;
}
when trying to run the code I get errors on the second loop where i = 1 in this portion:
gpx.DrawImage(newBitmap,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap.Width,
newBitmap.Height);
//given error: System.ArgumentException: 'Parameter is not valid.' //Does not point to anything specific.
the comps list contains lists formatted as such:
//These listed items are defaulted at program start but more are added while running
// p elementName type sub type x y sX sY R G B O i
eleComps = new List<Object> {0,"Background","Static","BlankBackground",0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
eleComps = new List<Object> {0,"Border", "Static","CardBorder_02", 0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
eleComps = new List<Object> {0,"Cut Line", "Static","00_Card_CutLine",0,0,100,100,50,50,50,0,0};
comps.Add(eleComps);
Each possible "sub type" is a bitmap file in resources, and there are more than just those listed above.
What I have tried:
I have consulted the miracle that is Google and YouTube and haven't been able to solve this one. Although I feel like I'm missing something basic I have been looking at it all day and may need some fresh eyes on it to correct something that may be obvious but for some reason I seem to be missing.
What I am looking for:
I am trying to create an image viewer that will overlap images or parts of images (.png) at specific coordinates in an Picture Box. The comps list will be added to and removed from actively during application use and as such using a variable to reference the Resources is necessary and that portion of the code has given me the most difficult time out of the supplied code samples.
UPDATE 1: changed image box to picture box to reduce confusion.
UPDATE 2: NOTE: This sample achieves the desired end result, but does not allow me to use the variables as needed to change positions, scale, color, and opacity.
//Clear Img Area
templateArea.InitialImage = null;
//Find Bitmap(s)
string p = #"C:\Users\david\Desktop\Test Fields\Default\";
Bitmap img1 = new Bitmap(p + "00_Empty.png", true);
Bitmap img2 = new Bitmap(p + "CardBorder_02.png", true);
Bitmap img3 = new Bitmap(p + "00_Card_CutLine.png", true);
//Set initial image for image area
Graphics gpx = Graphics.FromImage(img1);
//place an image at x, y, Width, Height
gpx.DrawImage(img2, 0, 0, img2.Width, img2.Height);
gpx.DrawImage(img3, 0, 0, img3.Width, img3.Height);
//clear gpx cache
gpx.Dispose();
//set image area to modified img1
templateArea.Image = img1;
Also please let me know if I'm just plain going about this in the wrong way, and what I need to do to get it right.
UPDATE 3: I seem to have gotten it to work properly with this variation, using the originally supplied list of listed parameters. Thanks to Idle_Mind's pointers I was able to figure it out.
Bitmap newBitmap_01 = new Bitmap(templateArea.Width, templateArea.Height);
Graphics gpx = Graphics.FromImage(newBitmap_01);
string bitmapToLoad;
ResourceManager rm = Resources.ResourceManager;
Bitmap newBitmap_02;
for (int i = 0; i < comps.Count; i++)
{
bitmapToLoad = Convert.ToString(comps[i][3]);
newBitmap_02 = (Bitmap)rm.GetObject(bitmapToLoad);
gpx.DrawImage(newBitmap_02,
Convert.ToInt32(comps[i][4]),
Convert.ToInt32(comps[i][5]),
newBitmap_02.Width,
newBitmap_02.Height);
}
gpx.Dispose();
templateArea.Image = newBitmap_01;

GameCard value recognition gives wrong result with AForge.NET

Actually I'm using AForge.NET to recognize suit and value from a gamecard.
Here is a snippet where I decide which suit/value is there:
public static CardTemplateIdentifier GetBestMatchingIdentifier(this List<CardTemplateIdentifier> templates, Bitmap bitmap)
{
float maxSimilar = 0f;
CardTemplateIdentifier result = null;
foreach (var template in templates)
{
// Identify similarity between template and bmp
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0);
TemplateMatch[] matchings = tm.ProcessImage(bitmap, template.Sample);
// If the currently tested template fits better than the best one so far,
// set the value as the identified card value
if (matchings.Length > 0 && matchings[0].Similarity > maxSimilar)
{
maxSimilar = matchings[0].Similarity;
result = template;
}
}
return result;
}
CardTemplateIdentifier contains a list with all possible cards and their comparism sample.
...
Now If trying to recognize the images, I can not rely on this being reliably recognized: Sometimes the scaling of the images differs from the one I made the sample. Or the font has got less thicknes, see example below:
I think I'm on the wrong way to detect the values from the images. Is there a better or convience way how to solve this problem?

Cropping rectangle arround a face/coordinates

I looking for a way to calculate a rectangle (x,y,width & height) which can be used for cropping an image around the coordinates of a selected face.
I have an image 995x1000 (https://tourspider.blob.core.windows.net/img/artists/original/947a0903-9b64-42a1-8179-108bab2a9e46.jpg) by which the center of the face is located at 492x325. I can find this information using various services so even for multiple faces in an image I'm ableto find the most prominent - hence a single coordinate.
Now i need to make various sized cropped images from the source image (200x150, 200x200 & 750x250). Now I can't seem to solve how to best calculate a rectangle around the center coordinates while taking into account the edges of the images. The face should be as central as possible in the image.
Even after experimenting with various services (https://www.microsoft.com/cognitive-services/en-us/computer-vision-api) the result are pretty poor as the face, mainly in the 750x250, is sometimes not even present.
I'm also experimenting with the ImageProcessor (http://imageprocessor.org/) library with which you can use anchors for resizing but can't get the desired result.
Does anybody has an idea on how best crop around predefined coordinates?
Using Imageprocessor I created the following solution. It is not yet perfect but goes a long way ;)
public static void StoreImage(byte[] image, int destinationWidth, int destinationHeight, Point anchor)
{
using (var inStream = new MemoryStream(image))
using (var imageFactory = new ImageFactory())
{
// Load the image in the image factory
imageFactory.Load(inStream);
var originalSourceWidth = imageFactory.Image.Width;
var originalSourceHeight = imageFactory.Image.Height;
// Resizes the image until the shortest side reaches the set given dimension.
// This will maintain the aspect ratio of the original image.
imageFactory.Resize(new ResizeLayer(new Size(destinationWidth, destinationHeight), ResizeMode.Min));
var resizedSourceWidth = imageFactory.Image.Width;
var resizedSourceHeight = imageFactory.Image.Height;
//Adjust anchor position
var resizedAnchorX = anchor.X/(originalSourceWidth / resizedSourceWidth);
var resizedAnchorY = anchor.Y/(originalSourceHeight/resizedSourceHeight);
if (anchor.X > originalSourceWidth || anchor.Y > originalSourceHeight)
{
throw new Exception($"Invalid anchor point. Image: {originalSourceWidth}x{originalSourceHeight}. Anchor: {anchor.X}x{anchor.Y}.");
}
var cropX = resizedAnchorX - destinationWidth/2;
if (cropX < 0)
cropX = 0;
var cropY = resizedAnchorY - destinationHeight/2;
if (cropY < 0)
cropY = 0;
if (cropY > resizedSourceHeight)
cropY = resizedSourceHeight;
imageFactory
.Crop(new Rectangle(cropX, cropY, destinationWidth, destinationHeight))
.Save($#"{Guid.NewGuid()}.jpg");
}
}

Desktop capture in png format

I will like to get a screen capture and save it in the format of png of the entire screen. How can I do that?
Could I use the Snipping Tool library to accomplish this? There are some tutorials on the internet that show you how to do this with windows forms and the image is in the format of bitmap.
Here's a little method to capture the contents of any screen.
private static void CaptureScreen(Screen window, string file)
{
try
{
Rectangle s_rect = window.Bounds;
using (Bitmap bmp = new Bitmap(s_rect.Width, s_rect.Height))
{
using (Graphics gScreen = Graphics.FromImage(bmp))
gScreen.CopyFromScreen(s_rect.Location, Point.Empty, s_rect.Size);
bmp.Save(file, System.Drawing.Imaging.ImageFormat.Png);
}
}
catch (Exception) { /*TODO: Any exception handling.*/ }
}
Example of usage:
CaptureScreen(Screen.PrimaryScreen, #"B:\exampleScreenshot.png");
EDIT: Coming back to this later I realized it's probably more useful to return an Image object from the function so you can choose how to use the captured bitmap.
I've also made the function a bit more robust now so that it can capture multiple screens (i.e. in a multi-monitor setup). It should accommodate screens of varying heights, but I can't test this myself.
public static Image CaptureScreens(params Screen[] screens) {
if (screens == null || screens.Length == 0)
throw new ArgumentNullException("screens");
// Order them in logical left-to-right fashion.
var orderedScreens = screens.OrderBy(s => s.Bounds.Left).ToList();
// Calculate the total width needed to fit all the screen into a single image
var totalWidth = orderedScreens.Sum(s => s.Bounds.Width);
// In order to handle screens of different sizes, make sure to make the Bitmap large enough to fit the tallest screen
var maxHeight = orderedScreens.Max(s => s.Bounds.Top + s.Bounds.Height);
var bmp = new Bitmap(totalWidth, maxHeight);
int offset = 0;
// Copy each screen to the bitmap
using (var g = Graphics.FromImage(bmp)) {
foreach (var screen in orderedScreens) {
g.CopyFromScreen(screen.Bounds.Left, screen.Bounds.Top, offset, screen.Bounds.Top, screen.Bounds.Size);
offset += screen.Bounds.Width;
}
}
return bmp;
}
New example:
// Capture all monitors and save them to file
CaptureScreens(Screen.AllScreens).Save(#"C:\Users\FooBar\screens.png");

Best way to get a glow effect windows phone 7

I'm messing around with the Windows Phone 7 sdk and I'm trying to make the screen look like an old fashion digital display. Right now I'm trying to figure out how to make the text "glow" like one of those cool digital clocks. This is the sort of thing I'd assume you would look in to using shaders for, but it seems that shaders are disabled for use on the Windows Phone 7 OS. Any ideas? To be more specific, I want the text to look as though it is a light source and have the color "bleed" out slightly from the actual font.
I'd say its a choice between using an image as a font or blurring with WriteableBitmap.
Using a pre-made font image allows you to make the letters a complex as you want and should perform well. SpriteFont2 is convenient as it can generate the SpriteSheet with effects like glow, stroke, shadow and export an xml file containing the letter positions.
Add the generated png, and xml files to your solution and change the Build Action to content also check you have referenced System.Xml.Linq.
The following class can then be used.
public static class BitmapFont
{
private class FontInfo
{
public FontInfo(WriteableBitmap image, Dictionary<char, Rect> metrics)
{
this.Image = image;
this.Metrics = metrics;
}
public WriteableBitmap Image { get; private set; }
public Dictionary<char, Rect> Metrics { get; private set; }
}
private static Dictionary<string, FontInfo> fonts = new Dictionary<string, FontInfo>();
public static void RegisterFont(string fontFile, string fontMetricsFile)
{
string name = System.IO.Path.GetFileNameWithoutExtension(fontFile);
BitmapImage image = new BitmapImage();
image.SetSource(App.GetResourceStream(new Uri(fontFile,UriKind.Relative)).Stream);
var metrics = XDocument.Load(fontMetricsFile);
var dict = (from c in metrics.Root.Elements()
let key = (char)((int)c.Attribute("key"))
let rect = new Rect((int)c.Element("x"), (int)c.Element("y"), (int)c.Element("width"), (int)c.Element("height"))
select new { Char = key, Metrics = rect }).ToDictionary(x => x.Char, x => x.Metrics);
fonts.Add(name,new FontInfo(new WriteableBitmap(image),dict));
}
public static WriteableBitmap DrawFont(string text, string fontName)
{
var font = fonts[fontName];
var letters = text.Select(x => font.Metrics[x]).ToArray();
var height = (int)letters.Max(x => x.Height);
var width = (int)letters.Sum(x => x.Width);
WriteableBitmap bmp = new WriteableBitmap(width, height);
int[] source = font.Image.Pixels, dest = bmp.Pixels;
int sourceWidth = font.Image.PixelWidth;
int destX = 0;
foreach (var letter in letters)
{
for (int sourceY = (int)letter.Y, destY = 0; destY < letter.Height; sourceY++, destY++)
{
Array.Copy(source, (sourceY * sourceWidth) + (int)letter.X, dest, (destY * width) + destX, (int)letter.Width);
}
destX += (int)letter.Width;
}
return bmp;
}
public static Rectangle[] GetElements(string text, string fontName)
{
var font = fonts[fontName];
return (from c in text
let r = font.Metrics[c]
select new Rectangle
{
Width = r.Width,
Height = r.Height,
Fill = new ImageBrush {
ImageSource = font.Image,
AlignmentX=AlignmentX.Left,
AlignmentY=AlignmentY.Top,
Transform = new TranslateTransform { X = -r.X, Y = -r.Y },
Stretch=Stretch.None
},
}).ToArray();
}
}
Usage
//Register the font once.
BitmapFont.RegisterFont("Font.png", "Metrics.xml");
//Draws the text to a new bitmap, font name is image name without extension.
image.Source = BitmapFont.DrawFont(DateTime.Now.ToLongTimeString(), "Font");
//Alternatively put these elements in a horizontal StackPanel, or ItemsControl
//This doesn't create any new bitmaps and should be more efficient.
//You could alter the method to transform each letter too.
BitmapFont.GetElements(DateTime.Now.ToLongTimeString(), "Font");
If you want to blur see a BoxBlur implementation here or use WriteableBitmapEx.Convolute.
You should make a copy of the TextBlock you want to give the glow to. Bind the text property of the new item to the original item's text property (using ElementName binding). Do the same for location / height / width etc or any other property that you feel that will change on the original item. Set a transparency on the new item as well as a Blur Effect. This will give you a cool glow effect you want.
Several ways
Using a Border for a Glow Effect in Silverlight 3
Creating a Glowing effect in Silverlight
Glow effect in Silverlight
I believe all of them can be used in WP7 as well.

Categories