Actually I'm using AForge.NET to recognize suit and value from a gamecard.
Here is a snippet where I decide which suit/value is there:
public static CardTemplateIdentifier GetBestMatchingIdentifier(this List<CardTemplateIdentifier> templates, Bitmap bitmap)
{
float maxSimilar = 0f;
CardTemplateIdentifier result = null;
foreach (var template in templates)
{
// Identify similarity between template and bmp
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0);
TemplateMatch[] matchings = tm.ProcessImage(bitmap, template.Sample);
// If the currently tested template fits better than the best one so far,
// set the value as the identified card value
if (matchings.Length > 0 && matchings[0].Similarity > maxSimilar)
{
maxSimilar = matchings[0].Similarity;
result = template;
}
}
return result;
}
CardTemplateIdentifier contains a list with all possible cards and their comparism sample.
...
Now If trying to recognize the images, I can not rely on this being reliably recognized: Sometimes the scaling of the images differs from the one I made the sample. Or the font has got less thicknes, see example below:
I think I'm on the wrong way to detect the values from the images. Is there a better or convience way how to solve this problem?
Related
I've got a strange problem. There is an image. Unfortunately it's too big to show it in question. But you can download it. If you can't do it from OneDrive this is another way.
This image seems to be ordinary, but it's not.
When we open properties we will see this:
We need to keep in mind dimensions of this picture: Width is 3000px and Height is 4000px. It looks correct because image is portret.
Then lets try to read it with C#:
private static void TestImage()
{
using (FileStream file1 = new FileStream("DSC_2446.JPG", FileMode.Open))
{
Console.WriteLine("DSC_2446.JPG :");
using (var img1 = System.Drawing.Image.FromStream(file1))
{
Console.WriteLine($" Width = {img1.Width}");
Console.WriteLine($" Height = {img1.Height}");
}
}
Console.Read();
}
And in results we see some magic!!!
So I've got completely wrong values. Values are switched between properties.
Does someone know why it can happens and how to detect/fix this behavior?
The problem is the EXIF version. You can use this site to get the real data https://exif.tools/meta/Exif-Version/0231 and you will see
Also according to this Post you can get your image's orientation by
var orientation = (int)img1.GetPropertyItem(274).Value[0];
//orientation = 6
the value 6 means rotate 90 degrees.
There is the reference of the value 6. https://exiftool.org/TagNames/EXIF.html
Basically i'm trying to check if a face is upside down in an image using this library https://github.com/takuya-takeuchi/FaceRecognitionDotNet.
Take example of the image below
This is an image that is successfully detected using the FaceRecognition.Net library.The image is upside down.I have marked all face landmarks in the image with a blue ellipses.
This is the approach i follow
// Finding faceparts
var faceparts = dparameters._FaceRecognition.FaceLandmark(dparameters.FCImage);
// Drawing Ellipses over all points got from faceparts
foreach(var facepart in faceparts) {
foreach(var mypoint in facepart.Values) {
foreach(var x in mypoint) {
tempg.DrawEllipse(Pens.Blue, x.Point.X, x.Point.Y, 2, 2);
}
}
}
Now i'm checking if the image is rotated by comparing maximum Y coordinates of the lip and eyepoints
var temp = faceparts.FirstOrDefault();
IEnumerable < FacePoint > lippoints;
temp.TryGetValue(FacePart.BottomLip, out lippoints);
IEnumerable < FacePoint > eyepoints;
temp.TryGetValue(FacePart.LeftEye, out eyepoints);
var lippoint = lippoints.Max(r => r.Point.Y);
var topeyepoint = eyepoints.Max(r => r.Point.Y);
if (lippoint > topeyepoint) {
bool isinverted = true;
} else {
bool isinverted = false;
}
The issue is that even when the image is not upside down, the eyecoordinate is less than the face coordinate.This is because a false face is detected as outlined in the image.How to get over this issue?
It looks like this library does not provide a confidence ratio for the results. Otherwise, I would suggest to try both the input and its flipped copy and take the one with higher confidence before doing the "eye over mouth" check.
So maybe what could help is:
using the CNN model, in the original library it is called by
face_locations = face_recognition.face_locations(image, number_of_times_to_upsample=0, model="cnn")
in the C# port it should be
_FaceRecognition.FaceLocations(image, 0, Model.Cnn)
That should give you a more accurate face bounding box which you can then compare with the bounding box of the landmarks. If you do the same for a flipped copy of the image, you can "emulate" the confidence I mentioned earlier and assume the orientation where the boxes match better. Then you can identify the orientation by the "eyes over mouth" test.
as far as I noticed the library does not provide pre-trained data, so in order to use the Cnn model you need to train it by yourself. Selection of the dataset for training is of course very important. If you already performed the training, more/better training data might improve the accuracy.
I have a MVC C# application that includes a .Net wrapper for tesseract-ocr nuget. The current version I am using is v4.1.0-beta1. The image that I am try to scan is shown below
My aim is to extract the player name and the number just above them to the left.
I tried making the OCR scan the field/pitch area but the results are way off base. So, I decided to section off all player names and all numbers as seen in the image below. Ratings area marked in blue and player names marked in red. As you can see the name and rating are always the same distance apart.
My current code setup is shown below.
public void Get(HttpPostedFileBase file)
{
using (var engine = new TesseractEngine(Path.Combine(HttpRuntime.AppDomainAppPath, "tessdata"), "eng+deu", EngineMode.Default))
{
var bitmap = (Bitmap)Image.FromStream(file.InputStream, true, true);
using (var img = PixConverter.ToPix(bitmap))
{
SetPlayerRatings(engine, img);
}
}
}
private void SetPlayerRatings(TesseractEngine engine, Pix img)
{
var width = 285;
var height = 76;
var textPositions = Service.Get<Formation>(this.FormationId).TextPositions.ToList();
foreach (var textPosition in textPositions)
{
var playerRating = GetPlayerData(engine, img, new Rect(textPosition.X, textPosition.Y, width, height));
}
}
private static PlayerRating GetPlayerData(TesseractEngine engine, Pix img, Rect region)
{
using (var page = engine.Process(img, region, PageSegMode.Auto))
{
var playerName = page.GetText();
}
var ratingRegion = new Rect(region.X1, region.Y1 - 52, 80, 50);
using (var page = engine.Process(img, ratingRegion, PageSegMode.Auto))
{
var playerRating = page.GetText();
}
}
This code is producing the correct results for the 1st image.
Is there any way to train OCR so that I dont have to workout the X and Y co-ordinates for each player position? I would like to just specify the area of the pitch and have OCR read in the rating followed by the player name.
With specifying coordinates you solved several problems regarding image processing. So if you do not want to specify coordinates, you have to deal with them: e.g. removing graphics component from OCR area like T-shirt, lines.
Next idea: Tesseract API has option GetComponentImages (I expect C# wrapper should provide it too - I am not familiar with C#), so you can iterate over found components.
I'm hoping someone can help me out here. My ultimate goal with this code is to extract the color of the sweater I am wearing. Like the title suggests, I'm trying to exctract RBG values from a certain Skeleton point (ie. skeleton.Joint[JointType.Spine].Position). I do this using the following mapping:
All of the following code is within the SensorAllFramesReady event:
private void SensorAllFramesReady(object sender, AllFramesReadyEventArgs e)
{
Skeleton[] skeletons = new Skeleton[0];
using (SkeletonFrame skeletonFrame = e.OpenSkeletonFrame())
{
if (skeletonFrame != null)
{
skeletons = new Skeleton[skeletonFrame.SkeletonArrayLength];
skeletonFrame.CopySkeletonDataTo(skeletons);
}
}
if (skeletons.Length != 0)
{
foreach (Skeleton skel in skeletons)
{
if (skel.TrackingState == SkeletonTrackingState.Tracked)
{
colorPoint = this.SkeletonPointToColor(skel.Joints[JointType.Spine].Position);
}
}
}
}
private Point SkeletonPointToColor(SkeletonPoint skelpoint)
{
ColorImagePoint colorPoint = this.sensor.CoordinateMapper.MapSkeletonPointToColorPoint(skelpoint, ColorImageFormat.RgbResolution640x480Fps30);
return new Point(colorPoint.X, colorPoint.Y);
}
I assign the returned Point to a variable "ColorPoint", and here is how I (somewhat successful) extract the RBG values:
using (ColorImageFrame colorFrame = e.OpenColorImageFrame())
{
if (colorFrame != null)
{
int arrayLength = colorFrame.PixelDataLength;
this.colorPixelData = new byte[arrayLength];
colorFrame.CopyPixelDataTo(this.colorPixelData);
blue = (int)colorPixelData[(int)(4* (colorPoint.X + (colorPoint.Y * colorFrame.Width)))+0];
green = (int)colorPixelData[(int)(4 * (colorPoint.X + (colorPoint.Y * colorFrame.Width))) + 1];
red = (int)colorPixelData[(int)(4 * (colorPoint.X + (colorPoint.Y * colorFrame.Width))) + 2];
}
}
I then draw an ellipse on my Windows Form window using the retrieved RBG values. Now this works, kind of. I do get a color which resembles the color of the sweater I'm wearing, but even if I do my best to stand very still the color is always changing. It's almost as if I'm getting random RBG values within a certain range, and only the range is dictated by the color of my sweater. Why is this? Is there another way I should be solving this problem?
Thank you for reading!
EDIT: I apologise for the formatting, this is my firs time submitting a question and I realise the formatting in the first code block is a bit off. The SkeletonPointToColor method is naturally not within the SensorAllFramesReady method. My apologies
First of all I really recomend you have a look at the Coding4Fun Kinect Toolkit which provides many useful functions for dealing with the Kinect data. There is a for instance an extension for returning a bitmap named ToBitmapSource() which should be of use.
Another observation is that without using some algorithm to get an average of the color values received it's quite normal to have the color values jump around. Also not confident that you can expect the skeleton and image frames to be 100% in sync.
I'm trying to build an application that solves a puzzle (trying to develop a graph algorithm), and I don't want to enter sample input by hand all the time.
Edit: I'm not trying to build a game. I'm trying to build an agent which plays the game "SpellSeeker"
Say I have an image (see attachment) on the screen with numbers in it, and I know the locations of the boxes, and I have the exact images for these numbers. What I want to do is simply tell which image (number) is on the corresponding box.
So I guess I need to implement
bool isImageInsideImage(Bitmap numberImage,Bitmap Portion_Of_ScreenCap) or something like that.
What I've tried is (using AForge libraries)
public static bool Contains(this Bitmap template, Bitmap bmp)
{
const Int32 divisor = 4;
const Int32 epsilon = 10;
ExhaustiveTemplateMatching etm = new ExhaustiveTemplateMatching(0.9f);
TemplateMatch[] tm = etm.ProcessImage(
new ResizeNearestNeighbor(template.Width / divisor, template.Height / divisor).Apply(template),
new ResizeNearestNeighbor(bmp.Width / divisor, bmp.Height / divisor).Apply(bmp)
);
if (tm.Length == 1)
{
Rectangle tempRect = tm[0].Rectangle;
if (Math.Abs(bmp.Width / divisor - tempRect.Width) < epsilon
&&
Math.Abs(bmp.Height / divisor - tempRect.Height) < epsilon)
{
return true;
}
}
return false;
}
But it returns false when searching for a black dot in this image.
How can I implement this?
I'm answering my question since I've found the solution:
this worked out for me:
System.Drawing.Bitmap sourceImage = (Bitmap)Bitmap.FromFile(#"C:\SavedBMPs\1.jpg");
System.Drawing.Bitmap template = (Bitmap)Bitmap.FromFile(#"C:\SavedBMPs\2.jpg");
// create template matching algorithm's instance
// (set similarity threshold to 92.5%)
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0.921f);
// find all matchings with specified above similarity
TemplateMatch[] matchings = tm.ProcessImage(sourceImage, template);
// highlight found matchings
BitmapData data = sourceImage.LockBits(
new Rectangle(0, 0, sourceImage.Width, sourceImage.Height),
ImageLockMode.ReadWrite, sourceImage.PixelFormat);
foreach (TemplateMatch m in matchings)
{
Drawing.Rectangle(data, m.Rectangle, Color.White);
MessageBox.Show(m.Rectangle.Location.ToString());
// do something else with matching
}
sourceImage.UnlockBits(data);
The only problem was it was finding all (58) boxes for said game. But changing the value 0.921f to 0.98 made it perfect, i.e. it finds only the specified number's image (template)
Edit: I actually have to enter different similarity thresholds for different pictures. I found the optimized values by trying, in the end I have a function like
float getSimilarityThreshold(int number)
A better approach is to build a custom class which holds all the information you need instead of relying on the image itself.
For example:
public class MyTile
{
public Bitmap TileBitmap;
public Location CurrentPosition;
public int Value;
}
This way you can "move around" the tile class and read the value from the Value field instead of analyzing the image. You just draw whatever image the class hold to the position it's currently holding.
You tiles can be held in an array like:
private list<MyTile> MyTiles = new list<MyTile>();
Extend class as needed (and remember to Dispose those images when they are no longer needed).
if you really want to see if there is an image inside the image, you can check out this extension I wrote for another post (although in VB code):
Vb.Net Check If Image Existing In Another Image