I'm currently working on an educational Windows store app in which the user will be able to trace over letters (e.g., A, B, C, but not limited to English) to learn the basics of writing.
How can I detect input and then compare it to an image mask of a letter using C# and XAML?
To do this you will need to have some way to rasterize your text that you want the user to trace over. Then, in order to provide feedback on whether they traced it correctly you'll need to continuously listen for the draw event and compare the input to what they should be drawing.
Basically, if a user draws a certain path or set of paths on a canvas, you'll want to be able to provide instant feedback on if they got it right yet. To give you some direction for this, I recommend you read this answer on SO, which roughly describes how to capture input and draw it on a canvas.
From there you should be thinking in terms of matching the user's input to an image of the letter they're supposed to be drawing. This requires some amount of image matching. To get you started, I recommend reading through all the answers to this post on SO.
Since you seem to be lacking direction in general, here's an idea of how your program could be structured:
Load the current letter to be drawn, and make sure to perform the appropriate calculations to pre-determine as much as possible for comparing to the input. Based on the second link above, this means you should call GetPixel for the letter to be drawn before the user is allowed to start tracing it (also note that you may want to downscale the image for better performance). You will also need to decide what your match threshold will be. Try starting with something like 70%.
Capture the user's input on a canvas, as explained in the first link. You'll probably want to adjust the brush width, but that post is a great start.
In the MouseMove event, you will also want to occasionally check to see how well it matches with the letter they're supposed to be drawing.
Once the user's input is within your match threshold, move on to the next letter. You may want to consider providing them with how well they did, based on the match percentage.
Experiment with values such as the brush width, image resolution, and how often you compare the input to the letter. Also try to do as much of the image processing as you can while displaying a 'loading' prompt when moving to the next letter to be drawn.
Related
I'm learning how to code C# from scratch. Right now, I'm learning how to build a scene on the command line and update it as necessary.
Part of what i want to do is to "scroll text as it is generated".
The scene is a square command line (Size 100 by 40) and outlined by a box of "#" characters. This is the canvas on which everything is presented.
As the user interacts via simple inputs, the program will provide textual feedback, like a text-based game.
However, unlike a normal text-based game, I cannot allow for the natural flow of the command line to move the canvas. In other words, if I allow for the command line to act naturally, then the canvas would slowly travel upwards until it's no longer visible.
I need a way to read whatever is already being presented on the screen and storing it on memory. Then, I can delete a portion of the screen an paste the previously copied information shifted upwards, making space for new information.
Here is a mockup of what I want to do:
Image 1: desirable outcome
My question is: Is there a way to "read" only a specific portion of the command space and store into memory?
I know you can store whatever is being printed at the moment of printing, or I could keep track of a certain number of previously displayed information and print it again, but I would like to "copy, then paste shifted" a portion of the screen.
PS: This is what I'm trying to avoid
Image 2: Undesirable outcome
Since the top line needs to be moved too, really the only way to do this is to clear the screen and redraw everything. Instead of printing directly to the screen, save the new line to a list, clear the screen, then print that list to the screen, together with the border.
While I haven't found a way to store a portion of the command line to a variable for later use, I've succesfully "scrolled" the text, and found a workaround to "store" it.
Simply put, Console.MoveBufferArea() allows one to select a portion of the command line buffer and move it somewhere else on the buffer.
This way I can:
-Clear whatever space i need at the top by moving the cursor to, say, (1,1) and writing a box of spaces via a string.
-Console.MoveBofferArea() the portion of the buffer a few rows upwards
-Write the new text at the bottom
As a byproduct, i can also "store" a portion of the screen for late use by:
-Setting the buffer to a wider area with Console.SetBufferSize()
-Use Console.MoveBufferArea() to the buffer portion that is outside of normal view
-Do whatever i need to do on the screen
-Use Console.MoveBufferArea() to retrieve the "stored" portion back to where i need it
-Reduce the buffer area (I could also just leave it be)
I have no experience with images. I have to detect simple object in static image. For example I have image like:
I want to detect edges and remove background. Just to compare them.
Something like this.
Do u have any solutions of this problem? Images have often white backgrounds.
I've just thought about detect edges, and take everything what they contains.
To segment out the shoe-
Anadptive Threshold to remove the smooth changing background.
Sobelx, which removes the apparent background line, which i assume is
common for images of this setup.
dilate, closing operation to separate out the shoe.
Find contours, bounding box etc as per your choice.
Do an additional threshold if you want to remove the shadow at the bottom.
I'm working on an experimental project in which the challenge is to identify and extract an image of the icon or control that the user is has clicked on/touched. The method I'm trying is as follows (I need some help with step 3):
1) Take a screen shot when the user clicks/touches the screen:
2) Apply edge detection:
3) Extract the possible icon images around the Point associated with the user's cursor (Don't know how to do this)
There are easier cases in which the mouse-over event will highlight the icon/control, which allows me to identify the control with a simple screen shot comparison (before and after mouse-over). The above method is specifically for cases in which the icon is not highlighted. I'm new to emgu, so if anyone has any pointers on how to better achieve this, I'm all ears.
Cheers! Matt
Instead of doing edge detection. Consider taking the following steps:
Only grab pixels which are within a certain radius of the point of the user's cursor. Create a new image with just these pixels.
Use thresholding to classify into foreground and background.
Calculate the centroid, (use mean x coordinate and mean y coordinate). Calculate deviation from the mean. Discard foreground pixels which are beyond a certain deviation from the mean. Eg: discard pixels that are more than 1.6 deviations from the mean.
(You may need to experiment with this step ).
Use a convex hull to find the area of the image with the icon in it.
I set up a draw rectangle to draw simple formatted text first aligned to the left as
*item 1
[1]Something
content
[2]Something else
<a> subsomething else
content
<b> another subsomething else
content
*item 2
The end.
and I would also like it to automatically create a new column (after checking for the longest string in the first column [drawn stuff on the left hand side]) to draw the rest into it.
In order to keep track of the paddings and itemized sections and subsections, I think of using a stack which I can push and pop the current and next positions needed to draw a text line each time I leave a content. Yet, I can't figure out how to jump back to a certain subsection position because stack doesn't offer an inline sub-scripting method.
Then I look into a hash-map (in C# I have tried Dictionary) to keep track of it and to access the value via specific key. For that I also use a external global variable to maintain the number of subsections the user may have entered and increase one each time a new subsection is created; and the float value is used to store the x-coordinate value for the drawstring to be done. This is complicated to me at least at present when I don't really have a nerve to go into it anymore. I can only receive false simulated outcomes.
So I am asking for an easier approach to tackle this problem, which I think is simple to many of you sure experiencing the same situation. I am desperately looking forward to seeing a short easy method to do this.
Draw formatted text using ..
..whatever works. I suggest a JLabel, which will render (simple) HTML/CSS formatted content.
See LabelRenderTest.java for an example.
Basically I'm reading vehicle license plates using tessract OCR, however despite being able to emphasise text easily enough via changing contrast, reducing noise an so on, some 'parts' of the vehicle remain on the image which does cause the OCR to throw bad results.
For example take:
I can change this easily enough, such as:
I'm looking to eliminate the edges off each plate, here's another example:
I could remove the edges using pixel manipulation algorithm, however I don't feel it's the right method, and would cause quite a lot of problems.
I've been using the following application to test various methods such as morphology and eliminating the unwanted data, so far I haven't been successful.
http://www.codeproject.com/KB/GDI-plus/Image_Processing_Lab.aspx
However someone with knowledge of this could use the application on the article above to achieve want I'm trying, so feel free to give it a try.
Thanks
Please try to use stroke width transformation concept.
This concept use to segment text from natural images.....
I already did such an algorithm. I just can say that it works great. The secret is, that you need to know that the light is coming just from one side perhaps. You cannot set the image to "black/white" just by using ONE threshold.
Detect the average luminance of parts of the image and use this luminance calculation to set the threshold for each region.
For example, if the left top is lighter, you need a lower threshold to make these parts not to bright. And if the bottom right has low light, you need to set the threshold higher to receive all existing light information.
Then, you need just to drive into the image from each side by using the method:
IsPixelAboveThreshold ?
If it is below, you are on the border, if it is above, you can say you are on the middle of the image with more brightness.
Regards