Find a red rectangle on the game screen - c#

I know there's been many topics similar to this one but none of them had a precise answer to what I'm searching for so please if anybody knows, and also I'm doing it in C#.
You all probably know (FPS) games and on the game screen with resolution say 1024x768 i need to find a red rectangle(which is the enemy) and move the mouse to it.
So my main problem is to find that red rectangle. OK so here's what I've tried so far:
I've tried the AForge and ran out of memory:
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0);
TemplateMatch[] matchings = tm.ProcessImage(image1.Clone(r,
System.Drawing.Imaging.PixelFormat.Format24bppRgb), image2);
I've used CopyfromScreen to create a image1 and image2 is a template I have.
I've tried the LockBits, so I can create the 2 dimensional code array for a bitmap and find the code for color red and try to ID if it's a rectangle but the idea seems very complicated been stuck here for a 4 days now.
The web is full of info on this but the more I go into the more I get confused :(
Anyway please ppl help me out here:

Well, first I need to say that this will be probably slow, so if the red rectangle is moving fast, you will need some other solution. C++, CUDA, etc...
First:
Save a image of the red rectangle.
Define the area of possible locations of the red rectangle.
Steps:
Capture the game image (you can use Graphics CopyFromScreen).Copy only the area that the red rectangle might be, to reduce processing time.
Use EmguCV MatchTemplate to find the position of the red
rectangle.
Move the mouse to the position.
Some thread sleep and Repeat 1...
To process the image use EmguCV
To control the mouse use MouseKeyboardActivityMonitor
Speedup note: EmguCV have some CUDA support now, so you can try to use the CUDA version of the method.
//You can check if a 8bpp image is enough for the comparison,
//since it will be much more faster. Otherwise, use 24bpp images.
//Bitmap pattern = "YOUR RED RECTANGLE IMAGE HERE!!"
Point location = Point.Empty;
//source is the screen image !!
Image<Bgr, Byte> srcBgr = new Image<Bgr, Byte>(source);
Image<Bgr, Byte> templateBgr = new Image<Bgr, Byte>(pattern);
Image<Gray, Byte> src;
Image<Gray, Byte> template;
src = srcBgr.Convert<Gray, Byte>();
template = templateBgr.Convert<Gray, Byte>();
Image<Gray, float> matchFloat;
matchFloat = src.MatchTemplate(template, TM_TYPE.CV_TM_CCOEFF_NORMED);
if (debug)
{
//matchFloat.Save(#"match.png");
//Process.Start(#"D:\match.png");
}
//Gets the max math value
double max;
double[] maxs, mins;
Point[] maxLocations, minLocations;
matchFloat.MinMax(out mins, out maxs, out minLocations, out maxLocations);
max = maxs[0];
if (max > threshold)
{
location = maxLocations[0];
//Point the mouse... Do your stuff

Related

Application C# image quality [duplicate]

This question already exists:
Application about C# image quality [closed]
Closed 2 years ago.
I am developing an application with C#. In the application I developed, I take the picture file with the user selection and transfer the picture onto the form.
After this step, I want to make changes to the bitmap of the picture. For example resizing the image in small sizes and converting to 1bpp color format.
I can do these now, I can resize the image and convert it to 1bpp color format, but at this point I think I have quality problems.
For example, when I take a screenshot of a text and send it to the program, I see that the letters in the text are unclearly bad when I view it in the resized 1bpp color format.
I show the algorithms I used and the screenshot of the application:
Bitmap bmp = new Bitmap(ResizeImage.filePath);
Bitmap bmpOriginalRGB = Helper.ImageResize(bmp, 512, 384);
pcbox1.Image = bmpOriginalRGB;
Bitmap bmpResizeRGB = Helper.ImageResize(bmp, ResizeWidth, ResizeHeight);
pcbox2.Image = bmpResizeRGB;
Bitmap bmpResize1BPP_1 = Helper.ConvertTo1BppImage(Helper.ImageResize(bmp, ResizeWidth, ResizeHeight));
pcbox3.Image = bmpResize1BPP_1;
Bitmap bmpResize1BPP_2 = bmpResizeRGB.Clone(new Rectangle(0, 0, ResizeWidth, ResizeHeight), PixelFormat.Format1bppIndexed);
pcbox4.Image = bmpResize1BPP_2;
Bitmap oledBitmap = bmpResize1BPP_2;
and
public static Bitmap ImageResize(Image image, int width, int height)
{
var destRect = new Rectangle(0, 0, width, height);
var destImage = new Bitmap(width, height);
destImage.SetResolution(image.HorizontalResolution, image.VerticalResolution);
using (var graphics = Graphics.FromImage(destImage))
{
graphics.CompositingMode = CompositingMode.SourceCopy;
graphics.CompositingQuality = CompositingQuality.HighQuality;
graphics.InterpolationMode = InterpolationMode.HighQualityBicubic;
graphics.SmoothingMode = SmoothingMode.HighQuality;
graphics.PixelOffsetMode = PixelOffsetMode.HighQuality;
using (var wrapMode = new ImageAttributes())
{
wrapMode.SetWrapMode(WrapMode.TileFlipXY);
graphics.DrawImage(image, destRect, 0, 0, image.Width, image.Height, GraphicsUnit.Pixel, wrapMode);
}
}
return destImage;
}
Here's a first screenshoot
Here's a second screenshoot
Here's a third screenshoot
For example, as in the second screen image, the shadow of the woman appears as black. And I don't want that. I want to see a clearer black and white image.
Do you think it is possible to further improve this picture quality?
Edit: Sorry for translate..
I don't believe your problem is the quality on resizing, it's rather the conversion from greyscale to black and white.
When resizing, the color of the adjacent pixels are averaged together to get a new color approximating the set of pixels. So going from a 200x200 pixel image to a 100x100 pixel image, a set of 4 pixels becomes a single pixel. It will still look fairly good, but the solid black text will become a series of gray pixels. The exact formula will vary by the interpolation method used. But when resizing a nice piece of black and white text, the text will end up lighter or more grey (which will be important later)
However, when going from greyscale (or full color) to black and white each pixel ends up being either black and white, there are no other options, but there are different algorithms used to decide which pixels end up black or white, often called dithering. (https://en.wikipedia.org/wiki/Dither)
Your first b/w image appears to use some form of error diffusion, quite possibly Floyd-Steinberg. It tends to work fairly well on real wold images and turns grey areas into spaced out black pixels visually approximating the greyness of the area they fall in.
Your second b/w image appears to be a simple threshold algorithm. Basically pixels darker than a certain color end up black, and all the rest end up white. You can adjust the image by simply setting which color is used as the threshold. Often this works well with text, but you will need to adjust the value used as the threshold, but I don't know if the libraries you are using allow for this or not. I have found what works well for programmatically selecting the threshold is to total up how many pixels there are of each color, and then assume some percent will be black (text tends to be mostly white space), then choose the threshold that gives you that number of black pixels.
And there are many other dithering algorithms that you can try, as well as edge detection algorithms. You can also try adjusting the contrast of the image before converting to b/w.
However, at the end of the day, when resolution is low (pixels per character), it may not be possible to easily convert them to b/w and have them still be readable (just try to fax small text on a fax machine in standard mode). Remember your resize removed a lot of information and the conversion from 8 bit to 1 bit removed another 87% of the information in the resized picture.

Extracting information from a video game screenshot

I am trying to use image processing to detect and extract information such
Attacker hero
Victim hero
However my method is not that accurate and I could use some ideas to help make it better.
I have a screenshot of a killfeed from a game:
I use Emgu to perform a canny edge detection of the image:
using (Image<Bgr, Byte> img = new Image<Bgr, byte>(bitmap))
{
//Convert the image to grayscale and filter out the noise
using (Image<Gray, Byte> gray = img.Convert<Gray, Byte>().PyrDown().PyrUp())
{
Image<Gray, Byte> cannyEdges = gray.Canny(40, 100);
//Smooth out found edges
cannyEdges._SmoothGaussian(3);
}
}
And get this result:
From the result I loop through the image to find a straight line with a width of 2 to then detect the border of the icon.
This however is really inaccurate as you can see the canny edged image has lines that I am not interested in, and the second icon does not even have a clear enough border for me to detect.
Is there any way better way to detect the icons?
Before using hard algorithm like cannyEdge and other fft things, I would recommand to use common sense and eyes to :
select a subset of the image you want to search In
clean it
For instance, you may want to :
filter on cyan color in order to detect the cyan square and/or cyan name (it also works with red).
search for the white arrow that is always the same, then deduce where the squares are
Once it's clean, you'll have only the zone you want to process (for instance both squares with heroes). Then all kinds of algorythms you will use will be much more effective.
If the heroes pics always are the same you even may want to perform a simple difference between screen-zone and reference.

Differences between two images

Using this image as a refernce I want to find the dirty spot at the bottom of this image.
I was able to turn second image to the same scale and orientation and now trying fo find the spot using absdiff but since images are not perfectly matchet I have edges on the diff image.
I'm thinking that instead of difference between pixels with same coordinates I need find minimal difference in an area like n by n pixels. So the question is: does OpenCV have something buitin for that and/or is there better solution to the problem?
EDIT: Solution using threshold and erode:
public static Image<Bgr, Byte> Diff(Image<Bgr, Byte> image1,
Image<Bgr, byte> image2,
int erodeIterations=2)
{
return Diff(image1, image2, new Bgr(50, 50, 50), erodeIterations);
}
public static Image<Bgr, Byte> Diff(Image<Bgr, Byte> image1,
Image<Bgr, byte> image2,
Bgr thresholdColor,
int erodeIterations)
{
var diff = image1.AbsDiff(image2);
diff = diff.ThresholdToZero(thresholdColor);
diff = diff.Erode(erodeIterations);
return diff;
}
Not sure about OpenCV, but solving this problem shouldn't be difficult. Align the two images and find the difference image as you've already done. Use a NxN sliding window across the difference image, and compute the number of pixels that are significantly different within the window i.e. ignore differences of up to say 10 grey levels. Find the maxima of these sums over the entire image, and this should highlight what you need.
You could create a similarity map where each pixel will be assinged with the histogram comparison value calculated on an n by b area focused on each pixel.

emgu finding image a in image b [duplicate]

This question already has answers here:
opencv finding image cordinates on another image
(3 answers)
Closed 9 years ago.
I'm new to emgu and would like some advice on where to start.
I've looked through the shape detection but its far too complex for what i need .. i think.. and my surfexample isn't working. I get this error:
Cannot get SURF example in EMGU.CV to work?
Anyway, this is what i would like to do: Find image A in image B. Image A is a simple square which always has the same grey 1 pixel border and always the same size (i believe) but the inner colour could be black or one of about 7 other colours (only ever a solid colour). i need to find the coordinates of image A in image b when i press a button. see the below images.
Image B
And
Image A
Goosebumps answer is correct, but I thought that a bit of code might be helpful also. This is my code using MatchTemplate to detect a template (image A) inside a source image (image B). As Goosebumps noted, you probably want to include some grey around the template.
Image<Bgr, byte> source = new Image<Bgr, byte>(filepathB); // Image B
Image<Bgr, byte> template = new Image<Bgr, byte>(filepathA); // Image A
Image<Bgr, byte> imageToShow = source.Copy();
using (Image<Gray, float> result = source.MatchTemplate(template, Emgu.CV.CvEnum.TM_TYPE.CV_TM_CCOEFF_NORMED))
{
double[] minValues, maxValues;
Point[] minLocations, maxLocations;
result.MinMax(out minValues, out maxValues, out minLocations, out maxLocations);
// You can try different values of the threshold. I guess somewhere between 0.75 and 0.95 would be good.
if (maxValues[0] > 0.9)
{
// This is a match. Do something with it, for example draw a rectangle around it.
Rectangle match = new Rectangle(maxLocations[0], template.Size);
imageToShow.Draw(match, new Bgr(Color.Red), 3);
}
}
// Show imageToShow in an ImageBox (here assumed to be called imageBox1)
imageBox1.Image = imageToShow;
You could have a look at http://docs.opencv.org/doc/tutorials/imgproc/histograms/template_matching/template_matching.html
This is probably what you are looking for. Your Black square would be the template. You may try to also include a little bit of grey around it. This will keep the detector from fireing on large black areas.

C#, Emgu webcam - choose capture size

I'm using the Emgu library for integrating the open CV webcam features in C#.
I use this code for choosing the capture device and setting its size:
camera = new Capture(0);
camera.SetCaptureProperty(CAP_PROP.CV_CAP_PROP_FRAME_WIDTH, videoSettings.width);
camera.SetCaptureProperty(CAP_PROP.CV_CAP_PROP_FRAME_HEIGHT, videoSettings.height);
Then I display it in an imageBox like this: imageBox1.Image = camera.QueryFrame();
Then to capture a snapshot of the current frame I use this code:
Image<Bgr, byte> snapshot = camera.QueryFrame();
snapshot.Save("snapshot.jpg");
Though I would want to be able to save the snapshot at a higher resolution than the preview window.
But the problem is that as far as I know I can't create a new "Capture" object using the same webcamera. So I'm wondering if it is maybe possible to set the camera.setCaptureProperty height and width to let's say 1028x720 but then in some way crop it for displaying it in the imageBox with the resolution of 514x360?
Or is there any other way to do this?
I solved this by using
imageBox1.SizeMode = PictureBoxSizeMode.StretchImage;
I solved this by using the Resize() method in QueryFrame()
currentFrame = grabber.QueryFrame().Resize(320, 240, Emgu.CV.CvEnum.INTER.CV_INTER_CUBIC);

Categories