I currently have a winform application that relies relatively heavily on reading mouse click points. It's a video application that allows the user to click on incoming video feed from a webcam and draw lines on it and do other things. The important part for this question is the drawing lines- a simple example is that if they click somewhere on the video feed, it instantly draws a horizontal line across the entire picture at the point of the mouse.
This works flawlessly on 96dpi, but when windows scales to 125% zoom (Control Panel\All Control Panel Items\Display -> Medium - 125%), which scales it to 120 dpi, it all goes to pot. Suddenly the application reads the mouse as being clicked much farther down the picture than it actually is.
All I'm doing is pulling the exact location of the mouse upon mouseclick. It is literally as simple as
using (Graphics g = Graphics.FromImage(tmpImage))
{
g.DrawLine(new Pen(lineColor, lineWidth),
new Point(0, e.Location.X), new Point(tmpImage.Width,
e.Location.Y));
}
In 96dpi, it draws exactly at the mouse. At 120dpi, it draws quite a bit south of the mouse. This is a huge problem for me.
Is there any way that I can either make my application constantly run at 96dpi, or some other solution to cause my application to work appropriately under the circumstances of 120+dpi?
Thank you!
In order to force your application to run at 96 DPI write this into Form1_Load:
Graphics g = this.CreateGraphics();
if (g.DpiX != 96)
{
MessageBox.Show("Your message.");
this.Close();
}
Take a look at AutoScaleMode property. this is what you need.
By default AutoScaleMode is set to AutoScaleMode.Font typically in *.designer.cs Change that to AutoScaleMode.None for no scaling or you can use AutoScaleMode.Dpi with AutoScaleDimensions
Hope this helps
Related
So I have searched and can't seem to find the right resources or tools for what I am wanting to do so I thought I'd ask for some help. Below is what I am wanting to do.
I am making a .Net core application that will simulate some key strokes and that is all working fine, but I want to also move the mouse based on where certain text appears on the screen, however, I will not have control over the application where the text appears, so I will need to somehow either be streaming the desktop view and using something to analyze it, or constantly taking a screen shot every 10 seconds or so and analyzing that screen shot if the text is there and then move the mouse to that position of text and simulate a left click.
I am not really confident in the screenshot approach as I am not sure how I'd get the mouse coordinates, so I have a feeling I will need to feed a desktop stream into something in order to get the image I am analyzing and then overlay my own transparent overlay on top of that in order to move the cursor to appropriate position.
I hope this makes sense and any libraries or whatever that is recommended to do this is appreciated. I know how to move the cursor on the screen and left click, just haven't found on google the right library for analyzing a desktop screen in real time and getting coordinates based on searching conditions.
Thank you everyone.
So I figured out what I needed to do. Trying to match the text with and OCR proved to be impossible as the text was layered into an image where they actively tried to prevent detection, this is because its a video game and they don't want automation I guess. So what I did was determine I will always know what the button is going to look like, so I took a screen shot of the button and now I take screenshots of the game every 10 seconds and then search that image for my button template image. If the image has a match, then I generate coordinates and send mouse there and click it. Below is the code I used and you will need the libraries from AForge to use this code.
Bitmap sourceImage = (Bitmap)Bitmap.FromFile(#"C:\testjpg.jpg");
Bitmap template = (Bitmap)Bitmap.FromFile(#"C:\buttonjpg.jpg");
// create template matching algorithm's instance
//Resize value
var resizePercent = 0.4;
//Resizing images to increase process time. Please note this will result in skewed coordinates.
//You must divide the coordinates by your resizePercent to get native coordinates taken from screen shots.
sourceImage = new ResizeBicubic((int)(sourceImage.Width * resizePercent), (int)(sourceImage.Height * resizePercent)).Apply(sourceImage);
template = new ResizeBicubic((int)(template.Width * resizePercent), (int)(template.Height * resizePercent)).Apply(template);
// (set similarity threshold to 0.951f = 95.1%)
ExhaustiveTemplateMatching tm = new ExhaustiveTemplateMatching(0.951f);
// find all matchings with specified above similarity
TemplateMatch[] matchings = tm.ProcessImage(sourceImage, template);
// highlight found matchings
BitmapData data = sourceImage.LockBits(
new Rectangle(0, 0, sourceImage.Width, sourceImage.Height),
ImageLockMode.ReadWrite, sourceImage.PixelFormat);
foreach (TemplateMatch m in matchings)
{
Drawing.Rectangle(data, m.Rectangle, Color.White);
Console.WriteLine(m.Rectangle.Location.ToString());
var x = m.Rectangle.Location.X;
var y = m.Rectangle.Location.Y;
//Fixing the coordinates to reflex the origins of the original screenshot, not the resized one.
var xResized = (int)(x / resizePercent);
var yResized = (int)(y / resizePercent);
// do something else with matching
}
sourceImage.UnlockBits(data);
I did notice my coordinates were off usually around ~4.5%-8% from dead center(the button still gets clicked, but I want a perfect center click), so for me I am calculating a range of coordinates, which in this range are not very many to ensure the button I want to click is clicked and not missed and just having program click all the potential coordinates.
So I hope this helps someone!
Kind Regards,
Aaron
Depending on the application there are a number of approaches you might want to try.
For example, using Application Insights and UI Automation, you might be able to capture the text directly.
If you have the exe file, then you might be able to decompile it with something like dotPeek and then use Visual Studio's Live debugging feature to access the text.
You might be able to inspect the heap of the running process via a tool like HeapMemView.
Optical Character Recognition might be an option worth considering. In that case Tesseract is an open source OCR engine worth considering. This can be used in conjunction with screenshots to extract text.
I am developing a WinForms application in C#. It uses a panel to draw an image of the Mandelbrot fractal. In the manual for the program and whenever I post about it somewhere, I recommend people to set their scaling setting to 100%, as otherwise the images won't look nice. This is because on other settings, the image is scaled up after drawing it, and it becomes blurry. All other controls are blurry too.
For example: my panel is 500x500. The scaling in my Windows is set to 125%. When I run the program, the panel is internally still 500x500, but it appears as 625x625, blurry. Instead, when the program is run, I want the panel to internally resize to 625x625, and appear as 625x625 too.
I have found the following solution: I found out about SetProcessDPIAware() (from here). Setting that makes the window not scale (it appears as if it was at the 100% scale setting), but the text does (and without becoming blurry). I can then, at the start of the program, calculate the appropriate multiplier (dpi = DpiX / 96) and give that to a huge method that includes commands like
xentrylabel.Location = new Point((int)(xentrylabel.Location.X * dpi),(int)(xentrylabel.Location.Y * dpi));
xentry.Location = new Point((int)(xentry.Location.X * dpi), (int)(xentry.Location.Y * dpi));
xentry.Size = new Size((int)(xentry.Width * dpi), (int)(xentry.Height * dpi));
One for every control property that might need to be updated. While writing this question, I got this idea and got started with it. However, I realised that this will need very many lines of code, so I wonder if there isn't a built-in way to do this. It seems like an option that many would like to go for, rather than their applications becoming blurry or hard to read on screens with high dpi.
Is this way of manually correcting positions and sizes the way to go, or is there a built-in way to scale the form for other DPI settings by actually scaling everything in the form, instead of scaling the end result?
Edit: From some comments it seems as if SetProcessDPIAware() alone should scale up everything. But in my experience, it doesn't. Here are some screenshots:
Application on 100% scale setting:
https://i.stack.imgur.com/pws9B.png
Application on 125% scale setting without SetProcessDPIAware():
https://i.stack.imgur.com/cE4Wx.png
Application on 125% scale setting with SetProcessDPIAware():
https://i.stack.imgur.com/oVA8W.png
We solved the issue of rendering controls for high DPI screens with scaling greater than 100% in Chem4Word by creating a WPF user control which is hosted in an ElementHost control, which is set to fill the form.
WinForm
+- ElementHost
+- WPFUserControl
Looking at your uploaded images, that's exactly the problem which is solved by using a WPF user control.
I hope you can help me with this problem, attached videos to explain in a simpler way.
First example
Panel (has a textured background) with labels (the labels have a png image without background)
Events: MouseDown, MouseUp and MouseMove.
As you will notice in the video to drag the label the background turns white panel and regains its background image when I stop dragging the label
Panel controls have a transparent background as property, but changing the background with any color, let the problem occurred related to the substance, I do not understand why this happens and how to fix less.
Second Example
Contains the above, with the only difference that the panel controls instead of having transparent background, I chose black color for that property
You have to use double buffer and you don't have to stop using an image on the background, you can have everything running smoothly.
You have a couple of ways to do this, the fast way (not enough most of the time) is to enable doublebuffer of the panel.
The "slow" but better way is to do your own Double Buffer using a Bitmap object as a buffer.
This example creates a "side buffer" and accepts an image as parameter and draws it using created buffer.
public void DrawSomething(Graphics graphics, Bitmap yourimage)
{
Graphics g;
Bitmap buffer = new Bitmap(yourimage.Width, yourimage.Height, graphics);
g = Graphics.FromImage(buffer);
g.SmoothingMode = System.Drawing.Drawing2D.SmoothingMode.HighQuality;
g.DrawImage(yourimage, 0, 0);
graphics.DrawImage(buffer, 0, 0);
g.Dispose();
}
Call this on your OnPaint event.
BTW... this is just a double buffer example.
Cheers
Change DoubleBuffered to true for both form and panel. I think that should solve your problem.
this is totally normal, because System.Windows.Forms.Control based items were not designed to do this kind of advanced Graphics operations.
in fact the reason that this effect happens here, is that when you assign any value other than 255 to the alpha component of a control BackColor, the form does the following when you change the control size or position:
it sets the new control position
it redraws the parent control
it gets the background of the control's parent as an image
it draws acquired image into the control body to seem as if the control is transparent
the control body gets drawn on top of the previously drawn background
the control children are drawn
* this is is a simplified explanation for the sake of illustration to deliver the idea
steps 1, 2 are responsible for the flickering effect that you see.
but you have two ways to solve this,
-the first is some kinda advanced solution but it's very powerful, which is you would have to create a double buffered custom control that would be your viewport.
the second is to use WPF instead of windows forms, as WPF was designed exactly to do this kind of things.
if you can kindly provide some code, i can show you how to do both.
I want to draw directly on the desktop in C#. From searching a bit, I ended up using a Graphics object from the Desktop HDC (null). Then, I painted normally using this Graphics object.
The problem is that my shapes get lost when any part of the screen is redrawn. I tried a While loop, but it actually ends up drawing as fast as the application can, which is not the update rate of the desktop.
Normally, I would need to put my drawing code in a "OnPaint" event, but such thing does not exist for the desktop.
How would I do it?
Example code: https://stackoverflow.com/questions/1536141/how-to-draw-directly-on-the-windows-desktop-c
I posted two solutions for a similar requirement here
Basically you have two options.
1- Get a graphics object for the desktop and start drawing to it. The problem is if you need to start clearing what you have previously drawn etc.
Point pt = Cursor.Position; // Get the mouse cursor in screen coordinates
using (Graphics g = Graphics.FromHwnd(IntPtr.Zero))
{
g.DrawEllipse(Pens.Black, pt.X - 10, pt.Y - 10, 20, 20);
}
2- The second option that I provide in the link above is to create a transparent top-most window and do all your drawing in that window. This basically provides a transparent overlay for the desktop which you can draw on. One possible downside to this, as I mention in the original answer, is that other windows which are also top-most and are created after your app starts will obscure your top most window. This can be solved if it is a problem though.
For option 2, making the form transparent is as simple as using a transparency key, this allows mouse clicks etc. to fall through to the underlying desktop.
BackColor = Color.LightGreen;
TransparencyKey = Color.LightGreen;
When you draw to HDC(NULL) you draw to the screen, in an unmanaged way. As you've discovered, as soon as windows refreshes that part of the screen, your changes are overwritten.
There are a couple of options, depending upon what you want to achieve:
create a borderless, possibly
non-rectangular window. (Use
SetWindowRgn to make a window
non-rectangular.) You can make this a child of the desktop window.
subclass the desktop window. This is not straightforward, and involves
injecting a DLL into the
Explorer.exe process.
To get an OnPaint for the desktop you would need to subclass the desktop window and use your drawing logic when it receives a WM_PAINT/WM_ERASEBKGND message.
As the thread you linked to says, you can only intercept messages sent to a window of an external process using a hook (SetWindowsHookEx from a DLL).
As mentioned a transparent window is another way to do it, or (depending on the update frequency) copying, drawing and setting a temporary wallpaper (as bginfo does).
This is difficult to do correctly.
It will be far easier, and more reliable, to make your own borderless form instead.
I want to draw directly on the desktop in C#. From searching a bit, I ended up using a Graphics object from the Desktop HDC (null). Then, I painted normally using this Graphics object.
The problem is that my shapes get lost when any part of the screen is redrawn. I tried a While loop, but it actually ends up drawing as fast as the application can, which is not the update rate of the desktop.
Normally, I would need to put my drawing code in a "OnPaint" event, but such thing does not exist for the desktop.
How would I do it?
Example code: https://stackoverflow.com/questions/1536141/how-to-draw-directly-on-the-windows-desktop-c
I posted two solutions for a similar requirement here
Basically you have two options.
1- Get a graphics object for the desktop and start drawing to it. The problem is if you need to start clearing what you have previously drawn etc.
Point pt = Cursor.Position; // Get the mouse cursor in screen coordinates
using (Graphics g = Graphics.FromHwnd(IntPtr.Zero))
{
g.DrawEllipse(Pens.Black, pt.X - 10, pt.Y - 10, 20, 20);
}
2- The second option that I provide in the link above is to create a transparent top-most window and do all your drawing in that window. This basically provides a transparent overlay for the desktop which you can draw on. One possible downside to this, as I mention in the original answer, is that other windows which are also top-most and are created after your app starts will obscure your top most window. This can be solved if it is a problem though.
For option 2, making the form transparent is as simple as using a transparency key, this allows mouse clicks etc. to fall through to the underlying desktop.
BackColor = Color.LightGreen;
TransparencyKey = Color.LightGreen;
When you draw to HDC(NULL) you draw to the screen, in an unmanaged way. As you've discovered, as soon as windows refreshes that part of the screen, your changes are overwritten.
There are a couple of options, depending upon what you want to achieve:
create a borderless, possibly
non-rectangular window. (Use
SetWindowRgn to make a window
non-rectangular.) You can make this a child of the desktop window.
subclass the desktop window. This is not straightforward, and involves
injecting a DLL into the
Explorer.exe process.
To get an OnPaint for the desktop you would need to subclass the desktop window and use your drawing logic when it receives a WM_PAINT/WM_ERASEBKGND message.
As the thread you linked to says, you can only intercept messages sent to a window of an external process using a hook (SetWindowsHookEx from a DLL).
As mentioned a transparent window is another way to do it, or (depending on the update frequency) copying, drawing and setting a temporary wallpaper (as bginfo does).
This is difficult to do correctly.
It will be far easier, and more reliable, to make your own borderless form instead.