I need to send global keystrokes and mouse events to another application, which is coincidentally using using DirectX. (No controls/handles other than the window itself)
For example, I need to hold key X for 2 seconds and then release it...
I need to push Right Click down on coordinates x:600 and y:350, move the mouse 100 pixels down and then release the Right Click.
I also need to push 2 or more keys at once, like X and Y, and stop X after 2 seconds and Y after 2 more seconds.
So basically I would need full control of the input system...
It would also be ideal if I could control the application while maximized or in background. (optionally)
For the skeptics... The teacher made a DirectX application for drawing for our school. I am asked to make an application that draws samples on it, like a train or flower or something... I will be reading images and use the input to set the color and click on the canvas...
There are some possibilities. You may have a look at System.Windows.Forms.SendKeys and you can pInvoke some Win32 functions like SetForegroundWindow(), LockSetForegroundWindow() from gdi32.dll or from user32.dll SetCursorPos() and mouse_event to perform clicks:
Here a snippet for the Mouse events I used a while ago.
/**
* Mouse functions
*/
[DllImport("user32.dll", ExactSpelling=true)]
public static extern long mouse_event(Int32 dwFlags, Int32 dx, Int32 dy, Int32 cButtons, Int32 dwExtraInfo);
[DllImport("user32.dll", ExactSpelling=true)]
public static extern void SetCursorPos(Int32 x, Int32 y);
public const Int32 MOUSEEVENTF_ABSOLUTE = 0x8000;
public const Int32 MOUSEEVENTF_LEFTDOWN = 0x0002;
public const Int32 MOUSEEVENTF_LEFTUP = 0x0004;
public const Int32 MOUSEEVENTF_MIDDLEDOWN = 0x0020;
public const Int32 MOUSEEVENTF_MIDDLEUP = 0x0040;
public const Int32 MOUSEEVENTF_MOVE = 0x0001;
public const Int32 MOUSEEVENTF_RIGHTDOWN = 0x0008;
public const Int32 MOUSEEVENTF_RIGHTUP = 0x0010;
public static void PerformLeftKlick(Int32 x, Int32 y)
{
SetCursorPos(x, y);
mouse_event(MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_LEFTDOWN, 0, 0, 0, 0);
mouse_event(MOUSEEVENTF_ABSOLUTE | MOUSEEVENTF_LEFTUP, 0, 0, 0, 0);
}
Hope that pushes you in the right direction. A good resource is http://pinvoke.net/
If you want to use a library for C# that will make your work easier then read following link -
http://www.codeproject.com/Articles/117657/InputManager-library-Track-user-input-and-simulate
Other than .Net C# you can use other language alternative like in Java where, there is no confusion of direct x or normal input -
http://docs.oracle.com/javase/7/docs/api/java/awt/Robot.html
Related
Some background
One of my current clients runs a chain of Internet points where customers an access the net through PC:s set up as "kiosks" (a custom-built application "locks" the computer until a user has signed in, and the running account is heavily restricted through the Windows group policy). Currently, each computer is running Windows XP and uses Active Desktop to display advertisements in the background. However, since my client has got problems with Active Desktop crashing on a daily basis (in addition to generally slowing down the computer) I have been asked to develop an application that replaces it.
The problem
I am trying to investigate whether it is possible to build a Windows forms application (using C#) that always stays in the background. The application should lie above the desktop (so that it covers any icons, files etc) but always behind all other running applications. I guess I'm really looking for a BottomMost property of the Form class (which doesn't exist, of course).
Any tips or pointers on how to achieve this would be highly appreciated.
This isn't directly supported by the .NET Form class, so you have two options:
1) Use the Win32 API SetWindowPos function.
pinvoke.net shows how to declare this for use in C#:
[DllImport("user32.dll")]
static extern bool SetWindowPos(IntPtr hWnd, IntPtr hWndInsertAfter, int X, int Y, int cx, int cy, uint uFlags);
static readonly IntPtr HWND_BOTTOM = new IntPtr(1);
const UInt32 SWP_NOSIZE = 0x0001;
const UInt32 SWP_NOMOVE = 0x0002;
const UInt32 SWP_NOACTIVATE = 0x0010;
So in your code, call:
SetWindowPos(Handle, HWND_BOTTOM, 0, 0, 0, 0, SWP_NOMOVE | SWP_NOSIZE | SWP_NOACTIVATE);
As you commented, this moves the form to the bottom of the z-order but doesn't keep it there. The only workaround I can see for this is to call SetWindowPos from the Form_Load and Form_Activate events. If your application is maximized and the user is unable to move or minimise the form then you might get away with this approach, but it's still something of a hack. Also the user might see a slight "flicker" if the form gets brought to the front of the z-order before the SetWindowPos call gets made.
2) subclass the form, override the WndProc function and intercept the WM_WINDOWPOSCHANGING Windows message, setting the SWP_NOZORDER flag (taken from this page).
I think the best way to do so is using the activated event handler and SendToBack method, like so:
private void Form1_Activated(object sender, EventArgs e)
{
this.SendToBack();
}
Set your window to be a child window of the desktop (the "Program Manager" or "progman" process). I've succeeded with this method in Windows XP (x86) and Windows Vista (x64).
I stumbled on this method while searching for a way to make a screensaver display as if it were wallpaper. It turns out, this is sort of built in to the system's .scr handler. You use screensaver.scr /p PID, where PID is the process id of another program to attach to. So write a program to find progman's handle, then invoke the .scr with that as the /p argument, and you have screensaver wallpaper!
The project I'm playing with now is desktop status display (shows the time, some tasks, mounted disks, etc), and it's built on Strawberry Perl and plain Win32 APIS (mainly the Win32::GUI and Win32::API modules), so the code is easy to port to or understand any dynamic language with similar Win32 API bindings or access to Windows' Scripting Host (eg, ActivePerl, Python, JScript, VBScript). Here's a relevant portion of the class that produces the window:
do { Win32::API->Import(#$_) or die "Win32::API can't import #$_ ($^E)" } for
[user32 => 'HWND FindWindow(LPCTSTR lpClassName, LPCTSTR lpWindowName)'],
[user32 => 'HWND SetParent(HWND hWndChild, HWND hWndNewParent)'],
sub __screen_x {
Win32::GUI::GetSystemMetrics(SM_CXSCREEN)
}
sub __screen_y {
Win32::GUI::GetSystemMetrics(SM_CYSCREEN)
}
sub _create_window { # create window that covers desktop
my $self = shift;
my $wnd = $$self{_wnd} = Win32::GUI::Window->new(
-width => __screen_x(), -left => 0,
-height => __screen_y(), -top => 0,
) or die "can't create window ($^E)";
$wnd->SetWindowLong(GWL_STYLE,
WS_VISIBLE
| WS_POPUP # popup: no caption or border
);
$wnd->SetWindowLong(GWL_EXSTYLE,
WS_EX_NOACTIVATE # noactivate: doesn't activate when clicked
| WS_EX_NOPARENTNOTIFY # noparentnotify: doesn't notify parent window when created or destroyed
| WS_EX_TOOLWINDOW # toolwindow: hide from taskbar
);
SetParent($$wnd{-handle}, # pin window to desktop (bottommost)
(FindWindow('Progman', 'Program Manager') or die "can't find desktop window ($^E)")
) or die "can't pin to desktop ($^E)";
Win32::GUI::DoEvents; # allow sizing and styling to take effect (otherwise DC bitmaps are the wrong size)
}
This program buffers output to prevent flickering, which you'll probably want to do as well. I create a DC (device context) and PaintDesktop to it (you could use any bitmap with only a couple more lines -- CreateCompatibleBitmap, read in a file, and select the bitmap's handle as a brush), then create a holding buffer to keep a clean copy of that background and a working buffer to assemble the pieces -- on each loop, copy in background, then draw lines and brush bitmaps and use TextOut -- which is then copied to the original DC, at which time it appears on screen.
Yes, function SetWindowPos with flag HWND_BOTTOM should help you. But, from my experience: even after calling SetWindowPos as result of some user operations your window may bring to front.
subclass the form, override the WndProc function and intercept the Windows message(s) that are responsible for moving it up the z-order when it gets activated.
Create a Panel that cover your form, but what ever you want on that Panel, then in the Panel's Click-Event write this.sendback .
I've managed to get rid of the flickering when using setwindowpos...
const UInt32 SWP_NOSIZE = 0x0001;
const UInt32 SWP_NOMOVE = 0x0002;
const UInt32 SWP_NOACTIVATE = 0x0010;
const UInt32 SWP_NOZORDER = 0x0004;
const int WM_ACTIVATEAPP = 0x001C;
const int WM_ACTIVATE = 0x0006;
const int WM_SETFOCUS = 0x0007;
static readonly IntPtr HWND_BOTTOM = new IntPtr(1);
const int WM_WINDOWPOSCHANGING = 0x0046;
[DllImport("user32.dll")]
static extern bool SetWindowPos(IntPtr hWnd, IntPtr hWndInsertAfter, int X,
int Y, int cx, int cy, uint uFlags);
[DllImport("user32.dll")]
static extern IntPtr DeferWindowPos(IntPtr hWinPosInfo, IntPtr hWnd,
IntPtr hWndInsertAfter, int x, int y, int cx, int cy, uint uFlags);
[DllImport("user32.dll")]
static extern IntPtr BeginDeferWindowPos(int nNumWindows);
[DllImport("user32.dll")]
static extern bool EndDeferWindowPos(IntPtr hWinPosInfo);
private void Window_Loaded(object sender, RoutedEventArgs e)
{
IntPtr hWnd = new WindowInteropHelper(this).Handle;
SetWindowPos(hWnd, HWND_BOTTOM, 0, 0, 0, 0, SWP_NOSIZE | SWP_NOMOVE | SWP_NOACTIVATE);
IntPtr windowHandle = (new WindowInteropHelper(this)).Handle;
HwndSource src = HwndSource.FromHwnd(windowHandle);
src.AddHook(new HwndSourceHook(WndProc));
}
private IntPtr WndProc(IntPtr hWnd, int msg, IntPtr wParam, IntPtr lParam, ref bool handled)
{
if (msg == WM_SETFOCUS)
{
IntPtr hWnd = new WindowInteropHelper(this).Handle;
SetWindowPos(hWnd, HWND_BOTTOM, 0, 0, 0, 0, SWP_NOSIZE | SWP_NOMOVE | SWP_NOACTIVATE);
handled = true;
}
return IntPtr.Zero;
}
private void Window_Closing(object sender, System.ComponentModel.CancelEventArgs e)
{
IntPtr windowHandle = (new WindowInteropHelper(this)).Handle;
HwndSource src = HwndSource.FromHwnd(windowHandle);
src.RemoveHook(new HwndSourceHook(this.WndProc));
}
I have an multithreaded application that needs to be able to preform multiple mouse click at the same time.
I have an IntPtr intptr to a process on which i need to send a mouse click to.
I have tried to find this information on the web and there are some examples which i have tried. But I have not got any of them to work.
As I understand the correct way to solv my issue is to use the function
SendMessage(IntPtr hWnd, int Msg, IntPtr wParam, IntPtr lParam);
hWnd is the IntPtr to the process.
Msg is the wanted action, which I want a left click, int WM_LBUTTONDBLCLK = 0x0203;
IntPtr wParam is of no intrest to this problem ( as I understand)
And the coordinates to the click is in lParam.
I construct lParam like,
Int32 word = MakeLParam(x, y);
private int MakeLParam(int LoWord, int HiWord)
{
return ((HiWord << 16) | (LoWord & 0xffff));
}
But as you might understand, I cant get this to work.
My first question is, the coordinates are they within the window of this process or are
the absolut screen coordinates?
And my second question, what am I doing wrong?
I was trying to simulate mouse clicks in C# just recently, I wrote this little helper class to do the trick:
public static class SimInput
{
[DllImport("user32.dll")]
static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint dwData, UIntPtr dwExtraInfo);
[Flags]
public enum MouseEventFlags : uint
{
Move = 0x0001,
LeftDown = 0x0002,
LeftUp = 0x0004,
RightDown = 0x0008,
RightUp = 0x0010,
MiddleDown = 0x0020,
MiddleUp = 0x0040,
Absolute = 0x8000
}
public static void MouseEvent(MouseEventFlags e, uint x, uint y)
{
mouse_event((uint)e, x, y, 0, UIntPtr.Zero);
}
public static void LeftClick(Point p)
{
LeftClick((double)p.X, (double)p.Y);
}
public static void LeftClick(double x, double y)
{
var scr = Screen.PrimaryScreen.Bounds;
MouseEvent(MouseEventFlags.LeftDown | MouseEventFlags.LeftUp | MouseEventFlags.Move | MouseEventFlags.Absolute,
(uint)Math.Round(x / scr.Width * 65535),
(uint)Math.Round(y / scr.Height * 65535));
}
public static void LeftClick(int x, int y)
{
LeftClick((double)x, (double)y);
}
}
The coordinates are a fraction of 65535, which is a bit odd, but this class will handle that for you.
I'm not 100% sure I understand what you're trying to accomplish. But if you want to simulate mouse input then I'd recommend using the SendInput API.
You can provide an array of inputs to be inserted into the input stream.
See also: PInvoke reference
I don't understand why anyone would want to send multiple mouse clicks simultaneously. If it's to test your GUI, it's the wrong test. No one can physically click something multiple times in the same time space.
But going back to your question, using SendMessage won't help you, because it is basically a blocking call. Even if you tried to use PostMessage, you won't be able to accomplish simultaneous clicks, because the message queue is getting pumped from the UI thread and has messages popped off and handled sequentially.
I used this code to click left button in handle
public static void MouseLeftClick(Point p, int handle = 0)
{
//build coordinates
int coordinates = p.X | (p.Y << 16);
//send left button down
SendMessage(handle, 0x201, 0x1, coordinates);
//send left button up
SendMessage(handle, 0x202, 0x1, coordinates);
}
If you set no handle with calling - then it sends click to Desktop, so coordinates should be for whole screen, if you will set handle, then message will be sent to handle's window and you should set coordinates for window.
How about just using VirtualMouse? I use it in C# and it works great.
public partial class Form1 : Form
{
private VirtualMouse vm = new VirtualMouse();
public Form1()
{
InitializeComponent();
}
private void MouseClickHere(Point myPoint)
{
vm.ClickIt(myPoint, 150);
}
private void Clicker()
{
MouseClickHere(new Point(250,350));
}
}
I have a kind of funny and weird requirement this time. I have my account on Facebook and as you all know it is very popular for playing games. One of the applications that i came across was Click game in which a person has to click as many times as he can in span of 10 seconds. Well, one friend said he created some .Net code in C# that would automate the process of clicking on the button. Is it really possible or is he bluffing? If so, can anybody tell me how? I personally haven't seen him doing it. But he mentions this thing in front of my other friends. Any guidelines would be helpful. With much effort i clicked 92 times in 10 seconds and he said using some C# code he just kept a loop and clicked for 1500 times. Now i feel kind of inferior in front of him :p. Just 92 as against his 1500.
Thanks in advance :)
Even this code doesn't work. I can't see even a single click on my page made to facebook :-
[DllImport("user32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall)]
static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint dwData,int dwExtraInfo);
private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_LEFTUP = 0x04;
private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
private const int MOUSEEVENTF_RIGHTUP = 0x10;
public void DoMouseClick()
{
int X = Cursor.Position.X;
int Y = Cursor.Position.Y;
for (int x = 0; x < 1000; x++)
{
for (int y = 0; y < 600; y++)
{
mouse_event((uint)MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTDOWN, (uint)x, (uint)y, 0, 0);
}
}
}
Probably it doesn't work because mouse click is sent to OS not facebook.
In .net, you can interact with the page scripts in a WebBrowser control with the InvokeScript API and interact with the page DOM via the Document property.
I've seen what you're talking about, and I, too, know people who have ridiculous numbers in that game. Most likely what they are doing is manipulating the JavaScript call that gets passed back to the server and relaying a fake number.
It is possible, however, to simulate a mouse click in .Net. Here's the code to trigger it:
[DllImport("user32.dll",CharSet=CharSet.Auto, CallingConvention=CallingConvention.StdCall)]
public static extern void mouse_event(long dwFlags, long dx, long dy, long cButtons, long dwExtraInfo);
private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_LEFTUP = 0x04;
private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
private const int MOUSEEVENTF_RIGHTUP = 0x10;
public void DoMouseClick()
{
int X = Cursor.Position.X;
int Y = Cursor.Position.Y;
mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, X, Y, 0, 0);
}
As you can see, we're getting the X and Y positions from the mouse's current location; however, you can simulate a click anywhere on the screen so long as you know the coordinates.
If you run this code in a loop, and you get the X and Y coordinates of the button you're trying to press (possibly by delaying the click routine for a few seconds after execution so you have time to move your mouse to where the button is), you can accomplish what you're trying to do.
Note that I don't think this is how people are getting such large numbers in the game. Most likely you can edit the JavaScript calls via FireBug or similar developer tool and then send back fake data to the server.
A Test Framework like Selenium could used for such a challenge
How could I programmatically trigger a left-click event on the mouse?
Thanks.
edit: the event is not triggered directly on a button. I'm aiming for the Windows platform.
To perform a mouse click:
[DllImport("user32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall)]
public static extern void mouse_event(long dwFlags, long dx, long dy, long cButtons, long dwExtraInfo);
private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_LEFTUP = 0x04;
private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
private const int MOUSEEVENTF_RIGHTUP = 0x10;
public static void DoMouseClick()
{
mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, 0, 0, 0, 0);
}
To move the cursor where you want:
[DllImport("user32.dll")]
static extern bool SetCursorPos(int X, int Y);
public static void MoveCursorToPoint(int x, int y)
{
SetCursorPos(x, y);
}
If it's right on a button, you can use
button1.PerformClick();
Otherwise, you can check out this MSDN article which discusses simulating mouse (and keyboard) input.
Additionally, this project may be able to help you out as well. Under the covers, it uses SendInput.
https://web.archive.org/web/20140214230712/http://www.pinvoke.net/default.aspx/user32.sendinput
Use the Win32 API to send input.
Update:
Since I no longer work with Win32 API, I will not update this answer to be correct when the platform changes or websites become unavailable. Since this answer doesn't even conform to Stackoverflow standards (does not contain the answer itself, but rather a link to an external, now defunct resource), there's no point giving it any points or spending any more time on it.
Instead, take a look at this question on Stackoverflow, which I think is a duplicate:
How to simulate Mouse Click in C#?
I have created a custom action for my setup project and have successfully implemented a form that displays a progress bar for a download step in my install (I'm using a WebClient in my custom action code). So I have two questions that relate to each other.
Is there any way to show a download progress bar in the main setup window rather than creating a separate form that I display as I have done? I would prefer this.
If not, then what can I do to cause my form to display in front of the actual setup window when I call form.ShowDialog()? I've also called BringToFront() on it which doesn't work either. It's there, but it's always behind the main setup window. Seems there has to be some way to get the correct z-order.
Thanks for your help.
So I gave up on the idea of integrating the progress bar into the actual installer screen, but it's just plain ridiculous what it takes to get the Windows Form to display on top. I have to get a handle to the installer Window and send it to the background because bringing the progress bar window forward simply won't work. I've moved to Mac development now so coming back to this is just frustrating. I remember thinking C# .NET was pretty cool. It's got NOTHING on Cocoa/Objective-C.
It's infuriating having a method called BringToFront() that simply ignores you. Why do I have to drop down to Windows API code to do something as fundamental to a GUI as managing the the Z-Order? Z-Order? Seriously?
In case you're wondering, here's what I ended up doing (via google):
[DllImport("user32.dll", EntryPoint = "SetWindowPos")]
public static extern bool SetWindowPos(
IntPtr hWnd, // window handle
IntPtr hWndInsertAfter, // placement-order handle
int X, // horizontal position
int Y, // vertical position
int cx, // width
int cy, // height
uint uFlags); // window positioning flags
public const uint SWP_NOSIZE = 0x1;
public const uint SWP_NOMOVE = 0x2;
public const uint SWP_SHOWWINDOW = 0x40;
public const uint SWP_NOACTIVATE = 0x10;
[DllImport("user32.dll", EntryPoint = "GetWindow")]
public static extern IntPtr GetWindow(
IntPtr hWnd,
uint wCmd);
public const uint GW_HWNDFIRST = 0;
public const uint GW_HWNDLAST = 1;
public const uint GW_HWNDNEXT = 2;
public const uint GW_HWNDPREV = 3;
public static void ControlSendToBack(IntPtr control)
{
bool s = SetWindowPos(
control,
GetWindow(control, GW_HWNDLAST),
0, 0, 0, 0,
SWP_NOSIZE | SWP_NOMOVE | SWP_NOACTIVATE);
}
I get a handle to the installer window and then call ControlSendToBack() on it. It works, but it sends it to the very back. I tried another method that would just send it back one position, but this wouldn't work either. Windows programming--as good as it was in 1995. Cool.
Another way of doing this is to use a BackgroundWorker. You let the Background Worker handle the downloading of the file so it doesn't prevent the UI being updated.
See this link on donnetperls