C# Simulate mouse movement does not affect foreground application - c#

I have two machines: A and B
On A, I get the current (local) mouse position (x & y) and send send that mouse position over my local network to machine B.
Machine B takes the incoming position X and Y and simulate the mouse movement using the example found here. It all works fine and dandy - I can see the mouse moving, but for some reason, it does not affect the Window in the foreground on machine B.
What is this "Window"? It is a Unity3D application - a game. I expect that the mouse movement would cause the in-game camera to move around. Interestingly, if I do as described and then stop moving the mouse on machine A... and then move the mouse via the touchpad (or regular mouse) on machine B, it moves the in-game camera, as expected!
What is going on?

It seems that, for whatever reason, the code taken from here
[DllImport("user32.dll")]
static extern bool SetCursorPos(int X, int Y);
does not work...properly! My mouse would move, as expected, on the remote machine but the application that I was bringing to the foreground was not detecting the mouse movements.
However, I did manage to get this to work using mouse_event:
[Flags]
public enum MouseEventFlags
{
LeftDown = 0x00000002,
LeftUp = 0x00000004,
MiddleDown = 0x00000020,
MiddleUp = 0x00000040,
Move = 0x00000001,
Absolute = 0x00008000,
RightDown = 0x00000008,
RightUp = 0x00000010
}
[DllImport("user32.dll")]
private static extern void mouse_event(int dwFlags, int dx, int dy, int dwData, int dwExtraInfo);
public static void MouseEvent(MouseEventFlags value, Point position)
{
MousePoint position = position;
mouse_event
((int)value,
position.X,
position.Y,
0,
0)
;
}
[StructLayout(LayoutKind.Sequential)]
public struct MousePoint
{
public int X;
public int Y;
public MousePoint(int x, int y)
{
X = x;
Y = y;
}
}
To use it, just call:
mouse_event(MouseEventFlags.Move,new MousePoint{X = YOUR_X, Y = YOUR_Y});
All credit goes to this SO answer.

Related

C# Moving/Clicking Mouse Winforms

i want to make an application that will automatically move the mouse and click it with just a press of a button in the background.. I'm from Sales and Inventory / HTML Shop website programming and this is my first time making an application that involves control. please help me because i want to push my programming skills.
This is what i'm trying to do and my Idea.
*i will put an loop counter for the repetition of the moves
1.get the x/y of current cursor and save it to variable named (coordinate) (Point A)
2.Right click it and move lower right (Point B)
3.wait 2 seconds
4.Move back to the first position by using variable (coordinate)
5.End loop repeat.
that's my idea and my algorithm my problem is i don't have any idea how to move a mouse and make it stop.
In Window Form projects to move a cursor to a specific point on your screen, you can use this static method.
System.Windows.Forms.Cursor.Position = new Point (X,Y);
and to perform a click event you can use this method.
[DllImport("user32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall)]
public static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint cButtons, uint dwExtraInfo);
public void DoMouseClick()
{
//Call the imported function with the cursor's current position
uint X = (uint)System.Windows.Forms.Cursor.Position.X;
uint Y = (uint)System.Windows.Forms.Cursor.Position.Y;
mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, X, Y, 0, 0);
}
You can move mouse writing some function or code like this:
private void MoveCursor()
{
// Set the Current cursor, move the cursor's Position,
// and set its clipping rectangle to the form.
this.Cursor = new Cursor(Cursor.Current.Handle);
Cursor.Position = new Point(Cursor.Position.X - 50, Cursor.Position.Y - 50);
Cursor.Clip = new Rectangle(this.Location, this.Size);
}
How To Move mouse in C#
And to find location of any control on your Form you can use the following code
Point locationOnForm = control.FindForm().PointToClient(
control.Parent.PointToScreen(control.Location));
How to get controls location in Win Forms

Move mouse to specified point

I'd like to make a c# app that moves the mouse to a specified x,y position on the screen. I've tried a few codes I found online but none seem to work.
Try this:
System.Windows.Forms.Cursor.Position = new Point { X = xxx, Y = yyy };
Or try to use the native WinAPI for XP (or earlier):
[DllImport("user32.dll")]
public static extern long SetCursorPos(int x, int y);
public void SetCursorPosition(Point p)
{
SetCursorPos(p.X, p.Y);
}

Obtain mouse click position relative to active window

Having worked out how to obtain the mouse click position anywhere along the monitor boundaries using low level hooks I receive an X Y coordinate that will contain a value typically between x: -1680 to +1920 and y: 0 to 1200 in my pcs case. Easy enough!
Now the problem is that I now want to calculate the mouse position relative to a given window that I have so I use GetForegroundWindow() and GetWindowRect(HandleRef hWnd, out RECT lpRect) to obtain my active window coordinates.
Where I am stuck is I require the current active desktop (By active I mean which monitor the click occurred on) to calculate the coordinates of my mouse click relative to a window.
Unfortunately I have not been able to find an API call like GetActiveMonitor() or similar so hopefully someone can point me in the right direction?
[DllImport("user32.dll", SetLastError = true)]
[return: MarshalAs(UnmanagedType.Bool)]
static extern bool GetWindowRect(IntPtr hWnd, ref RECT lpRect);
[StructLayout(LayoutKind.Sequential)]
private struct RECT
{
public int Left;
public int Top;
public int Right;
public int Bottom;
}
Call it as:
RECT rct = new RECT();
GetWindowRect(hWnd, ref rct);
after get your mouse position like this
int mouserelativepositionX = mousePosition.X - rct.Left;
int mouserelativepositionY = mousePosition.Y - rct.Top;
My guess is that you can know where your mouse is by using an if:
if(mousePosition.X > -1680 && mousePosition.X < 0)
//We are in monitor 1;
else
//Monitor 2;

How to programmatically (C# or Java) launch an app in Windows and invoke click in it's window?

There is a simple application that works in Windows. It has very simple interface: squre window with buttons in fixed coordinates.
I need to write a program that makes use of this application: to launch it and to click one of buttons (let's say invoke a click at (150,200)).
Is there any way to do it in Java or .NET?
The Java based solution is to launch the app. in a Process and use the Robot to interact with it.
The best solution on this thread was by #HFoE but deleted by a moderator. For reference, it basically came down to..
If you want to control another Windows application, use a tool that was built specifically for this such as AutoIt V3.
Since "Don't do it" seems to be considered a valid answer when an alternative is supplied (by general opinion on Meta), I cannot understand why the answer was deleted.
As Hovercraft Full Of Eels if you can - use autoit - it's much easier. If AutoIt is not an option then you will need to use winAPI functions in order to do it.
For example to call mouseclick at coordinates:
[DllImport("user32.dll")]
static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")]
static extern bool GetCursorPos(ref Point lpPoint);
[DllImport("user32.dll")]
public static extern void mouse_event(int dwFlags, int dx, int dy, int cButtons, int dwExtraInfo);
public void LeftMouseClick(int xpos, int ypos) //Make a click at specified coords and return mouse back
{
Point retPoint = new Point();
GetCursorPos(ref retPoint); // set retPoint as mouse current coords
SetCursorPos(xpos, ypos); //set mouse cursor position
mouse_event(MOUSEEVENTF_LEFTDOWN, xpos, ypos, 0, 0);
mouse_event(MOUSEEVENTF_LEFTUP, xpos, ypos, 0, 0); //click made
SetCursorPos(retPoint.X, retPoint.Y); //return mouse position to coords
}
But be aware, that to make click inside a window it needs to be at front of you - you cannot click to a minimized app for example.
If you want to try - you can find all needed functions(how to run a programm, get needed window by hwnd and so on) at PInvoke
For .Net you can pretty much use AutomationElement which I prefer. There's a bit of learning time, but it shouldn't take much. You can start your app with ProcessStartInfo.
If you have VS2010 Pro or Ultimate you can use the CodedUITests to generate a couple of button pushes.
As #Hovercraft Full Of Eels suggested - Autoit, Python could do the same
Yes - in C#...
Use the Process class to start the process (there are plenty of resources on the web on how to do this.
Wait until the process has started (either just wait for a fixed amount of time which is probably going to be long enough, or you could try and do something fancy like IPC or monitoring for a window being created)
To simulate the click take a look at How to simulate Mouse Click in C#? which uses a P/Invoke call to the mouse_event function.
However note that there are several things that can go wrong with this
Someone might move the window, or place another window on top of that window in the time it takes to launch the application
On a slower PC it may take longer to load the application (this risk can be mitigated by doing things like monitoring open windows and waiting for the expected application window to appear)
In .net you can Process.Start from System.Diagnostics to launch an application, you can even pass parameters, and to simulate mouse events you can use P/Invoke there is already an answer to that on SO here
Here is my working test app to play with clicking in windows.
We just start some app and hope to click it in right place)
It would be nice to have some solution for capturing windows this way =)
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Threading;
namespace ConsoleApplication8
{
class Program
{
static void Main(string[] args)
{
var startInfo = new ProcessStartInfo(#"C:\Users\Bodia\Documents\visual studio 2010\Projects\ConsoleApplication8\WindowsFormsApplication1\bin\Debug\WindowsFormsApplication1.exe");
startInfo.WindowStyle = ProcessWindowStyle.Maximized;
Console.WriteLine(1);
var process = Process.Start(startInfo);
Console.WriteLine(2);
Thread.Sleep(400);
Console.WriteLine(3);
LeftMouseClick(1000, 200);
Console.WriteLine(4);
}
static void CursorFun()
{
Point cursorPos = new Point();
GetCursorPos(ref cursorPos);
cursorPos.X += 100;
Thread.Sleep(1000);
SetCursorPos(cursorPos.X, cursorPos.Y);
cursorPos.X += 100;
Thread.Sleep(1000);
SetCursorPos(cursorPos.X, cursorPos.Y);
cursorPos.X += 100;
Thread.Sleep(1000);
SetCursorPos(cursorPos.X, cursorPos.Y);
cursorPos.X += 100;
Thread.Sleep(1000);
SetCursorPos(cursorPos.X, cursorPos.Y);
}
[DllImport("user32.dll")]
static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")]
static extern bool GetCursorPos(ref Point lpPoint);
[DllImport("user32.dll")]
public static extern void mouse_event(int dwFlags, int dx, int dy, int cButtons, int dwExtraInfo);
public static void LeftMouseClick(int xpos, int ypos) //Make a click at specified coords and return mouse back
{
Point retPoint = new Point();
GetCursorPos(ref retPoint); // set retPoint as mouse current coords
SetCursorPos(xpos, ypos); //set mouse cursor position
mouse_event(MOUSEEVENTF_LEFTDOWN, xpos, ypos, 0, 0);
mouse_event(MOUSEEVENTF_LEFTUP, xpos, ypos, 0, 0); //click made
SetCursorPos(retPoint.X, retPoint.Y); //return mouse position to coords
}
struct Point
{
public int X;
public int Y;
}
private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_LEFTUP = 0x04;
private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
private const int MOUSEEVENTF_RIGHTUP = 0x10;
}
}

Virtual mouse click c#

I have an multithreaded application that needs to be able to preform multiple mouse click at the same time.
I have an IntPtr intptr to a process on which i need to send a mouse click to.
I have tried to find this information on the web and there are some examples which i have tried. But I have not got any of them to work.
As I understand the correct way to solv my issue is to use the function
SendMessage(IntPtr hWnd, int Msg, IntPtr wParam, IntPtr lParam);
hWnd is the IntPtr to the process.
Msg is the wanted action, which I want a left click, int WM_LBUTTONDBLCLK = 0x0203;
IntPtr wParam is of no intrest to this problem ( as I understand)
And the coordinates to the click is in lParam.
I construct lParam like,
Int32 word = MakeLParam(x, y);
private int MakeLParam(int LoWord, int HiWord)
{
return ((HiWord << 16) | (LoWord & 0xffff));
}
But as you might understand, I cant get this to work.
My first question is, the coordinates are they within the window of this process or are
the absolut screen coordinates?
And my second question, what am I doing wrong?
I was trying to simulate mouse clicks in C# just recently, I wrote this little helper class to do the trick:
public static class SimInput
{
[DllImport("user32.dll")]
static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint dwData, UIntPtr dwExtraInfo);
[Flags]
public enum MouseEventFlags : uint
{
Move = 0x0001,
LeftDown = 0x0002,
LeftUp = 0x0004,
RightDown = 0x0008,
RightUp = 0x0010,
MiddleDown = 0x0020,
MiddleUp = 0x0040,
Absolute = 0x8000
}
public static void MouseEvent(MouseEventFlags e, uint x, uint y)
{
mouse_event((uint)e, x, y, 0, UIntPtr.Zero);
}
public static void LeftClick(Point p)
{
LeftClick((double)p.X, (double)p.Y);
}
public static void LeftClick(double x, double y)
{
var scr = Screen.PrimaryScreen.Bounds;
MouseEvent(MouseEventFlags.LeftDown | MouseEventFlags.LeftUp | MouseEventFlags.Move | MouseEventFlags.Absolute,
(uint)Math.Round(x / scr.Width * 65535),
(uint)Math.Round(y / scr.Height * 65535));
}
public static void LeftClick(int x, int y)
{
LeftClick((double)x, (double)y);
}
}
The coordinates are a fraction of 65535, which is a bit odd, but this class will handle that for you.
I'm not 100% sure I understand what you're trying to accomplish. But if you want to simulate mouse input then I'd recommend using the SendInput API.
You can provide an array of inputs to be inserted into the input stream.
See also: PInvoke reference
I don't understand why anyone would want to send multiple mouse clicks simultaneously. If it's to test your GUI, it's the wrong test. No one can physically click something multiple times in the same time space.
But going back to your question, using SendMessage won't help you, because it is basically a blocking call. Even if you tried to use PostMessage, you won't be able to accomplish simultaneous clicks, because the message queue is getting pumped from the UI thread and has messages popped off and handled sequentially.
I used this code to click left button in handle
public static void MouseLeftClick(Point p, int handle = 0)
{
//build coordinates
int coordinates = p.X | (p.Y << 16);
//send left button down
SendMessage(handle, 0x201, 0x1, coordinates);
//send left button up
SendMessage(handle, 0x202, 0x1, coordinates);
}
If you set no handle with calling - then it sends click to Desktop, so coordinates should be for whole screen, if you will set handle, then message will be sent to handle's window and you should set coordinates for window.
How about just using VirtualMouse? I use it in C# and it works great.
public partial class Form1 : Form
{
private VirtualMouse vm = new VirtualMouse();
public Form1()
{
InitializeComponent();
}
private void MouseClickHere(Point myPoint)
{
vm.ClickIt(myPoint, 150);
}
private void Clicker()
{
MouseClickHere(new Point(250,350));
}
}

Categories