I'm working on an app to automate some input into another application. And i'm running into a problem. Below is the function code i'm using
public class MouseClick
{
[DllImport("user32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall)]
public static extern void mouse_event(int dwFlags, int dx, int dy, int cButtons, int dwExtraInfo);
}
public enum MouseButton
{
MOUSEEVENTF_LEFTDOWN = 0x02,
MOUSEEVENTF_LEFTUP = 0x04,
MOUSEEVENTF_RIGHTDOWN = 0x08,
MOUSEEVENTF_RIGHTUP = 0x10
}
and here is the code i'm using to move and click
Point LocPoint = GetLocation(Column, Row, Item);
Console.WriteLine("Column: {0}\tRow: {1}\tItem: {2}\tPoints: {3}\tCursor: {4}", Column, Row, Item, Points, LocPoint.X + "," + LocPoint.Y);
Thread.Sleep(200);
Cursor.Position = LocPoint;
Thread.Sleep(10);
MouseClick.mouse_event((int)MouseButton.MOUSEEVENTF_LEFTDOWN | (int)MouseButton.MOUSEEVENTF_LEFTUP, 0, 0, 0, 0);
Thread.Sleep(200);
However here is where things get interesting when I don't have the application I want to input into as the active window but say mspaint the code runs fine and I get dots from the paint brush where I want to click however when the application I want to run this in is active the mouse doesn't ever move and no click is registered it is as if the application is intercepting these calls and ignoring them. So this leads me to two questions
Is this possible? How does it detect difference between a mouse and setting the cords
Is there a way around this or another method to use?
Ok so everyone is telling me to use SendInput instead. Ok so I changed the code to use SendInput. I've also tried C# SendKeys as a test as well. Currently I've gone back to the basic and i'm just trying to input the letter A into a text input box that I make the target manually. When I run it in Notepad both SendInput and SendKeys both type the letter A however when i'm inside the other application i'm trying to automate this to nothing shows up. Here is the SendInput code i'm using.
INPUT[] Inputs = new INPUT[2];
Inputs[0].type = WindowsAPI.INPUT_KEYBOARD;
Inputs[0].ki.wVk = 0;
Inputs[0].ki.dwFlags = WindowsAPI.KEYEVENTF_UNICODE;
Inputs[0].ki.wScan = 0x41;
Inputs[0].type = WindowsAPI.INPUT_KEYBOARD;
Inputs[0].ki.wVk = 0;
Inputs[1].ki.dwFlags = WindowsAPI.KEYEVENTF_KEYUP;
Inputs[0].ki.wScan = 0x41;
WindowsAPI.SendInput((uint)Inputs.Length, Inputs, Marshal.SizeOf(Inputs[0]));
So after just frustration I decided to run the app outside of Visual Studio's debug mode which resulted in the same result nothing. However I decided to run the app "As Administrator" even through I have UAC turned all the down and to my surprise the application properly moved the mouse, clicked and inputted text. I do not know why this is required as I've pinvoked methods before and never had to do this however that seems to be the solution.
Related
Let me first explain my current situation and why I think I need this. It may very well be the case that I am handling this completely the wrong way and am thus open to suggestions.
We have a c# program which uses winforms and only has an embededded System.Windows.Forms.WebBrowser which shows a website with an embedded java applet.
Our users have several of these programs opened at a time. The java applet sometimes needs some time to calculate data and the user meanwhile just uses one of his other windows. When the java applet finishes it sets focus to itself and the whole window pops on top which interrupts the other task the user was doing. We don't have access to the java applets source and can't modify it in any way. Calling a java script function which was hooked up through COM to the WebBrowser has the same effect btw.
To counter this we created the event "Deactivate" on the whole form. When called it sets the embedded WebBrowser.Enabled = false. In the corresponding event "Activated" the WebBrowser gets enabled again.
This works really nice: windows don't pop on top anymore just because the java applet wants to set the focus to itself while in the background.
The problem we now have is that when a user clicks on a deactivated window the window gets activated but the mouse click doesn't get forwarded to the WebBrowser. So for example a user has to click twice to press a button.
So I think my question is how to forward the mouse click which activated the window to the WebBrowser.
Thanks in advance
Markus
I think I found a solution. It works but I don't like it:
[DllImport( "user32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall )]
public static extern void mouse_event( uint dwFlags, uint dx, uint dy, uint cButtons, uint dwExtraInfo );
private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
protected override void WndProc( ref Message m )
{
// If the panel which contains the WebBrowser was diabled, the messages WM_LBUTTONDOWN and WM_RBUTTONDOWN will not reach the panel
// but will reach the window instead. We then resend them.
if( m.Msg == 0x0201 ) // WM_LBUTTONDOWN
{
short x, y;
MouseCoordsFromMessage( m, out x, out y );
mouse_event( MOUSEEVENTF_LEFTDOWN, (uint)x, (uint)y, 0, 0 );
}
else if( m.Msg == 0x0204 ) // WM_RBUTTONDOWN
{
short x, y;
MouseCoordsFromMessage( m, out x, out y );
mouse_event( MOUSEEVENTF_RIGHTDOWN, (uint)x, (uint)y, 0, 0 );
}
else
{
base.WndProc( ref m );
}
}
private static void MouseCoordsFromMessage( Message m, out short x, out short y )
{
x = unchecked( (short)(long)m.LParam );
y = unchecked( (short)( (long)m.LParam >> 16 ) );
}
I am not sure how fragile this is and would like to hear other opinions.
I am trying to change the windows cursors (the default is Windows Custom Scheme) to my custom cursors (It named Cut the rope):
Is there any idea to change all of cursors (Arrow, Busy, Help Select, Link select,...) to my Cut the rope?
If you want to change the default Mouse Cursor theme:
You can just change it in the registry:
There are three main registry keys that come into play.
The registry key HKEY_CURRENT_USER\Control Panel\Cursors contains the active user cursors
1a) The values underneath this are the different types of cursors
1b) The Scheme Source specifies the type of cursor scheme that is currently being used.
The different values are:
"0" – Windows Default
"1" – User Scheme
"2" – System Scheme
The registry key HKEY_CURRENT_USER\Control Panel\Cursors contains the user defined cursor schemes (i.e. Scheme Source = 1)
The registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Control Panel\Schemes contains the system cursor schemes (i.e. Scheme Source = 2)
If you already changed the path to one of the cursor type in HKCU\Control Panel\Cursors and realized that it did not do anything. You are correct, just updating a key – HKCU\Control Panel\Cursors\Arrow, for instance – isn’t enough. You have to tell windows to load the new cursor.
This is where the SystemParametersInfo call comes in. To try this out let’s go ahead and change HKCU\Control Panel\Cursors\Arrow to C:\WINDOWS\Cursors\appstar3.ani (assuming you have this icon) and then make a call to SystemParametersInfo.
In AutoHotKey Script:
SPI_SETCURSORS := 0x57
result := DllCall("SystemParametersInfo", "UInt", SPI_SETCURSORS, "UInt", 0, "UInt", 0, "UInt", '0')
MsgBox Error Level: %ErrorLevel% `nLast error: %A_LastError%`nresult: %result%
Translated to C#:
[DllImport("user32.dll", EntryPoint = "SystemParametersInfo")]
public static extern bool SystemParametersInfo(uint uiAction, uint uiParam, uint pvParam, uint fWinIni);
const int SPI_SETCURSORS = 0x0057;
const int SPIF_UPDATEINIFILE = 0x01;
const int SPIF_SENDCHANGE = 0x02;
To call it:
SystemParametersInfo(SPI_SETCURSORS, 0, 0, SPIF_UPDATEINIFILE | SPIF_SENDCHANGE);
Changing to the Default Windows Cursor
Now the tricky part. If you look at HKLM\SOFTWARE\Microsoft\Windows\CurrentVersion\Control Panel\Schemes you will notice that “Windows Default” is defined as “,,,,,,,,,,,,,” or in other words no pointers to actual cursors!
What to do now? Don’t worry. All you have to do is set the different cursor types to empty string and then make the SystemParametersInfo call as usual. In fact, you can set any of the cursor type to empty string in any scheme and Windows will default it to it’s equivalent in the “Windows Default” scheme.
REF:
https://thebitguru.com/articles/programmatically-changing-windows-mouse-cursors/3
https://social.msdn.microsoft.com/Forums/vstudio/en-US/977e2f40-3222-4e13-90ea-4e8d0cdf289c/faq-item-how-to-change-the-systems-cursor-using-visual-cnet?forum=csharpgeneral
You can do like this. Get the Cursor.cur file to load custom cursor. On MouseLeave set the Default cursor for form.
public static Cursor ActuallyLoadCursor(String path)
{
return new Cursor(LoadCursorFromFile(path));
}
[DllImport("user32.dll")]
private static extern IntPtr LoadCursorFromFile(string fileName);
Button btn = new Button();
btn.MouseLeave += Btn_MouseLeave;
btn.Cursor = ActuallyLoadCursor("Cursor.cur");
private static void Btn_MouseLeave(object sender, EventArgs e)
{
this.Cursor = Cursors.Default;
}
There is a simple application that works in Windows. It has very simple interface: squre window with buttons in fixed coordinates.
I need to write a program that makes use of this application: to launch it and to click one of buttons (let's say invoke a click at (150,200)).
Is there any way to do it in Java or .NET?
The Java based solution is to launch the app. in a Process and use the Robot to interact with it.
The best solution on this thread was by #HFoE but deleted by a moderator. For reference, it basically came down to..
If you want to control another Windows application, use a tool that was built specifically for this such as AutoIt V3.
Since "Don't do it" seems to be considered a valid answer when an alternative is supplied (by general opinion on Meta), I cannot understand why the answer was deleted.
As Hovercraft Full Of Eels if you can - use autoit - it's much easier. If AutoIt is not an option then you will need to use winAPI functions in order to do it.
For example to call mouseclick at coordinates:
[DllImport("user32.dll")]
static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")]
static extern bool GetCursorPos(ref Point lpPoint);
[DllImport("user32.dll")]
public static extern void mouse_event(int dwFlags, int dx, int dy, int cButtons, int dwExtraInfo);
public void LeftMouseClick(int xpos, int ypos) //Make a click at specified coords and return mouse back
{
Point retPoint = new Point();
GetCursorPos(ref retPoint); // set retPoint as mouse current coords
SetCursorPos(xpos, ypos); //set mouse cursor position
mouse_event(MOUSEEVENTF_LEFTDOWN, xpos, ypos, 0, 0);
mouse_event(MOUSEEVENTF_LEFTUP, xpos, ypos, 0, 0); //click made
SetCursorPos(retPoint.X, retPoint.Y); //return mouse position to coords
}
But be aware, that to make click inside a window it needs to be at front of you - you cannot click to a minimized app for example.
If you want to try - you can find all needed functions(how to run a programm, get needed window by hwnd and so on) at PInvoke
For .Net you can pretty much use AutomationElement which I prefer. There's a bit of learning time, but it shouldn't take much. You can start your app with ProcessStartInfo.
If you have VS2010 Pro or Ultimate you can use the CodedUITests to generate a couple of button pushes.
As #Hovercraft Full Of Eels suggested - Autoit, Python could do the same
Yes - in C#...
Use the Process class to start the process (there are plenty of resources on the web on how to do this.
Wait until the process has started (either just wait for a fixed amount of time which is probably going to be long enough, or you could try and do something fancy like IPC or monitoring for a window being created)
To simulate the click take a look at How to simulate Mouse Click in C#? which uses a P/Invoke call to the mouse_event function.
However note that there are several things that can go wrong with this
Someone might move the window, or place another window on top of that window in the time it takes to launch the application
On a slower PC it may take longer to load the application (this risk can be mitigated by doing things like monitoring open windows and waiting for the expected application window to appear)
In .net you can Process.Start from System.Diagnostics to launch an application, you can even pass parameters, and to simulate mouse events you can use P/Invoke there is already an answer to that on SO here
Here is my working test app to play with clicking in windows.
We just start some app and hope to click it in right place)
It would be nice to have some solution for capturing windows this way =)
using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Threading;
namespace ConsoleApplication8
{
class Program
{
static void Main(string[] args)
{
var startInfo = new ProcessStartInfo(#"C:\Users\Bodia\Documents\visual studio 2010\Projects\ConsoleApplication8\WindowsFormsApplication1\bin\Debug\WindowsFormsApplication1.exe");
startInfo.WindowStyle = ProcessWindowStyle.Maximized;
Console.WriteLine(1);
var process = Process.Start(startInfo);
Console.WriteLine(2);
Thread.Sleep(400);
Console.WriteLine(3);
LeftMouseClick(1000, 200);
Console.WriteLine(4);
}
static void CursorFun()
{
Point cursorPos = new Point();
GetCursorPos(ref cursorPos);
cursorPos.X += 100;
Thread.Sleep(1000);
SetCursorPos(cursorPos.X, cursorPos.Y);
cursorPos.X += 100;
Thread.Sleep(1000);
SetCursorPos(cursorPos.X, cursorPos.Y);
cursorPos.X += 100;
Thread.Sleep(1000);
SetCursorPos(cursorPos.X, cursorPos.Y);
cursorPos.X += 100;
Thread.Sleep(1000);
SetCursorPos(cursorPos.X, cursorPos.Y);
}
[DllImport("user32.dll")]
static extern bool SetCursorPos(int x, int y);
[DllImport("user32.dll")]
static extern bool GetCursorPos(ref Point lpPoint);
[DllImport("user32.dll")]
public static extern void mouse_event(int dwFlags, int dx, int dy, int cButtons, int dwExtraInfo);
public static void LeftMouseClick(int xpos, int ypos) //Make a click at specified coords and return mouse back
{
Point retPoint = new Point();
GetCursorPos(ref retPoint); // set retPoint as mouse current coords
SetCursorPos(xpos, ypos); //set mouse cursor position
mouse_event(MOUSEEVENTF_LEFTDOWN, xpos, ypos, 0, 0);
mouse_event(MOUSEEVENTF_LEFTUP, xpos, ypos, 0, 0); //click made
SetCursorPos(retPoint.X, retPoint.Y); //return mouse position to coords
}
struct Point
{
public int X;
public int Y;
}
private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_LEFTUP = 0x04;
private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
private const int MOUSEEVENTF_RIGHTUP = 0x10;
}
}
I have a kind of funny and weird requirement this time. I have my account on Facebook and as you all know it is very popular for playing games. One of the applications that i came across was Click game in which a person has to click as many times as he can in span of 10 seconds. Well, one friend said he created some .Net code in C# that would automate the process of clicking on the button. Is it really possible or is he bluffing? If so, can anybody tell me how? I personally haven't seen him doing it. But he mentions this thing in front of my other friends. Any guidelines would be helpful. With much effort i clicked 92 times in 10 seconds and he said using some C# code he just kept a loop and clicked for 1500 times. Now i feel kind of inferior in front of him :p. Just 92 as against his 1500.
Thanks in advance :)
Even this code doesn't work. I can't see even a single click on my page made to facebook :-
[DllImport("user32.dll", CharSet = CharSet.Auto, CallingConvention = CallingConvention.StdCall)]
static extern void mouse_event(uint dwFlags, uint dx, uint dy, uint dwData,int dwExtraInfo);
private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_LEFTUP = 0x04;
private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
private const int MOUSEEVENTF_RIGHTUP = 0x10;
public void DoMouseClick()
{
int X = Cursor.Position.X;
int Y = Cursor.Position.Y;
for (int x = 0; x < 1000; x++)
{
for (int y = 0; y < 600; y++)
{
mouse_event((uint)MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTDOWN, (uint)x, (uint)y, 0, 0);
}
}
}
Probably it doesn't work because mouse click is sent to OS not facebook.
In .net, you can interact with the page scripts in a WebBrowser control with the InvokeScript API and interact with the page DOM via the Document property.
I've seen what you're talking about, and I, too, know people who have ridiculous numbers in that game. Most likely what they are doing is manipulating the JavaScript call that gets passed back to the server and relaying a fake number.
It is possible, however, to simulate a mouse click in .Net. Here's the code to trigger it:
[DllImport("user32.dll",CharSet=CharSet.Auto, CallingConvention=CallingConvention.StdCall)]
public static extern void mouse_event(long dwFlags, long dx, long dy, long cButtons, long dwExtraInfo);
private const int MOUSEEVENTF_LEFTDOWN = 0x02;
private const int MOUSEEVENTF_LEFTUP = 0x04;
private const int MOUSEEVENTF_RIGHTDOWN = 0x08;
private const int MOUSEEVENTF_RIGHTUP = 0x10;
public void DoMouseClick()
{
int X = Cursor.Position.X;
int Y = Cursor.Position.Y;
mouse_event(MOUSEEVENTF_LEFTDOWN | MOUSEEVENTF_LEFTUP, X, Y, 0, 0);
}
As you can see, we're getting the X and Y positions from the mouse's current location; however, you can simulate a click anywhere on the screen so long as you know the coordinates.
If you run this code in a loop, and you get the X and Y coordinates of the button you're trying to press (possibly by delaying the click routine for a few seconds after execution so you have time to move your mouse to where the button is), you can accomplish what you're trying to do.
Note that I don't think this is how people are getting such large numbers in the game. Most likely you can edit the JavaScript calls via FireBug or similar developer tool and then send back fake data to the server.
A Test Framework like Selenium could used for such a challenge
I have created a custom action for my setup project and have successfully implemented a form that displays a progress bar for a download step in my install (I'm using a WebClient in my custom action code). So I have two questions that relate to each other.
Is there any way to show a download progress bar in the main setup window rather than creating a separate form that I display as I have done? I would prefer this.
If not, then what can I do to cause my form to display in front of the actual setup window when I call form.ShowDialog()? I've also called BringToFront() on it which doesn't work either. It's there, but it's always behind the main setup window. Seems there has to be some way to get the correct z-order.
Thanks for your help.
So I gave up on the idea of integrating the progress bar into the actual installer screen, but it's just plain ridiculous what it takes to get the Windows Form to display on top. I have to get a handle to the installer Window and send it to the background because bringing the progress bar window forward simply won't work. I've moved to Mac development now so coming back to this is just frustrating. I remember thinking C# .NET was pretty cool. It's got NOTHING on Cocoa/Objective-C.
It's infuriating having a method called BringToFront() that simply ignores you. Why do I have to drop down to Windows API code to do something as fundamental to a GUI as managing the the Z-Order? Z-Order? Seriously?
In case you're wondering, here's what I ended up doing (via google):
[DllImport("user32.dll", EntryPoint = "SetWindowPos")]
public static extern bool SetWindowPos(
IntPtr hWnd, // window handle
IntPtr hWndInsertAfter, // placement-order handle
int X, // horizontal position
int Y, // vertical position
int cx, // width
int cy, // height
uint uFlags); // window positioning flags
public const uint SWP_NOSIZE = 0x1;
public const uint SWP_NOMOVE = 0x2;
public const uint SWP_SHOWWINDOW = 0x40;
public const uint SWP_NOACTIVATE = 0x10;
[DllImport("user32.dll", EntryPoint = "GetWindow")]
public static extern IntPtr GetWindow(
IntPtr hWnd,
uint wCmd);
public const uint GW_HWNDFIRST = 0;
public const uint GW_HWNDLAST = 1;
public const uint GW_HWNDNEXT = 2;
public const uint GW_HWNDPREV = 3;
public static void ControlSendToBack(IntPtr control)
{
bool s = SetWindowPos(
control,
GetWindow(control, GW_HWNDLAST),
0, 0, 0, 0,
SWP_NOSIZE | SWP_NOMOVE | SWP_NOACTIVATE);
}
I get a handle to the installer window and then call ControlSendToBack() on it. It works, but it sends it to the very back. I tried another method that would just send it back one position, but this wouldn't work either. Windows programming--as good as it was in 1995. Cool.
Another way of doing this is to use a BackgroundWorker. You let the Background Worker handle the downloading of the file so it doesn't prevent the UI being updated.
See this link on donnetperls