I have been trying to send mouse clicks to a WebBrowser control inside of my form using PostMessage(), and I have run into a rather significant issue. What I am trying to achieve is to simulate mouse clicks on this WebBrowser while my form is minimized. Usually PostMessage() would work just fine doing this, but it seems that it only works while my form has focus. This leads me to believe that there is some check going on to see if the particular website I am loading into my WebBrowser control is in focus before it handles mouse events.
This is how I send the clicks with my program:
private void SendClick(Point location)
{
resetHandle = true;
StringBuilder className = new StringBuilder(100);
while (className.ToString() != "Internet Explorer_Server")
{
handle = GetWindow(handle, 5); // 5 == child
GetClassName(handle, className, className.Capacity);
//MessageBox.Show(className.ToString());
}
IntPtr lParam = (IntPtr)((location.Y << 16) | location.X);
IntPtr wParam = IntPtr.Zero;
const uint downCode = 0x201;
const uint upCode = 0x202;
const uint moveCode = 0x200;
PostMessage(handle, moveCode, wParam, lParam); //move mouse
PostMessage(handle, downCode, wParam, lParam); // mousedown
PostMessage(handle, upCode, wParam, lParam); // mouseup
}
This is what the resetHandle does:
private void timer3_Tick(object sender, EventArgs e)
{
if (resetHandle == true)
{
handle = webBrowser1.Handle;
resetHandle = false;
}
}
I'm not sure if there is a better way of sending mouse events to a background window and I am open to any ideas. What I am really asking though is if it is at all possible to make a window believe it is in focus when it is actually still minimized?
Any help at all would be much appreciated!
Rather than keep the window minimized, keep it normal (restored), but set its X or Y coordinate so that it is positioned off screen.
If you want give the user the illusion of minimizing and restoring it, use HwndSource.AddHook to watch for SC_MINIMIZE. In your HwndSourceHook handler, move the window on or off screen according to the pseudo-minimized state, and set handled to true.
Related
In Windows you have a setting to "Show window contents while dragging". When this is off you are instead resizing with an outline of the window.
My WPF application has many controls on it, so it is extremely slow to resize. Is there a way to make my application resize by showing only the window outline instead of always updating the contents?
I found this question regarding WinForms but unfortunately I'm not able to adapt it to WPF. I can hook onto the HwndSource, but the message numbers may have changed in Windows 10, so the if statement in that answer is never entered... or there may be other things at work.
Also, inside the if it calls the WndProc base after is has changed a system parameter, then resetting the system parameter when it has finished calling it. But calling that method is not an option in WPF as a Window object does not have a way to forward the message.
public void OnViewLoaded() {
HwndSource source = HwndSource.FromHwnd(
new WindowInteropHelper(this).Handle);
source?.AddHook(WndProc);
}
private static IntPtr WndProc(IntPtr hwnd, int msg, IntPtr wParam,
IntPtr lParam, ref bool handled) {
if (msg == WM_SYSCOMMAND && (wParam.ToInt32() & 0xfff0) == SC_SIZE) {
// This if is never entered
int isDragFullWindow;
GetSystemParametersInfo(SPI_GETDRAGFULLWINDOWS, 0, out isDragFullWindow, 0);
if (isDragFullWindow != 0)
SetSystemParametersInfo(SPI_SETDRAGFULLWINDOWS, 0, 0, 0);
// How to call this?
base.WndProc(ref m);
if (isDragFullWindow != 0)
SetSystemParametersInfo(SPI_SETDRAGFULLWINDOWS, 1, 0, 0);
}
}
My problem is not being able to get the a flash child control inside Internet Explorer but I can do it within a webbrowser control. The webbrowser control is buggy with the flash object, so I need to do the work through Internet explorer and automate flash testing.
Below is the code for the webbrowser control which I am trying to replicate in IE 11.
private async Task<bool> clickCoorindate(Point point)
{
IntPtr handle = null;
Process[] processes = Process.GetProcessesByName("iexplore");
foreach (Process p in processes)
{
handle = p.MainWindowHandle;
}
//webBrowser1.Focus();
int x = point.X;
int y = point.Y;
// IntPtr handle = webBrowser1.Handle;
StringBuilder className = new StringBuilder(100);
while (className.ToString() != "MacromediaFlashPlayerActiveX")
{
handle = GetWindow(handle, 5); // Get a handle to the child window
GetClassName(handle, className, className.Capacity);
}
IntPtr lParam = (IntPtr)((y << 16) | x); // The coordinates
IntPtr wParam = IntPtr.Zero; // Additional parameters for the click (e.g. Ctrl)
const uint downCode = 0x201; // Left click down code
const uint upCode = 0x202; // Left click up code
const uint moveCode = 0x200;
SendMessage(handle, downCode, wParam, lParam); // Mouse button down
SendMessage(handle, upCode, wParam, lParam); // Mouse button up
Thread.Sleep(20);
SendMessage(handle, downCode, wParam, lParam); // Mouse button down
SendMessage(handle, upCode, wParam, lParam); // Mouse button up
return true;
}
When I use Spy++, the Flash object window is a child window of the IE window.
When I pass the IE mainwindow handle in the code above, it never finds the MacromediaFlashPlayerActiveX handle.
When testing this code in webbrowser control and passing the webbrowser control handle, it finds the flash object fine.
Any idea how to make this work with IE?
Someone asked a question similar here and found the solution, but unfortunately did not add it.
private static bool EnumWindow(IntPtr handle, IntPtr pointer) doesn't run
The solution comment from link above:
I found the solution :). Basically the iexplorer process had defined many different windows handlers. I had to use the EnumThreadWindows and get all the Windows Handler for that process. After that, just look for the Class Name I was interested in.
After spending 5 days of my life, I'm about to give up on this, but am consulting the experts once before that.
I have a WebBrowser control that loads a webpage and I programmatically scrape its contents. Clicking a particular menu item in the page brings up File Open dialog when done in IE (or any other browser). But clicking the same button in WebBrowser control using InvokeMember() apparently doesn't do anything no matter what. I've gone through several SO questions such as Setting Browser Features to make sure my control behaves exactly like IE, but that hasn't succeeded.
I went as far as inspecting the actual javascript function that the button is executing behind the scene and calling it manually using HtmlDocument.InvokeScript() but couldn't do that because the underlying function takes an argument of MouseEvent type (the click event actually) and I'm not sure how can I create that object in C#.
Another approach was to set focus to that particular button and then try SendKeys, but that won't work because the WebBrowser control is not visible. It is just an in-memory instance. To be more specific, the WebBrowser
EDIT
On a reader's request, here's the simple code that I'm using to find the element:
var MyButton = WB.Document.GetElementById("processfilelink");
processfilelink is an anchor tag (<a href='#' ... >) and I have confirmed that this element actually exists in the body of the document. The webpage uses jQuery's delegate feature to bind this anchor's click event to the target function. After locating the button, I simply call InvokeMember() like this:
MyButton.InvokeMember("click");
Note: I also see bindings for mousedown, mouseup and focus events in the page code. I expect all these events to automatically fire when one invokes click, but just to be sure I added InvokeMember calls for these events too. Results are no better.
From the comments:
... load this page in full IE browser, use F12 Tools to debug it and
execute button.click() in JavaScript console. Does it work as expected
this way?
So, you've tried that, and the result is:
... now that's interesting. It doesn't work! But clicking on the item
by hand does work flawlessly. What's going on here?
I suspected that as MyButton.InvokeMember("click") doesn't work. Apparently, the page handles this click by other means than via onclick event. Most likely, it uses onmousedown or onmouseup events. Study the page's scripting logic to verify if that's the case, use F12 debugger and put some break points.
Updated, if it turns out the page indeed uses onmousedown/onmouseup, you'd need to make your WebBrowser visible and automate it by posting WM_LBUTTONDOWN:
using System;
using System.Drawing;
using System.Runtime.InteropServices;
using System.Windows.Forms;
namespace WindowsFormsApplication_22979038
{
public partial class MainForm : Form
{
public MainForm()
{
InitializeComponent();
}
private void MainForm_Load(object sender, EventArgs e)
{
this.webBrowser.DocumentText = "<a id='goLink' href='javascript:alert(\"Hello!\"),undefined'>Go</a><script></script>";
this.webBrowser.DocumentCompleted += webBrowser_DocumentCompleted;
}
void webBrowser_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
var element = this.webBrowser.Document.GetElementById("goLink");
element.Focus();
var hwnd = GetFocus();
if (!IsChild(this.webBrowser.Handle, hwnd))
throw new ApplicationException("Unexpected focused window.");
var rect = GetElementRect(element);
IntPtr wParam = (IntPtr)MK_LBUTTON;
IntPtr lParam = (IntPtr)(rect.Left | rect.Top << 16);
PostMessage(hwnd, WM_LBUTTONDOWN, wParam, lParam);
PostMessage(hwnd, WM_LBUTTONUP, wParam, lParam);
}
// get the element rect in window client area coordinates
static Rectangle GetElementRect(HtmlElement element)
{
var rect = element.OffsetRectangle;
int left = 0, top = 0;
var parent = element;
while (true)
{
parent = parent.OffsetParent;
if (parent == null)
return new Rectangle(rect.X + left, rect.Y + top, rect.Width, rect.Height);
var parentRect = parent.OffsetRectangle;
left += parentRect.Left;
top += parentRect.Top;
}
}
// interop
const int MK_LBUTTON = 0x0001;
const int WM_LBUTTONDOWN = 0x0201;
const int WM_LBUTTONUP = 0x0202;
[StructLayout(LayoutKind.Sequential)]
public struct POINT
{
public int x;
public int y;
}
[DllImport("User32.dll", CharSet = CharSet.Auto)]
static extern int PostMessage(IntPtr hwnd, int msg, IntPtr wparam, IntPtr lparam);
[DllImport("User32.dll", ExactSpelling = true, CharSet = CharSet.Auto)]
static extern IntPtr GetFocus();
[DllImport("User32.dll", SetLastError = true, ExactSpelling = true, CharSet = CharSet.Auto)]
static extern bool IsChild(IntPtr hWndParent, IntPtr hWnd);
}
}
Intention
Using the following code, I managed to load some applications in my windows form.
Code
What this function does is...
stating a process
embedding the process into a panel of my form
maximizing the embedded process
adding a resize event handler to the panel to update the size of the embedded process on panel resize
adding a closed event handler to the form to terminate the embedded process on form close
Usings
using System;
using System.Diagnostics;
using System.Runtime.InteropServices;
using System.Windows.Forms;
Constants
const int GWL_STYLE = -16;
const long WS_VISIBLE = 0x10000000,
WS_MAXIMIZE = 0x01000000,
WS_BORDER = 0x00800000,
WS_CHILD = 0x40000000;
Function
IntPtr LoadExtern(Control Panel, string Path)
{
try
{
Process Process = Process.Start(Path);
Process.WaitForInputIdle();
IntPtr Handle = Process.MainWindowHandle;
SetParent(Handle, Panel.Handle);
SetWindowLong(Handle, GWL_STYLE, (int)(WS_VISIBLE+(WS_MAXIMIZE|WS_BORDER)));
MoveWindow(Handle, 0, 0, Panel.Width, Panel.Height, true);
Panel.Resize += new EventHandler(
delegate(object sender, EventArgs e)
{
MoveWindow(Handle, 0, 0, Panel.Width, Panel.Height, true);
}
);
this.FormClosed += new FormClosedEventHandler(
delegate(object sender, FormClosedEventArgs e) {
SendMessage(Handle, 83, 0, 0);
Thread.Sleep(1000);
Handle = IntPtr.Zero;
}
);
return Handle;
}
catch (Exception e) { MessageBox.Show(this, e.Message, "Error"); }
return new IntPtr();
}
DLL Imports
[DllImport("user32.dll")]
static extern IntPtr SetParent(IntPtr hWndChild, IntPtr hWndNewParent);
[DllImport("user32.dll")]
static extern int SetWindowLong(IntPtr hWnd, int nIndex, int dwNewLong);
[DllImport("user32.dll")]
static extern bool MoveWindow(IntPtr Handle, int x, int y, int w, int h, bool repaint);
[DllImport("user32.dll")]
static extern IntPtr SendMessage(IntPtr Handle, int Msg, int wParam, int lParam);
Result
This code works nice with some applications, like the windows notepad. Notepad is started and included in the panel of my form. There is no caption and the are no borders, as it should be.
LoadExtern(panel1, "notepad.exe");
After closing the form the embedded process gets terminated like expected.
Problem
Unfortunately my code doesn't work for some other (bigger) applications like firefox or sublimetext.
LoadExtern(panel2, #"C:\Program Files (x86)\Mozilla Firefox\firefox.exe");
What happens is that my form starts and firefox starts, but in its own window. Could you help me to include sublimetext or firefox in my applications?
Part of the solution
Thanks to Sheng Jiang's answers, I got it working for some more applications. What I did is to wait for a main window handle.
Process.WaitForInputIdle();
IntPtr Handle = new IntPtr();
for (int i = 0; Handle == IntPtr.Zero && i < 300; i++)
{
Handle = Process.MainWindowHandle;
Thread.Sleep(10);
}
But I still can't embed applications like the windows explorer.
Your code worked nice by coincidence.
WaitForInputIdle wouldn't necessary wait for the UI thread. For example an input method or a hook created by some other program may create a simple thread that becomes idle while the UI thread is still busy doing initialization.
MainWindowHandle searches for the first visible top level window. It won't return the logical main window when
The main window is not the first visible window created (e.g. a login dialog is created first)
The main window is not created with the visible style (think about a program that has only an icon in the notification area on the system tray)
There is no main window created at all (e.g. some applications open new documents/urls in an existing instance, like browsers and Windows Explorer)
There isn't a main window but multiple top level windows that have equal status. Think about IE6/Outlook/Word.
Even if the main window is created visibly and in fact is the first visible window in the new process, you may still have issues.
From the documentation of SetParent:
An application can use the SetParent function to set the parent window of a pop-up, overlapped, or child window.
It does not say you can reparent a top level window. In fact the top level window offers a lot of services that the program may be relying on, such as
Act as the measuring tool to determine if a full screen request is complete (conflicts with your requirement that new program needs to appear inside your panel)
Getting notified when a new DDE conversation starts, when the active window/program changes, when new hardware arrives, when system setting changes, when the user is logging off, when Windows Explorer is started, when the user pressed the Enter key on a nested dialog, etc. The list of window messages that only sent to top level windows is too long to list here,
Act as the default owner window of modal dialogs if the program choose to (and if you display modal dialog in your program as well, watch out for crashes)
This code works for most applcations. I embedded the file explorer simply using a webbrowser control on my form and set its url to a file location. The internet explorer control magically turns into a file explorer then.
This is my final code, feel free to use this for you own projects.
IntPtr EmbedProcess(Control Panel, string Path)
{
string Name = NameFromPath(Path);
foreach (Process Task in Process.GetProcesses())
{
if (NameFromPath(Task.ProcessName).Contains(Name))
{
try { Task.Kill(); }
catch (Exception e) { }
}
}
try
{
Process Task = Process.Start(Path);
Task.WaitForInputIdle();
IntPtr Handle = new IntPtr();
for (int i = 0; Handle == IntPtr.Zero && i < 10; i++) { Handle = Task.MainWindowHandle; Thread.Sleep(100); }
SetParent(Handle, Panel.Handle);
SetWindowLong(Handle, GWL_STYLE, (int)(WS_VISIBLE + (WS_MAXIMIZE | WS_BORDER)));
MoveWindow(Handle, 0, 0, Panel.Width, Panel.Height, true);
Panel.Resize += new EventHandler(delegate(object sender, EventArgs e) { MoveWindow(Handle, 0, 0, Panel.Width, Panel.Height, true); });
this.FormClosed += new FormClosedEventHandler(delegate(object sender, FormClosedEventArgs e)
{
SendMessage(Handle, 83, 0, 0);
Thread.Sleep(100);
Handle = IntPtr.Zero;
});
return Handle;
}
catch (Exception e) { MessageBox.Show(this, e.Message, "Error"); }
return new IntPtr();
}
I somebody is interested in the hole C# classes for embedding window processes and console processes into your form, check out this github repository.
I send mouse events to another application in the following way. The problem is, this works for some applications but not for others.
Why?
using System.Runtime.InteropServices;
using System.Diagnostics;
namespace WpfApplication1
{
public partial class MainWindow : Window
{
[DllImport("user32.dll")]
private static extern int SendMessage(IntPtr hWnd, uint Msg, IntPtr wParam, IntPtr lParam);
private const int downclick = 0x201;
private const int upclick = 0x202;
IntPtr handle = IntPtr.Zero;
public MainWindow()
{
InitializeComponent();
}
private void button1_Click(object sender, RoutedEventArgs e)
{
foreach (Process p in Process.GetProcessesByName("mspaint"))
{
IntPtr handle = p.MainWindowHandle;
int X = 50;
int Y = 380;
IntPtr lParam = (IntPtr)((Y << 16) | X);
IntPtr wParam = IntPtr.Zero;
SendMessage(handle, downclick, wParam, lParam);
SendMessage(handle, upclick, wParam, lParam);
}
}
}
}
Using Spy++ I see that the application recieves the following data:
<00062> 0004052C S WM_LBUTTONDOWN fwKeys:0000 xPos:50 yPos:380
<00063> 0004052C R WM_LBUTTONDOWN
<00064> 0004052C S WM_LBUTTONUP fwKeys:0000 xPos:50 yPos:380
<00065> 0004052C R WM_LBUTTONUP
I assume that the events themselves are correct. But I don't know why it works for some software but not for others. How can I send mouse messages from one window to another?
The software where I want to send the messages is not always visible.
Is it possible at all?
No it's not possible in any reliable way - as you've found out in your testing. The mouse messages are only one part of the input. Windows keeps an input state and just sending messages will not update that input state. And you're also ignoring mouse move messages, etc.
For example in your WinForms application you can use the MousePosition property to get the current mouse positon. Sending messages can't simulate that.
Also you can't send the mouse message to the main window handle, you would have to find the exact button you want to click on and send the message directly to the correct button.
So maybe it will work if the application is only listening for mouse messages this will work, but if not they it won't.
They supported way to simulate mouse clicks, is the SendInput function. But that won't work with minimized applications. It literally goes through the entire Windows input process and will move the mouse cursor - which means that the application has to be visible on the screen.
Here's some information, it talks about keyboard events, but similar logic applies:
http://blogs.msdn.com/b/oldnewthing/archive/2005/05/30/423202.aspx
http://blogs.msdn.com/b/oldnewthing/archive/2010/12/21/10107494.aspx