Simulating mouse click in webbrowser control - c#

I'm building a screen scraping utility to automate the population of legacy web forms. After researching Selenium and other automation frameworks, I settled on simply using the C# WebBrower control and so far it's easy to use and working well. One of the webforms, however, utilizes a java-driven navigation pane that I can't control as I would with html elements. Therefore, my plan is to find the screen coordinates of the various navigation elements and simulate mouse clicks.
I found a few relevant posts and was led to try something like this (code below). It doesn't work, though, and I can't figure out why. I've tried passing handles to both the WebBrowser control and its parent form in the ClickOn() call but neither works. I looked at it using Spy++ and it appears that no messages are being triggered. Anyone know what the culprit might be?
Also, I should add that there's no java involved yet. I'm just trying to click on an html button at this point to validate the ClickOn method.
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
browser.DocumentCompleted += new WebBrowserDocumentCompletedEventHandler(onDocLoaded);
}
private void onLoad(object sender, EventArgs e)
{
browser.Navigate(#"http://www.dummysite.com/");
}
void onDocLoaded(object sender, WebBrowserDocumentCompletedEventArgs e )
{
ClickOn(browser.Handle, 899, 463);
}
[DllImport("user32.dll", EntryPoint = "PostMessage", CharSet = CharSet.Auto)]
static extern bool PostMessage1(IntPtr hWnd, uint Msg,
int wParam, int lParam);
private void ClickOn(IntPtr hControl, int x, int y)
{
uint WM_LBUTTONDOWN = 0x0201;
uint WM_LBUTTONUP = 0x0202;
PostMessage1(hControl, WM_LBUTTONDOWN, x, y);
PostMessage1(hControl, WM_LBUTTONUP, x, y);
}
}

try calling ClickOn from javascript after window.load event

Related

Identify web browser's vertical scroll is reached on last position in c#

I have load HTML content in WebBrowser control of Winforms. And I want to append another HTML content when vertical scroll is reached on last position. So I am stuck on it how can I achieve it.
Here is my sample code:
WebBrowser web = new WebBrowser();
web.Document.Write("Some html content");
web.Document.Window.AttachEventHandler("onscroll", OnScrollEventHandler); // Create scroll event for browser control
private void OnScrollEventHandler(object sender, EventArgs e)
{
// Identify vertical scroll reached on last position and append another HTML
}
Thanks in advance.
This will work
class KeyHandle
{
private static Int32 WM_KEYDOWN = 0x100;
private static Int32 WM_KEYUP = 0x101;
[return: MarshalAs(UnmanagedType.Bool)]
[DllImport("user32.dll", SetLastError = true)]
static extern bool PostMessage(IntPtr hWnd, int Msg,
System.Windows.Forms.Keys wParam, int lParam);
public static void SendKey(IntPtr hWnd, System.Windows.Forms.Keys key)
{
PostMessage(hWnd, WM_KEYDOWN, key, 0);
}
}
Call method:
KeyHandle.SendKey(this.webBrowser.Handle, Keys.PageDown);
You can try this approach: Make two-way communication between your Winforms app (C#) and browser-side javascript, then you'll be able to call C# code from javascript, like this: window.external.GetAdditionalContent() when the scrollbar reaches the bottom.
GetAdditionalContent = your COM-visible method.
How to: Implement Two-Way Communication Between DHTML Code and Client Application Code:
https://msdn.microsoft.com/en-us/library/a0746166.aspx
Check if a user has scrolled to the bottom:
Check if a user has scrolled to the bottom

WebBrowser control not responding to InvokeMember("click")

After spending 5 days of my life, I'm about to give up on this, but am consulting the experts once before that.
I have a WebBrowser control that loads a webpage and I programmatically scrape its contents. Clicking a particular menu item in the page brings up File Open dialog when done in IE (or any other browser). But clicking the same button in WebBrowser control using InvokeMember() apparently doesn't do anything no matter what. I've gone through several SO questions such as Setting Browser Features to make sure my control behaves exactly like IE, but that hasn't succeeded.
I went as far as inspecting the actual javascript function that the button is executing behind the scene and calling it manually using HtmlDocument.InvokeScript() but couldn't do that because the underlying function takes an argument of MouseEvent type (the click event actually) and I'm not sure how can I create that object in C#.
Another approach was to set focus to that particular button and then try SendKeys, but that won't work because the WebBrowser control is not visible. It is just an in-memory instance. To be more specific, the WebBrowser
EDIT
On a reader's request, here's the simple code that I'm using to find the element:
var MyButton = WB.Document.GetElementById("processfilelink");
processfilelink is an anchor tag (<a href='#' ... >) and I have confirmed that this element actually exists in the body of the document. The webpage uses jQuery's delegate feature to bind this anchor's click event to the target function. After locating the button, I simply call InvokeMember() like this:
MyButton.InvokeMember("click");
Note: I also see bindings for mousedown, mouseup and focus events in the page code. I expect all these events to automatically fire when one invokes click, but just to be sure I added InvokeMember calls for these events too. Results are no better.
From the comments:
... load this page in full IE browser, use F12 Tools to debug it and
execute button.click() in JavaScript console. Does it work as expected
this way?
So, you've tried that, and the result is:
... now that's interesting. It doesn't work! But clicking on the item
by hand does work flawlessly. What's going on here?
I suspected that as MyButton.InvokeMember("click") doesn't work. Apparently, the page handles this click by other means than via onclick event. Most likely, it uses onmousedown or onmouseup events. Study the page's scripting logic to verify if that's the case, use F12 debugger and put some break points.
Updated, if it turns out the page indeed uses onmousedown/onmouseup, you'd need to make your WebBrowser visible and automate it by posting WM_LBUTTONDOWN:
using System;
using System.Drawing;
using System.Runtime.InteropServices;
using System.Windows.Forms;
namespace WindowsFormsApplication_22979038
{
public partial class MainForm : Form
{
public MainForm()
{
InitializeComponent();
}
private void MainForm_Load(object sender, EventArgs e)
{
this.webBrowser.DocumentText = "<a id='goLink' href='javascript:alert(\"Hello!\"),undefined'>Go</a><script></script>";
this.webBrowser.DocumentCompleted += webBrowser_DocumentCompleted;
}
void webBrowser_DocumentCompleted(object sender, WebBrowserDocumentCompletedEventArgs e)
{
var element = this.webBrowser.Document.GetElementById("goLink");
element.Focus();
var hwnd = GetFocus();
if (!IsChild(this.webBrowser.Handle, hwnd))
throw new ApplicationException("Unexpected focused window.");
var rect = GetElementRect(element);
IntPtr wParam = (IntPtr)MK_LBUTTON;
IntPtr lParam = (IntPtr)(rect.Left | rect.Top << 16);
PostMessage(hwnd, WM_LBUTTONDOWN, wParam, lParam);
PostMessage(hwnd, WM_LBUTTONUP, wParam, lParam);
}
// get the element rect in window client area coordinates
static Rectangle GetElementRect(HtmlElement element)
{
var rect = element.OffsetRectangle;
int left = 0, top = 0;
var parent = element;
while (true)
{
parent = parent.OffsetParent;
if (parent == null)
return new Rectangle(rect.X + left, rect.Y + top, rect.Width, rect.Height);
var parentRect = parent.OffsetRectangle;
left += parentRect.Left;
top += parentRect.Top;
}
}
// interop
const int MK_LBUTTON = 0x0001;
const int WM_LBUTTONDOWN = 0x0201;
const int WM_LBUTTONUP = 0x0202;
[StructLayout(LayoutKind.Sequential)]
public struct POINT
{
public int x;
public int y;
}
[DllImport("User32.dll", CharSet = CharSet.Auto)]
static extern int PostMessage(IntPtr hwnd, int msg, IntPtr wparam, IntPtr lparam);
[DllImport("User32.dll", ExactSpelling = true, CharSet = CharSet.Auto)]
static extern IntPtr GetFocus();
[DllImport("User32.dll", SetLastError = true, ExactSpelling = true, CharSet = CharSet.Auto)]
static extern bool IsChild(IntPtr hWndParent, IntPtr hWnd);
}
}

SendMessage WM_SETTEXT to TextBox doesn't trigger TextChanged Event

I have code that gets the handle to a textbox control and uses the windows API to change the text.
The TextChanged event doesn't fire when the text is updated.
Is there a way to fire the TextBox.TextChanged event using the Windows API?
[Update]
I think the reason the event doesn't fire is because the textbox handle is sent though a DCOM interface.
The program is a National Instruments TestStand shell written in c# and uses the NI TestStand COM object for the core functionality. In the TS sequence file (a sort of TS script language) I created an object reference for the textbox handle and set it using the TS api in the shell form's load event. After that I send the handle to my c# DLL. I use SendMessage to update the textbox and that works good. The problem is that the TextChanged event doesn't fire.
I tried using the TS interface to send the textbox and the TextChanged delegate and I couldn't get it to work. I think there is an AppDomain issue doing that through the TS COM object.
As this program proves, the TextChanged event does fire when the control is sent a WM_SETTEXT message.
using System;
using System.Runtime.InteropServices;
using System.Windows.Forms;
namespace WindowsFormsApplication1
{
public partial class Form1 : Form
{
const uint WM_SETTEXT = 0x000C;
[DllImport("user32.dll", CharSet = CharSet.Auto)]
static extern IntPtr SendMessage(IntPtr hWnd, unit Msg,
IntPtr wParam, string lParam);
public Form1()
{
InitializeComponent();
}
private void textBox1_TextChanged(object sender, EventArgs e)
{
MessageBox.Show(textBox1.Text);
}
private void button1_Click(object sender, EventArgs e)
{
SendMessage(textBox1.Handle, WM_SETTEXT, IntPtr.Zero,
textBox1.Text + ", " + textBox1.Text);
}
}
}
Note that this original version of the answer was overly complex and used a SendMessage like this:
static extern IntPtr SendMessage(IntPtr hWnd, unit Msg,
IntPtr wParam, IntPtr lParam);
and consequently had to perform manual marshalling:
IntPtr text = Marshal.StringToCoTaskMemUni(textBox1.Text + ", "
+ textBox1.Text);
SendMessage(textBox1.Handle, WM_SETTEXT, IntPtr.Zero, text);
Marshal.FreeCoTaskMem(text);
Comments at this question (Automatic casting for string DllImport arguments vs Marshal.StringToCoTaskMemUni) persuaded me to update.
I'm not sure what text you're trying to change but I used a combination of PostMessages and keypresses (for numbers) into the textbox and it triggers the TextChangedEvent.
Look at method 2 of this. It basically sets the mouse to click on the textbox and then send key presses of the text you want to the textbox (letter by letter).

Send emulated events to another window

I send mouse events to another application in the following way. The problem is, this works for some applications but not for others.
Why?
using System.Runtime.InteropServices;
using System.Diagnostics;
namespace WpfApplication1
{
public partial class MainWindow : Window
{
[DllImport("user32.dll")]
private static extern int SendMessage(IntPtr hWnd, uint Msg, IntPtr wParam, IntPtr lParam);
private const int downclick = 0x201;
private const int upclick = 0x202;
IntPtr handle = IntPtr.Zero;
public MainWindow()
{
InitializeComponent();
}
private void button1_Click(object sender, RoutedEventArgs e)
{
foreach (Process p in Process.GetProcessesByName("mspaint"))
{
IntPtr handle = p.MainWindowHandle;
int X = 50;
int Y = 380;
IntPtr lParam = (IntPtr)((Y << 16) | X);
IntPtr wParam = IntPtr.Zero;
SendMessage(handle, downclick, wParam, lParam);
SendMessage(handle, upclick, wParam, lParam);
}
}
}
}
Using Spy++ I see that the application recieves the following data:
<00062> 0004052C S WM_LBUTTONDOWN fwKeys:0000 xPos:50 yPos:380
<00063> 0004052C R WM_LBUTTONDOWN
<00064> 0004052C S WM_LBUTTONUP fwKeys:0000 xPos:50 yPos:380
<00065> 0004052C R WM_LBUTTONUP
I assume that the events themselves are correct. But I don't know why it works for some software but not for others. How can I send mouse messages from one window to another?
The software where I want to send the messages is not always visible.
Is it possible at all?
No it's not possible in any reliable way - as you've found out in your testing. The mouse messages are only one part of the input. Windows keeps an input state and just sending messages will not update that input state. And you're also ignoring mouse move messages, etc.
For example in your WinForms application you can use the MousePosition property to get the current mouse positon. Sending messages can't simulate that.
Also you can't send the mouse message to the main window handle, you would have to find the exact button you want to click on and send the message directly to the correct button.
So maybe it will work if the application is only listening for mouse messages this will work, but if not they it won't.
They supported way to simulate mouse clicks, is the SendInput function. But that won't work with minimized applications. It literally goes through the entire Windows input process and will move the mouse cursor - which means that the application has to be visible on the screen.
Here's some information, it talks about keyboard events, but similar logic applies:
http://blogs.msdn.com/b/oldnewthing/archive/2005/05/30/423202.aspx
http://blogs.msdn.com/b/oldnewthing/archive/2010/12/21/10107494.aspx

Embedding a File Explorer instance in a Windows Forms application form

My (C#, .NET 3.5) application generates files and, in addition to raising events that can be caught and reacted to, I want to display the target folder to the user in a form. The file-list is being shown within the same form as other information.
I'm using an instance of the WebBrowser control (System.Windows.Forms.WebBrowser), then navigating to the folder. This shows some default view of the explorer window, with the file summary panel on the left and the files in the 'Tiles' (large icon and text) view.
For example,
wb.Navigate(#"c:\path\to\folder\");
I'd like to suppress the panel and to view the file list in the Details view. The user can get to this via a right-click, context menu, but I'd like it to come up automatically.
I'd rather not have to build my own TreeView, DataGridView or whatever; the WebBrowser control does all the updating and re-sorting and whatnot 'for free'.
Is there a better way? A different control to use or some additional arguments to pass to the control?
And if I could trap events (for example, files being selected/renamed/double-clicked, etc.) then all the better!
WARNING: Long post with lots of code.
When you navigate the web browser control to a file system folder the web browser control hosts a shell view window that in turn hosts the explorer list view. In fact this is exactly the same thing that the Explorer process does as well as the file dialogs and Internet Explorer. This shell window is not a control so there are no methods that can be called on it or events that can be subscribed to but it can receive windows messages and it can be sub-classed.
It turns out that the part of your question dealing with setting the view to Details automatically is actually quite easy. In your web browser control's Navigated event simply find the handle to the shell view window and send it a WM_COMMAND message with a particular shell constant (SHVIEW_REPORT). This is an undocumented command but it is supported on all Windows platforms up to and including Windows 2008 and almost certainly will be on Windows 7. Some code to add to your web browser's form demonstrates this:
private delegate int EnumChildProc(IntPtr hwnd, IntPtr lParam);
[DllImport("user32.dll", SetLastError = true)]
private static extern IntPtr SendMessage(IntPtr hWnd, int Msg,
IntPtr wParam, IntPtr lParam);
[DllImport("user32.dll", SetLastError = true)]
private static extern int EnumChildWindows(IntPtr hWndParent,
EnumChildProc lpEnumFunc, IntPtr lParam);
[DllImport("user32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
private static extern int GetClassName(IntPtr hWnd, StringBuilder lpClassName,
int nMaxCount);
private const int WM_COMMAND = 0x0111;
private const int SHVIEW_REPORT = 0x702C;
private const string SHELLVIEW_CLASS = "SHELLDLL_DefView";
private IntPtr m_ShellView;
void webBrowser1_Navigated(object sender, WebBrowserNavigatedEventArgs e)
{
m_ShellView = IntPtr.Zero;
EnumChildWindows(webBrowser1.Handle, EnumChildren, IntPtr.Zero);
if (m_ShellView != IntPtr.Zero)
{
SendMessage(m_ShellView, WM_COMMAND, (IntPtr)SHVIEW_REPORT, (IntPtr)0);
}
}
private int EnumChildren(IntPtr hwnd, IntPtr lParam)
{
int retval = 1;
StringBuilder sb = new StringBuilder(SHELLVIEW_CLASS.Length + 1);
int numChars = GetClassName(hwnd, sb, sb.Capacity);
if (numChars == SHELLVIEW_CLASS.Length)
{
if (sb.ToString(0, numChars) == SHELLVIEW_CLASS)
{
m_ShellView = hwnd;
retval = 0;
}
}
return retval;
}
Every time the web browser navigates to a new window (including when a folder is opened from within the explorer view) a new shell view window is created so the message must be re-sent to the new window in every Navigated event.
For the second part of your question you would like to receive events from the explorer list view. This is quite a bit more difficult than the first part. To do this you would need to sub-class the list view window and then monitor the windows messages for ones that interest you (such as WM_LBUTTONDBLCLK). In order to sub-class a window you would need to create your own class derived from the NativeWindow class and assign it the handle of the window that you need to monitor. You can then override its Window procedure and handle the various messages as you wish. Below is an example of creating a double click event - it is relatively simple but to get extensive access to the explorer list view may involve a lot more work than you are willing to do.
Add this to your form:
private ExplorerListView m_Explorer;
void OnExplorerItemExecuted(object sender, ExecuteEventArgs e)
{
string msg = string.Format("Item to be executed: {0}{0}{1}",
Environment.NewLine, e.SelectedItem);
e.Cancel = (MessageBox.Show(msg, "", MessageBoxButtons.OKCancel)
== DialogResult.Cancel);
}
and these two lines to the Navigated event handler (right after the SendMessage):
m_Explorer = new ExplorerListView(m_ShellView);
m_Explorer.ItemExecuted += OnExplorerItemExecuted;
Then add the following classes:
class ExplorerListView : NativeWindow
{
public event EventHandler<ExecuteEventArgs> ItemExecuted;
[DllImport("user32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
private static extern IntPtr SendMessage(IntPtr hWnd, int Msg,
IntPtr wParam, IntPtr lParam);
[DllImport("user32.dll", SetLastError = true, CharSet = CharSet.Unicode)]
private static extern IntPtr FindWindowEx(IntPtr hwndParent,
IntPtr hwndChildAfter, string lpszClass, string lpszWindow);
private const int WM_LBUTTONDBLCLK = 0x0203;
private const int LVM_GETNEXTITEM = 0x100C;
private const int LVM_GETITEMTEXT = 0x1073;
private const int LVNI_SELECTED = 0x0002;
private const string EXPLORER_LISTVIEW_CLASS = "SysListView32";
public ExplorerListView(IntPtr shellViewHandle)
{
base.AssignHandle(FindWindowEx(shellViewHandle, IntPtr.Zero,
EXPLORER_LISTVIEW_CLASS, null));
if (base.Handle == IntPtr.Zero)
{
throw new ArgumentException("Window supplied does not encapsulate an explorer window.");
}
}
protected override void WndProc(ref Message m)
{
switch (m.Msg)
{
case WM_LBUTTONDBLCLK:
if (OnItemExecution() != 0) return;
break;
default:
break;
}
base.WndProc(ref m);
}
private int OnItemExecution()
{
int cancel = 0;
ExecuteEventArgs args = new ExecuteEventArgs(GetSelectedItem());
EventHandler<ExecuteEventArgs> temp = ItemExecuted;
if (temp != null)
{
temp(this, args);
if (args.Cancel) cancel = 1;
}
return cancel;
}
private string GetSelectedItem()
{
string item = null;
IntPtr pStringBuffer = Marshal.AllocHGlobal(2048);
IntPtr pItemBuffer = Marshal.AllocHGlobal(Marshal.SizeOf(typeof(LVITEM)));
int selectedItemIndex = SendMessage(base.Handle, LVM_GETNEXTITEM, (IntPtr)(-1), (IntPtr)LVNI_SELECTED).ToInt32();
if (selectedItemIndex > -1)
{
LVITEM lvi = new LVITEM();
lvi.cchTextMax = 1024;
lvi.pszText = pStringBuffer;
Marshal.StructureToPtr(lvi, pItemBuffer, false);
int numChars = SendMessage(base.Handle, LVM_GETITEMTEXT, (IntPtr)selectedItemIndex, pItemBuffer).ToInt32();
if (numChars > 0)
{
item = Marshal.PtrToStringUni(lvi.pszText, numChars);
}
}
Marshal.FreeHGlobal(pStringBuffer);
Marshal.FreeHGlobal(pItemBuffer);
return item;
}
struct LVITEM
{
public int mask;
public int iItem;
public int iSubItem;
public int state;
public int stateMask;
public IntPtr pszText;
public int cchTextMax;
public int iImage;
public IntPtr lParam;
public int iIndent;
public int iGroupId;
int cColumns; // tile view columns
public IntPtr puColumns;
public IntPtr piColFmt;
public int iGroup;
}
}
public class ExecuteEventArgs : EventArgs
{
public string SelectedItem { get; private set; }
public bool Cancel { get; set; }
internal ExecuteEventArgs(string selectedItem)
{
SelectedItem = selectedItem;
}
}
This should give you an idea of what you would need to do. If you want more than fairly simple events you may want to look for a alternative control, though from what I have seen in the free and low cost areas there are some pretty decent controls but they all have some quirks and will not give a seamless explorer experience.
Remember this code was put together fairly quickly without error handling or comments and ignoring several issues such as multiple selected items, so use it as a guideline and at your own risk.
In order to handle renaming, deleting and make other customization you need to write your own file explorer. WebBrowser control is not suitable for your needs. It's just a wrapper over ActiveX component.
You should check this codeproject article. It contains an implementation of file explorer. There are few more samples of file browser:
one
two
LogicNP Software has two controls (FileView and ShComboBox) that do what your looking for:
http://www.ssware.com/fldrview.htm
You can download a trial from their page, however it's ~130$ for the license.
I have written a library that might be able to help you. You can find it at: http://gong-shell.sourceforge.net/
The control you're looking for is the ShellView. There's tutorials there on how to create a simple Windows Explorer clone in only a few lines too.
Note for .NET 4.0 users: Gong-shell is currently broken for 4.0. The framework introduced changes in Interop and it will build just fine but cause different issues when interfacing with shell32 (notably the shellicon api, leading to an unmanaged null pointer dereference).
Check out this article here, it shows how to do this in .NET and WinForms. Doing it this way gives full-control over what the user sees.
I've used it in one of my applications and it works really well. You can show icon/details/list view and it stops the user moving to other directories (which is often the problem of showing the standard file/directory dialogs.
I use it to show the screen like the one below below http://img7.imageshack.us/img7/7647/screenshotbaf.png:
You may want to look at the ExplorerBrowser object.
See http://blogs.msdn.com/ieinternals/archive/2009/12/30/Windows-7-Web-Browser-Control-will-not-browse-file-system.aspx for more details.
If you are happy being Windows Vista only and wrapping a COM control, IExplorerBrowser might be acceptable for you needs.
This The Code Project article shows its use within an MFC program but at least one other person seems to have got it to work in C# after some effort.
The newer API exposes considerably more programmability than simply intercepting messages, but it is (obviously) useless for older platforms.
If you want to open a different window to display the target folder's content you can use System.Windows.Forms.OpenFileDialog, or SaveFileDialog, or inherit from FileDialog and extend it.
To allow the user to select a folder you can use FolderBrowserDialog, though as a user I don't like that control.
Does this help or you absolutely have to embed a control in your form?
Asaf

Categories