My app screencaptures another window that runs on a second monitor. Now I'd also like to forward mouse clicks made in my app to that window. I tried using SendMessage in user32.dll for this, but this also makes window focus switch, which causes some issues, like the two windows rapidly fighting for focus. Is there are way to place those mouse events without making the hidden window active and losing focus on the main app?
Is there are way to place those mouse events without making the hidden window active and losing focus on the main app?
No, there is not even a way to forward mouse input to another receiver. Messages are only part of the input processing. The system also does internal bookkeeping and you cannot replicate that.
The only reliable way to inject input is by calling SendInput. Doing so doesn't allow you to specify a receiver. Input goes to whichever thread is determined to be the receiver by the system.
Although, more often than not, this question is asked when the problem that needs to be solved is a different one altogether: How do you automate a UI? The answer to that question is UI Automation.
Related
What I am trying to do is have a helper application that a user can use touch input to affect a second application. I have been able to send keystrokes to the second application, but the problem I am having is when I want to hold a button down.
For example on my application, I want to be able to hold down a button which would simulate a ctrl key down. And while this button is touched, I want to be able to interact with the second application. And if the user lets go of the button, then the ctrl key is undressed. I can kind of get this working, except when the user does anything on the second application, the button that was held down is unpressed (because the other application gained focus).
I don't care if I have to go WPF or windows forms, just as long as I can get it working. Windows 8 or 8.1 only is acceptable as well (all clients will be 8.1).
Any help would be appreciated!
Note I added to a comment below.
The second application is one I haven't created, it could be anything really. A scenario would be my application having a ctrl button that you could hold press and hold, for example, and in outlook click a link. Or pressing and holding a shift button in my app, while drawing with a pen in photoshop to draw a straight line. I am able to send key strokes, but just can't handle the "hold" touch command.
Since it's been so long, I'm creating a new answer. I did the research, and I'm pretty sure I know what's going on. But I'm going to mention all the official resources I examined before coming to my conclusion.
Possible packaged solutions
First off, the new Windows Input Simulator might fix all your troubles right out of the box. If you need the Windows API, which I'll be talking about below, check PInvoke.net first to see if they have documentation for the call you're trying to make.
The Windows API way
The best place to start is the User Interaction article on MSDN. There's a bunch of new Winu8 Touch APIs there, but you're probably interest in the legacy Keyboard input article.
Every window for an application must have a Windows Procedure (a.k.a WindowsProc) that's responsible for reacting to messages it cares about (e.g. a button click, a message indicating the Window needs to draw its GUI, or the WM_QUIT event that alerts it to gracefully dispose of the resources held by the Window. This procedure is also responsible for handling messages from input devices, like mouse-clicks and keys on the keyboard.
In your case, you're more interested in making the Window think there's a message from the keyboard when there isn't. That's what the SendInput API call is for; it lets you insert an array of INPUT messages, be they keyboard, mouse, or other input device directly into the queue, bypassing the need for the user to physically act. This easy API call specifically accepts MOUSEINPUT, KEYBDINPUT, or HARDWAREINPUT messages.
For the keyboard, you'll get a message when a key is pressed (WM_KEYDOWN) and when it is released (WM_KEYUP), so to determine hotkeys like CTRL+C, you have to watch for WM_KEYDOWN message for the letter C that were received after a WM_KEYDOWN for the CTRL key but before its WM_KEYUP message.
Managing input device messages
To simulate input devices, use SendInput to pass along the WM_KEYDOWN and/or WM_KEYUP message(s) to the target Window. But don't forget that an application can have more than one window. There are API calls to get the different Windows, but it'll be up to you to write code to find it before you can use SendInput on it.a
To find out what a window believes about an input device, use GetAsyncKeyState. You may not be able to trust it if you've meddled with APIs related to input devices.
There is a BlockInput call on a window which denies all messages except SendInput calls from the thread which blocked it. In most cases, re-enabling input as soon as possible is the right thing. The documentation say that if the blocking thread dies, BlockInput is disabled. A similar but less harsh call is EnableWindow which prevents a window from receiving input focus.
The API for windows includes the ability to register hooks, which let you specify kinds of messages and/or certain windows to be reviewed by a user-specified function.
I would really like to know why you need this to be in two different applications, but here's the best I can think of.
In the applications, you should be able to subscribe to KeyDown, KeyUp, Focus, and Blur (lost focus). I'm not clear on if this is an actual button or if its touch input, but whatever the case may be, assume KeyDown is whatever event fires when the user is "simulating" the ctrl key being pressed, and KeyUp is whatever event fires when the user is ceases to "simulate" the ctrl key being down.
Set up the App1 so when it gains focus, it communicates with the App2 the state: depressed, or not depressed. Every time KeyDown or KeyUp fires, send a message to App2.
When App1's Blur event fires, stop sending messages to App2. Even though App1 will no longer have the button depressed, App2 won't know it and can continue to behave as though the button was depressed until App2 regains focus and can go back to sending messages again.
If it were me, I would have App2 have all the same logic as App1, so the moment App2 gets in Focus, it begins handling the up/down state itself. You may want to have the two applications do some kind of "handshake" when a blur/focus event happens to make sure the state is preserved when switching between. When App2 gets the Blur event, it transfers to App1 the state and they shake hands again, so App1 knows its now responsible for managing the state.
This is basically having the apps cooperate via "tag-team." They keep some state synchronized between each other, "handing off" the responsibility when the blur/focus events fire. Since you cannot know that Blur will fire on one app before Focus fires on the other, you will need to use the same mechanism that communicates the state of this "simulated button" to coordinate the apps so they never interfere with each other.
Something tells me that this doesn't completely solve your problem, but hearing why it doesn't will certainly get everyone closer to thinking out the rest of the way. Let me know the twist ending, eh?
I need to be able to issue keyboard input to a particular window in the background on a periodic basis without interrupting the user. How could this be done?
Use case:
I have two windows open: one window is waiting for the input (window A), and I'm actively working in another window (window B). While working in window B, periodic commands are issued to window A through the keyboard without interrupting the interactivity going on in window B.
I'm hoping to accomplish this using Python or C#...whichever gets the job done cleanest and fastest (or a nice hybrid thereof).
Check winGuiAuto.py. If you can find the hwnd for the control whose input you want to change, you can send it to it even if it's not the active window.
Even it doesn't solve your problem directly, the source code is a good lesson on using the win32api with Python and should help you anyway.
According to this article on msdn, it is possible to set a specific window as the foreground window, and then send keystrokes to it... but I need to send keystrokes to a window in the background, so that whatever the user is doing can continue uninterrupted.
Maybe you should just catch the keystrokes in your foreground window and pass them to the background window (delegate). Or if they are somewhat complex you could use global hotkeys. Anyways passing the keys from your foreground form to your background form should be the better solution.
I found a nice example it's not about hotkeys but the concept should apply in your case as well.
http://www.codeproject.com/KB/cs/pass_data_between_forms.aspx
So basically a keystroke listener? Java has a robot class that listens for keystrokes and can act upon them (ex. me typing "Hello World" and the class will automatically write that to a file or other window).
I've created something similar in the past. You can create a java program to run in the background in a constant while loop and just listen for keystrokes. Set the program to write each keystroke to a file or specific place. Because this is a short while loop and only reacts when a key is pressed, it takes up almost no processing power and does not affect the performance of the program in the foreground.
I do this with shortcut keys. I have an application that has multiple child windows along the side. The main window enables the Form.KeyPreview property of the child windows and then registers for the appropriate event (KeyPress, KeyDown or KeyUp) on that child window. With this setup the main window can process all the keys from the child windows.
The technique I've used on Windows Mobile/CE is to hook the keyboard and then simply use PostMessage() to send them to the target window to be handled as normal. This technique should also work on the desktop. There are several examples on Code Project of hooking the keyboard - http://www.codeproject.com/KB/system/globalsystemhook.aspx. Your question was not clear if the foreground window was part of your application or not. If it is, wouldn't you simply subscribe to one of foreground form key events from the background form?
I'm working on a docking project, in which I need my form to dock itself into a window that doesn't belong to my application. I have no problem detecting the location of the window, however I haven't got a clue on how to intercept the event of movement of the window. Does anyone know how can I intercept the event of movement of a window?
Maybe this post will give you some clues.
An alternative way (but not very nice) would be to have a timer that fires every 500 milisecs and checks each time for window position and compare with the previous one. Then if different adjust your window accordingly.
Listening for window messages requires injecting code in the process whose window you want to dock to. You cannot inject C# code, you can't reliably get the CLR initialized in that process. You'll only have a fighting chance if you use native code. Google EasyHook. Black belt Win32 API skills are required to bring this to a good end.
My goal is to make a floating toolbar (as its own C# application), and when the user uses the scrollwheel over me I want to change the buttons that are visible. Sounds easy enough, should just be a matter of this one-liner:
MouseWheel += new MouseEventHandler(Form1_MouseWheel);
The problem I am having is that the mouse wheel handler is only invoked when my application has focus. That means the user has to first click, and then mousewheel. That won't do for what I'm trying to do.
I can hook the MouseHover event handler and call form.Activate() then, to get focus. That's suboptimal because if the user uses the scrollwheel immediately after mousing over my application (instead of waiting a little), the focus will still be on the previous app and it'll get the mousewheel event.
A natural thing to do would be to hook the MouseEnter event and call Activate() there, but then instead of my application coming to the front, its icon starts to blink on the task bar. I'm using Win7, but this problem is probably older than this.
Ideally, what I'd like to do would be to detect the mousewheel events without having to worry about whether my application has focus. It would really be better for the previous application to keep input focus, so for example if the user's in Notepad they can type, mouse over to my app, use the scroll wheel, look at what they see and decide to just resume typing in Notepad. Ideally I don't want them to have to click even once in this scenario.
I'll settle for a solution that switches focus to my application, though, if there's no other way.
What I have so far uses C# and Windows Forms, but I'd be open to using something different if that can solve my problems.
So: how can I see those mousewheel events without the user having to click to focus my application first?
If you need to catch mouse events outside your application, you can use a global system hook. There's a good .NET implementation here