I have a system where I have multiple keyboards and really need to know which keyboard the key stroke is coming from. To explain the set up:
I have a normal PC and USB keyboard
I have an external VGA screen with some hard-keys
The hard keys are mapped as a standard USB keyboard, sending a limited number of key-codes (F1, F2, Return, + and -)
I have a low-level hook (in C# but actually calling upon Win32 functionality) which is able to deal with the input even when my application is not focused.
The problem is that when using the normal keyboard, some of the mapped key-codes at picked up by the application being driven on the external screen. One of the key-presses sent by the external screen and used for confirmation is VK_RETURN. Unless I can identify the "device" and filter upon it, the user could be performing actions and confirming them on a screen their not even looking at.
How do I know which keyboard was responsible for the key-press?
Yes I stand corrected, my bad, learning something new every day.
Here's my attempt at making up for it :) :
Register the devices you want to use for raw input (the two keyboards) with ::RegisterRawInputDevices().
You can get these devices from GetRawInputDeviceList()
After you've registered your devices, you will start getting WM_INPUT messages.
The lParam of the WM_INPUT message contains a RAWKEYBOARD structure that you can use to determine the keyboard where the input came from, plus the virtual keycode and the type of message (WM_KEYDOWN, WM_KEYUP, ...)
So you can set a flag of where the last message came from and then dispatch it to the regular keyboard input handlers.
No way to do this. Windows abstracts this for you. As mentioned, you need to write/modify a device driver.
Related
I'm working with the RPA software UIPath. UIPath simulates keystrokes and mouse clicks. What I want to do is differentiate between when I open notepad and type a keystroke manually as the user and when I run my UIPath robot and have the bot type into notepad.
The reason is I am trying to filter out the difference between a user manually typing and the UIPath bot typing. I need to do this for another piece of software that's running that is monitoring keystrokes and mouse clicks. I want the software to not record keystrokes and mouse clicks from the bot and would like to be able to find the execution path of the UIPath bot so I can set up a filter.
My solutions is I'd like to run some C# code and have the console log the different execution paths between me typing manually in the keyboard and then compare it with UIPath bot typing. I'm unsure of how to do this and all I can do is just record in the console if 'a','b','7' etc... was typed, but not exactly the path that was taken for 'a', 'b', '7', etc to be typed. Any help or other suggestions on how to solve this problem would be really appreciated.
Following are the steps you can code to detect if a keyboard or mouse input is simulated or not :
You need to capture low-level keyboard and mouse events using SetWindowsHookEx() i.e.
WH_KEYBOARD_LL and WH_MOUSE_LL
The hook procedure for WH_KEYBOARD_LL will receive KBDLLHOOKSTRUCT structure and for WH_MOUSE_LL it will be MSLLHOOKSTRUCT structure.
KBDLLHOOKSTRUCT has flags field . If LLKHF_INJECTED is set for this field , it would mean that this keyboard input was injected. Keyboard events produced by driver doesn't have this flag set.
Similarly MSLLHOOKSTRUCT has flags field. If LLKHF_INJECTED is set for this field , it would mean that this mouse input was injected. Mouse events produced by driver doesn't have this flag set.
These flag are set on windows kernel level and it is not possible to change them using winapi. I had read about this technique being used by anti-cheat system for games in the book Practical Video Game Bots. Please note that if you use C# library like globalmousekeyhook, this data might not be available in the MouseEventExtArgs/KeyboardEventArgsEx that is being passed to callback function. Please check any third party library you might be using for availability of this data.
What I am trying to do is have a helper application that a user can use touch input to affect a second application. I have been able to send keystrokes to the second application, but the problem I am having is when I want to hold a button down.
For example on my application, I want to be able to hold down a button which would simulate a ctrl key down. And while this button is touched, I want to be able to interact with the second application. And if the user lets go of the button, then the ctrl key is undressed. I can kind of get this working, except when the user does anything on the second application, the button that was held down is unpressed (because the other application gained focus).
I don't care if I have to go WPF or windows forms, just as long as I can get it working. Windows 8 or 8.1 only is acceptable as well (all clients will be 8.1).
Any help would be appreciated!
Note I added to a comment below.
The second application is one I haven't created, it could be anything really. A scenario would be my application having a ctrl button that you could hold press and hold, for example, and in outlook click a link. Or pressing and holding a shift button in my app, while drawing with a pen in photoshop to draw a straight line. I am able to send key strokes, but just can't handle the "hold" touch command.
Since it's been so long, I'm creating a new answer. I did the research, and I'm pretty sure I know what's going on. But I'm going to mention all the official resources I examined before coming to my conclusion.
Possible packaged solutions
First off, the new Windows Input Simulator might fix all your troubles right out of the box. If you need the Windows API, which I'll be talking about below, check PInvoke.net first to see if they have documentation for the call you're trying to make.
The Windows API way
The best place to start is the User Interaction article on MSDN. There's a bunch of new Winu8 Touch APIs there, but you're probably interest in the legacy Keyboard input article.
Every window for an application must have a Windows Procedure (a.k.a WindowsProc) that's responsible for reacting to messages it cares about (e.g. a button click, a message indicating the Window needs to draw its GUI, or the WM_QUIT event that alerts it to gracefully dispose of the resources held by the Window. This procedure is also responsible for handling messages from input devices, like mouse-clicks and keys on the keyboard.
In your case, you're more interested in making the Window think there's a message from the keyboard when there isn't. That's what the SendInput API call is for; it lets you insert an array of INPUT messages, be they keyboard, mouse, or other input device directly into the queue, bypassing the need for the user to physically act. This easy API call specifically accepts MOUSEINPUT, KEYBDINPUT, or HARDWAREINPUT messages.
For the keyboard, you'll get a message when a key is pressed (WM_KEYDOWN) and when it is released (WM_KEYUP), so to determine hotkeys like CTRL+C, you have to watch for WM_KEYDOWN message for the letter C that were received after a WM_KEYDOWN for the CTRL key but before its WM_KEYUP message.
Managing input device messages
To simulate input devices, use SendInput to pass along the WM_KEYDOWN and/or WM_KEYUP message(s) to the target Window. But don't forget that an application can have more than one window. There are API calls to get the different Windows, but it'll be up to you to write code to find it before you can use SendInput on it.a
To find out what a window believes about an input device, use GetAsyncKeyState. You may not be able to trust it if you've meddled with APIs related to input devices.
There is a BlockInput call on a window which denies all messages except SendInput calls from the thread which blocked it. In most cases, re-enabling input as soon as possible is the right thing. The documentation say that if the blocking thread dies, BlockInput is disabled. A similar but less harsh call is EnableWindow which prevents a window from receiving input focus.
The API for windows includes the ability to register hooks, which let you specify kinds of messages and/or certain windows to be reviewed by a user-specified function.
I would really like to know why you need this to be in two different applications, but here's the best I can think of.
In the applications, you should be able to subscribe to KeyDown, KeyUp, Focus, and Blur (lost focus). I'm not clear on if this is an actual button or if its touch input, but whatever the case may be, assume KeyDown is whatever event fires when the user is "simulating" the ctrl key being pressed, and KeyUp is whatever event fires when the user is ceases to "simulate" the ctrl key being down.
Set up the App1 so when it gains focus, it communicates with the App2 the state: depressed, or not depressed. Every time KeyDown or KeyUp fires, send a message to App2.
When App1's Blur event fires, stop sending messages to App2. Even though App1 will no longer have the button depressed, App2 won't know it and can continue to behave as though the button was depressed until App2 regains focus and can go back to sending messages again.
If it were me, I would have App2 have all the same logic as App1, so the moment App2 gets in Focus, it begins handling the up/down state itself. You may want to have the two applications do some kind of "handshake" when a blur/focus event happens to make sure the state is preserved when switching between. When App2 gets the Blur event, it transfers to App1 the state and they shake hands again, so App1 knows its now responsible for managing the state.
This is basically having the apps cooperate via "tag-team." They keep some state synchronized between each other, "handing off" the responsibility when the blur/focus events fire. Since you cannot know that Blur will fire on one app before Focus fires on the other, you will need to use the same mechanism that communicates the state of this "simulated button" to coordinate the apps so they never interfere with each other.
Something tells me that this doesn't completely solve your problem, but hearing why it doesn't will certainly get everyone closer to thinking out the rest of the way. Let me know the twist ending, eh?
The idea is to do something like what a keystroke catcher does, but with multiple input devices. I want a window to record input from devices even if the focus is on another window. What libraries or methodologies would allow me to accomplish this?
This is typically handled via a low level Hook. There is no C# library which will handle this directly, though there is a Microsoft KB article showing How to set a Windows hook in Visual C# .NET.
Is it possible to write a service program that would catch an event of someone trying to take a screen capture? Not that of pressing a printscreen button or any other key combination, though, but of the event itself, even if it is done by some external tool?
You can't realistically, because anyone could write a simple application that can take a screenshot.
The command being, GetDC(NULL) (msdn GetDC).
Hooking this function also wouldn't work since apps can make use of this for legitimate reasons so you'd have to try and filter out false positives.
Also, you've only prevented one class of problem, what if the user emails the file to someone? What if the user copies the text rather than uses a screen shot? What if the user takes a photo using their mobile phone?
If you have sensitive information that must be protected the only real option is to educate users and/or restrict their access. E.g. machines with sensitive information are not connected to a public network, USB drives are disabled and user's are not allowed to bring anything with a camera or data storage near the machines.
If it's just to avoid silly mistakes, then you could hook the keyboard and look for the print screen key. This however will not detect applications (such as Snipping Tool).
This could be done via a clentscript and using AJAX to send a message back to your servers, however you cannot guarantee the user wont have javascript turned off, or the user gets around this by running through a proxy or vm environment.
In short you could do this but there is no way to 100% guarantee its effectiveness.
Just the functionality of intercepting the keyboard entry , transforming it and sending it back is required.eg if user press a i wold like to send e etc.
To make a application that accepts the entry from keyboard modify it and send it to the active window (may be application like word, excel,notepad windows screens ) etc.
The feature is just like google's transliteration application for windows .
I would like to do it in c# .
I think this involves hooking in the keyboard and sending keys (like sendkey in vb) to the current window.
The working is just like Google transliteration .However the logic for transformation is quite different.
I'm not sure what the question is exactly, so it may get closed for being too vague. However, it sounds like you'll have to buffer all of the keystrokes, send words or phrases to some service asynchronously to be transliterated and then push them back into keyboard buffer with something like sendkeys as you mentioned. The problem is that if your app has focus, we don't know where the user wants the transliterated text to show up, and if the app such as Word has focus, it will receive both the real keystrokes and the transliterated text, which will then be mixed together (a mess). Seems better just to transliterate chunks of text rather than trying to interfere with the keyboard buffer - besides, backspace, undo, etc. will be nearly impossible to honor.
You need to write a dll which uses Windows Text Service framework or legacy IME framework .
This dll will get loaded into langauge bar and when you will activate it ; your code , which has implemented a set of APIS required by TSF or IME framework , will get the key event , You can then pass it to your code which translates it and passes a string back to you.Another of your implemented API can then input to the Current text editor of the windows Application.