Is it possible to get UI text from an external application in C#.
In particular, is there a way to read Unicode text from a label (I assume it's a normal Windows label control) from an external Win32 app that was written by a 3rd party? The text is visible, but not selectable by mouse in the UI.
I assume there is some accessibility API (e.g. meant for screen readers) that allows this.
Edit: Currently looking into using something like the Managed Spy App but would still appreciate any other leads.
If you just care about the standard Win32 label, then WM_GETTEXT will work fine, as outlined in the other answers.
--
There is an accessibility API - UIAutomation - for standard labels, it too uses WM_GETTEXT behind the scenes. One advantage to it, however, is that it can get text from several other types of controls, including most system controls, and often UI using non-system controls - including WPF, text in IE and Firefox, and others.
// compile as:
// csc file.cs /r:UIAutomationClient.dll /r:UIAutomationTypes.dll /r:WindowsBase.dll
using System.Windows.Automation;
using System.Windows.Forms;
using System;
class Test
{
public static void Main()
{
// Get element under pointer. You can also get an AutomationElement from a
// HWND handle, or by navigating the UI tree.
System.Drawing.Point pt = Cursor.Position;
AutomationElement el = AutomationElement.FromPoint(new System.Windows.Point(pt.X, pt.Y));
// Prints its name - often the context, but would be corresponding label text for editable controls. Can also get the type of control, location, and other properties.
Console.WriteLine( el.Current.Name );
}
}
You could do it if that unicode text is actually a window with a caption by sending a WM_GETTEXT message.
[DllImport("user32.dll")]
public static extern int SendMessage (IntPtr hWnd, int msg, int Param, System.Text.StringBuilder text);
System.Text.StringBuilder text = new System.Text.StringBuilder(255) ; // or length from call with GETTEXTLENGTH
int RetVal = Win32.SendMessage( hWnd , WM_GETTEXT, text.Capacity, text);
If it is just painted on the canvas you might have some luck if you know what framework the application uses. If it uses WinForms or Borland's VCL you could use that knowledge to get to the text.
didn't see the values for wm_gettext or wm_gettextlength in that article, so just in case..
const int WM_GETTEXT = 0x0D;
const int WM_GETTEXTLENGTH = 0x0E;
Related
I have an Windows Forms application written in C# .NET 5 with a webview2 control which hosts a kiosk based application which will be run full screen on a touchscreen.
The application will use the on screen keyboard for user input. The application is hosted on our own machines, which will only be used for our application so changing the system input language is exactly what we want to achieve.
I need to allow the users to pick their own input language. In 4.7.2 I can achieve this by setting InputLanguage.CurrentInputLanguage, which dynamically updates the on screen keyboard input language, you also see the system input language update in the task bar (which in production will not be visible to users).
In .NET 5, setting the same value does not have the desired effect, the on screen keyboard does not reflect the changed input language, not does the input language update in the taskbar.
I notice the libraries behind the scenes have changed which is clearly why I am having this issue.
I have tried the following without success in a simple dummy app:
private void btnEnglish_Click(object sender, EventArgs e)
{
var language = InputLanguage.FromCulture(System.Globalization.CultureInfo.GetCultureInfo("en-GB"));
if (InputLanguage.InstalledInputLanguages.IndexOf(language) >= 0)
InputLanguage.CurrentInputLanguage = language;
ChangeSystemInputLanguage(language);
}
private static void ChangeSystemInputLanguage(InputLanguage language)
{
Application.OleRequired();
IntPtr handleOld = ActivateKeyboardLayout(new HandleRef(language, language.Handle), 0);
if (handleOld == IntPtr.Zero)
{
throw new ArgumentException("ErrorBadInputLanguage", nameof(language));
}
}
[DllImport("user32.dll", CharSet = CharSet.Auto)]
public static extern IntPtr ActivateKeyboardLayout(HandleRef hkl, int uFlags);
Short of downgrading to 4.7.2 or re-writting in WPF (and such using the InputLanguageManager), can anyone make any recommendations how I might achieve the desired effect in .NET 5.0?
For anyone having this issue, the answer was incredibly simple...
before updating the CurrentInputLanguage, set focus on a control on in your form!
TL;DR: Trying to solve the issue that InputLanguage changes the input layout but does not update the Language Bar display.
I am writing a custom plugin for Trados Studio. Part of this is interfacing with languages both in the app itself and in Word, as in this question: List of all available languages for Windows .NET framework
The last problem I seem to cannot solve is that in a part of the code I am using InputLanguage to set the keyboard input to en-US.
To clarify, there is a limited API, so I have to be really inventive in automating certain aspects. The best workable approach was to use the default shortcuts in the application:
First I change the input language to en-US.
Then I send some keys to the application.
Then I change the input language back to what it was before.
Then I show a form.
Here is the code:
//change input language to English
InputLanguage currentLang = InputLanguage.CurrentInputLanguage;
InputLanguage newLang = InputLanguage.FromCulture(System.Globalization.CultureInfo.GetCultureInfo("en-US"));
if (newLang == null)
{
MessageBox.Show("The Upload Project function requires the En-US keyboard installed.", "Missing keyboard", MessageBoxButtons.OK, MessageBoxIcon.Exclamation);
return;
}
else
{
InputLanguage.CurrentInputLanguage = newLang;
}
//Save the document
SendKeys.SendWait("^s");
InputLanguage.CurrentInputLanguage = currentLang;
//get url and credentials from a custom input form
string[] psw = UploadData.GetPassword(
Settings.GetValue("Upload", "Uri", "https://www.scntranslations.org/ws/services"),
Vars.wsUsername == null ? Settings.GetValue("Upload", "User", "") : Vars.wsUsername,
Vars.wsPassword == null ? "" : Vars.wsPassword
);
Application.DoEvents();
The manifestation that I have is that the Language Bar changes with a delay to EN but by the time the form shows up it should be HU yet it stays EN.
However if I test it with Debug.WriteLine(InputLanguage.CurrentInputLanguage.LayoutName) then the output is the correct language (in my case "Hungarian").
Even after the form is hidden, the language remains EN, yet the keyboard types in Hungarian and the Debug.WriteLine(InputLanguage.CurrentInputLanguage.LayoutName) returns "Hungarian".
I looked at dozens of pages on SO and the web and I tried everything I could think of including System.Threading.Thread.Sleep(1000); and Application.DoEvents() and Sendkeys.Flush(), but nothing triggers Windows to update the Language Bar, and I could not find any solution to this issue.
The earlier version of this problem as given in this question: Change keyboard layout from C# code with .NET 4.5.2
I have made this work now fully with the above implementation of the InputLanguage bar this last hick-up.
Can anybody help in:
giving an explanation for this behavior?
suggesting a solution that triggers the update of the Language Bar from C# code?
UPDATE: The code now fully works. I implemented the BlockInput WinAPI.
[System.Runtime.InteropServices.DllImportAttribute("user32.dll", EntryPoint = "BlockInput")]
[return: System.Runtime.InteropServices.MarshalAsAttribute(System.Runtime.InteropServices.UnmanagedType.Bool)]
public static extern bool BlockInput([System.Runtime.InteropServices.MarshalAsAttribute(System.Runtime.InteropServices.UnmanagedType.Bool)] bool fBlockIt);
And also turning off the Caps Lock using Win API:
[DllImport("user32.dll")]
public static extern void keybd_event(byte bVk, byte bScan, uint dwFlags, UIntPtr dwExtraInfo);
private void TurnOffCapsLock()
{
if (Control.IsKeyLocked(Keys.CapsLock)) // Checks Capslock is on
{
const int KEYEVENTF_EXTENDEDKEY = 0x1;
const int KEYEVENTF_KEYUP = 0x2;
NativeMethods.keybd_event(0x14, 0x45, KEYEVENTF_EXTENDEDKEY, (UIntPtr)0);
NativeMethods.keybd_event(0x14, 0x45, KEYEVENTF_EXTENDEDKEY | KEYEVENTF_KEYUP,
(UIntPtr)0);
}
}
So what's left really is getting the Language Bar to correctly show the actual current input language.
I do have the option of sending Alt+Shifts, but I really don't want to do that.
I'm running a small tool (on Windows 7, 32 Bit) that I would like to see what document is open in another application I've tried this, which works for NotePad on Windows.
var myProcess = Process.GetProcessesByName("NotePad");
string title = myProcess[0].MainWindowTitle;
MessageBox.Show(title);
the output is:
"New Text Document - Notepad"
Now, if I try another application it doesn't always give me the correct title but I noticed that most microsoft applications seem to be fine - NotePad, WordPad, EXCEL etc. It's this other software that is an issue. It has a long title but just returns a very simple name.
Here's what I get from my application which has processName = "FooBar"
The actual running window has this up the top:
"FooBar Software Verson 1.2 - [Results]"
and my code gives:
"FooBar"
any ideas?
[EDIT} 2012-11-19
The very crux of this issue is that I was trying to get the name of the open file from the window. It now seems that the software I'm using doesn't display it there. What I've discovered is that a program called "AutoIT3 Window Spy" can get the text I need as the text of the open file is on the window and not only in the title. I downloaded the source (it's part of http://www.autohotkey.com/ which is open source. It seems to rely on many of the suggestions already made but I'm not able to figure it out just yet.) The source code that I"m looking at is c++ and is located here https://github.com/AutoHotkey/AutoHotkey
So I think the solution to my problem may lay elsewhere. This one may go unanswered.
The main window title is what you see when you go in to task manager and look at the Description column, not the window title itself.
It's the title of the process, not the title of a particular window in the process. A given process may have any number of windows open at one time.
If you need the actual window title, you have to hook user32 something like this:
using System;
using System.Runtime.InteropServices;
using System.Text;
using System.Security;
namespace Application
{
public class Program
{
public static void Main ( )
{
IntPtr hwnd = UnsafeNativeMethods.FindWindow("Notepad", null);
StringBuilder stringBuilder = new StringBuilder(256);
UnsafeNativeMethods.GetWindowText(hwnd, stringBuilder, stringBuilder.Capacity);
Console.WriteLine(stringBuilder.ToString());
}
}
[SuppressUnmanagedCodeSecurity]
internal static class UnsafeNativeMethods
{
[DllImport("user32.dll", SetLastError = true, CharSet = CharSet.Auto)]
internal static extern int GetWindowText ( IntPtr hWnd, [Out] StringBuilder lpString, int nMaxCount );
[DllImport("user32.dll", SetLastError = true)]
internal static extern IntPtr FindWindow ( string lpClassName, string lpWindowName );
}
}
It's possible that the 'title' you're seeing is some kind of owner-drawn UI element which is overriding the visual representation of the title, while the Windows APIs are likely to ignore that sort of thing.
I'd recommend examining the window with a tool like Spy++ to see if that's the case.
It's also possible that the author of the application decided to override the WM_GETTEXT message and is returning that instead of what's actually in the titlebar, although I'm not 100% sure if GetWindowText() is being called here and whether or not it sends the message explicitly or works some other way.
The developer has stated the following:
"I think the failure may be due to a discontinued property Form.Caption in VB 6.0 that was replaced with a Form.Text in. NET"
Thank you all for your valuable suggestions along the way!
I want the user to be able to select the text (just like in Commandprompt) where you right click on the console application's surface and a menu will show, the user can then choose same functions as in commandprompt:
Mark
Copy (Shortcut: Enter)
Paste
Select All
Scroll
Find
I have tried to Google after things like "C# Console Application select text" and other kind of things but can't seem to find a proper solution for this, since the user should be able to mark the text he/she wan't to copy or replace (with paste).
Do you have a solution for my question?
There are no managed methods to do this, but quick edit mode can be enabled through P/Invoke. Quick edit mode allows console text to be selected with the mouse and copied, and for text to be pasted with the right-moue button. (See this article for a description of quick edit mode.)
// using System.Runtime.InteropServices;
[DllImport("kernel32.dll")]
static extern bool SetConsoleMode(IntPtr hConsoleHandle, int mode);
[DllImport("kernel32.dll")]
static extern bool GetConsoleMode(IntPtr hConsoleHandle, out int mode);
[DllImport("kernel32.dll")]
static extern IntPtr GetStdHandle(int handle);
const int STD_INPUT_HANDLE = -10;
const int ENABLE_QUICK_EDIT_MODE = 0x40 | 0x80;
public static void EnableQuickEditMode()
{
int mode;
IntPtr handle = GetStdHandle(STD_INPUT_HANDLE);
GetConsoleMode(handle, out mode);
mode |= ENABLE_QUICK_EDIT_MODE;
SetConsoleMode(handle, mode);
}
maybe I didn't get you but when you execute your console application it will be hosted into a command-prompt window which allows you to copy end past text where ever you like.
You can't do context menu in console apps or in the command prompt.
Console Apps act exactly like the default cmd.exe. You need to go to the menu by clicking the icon on the top left, and the edit menu will give you the options you've listed.
You can also go to properties to turn quick edit on.
If you build the command prompt app then you'll get the select/copy/paste behavior for free. If you want to implement a right click menu (context menu) I don't think you can.
Maybe to simple but you can implement a simple switch based menu:
My application is something like the Spy++ application: i want to be able to automatically retreive all the different controls of the active window (any application) and their children, and for each control i want to know the type, the name, and the value (caption or text).
I am using a C# windows app.
what is the solution to iterate all the controls of the foreground window and their children (and so on) and retrieve name, type and value?
To enumerate top level windows use EnumWindows(), to get their child windows use EnumChildWindows().
Using theHWNDs from the enumeration, a top level window with a title bars value can be read via GetWindowText(), for other windows you can use the WM_GETTEXT message, or depending on exactly what you want, a message specific to the windows class such as LB_GETTEXT for a listbox.
RealGetWindowClass() will give you the windows class.
Window API reference; http://msdn.microsoft.com/en-us/library/ff468919%28v=VS.85%29.aspx
There are a number of Win32 API functions you can use to write your own Spy++ program. This link explains how to write a Spy++ clone in Visual Basic. I know, you probably don't use Visual Basic, but this document does show you how to duplicate Spy++ using the Win32 API. It should not require much effort to translate this to C#.
Yes you will have to use the windows API if its a window thats not part of your current application. This will get you the currently active window:
using System;
using System.Windows.Forms;
using System.Runtime.InteropServices;
using System.Text;
public class MainClass
// Declare external functions.
[DllImport("user32.dll")]
private static extern IntPtr GetForegroundWindow();
[DllImport("user32.dll")]
private static extern int GetWindowText(IntPtr hWnd, StringBuilder text, int count);
public static void Main() {
int chars = 256;
StringBuilder buff = new StringBuilder(chars);
// Obtain the handle of the active window.
IntPtr handle = GetForegroundWindow();
// Update the controls.
if (GetWindowText(handle, buff, chars) > 0)
{
Console.WriteLine(buff.ToString());
Console.WriteLine(handle.ToString());
}
}
}
It uses the GetWindowText() function to find the name of the window, so I assume it shouldn't be a problem to find out other properties of the windows such as its controls etc.