This isn't my area of programming, but I have one of my junior programmers working on a Windows application that needs to reference the UI elements of a Microsoft Office application, namely to get the coordinates of a button or menu item, such as the coordinates of the compose email button in Outlook or the edit menu item label frame in Excel, for example.
Specifically, is there a way of programatically obtaining the exact coordinates of a UI element from an Office application that is running with a visible window? Is there some kind of API Microsoft provides to programmers to allow determining object metrics.
So far, this is all I could find:
https://msdn.microsoft.com/en-
us/library/system.windows.controls.frame(v=vs.110).aspx
Thanks for any help!
You can extract the coordinate in developer mode of where the mouse is. So by hovering over a button you can see where the coordinate is but any change in resolution or screen size will throw this off.
Many UI options have hotkeys. Have you thought about using those? e.g if you wanna code it to click on file, fire a "alt+f" event and it will
Related
I would like to know if it is possible to, while running a WPF window application in Visual Studio, wait for the user to click anywhere on the screen (not necessarily inside the window of my application - for the purpose of my application, the click would most likely occur inside a browser page) and then gather the information about the click (like inside the window of which application the user clicked, or the selector of the html element the user clicked)? I know this question might be very confunsing but this is basically my last resort since researching on the Internet hasn't helped me much. Just to provide a better idea of what I seek, it's like what the 'Extract Structured Data' Activity does in UiPath. Oh and I'm using C# by the way.
You can try and use this external library called GlobalMouseHook.
This library allows you to tap keyboard and mouse, detect and record their activity even when an application is inactive and runs in background.
Here is what you can do with this library:
Mouse coordinates
Mouse buttons clicked
Mouse drag actions
Mouse wheel scrolls
Key presses and releases
Special key states
Hope this helps.
How can I show a message like the "Follow up" and "Assign policy" do while composing a new email? (see the highlighted area in the screenshot)
After searching I could not find anything that to lead me in the right direction. It must be something not directly tied with just Inspectors as that area is available also in the Explorer..
The closest thing I found are Form Regions but I would like to use the builtin area for this task. Thanks.
That is MailTips.
The Outlook object model (nor VSTO) doesn't provide anything for that. You may consider using Outlook Form Regions, but they allow placing the form at the bottom of the inspector window.
Add-in Express provides a layout for placing your forms at the top of the inspector windows:
And the last resort is to subclass Outlook windows using Windows API and create an adjacent window, see Creating Adjacent Windows In Outlook for more information.
I have tried inspecting this window using Inspect.exe provided on Windows' 8.1 Software Development Kit Testing Tools. I couldn't seem to identify the control handle for this huge rich textbox like control. I could however get the control handle of the edit box below the textbox. I was hoping I could talk directly to the control and use a library such as WindowScrape to get the text.
I believe the external program is created in VC++ . If that helps.
This is the window. I am trying to get text from that is boxed in red color. This is a chat window , messages come in every 5 to 10 seconds. This box scrolls automatically. I want to tail through the text and get the latest message.
Is there any other way to accomplish this?
Tesseract maybe, but I do not know where to start to monitor a specific position relative to the window. So that when I move the window it will still be able to monitor that window.
Can you provide a screenshot of Inspect results showing both the tree and details panes while selecting the chat box?
I have a TagVisualizer covering the entire screen. Nested inside is a fullscreen ScatterView. Inside one of my ScatterViewItems is UserControl which lists n amount of SurfaceTextBoxs. The UserControl and ScatterViewItems are always visible.
E.g.
<TagVisualizer Panel.ZIndex="1">
<TagVisualizer.Definitions>
...
<TagVisualizer.Definitions>
<ScatterView Panel.ZIndex="2">
<ScatterViewItem />
<ScatterViewItem />
<ScatterViewItem >
<UserControl />
</ScatterViewItem>
...
</ScatterView>
</TagVisualizer>
The problem I have is when there is no tags being recognized on the table the Touch input works and the Keyboard pops up and functions correctly. However when there is a tag on the table being recognized the keyboard no longer focuses on the textblock and wont react to user touch input. I thought layering the content zindex would solve this but I was wrong...
What is frustrating is that my code works with the Surface Input Simulator tools but not on the device itself. (tag and touch tool options, not mouse).
The tags are precisely cutout and do not appear as blobs / finger
touches on the table.
Dragging and dropping of content from
scatterview to librarybars etc all work as intended.
Other buttons and touch elements are all working as intended
The Source for each tag is IsHitTestVisible=false and has no buttons etc (only a Ellipse which shows me where the tag is if being recognised)
Only the
keyboard functionality is not functioning as I thought it was
intended.
I believe the reason behind this is due to that the Tag captures the Mouse context and won't let go and the default Keyboard functionality needs not the touch input but the single mouse event. How would I go about making the Keyboard work on a SurfaceTextBox - Inside a UserControl - when a Tag is being recognized?
I'm new to WPF and this problem is causing me a minor headache...I have looked at the SDK examples and I cannot find a solution for this.. Is there something stupid I am doing or something simple not doing/have missed?
Any help would be appreciative :)
After researching more about the keyboard structure of Surface 2.0 programs and again following the Shopping Cart example you have to use the Surface 2.0 Keyboard, not the Tablet one (the one you can toggle on/off in the control panel). I made the foolish error in thinking that both the keyboards can operate the same... nope.
The Surface 2.0 keyboard accepts touch input and works with TagVisualizers. This is due binding the keyboard to certain SurfaceTextBoxes and it taking in touch input even when the mouse has not clicked on it.
The Tablet windows default keyboard does not work with touch input unless a mouse input has touched it (when there is no other input the last and current active touch input is promoted to a mouse). This keyboard always shows when no over keyboard is specified and it is enabled in the Control Panel. This is tedious to work with TagVisualizers as when there is no mouse/touch input but there is a recognized tag.. this tag will steal the contact and until you take it off.
This problem does not exist with the Surface 2.0 keyboard. When you develop with the Surface 2.0 keyboard the Tablet keyboard will take it's place (annoyingly) until you run your software via the Surface Shell...
To develop your program and see the Surface 2.0 Keyboard you will have to have a development environment for and on a set up for Surface. E.g. develop on a Samsung SUR40 etc
http://msdn.microsoft.com/en-us/library/ff727875.aspx Testing in Windows mode
http://msdn.microsoft.com/en-us/library/ff727840.aspx Testing/debugging for surface
http://msdn.microsoft.com/en-us/library/ff727766.aspx Surface Keyboard
P.s. I believe this is correct... I am still researching on this issue and if I find a way around this Ill update my answer...
Edit:
To extend this answer.. You should code the application to use the Surface Shell, via the launcher you should either launch your application from the selection of available applications and / or set the shell to one application mode if you're programming for a store environment.
I have a NotifyIcon control that appears in the system tray. How can I find the location (x, y) of the icon on the screen?
Thanks
Actually you can do this, here is code that shows you how and much more.
In any Mouse Event from NotifyIcon, simply look at Control.MousePosition, this contains the (x,y) of the mouse. The same can be done to position a ContextMenu on a form/control exactly where the form/control was clicked using these (x,y) values.
A good example of both of these is here:
More Here: http://code.msdn.microsoft.com/TheNotifyIconExample
After a long journey of trial and error, and the development of my own notify icon locator, I developed a way to find the best position icon, so far so good, until I saw this amazing project by #zhwang SuperNotifyIcon. This project has three ways to locate the notification icon, ranging from the inaccurate (like my own development) to the use of new in Windows 7 API. This is the definitive answer.
You can't. It isn't hard to get the window handle for the system tray, GetClassName() returns "ToolbarWindow32", the class name of the standard TOOLBAR common control. Beware that a 64-bit OS has two of them. Then you can send messages to it like TB_GETBUTTONINFO. Chief hang-ups are that you won't know what button ID to choose and the returned info does not include the button position.
Which is for the better, buttons move around without you being able to lock them.