So for work they have me writing a simple program for tracking employee efficiency within their workflow (things like using keyboard shortcuts, window locations, how often they need to look stuff up). Currently we want to track the 'F5' key (brings up next work item), 'Alt+Tab' (changes windows), 'Ctrl+V' (paste), but may be expanded as they find there are more shortcuts or things they want to track.
Note We are on windows 7, and using c# to write the tracking program.
In order to do this I wrote a low-level hooking library to capture the chosen keystrokes, send off the message down the hook chain and then add a note to a db that the key was used. The hooking library works great in All web browsers and most normal programs (except we don't actually care about browsers so we ignore everything done in them).
The issue is that the application that they use for managing their work (the program we actually care about tracking) some how stops our hooks from hooking and I do not know how. The application in question is TA2000 Desktop.
I know that with the way hooks work if an application fails to call callnexthook() within the LowLevelHooksTimeout period that the system kills the hook. So figuring maybe TA2000 was just taking to long or something I bumped up the timeout to 30 seconds (yes I know this is significantly more time than a hook should even need) but this had no effect.
The next thing I tried was implementing a tracking system based on the Raw Input API. And once again the tracking tracks on browsers, Microsoft office, notepad, and all the other programs I opened except it still is unable to track key press in TA2000. This really surprised me because according to MSDN
An application does not have to detect or open the input device.
An application gets the data directly from the device, and processes the data for its needs
An application can distinguish the source of the input even if it is from the same type of device. For example, two mouse devices.
So if I am getting the data directly from the device how is TA2000 preventing me from also getting the key press?
The last thing I could think of trying was using dll injection on TA2000 to inject a hook. However this method seems risky because It is something neither I nor any other developer here has any experience with and the application we want to track is operation critical so messing it up can not happen and injecting code into its memory space seems like a good way to mess things up.
If someone could explain how TA2000 could be stopping me from tracking keystrokes and how to beat it or point me in a good direction I would be very appreciative.
p.s. This felt questionable as an appropriate question for the SO format but it also feels specific enough to be a viable question. So sorry if this is not a good question but I am at my wits end with this.
This financial software package is secured to prevent snooping. Running the key logging software as Administrator appears to fix this specific problem. The security was identified initially using Sysinternals' Process Explorer, which is a great starting point for unexpected problems like this.
Related
Disclaimer: Yes, I know that the general answer to whether or not to use GC.Collect() is a resounding "NO!". This is the first time in several years of programming that I ever consider using it at all.
Well then, here's the situation: We have developed a C# scripting tool based on the Microsoft.CodeAnalysis.CSharp.Scripting libraries (v3.6.0). It's a Winform GUI with editor etc., not unlike others out there. We use it for the validation of integrated circuits, meaning that its primary task is interfacing lab equipment such as power supplies, pattern generators, meters and the like. For the communication to said instruments we predominantly rely on National Instrument's VISA framework, albeit not exclusively. Some devices are controlled directly via DLLs from their respective manufacturers. In general, this system is working beautifully and by now it is successfully used by quite a lot of design engineers who do not know the first thing about the intricacies of .NET and C#.
At this point I should explain that the user can simply write a method (i.e. on "top-level") and then execute it. The Roslyn-part behind this is that the input is fed to CSharpScript.Create() and then compiled. The execution of a method is done via Script.ContinueWith("method name"). Inside of such a method the user can construct an object like, say, new VISA("connection string"), which connects to the device and then communicate with the device via this object. Nothing forces him or her to care about disposing the object (i.e. closing the connection).
Now, the problem is this: recently, very sporadic crashes of the GUI application have occurred with no feedback at all from the system - the form just closes and that's it. By trial-and-error we are currently 99% sure that if all connection objects are explicitely disposed within a method, the crashes do not occur. So, rewriting the method to something like this fixes the problem:
using(var device = new VISA("connection string"))
{
device.Query("IDN?");
}
The reason why I look into the GC's direction at all is that there is no discernible correlation to any actions from the user. The guys might run such methods for an hour without a problem and then, when scrolling in the editor, when no method is currently being executed, the GUI closes without comment. And that's why I'd like to get some input from people more knowledgeable about Roslyn and the GC:
Are there known issues with this scripting library and GC? (I would very much assume that there aren't)
Since the explicit disposal of objects seem to prevent the issue, might this be one of the extremely scarce situations where the use of GC.Collect() might be warranted? (admittedly, I could not yet test whether that also prevents the problem thanks to of home office)
Any ideas what can cause a .NET application to crash without any kind of feedback and how to obtain more information about such a crash? (the scripting engine is a separate DLL, as are the device drivers; the GUI only handles the graphics)
I am fully aware that this is a rather vague description of the problem with very little source code. This is due to the fact that the application comprises of quite a lot of source code and I have no idea what might be relevant here. Also, all namespaces in the above text refer to Microsoft.CodeAnalysis.CSharp.Scripting, except for VISA, which is self-defined. Obviously, I will gladly answer any follow-up questions for getting to the bottom of this.
Thanks in advance.
Short answer: No. It's not only not warranted, it's completely missing the actual issue.
Further explanation: #canton7 instantly hit the nail on the head when writing
I'd argue that your application shouldn't crash even if a finalizer does end up being called
The root issue hid inside a 3rd party DLL in form of an, at the very least, suboptimal implementation of IDisposable. Once I zoomed in on that, it was rather easy to produce a workaround for that.
My original question is so very misguided that I'd like to state the one that I should have asked:
How do I trace a crash of my C# application when my application's logging does not show anything?
This question has been answered comprehensively in a number of posts. In my case, the crash could be seen in the Windows event log.
I apologize for the length of the question, but I believe it is difficult to understand the “why” without the background.
Background: I have two applications running in a Windows Embedded Standard 7 environment. They should be the only two applications running on the machine. One, called “Controller”, is written in C++ the other, “DBconnector”, is written in c#. This is not new code. It has been in active use and development for almost 20 years.
The purpose of the software is to run a manufacturing machine for producing parts. These machines are big and dangerous if the program crashes. Long ago, I discovered that if the network went down for some reason, all the threads in the application would stall – not just the network thread. This was disastrous since leaving the controller in a state with the wrong relays on in extremely rare circumstances could cause the machine to literally explode. Note: Several things have been added to the software and hardware to prevent this now. While this danger doesn’t really exist anymore, stability is still extremely important. I never want the operator to be stuck in a state where they can’t hit the reset button. My solution at the time was to move the networking tasks into a separate application. The OS was windows XP based at the time. I have no idea if the problem still exists in windows 10 since I really don’t want to rewrite hundreds of thousands of lines of code to try and merge the two programs now.
The development of the two programs diverged such that the one that controlled the machine, Controller, was designed for extreme stability and the other, DBconnector, was where dangerous things like networking and most file I/O happened. Communication between the two programs is facilitated using a memory mapped file that they both can access. I have no problem sharing window handles or process id’s or any other data that might be needed between the two programs.
Here is my question. How can I make the Controller application display the GUI of DBconnector? For example, I have started to add functionality to Controller that requires DBconnector to display the quality control sheets that are held on a web site on company servers. I want for an operator to be able to pull up the quality control sheet directly on the machine. The operator currently only interacts with the Controller application. I don’t want Controller to be able to access the network. Also, C# has some tools to make displaying a web page easy. It seems to me that the place to do this is DBconnector. The problem is that DBconnector runs in the background and cannot currently be seen or accessed by a user. So, the question is how to solve this.
First option I have tried is to tell DBconnector to come forward and put Controller in the background. Then, when the user is done, Controller comes back to the front. I have made this to work using some hacks, but it is inconsistent. The trick I used was to minimize and then maximize DBconnector which seems to bring it to the front most of the time and try to hold focus on one or the other. There still might be a way to do it this way, but it needs to be something that is consistent.
The second option is to run the DBconnector application inside of one of Controller’s windows. I have no idea how to do this. I thought about using ATL or COM, but I think these run as threads within Controllers process rather than as a separate application.
The third option I’ve considered is to create a window inside Controller that intercepts and passes all user input messages directly to Dbconnector using a windows message handle and takes a screenshot of DBconnector whenever the it is invalidated and passes it through the memory mapped file. Currently, this is what I am swaying towards.
Are there any suggestions on how to do the first and last option better, or how to do the second option at all, or another solution that I have missed? Keep in mind that our current hardware is running Windows Embedded Standard 7. The project is currently in visual studio 2015. The C++ window technology is MFC implemented using libraries originally from around 2003 I think. DBconnector is in .NET framework 4 in C#.
unfortunatly, when you listen to WM_QUERYENDSESSION, you do not get the information if the user has requested a reboot or a shutdown. This is really bad design, but it's the way Windows is, so I was thinking of hooking the call to NTShutdownSystem, which gets a parameter telling the system to perform a reboot or to shutdown.
The question is: how can this actually be achieved in C#? I want to get some kind of hook that I can use to determine the parameters passed to NTShutdownSystem, and then save that information. After that, I want to call the "real" NTShutdownSystem the way it was intended by the user.
Do you have any sample code illustrating this?
The reason why WM_QUERYENDSESSION does not give a shutdown reason is that the user may just be logging out at that time, rather than shutting down the system.
This generally falls under the category of kernel level hooking and has generally not been considered a good thing as it can influence stability of the system. Most of them are written in C or C++, and generally have to go to a lot of effort to perform the hook across all the programs that are executing - e.g. hooking the routines at program load-time.
This is not a trivial, but there are some frameworks that have been written to help with trying to hook routines like this using managed code (e.g. C#)
The next question to ask is why do you care?
edit NTShutdownSystem is invoked very late in the shutdown process - at that point you probably have no UI and no way of doing anything. I would recommend intercepting ExitWindowsEx, InitiateShutdown, InitiateSystemShutdown and InitiateSystemShutdownEx - I don't know if some of them are called by the other, but you should probably only record the reason and then react to the reason in the WM_QUERYENDSESSION code of your standard app.
I am working on a program that uses keyboard hooks. However, when the PC that the program is running on is just slightly overloaded, it causes Windows to disconnect the hook from the program, causing it to no longer respond to keystrokes.
Is there a way to prevent this, or even better, propose a different way of solving the exact same problem, by using a different architecture, maybe involving a pipeline?
You can't "detect" this, and you absolutely shouldn't need to. What you're describing is a feature, specifically one introduced in Windows 7 to protect your system from rogue applications.
The applicable documentation describes it thusly (pay particular attention to the bolded section):
The hook procedure should process a message in less time than the data entry specified in the LowLevelHooksTimeout value in the following registry key:
HKEY_CURRENT_USER\Control Panel\Desktop
The value is in milliseconds. If the hook procedure times out, the system passes the message to the next hook. However, on Windows 7 and later, the hook is silently removed without being called. There is no way for the application to know whether the hook is removed.
The solution here is most certainly not to figure out a way to "detect" when the hook is uninstalled and reinstall it. You should have figured out that you're doing something wrong when the operating system uninstalled the hook the first time.
The actual solution is to redesign your application to return from the hook procedure more quickly. Ideally, you should return almost immediately. If you need to run some type of intensive calculation in response to the low level messages (and I can't really imagine why you would), then you should store the information you receive, return from the hook procedure, and do your processing at a later time (probably on a separate thread).
In fact, that's almost exactly what the documentation continues on to suggest:
Note: Debug hooks cannot track this type of low level keyboard hooks. If the application must use low level hooks, it should run the hooks on a dedicated thread that passes the work off to a worker thread and then immediately returns. In most cases where the application needs to use low level hooks, it should monitor raw input instead. This is because raw input can asynchronously monitor mouse and keyboard messages that are targeted for other threads more effectively than low level hooks can. For more information on raw input, see Raw Input.
I am not so sure that the keyboard hook is always to blame. We all seem to agree that under ideal or average conditions everything should be responsive. But during its lifetime from startup to shutdown a hook also has to survive some worst-case scenarios. In my company we wrote some keyboard hooks, and they are as lightweight and asynchronous as possible, yet they still occasionally appear to get disconnected.
As a user, every day I type several ten-thousand characters. I reboot once a month at best, and on occasions I am guilty of skipping a Windows Update. With options like suspend and hibernate, I don't think I am alone. As a developer, I have to make sure that the hook keeps running from beginning to end, regardless of what happens to the system.
Normally my system is very responsive. But there can be brief, exceptional moments where it is on its knees to the point that even Windows Aero gets switched off. Right now it is very snappy. But if I press the Show Desktop button, the mouse will freeze for at least a second during the time that my 65 or so open windows are all collapsed. What if I press a key during one of these moment?
If Windows can freeze the mouse for one second, if Windows can even switch off Aero during a brief moment of heavy load, why can't a keyboard hook be allowed to survive a similar exceptional time of overload? Instead, because of one exceptional moment, Windows pulls the plug, silently affecting the remaining computing experience until system shutdown. For 200 ms affecting just one keypress out of the thousands we press every day, for just that fraction of a second, I, the user, have to reboot the system because that's the only way I understand will bring back the keyboard macro or whatever utility I depend on for my productive work.
Even if it was guaranteed to prevent the above (which worst-case experience seems to suggest is not the case), I am not so sure that everything can easily be done in a separate thread. For example, let's say the user sets up a hotkey that is to start an application. Couldn't there be reasons for starting the application from the current context (foreground window, privileges, etc.)? And wouldn't it be reasonable for the user to actually expect and accept a delay, because he knows that, using the keyboard, he just started something that takes longer? I don't know if the example is technically sound, but I wanted to illustrate how sometimes things that otherwise may be unacceptable could be acceptable, as a result of a known event.
Keyboard hooks can be very useful for many things, from macros to error correction to launching things, and this behavior introduced in Windows 7 is putting the good and the bad, the acceptable and the unacceptable, the average and the exceptional, all in the same basket. This hurts both users, because quality keyboard hooks may get killed, and developers. Just imagine what a support nightmare it is when you have no official solution to your application working well under normal conditions, but being killed at random under some heavy load (or other inexplicable) circumstance.
To conclude this with a question, does anyone know what the status is under Windows 8, has anything changed?
Cody Gray's response is excellent, but FWIW, and purely for testing purposes, you may be able to detect a disconnected hook.
When your hook proc is invoked, store the current tick count in a variable accessible to the main thread. In the main thread, periodically call GetLastInputInfo. It will give you the tick count for the last input event in the system (not just in your process). If the value provided by GetLastInputInfo is significantly later (newer) than your last hook proc tick count, it's a good guess that hook has been disconnected.
I think there are some "bad-performance" code in your hook . that's the reason why makes slightly overload .
"it causes Windows to disconnect the hook from the program"
Does any error raise in your hook and you don't handle it ?
AFAIK, Windows wouldn't disconnect the hook if it works well by itself.
Try increasing priority of the process of your application.
is there a way to be notified when a program is executed or terminated by a user in c#? I am talking about all the programs that a user can execute, not just a certain program. I would like to be notified whenever a user execute .exe files.
I can't even think of a keyword to google.
any suggestions would be appreciated!
The closest thing I know of would be to use Hooks.
You can use WH_SHELL Hooks to receive notification any time a new, non-owned, top level window is created or destroyed by the system. This isn't the same as a process, but it's pretty close in many cases, and potentially more useful in others (since it'd show a new word document window opening after one was already opened, even though they're using a shared process).
You might be able to combine that with EnumProcess to check to see if the process list has changed. This would work for tracking windows applications (but not services or console-based applications running in an existing console).
In Microsoft .NET Framework 3.5, you can get a list of Processes and register for the Process.Exited event. I suppose someone could implement a polling system in which they continually looked for new Processes, but that doesn't really notify you when something launches.
-- EDIT --
You might find this article useful. If you're willing to write a kernel mode driver, you can control every process start and finish.
Now, if you really want to get wild, you can hook into Microsoft Detours. Here is an interesting article about that.