I thought this could've been a common question, but it has been very difficult to find an answer. I've tried searching here and other forums with no luck.
I'm writing a C# (.net version 4) program to monitor a process. It already raises an event when the process starts and when it stops, but I also need to check where is this process reading from and writing to; specially writing to since I know this process writes a large amount of data every time it runs. We process batches of data, and the path where the process writes to contains the Batch ID, which is an important piece of information to log the results of the process.
I've looked into the System.Diagnostics.Process.BeginOutputReadLine method, but since the documentation says that StandardOutput must be redirected, I'm not sure if this can be done on a process that is currently running, or if it affects the write operation originally intended by the process.
It is a console application in C#. If anyone have any idea on how to do this, it would be much appreciated.
Thanks in advance!
Output redirection would only help you solve the problem of intercepting the process' standard output stream. This would have no effect on read/write operations to other files or streams that the program would use.
The easiest way to do this would be to avoid reverse engineering this information and exert some control over where the process writes its data (e.g. pass a command line parameter to it to specify the output path and you can monitor that output path yourself).
If that is impossible for some reason, you can look into these approaches, all of which are quite advanced and have various drawbacks:
Use Detours to launch the process and redirect calls to CreateFile to a function that you define (e.g. you could call into some other function to track the file name that it used and then call the real CreateFile). Note that a license to use Detours costs money and it requires you to build an an unmanaged DLL to define your replacement function.
Read the data from the Microsoft-Windows-Kernel-File event tracing provider. This provider tracks all file operations for everything on the system. Using this data requires advanced knowledge of ETW and a lot of P/Invoke calls if you are trying to consume it from C#.
Enumerate the open handles of the process once it is started. A previous stackoverflow.com question has several possible solutions. Note that this is not foolproof as it only gives you a snapshot of the activity at a point in time (e.g. the process may open and close handles too quickly for you to observe it between calls to enumerate them) and most of those answers require calling into undocumented functions.
I came across this implementation recently: DetectOpenFiles but i have not used and/or test it. Feel free to try it. It seems to deliver open file handle information for a given process id. Looking forward to read your experience with it! ;-)
Related
Don't close it as duplicate, since I have a subtle but significant change from the similiar questions:
Is it possible to capture output of an external process (i.e. stdout) in java, when I didn't create the process, and all I know is the process name?
I'm running on windows 7.
EDIT:
If there is a way to do it in other language (C#\C++), then I can write a "CaptureOutput" program that capture the output, write to stdout, and in my java code to launch "CaptureOutput" and read its stdput.
Ugly, but might work.
So answer in other languages is also okay with me.
First let me say that what you're asking breaks all the rules of process isolation.
If your process does not create the process whose output you want to capture, and you also don't have access to modify the calling process (command shell? service manager? you haven't said which). Then your only chance, and it is a slim one at best, is to inject a thread into the process and while all its other threads are suspended, alter the global stdout (and stderr?). This can only be done by a process with full access privileges to the target process. Performing such surgery on a running process is not for the faint of heart.
What you are trying to do is pretty dangerous. It would be very easy to accidentally corrupt the memory of process you're trying to get into. Test, test, test. Then test some more. And good luck - I know I wouldn't want to have to pull this off.
This article - API Hooking - explains how to get started with what you want (using C++). Once you have your code injected into a running process, there are other Windows API calls to replace STDOUT (e.g. SetStdHandle).
Do you have any control over when the process starts? If so, you could start the process and have it pipe its stdout to a file which could be read or to another program you write that could log it in a database, event viewer, etc.
Under Linux, check out the operating system's IPC mechanisms such as message queues, pipes, shared memory, and sockets. These mechanisms allow for Inter-process communication. Although, if your particularly interested in a program's output, a work-around could just have the first process output the data out to disk onto a file, and read with a separate process. In this way, you could use multiple languages for the task. A simple example would be to have C++ write some data out to a file, and use JAVA read/use the data, given the same file. Hope I came close to answering, if at all.
I've written a ssh server in c# and I thought it'd be neat to hook up powershell as a shell. I've tried 2 methods to get this to work properly but both are far from perfect. Here's what I've tried:
Launch powershell.exe and redirect it's std(in/out). This doesn't
work well since powershell.exe detects it is redirected, changes
it's behaviour. What's more, it expects input data on the stdid, not
commands. So it uses the console api to read commands.
Host powershell in a "wrapper" application. This has the advantage of
being able to provide a "console" implementation to powershell (via
the PSHostRawUserInterface). This works better, but you can still invoke
commands (mostly real console applications) like "... | more", that expect
to be able to use the console api, and subsequently try to read from the
console of the wrapper process.
So what I'd like to do is have a set of functions replace the regular console input/output functions that console applications use, so I can handle them. But that seems rather drastic to the point of being a bad design idea (imo).
Right now I am on the idea of manipulating the console by sending the relevant keys with native/Pinvoke functions like WriteConsoleInput. I gather that it might be possible to fake the console that way. But I don't see how I would then "read" what happens on the console.
Also keep in mind, it's a service, so preferably it shouldn't spawn an actual console window, although perhaps in windows session 0 that wouldn't show up and not matter.
You've got PSSession for this purpose and the Enter-PSSession CmdLet. What will your SSH with Powershell do that PSSession is not doing ?
But if you want to do that here is a solution whithout writting anything : Using PowerShell through SSH
Edited 02/11/2011
PowerShell inside provide another way to do it whithout writting anything (free for personal usage).
Host03 sample, can perhaps provide basic code to do what you wat to do.
I installed PowerShellInside as suggested by JPBlanc, but didn't use it for very long. The one connection thing is just too limiting, and I don't like being limited (especially if that limitation is profit based but thats a whole other discussion i shouldn't get into). And despite being a solution to the original problem, it feels unsatisfactory because it doesn't solve the programming problem I ran into.
However, I did eventually manage to solve said problem, indeed by using the windows api calls in a wrapper process. Because there are quite a few pitfalls, I decided to anwser my own question and give others looking at the same problem some pointers. The basic structure is as follows:
Start the wrapper process with redirected stdin/-out (and stderr if you want). (In my case stdin and out will be streams of xterm control sequences and data, because that is the ssh way)
Using GetStdHandle() retrive the redirected input and output handles. Next SetStdHandle()'s to the CreateFile() of "CONIN$" and "CONOUT$", such that child processes inherits the the console and does not have the redirections of the wrapper process. (Note that a security descriptor allowing inheriting is needed for createfile)
Setup the console mode, size, title, Ctrl-C handler's, etc. Note: be sure to set a font if you want unicode support, I used Lucida Console (.FontFamily = 54, .FaceName = "Lucida Console"). Without this, reading the characters from your console output will return codepaged versions, which are horrible to work with in managed code.
Reading output can be done with the SetWinEventHook(), be sure to use out-of-context notification, because I'm pretty sure that having your managed application suddenly run in another process context/address space is a Bad Idea™ (I'm so sure that I didn't even try). The event will fire for every console window, not just your own. So filter all calls to the callback by window handle. Retrive the window handle of the current console application with GetConsoleWindow(). Also don't forget to unhook the callback when the application is done.
Note, upto this point be sure not to use (or do anything that causes the load of) the System.Console class, or things more than likely will go wrong. Usage after this point will behave as if the sub process had written to the output.
Spawn the needed sub process (Note, you must use .UseShellExecute = false or it will not inherit the console)
You can start providing input to the console using WriteConsoleInput()
At this point (or on a separate thread) you have to run a windows message loop or you will not recieve console event notification callbacks. You can simply use the parameterless Application.Run() to do this. To break the message loop, you must at some point post an exit message to your message loop. I did this with Application.Exit() in the subprocess's .Exited event. (Note use .EnableRaisingEvents for this to work)
Calls will now be made to your win event callback when something on your console changes. Pay attention to the scroll event, this might work somewhat unexpected. Also make no assumptions about synchronous delivery. If the sub process writes 3 lines, by the time you are processing the first event, the remaining 3 lines might already have been written. To be fair, windows does a nice job of composing events such that you don't get swamped with single character changes and can keep up with the changes.
Be sure to mark all PInvoke definitions with CharSet=CharSet.Unicode if they contain a character anywhere in the input or output. PInvoke.net missed quite a few of these.
The net result of all of this: a wrapper application for the windows console api. The wrapper can read/write the redirected stdin and stdout to communicate with the world. Ofcourse if you want to get fancy you could use any stream here (named pipe, tcp/ip, etc..). I implemented a few xterm control sequences and managed to get a fully working terminal wrapper that should be capable of wrapping any windows console process, translate the xterm input to input on the target application's console input and process the application's output to xterm control sequences. I even got the mouse to work. Starting powershell.exe as sub process now solves the original problem of running powershell in a ssh session. Cmd.exe also works. If anyone is interrested I'll see about posting the full code somewhere.
I've been having a hard time getting the output of a "sub-process" (one launched internally by a blackbox process that I'm monitoring via c# System.Diagnostics.Process)
I took the advice given by the answer of my previous post:
here. And there you can find the details of what I've been going through.
At this point, although I'm able to locate the ssh process spawned by process1.exe, that I'm monitoring. I can't redirect the output to my c# program, because it is an "already running process", and wasn't launched directly from C#.
It seems that, all the properties that you set on a System.Diagnostics.Process object, only take effect if you are explicitly launching that process from your c# application; if some other "unmanaged process" has launched the process, setting this redirection has no effect, because the process has already been launched by something that didn't specify the redirection I need.
Is there any way to redirect output of a process that has already been launched (a process launched by a program for which I have no scope to pre-specify redirection before this process is launched)?
Assuming there's no more straightforward solution, you could try to run a piece a code in another process through CreateRemoteThread(), explained here.
Instead of redirecting the output directly from the running process, can you capture the output as it leaves the process A at the intended destination, the pass it into your new process?
Perhaps you can look at this code. I found it when searching for a solution to do the same kind of thing; however, it was not really inter-process.
If that doesn't help you might be able to look at P/Invoking SetStdHandle and GetStdHandle which are supposed to be used when redirecting standard output. I think this is what the code sample linked to does to make the redirection happen.
Note: I just looked at this stuff and didn't actually get it to work properly. (I had a better solution available to me because I had access to the source code outputting to the console)
I've got the same conundrum. It's not an option for me to invoke anything internal in the slave process. It's already running, which code is beyond my control. But I do know it spits out Standard Output, which I want to monitor, process, etc. It's one thing I kick off the process, I can configure the redirection, but in this instance, the process will be launched secondarily to my kicking off the primary slave process. So I do not have that option. I haven't found a way for .NET Process to work under these conditions. Perhaps there is another way, maybe C++ is the way to go here? This would be marginally acceptable, but I would like to approach it from a .NET C# perspective if possible.
is there a way to be notified when a program is executed or terminated by a user in c#? I am talking about all the programs that a user can execute, not just a certain program. I would like to be notified whenever a user execute .exe files.
I can't even think of a keyword to google.
any suggestions would be appreciated!
The closest thing I know of would be to use Hooks.
You can use WH_SHELL Hooks to receive notification any time a new, non-owned, top level window is created or destroyed by the system. This isn't the same as a process, but it's pretty close in many cases, and potentially more useful in others (since it'd show a new word document window opening after one was already opened, even though they're using a shared process).
You might be able to combine that with EnumProcess to check to see if the process list has changed. This would work for tracking windows applications (but not services or console-based applications running in an existing console).
In Microsoft .NET Framework 3.5, you can get a list of Processes and register for the Process.Exited event. I suppose someone could implement a polling system in which they continually looked for new Processes, but that doesn't really notify you when something launches.
-- EDIT --
You might find this article useful. If you're willing to write a kernel mode driver, you can control every process start and finish.
Now, if you really want to get wild, you can hook into Microsoft Detours. Here is an interesting article about that.
Say I want to be informed whenever a file copy is launched on my system and get the file name, the destination where it is being copied or moved and the time of copy.
Is this possible? How would you go about it? Should you hook CopyFile API function?
Is there any software that already accomplishes this?
Windows has the concept of I/O filters which allow you to intercept all I/O operations and choose to perform additional actions as a result. They are primarily used for A/V type scenarios but can be programmed for a wide variety of tasks. The SysInternals Process Monitor for example uses a I/O filter to see the file level access.
You can view your current filters using MS Filter Manager, (fltmc.exe from a command prompt)
There is a kit to help you write filters, you can get the drivers and develop your own.
http://www.microsoft.com/whdc/driver/filterdrv/default.mspx is a starting place to get in depth info
As there is a .NET tag on this question, I would simply use System.IO.FileSystemWatcher that's in the .NET Framework. I'm guessing it is implemented using the I/O Filters that Andrew mentions in his answer, but I really do not know (nor care, exactly). Would that fit your needs?
As Andrew says a filter driver is the way to go.
There is no foolproof way of detecting a file copy as different programs copy files in different ways (some may use the CopyFile API, others may just read one file and write out the contents to another themselves). You could try calculating a hash in your filter driver of any file opened for reading, and then do the same after a program finishes writing to a file. If the hashes match you know you have a file copy. However this technique may be slow. If you just hook the CopyFile API you will miss file copies made without that API. Java programs (to name but one) have no access to the CopyFile API.
This is likely impossible as there is no guaranteed central method for performing a copy/move. You could hook into a core API (like CopyFile) but of course that means that you will still miss any copy/move that any application does without using this API.
Maybe you could watch the entire filesystem with IO filters for open files and then just draw conclusions yourself if two files with same names and same filesizes are open at the same time. But that no 100% solution either.
As previously mentioned, a file copy operation can be implemented in various ways and may involve several disk and memory transfers, therefore is not possible to simply get notified by the system when such operation occurs.
Even for the user, there are multiple ways to duplicate content and entire files. Copy commands, "save as", "send to", move, using various tools. Under the hood the copy operation is a succession of read / write, correlated by certain parameters. That is the only way to guarantee successful auditing. Hooking on CopyFile will not give you the copy operations of Total Commander, for example. Nor will it give you "Save as" operations which are in fact file create -> file content moved -> closing of original file -> opening of the new file. Then, things are different when dealing with copy over network, impersonated copy operations where the file handle security context is different than the process security context, and so on. I do not think that there is a straightforward way to achieve all of the above.
However, there is a software that can notify you for most of the common copy operations (i.e. when they are performed through windows explorer, total commander, command prompt and other applications). It also gives you the source and destination file name, the timestamp and other relevant details. It can be found here: http://temasoft.com/products/filemonitor.
Note: I work for the company which develops this product.