Above is a screen shot I took in Windows PE environment while applying an image to a machine. As you can see I have filtered the results of Windows Sysinternals Process Monitor so that only ReadFile operations on the Win8 32-bit image by the Dism.exe process are shown.
I am in need of finding a way to access this same information using C/C#/C++ without the .NET framework (because supposedly the .NET framework doesn't exist in PE environment). I have searched, asked around, and searched again for Namespaces and Classes that would allow me to extract this information from the kernel without luck.
I am most interested in having the ability to access the value "Offset" that you can see in the image. This number is the location offset of the Read operation on the file and I need this value for the program I am required to make.
Does anybody know how I can achieve my goal? This is obviously possible to do in the PE environment because Process Monitor does it, so how can I do the same thing?
You might want to look at file mini-filters. Process Explorer basically installs one on the system when you start it for the first time. The mini-filter can be installed and started without the need for a reboot.
The mini-filter runs inside the kernel like any other device drivers and from there you can monitor any file activity (read, write, create, delete, close, execute, etc.)
You'd have to write this in C. You can also get info from the OSR distribution list.
Related
There are several threads on SO that describe how to check which application creates a file with tools like Sysinternals process monitor. Is something like this possible programmatically from .net?
Background: My program has to remote-control a proprietary third party application using its automation interface, and one of the functions I need from this application has a bug where it creates a bunch of temporary files in %TEMP% that are called tmpXXXX.tmp (the same as .net's Path.GetTempFileName() does) but does not delete them. This causes the C drive to become full over time, eventually failing the application. I already filed a bug to the manufacturer, but we need a temporary workaround for the time being, so I thought of putting a FileSystemWatcher on %TEMP% that watches tmp*.tmp, collects these files, and after the operation on the third-party application finishes, deletes them. But this is risky as another application might also write files with the same file name pattern to %TEMP% so I only want to delete those created by NastyBuggyThirdPartyApplication.exe.
Is this anyhow possible?
This kind of things is possible, but maybe a bit tricky.
To know who created the file, look at the user that owns it. Therefore you might need to create a specific user, and that application will run under this specific user. In order to do that, you need to create a small application that will start your buggy app by impersonating another user, so anything done within the app will be under this user so as file creating...
I don't know how to monitor and get triggered when a file is created, but nothing can prevent you from setting a timer that wakes up every five or ten minutes, then checks if any file in the directory is owned by the application user and closed, so it deletes it.
Maybe if they react fast for this bug fixing, you won't need your app very long time. So another solution, if possible might just to change the Temp folder into another drive, which has lots of space...
One solution is that you use a FileWatcher to automatically delete all the files but before deleting you should check if the file is not currently locked or used by other process, for example the Sysinternal Suite has a tool called handle.exe that can do this. Use it from the command line:
handle.exe -a
You can invoke this from a c# program (there might be some performance issues though)
So what you would do is when a file is created you verify if it is in use or locked (for example u can use the code provided in Is there a way to check if a file is in use?) and then delete it.
Most of the time when an app is using a temp file it will lock it to prevent just what you fear, that you might delete files from other processes.
As far as I can tell there is no sure way to identify which process created a specific file.
I am looking for a programmatic way to copy files from one network drive to another. I have created a program that does a simple copy but this is not enough as the line is not reliable and the files ends up being corrupted. Is there a technology that syncs the folders and does a sum-check or something like that.
I need it to be a c# app as I need to know when the copy is completed so I can process the files on the other side. I am using windows services to co-ordinate this.
I'm just looking for someone to point me in the right direction with tutorials if necessary.
What you are looking for is Robocopy (aka Robust Copy), it is built in to windows Vista and newer and it has features to retry on network failure.
One down side is there is no "Verify" functionality built in, but if you need to you can use a 3rd party file hashing program and put the copy in a script that verifies the hashes after the copy completes.
Here is a query that copies all files and sub-folders in a restartable mode, if a copy fails it will re-try 1,000,000 times by default waiting 30 seconds between tries. (you can change that with /r:<N> and /w:<N> where <N> is a number)
robocopy C:\SourceFolder \\DestComputer\DestFolder /zb /e
The only time I have ever done anything like this has been during the deploy process to distributed servers. In that instance I zipped the files, moved and unzipped them using PSEXEC.
If I were you, I would begin by figuring out how to zip the files (To prevent the corruption issues you have described) and then worry about how to move them.
I have a program that is doing raw IO to disks within Windows.
All works fine if the target disk is online. However, the default behavior in some Windows OSes is to have new disks initially offline.
I am having a hard time finding the correct API to do this on Windows. The command line equivalent would be something like:
"select disk 2", "online disk" | diskpart
However I need to be able to do this in code. I looked through the DeviceIoControl Win32 API (which I think is right) but cannot determine which control code to use. The fact that I can't find it makes me think I might be missing a better API to use.
For future generations, the answer (on Win 2k3/Vista and later) is the Virtual Disk Service (VDS). There's some work getting it all together, especially if you don't use COM objects within .NET that much.
Disk online/offline is done with IVdsDrive::SetStatus. At least it should; I found that I could solve my problem with simply disabling read-only status on my disk. I was able to do this with IVdsDisk::SetFlags with the appropriate flag value.
This question has a couple useful links to the Windows API, including the DeviceIOControl method.
After looking through all of the enumerations, I could not find anything related to bringing a disk online, or make any interesting change to the disk beyond formatting/partitions. This is likely because only hot-swappable hard drives are supported by this functionality. The market for hot-swappable hard drives is very small, and the vast majority of those situations there are drivers to support any needed operations. Finally the remainder should be able to use the diskpart tool for whatever is necessary.
You need to look again at your requirements I think. You are running a process that has the rights necessary to online a hard disk, but cannot access a command line program? Here are some suggestions for common reasons to not use a command line program:
Can't have a black screen pop up - tons of solutions to this problem available online
Security team won't allow it - you are already running the process as an administrator so you trust it, why wouldn't you trust the built in Windows function
Technical problems preclude calling other processes - I would be interested in how this was managed given the process is running as an administrator
Coding guidelines such as "Always use the API" - there isn't one due to lack of need
Not sure about C#, but I'm using this in C++:
Try calling DeviceIoControl() with IOCTL_DISK_SET_DISK_ATTRIBUTES. The file handle must have read and write access. I think it requires at least Windows 7. It doesn't work on Windows 2003 x64. Windows 8 successfully takes the disk offline and then you can rewrite it from a backup.
BOOL disk_offline(HANDLE h_file, bool enable){
DWORD bytes_returned = 0;
BOOL b_offline = 0;
if(get_size_volume_disk(h_file)){
SET_DISK_ATTRIBUTES disk_attr;
ZeroMemory(&disk_attr, sizeof(disk_attr));
disk_attr.Version = sizeof(SET_DISK_ATTRIBUTES);
disk_attr.Attributes = enable? DISK_ATTRIBUTE_OFFLINE: 0;
disk_attr.AttributesMask = DISK_ATTRIBUTE_OFFLINE;
b_offline = DeviceIoControl(h_file, IOCTL_DISK_SET_DISK_ATTRIBUTES, &disk_attr, disk_attr.Version, NULL, 0, &bytes_returned, NULL);
// Invalidates the cached partition table and re-enumerates the device.
if(!enable) BOOL b_update = DeviceIoControl(h_file, IOCTL_DISK_UPDATE_PROPERTIES, NULL, 0, NULL, 0, &bytes_returned, NULL);
}
return b_offline;
}
Using DeviceIoControl and IOCTL_DISK_IS_WRITABLE control code, it is possible to check if disk is writable. If the disk is offline it returns false. This means that it is possible to determine if disk is offline and it works fine with Windows 2003 and after. However, I could not find any useful IOCTL to bring the disk online on Windows 2003. IOCTL_DISK_SET_DISK_ATTRIBUTES only works with Windows 2008 and after.
I want to write a program that will encrypt an entire folder and it's sub-folders, I have no problem doing this but I would like to make the entire encryption process rather transparent by letting a user double click it and have it open as if it weren't encrypted, say if it were a picture or a word document and it'd open in it's respective application.
How can a running program of mine become notified about the opening of a target file, stop the file from opening, do what it needs to do (decrypt), followed by running the resulting decrypted file.
How can I watch a file and do this in C#? Can I watch for other interactions like the user copying a watched file (since it won't be in a watched folder, it should be decrypted i.e. it's dragged to a USB device), or for deleting a watched file (say if I want to shred a file before deletion)?
P.S. The FileSystemWatcher doesn't quite meet my needs. EDIT: What I mean is that FileSystemWatcher will tell me when a file is being opened, deleted and all those events, but it won't let me step in real quick, decrypt the file, and hand it back to the process that normally opens that file.
You can rename files, add them your own extension, like thepicture.jpg.encrypted. Set your program as a default program for this extension and handle opening them
It's impossible in C#. the bare minimum would need you to use user-mode hooks on NtCreateFile, NtOpenFile, etc. You can't achieve that in C#. That wouldn't even work properly due to kernel-mode code which may try to access your files. The proper way of doing this would be to write a I/O minifilter (in C of course).
EDIT: If you're really desperate, try EasyHook - it allows you to hook functions from C#. I haven't tried it though, and it does seem risky hooking vital functions like NtCreateFile. Plus you need a fair bit of Native API knowledge.
Are you using Windows? If so, why not use the built-in BitLocker?
See this link:
BitLocker drive encryption
If you are thinking about a competitive application to BitLocker, add a comment, as I can point you in that direction as well.
Instead of trying to reinvent the wheel, use NTFS file encryption. You can encrypt single files or entire folders or drives. Plus it's completely transparent to the user and does exactly what you asks (e.g. automatically decrypt when copying to a UBS drive, etc). Just use System.IO.File.Encrypt(string) - there couldn't be anything easier.
You can't do this from usermode.
Unfortunately the only way to do this is to write a minifilter driver. Minifilter drivers allow you to intercept IO requests to files, you can then encrypt/decrypt the files you care about on the fly.
It sounds simple, but encryption minifilter drivers are very, very, difficult to get right. You will have to end up shadowing file objects which is a real challenge. Check with www.osr.com, they have a ton of information on doing exactly what you want to do.
If you choose to go this route I would recommend getting a copy of VMWare Workstation and download VirtualKD. It will let you debug at near fire-wire speeds into a VM. I would start with x64 Win7 and get remote shares working first.
One of my favorite things about owning a USB flash storage device is hauling around a bunch of useful tools with me. I'd like to write some tools, and make them work well in this kind of environment. I know C# best, and I'm productive in it, so I could get a windows forms application up in no time that way.
But what considerations should I account for in making a portable app? A few I can think of, but don't know answers to:
1) Language portability - Ok, I know that any machine I use it on will require a .NET runtime be installed. But as I only use a few windows machines regularly, this shouldn't be a problem. I could use another language to code it, but then I lose out on productivity especially in regards to an easy forms designer. Are there any other problems with running a .NET app from a flash drive?
2) Read/Write Cycles - In C#, how do I make sure that my application isn't writing unnecessarily to the drive? Do I always have control of writes, or are there any "hidden writes" that I need to account for?
3) Open question: are there any other issues relating to portable applications I should be aware of, or perhaps suggestions to other languages with good IDEs that would get me a similar level of productivity but better portability?
1) There shouldn't be any problems
running a .NET app from a flash
drive.
2) You should have control of
most writes. Be sure you write to
temp or some other location on the
hard drive, and not on the flash
drive. But write-cycles shouldn't be
a problem - even with moderate to heavy
usage most flashdrives have a life
time of years.
3) Just treat it
like's it any app that has xcopy
style deployment and try to account
for your app gracefully failing if
some dependency is not on the box.
If you want to use com objects, use reg-free com and include the com objects with your program.
You should always have control of your writes. Applications should be loaded into RAM at startup, and then memory past that is allocated in RAM, so nothing is written to the flash drive.
The most important thing for a portable application is that basically no installation is necessary for your application. You do not want to be dependant on registry values especially, since your application will not be 'installed' on other computers.
One of the issues with portable applications you may consider is data persistence. Generally, you write to a user's Application Data folder to save data. If this is the case, any data saved will only apply to the user on that computer. If you want some local application data, you may wish to create a Seralized XML file for your settings and store it locally within your application's directory. This file writing would then likely be the only write actions you'd need to worry about.
For your .NET portability issue, you could also write a small entry program in C++, which checks if the computer has .NET installed. .NET has registry values you can check to see the versions installed, so if .NET is installed, run your application, else display a message stating that .NET needs to be installed first.
Edit: I'd like to add that I do application development for Ultrasound machines using XAML in C# 3.0. The application I write works perfectly from a USB Flash Drive, while all user settings are stored on a local AppData basis, so nothing is written to the USB. While the application can be installed through an .exe installer, the installer does not write any registry values the application depends on.
I don't really have answers for #1 or #3. But for #2, the .NET CLR shouldn't be writing to an app's "installation" folder (i.e. the flash drive) unless your code specifically tells it to or is using and modifying file-based settings (ini, xml, etc) that live with the app.
Number 1 is really the kicker if you're not just writing things for personal use. Obviously hosting a portable copy of the full CLR on the thumb drive is impossible. But there are tools that can scan your assembly for its dependencies and package them up into a standalone .exe so that the CLR doesn't necessarily need to be installed on the target system.
I don't actually have any experience with this so it might be best to take what I say with a pinch of salt. But here is my take on it:
You don't need to do anything special.
It is not really a consideration of an application developer as to how and when writes are made to a drive, that is something that is far better controlled by the OS. I know that Windows caches writes to USB drives so I would trust it do handle that.
The only thing you need to consider is that your application will not be installed. So you need to make sure that you design it to run entirely self contained within the directory it is deployed to. You could optionally also make some writes to the users home directory but this needs to be done through the appropriate environment variables.
I would get writing and see if there is anything special about a flash drive that the OS doesn't handle.