I'm currently working on a component of my application which monitors a specific directory for files and keeps them in a DB together with special metadata of those files.
I use a FileSystemWatcher to track additions, renaming, deletion and changes to those files / directories. So far this works perfectly fine. But since FileSystemWatcher is not capable of tracking movements, I'm using hashes to keep track of the files.
Now in order to keep this working I must update the hash in the database once a file was changed. But my first test was proof of failure. Photoshop obviously has some tricky ways to save a file. I always catch a "psXXXXX.tmp" file being changed which I filter with the FileAttributes.Archive. The problem is: The watcher does not throw again for the real changed file. Since I need the real filename and not the temporary one, I can't update the DB.
Even if the file changes in size or the write time changed - nothing happens. It gets completely ignored. The event rises once when the temporary file gets created (or the original the renamed, I don't know) and then nothing happens anymore.
Can anybody explain this behaviour? Has anybody an idea how to avoid this?
Any workarounds or other solutions?
Thanks
I had that problem with PS, I only had to add:
watcher.Renamed += new RenamedEventHandler(OnChanged);
and all was good.
Related
I need to monitor a file for changes and see exactly where it has been modified.
It seems like I can't use FileSystemWatcher as I only get notified that a file has been modified but I can't know exactly what changed, and storing copies of the file to do a diff after the event is raised is not feasible since the file is really big.
I would like to implement something like what SysInternals have on their Process Monitor software, that tells us, for a WriteFile event, the modified part of the file given an offset and length.
I read that there is a Windows API that does this, but I couldn't find anything on how or where to start implementing it on a .net 3.5 application.
From what I have investigated, the .NET platform provides this event for handling file change events. However, this alone will not let you know what part of the file has changed.
You will need to store somewhere a copy of the file that is compared to the file whenever it changes, and then update that copy when the comparison is done. This is a very basic solution and can get really inefficient if the file is too large.
Using the FileSystemWatcher you can subscribe to the changed event ChangedEvent
FileSystemEventArgs contains a property FullPath which can be passed in to FileInfo. This will give you some basic file information, which can be stored to a database for example. You can keep a history of the file changes this way.
If you need to compare the content of the files it is more challenging and the approach will be different depending on file type.
I am working on an application that keeps track and uses files stored on the file system. Users are allowed to open, create, delete and move files in the file system. Meanwhile my application is not constantly running so I can't keep track of all changes real-time. Afterwards my application has to find out what file is whom (i.c. as identified in my application).
The most preferred solution for the users is that the application solves every change itself. Each user-interaction is less preferred.
One of my ideas was to use an attribute of a file and assign a key-value to it so when it has been identified once, it can always been recognized afterwards. But I don't know if there is such an attribute. This article didn't give much hope: There is in Windows file systems a pre computed hash for each file?.
Does somebody know if there is such an attribute I can use? And how can I used it in C#?
Is there anyone who is running up against this problem? And how did you solve it?
I'd like to hear good suggestions.
regards, Jaap
If your files don't leave NTFS, this is easily achievable by alternative data streams, where you can store your data along with files. This is more-or-less good article about ADS: http://www.flexhex.com/docs/articles/alternate-streams.phtml
There is another suitable method - it's very efficient, but also very complicated to use, it requires quite good knowledge about NTFS internals - USN Change Journal; see http://msdn.microsoft.com/en-us/library/windows/desktop/aa363798.aspx. With USN Change Journal, you can "get" very efficiently all files that were changed (even all change events) within specified time period.
Nevertheless, if your files leave NTFS realm, e.g. if it's copied to FAT32, contents of ADS is lost.
Relying on a File attribute is "dangerous" in that some user could alter the attribute while your program isn't running. This could lead you to believe that a certain file is (or isn't) tracked by the program while it really isn't.
I would suggest to keep track of the files in a database, XML, or some other file. When your application starts you read the file/db and check for new/deleted/editted files.
You could store a Hash of the files to find out if a file has been moved/editted. Keeping track of files that are moved AND editted is going to be pretty difficult. (I have no clue how you could achieve it)
PS: Have you considered making your application a Windows service? Having the file-management running in the background no matter if the GUI part of your application is running or not?
Context: A team of operators work with large batch files up to 10GB in size in a third party application. Each file contains thousands of images and after processing every 50 images, they hit the save button. The work place has unreliable power and if the power goes out during a save, the entire file becomes corrupt. To overcome this, I am writing a small utility using the FileSystemWatcher to detect saves and create a backup so that it may be restored without the need to reprocess the entire batch.
Problem: The FileSystemWatcher does a very good job of reporting events but there is a problem I can't pinpoint. Since the monitored files are large in size, the save process takes a few seconds. I want to to be notified once the save operation is complete. I suspect that every time the file buffer is flushed to disk, it triggers an unwanted event. The file remains locked for writing whether or not the a save is in progress so I cannot tell that way.
Creating a backup of the file DURING a save operation defeats the purpose since it corrupts the backed file.
Question:
Is there a way to use the FileSystemWatcher to be notified after the save operation is complete?
If not, how else could I reliably check to see if the file is still being written to?
Alternatives: Any alternative suggestions would be welcome as well.
There's really no direct way to do exactly what you're trying to do. The file system itself doesn't know when a save operation is completed. In logical terms, you may think of it as a series of saves simply because the user clicks the Save button multiple times, but that isn't how the file system sees it. As long as the application has the file locked for writing, as far as the file system is concerned it is still in the process of being saved.
If you think about it, it makes sense. If the application holds onto write access to the file, how would the file system know when the file is in a "corrupt" state and when it's not? Only the application writing the file knows that.
If you have access to the application writing the file, you might be able to solve this problem. Failing that, you might be able to get something with the last modified date, creating a backup only if the file isn't modified for a certain period of time, but that is bound to be buggy and unreliable.
I have come across several "solutions" here and across the web but none seem to fit the bill.
What I am looking to do is have an app monitor a folder for new files (either by creation, a move, or a copy) and perform actions on those objects. That being the scenario, I turned to the FileSystemWatcher class to perform this action.
The problem is that the file FileSystemWatcher.Created event is fired before the entire file is created (most noticeably seen through a copy of a large file).
Is there any way to have this event fire at the conclusion of the file creation as opposed to the beginning? I have tried various combination's of the FileSystemWatcher.NofityFilter property with no success.
Thanks in advance! :)
I have used a couple of solutions for this situation.
If you can work with the creator of the file and use a renaming scheme for the file. EG. Create the File as __Name_ while being created and at the end of the process rename it to Name and the event will fire and you have a complete file.
When your trigger fires check if you can get an exclusive readonly lock on the file. If you can then the write operation has been completed to the file. (I wrote something about this in another question Keep settings in sync between forms application and windows service (or any n-tier, really))
You could possibly integrate something like #2 into your Changed Event and then you'll get the result.
Hmm interesting problem. I never used the object while watching for big files. Did a little searching and seems one solution is to monitor the Changed event as well. Because once the file is done copying (after created is fired) a changed event is thrown as well (cause the file increased in size)
More details from what I read here: http://social.msdn.microsoft.com/forums/en-US/vblanguage/thread/f84bb7c8-b7d5-44da-b0f3-6d1a70415d11/
I know, that what I am going to tell you does not look elegant. I had also to monitor files that arrive from different places, some of them were large and some small. We found out, that FileSystemWatcher is not reliable for this purpose. If you want to be 100% sure, you can check once in a while, using Timer class and its Elapsed event.
You would need to track closing of the file after it's creation and I doubt it's possible with FileSystemWatcher.
If you don't find a solution with FileSystemWatcher, take a look at our CallbackFilter product, which lets you track all operations in real-time.
I need to know which user deleted file in filesystem from c# code.
Only one idea is to use audit, but it seem to be very slow...
You can use the FileSystemWatcher.Deleted event to capture deletes happening on the filesystem.
Depending on the application, you may at that point also be able to find out what user caused this event to occur (it is not part of FileSystemEventArgs).
I don't know if this can be retrieved from the filsystem, but one possible way is to use av FileSystemWatcher object to trigger an event on Deleted. The downside is that you need to have the watcher application running all the time. One upside is that you can monitor just a spesific folder if that's feasible.