I have a FileSystemWatcher that checks multiple directories if there are any files created.
((System.ComponentModel.ISupportInitialize)(FileMonitor)).BeginInit();
FileMonitor.EnableRaisingEvents = true;
FileMonitor.Created += new FileSystemEventHandler(FileMonitor_Created);
FileMonitor.Path = Path.ToString();
FileMonitor.IncludeSubdirectories = true;
FileMonitor.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite | NotifyFilters.FileName | NotifyFilters.DirectoryName | NotifyFilters.Attributes;
((System.ComponentModel.ISupportInitialize)(FileMonitor)).EndInit();
For some reason the FileMonitor_Created event is not always fired when running the application, even though it should. It feels random...
However, if I put a breakpoint at the FileMonitor_Created method, it works perfectly: The event fires everytime it should, if the breakpoint is set.
I've tried setting an InterBufferSize for the FileMonitor, but that had no effect.
Update
I added the Changed event to the Filemonitor and gave it the same handler as for the Created event. Somehow it works now, although the file is actually created, not changed.
I'm still curious why it always worked 'the old way' when setting a breakpoint.
How many changes are you making?
The Windows operating system notifies your component of file changes
in a buffer created by the FileSystemWatcher. If there are many
changes in a short time, the buffer can overflow. This causes the
component to lose track of changes in the directory, and it will only
provide blanket notification. Increasing the size of the buffer with
the InternalBufferSize property is expensive, as it comes from
non-paged memory that cannot be swapped out to disk, so keep the
buffer as small yet large enough to not miss any file change events.
To avoid a buffer overflow, use the NotifyFilter and
IncludeSubdirectories properties so you can filter out unwanted change
notifications.
Taken from MSDN
If you have a breakpoint its working, but if you don't, its not?
Are you sure that there's not something in your event handler? Like there's an exception occurring which makes the program 'feel' like its not doing anything? Can you post the code in the handler?
Separate your business logic from the FileMonitor_Created event.
In the event you should store the event parameters and return.
E.g. store the event parameters in a queue, then process these events async.
FileMonitor.Created fires when the file created and not replaced with the previous file with the same created date time.
Scenario 1) Copy paste and over right the same abc.txt File in input folder without any changes to the file creation date or file content- - File watcher doesn't recognize the file.
Scenario 2) Copy paste and over right the same file in input folder with new created date File watcher recognizes the file
So the created event works with the second scenario, it might not be your situation but looks hidden behavior for my first view.
When event is raised, file processing might take some time. During that time another file might be created and event handler will not handle the second file because it is still handling the first file. Therefore, second file is missed by FileSystemWatcher.
Solution is to separate file detection form file processing into two threads and connect then by a queue. It is producer-consumer queue.
File detection should be as short as can be. It should only detect a file, enqueue its file name in queue that file processing thread can process and close so another file can be detected. File processing thread can dequeue file name and take as much time as it needs to process it.
I explained this in detail with the code in this article: FileSystemWatcher skips some events
Related
I need to monitor a file for changes and see exactly where it has been modified.
It seems like I can't use FileSystemWatcher as I only get notified that a file has been modified but I can't know exactly what changed, and storing copies of the file to do a diff after the event is raised is not feasible since the file is really big.
I would like to implement something like what SysInternals have on their Process Monitor software, that tells us, for a WriteFile event, the modified part of the file given an offset and length.
I read that there is a Windows API that does this, but I couldn't find anything on how or where to start implementing it on a .net 3.5 application.
From what I have investigated, the .NET platform provides this event for handling file change events. However, this alone will not let you know what part of the file has changed.
You will need to store somewhere a copy of the file that is compared to the file whenever it changes, and then update that copy when the comparison is done. This is a very basic solution and can get really inefficient if the file is too large.
Using the FileSystemWatcher you can subscribe to the changed event ChangedEvent
FileSystemEventArgs contains a property FullPath which can be passed in to FileInfo. This will give you some basic file information, which can be stored to a database for example. You can keep a history of the file changes this way.
If you need to compare the content of the files it is more challenging and the approach will be different depending on file type.
I'm currently working on a component of my application which monitors a specific directory for files and keeps them in a DB together with special metadata of those files.
I use a FileSystemWatcher to track additions, renaming, deletion and changes to those files / directories. So far this works perfectly fine. But since FileSystemWatcher is not capable of tracking movements, I'm using hashes to keep track of the files.
Now in order to keep this working I must update the hash in the database once a file was changed. But my first test was proof of failure. Photoshop obviously has some tricky ways to save a file. I always catch a "psXXXXX.tmp" file being changed which I filter with the FileAttributes.Archive. The problem is: The watcher does not throw again for the real changed file. Since I need the real filename and not the temporary one, I can't update the DB.
Even if the file changes in size or the write time changed - nothing happens. It gets completely ignored. The event rises once when the temporary file gets created (or the original the renamed, I don't know) and then nothing happens anymore.
Can anybody explain this behaviour? Has anybody an idea how to avoid this?
Any workarounds or other solutions?
Thanks
I had that problem with PS, I only had to add:
watcher.Renamed += new RenamedEventHandler(OnChanged);
and all was good.
Specific question. I've set up a program that watches for changes to a text file with numbers.
FileSystemWatcher mikeWatcher = new FileSystemWatcher();
System.IO.WaitForChangedResult postMike;
string filePath = #"c:\path.txt";
mikeWatcher.Path = Path.GetDirectoryName(filePath);
mikeWatcher.Filter = Path.GetFileName(filePath);
// Wait for change to the file
postMike = mikeWatcher.WaitForChanged(System.IO.WatcherChangeTypes.Changed);
After this the code continues with the process once the WaitForChanged flag gets complete.
...
The issue I'm having is that while the flag is successfully triggered when I manually change the text file and hit save, when the other program (matlab) writes to and saves to that same file, the flag doesn't seem to be triggered.
I've confirmed that it is indeed the same file being changed (Path is correct) and that the change is registered in the "last modified date". Also, it does appear that the matlab process is closing the text file after saving so its not a release issue, I don't think.
Any thoughts or advice? It'd be easiest for me to change my c# code then the matlab code. All suggestions welcome!
The solution here is to stop using FileSystemWatcher, and instead use a timer to scan the directory for changes.
I have tried to use FileSystemWatcher in two projects and in both cases I found it too unreliable. It would miss some events, delay some events, and report some events twice. I gave up, and haven't used it since.
This answer may not be popular but I'm convinced that the only correct way to use FileSystemWatcher is in comments, like this:
// Tried FileSystemWatcher, but it was far too unreliable.
I think the problem is that it's possible that matlab deletes the file and creates new one with exactly a same name so the file ends up not being changed but newly created. You should trap Deleted and Created events also, not changed only.
Try the code from this example: http://msdn.microsoft.com/en-us/library/t6xf43e0(v=vs.110).aspx
I had the exact same problem and solved it by setting FW.EnableRaisingEvents = true;.
Strange because I don't use the events but it works.
I am trying to use the FileSystemWatcher - and am having some luck..
The goal is to MOVE the file that gets created, from the monitored folder, to a new folder.
But... have hit 2 snags. Firstly, if I move 3 files into a folder at once (Select 3 files, ctrl+x, and then ctrl+c into my Monitor Folder), the monitor only triggers for the first file. The other 2 don't get processed.
FileSystemWatcher fsw = new FileSystemWatcher(FolderToMonitor);
fsw.Created += new FileSystemEventHandler(fsw_Created);
bool monitor = true;
while (monitor)
{
fsw.WaitForChanged(WatcherChangeTypes.All, 2000);
if (Console.KeyAvailable)
{
monitor = false;
}
}
Show("User has quit the process...", ConsoleColor.Yellow);
Console.ReadKey();
Is there a way to make it see all 3?
Secondly, if I move a file into the monitor folder, from a different drive, it takes a few seconds to copy the file into the folder. However, the monitor triggers as soon as the file starts copying in.. so therefore, is read only, and not ready to be moved.
Is there a way I can wait for the file to complete it's copy into the monitor folder, before I process it?
According to the msdn documentation:
The Windows operating system notifies
your component of file changes in a
buffer created by the
FileSystemWatcher. If there are many
changes in a short time, the buffer
can overflow. This causes the
component to lose track of changes in
the directory, and it will only
provide blanket notification.
Increasing the size of the buffer with
the InternalBufferSize property is
expensive, as it comes from non-paged
memory that cannot be swapped out to
disk, so keep the buffer as small yet
large enough to not miss any file
change events. To avoid a buffer
overflow, use the NotifyFilter and
IncludeSubdirectories properties so
you can filter out unwanted change
notifications.
Perhaps that explains your issue?
Also note that cutting and pasting files from one directory to another is basically a mere renaming operation, therefore you should use the Renamed event to detect them.
As for your other problem: try using the Changed event together with Created, as I believe both will be raised exactly once for a file (note that moving a file from another drive in not a simple renaming operation: it's copy and delete), so the Changed event should indicate when the file copy operation has been completed (i.e. it won't fire until the file is complete).
I have come across several "solutions" here and across the web but none seem to fit the bill.
What I am looking to do is have an app monitor a folder for new files (either by creation, a move, or a copy) and perform actions on those objects. That being the scenario, I turned to the FileSystemWatcher class to perform this action.
The problem is that the file FileSystemWatcher.Created event is fired before the entire file is created (most noticeably seen through a copy of a large file).
Is there any way to have this event fire at the conclusion of the file creation as opposed to the beginning? I have tried various combination's of the FileSystemWatcher.NofityFilter property with no success.
Thanks in advance! :)
I have used a couple of solutions for this situation.
If you can work with the creator of the file and use a renaming scheme for the file. EG. Create the File as __Name_ while being created and at the end of the process rename it to Name and the event will fire and you have a complete file.
When your trigger fires check if you can get an exclusive readonly lock on the file. If you can then the write operation has been completed to the file. (I wrote something about this in another question Keep settings in sync between forms application and windows service (or any n-tier, really))
You could possibly integrate something like #2 into your Changed Event and then you'll get the result.
Hmm interesting problem. I never used the object while watching for big files. Did a little searching and seems one solution is to monitor the Changed event as well. Because once the file is done copying (after created is fired) a changed event is thrown as well (cause the file increased in size)
More details from what I read here: http://social.msdn.microsoft.com/forums/en-US/vblanguage/thread/f84bb7c8-b7d5-44da-b0f3-6d1a70415d11/
I know, that what I am going to tell you does not look elegant. I had also to monitor files that arrive from different places, some of them were large and some small. We found out, that FileSystemWatcher is not reliable for this purpose. If you want to be 100% sure, you can check once in a while, using Timer class and its Elapsed event.
You would need to track closing of the file after it's creation and I doubt it's possible with FileSystemWatcher.
If you don't find a solution with FileSystemWatcher, take a look at our CallbackFilter product, which lets you track all operations in real-time.