FileSystemWatcher missing file changes from other program - c#

Specific question. I've set up a program that watches for changes to a text file with numbers.
FileSystemWatcher mikeWatcher = new FileSystemWatcher();
System.IO.WaitForChangedResult postMike;
string filePath = #"c:\path.txt";
mikeWatcher.Path = Path.GetDirectoryName(filePath);
mikeWatcher.Filter = Path.GetFileName(filePath);
// Wait for change to the file
postMike = mikeWatcher.WaitForChanged(System.IO.WatcherChangeTypes.Changed);
After this the code continues with the process once the WaitForChanged flag gets complete.
...
The issue I'm having is that while the flag is successfully triggered when I manually change the text file and hit save, when the other program (matlab) writes to and saves to that same file, the flag doesn't seem to be triggered.
I've confirmed that it is indeed the same file being changed (Path is correct) and that the change is registered in the "last modified date". Also, it does appear that the matlab process is closing the text file after saving so its not a release issue, I don't think.
Any thoughts or advice? It'd be easiest for me to change my c# code then the matlab code. All suggestions welcome!

The solution here is to stop using FileSystemWatcher, and instead use a timer to scan the directory for changes.
I have tried to use FileSystemWatcher in two projects and in both cases I found it too unreliable. It would miss some events, delay some events, and report some events twice. I gave up, and haven't used it since.
This answer may not be popular but I'm convinced that the only correct way to use FileSystemWatcher is in comments, like this:
// Tried FileSystemWatcher, but it was far too unreliable.

I think the problem is that it's possible that matlab deletes the file and creates new one with exactly a same name so the file ends up not being changed but newly created. You should trap Deleted and Created events also, not changed only.
Try the code from this example: http://msdn.microsoft.com/en-us/library/t6xf43e0(v=vs.110).aspx

I had the exact same problem and solved it by setting FW.EnableRaisingEvents = true;.
Strange because I don't use the events but it works.

Related

C# Monitor File Activity

I need to monitor a file for changes and see exactly where it has been modified.
It seems like I can't use FileSystemWatcher as I only get notified that a file has been modified but I can't know exactly what changed, and storing copies of the file to do a diff after the event is raised is not feasible since the file is really big.
I would like to implement something like what SysInternals have on their Process Monitor software, that tells us, for a WriteFile event, the modified part of the file given an offset and length.
I read that there is a Windows API that does this, but I couldn't find anything on how or where to start implementing it on a .net 3.5 application.
From what I have investigated, the .NET platform provides this event for handling file change events. However, this alone will not let you know what part of the file has changed.
You will need to store somewhere a copy of the file that is compared to the file whenever it changes, and then update that copy when the comparison is done. This is a very basic solution and can get really inefficient if the file is too large.
Using the FileSystemWatcher you can subscribe to the changed event ChangedEvent
FileSystemEventArgs contains a property FullPath which can be passed in to FileInfo. This will give you some basic file information, which can be stored to a database for example. You can keep a history of the file changes this way.
If you need to compare the content of the files it is more challenging and the approach will be different depending on file type.

Tracking file changes from Photoshop with FileSystemWatcher

I'm currently working on a component of my application which monitors a specific directory for files and keeps them in a DB together with special metadata of those files.
I use a FileSystemWatcher to track additions, renaming, deletion and changes to those files / directories. So far this works perfectly fine. But since FileSystemWatcher is not capable of tracking movements, I'm using hashes to keep track of the files.
Now in order to keep this working I must update the hash in the database once a file was changed. But my first test was proof of failure. Photoshop obviously has some tricky ways to save a file. I always catch a "psXXXXX.tmp" file being changed which I filter with the FileAttributes.Archive. The problem is: The watcher does not throw again for the real changed file. Since I need the real filename and not the temporary one, I can't update the DB.
Even if the file changes in size or the write time changed - nothing happens. It gets completely ignored. The event rises once when the temporary file gets created (or the original the renamed, I don't know) and then nothing happens anymore.
Can anybody explain this behaviour? Has anybody an idea how to avoid this?
Any workarounds or other solutions?
Thanks
I had that problem with PS, I only had to add:
watcher.Renamed += new RenamedEventHandler(OnChanged);
and all was good.

How to detect new changes in a file?

I am trying to build a logViewer which displays log from multiple files. I want to display the new changes in the files as soon as they are modified. I am using a FileSystemWatcher to detect if a file has been changed. But I am not sure how do I detect the change that has been made in the file that I am monitoring. I have seen some of the questions in StackOverflow but they are not helpful. How do I do this?
Edit:
All I want is to get the line/lines that have been added. If there are any method other than using FileSystemWatcher I am fine with that also.
Use the filesystemwatcher to detect changes and get new lines using last read position and seek the file.
https://stackoverflow.com/a/19230599/1583653
If you are looking to do this programmatically you will need to have both the previous version and the current version. Then you will have to make a comparison somehow: letter by letter, word by word, etc, and then go from there.
FileSystemWatcher only detects if the change has happened, it doesn't detect the change itself.
Besides FileSystemWatcher your task can be solved (quite efficiently) using a filesystem filter driver. With such driver you'll know about the changes right when (before or after) they are made, and you know what exactly is done.
You can write a filter yourself if you have experience with kernel-mode development or use our CallbackFilter product, though the latter is an overkill for your needs.

FileSystemWatcher skips Created-events

I have a FileSystemWatcher that checks multiple directories if there are any files created.
((System.ComponentModel.ISupportInitialize)(FileMonitor)).BeginInit();
FileMonitor.EnableRaisingEvents = true;
FileMonitor.Created += new FileSystemEventHandler(FileMonitor_Created);
FileMonitor.Path = Path.ToString();
FileMonitor.IncludeSubdirectories = true;
FileMonitor.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite | NotifyFilters.FileName | NotifyFilters.DirectoryName | NotifyFilters.Attributes;
((System.ComponentModel.ISupportInitialize)(FileMonitor)).EndInit();
For some reason the FileMonitor_Created event is not always fired when running the application, even though it should. It feels random...
However, if I put a breakpoint at the FileMonitor_Created method, it works perfectly: The event fires everytime it should, if the breakpoint is set.
I've tried setting an InterBufferSize for the FileMonitor, but that had no effect.
Update
I added the Changed event to the Filemonitor and gave it the same handler as for the Created event. Somehow it works now, although the file is actually created, not changed.
I'm still curious why it always worked 'the old way' when setting a breakpoint.
How many changes are you making?
The Windows operating system notifies your component of file changes
in a buffer created by the FileSystemWatcher. If there are many
changes in a short time, the buffer can overflow. This causes the
component to lose track of changes in the directory, and it will only
provide blanket notification. Increasing the size of the buffer with
the InternalBufferSize property is expensive, as it comes from
non-paged memory that cannot be swapped out to disk, so keep the
buffer as small yet large enough to not miss any file change events.
To avoid a buffer overflow, use the NotifyFilter and
IncludeSubdirectories properties so you can filter out unwanted change
notifications.
Taken from MSDN
If you have a breakpoint its working, but if you don't, its not?
Are you sure that there's not something in your event handler? Like there's an exception occurring which makes the program 'feel' like its not doing anything? Can you post the code in the handler?
Separate your business logic from the FileMonitor_Created event.
In the event you should store the event parameters and return.
E.g. store the event parameters in a queue, then process these events async.
FileMonitor.Created fires when the file created and not replaced with the previous file with the same created date time.
Scenario 1) Copy paste and over right the same abc.txt File in input folder without any changes to the file creation date or file content- - File watcher doesn't recognize the file.
Scenario 2) Copy paste and over right the same file in input folder with new created date File watcher recognizes the file
So the created event works with the second scenario, it might not be your situation but looks hidden behavior for my first view.
When event is raised, file processing might take some time. During that time another file might be created and event handler will not handle the second file because it is still handling the first file. Therefore, second file is missed by FileSystemWatcher.
Solution is to separate file detection form file processing into two threads and connect then by a queue. It is producer-consumer queue.
File detection should be as short as can be. It should only detect a file, enqueue its file name in queue that file processing thread can process and close so another file can be detected. File processing thread can dequeue file name and take as much time as it needs to process it.
I explained this in detail with the code in this article: FileSystemWatcher skips some events

C#: Monitoring copied or moved files with FileSystemWatcher

I have come across several "solutions" here and across the web but none seem to fit the bill.
What I am looking to do is have an app monitor a folder for new files (either by creation, a move, or a copy) and perform actions on those objects. That being the scenario, I turned to the FileSystemWatcher class to perform this action.
The problem is that the file FileSystemWatcher.Created event is fired before the entire file is created (most noticeably seen through a copy of a large file).
Is there any way to have this event fire at the conclusion of the file creation as opposed to the beginning? I have tried various combination's of the FileSystemWatcher.NofityFilter property with no success.
Thanks in advance! :)
I have used a couple of solutions for this situation.
If you can work with the creator of the file and use a renaming scheme for the file. EG. Create the File as __Name_ while being created and at the end of the process rename it to Name and the event will fire and you have a complete file.
When your trigger fires check if you can get an exclusive readonly lock on the file. If you can then the write operation has been completed to the file. (I wrote something about this in another question Keep settings in sync between forms application and windows service (or any n-tier, really))
You could possibly integrate something like #2 into your Changed Event and then you'll get the result.
Hmm interesting problem. I never used the object while watching for big files. Did a little searching and seems one solution is to monitor the Changed event as well. Because once the file is done copying (after created is fired) a changed event is thrown as well (cause the file increased in size)
More details from what I read here: http://social.msdn.microsoft.com/forums/en-US/vblanguage/thread/f84bb7c8-b7d5-44da-b0f3-6d1a70415d11/
I know, that what I am going to tell you does not look elegant. I had also to monitor files that arrive from different places, some of them were large and some small. We found out, that FileSystemWatcher is not reliable for this purpose. If you want to be 100% sure, you can check once in a while, using Timer class and its Elapsed event.
You would need to track closing of the file after it's creation and I doubt it's possible with FileSystemWatcher.
If you don't find a solution with FileSystemWatcher, take a look at our CallbackFilter product, which lets you track all operations in real-time.

Categories