We Keep Coding

C# and the FileSystemWatcher

i have written a service in C# which should move backup files (*.bak and *.trn) from a database server to a special backup server. This works quite well so far. The problem is that it tries to move single files twice. This fails of course. I have configured FileSystemWatcher as follows:
try
{
m_objWatcher = new FileSystemWatcher();
m_objWatcher.Filter = m_strFilter;
m_objWatcher.Path = m_strSourcepath.Substring(0, m_strSourcepath.Length - 1);
m_objWatcher.IncludeSubdirectories = m_bolIncludeSubdirectories;
m_objWatcher.NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.LastAccess; // | NotifyFilters.CreationTime;
m_objWatcher.Changed += new FileSystemEventHandler(objWatcher_OnCreated);
}
catch (Exception ex)
{
m_objLogger.d(TAG, m_strWatchername + "InitFileWatcher(): " + ex.ToString());
}
Is it possible that the Watcher produces an event twice for the same file? If I set the filter to CreationTime only, it does not react at all.
How do I have to set the Watcher to fire an event only once per file?
Thanks in advance for your help

The documentation states that common file system operations might raise more than one event. Check under the Events and Buffer Sizes heading.
Common file system operations might raise more than one event. For example, when a file is moved from one directory to another, several OnChanged and some OnCreated and OnDeleted events might be raised. Moving a file is a complex operation that consists of multiple simple operations, therefore raising multiple events. Likewise, some applications (for example, antivirus software) might cause additional file system events that are detected by FileSystemWatcher.
It also offers a few guidelines, including:
Keep your event handling code as short as possible.
To that end, you could use your FileSystemWatcher.Changed event to queue files for processing, then process them later. This is a quick example of what that might look like using an instance of System.Threading.Timer to process the queue.
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
public class ServiceClass
{
public ServiceClass()
{
_processing = false;
_fileQueue = new ConcurrentQueue<string>();
_timer = new System.Threading.Timer(ProcessQueue);
// Schedule the time to run in 5 seconds, then again every 5 seconds.
_timer.Change(5000, 5000);
}
private void objWatcher_OnChanged(object sender, FileSystemEventArgs e)
{
// Just queue the file to be processed later. If the same file is added multiple
// times, we'll skip the duplicates when processing the files.
_fileQueue.Enqueue(e.FilePath);
}
private void ProcessQueue(object state)
{
if (_processing)
{
return;
}
_processing = true;
var failures = new HashSet<string>();
try
{
while (_fileQueue.TryDequeue(out string fileToProcess))
{
if (!File.Exists(fileToProcess))
{
// Probably a file that was added multiple times and it was
// already processed.
continue;
}
var file = new FileInfo(fileToProcess);
if (FileIsLocked(file))
{
// File is locked. Maybe you got the Changed event, but the file
// wasn't done being written.
failures.Add(fileToProcess);
continue;
}
try
{
fileInfo.MoveTo(/*Your destination*/);
}
catch (Exception)
{
// File failed to move. Add it to the failures so it can be tried
// again.
failutes.Add(fileToProcess);
}
}
}
finally
{
// Add any failures back to the queue to try again.
foreach (var failedFile in failures)
{
_fileQueue.Enqueue(failedFile);
}
_processing = false;
}
}
private bool IsFileLocked(FileInfo file)
{
try
{
using (FileStream stream = file.Open(FileMode.Open, FileAccess.Read,
FileShare.None))
{
stream.Close();
}
}
catch (IOException)
{
return true;
}
return false;
}
private System.Threading.Timer _timer;
private bool _processing;
private ConcurrentQueue<string> _fileQueue;
}
Credit where it's due, I took FileIsLocked from this answer.
Some other things you might need to consider:
What happens if your FileSystemWatcher misses an event? [The documentation] does state that it is possible.
Note that a FileSystemWatcher may miss an event when the buffer size is exceeded. To avoid missing events, follow these guidelines:
Increase the buffer size by setting the InternalBufferSize property.
Avoid watching files with long file names, because a long file name contributes to filling up the buffer. Consider renaming these files using shorter names.
Keep your event handling code as short as possible.
What happens if your service crashes, but the process writing backup files continues to write them? When you restart your service, will it pick those files up and move them?

I tried all sorts of ideas to stop this. The events are too close together... it can't be stopped in the FileChanged Event. Here is my working solution:
private System.Timers.Timer timer;
private FileSystemWatcher fwatcher;
static void Main(string[] args)
{
new Program();
}
private Program()
{
timer = new System.Timers.Timer(100);
timer.Elapsed += new ElapsedEventHandler(OnTimedEvent);
timer.AutoReset = false; // only once
fwatcher = new FileSystemWatcher();
fwatcher.Path = filePath;
fwatcher.Filter = fileName;
fwatcher.NotifyFilter = NotifyFilters.LastWrite;
fwatcher.Changed += new FileSystemEventHandler(FileChanged);
fwatcher.EnableRaisingEvents = true;
while (IsRunning)
{
Thread.Sleep(100);
}
Thread.Sleep(100);
}
private void FileChanged(object sender, FileSystemEventArgs e)
{
timer.Start();
}
private void OnTimedEvent(object source, ElapsedEventArgs e)
{
Console.WriteLine("file has changed!");
}
The timer will only fire once each time the file is changed.

FileSystemWatcher - Reading last line of file after update?

I'm trying to write a simple console application that waits for a change to a file then reads just the last line of that file. The file watcher works and the change event fires. But I'm struggling to work out how read from the file.
static void Main(string[] args)
{
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.NotifyFilter = NotifyFilters.LastWrite;
watcher.Path = "E:\\myFilePath";
watcher.Filter = "";
watcher.EnableRaisingEvents = true;
watcher.Changed += new FileSystemEventHandler(watcher_Changed);
Console.ReadKey();
}
static void watcher_Changed(object sender, FileSystemEventArgs e)
{
Console.WriteLine(File.ReadLines(e.FullPath).Last());
}
when testing (editing the file in notepad) it will work once, then after a second edit I get the error...
System.IO.IOException: 'The process cannot access the file '' because
it is being used by another process.'

I still ran into problems using lines.GetEnumerator().Dispose();
You can add a pause before opening the file, just chose your poison for doing so.
static void watcher_Changed(object sender, FileSystemEventArgs e)
{
for (int x = 0; x <= 500000; x++)
{
int t = x;
}
Console.WriteLine(File.ReadLines(e.FullPath).Last());
}
I tried using SreamReader too but still ran into the same problem.
// Does not work
using (StreamReader r = new StreamReader(e.FullPath))
{
while (r.EndOfStream == false)
{
m = r.ReadLine();
}
r.Close();
}
Console.WriteLine("{0}\n", m);

The problem is due to an already open handle to the file. File.ReadLines uses an iterator internally which is Disposable. Use this to ensure Disposal during the lifetime of your event handler.
static void watcher_Changed(object sender, FileSystemEventArgs e)
{
var lines = File.ReadLines(e.FullPath);
Console.WriteLine(lines.Last());
lines.GetEnumerator().Dispose();
}
This should guarantee disposal before the end of event handler.
Edit: In the case that a different process is holding a write lock to the file, use Thread.Sleep(ms) with a safe time as inspecting and releasing file handles is not feasible in C#.

FileSystemWatcher not firing events when file written using fprintf or printf

Yes there are many similar questions but none of them address my unique situation.
There is a separate c++ process writing the file using c++ printf and fprintf.
filename i am trying to watch is info_20160525.log
My fileSystemWatcher in winform C# application gets notification when the writer process writes to the file AND I physically access the file i-e F5 the folder or have it open in textpad and click the opened file or right click the file but I never get any event notification when I dont physically interact with the file.
Also, when I shutdown the writer application I do get the notification.
Here is my code.
public bool StartUp(string fullfilepath, int linenumber)
{
if (!File.Exists(fullfilepath))
return false;
if (!LogClass.CheckPathExists(m_applicationPath))
return false;
try
{
FileInfo info = new FileInfo(fullfilepath);
m_filename = fullfilepath;
m_fileWatcher = new FileSystemWatcher(info.DirectoryName, info.Name);
m_fileWatcher.NotifyFilter = NotifyFilters.Attributes | NotifyFilters.LastAccess
| NotifyFilters.LastWrite | NotifyFilters.Size ;
m_fileWatcher.Changed += m_fileWatcher_Changed;
m_fileWatcher.Error += m_fileWatcher_Error;
m_fileWatcher.Created += m_fileWatcher_Created;
m_fileWatcher.Deleted += m_fileWatcher_Deleted;
m_fileWatcher.Disposed += m_fileWatcher_Disposed;
m_fileWatcher.Renamed += m_fileWatcher_Renamed;
m_linesRead = linenumber;
m_fileWatcher.EnableRaisingEvents = true;
}
catch(Exception e)
{
LogClass.LogError(e, "Trouble accessing the file" + fullfilepath, m_applicationPath);
}
return true;
}
These are the handlers. I have breakpoints in each one of them but I never get a trigger unless ofcourse I physically interact with the file.
void m_fileWatcher_Renamed(object sender, RenamedEventArgs e)
{
string S = "";
}
void m_fileWatcher_Disposed(object sender, EventArgs e)
{
string S = "";
}
void m_fileWatcher_Deleted(object sender, FileSystemEventArgs e)
{
string S = "";
}
void m_fileWatcher_Created(object sender, FileSystemEventArgs e)
{
string S = "";
}
void m_fileWatcher_Error(object sender, ErrorEventArgs e)
{
string S = "";
}
void m_fileWatcher_Changed(object sender, FileSystemEventArgs args)
{
if (args.ChangeType == WatcherChangeTypes.Changed)
{
while (ParseFile(args.FullPath))
{
}
}
}

I bet this thread has your answer --> FileSystemWatcher changed event (for “LastWrite”) is unreliable
The FileSystemWatcher uses an update to the LastWrite attribute of a file to fire events, however, the LastWrite is not updated in real time and should not be relied upon as trigger for an event.
If you have enough time and resources on your hands then you probably want to look into File System Filters and the simpler approach of a Filter Manager and Minifilter Driver. It is driver type development, however, it is a sure file way to accomplish your objective.
It is dug down a little deeper by system policy but gives you a wide array of events to latch onto. If I was doing this for anything like pci compliance or similar tasks then I would not use the FileSystemWatcher.

Make sure you set IncludeSubdirectories to true.
https://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher(v=vs.110).aspx

FileSystemWatcher Raising Multiple Events

I'm going to start off by apologizing if this is a duplicated question. There were a ton of FileSystemWatcher questions but I didn't see any that addressed my issue.
Ok, so I have a console application in C# that is monitoring a directory, which we will call Root. Root has many subfolders. The purpose of this application is to write to a log file if any .csv files are create, modified, or deleted within Root or any of its subfolders. I currently have this working fine, sort of. The only problem is that when a .csv file is created, modified, or deleted it actually raises the event for all 3.
For example, if I create a file in Root called test.csv, the log file would look like this:
10/04/2012: File F:/Root/test.csv Created
10/04/2012: File F:/Root/test.csv Changed
10/04/2012: File F:/Root/test.csv Created
10/04/2012: File F:/Root/test.csv Deleted
I'm not sure whats going on, so here is the code that sets up the FileSystemWatcher
_watchFolder.Path = ConfigurationManager.AppSettings["RootToWatch"];
_watchFolder.Filter = ConfigurationManager.AppSettings["FileNameToWatch"];
_watchFolder.NotifyFilter = NotifyFilters.FileName
| NotifyFilters.LastWrite;
_watchFolder.IncludeSubdirectories = true;
_watchFolder.Changed += new FileSystemEventHandler(OnChanged);
_watchFolder.Created += new FileSystemEventHandler(OnChanged);
_watchFolder.Deleted += new FileSystemEventHandler(OnChanged);
_watchFolder.Renamed += new RenamedEventHandler(OnRenamed);
try
{
_watchFolder.EnableRaisingEvents = true;
}
catch (ArgumentException ex)
{
AbortMonitoring(ex.Message);
}
And here is my OnChanged event (Renamed is same but has different argument)
protected static void OnChanged(object sender, FileSystemEventArgs e)
{
//compile message to insert into log file.
string message = "File: " + e.FullPath + " " + e.ChangeType;
UpdateLogFile(message);
}

Just add a handler only for the changed event. It should gather all change types. The other types of events like created are there if you want the event to be raised only on a particular change type.
watchFolder.IncludeSubdirectories = true;
_watchFolder.Changed += new FileSystemEventHandler(OnChanged);

FileSystemWatcher, unsubscribe from the event

I'm fooling around with the FileSystemWatcher in 4.0. I find this very useful but am getting caught in a loop. I'm trying to monitor whenever an ini is changed and change it back to the correct default (long story) however the change event copying over the new file is causing it to drop into a loop ... Any Ideas > ? I played around with the idea of deleting and recreating thefile to avoid triggering the changed event but this leads to another set of issues with the program that I'd rather avoid. Also I'd imagine I could overwrite the text but this also poses the same issue. Thanks in advance for the help
static void Main() { Watch (#"\\NoFault2010\Lexis\Data\Setup\", "tmconfig.ini", true); }
static void Watch (string path, string filter, bool includeSubDirs)
{
using (var watcher = new FileSystemWatcher (path, filter))
{
watcher.Changed += FileChanged;
watcher.EnableRaisingEvents = true;
Console.WriteLine("Do Not Close ... \n\nThis is a Temporary Configuration Manager for Time Matters ... \n\n\nI'm Listening ............");
Console.ReadLine();
}
}
static void FileChanged (object o, FileSystemEventArgs e)
{
string _right_stuff = #"\\NOFAULT2010\Lexis\Data\Templates\Programs\tmconfig.ini";
string _working = #"\\NOFAULT2010\Lexis\Data\Setup\tmconfig.ini";
System.Threading.Thread.Sleep(2000);
File.Copy(_right_stuff, _working, true);
Console.WriteLine("File {0} has been {1}", e.FullPath, e.ChangeType);
MAIL_IT("SQLMail#lcjlawfirm.com", "TM Master.INI has been altered", "Check the Master INI and Yell At Ecopy Guy " + e.ChangeType + e.FullPath);
}
How would I unsubscribe from the event to avoid entering into this loop.

To temporarily disable the event while you're fiddling with the file yourself:
static void FileChanged (object o, FileSystemEventArgs e)
{
watcher.Changed -= FileChanged;
... correct the file here...
watcher.Changed += FileChanged;
}
Alternatively, you can use a guard variable to detect reentrant calls:
static bool reentrant = false;
static void FileChanged (object o, FileSystemEventArgs e)
{
if (reentrant)
return;
reentrant = true;
... correct the file here...
reentrant = false;
}
Note that you will also want to do exception handling within the method or your file watcher may become permanently disabled if a problem occurs.

I've written an application that depends on filesystemwatcher - and also, sometimes the fsw handler makes a change to a file.
I approached it in two ways - the first was to take the view that my code would be very quick in changing the file - so I did
fsw.EnableRaisingEvents = false;
//make my change
fsw.EnableRaisingEvents = true;
However, if you feel that other files might get changed during that time, you could log the time that you make the change and store that data somewhere...
say, Dictionary mapFileNameTimeChanged ...here you could store the file name...so in your handler you could do something like....
fsw_Changed(object sender, FileSystemEventArgs e)
{
lock (m_mapFileNameChanged)
{
if (m_mapFileNameChanged.ContainsKey(e.FullPath))
{
FileInfo fileInfo = new FileInfo(e.FullPath);
if (fileInfo.LastAccessTime == m_mapFileNameChanged[e.FullPath]
{
return;//not been changed since you last did something with it....
}
}
else
{
m_mapFileNameChanged.Remove(e.FullPath);//discard this now..it has changed since you last looked at it...need to look at it again!
}
}
//do things in your event handler...
lock (m_mapFileNameChanged)
{
// copy or change the file here...
FileInfo fileInfo = new FileInfo(e.FullPath);
m_mapFileNameChanged[strFullPathToFile] = fileInfo.LastAccessTime;
}
}

You could add a boolean (again at the class level) that you could use to track whether the changes were caused by you, and if so, just immediately exit your FileChanged method, ie:
static bool inEdit;
static void FileChanged (object o, FileSystemEventArgs e)
{
if (inEdit)
return;
inEdit = true;
// Do processing
inEdit = false;
}

Unsubscribe is easy, so I wonder if that was the question:
watcher.Changed -= FileChanged
Also, I would create some object to be SynchronizationObject for watcher. There is a problem that by default watcher raises events in new thread, and thus if you unsubscribe after new thread is created, you might run into the problems.
Also of note that FileSystemWatcher may raises multiple events for something you consider as single event, and it might influence functioning of your program.

If you make the watcher a class variable instead of a local variable, then your FileChanged method should be able to access it. Then you should be able to do something like
static void FileChanged (object o, FileSystemEventArgs e)
{
watcher.EnableRaisingEvents = false;
// Edit the file here
watcher.EnableRaisingEvents = true;
}