We Keep Coding

FileSystemWatcher hangs after a few days

I have a Windows service, that watches a folder for newly created files and copies them to another folder. This folder has high traffic and the events are queued.
The service runs fine but after a few days, it hangs for many hours not doing any job. Later it just starts working again and continues the job, and the accumulated events are then processed. Meanwhile, the service is not crashing.
What went wrong?
The service starts like this:
protected override void OnStart(string[] args)
{
Log.AddLog("starting up");
FolderWatcher folderWatcher = new FolderWatcher();
folderWatcher.Start();
}
This is my file system watcher class:
public class FolderWatcher
{
private FileSystemWatcher watcher;
public void Start()
{
Dispatcher changeDispatcher = null;
ManualResetEvent changeDispatcherStarted = new ManualResetEvent(false);
Action changeThreadHandler = () =>
{
changeDispatcher = Dispatcher.CurrentDispatcher;
changeDispatcherStarted.Set();
Dispatcher.Run();
};
new Thread(() => changeThreadHandler()) { IsBackground = true }.Start();
changeDispatcherStarted.WaitOne();
watcher = new FileSystemWatcher(sourceFilesPath);
watcher.Filter = "*.xml";
watcher.IncludeSubdirectories = true;
watcher.InternalBufferSize = 65536;
watcher.EnableRaisingEvents = true;
watcher.Created += (sender, e) => changeDispatcher.BeginInvoke(new Action(() => OnCreated(sender, e)));
}
public static void OnCreated(Object sender, FileSystemEventArgs e)
{
//process the files here
}
}
thanks

C# Console Application - Inactivity in a folder

I am fairly new to C# coding.. I am trying to setup a code that will alert me when there is an inactivity in a folder. We have a current and archive folder. Once the file is processed in the current folder it will move onto the archive folder.
I have the code to check if there are files in the current folder that's the easy part
DirectoryInfo id = new DirectoryInfo(#"C\");
FileInfo[] TXTFiles = id.GetFiles("*.txt");
if (TXTFiles.Length == 0)
{
Console.WriteLine("Files does not ");
Console.WriteLine("Checking the last processed file in the Archive directory");
Console.Read();
}
if (TXTFiles.Length != 0)
{
Console.WriteLine("Files exists ");
Console.Read();
}
So in the logic where the file does not exist I want to have an additional step to get the timestamp of the last text file that was processed. This is to check for how long there hasnt been any activity .
I am not sure how to proceed. Also instead of writing this information to a console can i send a message to a webservice
Thanks

FileSystemWatcher will be your friend :)
https://msdn.microsoft.com/fr-fr/library/system.io.filesystemwatcher(v=vs.110).aspx
using System;
using System.IO;
using System.Security.Permissions;
public class Watcher
{
public static void Main()
{
Run();
}
[PermissionSet(SecurityAction.Demand, Name="FullTrust")]
public static void Run()
{
string[] args = System.Environment.GetCommandLineArgs();
// If a directory is not specified, exit program.
if(args.Length != 2)
{
// Display the proper way to call the program.
Console.WriteLine("Usage: Watcher.exe (directory)");
return;
}
// Create a new FileSystemWatcher and set its properties.
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = args[1];
/* Watch for changes in LastAccess and LastWrite times, and
the renaming of files or directories. */
watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite
| NotifyFilters.FileName | NotifyFilters.DirectoryName;
// Only watch text files.
watcher.Filter = "*.txt";
// Add event handlers.
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.Created += new FileSystemEventHandler(OnChanged);
watcher.Deleted += new FileSystemEventHandler(OnChanged);
watcher.Renamed += new RenamedEventHandler(OnRenamed);
// Begin watching.
watcher.EnableRaisingEvents = true;
// Wait for the user to quit the program.
Console.WriteLine("Press \'q\' to quit the sample.");
while(Console.Read()!='q');
}
// Define the event handlers.
private static void OnChanged(object source, FileSystemEventArgs e)
{
// Specify what is done when a file is changed, created, or deleted.
Console.WriteLine("File: " + e.FullPath + " " + e.ChangeType);
}
private static void OnRenamed(object source, RenamedEventArgs e)
{
// Specify what is done when a file is renamed.
Console.WriteLine("File: {0} renamed to {1}", e.OldFullPath, e.FullPath);
}
}
Source code from MSDN.

C# continuously watching file that is being recreated

I am trying to continuosly watch a log file that is being deleted and rewritten every it gets updated.
My current approach was to use the FileSystemWatcher. This works great when modifying the file but if I delete the file and make a new one with the same name it stops tracking it.
My current approach:
namespace LogReader
{
class Program
{
static void Main(string[] args)
{
Watch();
while (true)
{
}
}
public static void Watch()
{
var watch = new FileSystemWatcher();
watch.Path = #"C:\TEMP\test";
watch.Filter = "test.txt";
watch.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite;
watch.Changed += new FileSystemEventHandler(OnChanged);
watch.EnableRaisingEvents = true;
}
private static void OnChanged(object source, FileSystemEventArgs e)
{
if (e.FullPath == #"C:\TEMP\test\test.txt")
{
Console.Clear();
Stream stream = File.Open(#"C:\TEMP\test\test.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
StreamReader streamReader = new StreamReader(stream);
var lines = streamReader.ReadToEnd();
Console.Out.WriteLine(lines);
streamReader.Close();
stream.Close();
}
}
}
}

This is because Create and Delete operations will not Trigger OnChanged event of FileSystemWatcher. So you need to register those events and assign the same event handler OnChanged It will be like the following:
watch.Created += new FileSystemEventHandler(OnChanged);
watch.Deleted += new FileSystemEventHandler(OnChanged);
You can go through This For more information regarding FileSystemWatcher.

I was only watching for file changes not file creations. Changing the watch function to this fixed it.
public static void Watch()
{
var watch = new FileSystemWatcher();
watch.Path = #"C:\TEMP\test";
watch.Filter = "test.txt";
watch.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite | NotifyFilters.CreationTime; //more options
watch.Created += new FileSystemEventHandler(OnChanged);
watch.Changed += new FileSystemEventHandler(OnChanged);
watch.EnableRaisingEvents = true;
}

How can I watch for newly created files and wait for them to be unlocked?

First the watch method; I need to watch for any newly created jpg files, since I don't know yet the file names. My program creates each time a new jpg in the directory specified by a TextBox. So my first problem is how to know/get the file name when it's being created?
Second problem, how can I use all these methods, the two methods and the event changed (code below)? I have a button click event when I click it, it will create the new jpg file. Then in the button click event I want to start watching it and give a message on a label something like: "Creating file wait", then when the file is created and ready for use "File created".
private void watch()
{
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = SavePathTextBox.Text;
watcher.NotifyFilter = NotifyFilters.LastWrite;
watcher.Filter = "*.jpg";
watcher.Changed += watcher_Changed;
watcher.EnableRaisingEvents = true;
}
Then the event watcher_Changed:
void watcher_Changed(object sender, FileSystemEventArgs e)
{
}
And the method that checks if the file is locked or not
public static bool IsFileReady(String sFilename)
{
// If the file can be opened for exclusive access it means that the file
// is no longer locked by another process.
try
{
using (FileStream inputStream = File.Open(sFilename, FileMode.Open, FileAccess.Read, FileShare.None))
{
if (inputStream.Length > 0)
{
return true;
}
else
{
return false;
}
}
}
catch (Exception)
{
return false;
}
}
This is what i tried:
In the button click event:
private void TakePhotoButton_Click(object sender, EventArgs e)
{
try
{
if ((string)TvCoBox.SelectedItem == "Bulb") CameraHandler.TakePhoto((uint)BulbUpDo.Value);
else CameraHandler.TakePhoto();
watch();
}
catch (Exception ex) { ReportError(ex.Message, false); }
}
In the watch method:
private void watch()
{
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = SavePathTextBox.Text;
watcher.NotifyFilter = NotifyFilters.LastWrite;
watcher.Filter = "*.JPG";
watcher.Changed += watcher_Changed;
watcher.EnableRaisingEvents = true;
}
The event watcher_Changed
void watcher_Changed(object sender, FileSystemEventArgs e)
{
if (IsFileReady(e.FullPath) == false)
{
this.Invoke((Action)delegate { label6.Text = "Busy"; });
}
else
{
this.Invoke((Action)delegate { label6.Text = "File Ready"; });
}
}
And the method to find if file is locked or not:
public static bool IsFileReady(String sFilename)
{
// If the file can be opened for exclusive access it means that the file
// is no longer locked by another process.
try
{
using (FileStream inputStream = File.Open(sFilename, FileMode.Open, FileAccess.Read, FileShare.None))
{
if (inputStream.Length > 0)
{
return true;
}
else
{
return false;
}
}
}
catch (Exception)
{
return false;
}
}
The problem is that sometimes in most of the cases it's getting to the line inside the watcher_Changed event:
this.Invoke((Action)delegate { label6.Text = "File Ready"; });
And making this line twice or somtimes even 3 times in a row.
I can say that each click my camera take one photo and it's creating two files one for example with the name: IMG_0001.CR2 and the Jpg one: IMG_0001.JPG
But i'm not sure if that's why it's getting to the event and doing the line/s there more then once.
I also checked the file in the e.FullPath is always .jpg and never cr2.
The question is why it's getting there more then once and how can i make sure that the file is really ready ? ("File Ready")
Maybe i need somehow to track the file size from 0kb until the size not change any more and then to decide in the event that it's ready or not ?

I see some problems with the way You use watcher.
Watcher should run before You call CameraHandler.TakePhoto or there is a (small) chance You would miss it.
Either do not create new instance of watcher on every TakePhotoButton_Click or stop the old one. Otherwise You would end up with new running watcher with every click and get as many calls to watcher_Changed as many watchers You have.
I have read there is a chance watcher can be garbage collected. I am not sure if it is true, but rather be safe and save it to some local field.
What You are now waiting for is end of writing to some .jpg file. This may do what You need and in that case it is good. But if You want to wait for file create, You need different setting. This worked for me:
string watchDir = Application.StartupPath;
watcher = new FileSystemWatcher(watchDir, "*.jpg");
watcher.NotifyFilter |= NotifyFilters.Size;
watcher.Created += new FileSystemEventHandler(watcher_Created);
watcher.EnableRaisingEvents = true;
protected void watcher_Created(object sender, FileSystemEventArgs e)
{
string changeType = e.ChangeType.ToString();
if (changeType != "Created")
{
return;
}
// file is created, wait for IsFileReady or whatever You need
}

How to determine whether a folder has finished copying c#

I have a question: How do I determine whether a folder has finished copying from one location to another?
At the moment my FileSystemWatcher triggers several events as soon as a file within the directory being copied. What I want though, is one single event to be triggered when all the files within that folder has been successfully copied. My code right now looks like this:
static void Main(string[] args)
{
String path = #"D:\Music";
FileSystemWatcher mWatcher = new FileSystemWatcher();
mWatcher.Path = path;
mWatcher.NotifyFilter = NotifyFilters.LastAccess;
mWatcher.NotifyFilter = mWatcher.NotifyFilter | NotifyFilters.LastWrite;
mWatcher.NotifyFilter = mWatcher.NotifyFilter | NotifyFilters.DirectoryName;
mWatcher.IncludeSubdirectories = true;
mWatcher.Created += new FileSystemEventHandler(mLastChange);
mWatcher.Changed += new FileSystemEventHandler(mLastChange);
mWatcher.EnableRaisingEvents = true;
Console.WriteLine("Watching path: " + path);
String exit;
while (true)
{
exit = Console.ReadLine();
if (exit == "exit")
break;
}
}
private static void mLastChange(Object sender, FileSystemEventArgs e)
{
Console.WriteLine(e.ChangeType + " " + e.FullPath);
}

Unfortunately FileSystemWatcher doesn't tell you when a file is finished writing. So your options are...
Set a timeout after last write when it is assumed there are no more changes coming
Have the writing application put a lock file of some variety that tells any other program that it's done.
After re-reading your question... it doesn't sound like you have any control over the other application.
So you will need some kind of timeout value that determines when all the writing is done. Basically create a timer that resets after each filesystemwatcher event... when it times out then you fire the single event that says it's done.
Here is how you could add it to your code...
static void Main(string[] args)
{
Timer.Interval = 5000; // 5 seconds - change to whatever is appropriate
Timer.AutoReset = false;
Timer.Elapsed += TimeoutDone;
String path = #"D:\Music";
FileSystemWatcher mWatcher = new FileSystemWatcher();
mWatcher.Path = path;
mWatcher.NotifyFilter = NotifyFilters.LastAccess;
mWatcher.NotifyFilter = mWatcher.NotifyFilter | NotifyFilters.LastWrite;
mWatcher.NotifyFilter = mWatcher.NotifyFilter | NotifyFilters.DirectoryName;
mWatcher.IncludeSubdirectories = true;
mWatcher.Created += new FileSystemEventHandler(mLastChange);
mWatcher.Changed += new FileSystemEventHandler(mLastChange);
mWatcher.EnableRaisingEvents = true;
Console.WriteLine("Watching path: " + path);
Timer.Start();
String exit;
while (true)
{
exit = Console.ReadLine();
if (exit == "exit")
break;
}
}
private static Timer Timer = new Timer();
private static void TimeoutDone(object source, ElapsedEventArgs e)
{
Console.WriteLine("Timer elapsed!");
}
private static void mLastChange(Object sender, FileSystemEventArgs e)
{
Console.WriteLine(e.ChangeType + " " + e.FullPath);
if (Timer != null)
{
Timer.Stop();
Timer.Start();
}
}

It's horribly cheesy, but in the past I have dealt with this problem by creating a custom decorator for the FileSystemWatcher class. Internally it creates a FileSystemWatcher and registers for the Created and Changed events, wraps them, and throws its own Created and Changed events after the files are finished, similar to this:
private void Watcher_Changed(Object sender, FileSystemEVentArgs e)
{
while (true)
{
FileStream stream;
try
{
stream = File.Open(e.FullPath, FileMode.Open, FileAccess.ReadWrite, FileShare.None);
// If this succeeds, the file is finished
Changed();
}
catch (IOException)
{
}
finally
{
if (stream != null) stream.Close();
}
}
}
Some of this is drawn from the answer here. In reality, you shouldn't use an infinite loop. You probably want to add timeouts, sleep in between checks, etc. but this is the general idea.

If the destination is a local folder, you can use the filesystem filter driver to track file create and file close events. Knowing when all files, previously created, are closed will inform you that copying is complete.

I have created a Git repo with a class that extends FileSystemWatcher to trigger the events only when copy is done.
Download FileSystemSafeWatcher and add it to your project.
Then use it as a normal FileSystemWatcher and monitor when the events are triggered.
var fsw = new FileExamSystemWatcher(file);
fsw.EnableRaisingEvents = true;
// Add event handlers here
fsw.Created += fsw_Created;

Unfortunatly there is no ready solution. but I designed a simple tricky solution to trigger (copy finished) event.
you should use timer.
FileSystemWatcher fsw = new FileSystemWatcher();
string fullPath = "";
DateTime tempTime;
fsw.Path = #"C:\temp";
private void startwatching()
{
timer1.Start();
}
fsw.EnableRaisingEvents = true;
fsw.Created += Fsw_Created;
private void Fsw_Created(object sender, FileSystemEventArgs e)
{
tempTime = DateTime.Now.AddSeconds(-4);
fullPath = e.FullPath;
}
private void timer1_Tick(object sender, EventArgs e)
{
if (fullPath!=string.Empty)
{
timer1.Stop();
if (tempTime >= Directory.GetLastAccessTime(fullPath))
{
DirectoryInfo di = new DirectoryInfo(fullPath);
listBox1.Items.Add("Folder " + di.Name + " finished copying");
fullPath = string.Empty;
}
else
{
tempTime = DateTime.Now;
}
timer1.Start();
}
}