I am trying to use FileSystemWatcher to read a textfile as soon as anything gets updated into textfile in Windows Service.Now the problem that i am facing is not getting the way where i should put my FileSystemWatcher code so that i would get called as soon as textfile gets changed.Do i need to add this into OnStart() Method of Windows Service or anywhere else.
Here is my Code Structure..
protected override void OnStart(string[] args)
{
_thread = new Thread(startReadingTextFile);
_thread.Start();
}
public void startReadingTextFile() {
_freader = new AddedContentReader(TextFileLocation);
}
private void Watcher_Changed(object sender, FileSystemEventArgs e)
{
string addedContent = _freader.GetAddedLines();
}
Please help me .Thanks ..
Updated Code..
protected override void OnStart(string[] args)
{
if (lastLineReadOffset == 0)
{
_freader = new AddedContentReader(TextFileLocation);
}
//If you have saved the last position when the application did exit then you can use that value here to start from that location like the following
//_freader = new AddedContentReader("E:\\tmp\\test.txt",lastReadPosition);
else
{
_freader = new AddedContentReader(TextFileLocation, lastLineReadOffset);
}
FileSystemWatcher Watcher = new FileSystemWatcher("C:\\temp");
Watcher.EnableRaisingEvents = true;
Watcher.Changed += new FileSystemEventHandler(Watcher_Changed);
}
private void Watcher_Changed(object sender, FileSystemEventArgs e)
{
string addedContent = _freader.GetAddedLines();
//you can do whatever you want with the lines
using (StringReader reader = new StringReader(addedContent))
{
string line;
while ((line = reader.ReadLine()) != null)
{
// Call the Processing Function
}
}
}
Do i need to add this into OnStart()
Yes.
But, there is no need to create a thread for this purpose. Once FileSystemWatcher.EnableRaisingEvents is set, then events will be fired in the thread pool: you can return from OnStart.
Richard's answer is correct. However, the Changed event fires at least twice because by default FileSystemWatcher fires it once when the file is created and then again every time the file system flushes its contents to the disk. For large files you may get multiple Change events caused by multiple disk writes. If you try to open the file the first time Change fires you may get errors if the file is locked by the writing process or get an incomplete file content.
The most reliable way I have discovered is to set a timer with a short interval (a few seconds) on the very first Change event for a new file and then reset it every subsequent time the event fires for the same file. You then open the file in the timer's own Elapsed event when it does fire a few seconds after the last Change event was fired for the file.
This requires some extra code and variables:
First, create a Dictionary<string, Timer> to keep track of timers per filename.
Inside your Change event handler you need to check if the dictionary already contains the file's name as a key (inside a lock block to take care of thread concurrency issues).
If it isn't then:
create a new Timer instance
set its state object to the name of the file so when its Elapsed event fires you'll know which file you are supposed to process (the same end result can also be achieved using a closure and a lambda function but a state object is simpler)
add the new timer instance to the dictionary using the file name as the key
If it is (i.e. this is not the first Change event for that file):
look up the Timer instance in the dictionary
reset its interval to push its Elapsed event further
Then in the handler of the timer's Elapsed event you do your actual processing and cleanup:
get the file name from the timer's state object passed in the event arguments
look up the Timer instance in the dictionary by the file name and dispose of it
remove the timer from the dictionary, i.e. Remove(key) where key is the file name (the three actions above should happen inside a lock block)
open the file and do whatever you need with it.
Here's how you might want to implement this logic inside your service:
const int DELAY = 2000; // milliseconds
const WatcherChangeTypes FILE_EVENTS = WatcherChangeTypes.Created | WatcherChangeTypes.Changed | WatcherChangeTypes.Renamed;
FileSystemWatcher _fsw;
Dictionary<string, Timer> _timers = new Dictionary<string, Timer>();
object _lock = new object();
public void Start()
{
_fsw = new FileSystemWatcher(Directory, FileFilter)
{
IncludeSubdirectories = false,
EnableRaisingEvents = true
};
_fsw.Created += OnFileChanged;
_fsw.Changed += OnFileChanged;
}
private void OnFileChanged(object sender, FileSystemEventArgs e)
{
try
{
// When a file is created in the monitored directory, set a timer to process it after a short
// delay and add the timer to the queue.
if (FILE_EVENTS.HasFlag(e.ChangeType))
{
lock (_lock)
{
// File events may fire multiple times as the file is being written to the disk and/or renamed,
// therefore the first time we create a new timer and then reset it on subsequent events so that
// the file is processed shortly after the last event fires.
if (_timers.TryGetValue(e.FullPath, out Timer timer))
{
timer.Change(DELAY, 0);
}
else
{
_timers.Add(e.FullPath, new Timer(OnTimerElapsed, e.FullPath, DELAY, 0));
}
}
}
}
catch (Exception ex)
{
// handle errors
}
}
private void OnTimerElapsed(object state)
{
var fileName = (string)state;
lock (_lock)
{
try { _timers[fileName].Dispose(); } catch { }
try { _timers.Remove(fileName); } catch { }
}
// open the file ...
}
Related
i have written a service in C# which should move backup files (*.bak and *.trn) from a database server to a special backup server. This works quite well so far. The problem is that it tries to move single files twice. This fails of course. I have configured FileSystemWatcher as follows:
try
{
m_objWatcher = new FileSystemWatcher();
m_objWatcher.Filter = m_strFilter;
m_objWatcher.Path = m_strSourcepath.Substring(0, m_strSourcepath.Length - 1);
m_objWatcher.IncludeSubdirectories = m_bolIncludeSubdirectories;
m_objWatcher.NotifyFilter = NotifyFilters.LastWrite | NotifyFilters.LastAccess; // | NotifyFilters.CreationTime;
m_objWatcher.Changed += new FileSystemEventHandler(objWatcher_OnCreated);
}
catch (Exception ex)
{
m_objLogger.d(TAG, m_strWatchername + "InitFileWatcher(): " + ex.ToString());
}
Is it possible that the Watcher produces an event twice for the same file? If I set the filter to CreationTime only, it does not react at all.
How do I have to set the Watcher to fire an event only once per file?
Thanks in advance for your help
The documentation states that common file system operations might raise more than one event. Check under the Events and Buffer Sizes heading.
Common file system operations might raise more than one event. For example, when a file is moved from one directory to another, several OnChanged and some OnCreated and OnDeleted events might be raised. Moving a file is a complex operation that consists of multiple simple operations, therefore raising multiple events. Likewise, some applications (for example, antivirus software) might cause additional file system events that are detected by FileSystemWatcher.
It also offers a few guidelines, including:
Keep your event handling code as short as possible.
To that end, you could use your FileSystemWatcher.Changed event to queue files for processing, then process them later. This is a quick example of what that might look like using an instance of System.Threading.Timer to process the queue.
using System;
using System.Collections.Concurrent;
using System.Collections.Generic;
using System.IO;
public class ServiceClass
{
public ServiceClass()
{
_processing = false;
_fileQueue = new ConcurrentQueue<string>();
_timer = new System.Threading.Timer(ProcessQueue);
// Schedule the time to run in 5 seconds, then again every 5 seconds.
_timer.Change(5000, 5000);
}
private void objWatcher_OnChanged(object sender, FileSystemEventArgs e)
{
// Just queue the file to be processed later. If the same file is added multiple
// times, we'll skip the duplicates when processing the files.
_fileQueue.Enqueue(e.FilePath);
}
private void ProcessQueue(object state)
{
if (_processing)
{
return;
}
_processing = true;
var failures = new HashSet<string>();
try
{
while (_fileQueue.TryDequeue(out string fileToProcess))
{
if (!File.Exists(fileToProcess))
{
// Probably a file that was added multiple times and it was
// already processed.
continue;
}
var file = new FileInfo(fileToProcess);
if (FileIsLocked(file))
{
// File is locked. Maybe you got the Changed event, but the file
// wasn't done being written.
failures.Add(fileToProcess);
continue;
}
try
{
fileInfo.MoveTo(/*Your destination*/);
}
catch (Exception)
{
// File failed to move. Add it to the failures so it can be tried
// again.
failutes.Add(fileToProcess);
}
}
}
finally
{
// Add any failures back to the queue to try again.
foreach (var failedFile in failures)
{
_fileQueue.Enqueue(failedFile);
}
_processing = false;
}
}
private bool IsFileLocked(FileInfo file)
{
try
{
using (FileStream stream = file.Open(FileMode.Open, FileAccess.Read,
FileShare.None))
{
stream.Close();
}
}
catch (IOException)
{
return true;
}
return false;
}
private System.Threading.Timer _timer;
private bool _processing;
private ConcurrentQueue<string> _fileQueue;
}
Credit where it's due, I took FileIsLocked from this answer.
Some other things you might need to consider:
What happens if your FileSystemWatcher misses an event? [The documentation] does state that it is possible.
Note that a FileSystemWatcher may miss an event when the buffer size is exceeded. To avoid missing events, follow these guidelines:
Increase the buffer size by setting the InternalBufferSize property.
Avoid watching files with long file names, because a long file name contributes to filling up the buffer. Consider renaming these files using shorter names.
Keep your event handling code as short as possible.
What happens if your service crashes, but the process writing backup files continues to write them? When you restart your service, will it pick those files up and move them?
I tried all sorts of ideas to stop this. The events are too close together... it can't be stopped in the FileChanged Event. Here is my working solution:
private System.Timers.Timer timer;
private FileSystemWatcher fwatcher;
static void Main(string[] args)
{
new Program();
}
private Program()
{
timer = new System.Timers.Timer(100);
timer.Elapsed += new ElapsedEventHandler(OnTimedEvent);
timer.AutoReset = false; // only once
fwatcher = new FileSystemWatcher();
fwatcher.Path = filePath;
fwatcher.Filter = fileName;
fwatcher.NotifyFilter = NotifyFilters.LastWrite;
fwatcher.Changed += new FileSystemEventHandler(FileChanged);
fwatcher.EnableRaisingEvents = true;
while (IsRunning)
{
Thread.Sleep(100);
}
Thread.Sleep(100);
}
private void FileChanged(object sender, FileSystemEventArgs e)
{
timer.Start();
}
private void OnTimedEvent(object source, ElapsedEventArgs e)
{
Console.WriteLine("file has changed!");
}
The timer will only fire once each time the file is changed.
I am currently trying to keep a counter on C# on a local file folder for new files that are created.
I have two sub directories to CD and LP that I have to keep checking. With counters that makes sure that the count of folders made have not exceeded the count set by the user.
public static int LPmax { get; set; }
public static int CDmax { get; set; }
public static int LPcounter2 { get; set; }
public static int CDcounter2 { get; set; }
public static int LPCreated;
public static int CDCreated;
public static int oldLPCreated;
public static int oldCDCreated;
FileSystemWatcher CDdirWatcher = new FileSystemWatcher();
FileSystemWatcher LPdirWatcher = new FileSystemWatcher();
//watch method should run in the background as checker
public void watch()
{
CDdirWatcher.Path = #"C:\Data\LotData\CD";
CDdirWatcher.Filter = "EM*";
CDdirWatcher.NotifyFilter = NotifyFilters.DirectoryName | NotifyFilters.LastWrite;
CDdirWatcher.EnableRaisingEvents = true;
CDdirWatcher.Created += CDdirWatcher_Created;
LPdirWatcher.Path = #"C:\Data\LotData\LP";
LPdirWatcher.Filter = "EM*";
LPdirWatcher.NotifyFilter = NotifyFilters.DirectoryName;
LPdirWatcher.EnableRaisingEvents = true;
LPdirWatcher.Created += LPdirWatcher_Created;
Thread.Sleep(10);
}
private static void CDdirWatcher_Created(object sender, FileSystemEventArgs e)
{
CDCreated += 1;
}
private static void LPdirWatcher_Created(object sender, FileSystemEventArgs e)
{
LPCreated += 1;
}
The above method works fine, and the criteria is that it has to be less count then the one set
public void checker()
{
if(CDCreated>CDmax)
{
popupbx();
}
if(LPCreated>LPmax)
{
popupbx();
}
}
The problem is my main method where I have two threads which need to continuously check the two criteria to see if the counter has been exceeded.
public Form1()
{
InitializeComponent();
//Implementing Threads asynchronously
Thread oThreadone = new Thread(() =>
{
//Do what u wanna……
watch();
});
Thread oThreadtwo = new Thread(() =>
{
//Do what u wanna……
checker();
});
//Calling thread workers
oThreadone.Start();
oThreadone.IsBackground = true;
oThreadtwo.Start();
oThreadtwo.IsBackground = true;
}
Mkdir fires the counters in debug mode, but thread two doesn't check for the counters after they fire.
The first major thing wrong with your code is that neither of threads you create are needed, nor do they do what you want. Specifically, the FileSystemWatcher object itself is already asynchronous, so you can create it in the main thread. In fact, you should, because there you could set FileSystemWatcher.SynchronizingObject to the current form instance so that it will raise its events in that object's synchronization context. I.e. your event handlers will be executed in the main thread, which is what you want.
So the first method, watch(), rather than being executed in a thread, just call it directly.
Which brings me to the second method, checker(). Your method for the thread doesn't loop, so it will execute the two tests, and then promptly exit. That's the end of that thread. It won't stay around long enough to monitor the counts as they are updated.
You could fix it by looping in the checker() method, so that it never exits. But then you run into problems of excessive CPU usage. Which you'll fix by adding sleep statements. Which then you're wasting a thread most of the time. Which you could fix by using async/await (e.g. await Task.Delay()). Except that would just unnecessarily complicate the code.
Instead, you should just perform each check after each count is updated. In theory, you could just display the message immediately. But that will block the event handler subscribed to FileSystemWatcher, possibly delaying additional reports. So you may instead prefer to use Control.BeginInvoke() to defer display of the message until after the event handler has returned.
So, taking all that into account, your code might instead look more like this:
public Form1()
{
InitializeComponent();
watch();
}
private void CDdirWatcher_Created(object sender, FileSystemEventArgs e)
{
CDCreated += 1;
if (CDCreated > CDmax)
{
BeginInvoke((MethodInvoker)popupbx);
}
}
private static void LPdirWatcher_Created(object sender, FileSystemEventArgs e)
{
LPCreated += 1;
if (LPCreated > LPmax)
{
BeginInvoke((MethodInvoker)popupbx);
}
}
You can remove the checker() method altogether. The watch() method can remain as it is, though I would change the order of subscribing to the Created event and the assignment of EnableRaisingEvents. I.e. don't enable raising events until you've already subscribed to the event. The FileSystemWatcher is unreliable enough as it is, without you giving it a chance to raise an event before you're ready to observe it. :)
This is based on your current implementation. Note though that if files keep getting created, the message will be displayed over and over. If they are created fast enough, new messages will be displayed before the user can dismiss the previously-displayed one(s). You may prefer to modify your code to prevent this. E.g. unsubscribe from the event after you've already exceeded the max count, or at least temporarily inhibit the display of the message while one such message is already being displayed. Exactly what to do is up to you, and beyond the scope of your question and thus the scope of this answer.
I am working on a windows service which will look for backup files in a particular folder. When it finds one the service will move all the backup files from that location and move them to an archived folder.
I have used FileSystemWatcher before but since it doesn't work on Servers I am using DirectoryInfo to look for the files.
The requirement is to run this service at every 5 minutes interval to look for any new backup files coming in.
I am stuck with the timer implementation.
Following is my code:
I want to call MoveToProcessed(processed) method from the ElapsedEventHandler. but I am getting error CS0149 - Method name expected.
protected override void OnStart(string[] args)
{
DirectoryInfo dir = new DirectoryInfo(backupdirectory);
// Other unrelated code omitted
// Move all the Backup files to Processed folder at certain intervals.
createOrderTimer = new System.Timers.Timer();
//***** ERROR ON THE FOLLOWING LINE *****
createOrderTimer.Elapsed += new ElapsedEventHandler(MoveToProcessed(processed));
//***************************************
createOrderTimer.Interval = 300000; // 15 min
createOrderTimer.Enabled = true;
createOrderTimer.AutoReset = true;
createOrderTimer.Start();
}
private void MoveToProcessed(string processed)
{
// Code here backs up and restores files
}
You can easily call a method from an event handler, but the event handler method itself must match a specific signature. For the System.Timers class, it should look something like this:
private static void Timer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
So you can simply create a method with this signature, assign it as an event handler, and then call your method from there:
// Elapsed event handler calls our other method
private static void CreateOrderTimer_Elapsed(object sender, System.Timers.ElapsedEventArgs e)
{
// Not sure what the string argument represents, or how it should be set
MoveToProcessed("some string");
}
// We can assign the event handler above when creating the Timer:
protected override void OnStart(string[] args)
{
var createOrderTimer = new System.Timers.Timer
{
Interval = TimeSpan.FromMinutes(15).TotalMilliseconds,
AutoReset = true
};
createOrderTimer.Elapsed += CreateOrderTimer_Elapsed;
createOrderTimer.Start();
}
I am reading a text file in windows Service .As per my requirement i have to read the text file line by line and process the contents of the line and insert into the database.The nature of the text file is such that it is constantly getting updated.Sometimes in a span of minute 100 lines will get added and sometimes no.So there is no fixed rate at which lines are getting inserted into the text file.
At present i am reading the text file using StreamReader line by line with while loop but as soon as line gets over it comes back.Here is my structure of windows Service.
public partial class Service1 : ServiceBase
{
public Service1()
{
InitializeComponent();
}
private System.Threading.Thread _thread;
private ManualResetEvent _shutdownEvent = new ManualResetEvent(false);
public void OnDebug()
{
OnStart(null);
}
protected override void OnStart(string[] args)
{
_thread = new Thread(parseAndProcess);
_thread.Start();
}
public void parseAndProcess()
{
if (System.IO.File.Exists(FileToCopy) == true)
{
using (StreamReader reader = new StreamReader(FileToCopy))
{
string line;
while ((line = reader.ReadLine()) != null)
{
if (line != "")
{
//line processing logic goes here
}
reader.Close();
}
}
}
}
protected override void OnStop()
{
_shutdownEvent.Set();
_thread.Join(); // wait for thread to stop
}
}
}
This is the entire structure of my windows Service.Here reading stops if there is no next line in the text file available for that time and it will be stopped until the service is restarted which i dont want to do.
So how can i check for file updation in case streamreader gets stopped reading lines.
Here is my updated code with FileSystemWatcher
protected override void OnStart(string[] args)
{
_thread = new Thread(parseAndProcess);
_thread.Start();
FileSystemWatcher Watcher = new FileSystemWatcher("File Path");
Watcher.EnableRaisingEvents = true;
Watcher.Changed += new FileSystemEventHandler(Watcher_Changed);
}
// This event is raised when a file is changed
private void Watcher_Changed(object sender, FileSystemEventArgs e)
{
_thread = new Thread(parseAndProcess);
_thread.Start();
}
In this Code i have a doubt will my how to read only new lines added into the textfile when getting invoked by FileSystemWatcher Changed Event.
Please help.
Create a private copy of the file, that way the file is open to be changed by other processes.
Process the copy.
Delete the copy.
If the original file has changed (could use FileSystemWatcher) repeat and do not process lines that you already processed.
Detecting lines that you already processed might be the hardest but depends on the idempotency of the system and the actual content of the file.
Idempotency means that the system (database) you are feeding the lines to does not change when passing it the same line multiple times.
Recognizing the same line depends on the content of the line. It might be a line number, GUID, ID, Timestamp
I'm fooling around with the FileSystemWatcher in 4.0. I find this very useful but am getting caught in a loop. I'm trying to monitor whenever an ini is changed and change it back to the correct default (long story) however the change event copying over the new file is causing it to drop into a loop ... Any Ideas > ? I played around with the idea of deleting and recreating thefile to avoid triggering the changed event but this leads to another set of issues with the program that I'd rather avoid. Also I'd imagine I could overwrite the text but this also poses the same issue. Thanks in advance for the help
static void Main() { Watch (#"\\NoFault2010\Lexis\Data\Setup\", "tmconfig.ini", true); }
static void Watch (string path, string filter, bool includeSubDirs)
{
using (var watcher = new FileSystemWatcher (path, filter))
{
watcher.Changed += FileChanged;
watcher.EnableRaisingEvents = true;
Console.WriteLine("Do Not Close ... \n\nThis is a Temporary Configuration Manager for Time Matters ... \n\n\nI'm Listening ............");
Console.ReadLine();
}
}
static void FileChanged (object o, FileSystemEventArgs e)
{
string _right_stuff = #"\\NOFAULT2010\Lexis\Data\Templates\Programs\tmconfig.ini";
string _working = #"\\NOFAULT2010\Lexis\Data\Setup\tmconfig.ini";
System.Threading.Thread.Sleep(2000);
File.Copy(_right_stuff, _working, true);
Console.WriteLine("File {0} has been {1}", e.FullPath, e.ChangeType);
MAIL_IT("SQLMail#lcjlawfirm.com", "TM Master.INI has been altered", "Check the Master INI and Yell At Ecopy Guy " + e.ChangeType + e.FullPath);
}
How would I unsubscribe from the event to avoid entering into this loop.
To temporarily disable the event while you're fiddling with the file yourself:
static void FileChanged (object o, FileSystemEventArgs e)
{
watcher.Changed -= FileChanged;
... correct the file here...
watcher.Changed += FileChanged;
}
Alternatively, you can use a guard variable to detect reentrant calls:
static bool reentrant = false;
static void FileChanged (object o, FileSystemEventArgs e)
{
if (reentrant)
return;
reentrant = true;
... correct the file here...
reentrant = false;
}
Note that you will also want to do exception handling within the method or your file watcher may become permanently disabled if a problem occurs.
I've written an application that depends on filesystemwatcher - and also, sometimes the fsw handler makes a change to a file.
I approached it in two ways - the first was to take the view that my code would be very quick in changing the file - so I did
fsw.EnableRaisingEvents = false;
//make my change
fsw.EnableRaisingEvents = true;
However, if you feel that other files might get changed during that time, you could log the time that you make the change and store that data somewhere...
say, Dictionary mapFileNameTimeChanged ...here you could store the file name...so in your handler you could do something like....
fsw_Changed(object sender, FileSystemEventArgs e)
{
lock (m_mapFileNameChanged)
{
if (m_mapFileNameChanged.ContainsKey(e.FullPath))
{
FileInfo fileInfo = new FileInfo(e.FullPath);
if (fileInfo.LastAccessTime == m_mapFileNameChanged[e.FullPath]
{
return;//not been changed since you last did something with it....
}
}
else
{
m_mapFileNameChanged.Remove(e.FullPath);//discard this now..it has changed since you last looked at it...need to look at it again!
}
}
//do things in your event handler...
lock (m_mapFileNameChanged)
{
// copy or change the file here...
FileInfo fileInfo = new FileInfo(e.FullPath);
m_mapFileNameChanged[strFullPathToFile] = fileInfo.LastAccessTime;
}
}
You could add a boolean (again at the class level) that you could use to track whether the changes were caused by you, and if so, just immediately exit your FileChanged method, ie:
static bool inEdit;
static void FileChanged (object o, FileSystemEventArgs e)
{
if (inEdit)
return;
inEdit = true;
// Do processing
inEdit = false;
}
Unsubscribe is easy, so I wonder if that was the question:
watcher.Changed -= FileChanged
Also, I would create some object to be SynchronizationObject for watcher. There is a problem that by default watcher raises events in new thread, and thus if you unsubscribe after new thread is created, you might run into the problems.
Also of note that FileSystemWatcher may raises multiple events for something you consider as single event, and it might influence functioning of your program.
If you make the watcher a class variable instead of a local variable, then your FileChanged method should be able to access it. Then you should be able to do something like
static void FileChanged (object o, FileSystemEventArgs e)
{
watcher.EnableRaisingEvents = false;
// Edit the file here
watcher.EnableRaisingEvents = true;
}