We Keep Coding

How to monitor a logfile that seems to be open all the time (much like notepad++ does)?

I'm trying to build a small program to monitor my pfirewall.log, but I can't seem to open it.
I found quite many (simple) answers, that all kinda say
// use FilesystemWatcher
// open FileStream
// read from last position to end
// output new lines
The problem here is: The file seems to always be opened by another process already. I guess that's the windows process writing to the file, since it's getting written to all the time, as Notepad++ shows me.
Which means, Notepad++ can for some reason do what I can not: Read the file despite it being opened already.
I initialize my monitor in the constructor:
public FirewallLogMonitor(string path)
{
if (!File.Exists(path))
throw new FileNotFoundException("Logfile not found");
this.file = path;
this.lastPosition = 0;
this.monitor = new FileSystemWatcher(Path.GetDirectoryName(path), Path.GetFileName(path));
this.monitor.NotifyFilter = NotifyFilters.Size;
}
And try to read the file on monitor.Changed event:
private void LogFileChanged(object sender, FileSystemEventArgs e)
{
using (FileStream stream = new FileStream(e.FullPath, FileMode.Open, FileAccess.Read, FileShare.Read))
using (StreamReader reader = new StreamReader(stream))
{
stream.Seek(this.lastPosition, SeekOrigin.Begin);
var newLines = reader.ReadToEnd();
this.lastPosition = stream.Length;
var filteredLines = filterLines(newLines);
if (filteredLines.Count > 0)
NewLinesAvailable(this, filteredLines);
}
}
It always throws the IOException on new FileStream(...) to tell me the file is already in use.
Since Notepad++ does it, there has to be a way I can do it too, right?
**Edit: ** A button does this:
public void StartLogging()
{
this.IsRunning = true;
this.monitor.Changed += LogFileChanged;
this.monitor.EnableRaisingEvents = true;
}
**Edit2: ** This is not a duplicate of FileMode and FileAccess and IOException: The process cannot access the file 'filename' because it is being used by another process, since that one assumes I have control over the writing process. Will try the other suggestions, and report back with results.

If i understand your question you can use the notepad++ itself with a plugin to monitor you need to go to:
plugins -> Document Moniter -> Start to monitor
if you dont have this plugin you can download it here:
http://sourceforge.net/projects/npp-plugins/files/DocMonitor/

How to deal with File Exception: cannot access the file because it is being used by another process

How to deal with the this Exception. I have a text file. And this code rewrites the file every 3 sec. At the same time another MS Excel file reads the data from the file. And sometimes I get this exception.
Is there a way for StreamWriter to wait until MS Excel finishes reading the file and start rewriting the file right after that?
string path = #"C:\My Path\my_file.txt";
using (StreamWriter fs = new StreamWriter(File.OpenWrite(path)))
{
fs.Write(text);
fs.Close();
}

This is one way:
string path = #"C:\My Path\my_file.txt";
string text = "Declare text variable so this code compiles";
FileStream fileS = null;
bool done = false;
while (!done)
{
done = true;
try
{
fileS = File.OpenWrite(path);
}
catch (IOException ex)
{
done = false;
}
}
using (StreamWriter fs = new StreamWriter(fileS))
{
fs.Write(text);
fs.Close();
};
I would also add in a StopWatch call to have the above while loop timeout after so long, and possibly a small Thread.Sleep() so you don't spin at full speed while essentially trying to obtain the FileStream lock.

Does FileShare.None make threads wait until the filestream is closed?

When using a file stream, and setting FileShare to None, and say two users accessing the same function at the same time want to read/write to that file. Will FileShare.None make the second users request waiting or will the second user's request throw an exception?
//two users get to this this code at the same time
using (FileStream filestream = new FileStream(chosenFile, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None))
using (StreamReader sr = new StreamReader(filestream))
using (StreamWriter sw = new StreamWriter(filestream))
{
//reading and writing to file
}
Msdn says: None Declines sharing of the current file. Any request to open the file (by this process or another process) will fail until the file is closed.
But will requests keep trying until the filestream is closed?

When a process opean a file for Read/Write with FileShare.None any subsequent access by any process on this same file will result in Acess Denied Exception. To answer your question, Second user will get exception.
MSDN: FileShare.None - Declines sharing of the current file. Any request to open the
file (by this process or another process) will fail until the file is
closed.
There are many ways you can handle these kind of concurrent file access issues, Following code demonstrates a simple approach to tackle this situation.
//Retry 5 times when file access fails
int retryCounter = 5;
while (!isFileAccessSuccess && retryCounter > 0)
{
try
{
//Put file access logic here
//If the file has been accessed successfully set the flag to true
isFileAccessSuccess = true;
}
catch (Exception exception)
{
//Log exception
}
finally
{
//Decrease the retry count
--retryCounter;
}
if (!isFileAccessSuccess)
{
//Wait sometime until initiating next try
Thread.Sleep(10000);
}
}

No, IOException will be thrown an with HResult = -2147024864 and Message = The process cannot access the file 'path' because it is being used by another process.
if you want to synchronize access to a file you can use a named wait handle.
public class FileAcessSynchronizer
{
private readonly string _path;
private readonly EventWaitHandle _waitHandle;
public FileAcessSynch(string path)
{
_path = path;
_waitHandle = new EventWaitHandle(true, EventResetMode.AutoReset, "NameOfTheWaitHandle");
}
public void DoSomething()
{
try
{
_waitHandle.WaitOne();
using (FileStream filestream = new FileStream(chosenFile, FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.None))
using (StreamReader sr = new StreamReader(filestream))
using (StreamWriter sw = new StreamWriter(filestream))
{
//reading and writing to file
}
}
finally
{
_waitHandle.Set();
}
}
}
since named wait handle creates a critical section no two threads or processes of your application (that use same name as wait handle name) can execute the codes in it concurrently. So one thread or process enters the section, opens the file in the way that no one can access it (other applications), execute commands and at the end leaves the critical section to allow other threads or processes of your application enters the critical section.

Using StreamReader / StreamWriter to grab logs causes program to cease responding

I'm attempting to use StreamReader and StreamWriter to grab a temporary output log (.txt format) from another application.
The output log is always open and constantly written to.
Unhelpfully if the application closes or crashes, the log file ends up deleted - hence the need for a tool that can grab the information from this log and save it.
What my program currently does is:
Create a new .txt file, and stores the path of that file as the
string "destinationFile".
Finds the .txt log file to read, and stores the path of that file as
the string "sourceFile"
It then passes those two strings to the method below.
Essentially I'm trying to read the sourceFile one line at a time.
Each time one line is read, it is appended to destinationFile.
This keeps looping until the sourceFile no longer exists (i.e. the application has closed or crashed and deleted its log).
In addition, the sourceFile can get quite big (sometimes 100Mb+), and this program may be handling more than one log at a time.
Reading the whole log rather than line by line will most likely start consuming a fair bit of memory.
private void logCopier(string sourceFile, string destinationFile)
{
while (File.Exists(sourceFile))
{
string textLine;
using (var readerStream = File.Open(sourceFile,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
using (var reader = new StreamReader(readerStream))
{
while ((textLine = reader.ReadLine()) != null)
{
using (FileStream writerStream = new FileStream(destinationFile,
FileMode.Append,
FileAccess.Write))
using (StreamWriter writer = new StreamWriter(writerStream))
{
writer.WriteLine(textLine);
}
}
}
}
}
The problem is that my WPF application locks up and ceases to respond when it reaches this code.
To track down where, I put a MessageBox just before the writerStream line of the code to output what the reader was picking up.
It was certainly reading the log file just fine, but there appears to be a problem with writing it to the file.
As soon as it reaches the using (FileStream writerStream = new FileStream part of the code, it stops responding.
Is using the StreamWriter in this manner not valid, or have I just gone and dome something silly in the code?
Am also open to a better solution than what I'm trying to do here.

Simply what I understand is you need to copy a file from source to destination which may be deleted at any time.
I'll suggest you to use FileSystemWatcher to watch for source file changed event, then just simply copy the whole file from source to destination using File.Copy.

I've just solved the problem, and the issue was indeed something silly!
When creating the text file for the StreamWriter, I had forgotten to use .Dispose();. I had File.Create(filename); instead of File.Create(filename).Dispose(); This meant the text file was already open, and the StreamWriter was attempting to write to a file that was locked / in use.
The UI still locks up (as expected), as I've yet to implement this on a new thread as SteenT mentioned. However the program no longer crashes and the code correctly reads the log and outputs to a text file.
Also after a bit of refinement, my log reader/writer code now looks like this:
private void logCopier(string sourceFile, string destinationFile)
{
int num = 1;
string textLine = String.Empty;
long offset = 0L;
while (num == 1)
{
if (File.Exists(sourceFile))
{
FileStream stream = new FileStream(sourceFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
using (new StreamReader(stream))
{
stream.Seek(offset, SeekOrigin.Begin);
TextReader reader2 = new StreamReader(stream);
while ((textLine = reader2.ReadLine()) != null)
{
Thread.Sleep(1);
StreamWriter writer = new StreamWriter(destinationFile, true);
writer.WriteLine(textLine);
writer.Flush();
writer.Close();
offset = stream.Position;
}
continue;
}
}
else
{
num = 0;
}
}
}
Just putting this code up here in case anyone else is looking for something like this. :)

Why am I getting a Unhandled Exception: System.IO.IOException when trying to read from a file being written to?

I have two C# applications, one is reading a file (File A) line by line and writing its contents to a different file (File B).
The second application is using FileSystemWatcher for File B to see when it is updated and reporting the difference is line numbers between when the program was started and when the file was changed.
Thats all I am trying to do for now, ultimately I want to read the lines between when the file was last read and the current read but until I can get the line difference that is on hold.
The code that I have for application 1 is;
static void Main(string[] args)
{
String line;
StreamReader sr = new StreamReader("f:\\watch\\input.txt");
FileStream fs = new FileStream("f:\\watch\\Chat.log", FileMode.Create, FileAccess.Write, FileShare.ReadWrite);
StreamWriter sw = new StreamWriter(fs);
while ((line = sr.ReadLine()) != null)
{
sw.WriteLine(line);
Thread.Sleep(200);
Console.WriteLine(line);
sw.Flush();
}
sw.Close();
sr.Close();
}
The code that I have for application 2 is;
public static int lines = 0;
public static void Main()
{
Run();
}
public static void Run()
{
string[] args = System.Environment.GetCommandLineArgs();
if (args.Length != 2)
{
Console.WriteLine("Usage: Watcher.exe (directory)");
return;
}
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.Path = args[1];
watcher.NotifyFilter = NotifyFilters.LastAccess | NotifyFilters.LastWrite
| NotifyFilters.FileName | NotifyFilters.DirectoryName;
watcher.Filter = "Chat.log";
watcher.Changed += new FileSystemEventHandler(OnChanged);
watcher.EnableRaisingEvents = true;
lines = File.ReadAllLines(args[1] + "\\Chat.log").Length;
Console.WriteLine("File lines: " + lines);
while(Console.Read()!='q');
}
private static void OnChanged(object source, FileSystemEventArgs e)
{
Linework(e.FullPath);
Console.WriteLine("File: " + e.FullPath + " " + e.ChangeType);
}
public static string Linework(string path)
{
string newstring = " ";
using (FileStream stream = new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
int newlines = File.ReadAllLines(path).Length;
Console.WriteLine("Lines now: " + newlines);
}
return newstring;
}
Now when I try and run these two applications together I get an exception saying "Unhandled Exception: System.IO.IOException: The process cannot access the file because it is in use by another process".
I have both filestreams setup for ReadWrite access and I have one of the filestreams setup for FileAccess.Write and the other for FileAccess.Read.
Any clues as to why I would be getting this exception?
Thanks
Hew.

lines = File.ReadAllLines(args[1] + "\Chat.log").Length;
There's your problem. That method opens the file, reads all the lines and closes it again. It uses "normal" file share settings when opening the file, FileShare.Read. That denies write access to any other process that also has the file opened.
That cannot work here, you've already have the file opened with write access. The 2nd process cannot deny it. The IOException is the result.
You cannot use File.ReadAllLines() as-is here, you need to open a FileStream with FileShare.ReadWrite, pass it to a StreamReader and read all lines.
Beware the very troublesome race potential you've got here, there's no guarantee that the last line you'll read is a complete line. Getting only a \r and not the \n at the end of the line is a particularly tricky issue. This will strike randomly and infrequently, the hardest bugs to troubleshoot. Maybe your Flush() call fixes it, I've never been brave enough to put this to the test.

Allowing the second program ReadWrite access on the file would work in this case.
//lines = File.ReadAllLines(args[1] + "\\Chat.log").Length;
//Commenting the above lines as this would again open a new filestream on the chat.log
//without the proper access mode required.
using (FileStream fsReader = new FileStream(args[1] + "\\Chat.log", FileMode.Open, FileAccess.ReadWrite, FileShare.ReadWrite))
{
using (StreamReader sr = new StreamReader(fsReader))
{
while (sr.ReadLine() != null)
lines++;
}
}

StreamReader sr = new StreamReader("f:\\watch\\input.txt");
input.txt might not be available for reading?
Also use the using statement instead of Close() in the 1st application (in case an exception is thrown).
Otherwise it is OK. The file share may require additional permissions though (can't really affect that).
I have missed one piece of code:
int newlines = File.ReadAllLines(path).Length;
use the stream with a StreamReader for that.

MSDN offers two ways to not obtain an exclusive hold:
A FileStream object will not have an
exclusive hold on its handle when
either the SafeFileHandle property is
accessed to expose the handle or the
FileStream object is given the
SafeFileHandle property in its
constructor.
The documentation implies that the inverse is true:
Opening a FileStream without setting the SafeFileHandle means the
FileStream maintains an exclusive hold on the file handle (which is
inline with the IO exception that is supposed to be thrown).