We Keep Coding

How to monitor a logfile that seems to be open all the time (much like notepad++ does)?

I'm trying to build a small program to monitor my pfirewall.log, but I can't seem to open it.
I found quite many (simple) answers, that all kinda say
// use FilesystemWatcher
// open FileStream
// read from last position to end
// output new lines
The problem here is: The file seems to always be opened by another process already. I guess that's the windows process writing to the file, since it's getting written to all the time, as Notepad++ shows me.
Which means, Notepad++ can for some reason do what I can not: Read the file despite it being opened already.
I initialize my monitor in the constructor:
public FirewallLogMonitor(string path)
{
if (!File.Exists(path))
throw new FileNotFoundException("Logfile not found");
this.file = path;
this.lastPosition = 0;
this.monitor = new FileSystemWatcher(Path.GetDirectoryName(path), Path.GetFileName(path));
this.monitor.NotifyFilter = NotifyFilters.Size;
}
And try to read the file on monitor.Changed event:
private void LogFileChanged(object sender, FileSystemEventArgs e)
{
using (FileStream stream = new FileStream(e.FullPath, FileMode.Open, FileAccess.Read, FileShare.Read))
using (StreamReader reader = new StreamReader(stream))
{
stream.Seek(this.lastPosition, SeekOrigin.Begin);
var newLines = reader.ReadToEnd();
this.lastPosition = stream.Length;
var filteredLines = filterLines(newLines);
if (filteredLines.Count > 0)
NewLinesAvailable(this, filteredLines);
}
}
It always throws the IOException on new FileStream(...) to tell me the file is already in use.
Since Notepad++ does it, there has to be a way I can do it too, right?
**Edit: ** A button does this:
public void StartLogging()
{
this.IsRunning = true;
this.monitor.Changed += LogFileChanged;
this.monitor.EnableRaisingEvents = true;
}
**Edit2: ** This is not a duplicate of FileMode and FileAccess and IOException: The process cannot access the file 'filename' because it is being used by another process, since that one assumes I have control over the writing process. Will try the other suggestions, and report back with results.

If i understand your question you can use the notepad++ itself with a plugin to monitor you need to go to:
plugins -> Document Moniter -> Start to monitor
if you dont have this plugin you can download it here:
http://sourceforge.net/projects/npp-plugins/files/DocMonitor/

File Copy Program Doesn't Properly Copy File

Hello
I've been working on terminal-like application to get better at programming in c#, just something to help me learn. I've decided to add a feature that will copy a file exactly as it is, to a new file... It seems to work almost perfect. When opened in Notepad++ the file are only a few lines apart in length, and very, very, close to the same as far as actual file size goes. However, the duplicated copy of the file never runs. It says the file is corrupt. I have a feeling it's within the methods for reading and rewriting binary to files that I created. The code is as follows, thank for the help. Sorry for the spaghetti code too, I get a bit sloppy when I'm messing around with new ideas.
Class that handles the file copying/writing
using System;
using System.IO;
//using System.Collections.Generic;
namespace ConsoleFileExplorer
{
class FileTransfer
{
private BinaryWriter writer;
private BinaryReader reader;
private FileStream fsc; // file to be duplicated
private FileStream fsn; // new location of file
int[] fileData;
private string _file;
public FileTransfer(String file)
{
_file = file;
fsc = new FileStream(file, FileMode.Open);
reader = new BinaryReader(fsc);
}
// Reads all the original files data to an array of bytes
public byte[] ReadAllDataToArray()
{
byte[] bytes = reader.ReadBytes((int)fsc.Length); // reading bytes from the original file
return bytes;
}
// writes the array of original byte data to a new file
public void WriteDataFromArray(byte[] fileData, string path) // got a feeling this is the problem :p
{
fsn = new FileStream(path, FileMode.Create);
writer = new BinaryWriter(fsn);
int i = 0;
while(i < fileData.Length)
{
writer.Write(fileData[i]);
i++;
}
}
}
}
Code that interacts with this class .
(Sleep(5000) is because I was expecting an error on first attempt...
case '3':
Console.Write("Enter source file: ");
string sourceFile = Console.ReadLine();
if (sourceFile == "")
{
Console.Clear();
Console.ForegroundColor = ConsoleColor.DarkRed;
Console.Error.WriteLine("Must input a proper file path.\n");
Console.ForegroundColor = ConsoleColor.White;
Menu();
} else {
Console.WriteLine("Copying Data"); System.Threading.Thread.Sleep(5000);
FileTransfer trans = new FileTransfer(sourceFile);
//copying the original files data
byte[] data = trans.ReadAllDataToArray();
Console.Write("Enter Location to store data: ");
string newPath = Console.ReadLine();
// Just for me to make sure it doesnt exit if i forget
if(newPath == "")
{
Console.Clear();
Console.ForegroundColor = ConsoleColor.DarkRed;
Console.Error.WriteLine("Cannot have empty path.");
Console.ForegroundColor = ConsoleColor.White;
Menu();
} else
{
Console.WriteLine("Writing data to file"); System.Threading.Thread.Sleep(5000);
trans.WriteDataFromArray(data, newPath);
Console.WriteLine("File stored.");
Console.ReadLine();
Console.Clear();
Menu();
}
}
break;
File compared to new file
right-click -> open in new tab is probably a good idea
Original File
New File

You're not properly disposing the file streams and the binary writer. Both tend to buffer data (which is a good thing, especially when you're writing one byte at a time). Use using, and your problem should disappear. Unless somebody is editing the file while you're reading it, of course.
BinaryReader and BinaryWriter do not just write "raw data". They also add metadata as needed - they're designed for serialization and deserialization, rather than reading and writing bytes. Now, in the particular case of using ReadBytes and Write(byte[]) in particular, those are really just raw bytes; but there's not much point to use these classes just for that. Reading and writing bytes is the thing every Stream gives you - and that includes FileStreams. There's no reason to use BinaryReader/BinaryWriter here whatsover - the file streams give you everything you need.
A better approach would be to simply use
using (var fsn = ...)
{
fsn.Write(fileData, 0, fileData.Length);
}
or even just
File.WriteAllBytes(fileName, fileData);
Maybe you're thinking that writing a byte at a time is closer to "the metal", but that simply isn't the case. At no point during this does the CPU pass a byte at a time to the hard drive. Instead, the hard drive copies data directly from RAM, with no intervention from the CPU. And most hard drives still can't write (or read) arbitrary amounts of data from the physical media - instead, you're reading and writing whole sectors. If the system really did write a byte at a time, you'd just keep rewriting the same sector over and over again, just to write one more byte.
An even better approach would be to use the fact that you've got file streams open, and stream the files from source to destination rather than first reading everything into memory, and then writing it back to disk.

There is an File.Copy() Method in C#, you can see it here https://msdn.microsoft.com/ru-ru/library/c6cfw35a(v=vs.110).aspx
If you want to realize it by yourself, try to place a breakpoint inside your methods and use a debug. It is like a story about fisher and god, who gived a rod to fisher - to got a fish, not the exactly fish.
Also, look at you int[] fileData and byte[] fileData inside last method, maybe this is problem.

Using StreamReader / StreamWriter to grab logs causes program to cease responding

I'm attempting to use StreamReader and StreamWriter to grab a temporary output log (.txt format) from another application.
The output log is always open and constantly written to.
Unhelpfully if the application closes or crashes, the log file ends up deleted - hence the need for a tool that can grab the information from this log and save it.
What my program currently does is:
Create a new .txt file, and stores the path of that file as the
string "destinationFile".
Finds the .txt log file to read, and stores the path of that file as
the string "sourceFile"
It then passes those two strings to the method below.
Essentially I'm trying to read the sourceFile one line at a time.
Each time one line is read, it is appended to destinationFile.
This keeps looping until the sourceFile no longer exists (i.e. the application has closed or crashed and deleted its log).
In addition, the sourceFile can get quite big (sometimes 100Mb+), and this program may be handling more than one log at a time.
Reading the whole log rather than line by line will most likely start consuming a fair bit of memory.
private void logCopier(string sourceFile, string destinationFile)
{
while (File.Exists(sourceFile))
{
string textLine;
using (var readerStream = File.Open(sourceFile,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
using (var reader = new StreamReader(readerStream))
{
while ((textLine = reader.ReadLine()) != null)
{
using (FileStream writerStream = new FileStream(destinationFile,
FileMode.Append,
FileAccess.Write))
using (StreamWriter writer = new StreamWriter(writerStream))
{
writer.WriteLine(textLine);
}
}
}
}
}
The problem is that my WPF application locks up and ceases to respond when it reaches this code.
To track down where, I put a MessageBox just before the writerStream line of the code to output what the reader was picking up.
It was certainly reading the log file just fine, but there appears to be a problem with writing it to the file.
As soon as it reaches the using (FileStream writerStream = new FileStream part of the code, it stops responding.
Is using the StreamWriter in this manner not valid, or have I just gone and dome something silly in the code?
Am also open to a better solution than what I'm trying to do here.

Simply what I understand is you need to copy a file from source to destination which may be deleted at any time.
I'll suggest you to use FileSystemWatcher to watch for source file changed event, then just simply copy the whole file from source to destination using File.Copy.

I've just solved the problem, and the issue was indeed something silly!
When creating the text file for the StreamWriter, I had forgotten to use .Dispose();. I had File.Create(filename); instead of File.Create(filename).Dispose(); This meant the text file was already open, and the StreamWriter was attempting to write to a file that was locked / in use.
The UI still locks up (as expected), as I've yet to implement this on a new thread as SteenT mentioned. However the program no longer crashes and the code correctly reads the log and outputs to a text file.
Also after a bit of refinement, my log reader/writer code now looks like this:
private void logCopier(string sourceFile, string destinationFile)
{
int num = 1;
string textLine = String.Empty;
long offset = 0L;
while (num == 1)
{
if (File.Exists(sourceFile))
{
FileStream stream = new FileStream(sourceFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
using (new StreamReader(stream))
{
stream.Seek(offset, SeekOrigin.Begin);
TextReader reader2 = new StreamReader(stream);
while ((textLine = reader2.ReadLine()) != null)
{
Thread.Sleep(1);
StreamWriter writer = new StreamWriter(destinationFile, true);
writer.WriteLine(textLine);
writer.Flush();
writer.Close();
offset = stream.Position;
}
continue;
}
}
else
{
num = 0;
}
}
}
Just putting this code up here in case anyone else is looking for something like this. :)

"The process cannot access the file because it is being used by another process" with Images

I've seen many issues like this that have been solved and the problem was mostly due to streams not being disposed of properly.
My issue is slightly different, here follow a code snippet
foreach (Images item in ListOfImages)
{
newPath = Path.Combine(newPath, item.ImageName + item.ImageExtension);
File.Create(newPath);
File.WriteAllBytes(newPath, item.File);
}
Where Images is a custom struct and item.File is the raw data, byte[].
My issue is that at the line where the WriteAllBytes is called, an exception is thrown. The message reads:
The process cannot access the file because it is being used by another process
Again I have no clue how am I going to somehow close the process.

Since File.Create returns the stream i would dispose it properly:
using(var stream = File.Create(newPath)){}
File.WriteAllBytes(newPath, item.File);
or you can use the stream to write to the file directly:
using (FileStream fs = File.Create(newPath))
{
fs.Write(item.File, 0, item.File.Length);
}
or, probably the easiest, use File.WriteAllBytes alone:
File.WriteAllBytes(newPath, item.File);
Creates a new file, writes the specified byte array to the file, and
then closes the file. If the target file already exists, it is
overwritten.

You state that your problem has nothing to do with disposing streams but check this MSDN article:
http://msdn.microsoft.com/en-us/library/d62kzs03.aspx
What does File.Create return? A FileStream!!!!
And, at the end of the day, why are you using File.Create if File.WriteAllBytes creates a file if this doesn't exist? ;)
Creates a new file, writes the specified byte array to the file, and
then closes the file. If the target file already exists, it is
overwritten.
Check it on MSDN too: http://msdn.microsoft.com/en-us/library/system.io.file.writeallbytes.aspx

using (FileStream fs =
new FileStream(filePath,
FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
Your log may be write locked, so try with FileShare.ReadWrite.

The create method opens the file for writing and returns a FileStream object for you to work with. Just because you are not referencing it does not mean it does not need to be returned.
foreach (Images item in ListOfImages)
{
newPath = Path.Combine(newPath, item.ImageName + item.ImageExtension);
FileStream f = File.Create(newPath);
f.Write(item.File, 0, item.File.Length);
}

The File.WriteAllBytes creates the file if necessary. You can juts use:
foreach (Images item in ListOfImages)
{
newPath = Path.Combine(newPath, item.ImageName + item.ImageExtension);
File.WriteAllBytes(newPath, item.File);
}
And are you combine path correctly?

This is the most specific way to accomplish what you are trying to do:
foreach (Images item in ListOfImages)
{
using (System.IO.FileStream output = new System.IO.FileStream(Path.Combine(newPath, item.ImageName + item.ImageExtension),
System.IO.FileMode.Create, System.IO.FileAccess.Write))
{
output.Write(item.File, 0, item.File.Length);
output.Flush();
output.Close();
}
}
You also need to fix your logic for creating the path, which I have done in my example above. You were concatenating the newPath over and over again.

Force the garbage collector to clean.
GC.Collect();

Create a temporary file from stream object in c#

Given a stream object which contains an xlsx file, I want to save it as a temporary file and delete it when not using the file anymore.
I thought of creating a class that implementing IDisposable and using it with the using code block in order to delete the temp file at the end.
Any idea of how to save the stream to a temp file and delete it on the end of use?
Thanks

You could use the TempFileCollection class:
using (var tempFiles = new TempFileCollection())
{
string file = tempFiles.AddExtension("xlsx");
// do something with the file here
}
What's nice about this is that even if an exception is thrown the temporary file is guaranteed to be removed thanks to the using block. By default this will generate the file into the temporary folder configured on the system but you could also specify a custom folder when invoking the TempFileCollection constructor.

You can get a temporary file name with Path.GetTempFileName(), create a FileStream to write to it and use Stream.CopyTo to copy all data from your input stream into the text file:
var stream = /* your stream */
var fileName = Path.GetTempFileName();
try
{
using (FileStream fs = File.OpenWrite(fileName))
{
stream.CopyTo(fs);
}
// Do whatever you want with the file here
}
finally
{
File.Delete(fileName);
}

Another approach here would be:
string fileName = "file.xslx";
int bufferSize = 4096;
var fileStream = System.IO.File.Create(fileName, bufferSize, System.IO.FileOptions.DeleteOnClose)
// now use that fileStream to save the xslx stream
This way the file will get removed after closing.
Edit:
If you don't need the stream to live too long (eg: only a single write operation or a single loop to write...), you can, as suggested, wrap this stream into a using block. With that you won't have to dispose it manually.
Code would be like:
string fileName = "file.xslx";
int bufferSize = 4096;
using(var fileStream = System.IO.File.Create(fileName, bufferSize, System.IO.FileOptions.DeleteOnClose))
{
// now use that fileStream to save the xslx stream
}

// Get a random temporary file name w/ path:
string tempFile = Path.GetTempFileName();
// Open a FileStream to write to the file:
using (Stream fileStream = File.OpenWrite(tempFile)) { ... }
// Delete the file when you're done:
File.Delete(tempFile);
EDIT:
Sorry, maybe it's just me, but I could have sworn that when you initially posted the question you didn't have all that detail about a class implementing IDisposable, etc... anyways, I'm not really sure what you're asking in your (edited?) question. But this question: Any idea of how to save the stream to temp file and delete it on the end of use? is pretty straight-forward. Any number of google results will come back for ".NET C# Stream to File" or such.

I just suggest for creating file use Path.GetTempFileName(). but others depends on your usage senario, for example if you want to create it in your temp creator class and use it just there, it's good to use using keyword.