I'm attempting to use StreamReader and StreamWriter to grab a temporary output log (.txt format) from another application.
The output log is always open and constantly written to.
Unhelpfully if the application closes or crashes, the log file ends up deleted - hence the need for a tool that can grab the information from this log and save it.
What my program currently does is:
Create a new .txt file, and stores the path of that file as the
string "destinationFile".
Finds the .txt log file to read, and stores the path of that file as
the string "sourceFile"
It then passes those two strings to the method below.
Essentially I'm trying to read the sourceFile one line at a time.
Each time one line is read, it is appended to destinationFile.
This keeps looping until the sourceFile no longer exists (i.e. the application has closed or crashed and deleted its log).
In addition, the sourceFile can get quite big (sometimes 100Mb+), and this program may be handling more than one log at a time.
Reading the whole log rather than line by line will most likely start consuming a fair bit of memory.
private void logCopier(string sourceFile, string destinationFile)
{
while (File.Exists(sourceFile))
{
string textLine;
using (var readerStream = File.Open(sourceFile,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
using (var reader = new StreamReader(readerStream))
{
while ((textLine = reader.ReadLine()) != null)
{
using (FileStream writerStream = new FileStream(destinationFile,
FileMode.Append,
FileAccess.Write))
using (StreamWriter writer = new StreamWriter(writerStream))
{
writer.WriteLine(textLine);
}
}
}
}
}
The problem is that my WPF application locks up and ceases to respond when it reaches this code.
To track down where, I put a MessageBox just before the writerStream line of the code to output what the reader was picking up.
It was certainly reading the log file just fine, but there appears to be a problem with writing it to the file.
As soon as it reaches the using (FileStream writerStream = new FileStream part of the code, it stops responding.
Is using the StreamWriter in this manner not valid, or have I just gone and dome something silly in the code?
Am also open to a better solution than what I'm trying to do here.
Simply what I understand is you need to copy a file from source to destination which may be deleted at any time.
I'll suggest you to use FileSystemWatcher to watch for source file changed event, then just simply copy the whole file from source to destination using File.Copy.
I've just solved the problem, and the issue was indeed something silly!
When creating the text file for the StreamWriter, I had forgotten to use .Dispose();. I had File.Create(filename); instead of File.Create(filename).Dispose(); This meant the text file was already open, and the StreamWriter was attempting to write to a file that was locked / in use.
The UI still locks up (as expected), as I've yet to implement this on a new thread as SteenT mentioned. However the program no longer crashes and the code correctly reads the log and outputs to a text file.
Also after a bit of refinement, my log reader/writer code now looks like this:
private void logCopier(string sourceFile, string destinationFile)
{
int num = 1;
string textLine = String.Empty;
long offset = 0L;
while (num == 1)
{
if (File.Exists(sourceFile))
{
FileStream stream = new FileStream(sourceFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
using (new StreamReader(stream))
{
stream.Seek(offset, SeekOrigin.Begin);
TextReader reader2 = new StreamReader(stream);
while ((textLine = reader2.ReadLine()) != null)
{
Thread.Sleep(1);
StreamWriter writer = new StreamWriter(destinationFile, true);
writer.WriteLine(textLine);
writer.Flush();
writer.Close();
offset = stream.Position;
}
continue;
}
}
else
{
num = 0;
}
}
}
Just putting this code up here in case anyone else is looking for something like this. :)
Related
Our application needs to lock down a file to read-only when it is opened the first time. I have the code below, which doesn't seem to work anymore.
Now, for some reason, it never triggers the exception when I open a second file and just passes through the first statement every time.
From what I understand, the logic behind this code is this: The first file opens the file with Read/Write (FileAccess.ReadWrite) and sets the file to Read for subsequent users (FileShare.Read).
When it is opened the second time, it will try and open it with Read/Write but is restricted to Read-Only upon which it triggers the exception and goes to the second statement. There it will open the file as Read-Only(FileAccess.Read) and set it back to read-write(FileAccess.ReadWrite) for subsequent users, this to ensure that the first document does not get locked out of its already permitted Write rights, which caused an exception.
I tested this already, it used to work with below code. I can still verify that the read-lock is set. When I open the file in another PDF editor, it cannot save, just read. When I do it in my own application, I can save whenever I want.
Am I missing something, or was there a fluke so that it temporarily worked ?
try
{
m_FileStream = File.Open(fileName, FileMode.Open, FileAccess.ReadWrite, FileShare.Read);
m_bCanWrite = true;
}
catch (Exception)
{
m_FileStream = File.Open(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
m_bCanWrite = false;
}
I changed the code so it uses using blocks, with the same params, but the result is also the same.
try
{
FileInfo info = new FileInfo(fileName);
using (FileStream readWriteStream = info.Open(FileMode.Open, FileAccess.ReadWrite, FileShare.Read))
{
m_bcanWrite = true;
m_PdfDocument = new PDFDoc(readWriteStream);
}
}
catch (Exception)
{
FileInfo info = new FileInfo(fileName);
using (FileStream readStream = info.Open(FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
m_bcanWrite = false;
m_PdfDocument = new PDFDoc(readStream);
}
}
As we figured out in comments - the reason was premature closing of the file stream. Of course when you close filestream - all locks on it are released. It's not a good idea to hold the file open for the duration of the whole application run time as you suggested in comment - just close the file when you are done with it, not sooner but not later.
I'm trying to build a small program to monitor my pfirewall.log, but I can't seem to open it.
I found quite many (simple) answers, that all kinda say
// use FilesystemWatcher
// open FileStream
// read from last position to end
// output new lines
The problem here is: The file seems to always be opened by another process already. I guess that's the windows process writing to the file, since it's getting written to all the time, as Notepad++ shows me.
Which means, Notepad++ can for some reason do what I can not: Read the file despite it being opened already.
I initialize my monitor in the constructor:
public FirewallLogMonitor(string path)
{
if (!File.Exists(path))
throw new FileNotFoundException("Logfile not found");
this.file = path;
this.lastPosition = 0;
this.monitor = new FileSystemWatcher(Path.GetDirectoryName(path), Path.GetFileName(path));
this.monitor.NotifyFilter = NotifyFilters.Size;
}
And try to read the file on monitor.Changed event:
private void LogFileChanged(object sender, FileSystemEventArgs e)
{
using (FileStream stream = new FileStream(e.FullPath, FileMode.Open, FileAccess.Read, FileShare.Read))
using (StreamReader reader = new StreamReader(stream))
{
stream.Seek(this.lastPosition, SeekOrigin.Begin);
var newLines = reader.ReadToEnd();
this.lastPosition = stream.Length;
var filteredLines = filterLines(newLines);
if (filteredLines.Count > 0)
NewLinesAvailable(this, filteredLines);
}
}
It always throws the IOException on new FileStream(...) to tell me the file is already in use.
Since Notepad++ does it, there has to be a way I can do it too, right?
**Edit: ** A button does this:
public void StartLogging()
{
this.IsRunning = true;
this.monitor.Changed += LogFileChanged;
this.monitor.EnableRaisingEvents = true;
}
**Edit2: ** This is not a duplicate of FileMode and FileAccess and IOException: The process cannot access the file 'filename' because it is being used by another process, since that one assumes I have control over the writing process. Will try the other suggestions, and report back with results.
If i understand your question you can use the notepad++ itself with a plugin to monitor you need to go to:
plugins -> Document Moniter -> Start to monitor
if you dont have this plugin you can download it here:
http://sourceforge.net/projects/npp-plugins/files/DocMonitor/
for an application that uses a File as some sort of global storage for device reservations in a firm I need a way to read and write to a file (or lock a file, read from it, write to it, and unlock it). A little code snippet will shot what I mean:
FileStream in = new FileStream("storage.bin", FileMode.Open);
//read the file
in.Close();
//!!!!!
//here is the critical section since between reading and writing, there shouldnt
//be a way for another process to access and lock the file, but there is the chance
//because the in stream is closed
//!!!!!
FileStream out = new FileStream("storage.bin", FileMode.Create);
//write data to file
out.Close();
this should get something like this
LockFile("storage.bin");
//read from it...
//OVERwrite it....
UnlockFile("storage.bin");
the method should be absolute safe, since the program should run on 2000 devices at the same time
Simply holding a FileStream open with exclusive (not shared) access will prevent other processes from accessing the file. This is the default when opening a file for read/write access.
You can 'overwrite' a file that you currently hold open by truncating it.
So:
using (var file = File.Open("storage.bin", FileMode.Open))
{
// read from the file
file.SetLength(0); // truncate the file
// write to the file
}
the method should be absolute safe, since the program should run on 2000 devices at the same time
Depending on how often you're writing to the file, this could become a chokepoint. You probably want to test this to see how scalable it is.
In addition, if one of the processes tries to operate on the file at the same time as another one, an IOException will be thrown. There isn't really a way to 'wait' on a file, so you probably want to coordinate file access in a more orderly fashion.
You need a single stream, opened for both reading and writing.
FileStream fileStream = new FileStream(
#"c:\words.txt", FileMode.OpenOrCreate,
FileAccess.ReadWrite, FileShare.None);
Alternatively you can also try
static void Main(string[] args)
{
var text = File.ReadAllText(#"C:\words.txt");
File.WriteAllText(#"C:\words.txt", text + "DERP");
}
As per http://msdn.microsoft.com/en-us/library/system.io.fileshare(v=vs.71).aspx
FileStream s2 = new FileStream(name, FileMode.Open, FileAccess.Read, FileShare.None);
You need to pass in a FileShare enumeration value of None to open on the FileStream constructor overloads:
fs = new FileStream(#"C:\Users\Juan Luis\Desktop\corte.txt", FileMode.Open,
FileAccess.ReadWrite, FileShare.None);
I ended up writing this helper class to do this:
public static class FileHelper
{
public static void ReplaceFileContents(string fileName, Func<String, string> replacementFunction)
{
using (FileStream fileStream = new FileStream(
fileName, FileMode.OpenOrCreate,
FileAccess.ReadWrite, FileShare.None))
{
StreamReader streamReader = new StreamReader(fileStream);
string currentContents = streamReader.ReadToEnd();
var newContents = replacementFunction(currentContents);
fileStream.SetLength(0);
StreamWriter writer = new StreamWriter(fileStream);
writer.Write(newContents);
writer.Close();
}
}
}
which allows you to pass a function that will take the existing contents and generate new contents and ensure the file is not read or modified by anything else whilst this change is happening
You are likely looking for FileStream.Lock and FileStream.Unlock
I think you just need to use the FileShare.None flag in the overloaded Open method.
file = File.Open("storage.bin", FileMode.Open, FileShare.None);
Here is my code:
public static TextWriter twLog = null;
private int fileNo = 1;
private string line = null;
TextReader tr = new StreamReader("file_no.txt");
TextWriter tw = new StreamWriter("file_no.txt");
line = tr.ReadLine();
if(line != null){
fileNo = int.Parse(line);
twLog = new StreamWriter("log_" + line + ".txt");
}else{
twLog = new StreamWriter("log_" + fileNo.toString() + ".txt");
}
System.IO.File.WriteAllText("file_no.txt",string.Empty);
tw.WriteLine((fileNo++).ToString());
tr.Close();
tw.Close();
twLog.Close();
It throws this error:
IOException: Sharing violation on path C:\Users\Water Simulation\file_no.txt
What i'm trying to do is just open a file with log_x.txt name and take the "x" from file_no.txt file.If file_no.txt file is empty make log file's name log_1.txt and write "fileNo + 1" to file_no.txt.After a new program starts the new log file name must be log_2.txt.But i'm getting this error and i couldn't understand what am i doing wrong.Thanks for help.
Well, you're trying to open the file file_no.txt for reading and for writing using separate streams. This may not work as the file will be locked by the reading stream, so the writing stream can't be created and you get the exception.
One solution would be to read the file first, close the stream and then write the file after increasing the fileNo. That way the file is only opened once at a time.
Another way would be to create a file stream for both read and write access like that:
FileStream fileStream = new FileStream(#"file_no.txt",
FileMode.OpenOrCreate,
FileAccess.ReadWrite,
FileShare.None);
The accepted answer to this question seems to have a good solution also, even though I assume you do not want to allow shared reads.
Possible alternate solution
I understand you want to create unique log files when your program starts. Another way to do so would be this:
int logFileNo = 1;
string fileName = String.Format("log_{0}.txt", logFileNo);
while (File.Exists(fileName))
{
logFileNo++;
fileName = String.Format("log_{0}.txt", logFileNo);
}
This increases the number until it finds a file number where the log file doesn't exist. Drawback: If you have log_1.txt and log_5.txt, the next file won't be log_6.txt but log_2.txt.
To overcome this, you could enumerate all the files in your directory with mask log_*.txt and find the greatest number by performing some string manipulation.
The possibilities are endless :-D
Well this may be old but the accepted answer didn't work for me. This is caused when you try to Read or Write a file you just created from a separate stream. Solving this is very simple, just dispose the filestream you used in creating it and then you can access the file freely.
if (!File.Exists(myfile))
{
var fs = new FileStream(fav, FileMode.Create);
fs.Dispose();
string text = File.ReadAllText(myfile);
}
enter image description here
var stream = new System.IO.FileStream(filePath, System.IO.FileMode.Create);
resizedBitmap.Compress(Bitmap.CompressFormat.Png, 200, stream); //problem here
stream.Close();
return resizedBitmap;
In the Compress method, I was passing the value of the quality parameter as 200, which sadly doesn't allows values outside the range 0-100.
I changed back the value of quality to 100 and the issue got fixed.
None of the proposed options helped me. But I found a solution:
In my case, the problem was with Anti-Virus, with intensive writing to a file, Anti-Virus started scanning the file and at that moment there was a problem with writing to the file.
The StreamReader locks a text file whilst it is reading it.
Can I force the StreamReader to work in a "read-only" or "non locking" mode?
My workaround would be to copy the file to a temp location and read it from there but I would prefer to use the StreamReader directly if possible.
Any alternative suggetions?
Background:
I've written a small app to get some stats out of a log file. This file is constantly being updating (several times a second) by an outside program lets call AAXXYY.
Reviewing the output suggests that my app may be locking the file and preventing AAXXYY from writing.
This is what I'm doing
private void btnGetStats_Click(object sender, EventArgs e)
{
int countStarts = 0;
int countEnds = 0;
IList<string> sessions = new List<string>();
using(StreamReader stRead = new StreamReader(openFileDialog1.FileName,Encoding.Unicode))
{
while(!stRead.EndOfStream)
{
string line = stRead.ReadLine();
if(line.Contains("Session start"))
{
countStarts++;
sessions.Add(line.Substring(line.IndexOf("["), line.LastIndexOf("]") - line.IndexOf("[")));
}
if (line.Contains("Session end"))
{
countEnds++;
sessions.Remove(line.Substring(line.IndexOf("["), line.LastIndexOf("]") - line.IndexOf("[")));
}
}
}
txtStarts.Text = countStarts.ToString();
txtEnds.Text = countEnds.ToString();
txtDifference.Text = (countStarts - countEnds).ToString();
listBox1.DataSource = sessions;
}
You can pass a FileStream to the StreamReader, and create the FileStream with the proper FileShare value. For instance:
using (var file = new FileStream (openFileDialog1.FileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
using (var reader = new StreamReader (file, Encoding.Unicode)) {
}
Thought I'd add some context, StreamReader does not lock a file for reading only for writing whist it is being read. Take a look at the code below from the StreamReader class.
new FileStream(path, FileMode.Open, FileAccess.Read, FileShare.Read, FileOptions.SequentialScan);
Notice the default FileAccess.Read parameter taken for MSDN http://msdn.microsoft.com/en-us/library/system.io.fileshare.aspx
Allows subsequent opening of the file for reading. If this flag is not specified, any request to open the file for reading (by this process or another process) will fail until the file is closed. However, even if this flag is specified, additional permissions might still be needed to access the file.
Again taken from MSDN to allow reading and writing use FileAccess.ReadWrite instead (as suggested by Jb Evain).
Allows subsequent opening of the file for reading or writing. If this
flag is not specified, any request to open the file for reading or
writing (by this process or another process) will fail until the file
is closed. However, even if this flag is specified, additional
permissions might still be needed to access the file.