I have a thread producing lines in a log file:
var t1 = Task.Factory.StartNew(() =>
{
using (var fileStream = File.Open(file, FileMode.Create, FileAccess.Write, FileShare.Read))
using (var streamWriter = new StreamWriter(fileStream))
{
for (var i = 0; i < 10; i++)
{
streamWriter.WriteLine(i);
streamWriter.Flush();
Thread.Sleep(1000);
}
}
File.Delete(file);
});
And I have a thread reading lines from the same log file:
// Reads lines from the log file.
var t2 = Task.Factory.StartNew(() =>
{
Thread.Sleep(500); // Horrible wait to ensure file existence in this test case.
using (var fileStream = new FileStream(file, FileMode.Open, FileAccess.Read, FileShare.Delete | FileShare.Read | FileShare.Write))
using (var streamReader = new StreamReader(fileStream))
{
string line;
while ((line = streamReader.ReadLine()) != null) Console.WriteLine(line);
// FIXME: The stream reader stops, instead of doing a continous read.
Console.WriteLine("End of file");
}
});
The reader is supposed to read all written lines, therefore it should wait for more data instead of stopping at the first time it encounters EOF. I do not mind if the reader is never 'finished', so long as the file continues to exist, the reader is allowed to continue reading. How can I achieve this? Full code for reproduction purposes:
using System;
using System.IO;
using System.Threading;
using System.Threading.Tasks;
namespace PlayGround
{
internal static class Program
{
private static void Main()
{
const string file = "test.log";
// Writes lines to the log file.
var t1 = Task.Factory.StartNew(() =>
{
using (var fileStream = File.Open(file, FileMode.Create, FileAccess.Write, FileShare.Read))
using (var streamWriter = new StreamWriter(fileStream))
{
for (var i = 0; i < 10; i++)
{
streamWriter.WriteLine(i);
streamWriter.Flush();
Thread.Sleep(1000);
}
}
File.Delete(file);
});
// Reads lines from the log file.
var t2 = Task.Factory.StartNew(() =>
{
Thread.Sleep(500); // Horrible wait to ensure file existence in this test case.
using (
var fileStream = new FileStream(file, FileMode.Open, FileAccess.Read,
FileShare.Delete | FileShare.Read | FileShare.Write))
using (var streamReader = new StreamReader(fileStream))
{
string line;
while ((line = streamReader.ReadLine()) != null) Console.WriteLine(line);
// FIXME: The stream reader stops, instead of doing a continous read.
Console.WriteLine("End of file");
}
});
Task.WaitAll(t1, t2);
}
}
}
EDIT: As a practical example, this is useful for a scenario where a third party process is producing log entries which need to be read and processed. You could see this as a log file reader if that makes the application and use clearer.
You could perform a wait when the line == null, by checking the streamReader.EndOfFile property. Using Thread.Sleep(1000) is not an ideal solution, a bit hacky, and I guess there are other better alternative solutions out there. :-)
using (var fileStream = new FileStream(file, FileMode.Open, FileAccess.Read, FileShare.Delete | FileShare.Read | FileShare.Write))
using (var streamReader = new StreamReader(fileStream))
{
string line;
bool running = true; // we may want to terminate this loop in some condition.
while (running)
{
line = streamReader.ReadLine();
if (line != null)
{
Console.WriteLine(line);
}
else // as per edit, the whole else block can be omitted.
{
while (streamReader.EndOfStream)
{
Thread.Sleep(1000); // wait around for n time. This could end up in an infinte loop if the file is not written to anymore.
}
}
}
// FIXME: The stream reader stops, instead of doing a continous read.
Console.WriteLine("End of file");
}
EDIT: You can do without the else block:
else
{
while (streamReader.EndOfStream)
{
Thread.Sleep(1000)
}
}
Like this:
while (running)
{
line = streamReader.ReadLine();
if (line != null)
{
Console.WriteLine(line);
}
}
You need synchronization mechanism, in this case I use AutoResetEvent.
The required changes based on your code is.
const string file = "test.log";
// Adds new line.
AutoResetEvent signal = new AutoResetEvent(false);
streamWriter.Flush();
// Adds new line.
signal.Set();
File.Delete(file);
// Adds new line
signal.Set();
Thread.Sleep(500);
// Replaces with.
signal.WaitOne();
while ((line = streamReader.ReadLine()) != null) Console.WriteLine(line);
// Replaces with.
while ((line = streamReader.ReadLine()) != null)
{
signal.WaitOne();
Console.WriteLine(line);
}
Full code
const string file = "test.log";
AutoResetEvent signal = new AutoResetEvent(false);
// Writes lines to the log file.
var t1 = Task.Factory.StartNew(() =>
{
using (var fileStream = File.Open(file, FileMode.Create, FileAccess.Write, FileShare.Read))
{
using (var streamWriter = new StreamWriter(fileStream))
{
for (var i = 0; i < 10; i++)
{
streamWriter.WriteLine(i);
streamWriter.Flush();
signal.Set();
Thread.Sleep(10);
}
}
}
File.Delete(file);
signal.Set();
});
// Reads lines from the log file.
var t2 = Task.Factory.StartNew(() =>
{
signal.WaitOne();
using (var fileStream = new FileStream(file, FileMode.Open, FileAccess.Read, FileShare.Delete | FileShare.Read | FileShare.Write))
{
using (var streamReader = new StreamReader(fileStream))
{
string line;
while ((line = streamReader.ReadLine()) != null)
{
signal.WaitOne();
Console.WriteLine(line);
}
// FIXME: The stream reader stops, instead of doing a continous read.
Console.WriteLine("End of file");
}
}
});
Task.WaitAll(t1, t2);
Related
I'm using a FileStream to lock the File to be not writeable for other processes and also read and write to it, I'm using following method for it:
public static void ChangeOrAddLine(string newLine, string oldLine = "")
{
string filePath = "C:\\test.txt";
FileMode fm = FileMode.Create;
//FileMode fm = FileMode.OpenOrCreate;
using (FileStream fs = new FileStream(filePath, FileMode.Create, FileAccess.ReadWrite, FileShare.Read))
using (StreamReader sr = new StreamReader(fs))
using (StreamWriter sw = new StreamWriter(fs))
{
List<string> lines = sr.ReadToEnd().Split(new string[] { "\r\n" }, StringSplitOptions.None).ToList();
bool lineFound = false;
if (oldLine != "")
for (int i = 0; i < lines.Count; i++)
if (lines[i] == oldLine)
{
lines[i] = newLine;
lineFound = true;
break;
}
if (!lineFound)
lines.Add(newLine);
sw.Write(string.Join("\r\n", lines));
}
}
I want to overwrite it with the new content but i don't find the right FileMode, using FileMode.OpenOrCreate just appends the new content to the old and FileMode.Create deletes the file-content at the time, the FileStream fm has been initialized, so the file is empty.
I need to just clear the old content at the moment, when i write the new content to it without losing the write-lock on it during the method is running.
OpenOrCreate just appends ...
Because you don't reposition after the reading.
That also shows the main problem with your approach: The FileStream only has one Position, and the Reader and the Writer heavily use caching.
However, as long as you want to replace everything and really need that locking scheme:
using (FileStream fs = new FileStream(filePath,
FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.Read))
{
using (StreamReader sr = new StreamReader(fs))
{
... // all the reading
}
fs.Position = 0;
using (StreamWriter sw = new StreamWriter(fs))
{
sw.Write(string.Join("\r\n", lines));
}
fs.SetLength(fs.Position); // untested, something along this line
}
and maybe you have to convince the sw and sr to leave their stream open.
But I have to note that the FileShare.Read flag doesn't make too much sense in this scenario. A reader could see al sorts of inconsistent data, including torn lines and broken UTF8 characters.
I'm using a filestream to access a file (making an md5 ComputeHash). If I attempt to rename the file during this time (which fails as the file is being accessed). So far so good, but when I then try to open the file anew after the original filestream is closed I get the info that the file is open in another process.
Code:
using (Stream fileStream = new FileStream(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) {
MD5 md5 = MD5.Create();
byte[] mymd5computed = md5.ComputeHash(fileStream);
......
}
Thread.Sleep(50);
Thread a = new Thread (()=>{(FileStream sourceStream = File.Open(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)){....} });
Like I said if while during the computation of the MD5 I try to rename the file I get the info that the file is still locked.
The lock on a file sometimes isn't released right away when you close your stream, so there are some other solutions you can use to wait until you can access the file again. One of them is explained here: http://www.codeproject.com/Tips/164428/C-FileStream-Lock-How-to-wait-for-a-file-to-get-re.
Recap:
public static void Lock(string path, Action<FileStream> action) {
var autoResetEvent = new AutoResetEvent(false);
while(true)
{
try
{
using (var file = File.Open(path,
FileMode.OpenOrCreate,
FileAccess.ReadWrite,
FileShare.Write))
{
action(file);
break;
}
}
catch (IOException)
{
var fileSystemWatcher =
new FileSystemWatcher(Path.GetDirectoryName(path))
{
EnableRaisingEvents = true
};
fileSystemWatcher.Changed +=
(o, e) =>
{
if(Path.GetFullPath(e.FullPath) == Path.GetFullPath(path))
{
autoResetEvent.Set();
}
};
autoResetEvent.WaitOne();
}
}
}
Sample use:
Lock(#"c:\file",
(f) =>
{
try
{
f.Write(buf, 0, buf.Length);
}
catch(IOException ioe)
{
// handle IOException
}
});
Hope it helps! :)
I have a TextReader object.
Now, I want to stream the whole content of the TextReader to a File. I cannot use ReadToEnd() and write all to a file at once, because the content can be of high size.
Can someone give me a sample/tip how to do this in Blocks?
using (var textReader = File.OpenText("input.txt"))
using (var writer = File.CreateText("output.txt"))
{
do
{
string line = textReader.ReadLine();
writer.WriteLine(line);
} while (!textReader.EndOfStream);
}
Something like this. Loop through the reader until it returns null and do your work. Once done, close it.
String line;
try
{
line = txtrdr.ReadLine(); //call ReadLine on reader to read each line
while (line != null) //loop through the reader and do the write
{
Console.WriteLine(line);
line = txtrdr.ReadLine();
}
}
catch(Exception e)
{
// Do whatever needed
}
finally
{
if(txtrdr != null)
txtrdr.Close(); //close once done
}
Use TextReader.ReadLine:
// assuming stream is your TextReader
using (stream)
using (StreamWriter sw = File.CreateText(#"FileLocation"))
{
while (!stream.EndOfStream)
{
var line = stream.ReadLine();
sw.WriteLine(line);
}
}
I'm working now on a class that will allow editing very big text files (4Gb+). Well it may sound a little stupid but I do not understand how I can modify text in a stream.
Here is my code:
public long Replace(String text1, String text2)
{
long replaceCount = 0;
currentFileStream = File.Open(CurrentFileName, FileMode.Open, FileAccess.ReadWrite, FileShare.None);
using (BufferedStream bs = new BufferedStream(currentFileStream))
using (StreamReader sr = new StreamReader(bs))
{
string line;
while ((line = sr.ReadLine()) != null)
{
if (line.Contains(text1))
{
line.Replace(text1, text2);
// Here I should save changed line
replaceCount++;
}
}
}
return replaceCount;
}
You are not replacing it anywhere in your code. You should save all the text and then write it again to the file. Like,
public long Replace(String text1, String text2)
{
long replaceCount = 0;
currentFileStream = File.Open(CurrentFileName, FileMode.Open, FileAccess.ReadWrite, FileShare.None);
StringBuilder sb = new StringBuilder();
using (BufferedStream bs = new BufferedStream(currentFileStream))
using (StreamReader sr = new StreamReader(bs))
{
string line;
while ((line = sr.ReadLine()) != null)
{
string textToAdd = line;
if (line.Contains(text1))
{
textToAdd = line.Replace(text1, text2);
// Here I should save changed line
replaceCount++;
}
sb.Append(textToAdd);
}
}
using (FileStream fileStream = new FileStream(filename , fileMode, fileAccess))
{
StreamWriter streamWriter = new StreamWriter(fileStream);
streamWriter.Write(sb.ToString());
streamWriter.Close();
fileStream.Close();
}
return replaceCount;
}
I have a windows service writes its log in a text file in a simple format.
Now, I'm going to create a small application to read the service's log and shows both the existing log and the added one as live view.
The problem is that the service locks the text file for adding the new lines and at the same time the viewer application locks the file for reading.
The Service Code:
void WriteInLog(string logFilePath, data)
{
File.AppendAllText(logFilePath,
string.Format("{0} : {1}\r\n", DateTime.Now, data));
}
The viewer Code:
int index = 0;
private void Form1_Load(object sender, EventArgs e)
{
try
{
using (StreamReader sr = new StreamReader(logFilePath))
{
while (sr.Peek() >= 0) // reading the old data
{
AddLineToGrid(sr.ReadLine());
index++;
}
sr.Close();
}
timer1.Start();
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
private void timer1_Tick(object sender, EventArgs e)
{
using (StreamReader sr = new StreamReader(logFilePath))
{
// skipping the old data, it has read in the Form1_Load event handler
for (int i = 0; i < index ; i++)
sr.ReadLine();
while (sr.Peek() >= 0) // reading the live data if exists
{
string str = sr.ReadLine();
if (str != null)
{
AddLineToGrid(str);
index++;
}
}
sr.Close();
}
}
Is there any problem in my code in reading and writing way?
How to solve the problem?
You need to make sure that both the service and the reader open the log file non-exclusively. Try this:
For the service - the writer in your example - use a FileStream instance created as follows:
var outStream = new FileStream(logfileName, FileMode.Open,
FileAccess.Write, FileShare.ReadWrite);
For the reader use the same but change the file access:
var inStream = new FileStream(logfileName, FileMode.Open,
FileAccess.Read, FileShare.ReadWrite);
Also, since FileStream implements IDisposable make sure that in both cases you consider using a using statement, for example for the writer:
using(var outStream = ...)
{
// using outStream here
...
}
Good luck!
Explicit set up the sharing mode while reading the text file.
using (FileStream fs = new FileStream(logFilePath,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
{
using (StreamReader sr = new StreamReader(fs))
{
while (sr.Peek() >= 0) // reading the old data
{
AddLineToGrid(sr.ReadLine());
index++;
}
}
}
new StreamReader(File.Open(logFilePath,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
-> this doesn't lock the file.
The problem is when you are writing to the log you are exclusively locking the file down so your StreamReader won't be allowed to open it at all.
You need to try open the file in readonly mode.
using (FileStream fs = new FileStream("myLogFile.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
{
using (StreamReader sr = new StreamReader(fs))
{
while (!fs.EndOfStream)
{
string line = fs.ReadLine();
// Your code here
}
}
}
I remember doing the same thing a couple of years ago. After some google queries i found this:
FileStream fs = new FileStream(#”c:\test.txt”,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite);
i.e. use the FileShare.ReadWrite attribute on FileStream().
(found on Balaji Ramesh's blog)
Have you tried copying the file, then reading it?
Just update the copy whenever big changes are made.
This method will help you to fastest read a text file and without locking it.
private string ReadFileAndFetchStringInSingleLine(string file)
{
StringBuilder sb;
try
{
sb = new StringBuilder();
using (FileStream fs = File.Open(file, FileMode.Open))
{
using (BufferedStream bs = new BufferedStream(fs))
{
using (StreamReader sr = new StreamReader(bs))
{
string str;
while ((str = sr.ReadLine()) != null)
{
sb.Append(str);
}
}
}
}
return sb.ToString();
}
catch (Exception ex)
{
return "";
}
}
Hope this method will help you.