We Keep Coding

FileStream's FileMode.OpenOrCreate overwrites file

Documentation says FileMode.OpenOrCreate "specifies that the operating system should open a file if it exists; otherwise, a new file should be created", which sounds like it will open the file and write to it. Instead, the file seems to be overwritten.
How do I add to the file, rather than overwrite it?
class Logger : IDisposable
{
private FileStream fs;
private StreamWriter sw;
public Logger()
{
// INTENT (but not reality): Will create file if one does not exist, otherwise opens existing file
fs = new FileStream("log.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite);
sw = new StreamWriter(fs, Encoding.UTF8);
}
public void Log(string message)
{
sw.WriteLine(message);
sw.Flush();
fs.Flush();
}
public void Dispose()
{
sw?.Dispose();
fs?.Dispose();
}
}

The documentation is correct – but note that open and append are not synonymous.
FileMode.OpenOrCreate is not causing the FileStream constructor to delete the preexisting file wholesale; rather, it causes the stream to start at the beginning of the file. What you are observing is the file contents being overwritten by the StreamWriter, not the FileStream constructor replacing the file immediately.
You have to move the stream position to the end of the file to add text at the end. To do this, you could move the position with FileStream.Seek() or change the FileMode to FileMode.Append.
FileMode.Append is likely more idiomatic but also requires making the FileAccess write-only, rather than read-write. Otherwise, an exception is thrown upon calling the contructor.
Option 1 (FileMode.OpenOrCreate, FileAccess.ReadWrite)
public Logger()
{
// Will create file if one does not exist, otherwise opens existing file
fs = new FileStream("log.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite, FileShare.ReadWrite);
// Set stream position to end-of-file
fs.Seek(0, SeekOrigin.End);
sw = new StreamWriter(fs, Encoding.UTF8);
}
Option 2 (FileMode.Append, FileAccess.Write)
public Logger()
{
// Will create file if one does not exist, otherwise appends to existing file
fs = new FileStream("log.txt", FileMode.Append, FileAccess.Write, FileShare.ReadWrite);
sw = new StreamWriter(fs, Encoding.UTF8);
}

Using StreamReader / StreamWriter to grab logs causes program to cease responding

I'm attempting to use StreamReader and StreamWriter to grab a temporary output log (.txt format) from another application.
The output log is always open and constantly written to.
Unhelpfully if the application closes or crashes, the log file ends up deleted - hence the need for a tool that can grab the information from this log and save it.
What my program currently does is:
Create a new .txt file, and stores the path of that file as the
string "destinationFile".
Finds the .txt log file to read, and stores the path of that file as
the string "sourceFile"
It then passes those two strings to the method below.
Essentially I'm trying to read the sourceFile one line at a time.
Each time one line is read, it is appended to destinationFile.
This keeps looping until the sourceFile no longer exists (i.e. the application has closed or crashed and deleted its log).
In addition, the sourceFile can get quite big (sometimes 100Mb+), and this program may be handling more than one log at a time.
Reading the whole log rather than line by line will most likely start consuming a fair bit of memory.
private void logCopier(string sourceFile, string destinationFile)
{
while (File.Exists(sourceFile))
{
string textLine;
using (var readerStream = File.Open(sourceFile,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
using (var reader = new StreamReader(readerStream))
{
while ((textLine = reader.ReadLine()) != null)
{
using (FileStream writerStream = new FileStream(destinationFile,
FileMode.Append,
FileAccess.Write))
using (StreamWriter writer = new StreamWriter(writerStream))
{
writer.WriteLine(textLine);
}
}
}
}
}
The problem is that my WPF application locks up and ceases to respond when it reaches this code.
To track down where, I put a MessageBox just before the writerStream line of the code to output what the reader was picking up.
It was certainly reading the log file just fine, but there appears to be a problem with writing it to the file.
As soon as it reaches the using (FileStream writerStream = new FileStream part of the code, it stops responding.
Is using the StreamWriter in this manner not valid, or have I just gone and dome something silly in the code?
Am also open to a better solution than what I'm trying to do here.

Simply what I understand is you need to copy a file from source to destination which may be deleted at any time.
I'll suggest you to use FileSystemWatcher to watch for source file changed event, then just simply copy the whole file from source to destination using File.Copy.

I've just solved the problem, and the issue was indeed something silly!
When creating the text file for the StreamWriter, I had forgotten to use .Dispose();. I had File.Create(filename); instead of File.Create(filename).Dispose(); This meant the text file was already open, and the StreamWriter was attempting to write to a file that was locked / in use.
The UI still locks up (as expected), as I've yet to implement this on a new thread as SteenT mentioned. However the program no longer crashes and the code correctly reads the log and outputs to a text file.
Also after a bit of refinement, my log reader/writer code now looks like this:
private void logCopier(string sourceFile, string destinationFile)
{
int num = 1;
string textLine = String.Empty;
long offset = 0L;
while (num == 1)
{
if (File.Exists(sourceFile))
{
FileStream stream = new FileStream(sourceFile, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
using (new StreamReader(stream))
{
stream.Seek(offset, SeekOrigin.Begin);
TextReader reader2 = new StreamReader(stream);
while ((textLine = reader2.ReadLine()) != null)
{
Thread.Sleep(1);
StreamWriter writer = new StreamWriter(destinationFile, true);
writer.WriteLine(textLine);
writer.Flush();
writer.Close();
offset = stream.Position;
}
continue;
}
}
else
{
num = 0;
}
}
}
Just putting this code up here in case anyone else is looking for something like this. :)

One program writes to file while other program reads same file c#

1) I have created a program that has opened a twitter stream and writes everything to a file.
FileStream fs = new FileStream(#"\Database\twitterstream.txt", FileMode.Create);
TextWriter tmp = Console.Out;
StreamWriter sw = new StreamWriter(fs);
Console.SetOut(sw);
2) I have another program that I want to read said text file.
using (StreamReader sr = File.OpenText("C:\\Database\\twitterstream.txt"))
{
input = sr.ReadLine();
}
Because I want it to be in real time I am trying to have one program write, while at the same time the other program reads, however obviously it is throwing
"The process cannot access the file
'C:\Database\twitterstream.txt' because it is being used by another
process" back at me.
Is what I am trying to do possible? If so, how do I go about doing it?

Add a couple parameters to you FileStream constructor:
FileStream fs = new FileStream(
#"\Database\twitterstream.txt",
FileMode.Create, FileAccess.ReadWrite, FileShare.ReadWrite);
See FileStream on MSDN

Read and Write to File at the same time

for an application that uses a File as some sort of global storage for device reservations in a firm I need a way to read and write to a file (or lock a file, read from it, write to it, and unlock it). A little code snippet will shot what I mean:
FileStream in = new FileStream("storage.bin", FileMode.Open);
//read the file
in.Close();
//!!!!!
//here is the critical section since between reading and writing, there shouldnt
//be a way for another process to access and lock the file, but there is the chance
//because the in stream is closed
//!!!!!
FileStream out = new FileStream("storage.bin", FileMode.Create);
//write data to file
out.Close();
this should get something like this
LockFile("storage.bin");
//read from it...
//OVERwrite it....
UnlockFile("storage.bin");
the method should be absolute safe, since the program should run on 2000 devices at the same time

Simply holding a FileStream open with exclusive (not shared) access will prevent other processes from accessing the file. This is the default when opening a file for read/write access.
You can 'overwrite' a file that you currently hold open by truncating it.
So:
using (var file = File.Open("storage.bin", FileMode.Open))
{
// read from the file
file.SetLength(0); // truncate the file
// write to the file
}
the method should be absolute safe, since the program should run on 2000 devices at the same time
Depending on how often you're writing to the file, this could become a chokepoint. You probably want to test this to see how scalable it is.
In addition, if one of the processes tries to operate on the file at the same time as another one, an IOException will be thrown. There isn't really a way to 'wait' on a file, so you probably want to coordinate file access in a more orderly fashion.

You need a single stream, opened for both reading and writing.
FileStream fileStream = new FileStream(
#"c:\words.txt", FileMode.OpenOrCreate,
FileAccess.ReadWrite, FileShare.None);
Alternatively you can also try
static void Main(string[] args)
{
var text = File.ReadAllText(#"C:\words.txt");
File.WriteAllText(#"C:\words.txt", text + "DERP");
}
As per http://msdn.microsoft.com/en-us/library/system.io.fileshare(v=vs.71).aspx
FileStream s2 = new FileStream(name, FileMode.Open, FileAccess.Read, FileShare.None);
You need to pass in a FileShare enumeration value of None to open on the FileStream constructor overloads:
fs = new FileStream(#"C:\Users\Juan Luis\Desktop\corte.txt", FileMode.Open,
FileAccess.ReadWrite, FileShare.None);

I ended up writing this helper class to do this:
public static class FileHelper
{
public static void ReplaceFileContents(string fileName, Func<String, string> replacementFunction)
{
using (FileStream fileStream = new FileStream(
fileName, FileMode.OpenOrCreate,
FileAccess.ReadWrite, FileShare.None))
{
StreamReader streamReader = new StreamReader(fileStream);
string currentContents = streamReader.ReadToEnd();
var newContents = replacementFunction(currentContents);
fileStream.SetLength(0);
StreamWriter writer = new StreamWriter(fileStream);
writer.Write(newContents);
writer.Close();
}
}
}
which allows you to pass a function that will take the existing contents and generate new contents and ensure the file is not read or modified by anything else whilst this change is happening

You are likely looking for FileStream.Lock and FileStream.Unlock

I think you just need to use the FileShare.None flag in the overloaded Open method.
file = File.Open("storage.bin", FileMode.Open, FileShare.None);

What's the least invasive way to read a locked file in C# (perhaps in unsafe mode)?

I need to read a Windows file that may be locked, but I don't want to create any kind lock that will prevent other processes from writing to the file.
In addition, even if the file is locked for exclusive use, I'd like to see what's inside.
Although this isn't my exact use case, consider how to read a SQL/Exchange log or database file while it's in use and mounted. I don't want to cause corruption but I still want to see the insides of the file and read it.

You can do it without copying the file, see this article:
The trick is to use FileShare.ReadWrite (from the article):
private void LoadFile()
{
try
{
using(FileStream fileStream = new FileStream(
"logs/myapp.log",
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
{
using(StreamReader streamReader = new StreamReader(fileStream))
{
this.textBoxLogs.Text = streamReader.ReadToEnd();
}
}
}
catch(Exception ex)
{
MessageBox.Show("Error loading log file: " + ex.Message);
}
}

The accepted answer is not correct. If the file is really locked, you cannot just change the file share. This would work if the lock has been set with this fileshare option too but it does not mean that it is the case. In fact, you can test #CaffGeek solution pretty easily by opening the file without the FileShare.ReadWrite and than trying to open it with this flag to ReadWrite. You will get that the file is using by another process.
Code:
string content;
var filePath = "e:\\test.txt";
//Lock Exclusively the file
var r = File.Open(filePath, FileMode.Open, FileAccess.Write, FileShare.Write);
//CaffGeek solution
using (FileStream fileStream = new FileStream(
filePath,
FileMode.Open,
FileAccess.Read,
FileShare.ReadWrite))
{
using (StreamReader streamReader = new StreamReader(fileStream))
{
content = streamReader.ReadToEnd();
}
}
As you can see, it crashes. This result is the same with any FileStream method like the File.Open. It will crash what ever you put for FileShare during the open stage.
//OPEN FOR WRITE with exclusive
var r = File.Open(filePath, FileMode.Open, FileAccess.Write, FileShare.Write);
//OPEN FOR READ with file share that allow read and write
var x = File.Open(filePath, FileMode.Open, FileAccess.Read, FileShare.ReadWrite); //Crash
Copying the file is not also an option. You can try it your self by opening the file exclusively and try to copy the file on Windows Explorer or by code:
var filePath = "e:\\test.txt";
var filePathCopy = "e:\\test.txt.bck";
//Lock the file
var r = File.Open(filePath, FileMode.Open, FileAccess.Write, FileShare.Write);
File.Copy(filePath, filePathCopy);
var x = File.Open(filePathCopy, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
using (var reader = new StreamReader(x))
{
content = reader.ReadToEnd();
}
r.Close();
File.Delete(filePathCopy);
This code crash when you hit the File.Copy line. The exception is the same as before : file is being using by another process.
You need to kill the process that has the lock of the file if you want to read it OR if you have the source code of the file that is locking the file to change this one to use FileShare.ReadWrite instead of just FileShare.Write.

You can probably create a copy and read that, even if the file is locked.
Or maybe a StreamReader on a FileStream depending on how SQL opened the file?
new FileStream("c:\myfile.ext", FileMode.Open, FileAccess.Read, FileShare.ReadWrite);