We Keep Coding

Timers, Files and race conditions?

Hello and thanks for your help.
This time I am having a curious problem with a program (C#) I am writing and would like to hear your advice.
I am writing a normal program (not multithreaded) but then added a timer (System.Timers.Timer)
Also I am using a StreamWriter to write on a file. I open this like this
StreamWriter logStream=new StreamWriter(filename, true);
meaning that if the file exists, it appends , if not it creates.
Later I write in the file like this
logStream.WriteLine(message);
However, I write to the stream from both the main function and from the function that is called by the timer.
the problem symptoms
My program is throwing an error sometimes when I flush or write the stream saying that "Can not access a closed file" and other times "Can not access a closed TextWriter... (What is a "TextWriter"?)
However curiously, the file keeps being written without problem. (Even the "can not access a closed file" message is written in the supposed closed file)
I am not familiar with the inner workings of a Timer. (I suppose it runs a separate thread?)
My question is
Is it possible to use a StreamWriter from several threads? (in this case the main one and the Timer one)
Is it possible that there is happening a race condition or some problem like that?
One more thing: I made a logic mistake and close and reopen the file every time I want to write on it. Yes, it is a mistake and I should correct it. But maybe if I correct this the error I described above will disappear masking a more serious flaw.
My suspicions is that since I am closing and opening the file every time I write on it, maybe the both threads try to access them on a wrong time
Any help will be greatly appreciated

Closing and opening you file under this scenario will create a race condition like you suspect. You cannot keep the stream open and pass the object to the thread because you might end up with a similar issue if you call from different thread. Your best solution remain using a thread safe method that will write what you send to it.
the methods are static because the lock has to be accessible from all instance of the class.
private static ReaderWriterLockSlim readerWriterLockSlim = new ReaderWriterLockSlim();
public static void AppendToFile(string path, string text)
{
// Set to locked (other thread will freeze here until object is unlocked
readerWriterLockSlim.EnterWriteLock();
try
{
// Write that will append to the file
using (StreamWriter sw = File.AppendText(path))
{
// append the text
sw.WriteLine(text);
sw.Close();
}
}
finally
{
// Clear the lock
readerWriterLockSlim.ExitWriteLock();
}
}

How to check that a file can be read successfully? [duplicate]

This question already has answers here:
Detecting whether a file is locked by another process (or indeed the same process) [duplicate]
(4 answers)
Closed 9 years ago.
In my program I pass the path to a file which may be still being written to by another process (independent external non-c#).
How can I check the file is fully readable and accessible before I actually send the path back to the client? FileIOPermission with Demand for reading did not work for me, so I am wondering if there is some other way of doing this without attempting to read the whole file upfront.
Thank you.

The trouble with checking to see if a file is accessible and fully readable, and then opening the file for reading is that potentially, the state could change between the check and the open where you actually want to be doing something with it.
My suggestion is to go ahead and open the file for reading, being sure to catch the appropriate exception if a problem occurs. You can then wait a little and try again, or do something else.
Remember that just because your program has a sequence of events that are logical to you, many things are going on in an operating system and the state can easily change between two lines of code that seem watertight for you, but have an epoch between them at a multitasking level.

Moo-Juice is right. You can't check whether the file can be read successfully and then open it for reading... things can change and you might get an exception anyway. Best to just read it and catch the exception.
public bool TryReadFile(String path, out String contentsOfFile)
{
try
{
// Try reading file
contentsOfFile = File.ReadAllText(path);
// Success! Yay!
return true;
}
catch (IOException)
{
// Oops! Can't read that file!
// Return some default value and let the caller know we failed
contentsOfFile = String.Empty;
return false;
}

Best way would be to read the file in a normal try catch block, and then apply the logic to continue or not, based on if an exception was thrown or not.
Also, a secondary way is to check if a file is not of zero size, but purely as a secondary check.

Create a loop and try to open the file
If exception occur make your thread sleep for some seconds
and repeat the process

When the external process is complete, does it close? If so, you could modify the solution found here. It would look something like the following:
//While the process is found running
while (System.Diagnostics.Process.GetProcessesByName(process).Length != 0)
{
//Sleep for three seconds
System.Threading.Thread.Sleep(3000);
}
//open file for reading
If the process doesn't close when complete, however, the above won't work. You could try placing the following in a loop that tries to open the file exclusively until it's successful:
System.IO.File.Open(PathToFile, FileMode.Open, FileAccess.Read, FileShare.None);
Both of these methods should also have some kind of count added to ensure that they eventually stop trying and error out, or else you could end up with an infinite loop.

Check If File Is In Use By Other Instances of Executable Run

Before I go into too detail, my program is written in Visual Studio 2010 using C# .Net 4.0.
I wrote a program that will generate separate log files for each run. The log file is named after the time, and accurate up at millisecond (for example, 20130726103042375.log). The program will also generate a master log file for the day if it has not already exist (for example, *20130726_Master.log*)
At the end of each run, I want to append the log file to a master log file. Is there a way to check if I can append successfully? And retry after Sleep for like a second or something?
Basically, I have 1 executable, and multiple users (let's say there are 5 users).
All 5 users will access and run this executable at the same time. Since it's nearly impossible for all user to start at the exact same time (up to millisecond), there will be no problem generate individual log files.
However, the issue comes in when I attempt to merge those log files to the master log file. Though it is unlikely, I think the program will crash if multiple users are appending to the same master log file.
The method I use is
File.AppendAllText(masterLogFile, File.ReadAllText(individualLogFile));
I have check into the lock object, but I think it doesn't work in my case, as there are multiple instances running instead of multiple threads in one instance.
Another way I look into is try/catch, something like this
try
{
stream = file.Open(FileMode.Open, FileAccess.ReadWrite, FileShare.None);
}
catch {}
But I don't think this solve the problem, because the status of the masterLogFile can change in that brief millisecond.
So my overall question is: Is there a way to append to masterLogFile if it's not in use, and retry after a short timeout if it is? Or if there is an alternative way to create the masterLogFile?
Thank you in advance, and sorry for the long message. I want to make sure I get my message across and explain what I've tried or look into so we are not wasting anyone's time.
Please let me know if there's anymore information I can provide to help you help me.

Your try/catch is the way to do things. If the call to File.Open succeeds, then you can write to to the file. The idea is to keep the file open. I would suggest something like:
bool openSuccessful = false;
while (!openSuccessful)
{
try
{
using (var writer = new StreamWriter(masterlog, true)) // append
{
// successfully opened file
openSuccessful = true;
try
{
foreach (var line in File.ReadLines(individualLogFile))
{
writer.WriteLine(line);
}
}
catch (exceptions that occur while writing)
{
// something unexpected happened.
// handle the error and exit the loop.
break;
}
}
}
catch (exceptions that occur when trying to open the file)
{
// couldn't open the file.
// If the exception is because it's opened in another process,
// then delay and retry.
// Otherwise exit.
Sleep(1000);
}
}
if (!openSuccessful)
{
// notify of error
}
So if you fail to open the file, you sleep and try again.
See my blog post, File.Exists is only a snapshot, for a little more detail.

I would do something along the lines of this as I think in incurs the least overhead. Try/catch is going to generate a stack trace(which could take a whole second) if an exception is thrown. There has to be a better way to do this atomically still. If I find one I'll post it.

File.Create seems to lock the file. How can I avoid this?

I have 2 threads that communicate by sending files to each other. When Thread #1 does a
// Thread #1
File.Create(#"C:\somedir\response.done");
Then Thread #2 is supposed to delete it.
// Thread #2
while (!File.Exists(#"C:\somedir\response.done"))
Thread.Sleep(100);
while (File.Exists(#"C:\somedir\response.done"))
{
try {
File.Delete(#"C:\somedir\response.done");
}
catch { Thread.Sleep(1000); };
}
However, the file seems to be locked. There is generated a response.done file in the directory, but it is never deleted. When I try to manually remove it, then
"The action cannot be completed because the file is open in MyProgram. Close the file and try again."
How can I avoid this?

File.Create returns a FileStream. So... close it:
using(File.Create(path)) {
// add contents here if needed
}
The using ensures it is Dispose()d, hence closed. Note: it is also possible that some AV systems will interfere with file access, but that is usually not a huge problem.

You need to close FileStream created by File.Create(#"C:\somedir\response.done");.

You can also use .Close()
So your code would become
// Thread #1
File.Create(#"C:\somedir\response.done").Close();

change your code to
// Thread #1
using (FileStream FS = File.Create(#"C:\somedir\response.done") ) {};

New FileStream already closed/disposed?

I open a FileStream with FileMode.Open and FileAccess.Read. Shortly after that I call a function to handle the file's contents. I use Invoke to make the call because the call comes from a Thread and the function has to put the results on a Form. The function accepts any kind of Stream (I call it with MemoryStreams too without a problem) and uses XmlTextReader to read the XML in the FileStream, but on rare occasions for unknown reasons even the first Read() throws an ObjectDisposedException and the stream's CanRead property returns false if the stream was already closed.
In the Thread the FileStream is a local using variable, so I don't think another threads should be able to close it, and I don't close it until the Invoke returned. There are no Exceptions thrown so the file is definetly there (since there is no FileNotFoundException) and should be accessed properly (since there is no UnauthorizedAccessException and IOException).
How could my FileStream still look closed sometimes just after opened?
(It might matter that I'm running my code on a Windows CE 5 device with Compact Framework 3.5 and I wasn't able to reproduce the same behaviour on my desktop PC with XP yet.)
EDIT:
I know, that this Invoke is ugly but that alone can't be a reason to fail, can it? (And, in most of the cases it doesn't fail at all.)
//the code in the thread
//...
using (FileStream fs = File.Open(assemblyPath + "\\white.xml", FileMode.Open, FileAccess.Read))
{
mainForm.Instance.Invoke(new DataHandler(mainForm.Instance.handleData), new object[] { fs });
}
//...
//and the handler
public void handleData(Stream stream)
{
infoPanel.SuspendLayout();
try
{
using (XmlTextReader xml = new XmlTextReader(stream))
{
//it doesn't matter what is here
}
}
catch{}
}

There's one reason I can think of: the worker thread got aborted. This will run the finally block generated by the using statement and close the file. How it could be aborted is a secondary question. Is the thread's IsBackground property set to true? Is the program bombing on an unhandled exception elsewhere and shutting down? Just guesses of course.

Sure, this is expected behavior. You call Invoke, which marshals the call to another thread. The calling thread then continues to run and the using block exits, calling Dispose on the stream. This Dispose is happening before you are done (and maybe before you start) using the stream in the UI thread. The exact timing of these actions is going to depend on processor load and some other factors, but it's certainly unsafe.
Either don't put the stream in a using block or better yet have the thread do the read and pass the results to the UI via Invoke.
EDIT
As Hans points out in the comment, the above explanation should be for a BeginInvoke call, which underneath calls PostMessage. Invoke, on the other hand, uses SendMessage. Both propbably uses some WM_COPYDATA shenanigans (I've not looked to see) to marshal the data.
The Invoke call should be executing the entire handler you have posted, though the behavior you see indicates otherwise. From the code you posted there's no real way for us to determine what is closing the stream.
I would still refactor what you've done here because right now you're tying up both the UI and worker threads with the reader operation. I'd do the read work in the worker thread and then pass the results to the UI. This would decrease the odds of the reader work causing UI choppiness and would eliminate the possibility of the stream getting closed while you're reading from it.

I saw the same issue on some embedded board (ARM) I'm working on. Then I created a little test.
The following code (not involving any Threads!) crashes:
using (var w = new StreamWriter(File.Create("file.txt"), System.Text.Encoding.UTF8))
{
for (int i = 0; i < 1000; i++)
{
w.WriteLine("Test");
}
}
This code however does not crash:
using (var w = File.CreateText("file.txt"))
{
for (int i = 0; i < 1000; i++)
{
w.WriteLine("Test");
}
}
So, my guess can only be that the underlying native code treats text files differently than when you open the file using File.Create(). Both files are then written in UTF-8, so there is no difference about the encoding.
BTW: sorry I'm one year late on the answer, but I hope it'll help somebody