Before I go into too detail, my program is written in Visual Studio 2010 using C# .Net 4.0.
I wrote a program that will generate separate log files for each run. The log file is named after the time, and accurate up at millisecond (for example, 20130726103042375.log). The program will also generate a master log file for the day if it has not already exist (for example, *20130726_Master.log*)
At the end of each run, I want to append the log file to a master log file. Is there a way to check if I can append successfully? And retry after Sleep for like a second or something?
Basically, I have 1 executable, and multiple users (let's say there are 5 users).
All 5 users will access and run this executable at the same time. Since it's nearly impossible for all user to start at the exact same time (up to millisecond), there will be no problem generate individual log files.
However, the issue comes in when I attempt to merge those log files to the master log file. Though it is unlikely, I think the program will crash if multiple users are appending to the same master log file.
The method I use is
File.AppendAllText(masterLogFile, File.ReadAllText(individualLogFile));
I have check into the lock object, but I think it doesn't work in my case, as there are multiple instances running instead of multiple threads in one instance.
Another way I look into is try/catch, something like this
try
{
stream = file.Open(FileMode.Open, FileAccess.ReadWrite, FileShare.None);
}
catch {}
But I don't think this solve the problem, because the status of the masterLogFile can change in that brief millisecond.
So my overall question is: Is there a way to append to masterLogFile if it's not in use, and retry after a short timeout if it is? Or if there is an alternative way to create the masterLogFile?
Thank you in advance, and sorry for the long message. I want to make sure I get my message across and explain what I've tried or look into so we are not wasting anyone's time.
Please let me know if there's anymore information I can provide to help you help me.
Your try/catch is the way to do things. If the call to File.Open succeeds, then you can write to to the file. The idea is to keep the file open. I would suggest something like:
bool openSuccessful = false;
while (!openSuccessful)
{
try
{
using (var writer = new StreamWriter(masterlog, true)) // append
{
// successfully opened file
openSuccessful = true;
try
{
foreach (var line in File.ReadLines(individualLogFile))
{
writer.WriteLine(line);
}
}
catch (exceptions that occur while writing)
{
// something unexpected happened.
// handle the error and exit the loop.
break;
}
}
}
catch (exceptions that occur when trying to open the file)
{
// couldn't open the file.
// If the exception is because it's opened in another process,
// then delay and retry.
// Otherwise exit.
Sleep(1000);
}
}
if (!openSuccessful)
{
// notify of error
}
So if you fail to open the file, you sleep and try again.
See my blog post, File.Exists is only a snapshot, for a little more detail.
I would do something along the lines of this as I think in incurs the least overhead. Try/catch is going to generate a stack trace(which could take a whole second) if an exception is thrown. There has to be a better way to do this atomically still. If I find one I'll post it.
Related
Hello and thanks for your help.
This time I am having a curious problem with a program (C#) I am writing and would like to hear your advice.
I am writing a normal program (not multithreaded) but then added a timer (System.Timers.Timer)
Also I am using a StreamWriter to write on a file. I open this like this
StreamWriter logStream=new StreamWriter(filename, true);
meaning that if the file exists, it appends , if not it creates.
Later I write in the file like this
logStream.WriteLine(message);
However, I write to the stream from both the main function and from the function that is called by the timer.
the problem symptoms
My program is throwing an error sometimes when I flush or write the stream saying that "Can not access a closed file" and other times "Can not access a closed TextWriter... (What is a "TextWriter"?)
However curiously, the file keeps being written without problem. (Even the "can not access a closed file" message is written in the supposed closed file)
I am not familiar with the inner workings of a Timer. (I suppose it runs a separate thread?)
My question is
Is it possible to use a StreamWriter from several threads? (in this case the main one and the Timer one)
Is it possible that there is happening a race condition or some problem like that?
One more thing: I made a logic mistake and close and reopen the file every time I want to write on it. Yes, it is a mistake and I should correct it. But maybe if I correct this the error I described above will disappear masking a more serious flaw.
My suspicions is that since I am closing and opening the file every time I write on it, maybe the both threads try to access them on a wrong time
Any help will be greatly appreciated
Closing and opening you file under this scenario will create a race condition like you suspect. You cannot keep the stream open and pass the object to the thread because you might end up with a similar issue if you call from different thread. Your best solution remain using a thread safe method that will write what you send to it.
the methods are static because the lock has to be accessible from all instance of the class.
private static ReaderWriterLockSlim readerWriterLockSlim = new ReaderWriterLockSlim();
public static void AppendToFile(string path, string text)
{
// Set to locked (other thread will freeze here until object is unlocked
readerWriterLockSlim.EnterWriteLock();
try
{
// Write that will append to the file
using (StreamWriter sw = File.AppendText(path))
{
// append the text
sw.WriteLine(text);
sw.Close();
}
}
finally
{
// Clear the lock
readerWriterLockSlim.ExitWriteLock();
}
}
I have the following function, that accepts a string, and logs the string content to a log file.
private static void LogEvent(string sEvent)
{
sEvent = DateTime.Now.ToString("yyyy-MM-dd hh:mm:ss") + "|" + sEvent;
Console.WriteLine(sEvent);
try
{
using (StreamWriter oStreamWriter = new System.IO.StreamWriter("MyService_" + DateTime.Now.ToString("yyyyMMdd") + ".log", true))
{
oStreamWriter.WriteLine(sEvent);
}
}
catch
{
}
}
When the program calling the function is manually run from the command line or by double clicking executable, the log file is either created or appended to.
The problem I have is I've set this program to be called from a task scheduler every 10 minutes, and for some reason the code is executing correctly, except the program is not creating or appending to the log file.
The scheduled task is calling the program using the same user permissions as when I manually ran the program.
Why is this happening, and how can I fix it.
You're currently trying to write to the process's current working directory - which may well be something like C:\Windows\System32 when it's executed by the task scheduler. You're not going to be able to write there.
Specify an absolute filename and I think you'll be fine. It's not clear where you do want to write to, but you should think about that carefully - ideally you should separate your executable files from the data that it generates. Consider using Environment.GetFolderPath in conjunction with a suitable SpecialFolder member (e.g. ApplicationData.)
Note that using File.AppendAllText would make the code simpler, mind you.
The answer by Jon Skeet is right that the problem is with working directory permission and you should use SpecialFolder to write logs but I will like to add one more point,
Never have empty catch blocks like you did catch { } for me this catch statements sometime hides so many unknown problems.
I think they are something like teaser which says,
catch
{
// WHO CARES
}
Never hide this sort of exceptions under the carpet, take some action or inform the end user or flag somewhere what has happened. This will help the developer in later stages.
This question already has answers here:
Detecting whether a file is locked by another process (or indeed the same process) [duplicate]
(4 answers)
Closed 9 years ago.
In my program I pass the path to a file which may be still being written to by another process (independent external non-c#).
How can I check the file is fully readable and accessible before I actually send the path back to the client? FileIOPermission with Demand for reading did not work for me, so I am wondering if there is some other way of doing this without attempting to read the whole file upfront.
Thank you.
The trouble with checking to see if a file is accessible and fully readable, and then opening the file for reading is that potentially, the state could change between the check and the open where you actually want to be doing something with it.
My suggestion is to go ahead and open the file for reading, being sure to catch the appropriate exception if a problem occurs. You can then wait a little and try again, or do something else.
Remember that just because your program has a sequence of events that are logical to you, many things are going on in an operating system and the state can easily change between two lines of code that seem watertight for you, but have an epoch between them at a multitasking level.
Moo-Juice is right. You can't check whether the file can be read successfully and then open it for reading... things can change and you might get an exception anyway. Best to just read it and catch the exception.
public bool TryReadFile(String path, out String contentsOfFile)
{
try
{
// Try reading file
contentsOfFile = File.ReadAllText(path);
// Success! Yay!
return true;
}
catch (IOException)
{
// Oops! Can't read that file!
// Return some default value and let the caller know we failed
contentsOfFile = String.Empty;
return false;
}
Best way would be to read the file in a normal try catch block, and then apply the logic to continue or not, based on if an exception was thrown or not.
Also, a secondary way is to check if a file is not of zero size, but purely as a secondary check.
Create a loop and try to open the file
If exception occur make your thread sleep for some seconds
and repeat the process
When the external process is complete, does it close? If so, you could modify the solution found here. It would look something like the following:
//While the process is found running
while (System.Diagnostics.Process.GetProcessesByName(process).Length != 0)
{
//Sleep for three seconds
System.Threading.Thread.Sleep(3000);
}
//open file for reading
If the process doesn't close when complete, however, the above won't work. You could try placing the following in a loop that tries to open the file exclusively until it's successful:
System.IO.File.Open(PathToFile, FileMode.Open, FileAccess.Read, FileShare.None);
Both of these methods should also have some kind of count added to ensure that they eventually stop trying and error out, or else you could end up with an infinite loop.
Environment - C#, .NET 4.0, WPF, VS2010
I have an app that uses a keyboard hook to detect when a combination of four keys is pressed. When this occurs it calls Configuration.Save(), which has a call to myConfigDataSet.WriteXml(myConfigFile). And then on the next line it calls App.Current.Shutdown().
About half the time it works as expected. But many times it would insert XML content right into the middle of a previously existing configuration file, resulting in corrupt data.
I was able to fix the above issue by using...
if(File.Exists(myConfigFile)) { File.Delete(myConfigFile) }
...on the line just above the call to myConfigDataSet.WriteXml(myConfigFile)
But now many times it just writes a 0KB size file. I am pretty sure that all of my problems are being caused by App.Current.Shutdown() not waiting for the call to myConfigDataSet.WriteXml(myConfigFile) to finish.
Shouldn't this call block execution until the file has been written to disk? ...apparently not!!!
As a workaround I've already tried inserting Thread.Sleep(1000) to create a 1 second delay just before App.Current.Shutdown. But now sometimes the app errors out with a "StackOverFlow" exception...!!!
What would you guys recommend as a fix or workaround for this? What am I doing wrong?
You can't stop a App.Current.Shutdown
Instead, use Application.Current.MainWindow.Close().
In this case, you can intercept in the Close event, process what you need to process, and then you could call App.Current.Shutdown
You can try to put the call to myConfigDataSet.WriteXml(myConfigFile) in a try/finally block. The finally block is executed completely even when the thread is aborted (see here). Not sure if it works with App.Current.Shutdown though.
So instead of:
myConfigDataSet.WriteXml(myConfigFile)
Do the following:
try {
// Empty...
} finally {
myConfigDataSet.WriteXml(myConfigFile)
}
I need to accomplish the following task:
Attempt to move a file. If file is locked schedule for moving as soon as it becomes available.
I am using File.Move which is sufficient for my program. Now the problems are that:
1) I can't find a good way to check if the file I need to move is locked. I am catching System.IO.IOException but reading other posts around I discovered that the same exception may be thrown for different reasons as well.
2) Determining when the file gets unlocked. One way of doing this is probably using a timer/thread and checking the scheduled files lets say every 30 seconds and attempting to move them. But I hope there is a better way using FileSystemWatcher.
This is a .net 3.5 winforms application. Any comments/suggestions are appreciated. Thanks for attention.
You should really just try and catch an IOException. Use Marshal.GetHRForException to check for the cause of the exception.
A notification would not be reliable. Another process might lock the file again before File.Move is executed.
One possible alternative is by using MoveFileEx with a MOVEFILE_DELAY_UNTIL_REBOOT flag. If you don't have access to move the file right now, you can schedule it to be moved on the next reboot when it's guaranteed to be accessible (the moving happens very early in the boot sequence).
Depending on your specific application, you could inform the user a reboot is necessary and initiate the reboot yourself in addition to the moving scheduling.
It's simple:
static void Main(string[] args)
{
//* Create Watcher object.
FileSystemWatcher watcher = new FileSystemWatcher(#"C:\MyFolder\");
//* Assign event handler.
watcher.Created += new FileSystemEventHandler(watcher_Created);
//* Start watching.
watcher.EnableRaisingEvents = true;
Console.ReadLine();
}
static void watcher_Created(object sender, FileSystemEventArgs e)
{
try
{
File.Move(e.FullPath, #"C:\MyMovedFolder\" + e.Name);
}
catch (Exception)
{
//* Something went wrong. You can do additional proceesing here, like fire-up new thread for retry move procedure.
}
}
This is not specific to your problem, but generally you will always need to retain the 'try it and gracefully deal with a failure' mode of operation for this sort of action.
That's because however clever your 'detect that the file is available' mechanism is, there will always be some amount of time between you detecting that the file is available and moving it, and in that time someone else might mess with the file.
The scheduled retry on exception (probably increasing delays - up to a point) is probably the simplest way to achieve this (your (2) ).
To do it properly you're going to have to drop to system level (with Kernel code) hooks to trap the file close event - which has its own idiosynchrases. It's a big job - several orders of magnitude more complex than the scheduled retry method. It's up to you and your application case to make that call, but I don't know of anything effective in between.
Very old question, but google led me here, so when I found a better answer I decided to post it:
There's a nice code I found in the dotnet CLI repo:
/// <summary>
/// Run Directory.Move and File.Move in Windows has a chance to get IOException with
/// HResult 0x80070005 due to Indexer. But this error is transient.
/// </summary>
internal static void RetryOnMoveAccessFailure(Action action)
{
const int ERROR_HRESULT_ACCESS_DENIED = unchecked((int)0x80070005);
int nextWaitTime = 10;
int remainRetry = 10;
while (true)
{
try
{
action();
break;
}
catch (IOException e) when (e.HResult == ERROR_HRESULT_ACCESS_DENIED)
{
Thread.Sleep(nextWaitTime);
nextWaitTime *= 2;
remainRetry--;
if (remainRetry == 0)
{
throw;
}
}
}
}
There is also a method for just IOException. Here's the usage example:
FileAccessRetrier.RetryOnMoveAccessFailure(() => Directory.Move(packageDirectory.Value, tempPath));
Overall, this repo contains a lot of interesting ideas for file manipulations and installation/removal logic, like TransactionalAction, so I recommend it for reviewing. Unfortunately, these functions are not available as NuGet package.
Have a look at the FileSystemWatcher.
http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher(VS.90).aspx
Listens to the file system change
notifications and raises events when a
directory, or file in a directory,
changes