We Keep Coding

Best practice for writing big files

I need to write a big file in my project.
What I learned:
I should NOT write the big file directly to the destination path,
because this may leave a incomplete file in case the app crash while writing it.
Instead, I should write to a temporary file and move (rename) it. (called atomic file operation)
My code snippet:
[NotNull]
public static async Task WriteAllTextAsync([NotNull] string path, [NotNull] string content)
{
string temporaryFilePath = null;
try {
temporaryFilePath = Path.GetTempFileName();
using (var stream = new StreamWriter(temporaryFilePath, true)) {
await stream.WriteAsync(content).ConfigureAwait(false);
}
File.Delete(path);
File.Move(temporaryFilePath, path);
}
finally {
if (temporaryFilePath != null) File.Delete(temporaryFilePath);
}
}
My Question:
The file will be missing if the app crashes between File.Delete and File.Move. Can I avoid this?
Is there any other best practice for writing big files?
Is there any suggestion on my code?

The file will be missing if the app crashes between File.Delete and File.Move. Can I avoid this?
Not that I'm aware of, but you can detect it - and if you use a more predictable filename, you can recover from that. It helps if you tweak the process somewhat to use three file names: the target, a "new" file and an "old" file. The process becomes:
Write to "new" file (e.g. foo.txt.new)
Rename the target file to the "old" file (e.g. foo.txt.old)
Rename the "new" file to the target file
Delete the "old" file
You then have three files, each of which may be present or absent. That can help you to detect the situation when you come to read the new file:
No files: Nothing's written data yet
Just target: All is well
Target and new: App crashed while writing new file
Target and old: App failed to delete old file
New and old: App failed after the first rename, but before the second
All three, or just old, or just new: Something very odd is going on! User may have interfered
Note: I was unaware of File.Replace before, but I suspect it's effectively just a simpler and possibly more efficient way of doing the code you're already doing. (That's great - use it!) The recovery process would still be the same though.

You can use File.Replace instead of deleting and moving files. In case of hard fault (electricity cut or something like this) you will always lost data, you have to count with that.

Testing to see if a file is open doesn't give expected results

I wanted to test if a particular file is already open before trying to launch it, so I came up with this:
public void LaunchErrorLog()
{
var logFile = ConfigurationManager.AppSettings["Log"];
if (IsLogOpen(logFile))
return; //figure out how to give focus to other app later
var psi = new ProcessStartInfo(logFile);
psi.WindowStyle = ProcessWindowStyle.Maximized;
Process.Start(psi);
}
private bool IsLogOpen(string p)
{
try
{
using (var s = new FileStream(p, FileMode.Open, FileAccess.Read)){}
}
catch (IOException)
{
return true;
}
return false;
}
I'm testing using a .log file (just a text file) that I've got open in Baretail. The method always returns false regardless of whether or not the file is open. I tried opening it in Notepad, and it still returns false.
Basically, the end objective is to give focus to the application that has the file open, or launch the application/file if it's not already open. But this is always false so it just goes on and launches a new instance of Baretail with the file open.
Also tried the top solution found here;
Is there a way to check if a file is in use?

Notepad is a bad test application because it does not hold a lock open on the file. It streams in the file and closes the lock. Use Word to do the test and you will see different results. A file is only locked if a handle is kept open by an application. Word will lock files. The same holds true for "Baretail".
In other words, if "Baretail" streams the file in and closes the lock then this test will not work. You could do something hacky such as sniff around Win32 objects...window handles and title bars to extract the information...but be warned this kind of UI hacking is tricky and I would not consider the information to be reliable. There's no stopping some other program from using similar text in their title bars per se.

I believe that that method is always returning false because you are opening for read. Even if the file is open for write elsewhere, you should be able to read that file (thus no exception is thrown).
Try using a different FileAccess opetion ie
FileAccess.ReadWrite
Or you can also try different combinations of FileMode and FileOption. (Sorry not in front of a dev machine at this point)

Why isn't this working?

I am trying to just write an array of strings to a file, which SHOULD normally be an easy thing to do. However the following trivial code is throwing an IOException saying that the file is in use by another process. The problem is, the file doesn't even exist until this code is run. And I can guarantee you that there is no other process using the file. So how do I convince the stupid .NET framework that the file is not in use by another process and that it is okay to continue? Because this really shouldn't be that hard.
StreamWriter writer = new StreamWriter(ListFileName);
foreach (string s in InfoLineList)
{
writer.WriteLine(s);
}

This might be because you're not closing the stream when you're done with it, so some handle is getting stuck open somewhere. Perhaps the code is part of a web app, and the web server process keeps that lock around, or the code is being run multiple times. I'd recommend using the stream in a using block:
using(StreamWriter writer = new StreamWriter(ListFileName))
{
foreach (string s in InfoLineList)
{
writer.WriteLine(s);
}
}
This will make sure the StreamWriter is disposed of properly.
If you really want to know what has the file open, use SysInternal's Handle tool to check. I'd be willing to bet it's your own program.
Finally, as I said in my comments, the File.WriteAllLines() method can write an enumerable list of strings to a file all at once:
File.WriteAllLines(ListFileName, InfoListList);

File.Delete fails on attempting to delete an opened file but deletes on process exit?

While calling File.Delete(file_path) for a file which is opened in another process, the attempt fails with cannot access error, but on exiting the app (from where the attempt was made) the file gets deleted automatically. After File.Delete attempt, the path gets inaccessible for any other operation.
I need to either fail on delete and let the path be accessible through the app or delete the file completely, not on app exit.
Well my code is this :
private bool DeleteFilesAsync(string FileToDelete)
{
try
{
//Set file's attribute to normal if it is ReadOnly file
File.SetAttributes(FileToDelete, FileAttributes.Normal);
File.Delete(FileToDelete);
//Some bussiness logic to update file's status in database
return true;
}
catch (Exception ex)
{
//log the error
return false;
}
}
Something wrong here?

Windows does have a delete-on-last-close feature. All deletes are handled that way. The only reason that this almost never occurs in practice is because most apps do not open the file with FILE_SHARE_DELETE so it is not possible to delete a used file. But at the Kernel level all deletes are delete-on-close.
There might be a way to clear the delete flag by calling NtSetInformationFile to set the FileDispositionInfo class. This assumes that it is possible to clear the delete flag. It might well be.
Anyway, a more sane approach would be to open the file before deleting it to ensure that exclusive access is available:
using(new FileStream(path, FileMode.Open, FileAccess.ReadWrite, FileShare.None)) { }
After this line passes without exception, we know that the file was unused at the point of opening it. Of course, it might be opened by someone else immediately after this line, but maybe this solution is enough for you.

how to check if a file is being accessed by any other processs and release it?

I am trying to delete/open/edit some files in my C# .Net application.Sometimes i get exception stating the file/directory is being accessed by another process.Is there a way to check if a file/directory is being accessed by process and try to release the file from that process?

No. The only way to do this is to try to access the file, and handle the IOException.
Realistically this is the only safe way anyway. Suppose there was a IsFileInUse() method, and you called it, and it returned "nope, nobody's using that file," and you went ahead and accessed the file. The problem is that in the meantime some other process might have locked or deleted the file. So you'd need to put exception handling around your attempt to access the file anyway. The "test by acquiring" model is the only one that is 100% reliable.
If a file is in use by another process, .NET doesn't provide a way of determining which other process that might be. I believe this would require some pretty low-level unmanaged code though I could be wrong. It is a very low-level operation, if it is possible at all, to "release the file from that process" because that would violate the other process' expectations -- e.g. it thinks it is allowed to write to the file but you have deleted the file and garbaged the handle. I believe you would need to terminate the other process if it's not willing to give up its lock voluntarily.

First, I suppose there are 2 things that may help you:
consider using FileAccess and FileShare flags when opening files
if data from the file is needed only withing the scope of the function use the construction
using(FileStream stream = File.Open(...)) { <file operations> }
this will ensure that file is closed immediately after exiting 'using' block, and not when FileStream object is collected by GC.
Second, there is an unsafe way to get processes that use the file. It is based on debugging features provided by windows. The main idea is to get all system handles and iterate through them to find which are the files handle and additional information. This is done using functions that I'm not sure are documented. If you are interested use google to find more information, but I do not think it is not a good way.

public bool IsInUse(string path)
{
bool IsFree = true;
try
{
//Just opening the file as open/create
using (FileStream fs = new FileStream(path, FileMode.OpenOrCreate))
{
//we can check by using
fs.CanRead // or
fs.CanWrite
}
}
catch (IOException ex)
{
IsFree = false;
}
return IsFree;
}
string path = "D:\\test.doc";
bool IsFileFree = IsInUse(path);