We Keep Coding

C# - Unable to delete a file after copying and overwritting

I am working on an application which reads paths of all the text files from a folder into a list. It reads each file, creates a temporary output file, overwrites the original file with temporary output file and deletes the temporary output file.
Following is my code:
foreach (string lF in multipleFiles)
{
int lineNumber = 0;
using (StreamReader sr = new StreamReader(lF))
{
using (StreamWriter sw = new StreamWriter(lF + "Output"))
{
while (!sr.EndOfStream)
{
//LOGIC
sw.WriteLine(line);
}
}
}
File.Copy(lF + "Output", lF, true);
//File.Delete(lF + "Output");
try
{
File.Delete(lF + "Output"); <--- ERROR HERE
}
catch(Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
I am unable to delete the temporary output file due to the following error:
{"The process cannot access the file '' because it is being
used by another process."}
The error does not occur for every file but only a few. None of the files are open or being used by any other application.
How can the temporary file be deleted?
UPDATE: Refereed to Does FileStream.Dispose close the file immediately?
Added Thread.Sleep(1) before File.Delete(), The issue still exists. Tried increasing the sleep value to 5. No luck.

You always run the risk that an virus scanner or some other driver in the stack still holds on to that file or its directory entry. Use some retry mechanisms but that still doesn't guarantee you'll be able to remove that file as the file operations are not atomic, so any process can open that file between your calls trying to delete it.
var path = lf + "Output";
// we iterate a couple of times (10 in this case, increase if needed)
for(var i=0; i < 10; i++)
{
try
{
File.Delete(path);
// this is success, so break out of the loop
break;
} catch (Exception exc)
{
Trace.WriteLine("failed delete #{0} with error {1}", i, exc.Message);
// allow other waiting threads do some work first
// http://blogs.msmvps.com/peterritchie/2007/04/26/thread-sleep-is-a-sign-of-a-poorly-designed-program/
Thread.Sleep(0);
// we don't throw, we just iterate again
}
}
if (File.Exists(path))
{
// deletion still not happened
// this is beyond the code can handle
// possible options:
// store the filepath to be deleted on startup
// throw an exception
// format the disk (only joking)
}
Code slightly adapted from my answer here but that was in a different context.

How to rewrite txt file-why my code doesn't work

I want rewrite my txt file(I need delete lines). I use this code:
string cesta3 = cesta + "database.txt";
string file = new StreamReader(cesta3).ReadToEnd();
var linesToKeep = File.ReadLines(cesta3)
.Where(l => l != "Meno: " + textBox1.Text + " PN " + textBox2.Text);
But I don't know how to save my file with that same name.I try this:
File.WriteAllLines(cesta3, linesToKeep); // <== exception
var tempFile = Path.GetTempFileName();
File.Move(tempFile + ".txt", cesta3);
But it throws exception:
Additional information: Access to the path "'C:\Users\Lenovo\documents\visual studio 2015\Projects\attendancer\attendancer\bin\Debug\database.txt‌​' is denied."
How can I do ?

This line of code locks the file and prevent it from moving: string file = new StreamReader(cesta3).ReadToEnd();.
Fix:
you can properly close StreamReader after reading file text from it with using
alternatively since there is nothing in the sample using that StreamReader or result of ReadToEnd (file variable) you can simply remove that line.
Side note: File.Move will throw more exceptions after you are done with the first one as source file is unlikely to exist - I'm not sure what you were trying to do with that call.

Out of memory error archiving a log file

I am having a problem with a console job that runs and creates a daily log file that I archive at midnight.
This creates a blank log file for the next day and an archived file with yesterdays date in the name and the contents of the old file for debugging issues I may have had and not known about until the day after.
However since I cranked up the BOT's job I have been hitting issues with System Out of Memory errors when I try and archive the file.
At first I was just not able to get an archived file at all then I worked out a way to get at least the last 100,000 lines which is not nearly enough.
I wrap everything in 3 try/catches
I/O
System out of memory
standard exception
However it's always the OutOfMemoryException that I get e.g
System.OutOfMemoryException Error: Exception of type 'System.OutOfMemoryException' was thrown.;
To give you an example of size 100,000 lines of log is about 11MB file
A standard full log file can be anything from 1/2 a GB to 2GB
What I need to know is this:
a) what size of a standard text file will throw an out of memory error when trying to use File.ReadAllText or a custom StreamReader function I call ReadFileString e.g
public static string ReadFileString(string path)
{
// Use StreamReader to consume the entire text file.
using (StreamReader reader = new StreamReader(path))
{
return reader.ReadToEnd();
}
}
b) is it my computers memory (I have 16GB RAM - 8GB use at time of copying) or the objects I am using in C# that are failing with the opening and copying of files.
When archiving I first try with my custom ReadFileString function (see above), if that returns 0 bytes of content I try File.ReadAllText and then if that fails I try a custom function to get the last 100,000 lines, which is really not enough for debugging errors earlier in the day.
The log file starts at midnight when a new one is created and logs all day. I never used to have out of memory errors but since I have turned up the frequency of method calls the logging has expanded which means the file sizes have as well.
This is my custom function for getting the last 100,000 lines. I am wondering how many lines I could get without IT throwing an out of memory error and me not getting any contents of the last days log file at all.
What do people suggest for the maximum file size for various methods / memory needed to hold X lines, and what is the best method for obtaining as much of the log file as possible?
E.G some way of looping line by line until an exception is hit and then saving what I have.
This is my GetHundredThousandLines method and it logs to a very small debug file so I can see what errors happened during the archive process.
private bool GetHundredThousandLines(string logpath, string archivepath)
{
bool success = false;
int numberOfLines = 100000;
if (!File.Exists(logpath))
{
this.LogDebug("GetHundredThousandLines - Cannot find path " + logpath + " to archive " + numberOfLines.ToString() + " lines");
return false;
}
var queue = new Queue<string>(numberOfLines);
using (FileStream fs = File.Open(logpath, FileMode.Open, FileAccess.Read, FileShare.Read))
using (BufferedStream bs = new BufferedStream(fs)) // May not make much difference.
using (StreamReader sr = new StreamReader(bs))
{
while (!sr.EndOfStream)
{
if (queue.Count == numberOfLines)
{
queue.Dequeue();
}
queue.Enqueue(sr.ReadLine() + "\r\n");
}
}
// The queue now has our set of lines. So print to console, save to another file, etc.
try
{
do
{
File.AppendAllText(archivepath, queue.Dequeue(), Encoding.UTF8);
} while (queue.Count > 0);
}
catch (IOException exception)
{
this.LogDebug("GetHundredThousandLines - I/O Error accessing daily log file with ReadFileString: " + exception.Message.ToString());
}
catch (System.OutOfMemoryException exception)
{
this.LogDebug("GetHundredThousandLines - Out of Memory Error accessing daily log file with ReadFileString: " + exception.Message.ToString());
}
catch (Exception exception)
{
this.LogDebug("GetHundredThousandLines - Exception accessing daily log file with ReadFileString: " + exception.Message.ToString());
}
if (File.Exists(archivepath))
{
this.LogDebug("GetHundredThousandLines - Log file exists at " + archivepath);
success = true;
}
else
{
this.LogDebug("GetHundredThousandLines - Log file DOES NOT exist at " + archivepath);
}
return success;
}
Any help would be much appreciated.
Thanks

try:
keep the queue and stream position in class scope, try GC.Collect() when getting out of memory exception and call function again. seek stream to last position and continue.
or:
use one database like sqlite and keep newest 100000 record in each table.

IO Exception - File being used by another process (Can't open file following directory creation)

I have got a project on the go that monitors patients for a vet while they are being operated on and writes the result to a text file. While I was experimenting with the outputting I just let the files save in the Debug folder, which worked fine. However, I've now created a full directory that creates or opens a main folder, and then a sub folder (based on input text from the program), to save the text file into.
private void createDirectory()
{ //create output file in this folder using owner name and current date
//main folder path (contains all files output from system)
string rootDirectory = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments) + "\\Horse Monitoring Records";
//sub folder path (each patient has individual subfolder)
string subDirectory = rootDirectory + "\\" + txtPatName.Text + "-" + txtOwnerName.Text;
//file name (patient has file created for each operation)
fileName = subDirectory + "\\" + txtOwnerName.Text + "-" + DateTime.Now.Date.ToString("ddMMyyyy") + ".txt";
if (!Directory.Exists(rootDirectory)) //if main folder does not exist...
{
Directory.CreateDirectory(rootDirectory); //create it in My Documents
}
if (!Directory.Exists(subDirectory)) //if patient sub folder does not exist...
{
Directory.CreateDirectory(subDirectory); //create it in Patient-Owner format
}
if (!File.Exists(fileName)) //if monitoring file does not exist...
{
File.Create(fileName); //create it in Owner-Date format
}
}
This stage works fine, but as soon as you try to save some data to the text file, it throws to a run time error stating that
The file cannot be accessed because it is being used by another process.
The exception is brought up here:
private void saveFileDetails()
{
//Once case details have been entered, create new file using these details and add data input structure
StreamWriter consoleFile = new StreamWriter(fileName);
...
}
When I went and checked out the folder, the relevant sub-folder and file had been created but the text file was blank.
I'm guessing it's something to do with closing the text file after creating the directory, which means it's already open when the system tries to open it. I can't figure out how to sort this issue out though!
The two functions shown above are called like this:
private void btnStart_Click(object sender, EventArgs e)
{
...
//file details entered upon load written to new file - according to PatID
createDirectory();
saveFileDetails();
}
Any suggestions on where to go from here would be very much appreciated!
Thanks,
Mark

The issue here is that you do
if (!File.Exists(fileName)) //if monitoring file does not exist...
{
File.Create(fileName); //create it in Owner-Date format
}
Right before you try to write to the file. Because you've just created it (if it didn't exist), chances are that the operating system hasn't released the file yet.
Like #Jauch mentioned in the comments, you could skip this check completely and use the StreamWriter overload which will create file if it doesn't exist, or append to it if it does.
private void saveFileDetails()
{
//Once case details have been entered, create new file using these details and add data input structure
using (StreamWriter consoleFile = new StreamWriter(fileName, true))
{
// ...
}
}
Alternatively you can use the following to write all of your text at once:
File.AppendAllText(textToWrite, fileName);

File.Create(fileName) returns an open stream to the file which is never closed.
To create an empty file use File.WriteAllBytes(fileName, new byte[0]);
Otherwise the 2 methods can be shortend
private void SaveFileDetails()
{
string subDirectory = Path.Combine(
Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments),
"Horse Monitoring Records");
// create the folder hierarchy if not exists. does nothing if already there
Directory.CreateDirectory(subDirectory);
// each patient has individual file
var filepath = Path.Combine(subDirectory,
txtPatName.Text + "-" + txtOwnerName.Text + "-" + DateTime.Now.Date.ToString("yyyyMMdd") + ".txt");
// creates the file if not exists
using (var writer = new StreamWriter(filepath, append: true, encoding: Encoding.UTF8))
{
// write details
}
}
Note:
merged 2 methods
.NET naming conventions applied
changed dateformat to better sort by name in explorer

StreamWriter implements IDisposable, so wrapping it in a using block can manage closing and disposing the writer and ensuring it is available the next time you want to touch that file. It can also manage creating the text file if it doesn't exist, removing the need to explicitly call File.Create.
StreamWriter consoleFile = new StreamWriter(fileName);
becomes
using (StreamWriter writer = File.AppendText("log.txt"))
{
// writing, etc.
}
or
using (StreamWriter writer = new StreamWriter(fileName, true))
{ // true says "append to file if it exists, create if it doesn't"
// writing, etc.
}
Whatever seems more readable to you will work fine.

Error occured While writing to the file

I am creating the File. At first time after creating the file, I'm immediately going to open a file but it shows error:
"The process cannot access the file 'C:\ProjectWork\Websites3\LogsArpita\ErrorLogs\Error_Log_24_4_2014.txt' because it is being used by another process."
What does it means? How can I open a file immediately for further write operation. I have tried with following code.
FileName = String.Concat("Error_Log_", DateTimeStamp + ext);
if (!File.Exists(Server.MapPath("~/LogsArpita/ErrorLogs/" + FileName)))
{
File.Create(Server.MapPath("~/LogsArpita/ErrorLogs/" + FileName));
}
//Error occured here, below line
StreamWriter tw = new StreamWriter(Server.MapPath("~/LogsArpita/ErrorLogs/" + FileName), true);
tw.WriteLine("");
tw.Write("\"" + DateTimeStampLog + "\",");
tw.Write("\"Assignments.aspx\",");
tw.Write("\"" + ErrorMessage + "\",");
tw.Write("\"" + TransactVariable + "\"");
tw.Close();

You do not need the File.Create because the StreamWriter constructor will create the file if it does not exist
This is what the msdn documentation says:
Initializes a new instance of the StreamWriter class for the specified file by using the default encoding and buffer size. If the file exists, it can be either overwritten or appended to. If the file does not exist, this constructor creates a new file.