I am working on an application which reads paths of all the text files from a folder into a list. It reads each file, creates a temporary output file, overwrites the original file with temporary output file and deletes the temporary output file.
Following is my code:
foreach (string lF in multipleFiles)
{
int lineNumber = 0;
using (StreamReader sr = new StreamReader(lF))
{
using (StreamWriter sw = new StreamWriter(lF + "Output"))
{
while (!sr.EndOfStream)
{
//LOGIC
sw.WriteLine(line);
}
}
}
File.Copy(lF + "Output", lF, true);
//File.Delete(lF + "Output");
try
{
File.Delete(lF + "Output"); <--- ERROR HERE
}
catch(Exception ex)
{
MessageBox.Show(ex.ToString());
}
}
I am unable to delete the temporary output file due to the following error:
{"The process cannot access the file '' because it is being
used by another process."}
The error does not occur for every file but only a few. None of the files are open or being used by any other application.
How can the temporary file be deleted?
UPDATE: Refereed to Does FileStream.Dispose close the file immediately?
Added Thread.Sleep(1) before File.Delete(), The issue still exists. Tried increasing the sleep value to 5. No luck.
You always run the risk that an virus scanner or some other driver in the stack still holds on to that file or its directory entry. Use some retry mechanisms but that still doesn't guarantee you'll be able to remove that file as the file operations are not atomic, so any process can open that file between your calls trying to delete it.
var path = lf + "Output";
// we iterate a couple of times (10 in this case, increase if needed)
for(var i=0; i < 10; i++)
{
try
{
File.Delete(path);
// this is success, so break out of the loop
break;
} catch (Exception exc)
{
Trace.WriteLine("failed delete #{0} with error {1}", i, exc.Message);
// allow other waiting threads do some work first
// http://blogs.msmvps.com/peterritchie/2007/04/26/thread-sleep-is-a-sign-of-a-poorly-designed-program/
Thread.Sleep(0);
// we don't throw, we just iterate again
}
}
if (File.Exists(path))
{
// deletion still not happened
// this is beyond the code can handle
// possible options:
// store the filepath to be deleted on startup
// throw an exception
// format the disk (only joking)
}
Code slightly adapted from my answer here but that was in a different context.
Related
When working with an application on C# I am creating a few temporary files using the following logic:
Creating Temp File
private static string CreateTmpFile()
{
string fileName = string.Empty;
try
{
// Get the full name of the newly created Temporary file.
// Note that the GetTempFileName() method actually creates
// a 0-byte file and returns the name of the created file.
fileName = Path.GetTempFileName();
// Craete a FileInfo object to set the file's attributes
FileInfo fileInfo = new FileInfo(fileName);
// Set the Attribute property of this file to Temporary.
// Although this is not completely necessary, the .NET Framework is able
// to optimize the use of Temporary files by keeping them cached in memory.
fileInfo.Attributes = FileAttributes.Temporary;
Console.WriteLine("TEMP file created at: " + fileName);
}
catch (Exception ex)
{
Console.WriteLine("Unable to create TEMP file or set its attributes: " + ex.Message);
}
return fileName;
}
Writing to Temp File
private static void UpdateTmpFile(string tmpFile)
{
try
{
// Write to the temp file.
StreamWriter streamWriter = File.AppendText(tmpFile);
streamWriter.WriteLine("Hello from www.daveoncsharp.com!");
streamWriter.Flush();
streamWriter.Close();
Console.WriteLine("TEMP file updated.");
}
catch (Exception ex)
{
Console.WriteLine("Error writing to TEMP file: " + ex.Message);
}
}
I have also tried and followed some of the implementations found on this link for another question
and am using the following implementations in my code : Storing the file in the AppData Folder for using the ACL
However I have been asked to make sure that :
The temp files cannot be read by anyone(Not even the user) during application runtime,
And to make sure that they are deleted even when force closing the
application
For case 1: The temp files cannot be read by anyone(Not even the user) during application runtime,
How can I implement this for my application files? The temp files contain sensitive data which should not be readable even if the user themselves would like to read. Is there a way I can do that?
For case 2: To make sure that they are deleted even when force closing the
application
Here I would like to make sure than even with force close or a sudden restart the files are deleted.
If Force close: then delete the files before force close
If Restart: then delete the files on next startup
Are these doable?
I have a solution that acts as an interface between two systems, reading files that were dropped on an FTP site and importing any orders/products/etc. into the target system.
When a file is picked up, it is moved to a temp file in the same location, and then the contents are read into an XmlDocument.
string[] files = Directory.GetFiles(pickupFolder, fileFilter, SearchOption.TopDirectoryOnly);
foreach (string pathToFile in files)
{
FileInfo srcFile = new FileInfo(pathToFile);
string tmpFilename = Path.Combine(srcFile.DirectoryName, $"~{Path.GetFileNameWithoutExtension(srcFile.Name)}.tmp");
srcFile.MoveTo(tmpFilename);
XmlDocument srcXml = new XmlDocument();
try
{
using (FileStream fs = srcFile.Open(FileMode.Open, FileAccess.Read))
{
srcXml.Load(fs);
}
}
catch (XmlException ex)
{
throw new FileException($"Invalid XML in {srcFile.Name}.", ex);
}
}
Very, very occassionally, the interface will attempt to open the file so that it can be loaded into the XmlDocument while the moving process has not been completed, throwing an IOException. Is there some way to prevent this from happening?
What is the best way to create something like this that needs to iterate through and process files?
The file move operation will throw an exception when the FTP server is still having a lock on the file. That may happen when the file is still being uploaded and is not yet completed, but is "visible" on the disk. Such collisions are rare, but they happen.
Start by checking your FTP server settings and features if it can hide incomplete files during upload. Another way is if you control the system that uploads files, you could upload them with a special "do not download" extension, and rename them after the upload is complete (atomic operation). Finally, as other pointed out, you could simply catch this specific exception and retry with a delay.
As others have pointed out, if process runs periodically, you can simply wrap it with try / catch block:
try
{
srcFile.MoveTo(tmpFilename);
}
catch (Excption ex)
{
// Write log entry if required
continue;
}
If it's a one-off process, then you'll need to periodically attempt MoveTo until file is released and can be moved. Something like this may work:
int maxRetries = 60;
int retries = 0;
bool success = false;
while (retries < maxRetries)
{
try
{
retries++;
srcFile.MoveTo(tmpFilename);
success = true;
break;
}
catch (Excption ex)
{
// Log the error if required
Thread.Sleep(1000); // Wait 1 second
}
}
if (success == fale)
{
// Log the error
continue; // Skip the file if its still not released
}
The code tries to access the file every second during a minute. If it fails, then program skips this file and continues to next.
I am having a problem with a console job that runs and creates a daily log file that I archive at midnight.
This creates a blank log file for the next day and an archived file with yesterdays date in the name and the contents of the old file for debugging issues I may have had and not known about until the day after.
However since I cranked up the BOT's job I have been hitting issues with System Out of Memory errors when I try and archive the file.
At first I was just not able to get an archived file at all then I worked out a way to get at least the last 100,000 lines which is not nearly enough.
I wrap everything in 3 try/catches
I/O
System out of memory
standard exception
However it's always the OutOfMemoryException that I get e.g
System.OutOfMemoryException Error: Exception of type 'System.OutOfMemoryException' was thrown.;
To give you an example of size 100,000 lines of log is about 11MB file
A standard full log file can be anything from 1/2 a GB to 2GB
What I need to know is this:
a) what size of a standard text file will throw an out of memory error when trying to use File.ReadAllText or a custom StreamReader function I call ReadFileString e.g
public static string ReadFileString(string path)
{
// Use StreamReader to consume the entire text file.
using (StreamReader reader = new StreamReader(path))
{
return reader.ReadToEnd();
}
}
b) is it my computers memory (I have 16GB RAM - 8GB use at time of copying) or the objects I am using in C# that are failing with the opening and copying of files.
When archiving I first try with my custom ReadFileString function (see above), if that returns 0 bytes of content I try File.ReadAllText and then if that fails I try a custom function to get the last 100,000 lines, which is really not enough for debugging errors earlier in the day.
The log file starts at midnight when a new one is created and logs all day. I never used to have out of memory errors but since I have turned up the frequency of method calls the logging has expanded which means the file sizes have as well.
This is my custom function for getting the last 100,000 lines. I am wondering how many lines I could get without IT throwing an out of memory error and me not getting any contents of the last days log file at all.
What do people suggest for the maximum file size for various methods / memory needed to hold X lines, and what is the best method for obtaining as much of the log file as possible?
E.G some way of looping line by line until an exception is hit and then saving what I have.
This is my GetHundredThousandLines method and it logs to a very small debug file so I can see what errors happened during the archive process.
private bool GetHundredThousandLines(string logpath, string archivepath)
{
bool success = false;
int numberOfLines = 100000;
if (!File.Exists(logpath))
{
this.LogDebug("GetHundredThousandLines - Cannot find path " + logpath + " to archive " + numberOfLines.ToString() + " lines");
return false;
}
var queue = new Queue<string>(numberOfLines);
using (FileStream fs = File.Open(logpath, FileMode.Open, FileAccess.Read, FileShare.Read))
using (BufferedStream bs = new BufferedStream(fs)) // May not make much difference.
using (StreamReader sr = new StreamReader(bs))
{
while (!sr.EndOfStream)
{
if (queue.Count == numberOfLines)
{
queue.Dequeue();
}
queue.Enqueue(sr.ReadLine() + "\r\n");
}
}
// The queue now has our set of lines. So print to console, save to another file, etc.
try
{
do
{
File.AppendAllText(archivepath, queue.Dequeue(), Encoding.UTF8);
} while (queue.Count > 0);
}
catch (IOException exception)
{
this.LogDebug("GetHundredThousandLines - I/O Error accessing daily log file with ReadFileString: " + exception.Message.ToString());
}
catch (System.OutOfMemoryException exception)
{
this.LogDebug("GetHundredThousandLines - Out of Memory Error accessing daily log file with ReadFileString: " + exception.Message.ToString());
}
catch (Exception exception)
{
this.LogDebug("GetHundredThousandLines - Exception accessing daily log file with ReadFileString: " + exception.Message.ToString());
}
if (File.Exists(archivepath))
{
this.LogDebug("GetHundredThousandLines - Log file exists at " + archivepath);
success = true;
}
else
{
this.LogDebug("GetHundredThousandLines - Log file DOES NOT exist at " + archivepath);
}
return success;
}
Any help would be much appreciated.
Thanks
try:
keep the queue and stream position in class scope, try GC.Collect() when getting out of memory exception and call function again. seek stream to last position and continue.
or:
use one database like sqlite and keep newest 100000 record in each table.
I am trying to delete a folder but am getting the following error message:
The process cannot access the file .it is being used by another process.
string target_dir="D:\\projectpath\\page";
if (Directory.Exists(target_dir))
Directory.Delete(target_dir, false);
How can I resolve this error?
It looks like the file is locked by some other process. This could happen if when reading/writing to it you forgot to dispose the stream reader/writer and you leaked the unmanaged handler to the file.
For example if you used the following code to read from the file:
StreamReader reader = new StreamReader(fileName);
string contents = reader.ReadToEnd();
and you never release the reader, the file will be locked. The proper way is to wrap IDisposable resources such as Streams and StreamReaders in using statements:
using (StreamReader reader = new StreamReader(fileName))
{
string contents = reader.ReadToEnd();
}
If on the other hand the file is locked by some other external process to your application then there's very little you could do about it, other than killing this process.
I think on the surface, your problem should be apparent: the file is in use by something else, so you can't delete the directory it resides in. If there was a way to "force delete" the file, it could cause other programs to crash. I'd recommend catching the error and either logging it or displaying it to the user, so they can decide if they really want to delete the in-use file.
If you MUST delete the file, you could take a look at:
Using C#, how does one figure out what process locked a file?
And once you know what the process is, you can then kill it, which should free up the file. Again, this isn't a good practice and should only be used in exceptional circumstances.
To delete the diectory you must have the correct Permissions.
var target_dir = "D:\\projectpath\page";
var isWriteAccess = false;
try
{
var collection = Directory.GetAccessControl(target_dir)
.GetAccessRules(true, true, typeof(System.Security.Principal.NTAccount));
if (collection.Cast<FileSystemAccessRule>().Any(rule => rule.AccessControlType == AccessControlType.Allow))
{
isWriteAccess = true;
}
}
catch (UnauthorizedAccessException ex)
{
isWriteAccess = false;
}
catch (Exception ex)
{
isWriteAccess = false;
}
if (!isWriteAccess)
{
MessageBox.Show("no access to directory.");
// Handle here close and kill the blocking process
}
else
{
Directory.Delete(target_dir, false);
}
}
Why would the code below throw a io.system.directorynotfound exception? I can't recreate the problem myself but another user of my code does see it, any ideas why?
Thanks
try
{
//create path
string strAppData = Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData).ToString() + "\\MyApp\\Data\\logs";
//check path exists
if (!System.IO.File.Exists(strAppData))
{
System.IO.Directory.CreateDirectory(strAppData);
}
System.IO.DirectoryInfo dir = new System.IO.DirectoryInfo(strAppData);
int count = dir.GetFiles().Length;
if (count > 100)
{
string[] files = System.IO.Directory.GetFiles(strAppData);
foreach (string file in files)
{
System.IO.File.Delete(file);
}
}
this.fileName = fileName;
// delete the file if it exists
if (File.Exists(fileName))
{
//delete the file
File.Delete(fileName);
}
// write the data to the file
fs = File.OpenWrite(fileName);
sWriter = new StreamWriter(fs);
sWriter.WriteLine(headerText);
sWriter.Flush();
sWriter.Close();
}
catch (Exception exp)
{
throw new Exception(exp.Message);
}
Have you tried using System.IO.Directory.Exists rather than System.IO.File.Exists when checking to see if the path exists?
You're checking for the existence of a directory using System.IO.File rather than System.IO.Directory. It probably works on your machine because that directory already exists, and so the check doesn't matter.
Either way, you need to remember that the file system is volatile. Rather than checking existence, try to open the resource and handle the exception when it fails.
Check that the directory exists, not the file...
Although you're checking it, and creating it if it doesn't exist. You don't know if they have privelages to create the directory. So your Directory.CreateDirectory call may well be failing too and then sub-sequently the rest of the code will fail
http://msdn.microsoft.com/en-us/library/system.io.file.exists.aspx
"Remarks
The Exists method should not be used for path validation, this method merely checks if the file specified in path exists. Passing an invalid path to Existsl returns false. "
That is your error right there. Your validation does not ensure that the path to the file exists