I have an application that is looking through some files for old data. In order to make sure we don't corrupt good projects, I'm copying the files to a temporary location. Some of the directories I'm checking are source-code directories, and they have .svn folders. We use Subversion to manage our code.
Once I've searched through all of the files, I want to delete the temp cache. Sounds easy, right?
For some reason, all of my .svn directories won't delete from the cache. They crash the app.
For reasons (too deep to go into here), I have to use the temp folder, so just "scan the original file" is out of the question for political reasons.
I can go into explorer and delete them. No problem. No warnings. Just deletes. But the code crashes with "Access to {file} is denied." I'm at my wits end with this one, so any help would be appreciated.
While I've simplified the function a LITTLE for sake of your sanity, the code REALLY is about this simple.
List<string> tmpCacheManifest = new List<string>();
string oldRootPath = "C:\\some\\known\\directory\\";
string tempPath = "C:\\temp\\cache\\";
foreach (string file in ListOfFilesToScan)
{
string newFile = file.Replace(oldRootPath, tempPath);
// This works just fine.
File.Copy(file, newFile);
tmpCacheManifest.add(newFile);
}
// ... do some stuff to the cache to verify what I need.
// Okay.. I'm done.. Delete the cache.
foreach (string file in tmpCacheManifest)
{
// CRASH!
File.Delete(file);
}
* Update *: The exception is UnauthorizedAccessException. The text is "Access to the path 'C:\temp\cache\some-sub-dirs\.svn\entries' is denied."
It happens under XP, XP-Pro and Windows 7.
* Update 2 * None of my validation even ATTEMPTS to look at subversion files. I do need them, however. That's part of the political crap. I have to show that EVERY file was copied... wheter it was scanned or not.
And I realize what the usual suspects are for File.Delete. I realize what UnauthorizedAccessException means. I don't have access. That's a no-brainer. But I just copied the file. How can I NOT have access to the file?
* Update 3 *
The answer was in the "read-only" flag. Here's the code I used to fix it:
foreach (string file in ListOfFilesToScan)
{
string newFile = file.Replace(oldRootPath, tempPath);
// This works just fine.
File.Copy(file, newFile);
//// NEW CODE ////
// Clear any "Read-Only" flags
FileInfo fi3 = new FileInfo(fn);
if ((fi3.Attributes & FileAttributes.ReadOnly) == FileAttributes.ReadOnly)
{
fi3.Attributes = (FileAttributes)(Convert.ToInt32(fi3.Attributes) - Convert.ToInt32(FileAttributes.ReadOnly));
}
tmpCacheManifest.add(newFile);
}
// ... do some stuff to the cache to verify what I need.
As far as I recall, Subversion marks the files in its .svn subdirectories as read-only.
Try resetting the read-only attribute before deleting the file. I don't really know any C#, but a quick Google suggests this might do the trick:
File.SetAttributes(file, FileAttributes.Normal);
The only problem I see would be in this part:
// ... do some stuff to the cache to verify what I need.
If you do open the file and forget to close it, you still have exclusive access to it, and thus can't delete it later on.
Sounds like you don't have access to delete the file...
system.io.file.delete
The above link says you get UnauthorizedAccessException when:
The caller does not have the required permission.
-or-
path is a directory.
-or-
path specified a read-only file.
It's one of those.
Sounds like a permissions issue. Tricky one though as you obviously have write access if the File.Copy already works....
Only thing I could think of is the file still has a handle opened somewhere (as others have suggested perhaps in your do some stuff to the cache part).
First of all: "Crash" means an exception, right? Which one? Can you catch it and show it?
Second thing: You are copying subversion repositories, although you don't care about the subversion metadata? That's what svn export is about (no .svn directory in the target).
The answer to the first question is what you really need to provide. Maybe something grabs the .svn and locks some files. TortoiseSVN maybe (to give you nice overlay icons..)?
If a folder contains read only files, Directory.Delete won't delete it and raise the exception you're getting. For future visitors of this page, I've found a simple solution which doesn't require us to recurse through all the files and changing their read-only attribute:
Process.Start("cmd.exe", "/c " + #"rmdir /s/q C:\Test\TestDirectoryContainingReadOnlyFiles");
(Change a bit to not to fire a cmd window momentarily, which is available all over the internet)
Not understanding what you want to do so much, but what about chmoding it to 777 or 775. :-/
Edit:
Noticed your on windows. You'd have to change the permissions. Don't know how windows does that :-/
Related
I need to write a big file in my project.
What I learned:
I should NOT write the big file directly to the destination path,
because this may leave a incomplete file in case the app crash while writing it.
Instead, I should write to a temporary file and move (rename) it. (called atomic file operation)
My code snippet:
[NotNull]
public static async Task WriteAllTextAsync([NotNull] string path, [NotNull] string content)
{
string temporaryFilePath = null;
try {
temporaryFilePath = Path.GetTempFileName();
using (var stream = new StreamWriter(temporaryFilePath, true)) {
await stream.WriteAsync(content).ConfigureAwait(false);
}
File.Delete(path);
File.Move(temporaryFilePath, path);
}
finally {
if (temporaryFilePath != null) File.Delete(temporaryFilePath);
}
}
My Question:
The file will be missing if the app crashes between File.Delete and File.Move. Can I avoid this?
Is there any other best practice for writing big files?
Is there any suggestion on my code?
The file will be missing if the app crashes between File.Delete and File.Move. Can I avoid this?
Not that I'm aware of, but you can detect it - and if you use a more predictable filename, you can recover from that. It helps if you tweak the process somewhat to use three file names: the target, a "new" file and an "old" file. The process becomes:
Write to "new" file (e.g. foo.txt.new)
Rename the target file to the "old" file (e.g. foo.txt.old)
Rename the "new" file to the target file
Delete the "old" file
You then have three files, each of which may be present or absent. That can help you to detect the situation when you come to read the new file:
No files: Nothing's written data yet
Just target: All is well
Target and new: App crashed while writing new file
Target and old: App failed to delete old file
New and old: App failed after the first rename, but before the second
All three, or just old, or just new: Something very odd is going on! User may have interfered
Note: I was unaware of File.Replace before, but I suspect it's effectively just a simpler and possibly more efficient way of doing the code you're already doing. (That's great - use it!) The recovery process would still be the same though.
You can use File.Replace instead of deleting and moving files. In case of hard fault (electricity cut or something like this) you will always lost data, you have to count with that.
I've a plugin based web app that allows an administrator to assign various small pieces of functionality (the actual functionality is unimportant here) to users.
This functionality is configurable and an administrator having an understanding of the plugins is important. The administrator is technical enough to be able to read the very simple source code for these plugins. (Mostly just arithmetic).
I'm aware there are a couple of questions already about accessing source code from within a DLL built from that source:
How to include source code in dll? for example.
I've played with getting the .cs files into a /Resources folder. However doing this with a pre-build event obviously don't include these files in the project. So VS never copies them and I'm unable to access them on the Resources object.
Is there a way to reference the source for a particular class... from the same assembly that I'm missing? Or alternatively a way to extract COMPLETE source from the pdb for that assembly? I'm quite happy to deploy detailed PDB files. There's no security risk for this portion of the solution.
As I've access to the source code, I don't want to go about decompiling it to display it. This seems wasteful and overly complicated.
The source code isn't included in the DLLs, and it isn't in the PDBs either (PDBs only contain a link between the addresses in the DLL and the corresponding lines of code in the sources, as well as other trivia like variable names).
A pre-build event is a possible solution - just make sure that it produces a single file that's included in the project. A simple zip archive should work well enough, and it's easy to decompress when you need to access the source. Text compresses very well, so it might make sense to compress it anyway. If you don't want to use zip, anything else will do fine as well - an XML file, for example. It might even give you the benefit of using something like Roslyn to provide syntax highlighting with all the necessary context.
Decompilation isn't necessarily a terrible approach. You're trading memory for CPU, basically. You'll lose comments, but that shouldn't be a problem if your code is very simple. Method arguments keep their names, but locals don't - you'd need the PDBs for that, which is a massive overkill. It actually does depend a lot on the kind of calculations you're doing. For most cases, it probably isn't the best solution, though.
A bit roundabout way of handling this would be a multi-file T4 template. Basically, you'd produce as many files as there are source code files, and have them be embedded resources. I'm not sure how simple this is, and I'm sure not going to include the code :D
Another (a bit weird) option is to use file links. Simply have the source code files in the project as usual, but also make a separate folder where the same files will be added using "Add as link". The content will be shared with the actual source code, but you can specify a different build action - namely, Embedded Resource. This requires a (tiny) bit of manual work when adding or moving files, but it's relatively simple. If needed, this could also be automated, though that sounds like an overkill.
The cleanest option I can think of is adding a new build action - Compile + Embed. This requires you to add a build target file to your project, but that's actually quite simple. The target file is just an XML file, and then you just manually edit your SLN/CSPROJ file to include that target in the build, and you're good to go. The tricky part is that you'll also need to force the Microsoft.CSharp.Core.target to use your Compile + Embed action to be used as both the source code and the embedded resource. This is of course easily done by manually changing that target file, but that's a terrible way of handling that. I'm not sure what the best way of doing that is, though. Maybe there's a way to redefine #(Compile) to mean #(Compile;MyCompileAndEmbed)? I know it's possible with the usual property groups, but I'm not sure if something like this can be done with the "lists".
Taking from #Luaan's suggestion of using a pre-build step to create a single Zipped folder I created a basic console app to package the source files into a zip file at a specific location.
static void Main(string[] args)
{
Console.WriteLine("Takes a folder of .cs files and flattens and compacts them into a .zip." +
"Arg 1 : Source Folder to be resursively searched" +
"Arg 2 : Destination zip file" +
"Arg 3 : Semicolon List of folders to ignore");
if (args[0] == null || args[1] == null)
{
Console.Write("Args 1 or 2 missing");
return;
};
string SourcePath = args[0];
string ZipDestination = args[1];
List<String> ignoreFolders = new List<string>();
if (args[2] != null)
{
ignoreFolders = args[2].Split(';').ToList();
}
var files = DirSearch(SourcePath, "*.cs", ignoreFolders);
Console.WriteLine($"{files.Count} files found to zip");
if (File.Exists(ZipDestination))
{
Console.WriteLine("Destination exists. Deleting zip file first");
File.Delete(ZipDestination);
}
int zippedCount = 0;
using (FileStream zipToOpen = new FileStream(ZipDestination, FileMode.OpenOrCreate))
{
using (ZipArchive archive = new ZipArchive(zipToOpen, ZipArchiveMode.Create))
{
foreach (var filePath in files)
{
Console.WriteLine($"Writing {Path.GetFileName(filePath)} to zip {Path.GetFileName(ZipDestination)}");
archive.CreateEntryFromFile(filePath, Path.GetFileName(filePath));
zippedCount++;
}
}
}
Console.WriteLine($"Zipped {zippedCount} files;");
}
static List<String> DirSearch(string sDir, string filePattern, List<String> excludeDirectories)
{
List<String> filePaths = new List<string>();
foreach (string d in Directory.GetDirectories(sDir))
{
if (excludeDirectories.Any(ed => ed.ToLower() == d.ToLower()))
{
continue;
}
foreach (string f in Directory.GetFiles(d, filePattern))
{
filePaths.Add(f);
}
filePaths.AddRange(DirSearch(d, filePattern, excludeDirectories));
}
return filePaths;
}
Takes 3 parameters for source dir, output zip file and a ";" separated list of paths to exclude. I've just built this as a binary. Committed it to source control for simplicity and included it in the pre-build for projects I want the source for.
No error checking really and I'm certain it will fail for missing args. But if anyone wants it. Here it is! Again Thanks to #Luaan for clarifying PDBs aren't all that useful!
I want to make an exact copy of some files, directories and subdirectories that are on my USB drive I:/ and want them to be in C:/backup (for example)
My USB drive has the following structure:
(just to know, this is an example, my drive has more files, directories and subdirectories)
courses/data_structures/db.sql
games/pc/pc-game.exe
exams/exam01.doc
Well, I am not sure how to start with this but my first idea is to get all the files doing this:
string[] files = Directory.GetFiles("I:");
The next step could be to make a loop and use File.Copy specifying the destination path:
string destinationPath = #"C:/backup";
foreach (string file in files)
{
File.Copy(file, destinationPath + "\\" + Path.GetFileName(file), true);
}
At this point everything works good but not as I wanted cause this doesn't replicate the folder structure. Also some errors happen like the following...
The first one happens because my PC configuration shows hidden files for every folder and my USB has an AUTORUN.INF hidden file that is not hidden anymore and the loop tries to copy it and in the process generates this exception:
Access to the path 'AUTORUN.INF' is denied.
The second one happens when some paths are too long and this generates the following exception:
The specified path, file name, or both are too long. The fully
qualified file name must be less than 260 characters, and the
directory name must be less than 248 characters.
So, I am not sure how to achieve this and validate each posible case of error. I would like to know if there is another way to do this and how (maybe some library) or something more simple like an implemented method with the following structure:
File.CopyDrive(driveLetter, destinationFolder)
(VB.NET answers will be accepted too).
Thanks in advance.
public static void Copy(string src, string dest)
{
// copy all files
foreach (string file in Directory.GetFiles(src))
{
try
{
File.Copy(file, Path.Combine(dest, Path.GetFileName(file)));
}
catch (PathTooLongException)
{
}
// catch any other exception that you want.
// List of possible exceptions here: http://msdn.microsoft.com/en-us/library/c6cfw35a.aspx
}
// go recursive on directories
foreach (string dir in Directory.GetDirectories(src))
{
// First create directory...
// Instead of new DirectoryInfo(dir).Name, you can use any other way to get the dir name,
// but not Path.GetDirectoryName, since it returns full dir name.
string destSubDir = Path.Combine(dest, new DirectoryInfo(dir).Name);
Directory.CreateDirectory(destSubDir);
// and then go recursive
Copy(dir, destSubDir);
}
}
And then you can call it:
Copy(#"I:\", #"C:\Backup");
Didn't have time to test it, but i hope you get the idea...
edit: in the code above, there are no checks like Directory.Exists and such, you might add those if the directory structure of some kind exists at destination path. And if you're trying to create some kind of simple sync app, then it gets a bit harder, as you need to delete or take other action on files/folders that don't exist anymore.
This generally starts with a recursive descent parser. Here is a good example: http://msdn.microsoft.com/en-us/library/bb762914.aspx
You might want to look into the overloaded CopyDirectory Class
CopyDirectory(String, String, UIOption, UICancelOption)
It will recurse through all of the subdirectories.
If you want a standalone application, I have written an application that copies from one selected directory to another, overwriting newer files and adding subdirectories as needed.
Just email me.
File.Copy allows simple file copying. When a duplicate file name is encountered, File.Copy has a third parameter to determine if the original file is overwritten or not.
Is there a built-in .Net function that allows a third option to rename the copied file and hence keep both files?. For example, the file copy would automatically rename "readme.txt" to "readme - Copy.txt" if another readme.txt already existed in the destination folder - similar to Windows Explorer functionality?
I realize it can be written but didn't want reinvent the wheel if it exists.
Thanks in advance.
Nope, this functionality doesn't exist out of the box (thankfully, as it would introduce a responsibility to the framework which it ought not to have*,) so if you want this, then you will need to implement a bespoke solution.
*Which implementation should it take? Appending "- Copy", appending "(n)"? It becomes problematic rather sherpish.
There's nothing you can do all-in-one go with File.Copy, however you could test if the destination exists and then Move instead.
Move takes two parameters, and you can essentially rename at the same time.
if File.Exists(destinationPath) {
File.Move(source, destinationPathRenamed);
} else {
try {
File.Copy(source, destinationPath);
} catch (IOException ex) {
// destinationPath file already exists
File.Move(source, destinationPathRenamed);
}
}
See Move documentation
EDIT:
Updated code above. #xanatos makes a good point about atomic operation. I made no assumptions about whether there are other processes accessing the file(s).
Note that I haven't added other error handling for the source file being deleted before the operation begins either.
var destinationPath = c:\temp\archive.txt;
if(File.Exists(destinationPath))
destinationPath = string.Format("c:\temp\archive.{0}.txt", DateTime.Now.ToString("yyyyMMddHHmmffff"));
File.Move(filePath, destinationPath );
I have a very strange problem indeed! I wonder if the problem is in the framework, OS or maybe it's just me, misunderstanding things...
I have a file, which might be created a long time ago, I use the file, and then I want to archive it, by changing it's name. Then I want to create a new file, with the same name as the old file had, before it was renamed. Easy enough!
The problem that really puzzles me, is that the newly created file gets wrong "created"-timestamp! That's a problem since it's that timestamp that I want to use for determing when to archive and create a new file.
I've created a very small sample that shows the problem. For the sample to work, there must be a file 1.txt in the Files folder. Also, the file attribute must also be set back in time (with one of the tools available, I use Nomad.NET).
static void Main(string[] args)
{
// Create a directory, if doesnt exist.
string path = Path.GetDirectoryName(Application.ExecutablePath) + "\\Files";
Directory.CreateDirectory(path);
// Create/attach to the 1.txt file
string filename = path + "\\1.txt";
StreamWriter sw = File.AppendText(filename);
sw.WriteLine("testing");
sw.Flush();
sw.Close();
// Rename it...
File.Move(filename, path + "\\2.txt");
// Create a new 1.txt
sw = File.AppendText(filename);
FileInfo fi = new FileInfo(filename);
// Observe, the old files creation date!!
Console.WriteLine(String.Format("Date: {0}", fi.CreationTime.Date));
Console.ReadKey();
}
This is the result of an arcane "feature" going way back to the old days of Windows. The core details are here:
Windows NT Contains File System Tunneling Capabilities (Archive)
Basically, this is on purpose. But it's configurable, and an anachronism in most of today's software.
I think you can create a new filename first, then rename old->old.1, then new->old, and it'll "work". I don't remember honestly what we did when we ran into this last a few years back.
I recently ran into the same problem described in the question. In our case, if our log file is older than a week, we delete it and start a new one. However, it's been keeping the same date created since 2008.
One answer here describes renaming the old file and then creating a new one, hopefully picking up the proper Creation Date. However, that was unsuccessful for us, it kept the old date still.
What we used was the File.SetCreationTime method, and as its name suggests, it easily let us control the creation date of the file, allowing us to set it to DateTime.Now. The rest of our logic worked correctly afterwards.
File.SetCreationTime("file", DateTime.Now);