We Keep Coding

DirectoryInfo returns different values when called from different methods

I've been working on a sort of a directory info viewer.
1st stage was to create and populate a TreeView model and during coding I've nested a method to gather information about files on that dir.
All worked fine, values returned we're correct etc.
Now at the stage of code cleanup when I try to get that method out of TreeView creation one I get values that are not even close to original (like 90 MB instead of 1.2 TB).
Below code with issues marked:
private static int itemCount = 0;
private static long folderSizeInfo = 0;
public static void ListDirectory(string path)
{
(...)
var rootDirectoryInfo = new DirectoryInfo(path);
MainWindow.newWindowReport.Drzewko.Items.Add(CreateDirectoryNode(rootDirectoryInfo));
//this does not work if invoked from here <----------------- FIX ME!
//DirectoryMainOperation(rootDirectoryInfo);
//need to run this once more to get data from root
DirectoryContent_Operation(rootDirectoryInfo);
(...)
}
private static TreeViewItem CreateDirectoryNode(DirectoryInfo directoryInfo)
{
var directoryNode = new TreeViewItem { Header = directoryInfo.Name };
try
{
foreach (var directory in directoryInfo.GetDirectories())
{
if (!IsIgnorable(directory.Name))
{
directoryNode.Items.Add(CreateDirectoryNode(directory));
//this method somehow only works here <------------------------ FIX ME!
DirectoryContent_Operation(directory);
}
}
}
catch (UnauthorizedAccessException)
{
//Console.WriteLine("Path is not accessible: {0}", i);
}
return directoryNode;
}
//-------------------TO BE FIXED------------------
private static void DirectoryMainOperation(DirectoryInfo directoryInfo)
{
try
{
foreach (var directory in directoryInfo.GetDirectories())
{
if (!IsIgnorable(directory.Name))
{
DirectoryContent_Operation(directory);
}
}
}
catch (UnauthorizedAccessException)
{
//Console.WriteLine("Path is not accessible: {0}", i);
}
}
private static void DirectoryContent_Operation(DirectoryInfo targetDir)
{
try
{
foreach (var file in targetDir.EnumerateFiles("*", SearchOption.TopDirectoryOnly))
{
itemCount++;
folderSizeInfo += file.Length;
fileTable.Add(new FileExtension_List
{
FileFormat = file.Extension,
Category = extDB.Translation(file.Extension.ToUpper()),
Count = 1,
TotalSize = file.Length
});
}
}
catch (UnauthorizedAccessException)
{
}
catch (Exception ex)
{
}
}
The gist of it is that if I invoke "DirectoryContent_Operation(directory)" from:
"CreateDirectoryNode(DirectoryInfo directoryInfo)" it returns 1.2 TB (correct value)
"DirectoryMainOperation(DirectoryInfo directoryInfo)" it returns 90 MB.

From what it looks like DirectoryMainOperation returns prematurely due to an UnauthorizedAccessException, hence not returning all directories. In DirectoryContent_Operation you swallow every exception by catching Exception!
Please remove the try-catch (everywhere) to see what the (inner) exception message exactly is about and where the exception is thrown..
Note that swallowing exceptions is always a code smell and will likely introduce bugs, that can be really hard to identify.
If you can't handle the exception (put the application back into a stable state) the application must crash. Then fix the reason for the crash.
Before you think about if and how to handle an exception, think about how to avoid it.
Also from a performance perspective, it is highly recommended to avoid expensive exceptions.
Your problem shows how important it is to let exceptions crash the application in order to learn about implementation errors and also to prevent unwanted silent side effects like your incomplete directory list. Swallowing exceptions has put your application into an unpredictable state, which will lead to a lot faulty behavior, which may be unnoticed by users at first. This ca be very costly for a customer who uses your application to manage his business.
To avoid the UnauthorizedAccessException in your case, you can use DirectoryInfo.EnumerateDirectories(String, EnumerationOptions), which is available for .NET Core only (since version 2.1).
It avoids throwing an UnauthorizedAccessException exception by skipping forbidden directories by default.
Alternatively, only enumerate top level directories. Then check each child directory if it's forbidden, before you continue to recursively enumerate the child's top level directories. It is very likely that the following recursive enumeration will solve your problem.
public void ListDirectory(string path)
{
var rootDirectoryInfo = new DirectoryInfo(path);
var rootDirectoryNode = new TreeViewItem { Header = directoryInfo.Name };
MainWindow.newWindowReport.Drzewko.Items.Add(rootDirectoryNode);
CreateDirectoryNode(rootDirectoryInfo, rootDirectoryNode);
}
private static void CreateDirectoryNode(DirectoryInfo parentDirectory, TreeViewItem parentDirectoryNode)
{
foreach (DirectoryInfo childDirectory in parentDirectory.EnumerateDirectories("*", SearchOption.TopDirectoryOnly))
{
var childDirectoryNode = new TreeViewItem { Header = childDirectory.Name };
parentDirectoryNode.Items.Add(childDirectoryNode);
// Don't enter forbidden directories
// and optionally hidden directories too
if (childDirectory.Attributes.HasFlag(FileAttributes.System)
|| childDirectory.Attributes.HasFlag(FileAttributes.Hidden))
{
continue;
}
// Recursively iterate over child's subdirectories
CreateDirectoryNode(childDirectory, childDirectoryNode);
DirectoryContent_Operation(childDirectory);
}
}
private static void DirectoryMainOperation(DirectoryInfo parentDirectory)
{
foreach (DirectoryInfo childDirectory in parentDirectory.EnumerateDirectories("*", SearchOption.TopDirectoryOnly))
{
if (!IsIgnorable(childDirectory.Name))
{
DirectoryContent_Operation(childDirectory);
}
// Don't enter forbidden directories
// and optionally hidden directories too
if (childDirectory.Attributes.HasFlag(FileAttributes.System)
|| childDirectory.Attributes.HasFlag(FileAttributes.Hidden))
{
continue;
}
// Recursively iterate over child's subdirectories
DirectoryMainOperation(childDirectory);
}
}
Generally prefer DirectoryInfo.EnumerateDirectories over DirectoryInfo.GetDirectories.

How to scan several drives for specific directories?

I am not looking for specific files but rather specific directories to index all files in those directories. I am aware of how to search for file types and names but not how to do that with directories for indexing.

Try something like this: (as taken from MS code examples)
public class StackBasedIteration
{
static void Main(string[] args)
{
// Specify the starting folder on the command line, or in
// Visual Studio in the Project > Properties > Debug pane.
TraverseTree(args[0]);
Console.WriteLine("Press any key");
Console.ReadKey();
}
public static void TraverseTree(string root)
{
// Data structure to hold names of subfolders to be
// examined for files.
Stack<string> dirs = new Stack<string>(20);
if (!System.IO.Directory.Exists(root))
{
throw new ArgumentException();
}
dirs.Push(root);
while (dirs.Count > 0)
{
string currentDir = dirs.Pop();
string[] subDirs;
try
{
subDirs = System.IO.Directory.GetDirectories(currentDir);
}
// An UnauthorizedAccessException exception will be thrown if we do not have
// discovery permission on a folder or file. It may or may not be acceptable
// to ignore the exception and continue enumerating the remaining files and
// folders. It is also possible (but unlikely) that a DirectoryNotFound exception
// will be raised. This will happen if currentDir has been deleted by
// another application or thread after our call to Directory.Exists. The
// choice of which exceptions to catch depends entirely on the specific task
// you are intending to perform and also on how much you know with certainty
// about the systems on which this code will run.
catch (UnauthorizedAccessException e)
{
Console.WriteLine(e.Message);
continue;
}
catch (System.IO.DirectoryNotFoundException e)
{
Console.WriteLine(e.Message);
continue;
}
string[] files = null;
try
{
files = System.IO.Directory.GetFiles(currentDir);
}
catch (UnauthorizedAccessException e)
{
Console.WriteLine(e.Message);
continue;
}
catch (System.IO.DirectoryNotFoundException e)
{
Console.WriteLine(e.Message);
continue;
}
// Perform the required action on each file here.
// Modify this block to perform your required task.
foreach (string file in files)
{
try
{
// Perform whatever action is required in your scenario.
System.IO.FileInfo fi = new System.IO.FileInfo(file);
Console.WriteLine("{0}: {1}, {2}", fi.Name, fi.Length, fi.CreationTime);
}
catch (System.IO.FileNotFoundException e)
{
// If file was deleted by a separate application
// or thread since the call to TraverseTree()
// then just continue.
Console.WriteLine(e.Message);
continue;
}
}
// Push the subdirectories onto the stack for traversal.
// This could also be done before handing the files.
foreach (string str in subDirs)
dirs.Push(str);
}
}
}
I would simply modify to only get the files once you are in the directory you want to be looking in.

I was able to get the path of each logical drive to begin my search by adding System.IO.DriveInfo.GetDrives(); and storing the resulting list then passing those strings into the answer by Charles

How to ignore an exception and continue processing a foreach loop?

I have a test program that is supposed to loop over all the files under C:. It dies when it hits the "Documents and Settings" folder. I'd like to ignore the error and keep looping over the remaining files. The problem is that the exception is thrown in the foreach, so putting a try/catch around the foreach will cause the loop to exit. And putting a try/catch after the foreach never fires (because the exception is thrown in the foreach). Is there any way to ignore the exception and continue processing the loop?
Here's the code:
static void Main(string[] args)
{
IEnumerable<string> files = Directory.EnumerateFiles(#"C:\", "*.*",
SearchOption.AllDirectories);
foreach (string file in files) // Exception occurs when evaluating "file"
Console.WriteLine(file);
}

The problem is that IEnumerable<string> behaves lazily, (kind of like a stream). foreach takes one string after another from files, so only when it requests the problem directory, the enumerator crashes. (you can confirm this by calling ToList() or something like this:
//should now crash here instead
List<string> files = Directory.EnumerateFiles(#"C:\", "*.*",
SearchOption.AllDirectories).ToList();
I think you need to find a way to convert files into an enumerated collection, by manually getting each item from files and do a try/catch around the actual read from the enumerator.
edit: chris and servy has good points in comments. You probably shouldn't mess with the enumerator at all after it throws an exception. Try to be more conservative in your query of files would probably be the easiest solution.

There is no effective way for you to handle this case. If the iterator itself is throwing an exception when trying to get the next item then it is not going to be in a position to give you the item after that; it is no longer in a "valid" state. Once an exception is thrown you're done with that iterator.
Your only option here is to not query the entire drive using this method. Perhaps you could write your own iterator to traverse the drive manually in such a way that items that cannot be traversed are skipped more gracefully.
So, to do this manual traversal. We can start out with a generalize Traverse method that can traverse any tree (non-recursively, to better handle deep stacks efficiently):
public static IEnumerable<T> Traverse<T>(
this IEnumerable<T> source
, Func<T, IEnumerable<T>> childrenSelector)
{
var stack = new Stack<T>(source);
while (stack.Any())
{
var next = stack.Pop();
yield return next;
foreach (var child in childrenSelector(next))
stack.Push(child);
}
}
Now we just grab the root, traverse it, and use a method of getting the sub directories that won't throw an exception:
var root = new DirectoryInfo("C:\\");
var allFiles = new[] { root }.Traverse(dir => GetDirectoriesWithoutThrowing(dir))
.SelectMany(dir => GetFilesWithoutThrowing(dir));
The method to get the sub directories can start out as something like this, but may need to be fleshed out:
private static IEnumerable<DirectoryInfo> GetDirectoriesWithoutThrowing(
DirectoryInfo dir)
{
try
{
return dir.GetDirectories();
}
catch (Exception)//if possible catch a more derived exception
{
//TODO consider logging the exception
return Enumerable.Empty<DirectoryInfo>();
}
}
Note that you can play around with this particular part a bit. You may be able to get some of the sub directories out here even if you can't get others, you'll likely need to adjust the error handling, etc. The get files version will be essentially the same, but just with GetFiles instead.

You are probably getting an UnauthorizedAccessException because some files are hidden or a system file.Since you are working with strings(from enumeratefiles)and not fileinfos or directoryinfo objects i rewrote this to fit your design:
first add these 2 member variables:
private static Dictionary<DirectoryInfo, List<string>> DirectoryTree = new Dictionary<DirectoryInfo, List<string>>();
private static List<string> logger = new List<string>();
then place this method:
static void RecursiveSearch(string root)
{
string[] files = null;
string[] subDirs = null;
// First, process all the files directly under this folder
try
{
files = Directory.EnumerateFiles(root).ToArray();
}
catch (UnauthorizedAccessException e)
{
logger.Add(e.Message);
}
catch (System.IO.DirectoryNotFoundException e)
{
Console.WriteLine(e.Message);
}
if (files != null)
{
DirectoryTree.Add(new DirectoryInfo(root), files.ToList());
subDirs = Directory.GetDirectories(root);
foreach (string dir in subDirs)
{
RecursiveSearch(dir);
}
}
}
Then in Main you call it like so:
RecursiveSearch(#"c:\");
After your DirectoryTree dictionary and list logger will be filled.

static void Main(string[] args)
{
IEnumerable<string> files = Directory.EnumerateFiles(#"C:\", "*.*",
SearchOption.AllDirectories);
foreach (string file in files) // Exception occurs when evaluating "file"
{
try
{
Console.WriteLine(file);
}
Catch(Exception ex)
{
}
}
}

Thread try catch continue if access to file was denied

Looping through a folder using system thread, how to ignore and continue if access to file was denied.
// Start thread.
System.Threading.ThreadStart start = delegate { scanner(#"C:\", "*.html;*.txt"); };
System.Threading.Thread thread = new System.Threading.Thread(start);
thread.Start();
private static string scanstatus = string.Empty;
private static void scanner(string folder, string patterns)
{
try
{
// Get the patterns.
string[] pattern_array = patterns.Split(';');
// Search.
foreach (string pattern in pattern_array)
{
foreach (string path in System.IO.Directory.GetFiles(folder, pattern, System.IO.SearchOption.AllDirectories))
{
// trim path
scanstatus = (path.Length > 60) ? "../" + path.Substring(path.Length - 59, 59) : path;
System.Threading.Thread.Sleep(5000);
}
}
}
catch (System.Exception excpt)
{
Console.WriteLine(excpt.Message);
}
finally
{
Console.WriteLine("*************DONE*************");
}
}

As Daniel mentioned in the comment, basically when you want to keep going with the next iteration, you need to move the try/catch to inside the loop. Currently your catch is outside the outer loop, so if an exception is thrown, execution can't continue. There's no concept of "continue from where you'd got to" within C#.
I'd strongly suggest that you also limit what you catch. For example:
foreach (string pattern in patternArray)
{
try
{
foreach (string path in Directory.GetFiles(...))
{
// ...
}
}
catch (IOException e)
{
// Log it or whatever
}
// Any other exceptions you want to catch?
}
Notes:
Catching Exception is almost always a bad idea, except as a final backstop at the top level of your request (or whatever) handling
Underscores in variable names aren't conventional in .NET; typically you'd use camelCase for variables (and PascalCase for methods, classes etc)
With using directives you can avoid putting the fully-qualified type names in your code, which would make it much easier to read
This method will end up being quite long - I'd suggest extracting the inner loop; possibly including the try/catch, or possibly not.

How can I yield and also catch necessary exceptions in C#?

I have the following method I'm working on:
private IEnumerable<TreeNode> GetChildNodes(TreeNode parent)
{
string path = parent.Tag.ToString();
// Add Directories
string[] subdirs = Directory.GetDirectories(path);
foreach (string subdir in subdirs)
{
yield return GetChildNode(subdir);
}
// Add Files
string[] files = Directory.GetFiles(path);
foreach (string file in files)
{
var child = GetChildNode(file);
fileNodeMap[file] = child;
yield return child;
}
}
This works fine with the exception of Directory.GetDirectories() and Directory.GetFiles() can both throw exceptions that I want to catch.
I can't catch the pieces of code which utilize those methods due to my use of yield (yields can't be placed within the body of a try if there is a catch). I know I could remove the yield and simply add to my children to a collection but I'm curious how someone would catch IOExceptions from both of those methods and still utilize yield?

How about something like (for the first part):
string[] subdirs;
try
{
subdirs = Directory.GetDirectories(path);
}
catch (IOException e)
{
// Do whatever you need here
subdirs = new string[0];
}
And similarly for the second. You don't need to yield within that try block. If this doesn't help, please write whatever code you would want to be valid, so that we can see what you're planning to do if an exception is thrown.

Could you not catch exceptions outside, in the code that calls them?

You could make helper methods that add your special error handling sauce:
private string[] GetSubdirectoriesWithSpecialSauce(string path)
{
string[] subdirectories;
try
{
subdirectories = Directory.GetDirectories(path);
}
catch (IOException ioe)
{
ShutdownWOPR();
CallDrFalken();
}
return subdirectories;
}
And obviously substitute the relevant calls. I of course assumed you wanted to yield even on errors, but I humbly accept that this assumption may be wrong :)

I would caution against using exceptions as a method of control flow - if you aren't sure that the directory or path is going to return a valid result, check for it first - almost all of those exceptions can be prevented by argument checking, something generally like the below.
private IEnumerable<TreeNode> GetChildNodes(TreeNode parent)
{
string path = parent.Tag.ToString();
if (String.IsNullOrEmpty (path) || String.IsNullOrWhiteSpace (path))
yield break;
// I'm not aware of a constant/enum for the maximum allowed path length here :(
if (path.Length > 260 || path.Any (Path.GetInvalidPathChars ().Contains))
yield break;
if (!Directory.Exists (path))
yield break;
Func<string[], Func<string[],string>,> SafeIO = (fn, arg) => {
try {
return fn (p);
} catch (IOException) {
return new string[0];
}
};
// Add Directories
string[] subdirs = SafeIO (Directory.GetDirectories, path);
foreach (string subdir in subdirs)
yield return GetChildNode(subdir);
// Add Files
string[] files = SafeIO (Directory.GetFiles, path);
foreach (string file in files) {
var child = GetChildNode(file);
fileNodeMap[file] = child;
yield return child;
}
}
Plenty of room for optimization there (and ripe for further decomposition), and the usual comments apply about race conditions and the lack of a guarantee for checking if a directory exists before it is deleted on another thread, so now you can make this more robust by wrapping a try/catch around the Get{Directories,Files} calls like Jon or xanatos suggested (EDIT: and that I've now wrapped up here as SafeIO) - but now you can catch only the specific exception that is susceptible to this (IOException or DirectoryNotFoundException) and reserve it for truly exceptional cases.

The exception will be throw by the call of GetDirectories and GetFiles, so you can try-catch THEM instead of the for-each,