I have an idea for a C# program that works basically like the Windows Explorer. The aim is to display all files and folders, and to show specific information for each of them. One of the features I'm planning is to detect folder sizes which is something the Explorer cannot.
My idea for the algorithm is to cumulate the sizes of all files in the specific folder. However, I'm afraid of performance issues. For example, for displaying the sizes of all folders of C: I have to consider all the files on the whole drive. This will probably take a while and thus the calculation can't be done each time the user switches to a different folder or back.
So I'd like to cache some of the sizes. However, when files change, are added or removed, the cache data becomes outdated. But I do not want to monitor all file changes while the program is not running.
Is there any way I can find out if the cache is up-to-date, e.g. by retrieving some sort of checksum that doesn't require calculating all sizes again? Is there another memory and CPU-efficient way to find out if file sizes have changed since the last calculation? Or is there even another possibility?
Windows Explorer has the Folder size available (# files, size on disk etc) availble for the properties of any disk/folder. Directory Properties Example
As for writing a program, you can certainly use a recurisve DirectoryInfo.EnumerateFiles() to get all the files within a disk/folder.
As for monitoring, you can use the FileSystemWatcher class to monitor changes to any disk/folder.
To keep the cache up to date is going to be difficult because:
Depending on the Partition Formated Type [Fat, Fat32, NTFS, etc] you are limited to what each support.
Any new file (created date > cache date) means you still have to enumerate all the files to filter the list to new files.
Modified files (modified date > cache date) has the same issue.
Unless you use something VERY specific to the Formatted Type beyond what C# provides, updating a cache after the application launch will need to occur every time, and be very intense.
Windows Explorer is a pretty crafty program. It is filled with tricks that are designed to hide the fact that any file system is punishingly slow to iterate. The kind of tricks that I know about:
fake it. Show the folder hierarchy as a treeview and use the [+]
glyph to show that a folder has files or directories inside of it.
Even when it doesn't. That's visible, create an empty directory and
restart your machine. Note the [+] glyph, click it and notice that,
when forced to iterate the sub-directory, it smoothly change the [+]
glyph to a 'nothing there' glyph.
delay it. Harder to see, you need a subdirectory with a lot of
files. Explorer starts a background thread that iterates the
content of the folder. Once it figured it out, it smoothly changes
the status bar text.
tell me what happened. Explorer uses ReadDirectoryChangesW()
heavily. Wrapped in .NET by the FileSystemWatcher class. Key point
is that it gets a notification that something changed in the
subdirectory that the user is looking at. No polling required, that
would have horrible perf. Go back to bullet two.
Related
I have this code to add an Image to ImageList :
imageList1.Images.Add(Image.FromFile(path));
I want to somehow save the image in my application(resources maybe?!) just like the way you add Image to ImageList at design time so if I move my application files somewhere else the added images move with it. I don't want to save files in application path or database or other things like that.
Is there any way to do that ?
Can you logically explain how that should work? Did you think this through?
just like the way you add Image to ImageList at design time
If you do this, they get COMPILED INTO THE PROGRAM AS RESOURCE. Which not only requires the compiler and the source code (though you can put them into a resource only assembly), but also access to changing the program files.
Doable at compile time, totally not a sane approach at runtime.
It also effectively stored them in a resource assembly in the application path, which you rule out as a location.
I don't want to save files in application path or database or other things
like that.
latest "other things like that" would ALSO rule out modifying the program (as it would store them somewhere) and make this a total fallacy request. You want to store images but not store them. Grats. Even if not:
Programs should NOT NEVER EVER modify themselves. This is a high priviledge operation, normal users can not change the program files folder.
As you rule out all other places - where you want to store the iamge? Cloud? Magic?
There is no way to do what you want because you rule out all possibilities. And "Like at design time" only seems to think you think this works by magic.
So, no - the question as you have asked it has one answer: get realistic. You can not rule out all ways to save them and then want them saved. Requirements contradict themselves.
I have a portable audio player which lists directories and files (SD card FAT32 storage), as per the manual, in the order they are copied. That is indeed what it does: when manually copying one file at a time, it will display those files in the order copied. Now I already copied hundreds of files using a combination of Windows Explorer and Unison so as a result everything gets displayed in what is basically random order. I'd like everything to be listed alphabetically, so I wrote the simplest piece of C# code I can think of that lists directories alphabetically, idem for the files inside them, and uses Directory.SetLastWriteTime and File.SetLastWriteTime to set the time on each, and that time is incremented with 2 seconds on each iteration.
Reading everything back in using GetLastWriteTime confirms the timestamps are updated, however the player does not do what would be excpected.. It does order directories correctly, and some of the files (probably those that were already copied in alphabetical order before), but not all of them. Tried everything again using SetCreationTime but the same result.
So which file property has to be set then in order to make this work? Or is it some property of the FAT drive itself? And which functions to use? (eventually plain C apis are fine as well)
fatsort claims to do this so you can study its source code to find out how to re-order files.
Note that what you want to do is a fairly involved process because it relies on the internal structures of FAT, but it is doable.
I am working on an application that keeps track and uses files stored on the file system. Users are allowed to open, create, delete and move files in the file system. Meanwhile my application is not constantly running so I can't keep track of all changes real-time. Afterwards my application has to find out what file is whom (i.c. as identified in my application).
The most preferred solution for the users is that the application solves every change itself. Each user-interaction is less preferred.
One of my ideas was to use an attribute of a file and assign a key-value to it so when it has been identified once, it can always been recognized afterwards. But I don't know if there is such an attribute. This article didn't give much hope: There is in Windows file systems a pre computed hash for each file?.
Does somebody know if there is such an attribute I can use? And how can I used it in C#?
Is there anyone who is running up against this problem? And how did you solve it?
I'd like to hear good suggestions.
regards, Jaap
If your files don't leave NTFS, this is easily achievable by alternative data streams, where you can store your data along with files. This is more-or-less good article about ADS: http://www.flexhex.com/docs/articles/alternate-streams.phtml
There is another suitable method - it's very efficient, but also very complicated to use, it requires quite good knowledge about NTFS internals - USN Change Journal; see http://msdn.microsoft.com/en-us/library/windows/desktop/aa363798.aspx. With USN Change Journal, you can "get" very efficiently all files that were changed (even all change events) within specified time period.
Nevertheless, if your files leave NTFS realm, e.g. if it's copied to FAT32, contents of ADS is lost.
Relying on a File attribute is "dangerous" in that some user could alter the attribute while your program isn't running. This could lead you to believe that a certain file is (or isn't) tracked by the program while it really isn't.
I would suggest to keep track of the files in a database, XML, or some other file. When your application starts you read the file/db and check for new/deleted/editted files.
You could store a Hash of the files to find out if a file has been moved/editted. Keeping track of files that are moved AND editted is going to be pretty difficult. (I have no clue how you could achieve it)
PS: Have you considered making your application a Windows service? Having the file-management running in the background no matter if the GUI part of your application is running or not?
I have a project that uses the .net FileSystemWatcher to watch a Samba network share for video files. When it sees a file, it adds it to an encode queue. When files are dequeued, they are moved to a local directory where the process then encodes the file to several different formats and spits them out to an output directory.
The problem arises because the video files are so big, that it often takes several minutes for them to copy completely into the network directory, so when a file is dequeued, it may or may not have completely finished being copied to the network share. When the file is being copied from a windows machine, I am able to work around it because trying to move a file that is still being copied throws an IOException. I simply catch the exception and retry every few seconds until it is done copying.
When a file is dropped into the Samba share from a computer running OS X however, that IOException is not thrown. Instead, a partial file is copied to the working directory which then fails to encode because it is not a valid video file.
So my question is, is there any way to make the FileSystemWatcher wait for files to be completely written before firing its "Created" event (based on this question I think the answer to that question is "no")? Alternatively, is there a way to get files copied from OS X to behave similarly to those in windows? Or do I need to find another solution for watching the Samba share? Thanks for any help.
Option 3. Your best bet is to have a process that watches the incoming share for files. When it sees a file, note its size and/or modification date.
Then, after some amount of time (like, 1 or 2 seconds), look again. Note any files that were seen before and compare their new sizes/mod dates to the one you saw last time.
Any file that has not changed for some "sufficiently long" period of time (1s? 5s?) is considered "done".
Once you have a "done" file, MOVE/rename that file to another directory. It is from THIS directory that your loading process can run. It "knows" that only files that are complete are in this directory.
By having this two stage process, you are able to later possibly add other rules for acceptance of a file, since all of those rules must pass before the file gets moved to its proper staging area (you can check format, check size, etc.) beyond a simple rule of just file existence.
Your later process can rely on file existence, both as a start mechanism and a restart mechanism. When the process restarts after failure or shut down, it can assume that any files in the second staging are either new or incomplete and take appropriate action based on its own internal state. When the processing is done it can choose to either delete the file, or move it to a "finished" area for archiving or what not.
I'm creating an WP7 app that shows an inspirational text for every day and allows you to mark some of this texts as favorites. You can see the text for today, jump to an day in the calendar oder browse your favorites.
All texts are known prior roll out / installation, I don't want to lazy load them via cloud/web, I want to "install" them together with the app.
How should I store them? Should I use one of the open source databases for WP7 and create all rows on installation? Should I just hardcode them and save the favorites in an IsolatedStorage file?
EDIT: Is it possible to have the read only data in a XML file in the Visual Studio Project and mark it as a ressource? Will this later roll out the file automatically? Does this make sense?
If your concern is speed of loading / efficiency of reading the files then you'll have to test to see what works best. I'd start with what's simplest to implement and then change if necessary.
What is right for your app will depend on the total size of data and the size of individual pieces of text. As well as considering where you store the data, be sure to also consider the format you store it in as deserialization/parsing is also an overhaed you should consider.
Remember to test this on an actual device as the performance you see on the emulator is not likely to be realistic of what your users will see.
Update
If it's readonly data you probably want to add it as multiple content files (set the build Action) within the XAP.
The format of the files and how you divide the data between them will depend on the data and the app.
Having multiple files means you don't have to load all the data at once. (I assume you don't need to do that.) Just open the file you need.
Update 2
For an example of loading a resource file from the XAP see: http://blogs.msdn.com/b/silverlight_sdk/archive/2010/08/27/loading-a-static-xml-file-to-your-windows-phone-silverlight-app.aspx