We Keep Coding

Check inside loop if *txt file has been created

My code is searchcing inside a loop if a *txt file has been created.
If file will not be created after x time then i will throw an exception.
Here is my code:
var AnswerFile = #"C:\myFile.txt";
for (int i = 0; i <= 30; i++)
{
if (File.Exists(AnswerFile))
break;
await Task.Delay(100);
}
if (File.Exists(AnswerFile))
{
}
else
{
}
After the loop i check my file if has been created or not. Loop will expire in 3 seconds, 100ms * 30times.
My code is working, i am just looking for the performance and quality of my code. Is there any better approach than mine? Example should i use FileInfo class instead this?
var fi1 = new FileInfo(AnswerFile);
if(fi1.Exists)
{
}
Or should i use filewatcher Class?

You should perhaps use a FileSystemWatcher for this and decouple the process of creating the file from the process of reacting to its presence. If the file must be generated in a certain time because it has some expiry time then you could make the expiry datetime part of the file name so that if it appears after that time you know it's expired. A note of caution with the FileSystemWatcher - it can sometimes miss something (the fine manual says that events can be missed if large numbers are generated in a short time)
In the past I've used this for watching for files being uploaded via ftp. As soon as the notification of file created appears I put the file into a list and check it periodically to see if it is still growing - you can either look at the filesystem watcher lastwritetime event for this or directly check the size of the file now vs some time ago etc - in either approach it's probably easiest to use a dictionary to track the file and the previous size/most recent lastwritedate event.
After a minute of no growth I consider the file uploaded completely and I process it. It might be wise for you to implement a similar delay if using a file system watcher and the files are arriving by some slow generating method

Why you don't retrieve a list of files name, then search in the list? You can use Directory.GetFiles to get the files list inside a directory then search in this list.
This would be more fixable for you since you will create the list once, and reuse it across the application, instead of calling File.Exists for each file.
Example :
var path = #"C:\folder\"; // set the folder path, which contains all answers files
var ext = "*.txt"; // set the file extension.
// GET filename list (bare name) and make them all lowercase.
var files = Directory.GetFiles(path, ext).Select(x=> x.Substring(path.Length, (x.Length - path.Length) - ext.Length + 1 ).Trim().ToLower()).ToList();
// Search for this filename
var search = "myFile";
// Check
if(files.Contains(search.ToLower()))
{
Console.WriteLine($"File : {search} is already existed.");
}
else
{
Console.WriteLine($"File : {search} is not found.");
}

loop optimization in c#

I am working on a music player for pc using c# and all seems to go fine but, i have problem in loading all the music files from the music directory because it loads very slowly and this take time when opening the app, it could take 5 min depending on the amount of music files. I think this happens because i created a loop to loop through each music file and get the metadatas and also the picture to load on different picture boxes for each music file.
Please Help, i need it to be faster. Thank you.
the code is below...
public List<MusicDetails> Music_Library()
{
List<MusicDetails> files = new List<MusicDetails>();
string[] musicfolder = Directory.GetFiles(Environment.GetFolderPath(Environment.SpecialFolder.MyMusic),"*mp3", SearchOption.AllDirectories);
for (int i = 0; i <musicfolder.Length; i++)
{
try {
files.Add(new MusicDetails
{
title = TagLib.File.Create(musicfolder[i]).Tag.Title,
genre = TagLib.File.Create(musicfolder[i]).Tag.FirstGenre,
artist = TagLib.File.Create(musicfolder[i]).Tag.FirstPerformer,
path = musicfolder[i],
CoverArt = OrganiseAlbums.SingleAlbumImage(musicfolder[i],true)
});
}catch(Exception)
{
// OMIT FILE
}
}
return files;
}

You could try replacing your loop with a parallel foreach loop using background threads - add each item to the UI as it is processed, and let .Net determine the most efficient way to process everything. If you do it right, your UI will remain responsive, and your users will start out with "enough to look at..." Here is a link to get you started:
https://msdn.microsoft.com/en-us/library/dd460720(v=vs.110).aspx?cs-save-lang=1&cs-lang=csharp#code-snippet-1
If you have a ton of music files, though, you may not want to load them all into memory at once. I would look into some sort of a list control or layout control that allows virtualization so that you instantiate and display only the visible ones at any given time. Use a tutorial to see how to bind to your collection so that it is virtualized.
If you post a more specific question with examples of what you have tried, you will get more specific answers...

C# Loading text data into WPF control

I am searching for a very fast way of loading text content from a 1GB text file into a WPF control (ListView for example). I want to load the content within 2 seconds.
Reading the content line by line takes to long, so I think reading it as bytes will be faster. So far I have:
byte[] buffer = new byte[4096];
int bytesRead = 0;
using(FileStream fs = new FileStream("myfile.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) {
while((bytesRead = fs.Read(buffer, 0, buffer.Length)) > 0) {
Encoding.Unicode.GetString(buffer);
}
}
Is there any way of transforming the bytes into string lines and add those to a ListView/ListBox?
Is this the fastest way of loading file content into a WPF GUI control? There are various applications that can load file content from a 1GB file within 1 second.
EDIT: will it help by using multiple threads reading the file? For example:
var t1 = Task.Factory.StartNew(() =>
{
//read content/load into GUI...
});
EDIT 2: I am planning to use pagination/paging as suggested below, but when I want to scroll down or up, the file content has to be read again to get to the place that is being displayed.. so I would like to use:
fs.Seek(bytePosition, SeekOrigin.Begin);
but would that be faster than reading line by line, in multiple threads? Example:
long fileLength = fs.Length;
long halfFile = (fileLength / 2);
FileStream fs2 = fs;
byte[] buffer2 = new byte[4096];
int bytesRead2 = 0;
var t1 = Task.Factory.StartNew(() =>
{
while((bytesRead += fs.Read(buffer, 0, buffer.Length)) < (halfFile -1)) {
Encoding.Unicode.GetString(buffer);
//convert bytes into string lines...
}
});
var t2 = Task.Factory.StartNew(() =>
{
fs2.Seek(halfFile, SeekOrigin.Begin);
while((bytesRead2 += fs2.Read(buffer2, 0, buffer2.Length)) < (fileLength)) {
Encoding.Unicode.GetString(buffer2);
//convert bytes into string lines...
}
});

Using a thread won't make it any faster (technically there is a slight expense to threads so loading may take slightly longer) though it may make your app more responsive. I don't know if File.ReadAllText() is any faster?
Where you will have a problem though is data binding. If say you after loading your 1GB file from a worker thread (regardless of technique), you will now have 1GB worth of lines to databind to your ListView/ListBox. I recommend you don't loop around adding line by line to your control via say an ObservableCollection.
Instead, consider having the worker thread append batches of items to your UI thread where it can append the items to the ListView/ListBox per item in the batch.
This will cut down on the overhead of Invoke as it floods the UI message pump.

Since you want to read this fast I suggest using the System.IO.File class for your WPF desktop application.
MyText = File.ReadAllText("myFile.txt", Encoding.Unicode); // If you want to read as is
string[] lines = File.ReadAllLines("myFile.txt", Encoding.Unicode); // If you want to place each line of text into an array
Together with DataBinding, your WPF application should be able to read the text file and display it on the UI fast.
About performance, you can refer to this answer.
So use File.ReadAllText() instead of ReadToEnd() as it makes your code
shorter and more readable. It also takes care of properly disposing
resources as you might forget doing with a StreamReader (as you did in
your snippet). - Darin Dimitrov
Also, you must consider the specs of the machine that will run your application.

When you say "Reading the content line by line takes to long", what do you mean? How are you actually reading the content?
However, more than anything else, let's take a step back and look at the idea of loading 1 GB of data into a ListView.
Personally you should use an IEnumerable to read the file, for example:
foreach (string line in File.ReadLines(path))
{
}
But more importantly you should implement pagination in your UI and cut down what's visible and what's loaded immediately. This will cut down your resource use massively and make sure you have a usable UI. You can use IEnumerable methods such as Skip() and Take(), which are effective at using your resources effectively (i.e. not loading unused data).
You wouldn't need to use any extra threads either (aside from the background thread + UI thread), but I will suggest using MVVM and INotifyPropertyChanged to skip worrying about threading altogether.

C# - Waiting for a copy operation to complete

I have a program that runs as a Windows Service which is processing files in a specific folder.
Since it's a service, it constantly monitors a folder for new files that have been added. Part of the program's job is to perform comparisons of files in the target folder and flag non-matching files.
What I would like to do is to detect a running copy operation and when it is completed, so that a file is not getting prematurely flagged if it's matching file has not been copied over to the target folder yet.
What I was thinking of doing was using the FileSystemWatcher to watch the target folder and see if a copy operation is occurring. If there is, I put my program's main thread to sleep until the copy operation has completed, then proceed to perform the operation on the folder like normal.
I just wanted to get some insight on this approach and see if it is valid. If anyone else has any other unique approaches to this problem, it would be greatly appreciated.
UPDATE:
I apologize for the confusion, when I say target directory, I mean the source folder containing all the files I want to process. A part of the function of my program is to copy the directory structure of the source directory to a destination directory and copy all valid files to that destination directory, preserving the directory structure of the original source directory, i.e. a user may copy folders containing files to the source directory. I want to prevent errors by ensuring that if a new set of folders containing more subfolders and files is copied to the source directory for processing, my program will not start operating on the target directory until the copy process has completed.

Yup, use a FileSystemWatcher but instead of watching for the created event, watch for the changed event. After every trigger, try to open the file. Something like this:
var watcher = new FileSystemWatcher(path, filter);
watcher.Changed += (sender, e) => {
FileStream file = null;
try {
Thread.Sleep(100); // hack for timing issues
file = File.Open(
e.FullPath,
FileMode.Open,
FileAccess.Read,
FileShare.Read
);
}
catch(IOException) {
// we couldn't open the file
// this is probably because the copy operation is not done
// just swallow the exception
return;
}
// now we have a handle to the file
};
This is about the best that you can do, unfortunately. There is no clean way to know that the file is ready for you to use.

What you are looking for is a typical producer/consumer scenario. What you need to do is outlined in 'Producer/consumer queue' section on this page. This will allow you to use multi threading (maybe span a backgroundworker) to copy files so you don't block the main service thread from listening to system events & you can perform more meaningful tasks there - like checking for new files & updating the queue. So on main thread do check for new files on background threads perform the actual coping task. From personal experience (have implemented this tasks) there is not too much performance gain from this approach unless you are running on multiple CPU machine but the process is very clean & smooth + the code is logically separated nicely.
In short, what you have to do is have an object like the following:
public class File
{
public string FullPath {get; internal set;}
public bool CopyInProgress {get; set;} // property to make sure
// .. other properties if desired
}
Then following the tutorial posted above issue a lock on the File object & the queue to update it & copy it. Using this approach you can use this type approaches instead of constantly monitoring for file copy completion.
The important point to realize here is that your service has only one instance of File object per actual physical file - just make sure you (1)lock your queue when adding & removing & (2) lock the actual File object when initializing an update.
EDIT: Above where I say "there is not too much performance gain from this approach unless" I refere to if you do this approach in a single thread compare to #Jason's suggesting this approach must be noticeably faster due to #Jason's solution performing very expensive IO operations which will fail on most cases. This I haven't tested but I'm quite sure as my approach does not require IO operations open(once only), stream(once only) & close file(once only). #Jason approach suggests multiple open,open,open,open operations which will all fail except the last one.

One approach is to attempt to open the file and see if you get an error. The file will be locked if it is being copied. This will open the file in shared mode so it will conflict with an already open write lock on the file:
using(System.IO.File.Open("file", FileMode.Open,FileAccess.Read, FileShare.Read)) {}
Another is to check the file size. It would change over time if the file is being copied to.
It is also possible to get a list of all applications that has opened a certain file, but I don't know the API for this.

I know this is an old question, but here's an answer I spun up after searching for an answer to just this problem. This had to be tweaked a lot to remove some of the proprietary-ness from what I was working on, so this may not compile directly, but it'll give you an idea. This is working great for me:
void BlockingFileCopySync(FileInfo original, FileInfo copyPath)
{
bool ready = false;
FileSystemWatcher watcher = new FileSystemWatcher();
watcher.NotifyFilter = NotifyFilters.LastWrite;
watcher.Path = copyPath.Directory.FullName;
watcher.Filter = "*" + copyPath.Extension;
watcher.EnableRaisingEvents = true;
bool fileReady = false;
bool firsttime = true;
DateTime previousLastWriteTime = new DateTime();
// modify this as you think you need to...
int waitTimeMs = 100;
watcher.Changed += (sender, e) =>
{
// Get the time the file was modified
// Check it again in 100 ms
// When it has gone a while without modification, it's done.
while (!fileReady)
{
// We need to initialize for the "first time",
// ie. when the file was just created.
// (Really, this could probably be initialized off the
// time of the copy now that I'm thinking of it.)
if (firsttime)
{
previousLastWriteTime = System.IO.File.GetLastWriteTime(copyPath.FullName);
firsttime = false;
System.Threading.Thread.Sleep(waitTimeMs);
continue;
}
DateTime currentLastWriteTime = System.IO.File.GetLastWriteTime(copyPath.FullName);
bool fileModified = (currentLastWriteTime != previousLastWriteTime);
if (fileModified)
{
previousLastWriteTime = currentLastWriteTime;
System.Threading.Thread.Sleep(waitTimeMs);
continue;
}
else
{
fileReady = true;
break;
}
}
};
System.IO.File.Copy(original.FullName, copyPath.FullName, true);
// This guy here chills out until the filesystemwatcher
// tells him the file isn't being writen to anymore.
while (!fileReady)
{
System.Threading.Thread.Sleep(waitTimeMs);
}
}

wait for a TXT file to be readable c#

My application use "FileSystemWatcher()" to raise an event when a TXT file is created by an "X" application and then read its content.
the "X" application create a file (my application detect it successfully) but it take some time to fill the data on it, so the this txt file cannot be read at the creation time, so im
looking for something to wait until the txt file come available to reading. not a static delay but something related to that file.
any help ? thx

Create the file like this:
myfile.tmp
Then when it's finished, rename it to
myfile.txt
and have your filewatcher watch for the .txt extension

The only way I have found to do this is to put the attempt to read the file in a loop, and exit the loop when I don't get an exception. Hopefully someone else will come up with a better way...
bool FileRead = false;
while (!FileRead)
{
try
{
// code to read file, which you already know
FileRead = true;
}
catch(Exception)
{
// do nothing or optionally cause the code to sleep for a second or two
}
}

You could track the file's Changed event, and see if it's available for opening on change. If the file is still locked, just watch for the next change event.

You can open and read a locked file like this
using (var stream = new FileStream(#"c:\temp\file.txt", FileMode.Open, FileAccess.Read, FileShare.ReadWrite)) {
using (var file = new StreamReader(stream)) {
while (!file.EndOfStream) {
var line = file.ReadLine();
Console.WriteLine(line);
}
}
}
However, make sure your file writer flushes otherwise you may not see any changes.

The application X should lock the file until it closes it. Is application X also a .NET application and can you modify it? In that case you can simply use the FileInfo class with the proper value for FileShare (in this case FileShare.Read).
If you have no control over application X, the situation becomes a little more complex. But then you can always attempt to open the file exclusively via the same FileInfo.Open method. Provide FileShare.None in that case. It will attempt to open the file exclusively and will fail if the file is still in use. You can perform this action inside a loop until the file is closed by application X and ready to be read.

We have a virtual printer for creating pdf documents, and I do something like this to access that document after it's sent to the printer:
using (FileSystemWatcher watcher = new FileSystemWatcher(folder))
{
if(!File.Exists(docname))
for (int i = 0; i < 3; i++)
watcher.WaitForChanged(WatcherChangeTypes.Created, i * 1000);
}
So I wait for a total of 6 seconds (some documents can take a while to print but most come very fast, hence the increasing wait time) before deciding that something has gone awry.
After this, I also read in a for loop, in just the same way that I wait for it to be created. I do this just in case the document has been created, but not released by the printer yet, which happens nearly every time.

You can use the same class to be notified when file changes.
The Changed event is raised when changes are made to the size, system attributes, last write time, last access time, or security permissions of a file or directory in the directory being monitored.
So I think you can use that event to check if file is readable and open it if it is.

If you have a DB at your disposal I would recommend using a DB table as a queue with the file names and then monitor that instead. nice and transactional.

You can check if file's size has changed. Although this will require you to poll it's value with some frequency.
Also, if you want to get the data faster, you can .Flush() while writing, and make sure to .Close() stream as soon as you will finish writing to it.