I am currently developing a tool that will find all files, from the previous day, and move them from one folder to another on the same server. I am not hitting permission issues but I am getting stuck on the IF statement. Once the application finds a file, move to the other folder for further processing. My issue is that I am unable to find all files with just the date of yesterday and move them. I have supplied my code below and it is almost there (or at least that's what I tell myself). Thanks ahead of time for any assistance.
private void Form1_Load(object sender, EventArgs e)
{
DateTime past = DateTime.Today.AddDays(-1);
txtSourceFolderCount.Text = past.ToShortDateString();
//efile originally start here
var sourceDir = #"\\DIS2\EFilingXML\Archive";
//application moves to archive
var destDir = #"\\DIS2\EFilingXML";
//only XML files are accepted
//var pattern = "*.xml";
DirectoryInfo source = new DirectoryInfo(sourceDir);
// Get info of each file into the directory
foreach (FileInfo fi in source.GetFiles())
{
var creationTime = fi.LastWriteTime;
if (creationTime == past && creationTime < DateTime.Today)
{
fi.MoveTo(Path.Combine(destDir, fi.ToString()));
}
}
}
My issue is that I am unable to find all files with just the date of
yesterday and move them.
Two issues:
1) You are only using File.Name when you try to call the static System.IO.File.GetCreationTime method. This static method has no context other than the short File.Name with a value like "MyFile.xml" and no path attached.
2) You'll pickup files from today unless you add a second qualifier to your if statement like below:
//gets all files in source directory & moves to destination directory(archive)
foreach (var file in new DirectoryInfo(sourceDir).GetFiles(pattern))
{
DateTime dt = File.GetCreationTime(file.FullName);
if (dt >= DateTime.Today.AddDays(-1) && dt < DateTime.Today)
{
file.MoveTo(Path.Combine(destDir, file.Name));
}
}
Related
I am trying to create a simple “directory/file copy" console application in C#. What I need is to copy all folders and files (keeping the original hierarchy) from one drive to another, like from drive C:\Data to drive E:\Data.
However, I only want it to copy any NEW or MODIFIED files from the source to the destination.
If the file on the destination drive is newer than the one on the source drive, then it does not copy.
(the problem)
In the code I have, it's comparing file "abc.pdf" in the source with file "xyz.pdf" in the destination and thus is overwriting the destination file with whatever is in the source even though the destination file is newer. I am trying to figure out how to make it compare "abc.pdf" in the source to "abc.pdf" in the destination.
This works if I drill the source and destination down to a specific file, but when I back out to the folder level, it overwrites the destination file with the source file, even though the destination file is newer.
(my solutions – that didn’t work)
I thought by putting the “if (file.LastWriteTime > destination.LastWriteTime)” after the “foreach” command, that it would compare the files in the two folders, File1 source to File1 destination, but it’s not.
It seems I’m missing something in either the “FileInfo[]”, “foreach” or “if” statements to make this a one-to-one comparison. I think maybe some reference to the “Path.Combine” statement or a “SearchOption.AllDirectories”, but I’m not sure.
Any suggestions?
As you can see from my basic code sample, I'm new to C# so please put your answer in simple terms.
Thank you.
Here is the code I have tried, but it’s not working.
class Copy
{
public static void CopyDirectory(DirectoryInfo source, DirectoryInfo destination)
{
if (!destination.Exists)
{
destination.Create();
}
// Copy files.
FileInfo[] files = source.GetFiles();
FileInfo[] destFiles = destination.GetFiles();
foreach (FileInfo file in files)
foreach (FileInfo fileD in destFiles)
// Copy only modified files
if (file.LastWriteTime > fileD.LastWriteTime)
{
file.CopyTo(Path.Combine(destination.FullName,
file.Name), true);
}
// Copy all new files
else
if (!fileD.Exists)
{
file.CopyTo(Path.Combine(destination.FullName, file.Name), true);
}
// Process subdirectories.
DirectoryInfo[] dirs = source.GetDirectories();
foreach (DirectoryInfo dir in dirs)
{
// Get destination directory.
string destinationDir = Path.Combine(destination.FullName, dir.Name);
// Call CopyDirectory() recursively.
CopyDirectory(dir, new DirectoryInfo(destinationDir));
}
}
}
You can just take the array of files in "source" and check for a matching name in "destination"
/// <summary>
/// checks whether the target file needs an update (if it doesn't exist: it needs one)
/// </summary>
public static bool NeedsUpdate(FileInfo localFile, DirectoryInfo localDir, DirectoryInfo backUpDir)
{
bool needsUpdate = false;
if (!File.Exists(Path.Combine(backUpDir.FullName, localFile.Name)))
{
needsUpdate = true;
}
else
{
FileInfo backUpFile = new FileInfo(Path.Combine(backUpDir.FullName, localFile.Name));
DateTime lastBackUp = backUpFile.LastWriteTimeUtc;
DateTime lastChange = localFile.LastWriteTimeUtc;
if (lastChange != lastBackUp)
{
needsUpdate = true;
}
else
{/*no change*/}
}
return needsUpdate;
}
Update:
I modified my code with the suggestions above and all went well. It did exactly as I expected.
However, the problem I ran into was the amount of time it took run the application on a large folder. (containing 6,000 files and 5 sub-folders)
On a small folder, (28 files in 5 sub-folders) it only took a few seconds to run. But, on the larger folder it took 35 minutes to process only 1,300 files.
Solution:
The code below will do the same thing but much faster. This new version processed 6,000 files in about 10 seconds. It processed 40,000 files in about 1 minute and 50 seconds.
What this new code does (and doesn’t do)
If the destination folder is empty, copy all from the source to the destination.
If the destination has some or all of the same files / folders as the source, compare and copy any new or modified files from the source to the destination.
If the destination file is newer than the source, don’t copy.
So, here’s the code to make it happen. Enjoy and share.
Thanks to everyone who helped me get a better understanding of this.
using System;
using System.IO;
namespace VSU1vFileCopy
{
class Program
{
static void Main(string[] args)
{
const string Src_FOLDER = #"C:\Data";
const string Dest_FOLDER = #"E:\Data";
string[] originalFiles = Directory.GetFiles(Src_FOLDER, "*", SearchOption.AllDirectories);
Array.ForEach(originalFiles, (originalFileLocation) =>
{
FileInfo originalFile = new FileInfo(originalFileLocation);
FileInfo destFile = new FileInfo(originalFileLocation.Replace(Src_FOLDER, Dest_FOLDER));
if (destFile.Exists)
{
if (originalFile.Length > destFile.Length)
{
originalFile.CopyTo(destFile.FullName, true);
}
}
else
{
Directory.CreateDirectory(destFile.DirectoryName);
originalFile.CopyTo(destFile.FullName, false);
}
});
}
}
}
I'm using two DateTimePickers to specify a date range, then I'm using a CheckedListBox to specify some strings for filenames with wildcards to enumerate in each day's subdirectory contained within a system environment variable path. I want to copy from that source to a destination using FileInfo.Copy.
I have my code already creating the necessary directories. But I'm having trouble specifying the destination filenames -- they are not being specified at all with how I have this written.
I was thinking of using regular expressions, but after some digging I found this MSDN article that seems to do what I want already. I think I need to alter my code in order to use it. I could use some assistance fitting what I already have into what MSDN shows in its example.
I have been on this part of my program for a month now, which has led me to learn quite a bit about c#, parallel programming, async, lambda expressions, background workers, etc. What seems should be simple has become a big rabbit hole for me. For this question I just need a nudge in the right direction, and I will greatly appreciate it!
Here is my code as it stands:
private async void ProcessFiles()
{
// create a list of topics
var topics = topicsBox.CheckedItems.Cast<string>().ToList();
// create a list of source directories based on date range
var directories = new List<string>();
var folders = new List<string>();
for (DateTime date = dateTimePicker1.Value.Date;
date.Date <= dateTimePicker2.Value.Date;
date = date.AddDays(1))
{
directories.Add(_tracePath + #"\" + date.ToString("yyyy-MM-dd") + #"\");
folders.Add(#"\" + date.ToString("yyyy-MM-dd") + #"\");
}
// create a list of source files to copy and destination
// revise based on https://msdn.microsoft.com/en-us/library/kztecsys.aspx?f=255&MSPPError=-2147217396
foreach (var path in directories)
{
var path1 = path;
try
{
foreach (var files2 in folders)
{
// create the target directory
var destPath = textBox1.Text + #"\" + textBox4.Text + files2;
Console.WriteLine("Target directory is {0}", destPath);
Console.WriteLine("Destination filename is {0}", files2);
foreach (var files in topics)
{
foreach (string sourcePath in Directory.EnumerateFiles(path1, files + "*.*", SearchOption.AllDirectories))
{
// copy the files to the temp folder asynchronously
Console.WriteLine("Copy {0} to {1}", sourcePath, destPath);
Directory.CreateDirectory(sourcePath.Replace(sourcePath, destPath));
}
}
}
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
}
So sourcePath contains the source path and filename. You can easily construct the destination path from that, like so:
// Get just the filename of the source file.
var filename = Path.GetFileName(sourcePath);
// Construct a full path to the destination by combining the destination path and the filename.
var fullDestPath = Path.Combine(destPath, filename);
// Ensure the destination directories exist. Don't pass in the filename to CreateDirectory!
Directory.CreateDirectory(destPath);
Then you can copy the file (synchronously) like this:
File.Copy(sourcePath, fullDestPath);
I have a directory of around 30-40 folders that contain various backup files for a CRM system.
I have developed a script that downloads the files from a remote server, and places them in folders with YYYYMMDD, however due to space restrictions I now need to move the oldest folder from the directory. As the IT team at the company keep moving the folders between servers I cannot use the folder creation date!
What is the easiest option? I have looked at: deleting the oldest folder by identifying from the folder name and attempted to order the items then perform a move.
My other option was to take all of the folder names in the root directory, parse into a list of type time and date, select the lowest (oldest) option, then perform the file move?
how about something like this:
bool MoveOldestFolder(string initialFolderName, string destinationFolder)
{
// gets all top folders in your chosen location
var directories = System.IO.Directory.EnumerateDirectories(initialFolderName,"*", System.IO.SearchOption.TopDirectoryOnly);
// stores the oldest folder and it's date at the end of algorithm
DateTime outDate;
DateTime oldestDate = DateTime.MaxValue;
string resultFolder = string.Empty;
// just a temp variable
string tmp;
// using LINQ
directories.ToList().ForEach(p =>
{
tmp = new System.IO.FileInfo(p).Name; // get the name of the current folder
if (DateTime.TryParseExact(tmp,
"yyyyMMdd", // this is case sensitive!
System.Globalization.CultureInfo.InvariantCulture,
System.Globalization.DateTimeStyles.None,
out outDate)) // try using folder name as date that "should" be in yyyyMMdd format - if the conversion is successful and date is older than current outDate, then store folder name and date, else nothing
{
if (outDate.Date < oldestDate.Date)
{
oldestDate = outDate;
resultFolder = p;
}
}
});
// if we actually found a folder that is formatted in yyyyMMdd format
if (!oldestDate.Equals(DateTime.MaxValue))
{
try
{
System.IO.Directory.Move(resultFolder, destinationFolder);
return true;
}
catch(Exception ex)
{
// handle the excaption
return false;
}
}
else
{
// we didnt find anything
return false;
}
}
private void button1_Click(object sender, EventArgs e)
{
var initialFolderName = #"C:\initial";
var destinationFolder = #"c:\dest";
if (MoveOldestFolder(initialFolderName, destinationFolder))
{
// move was successful
}
else
{
// something went wrong
}
}
Other option would be to simply do what chrfin said but I wouldn't presume on everything being "dandy" in the folder structure. There is always a possibility that the folder name is not in YYYYMMDD format and that would probably cause some problems I imagine.
Anyway, the code could look something like this:
var directories = System.IO.Directory.EnumerateDirectories(initialFolderName,"*", System.IO.SearchOption.TopDirectoryOnly);
directories.ToList<string>().Sort();
var lastDir = directories.First();
my requirement is to move files from one directory to another after certain interval. So basic copy works, but during the next subsequent interval I want to move only the new files.
Following is my approach:
I am creating the file list of both source & target directory in target location, the idea is based on the difference of these two files copy only the files that are new from last iteration.
For 1st iteration it will create a blank file in target indicating copy everything. But my file comparison is hitting issues here, based on logic in below code it's creating this excpetion "System.Linq.Enumerable+d__99`1[System.String]"
Here's the code:
static void Main(string[] args)
{
create_source_fileList();
string source_dir = System.Configuration.ConfigurationSettings.AppSettings["SourceDir"];
string target_dir = System.Configuration.ConfigurationSettings.AppSettings["TargetDir"];
string dpath = target_dir + "Diff" + ".txt";
TextWriter df = new StreamWriter(dpath);
DirectoryInfo sourceinfo = new DirectoryInfo(source_dir);
DirectoryInfo targetinfo = new DirectoryInfo(target_dir);
string[] source_f_list = File.ReadAllLines(target_dir + "Source_File_List.txt");
string[] target_f_list = File.ReadAllLines(target_dir + "Target_File_List.txt");
IEnumerable<String> file_list_diff = source_f_list.Except(target_f_list);
df.WriteLine(file_list_diff);
df.Close();
if (!Directory.Exists(targetinfo.FullName))
{
Directory.CreateDirectory(targetinfo.FullName);
}
foreach (FileInfo fi in sourceinfo.GetFiles())
{
fi.CopyTo(Path.Combine(targetinfo.ToString(), fi.Name), true);
}
create_target_fileList();
}
Need help in fixing this issue,also will this approach down the line work in loop where I will iterate only the names in diff file.
Thanks!!
It's not creating an exception; rather, it's writing the result of calling ToString on an IEnumerable<string> instance. You need to replace:
df.WriteLine(file_list_diff);
with
df.WriteLine(string.Join(Environment.NewLine, file_list_diff));
or, prior to .NET 4:
df.WriteLine(string.Join(Environment.NewLine, file_list_diff.ToArray()));
I am using StraemWriter to log text messages to a log file. The log file should be created if it doesn't exist, appended to if the file creation date is less than a given time or recreated if created before that time. I am using the class/code below
public static class LogIt
{
private const string LOG_FNAME = #"Logfile.log";
public static void WriteMsg(string msg)
{
bool append = true;
if (File.Exists(LOG_FNAME))
{
//DateTime delDate = DateTime.Now.AddDays(-1);
DateTime delDate = DateTime.Now.AddMinutes(-30);
DateTime fileCreatedDate = File.GetCreationTime(LOG_FNAME);
if (DateTime.Compare(fileCreatedDate, delDate) < 0)
{
Console.WriteLine("DELETE FILE");
File.Delete(LOG_FNAME);
}
}
using (StreamWriter sw = new StreamWriter(LOG_FNAME, append))
{
sw.WriteLine(msg);
}
Console.WriteLine(msg);
}
}
This class is used by a simple console app run by the Task Scheduler which runs every x minutes.
The message are written as follows:
LogIt.WriteMsg("Log this message");
The messages are logged file when the file is initially created however when the file creation date is past the delete date, the file is recreated but no subsequent messages are ever written to the file.
Any ideas on why?
For some reason the file has the initial creation date (first time a file with that path ever created) as the creation date even if it is recreated after deleting. You can check the file properties and see that the log file creation date is always the same. A work around would be to update the file creation date in code whenever you recreate the file. You can use FileInfo class for that.
#MPD No problem. Here is the implementation of the workaround I suggested. Give it a try and let me know if that works.
private const string LOG_FNAME = #"Logfile.log";
public static void WriteMsg(string msg)
{
bool deleted = false;
bool append = true;
if (File.Exists(LOG_FNAME))
{
//DateTime delDate = DateTime.Now.AddDays(-1);
DateTime delDate = DateTime.Now.AddMinutes(-30);
DateTime fileCreatedDate = File.GetCreationTime(LOG_FNAME);
if (DateTime.Compare(fileCreatedDate, delDate) < 0)
{
Console.WriteLine("DELETE FILE");
File.Delete(LOG_FNAME);
//record that file was deleted and a new one will be created
deleted = true;
}
}
using (StreamWriter sw = new StreamWriter(LOG_FNAME, append))
{
sw.WriteLine(msg);
}
if (deleted)
{
//a new file is created. Make sure the creation time is set
FileInfo fi = new FileInfo(LOG_FNAME);
fi.CreationTime = DateTime.Now;
}
Console.WriteLine(msg);
}
I guess you are running this code on Windows 2003 (or maybe XP). If so: When you create a file in some directory at time T1 and the delete it and then re-create it; surprise surprise it has T1 as creation date!
I know this just because I had the same problem on Windows 2003!
BTW I use NLog now and (IMHO) it's perfect and has everything I need.