I have code set up to do an FTP PUT with a file to an FTP server. First I have a method that checks if the file exists at the target location. Then if it does I have another method that deletes the file. Then I perform the FTP PUT to the target location.
Currently, I'm performing these 3 methods by setting up 3 separate FTP connections to the same server. However, I want to perform all 3 methods with one connection to the server. The reason is because I'm getting the following error after opening multiple connections to same FTP server: "An existing connection was forcibly closed by the remote host."
Here are the 3 functions below. The first method, GetFileFromRemoteServer, is used to see if a file exists on the FTP server at target path. I use regex in some cases to get partial name match, or in other cases just do full name match.
I researched online that someone said it's possible to use the same ftp request object and just perform all methods you want and then close the connection. I tried to see if it works performing multiple methods on same request object and I got this error: This operation cannot be performed after the request has been submitted.
Is there a way to perform all of them using one connection to the server?
Thank you, I really appreciate your help!
public static List<FTPLineResult> GetFileFromRemoteServer(bool isSsl, string username, string password, string fileName, string dir, Regex regexPattern,
bool getSingleFile = false)
{
var output = new List<FTPLineResult>();
var parser = new FTPLineParser();
var isDone = false;
var request = (FtpWebRequest)WebRequest.Create(dir);
request.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
request.ConnectionGroupName = ConfigurationManager.AppSettings["ftpConnectionGroup"];
request.KeepAlive = true;
request.Credentials = new NetworkCredential(username, password);
request.UsePassive = true;
if (isSsl)
{
request.EnableSsl = true;
}
else
{
request.EnableSsl = false;
}
using (var response = (FtpWebResponse)request.GetResponse())
{
using (var responseStream = response.GetResponseStream())
{
using (var reader = new StreamReader(responseStream, Encoding.ASCII))
{
while (!isDone && !reader.EndOfStream)
{
var result = parser.Parse(reader.ReadLine());
//if "*" is in file name, which means get partial match, replacing * with real file name content
if (regexPattern != null)
{
if (regexPattern.IsMatch(result.Name.ToLower().Trim()))
{
output.Add(result);
}
}
else if (result.Name.ToLower().Trim() == fileName.ToLower().Trim())
{
output.Add(result);
isDone = true;
}
}
return output;
}
}
}
}
private void DeleteExistingTargetFile()
{
// Get the object used to communicate with the server.
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(_params.FinalFolderTarget);
request.Method = WebRequestMethods.Ftp.DeleteFile;
request.Credentials = new NetworkCredential(_params.Username, _params.Password);
request.UsePassive = true;
request.ConnectionGroupName = ConfigurationManager.AppSettings["ftpConnectionGroup"];
request.KeepAlive = true;
if (_params.IsSsl)
{
request.EnableSsl = true;
}
else
{
request.EnableSsl = false;
}
using (FtpWebResponse response = (FtpWebResponse)request.GetResponse())
{
var status = response.StatusDescription;
}
}
private void DoFtpPut(Dictionary<StatusEnum, string> statusDict)
{
int buffLength = 2048;
byte[] buff = new byte[buffLength];
System.IO.FileInfo _FileInfo = new System.IO.FileInfo(_params.SourceFilename);
var request = (FtpWebRequest)WebRequest.Create(new Uri(_params.TargetFilename));
request.Method = WebRequestMethods.Ftp.UploadFile;
request.ConnectionGroupName = ConfigurationManager.AppSettings["ftpConnectionGroup"];
request.KeepAlive = true;
request.Credentials = new NetworkCredential(_params.Username, _params.Password);
request.UsePassive = true;
if (_params.IsSsl)
{
request.EnableSsl = true;
}
else
{
request.EnableSsl = false;
}
using (var _Stream = request.GetRequestStream())
{
//read file one chunk at a time in order to avoid out of memory exception
using (var fileStream = _FileInfo.OpenRead())
{
var contentLen = fileStream.Read(buff, 0, buffLength);
while (contentLen != 0)
{
_Stream.Write(buff, 0, contentLen);
contentLen = fileStream.Read(buff, 0, buffLength);
}
}
}
statusDict[StatusEnum.ftpStatus] = Constants.SUCCESS_STATUS;
}
I couldn't figure out a way to do FTPPUT with only one connection using FtpWebRequest class. However, using FtpLib library allowed me to do exactly what I wanted, which was to check if file exists on ftp server target location, if it does then delete it, and then do ftp put, and finally move file to final location using a rename.
Here's where I downloaded ftplib library: ftplib.codeplex.com
Here's the code below:
using (FtpConnection ftp = new FtpConnection(host, _params.Username, _params.Password))
{
try
{
ftp.Open(); /* Open the FTP connection */
ftp.Login(); /* Login using previously provided credentials */
ftp.PutFile(_params.SourceFilename, _params.TargetFilename); /* upload /incoming/file.txt as file.txt to current executing directory, overwrite if it exists */
if (!ftp.DirectoryExists(_params.FinalDir)) /* check that a directory exists */
{
ftp.CreateDirectory(_params.FinalDir);
}
if (ftp.FileExists(_params.FinalLocation))
{
ftp.RemoveFile(_params.FinalLocation);
}
ftp.RenameFile(target, _params.FinalLocation);
statusDict[StatusEnum.ftpStatus] = Constants.SUCCESS_STATUS;
}
catch (Exception ex)
{
statusDict[StatusEnum.ftpStatus] = Constants.ERROR_STATUS;
}
}
Related
I would really need some help to change from FTP transfer to SFTP. I added some lines in the comment but I'm new to programming and I don't know for sure what I also need to add to fix this problem. The programme works fine but at the moment you transfer your data via FTP. I read some questions here but for now, I couldn't find the right tipp rewrite the program to SFTP.
public class FTP
{
private System.ComponentModel.BackgroundWorker bw = null;
private long filesSize = 0;
private long uploadSize = 0;
private const int bufferLength = 2048;
public FTP()
{
}
public FTP(System.ComponentModel.BackgroundWorker thread, long allFilesSize)
{
bw = thread;
filesSize = allFilesSize;
}
public string ReadUserFile()
{
WebClient req = new WebClient();
req.Credentials = new NetworkCredential("", "");
try
{
byte[] newFileData = req.DownloadData();
return System.Text.Encoding.Default.GetString(newFileData);
}
catch
{
return null;
}
}
public void UploadFile(
string ftpServer, string filePath, string username, string password)
{
//Create SFTP request
//Sftp client = new Sftp();
//client.Connect(hostname);
//client.Login(username, password);
//Create FTP request
FtpWebRequest request = (FtpWebRequest)FtpWebRequest.Create();
request.Method = WebRequestMethods.Ftp.UploadFile;
request.Credentials = new NetworkCredential(username, password);
request.UsePassive = true;
request.UseBinary = true;
request.KeepAlive = false;
//Load the file
//
//Load the file
FileStream stream = File.OpenRead(filePath);
//Upload to Sftp Server
//client.PutFile();
//Upload file
Stream reqStream = request.GetRequestStream();
byte[] buffer = new byte[bufferLength];
int count = 0;
int readBytes = 0;
do
{
readBytes = stream.Read(buffer, 0, bufferLength);
reqStream.Write(buffer, 0, readBytes);
count += readBytes;
if (bw != null)
bw.ReportProgress(CalculateProgress());
}
while (readBytes != 0);
//Disconnect
//client.Disconnect();
stream.Close();
reqStream.Close();
}
private int CalculateProgress()
{
uploadSize += bufferLength;
return (Int32)(uploadSize/(filesSize / 100));
}
public void DeleteFile(
string ftpServer, string filePath, string username, string password)
{
//Create SFTP request
//Sftp client = new Sftp();
//client.Connect(hostname);
//client.Login(username, password);
//Create FTP request
FtpWebRequest request = (FtpWebRequest)FtpWebRequest.Create();
request.Method = WebRequestMethods.Ftp.DeleteFile;
request.Credentials = new NetworkCredential(username, password);
request.UsePassive = true;
request.UseBinary = true;
request.KeepAlive = false;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
response.Close();
}
public string[] GetFileList(
string ftpServer, string username, string password)
{
string[] fileList;
StringBuilder result = new StringBuilder();
FtpWebRequest request;
try
{
request = (FtpWebRequest)FtpWebRequest.Create(new Uri());
request.UseBinary = true;
request.Credentials = new NetworkCredential(username, password);
request.Method = WebRequestMethods.Ftp.ListDirectory;
WebResponse response = request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string line = reader.ReadLine();
while (line != null)
{
if (line.StartsWith("./"))
line = line.Substring(2, line.Length - 2);
result.Append(line);
result.Append("\n");
line = reader.ReadLine();
}
result.Remove(result.ToString().LastIndexOf('\n'), 1);
reader.Close();
response.Close();
return result.ToString().Split('\n');
}
catch
{
//Error
fileList = null;
return fileList;
}
}
There's no simple way to switch from FTP to SFTP in C#/.NET, if you are currently using .NET FtpWebRequest API.
There's no support for SFTP in .NET framework.
You need 3rd party library: SFTP Libraries for .NET.
That also means that you basically need to scratch your current code and start from the very beginning. Not a single line in your current code would be useful for SFTP with any 3rd party library, as they have very different API to unusual FtpWebRequest.
There are libraries that offer an uniform interface to both FTP and SFTP protocols.
For example WinSCP .NET assembly supports FTP, FTPS, FTPES, SFTP and others (SCP, S3,
WebDAV and WebDAVS) over the same interface.
Though it is not a native .NET assembly. It's rather a thin wrapper over
a console application.
(I'm the author of WinSCP).
Looking for the best way to check for a given directory via FTP.
Currently i have the following code:
private bool FtpDirectoryExists(string directory, string username, string password)
{
try
{
var request = (FtpWebRequest)WebRequest.Create(directory);
request.Credentials = new NetworkCredential(username, password);
request.Method = WebRequestMethods.Ftp.GetDateTimestamp;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
FtpWebResponse response = (FtpWebResponse)ex.Response;
if (response.StatusCode == FtpStatusCode.ActionNotTakenFileUnavailable)
return false;
else
return true;
}
return true;
}
This returns false whether the directory is there or not. Can someone point me in the right direction.
Basically trapped the error that i receive when creating the directory like so.
private bool CreateFTPDirectory(string directory) {
try
{
//create the directory
FtpWebRequest requestDir = (FtpWebRequest)FtpWebRequest.Create(new Uri(directory));
requestDir.Method = WebRequestMethods.Ftp.MakeDirectory;
requestDir.Credentials = new NetworkCredential("username", "password");
requestDir.UsePassive = true;
requestDir.UseBinary = true;
requestDir.KeepAlive = false;
FtpWebResponse response = (FtpWebResponse)requestDir.GetResponse();
Stream ftpStream = response.GetResponseStream();
ftpStream.Close();
response.Close();
return true;
}
catch (WebException ex)
{
FtpWebResponse response = (FtpWebResponse)ex.Response;
if (response.StatusCode == FtpStatusCode.ActionNotTakenFileUnavailable)
{
response.Close();
return true;
}
else
{
response.Close();
return false;
}
}
}
The complete solution will now be:
public bool DoesFtpDirectoryExist(string dirPath)
{
try
{
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(dirPath);
request.Method = WebRequestMethods.Ftp.ListDirectory;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
return true;
}
catch(WebException ex)
{
return false;
}
}
// Calling the method (note the forwardslash at the end of the path):
string ftpDirectory = "ftp://ftpserver.com/rootdir/test_if_exist_directory/";
bool dirExists = DoesFtpDirectoryExist(ftpDirectory);
Originally, I was using,
string ftpDirectory = "ftp://ftpserver.com/rootdir/test_if_exist_directory";
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(ftpDirectory);
request.Method = WebRequestMethods.Ftp.ListDirectory;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
and waited for an exception in case the directory didn't exist. This method didn't throw an exception.
After a few hit and trials, I changed the directory from:
ftp://ftpserver.com/rootdir/test_if_exist_directory
to:
ftp://ftpserver.com/rootdir/test_if_exist_directory/
Now the code is working for me.
I think we should append a forward slash (/) to the URI of the FTP folder to get it to work.
I assume that you are already somewhat familiar with FtpWebRequest, as this is the usual way to access FTP in .NET.
You can attempt to list the directory and check for an error StatusCode.
try
{
FtpWebRequest request = (FtpWebRequest)WebRequest.Create("ftp://ftp.microsoft.com/12345");
request.Method = WebRequestMethods.Ftp.ListDirectory;
using (FtpWebResponse response = (FtpWebResponse)request.GetResponse())
{
// Okay.
}
}
catch (WebException ex)
{
if (ex.Response != null)
{
FtpWebResponse response = (FtpWebResponse)ex.Response;
if (response.StatusCode == FtpStatusCode.ActionNotTakenFileUnavailable)
{
// Directory not found.
}
}
}
I would try something along this lines:
Send MLST <directory> FTP command (defined in RFC3659) and parse it's output. It should return valid line with directory details for existing directories.
If MLST command is not available, try changing the working directory into the tested directory using a CWD command. Don't forget to determine the current path (PWD command) prior to changing to a tested directory to be able to go back.
On some servers combination of MDTM and SIZE command can be used for similar purpose, but the behavior is quite complex and out of scope of this post.
This is basically what DirectoryExists method in the current version of our Rebex FTP component does. The following code shows how to use it:
string path = "/path/to/directory";
Rebex.Net.Ftp ftp = new Rebex.Net.Ftp();
ftp.Connect("hostname");
ftp.Login("username","password");
Console.WriteLine(
"Directory '{0}' exists: {1}",
path,
ftp.DirectoryExists(path)
);
ftp.Disconnect();
User this code it may be your answer..
public bool FtpDirectoryExists(string directoryPath, string ftpUser, string ftpPassword)
{
bool IsExists = true;
try
{
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(directoryPath);
request.Credentials = new NetworkCredential(ftpUser, ftpPassword);
request.Method = WebRequestMethods.Ftp.PrintWorkingDirectory;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
IsExists = false;
}
return IsExists;
}
I have called this method as:
bool result = FtpActions.Default.FtpDirectoryExists( #"ftp://mydomain.com/abcdir", txtUsername.Text, txtPassword.Text);
Why use another library - create your own logic's.
I tried every which way to get a solid check but neither the WebRequestMethods.Ftp.PrintWorkingDirectory nor WebRequestMethods.Ftp.ListDirectory methods would work correctly. They failed when checking for ftp://<website>/Logs which doesnt exist on the server but they say it does.
So the method I came up with was to try to upload to the folder. However, one 'gotcha' is the path format which you can read about in this thread Uploading to Linux
Here is a code snippet
private bool DirectoryExists(string d)
{
bool exists = true;
try
{
string file = "directoryexists.test";
string path = url + homepath + d + "/" + file;
//eg ftp://website//home/directory1/directoryexists.test
//Note the double space before the home is not a mistake
//Try to save to the directory
req = (FtpWebRequest)WebRequest.Create(path);
req.ConnectionGroupName = "conngroup1";
req.Method = WebRequestMethods.Ftp.UploadFile;
if (nc != null) req.Credentials = nc;
if (cbSSL.Checked) req.EnableSsl = true;
req.Timeout = 10000;
byte[] fileContents = System.Text.Encoding.Unicode.GetBytes("SAFE TO DELETE");
req.ContentLength = fileContents.Length;
Stream s = req.GetRequestStream();
s.Write(fileContents, 0, fileContents.Length);
s.Close();
//Delete file if successful
req = (FtpWebRequest)WebRequest.Create(path);
req.ConnectionGroupName = "conngroup1";
req.Method = WebRequestMethods.Ftp.DeleteFile;
if (nc != null) req.Credentials = nc;
if (cbSSL.Checked) req.EnableSsl = true;
req.Timeout = 10000;
res = (FtpWebResponse)req.GetResponse();
res.Close();
}
catch (WebException ex)
{
exists = false;
}
return exists;
}
Navigate to the parent directory, execute the "ls" command, and parse the result.
I couldn't get this #BillyLogans suggestion to work....
I found the problem was the default FTP directory was /home/usr/fred
When I used:
String directory = "ftp://some.domain.com/mydirectory"
FtpWebRequest requestDir = (FtpWebRequest)FtpWebRequest.Create(new Uri(directory));
I found this gets turned into
"ftp:/some.domain.com/home/usr/fred/mydirectory"
to stop this change the directory Uri to:
String directory = "ftp://some.domain.com//mydirectory"
Then this starts working.
This was my best. get list from parent dir, and check if parent have correct child name
public void TryConnectFtp(string ftpPath)
{
string[] splited = ftpPath.Split('/');
StringBuilder stb = new StringBuilder();
for (int i = 0; i < splited.Length - 1; i++)
{
stb.Append(splited[i] +'/');
}
string parent = stb.ToString();
string child = splited.Last();
FtpWebRequest testConnect = (FtpWebRequest)WebRequest.Create(parent);
testConnect.Method = WebRequestMethods.Ftp.ListDirectory;
testConnect.Credentials = credentials;
using (FtpWebResponse resFtp = (FtpWebResponse)testConnect.GetResponse())
{
StreamReader reader = new StreamReader(resFtp.GetResponseStream());
string result = reader.ReadToEnd();
if (!result.Contains(child) ) throw new Exception("###");
resFtp.Close();
}
}
The only way which worked for me was an inversed logic by trying to create the directory/path (which will throw an exception if it already exists) and if so delete it again afterwards. Otherwise use the Exception to set a flag meaing that the directory/path exists. I'm quite new to VB.NET and I'm shure there's a nicer way to code this - but anyway here's my code:
Public Function DirectoryExists(directory As String) As Boolean
' Reversed Logic to check if a Directory exists on FTP-Server by creating the Directory/Path
' which will throw an exception if the Directory already exists. Otherwise create and delete the Directory
' Adjust Paths
Dim path As String
If directory.Contains("/") Then
path = AdjustDir(directory) 'ensure that path starts with a slash
Else
path = directory
End If
' Set URI (formatted as ftp://host.xxx/path)
Dim URI As String = Me.Hostname & path
Dim response As FtpWebResponse
Dim DirExists As Boolean = False
Try
Dim request As FtpWebRequest = DirectCast(WebRequest.Create(URI), FtpWebRequest)
request.Credentials = Me.GetCredentials
'Create Directory - if it exists WebException will be thrown
request.Method = WebRequestMethods.Ftp.MakeDirectory
'Delete Directory again - if above request did not throw an exception
response = DirectCast(request.GetResponse(), FtpWebResponse)
request = DirectCast(WebRequest.Create(URI), FtpWebRequest)
request.Credentials = Me.GetCredentials
request.Method = WebRequestMethods.Ftp.RemoveDirectory
response = DirectCast(request.GetResponse(), FtpWebResponse)
DirExists = False
Catch ex As WebException
DirExists = True
End Try
Return DirExists
End Function
WebRequestMethods.Ftp.MakeDirectory and WebRequestMethods.Ftp.RemoveDirectory are the Methods i used for this. All other solutions did not work for me.
Hope it helps
For what it is worth, You'll make your FTP life quite a bit easier if you use EnterpriseDT's FTP component. It's free and will save you headaches because it deals with the commands and responses. You just work with a nice, simple object.
Following test works...
public void test1()
{
string server="ftp://myserver.com/dev";
string userName="myusername";
string password="mypassword";
FtpWebRequest req = (FtpWebRequest)WebRequest.Create( server );
req.Credentials = new NetworkCredential( userName, password );
req.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
req.Timeout = 30000;
req.UseBinary = false;
req.EnableSsl = false;
req.UsePassive = false;
req.KeepAlive = true;
using( FtpWebResponse resp = (FtpWebResponse)req.GetResponse() )
{
using (StreamReader sr = new StreamReader(resp.GetResponseStream()))
{
string fileRecord = sr.ReadLine();
while (fileRecord != null)
{
Console.WriteLine( fileRecord );
fileRecord = sr.ReadLine();
}
}
}
}
While the following test fails...
public void test2()
{
string server="ftp://myserver.com/dev";
string userName="myusername";
string password="mypassword";
FtpWebRequest req = (FtpWebRequest)WebRequest.Create( server );
req.Credentials = new NetworkCredential( userName, password );
req.Method = WebRequestMethods.Ftp.GetDateTimestamp;
req.Timeout = 30000;
req.UseBinary = false;
req.EnableSsl = false;
req.UsePassive = false;
req.KeepAlive = true;
using( FtpWebResponse resp = (FtpWebResponse)req.GetResponse() )
{
using( StreamReader sr = new StreamReader( resp.GetResponseStream() ) )
{
Console.WriteLine( resp.LastModified );
}
}
}
with error message:
Test method test2 threw exception: System.Net.WebException: The remote server returned an error: (550) File unavailable (e.g., file not found, no access).
UPDATE: I tried with another ftp site (unix) that uses the default port#, so the url is "ftp://myserver.com/dev" - and the GetDateTimestamp() still dies with the same error.
I have updated the subject line and the body of the question to reflect my query properly.
Please add more information.
Guess so far: You are trying to
a) Do a ls on the FTP server (works)
b) Get at timestamp from the FTP
server (doesn't work)
Since everything else seems the same (address etc) I assume that both look at the same data. I would imagine that an ls just works when you're connected. But what timestamp are you trying to get there? The documentation for WebRequestMethods.Ftp.GetDateTimestamp says
Represents the FTP MDTM protocol
method that is used to retrieve the
date-time stamp from a file on an FTP
server.
(Emphasis by me)
Which file? As far as I can see you are only specifying a folder (not sure if that works)? Did you try this with "ftp://myserver.com/dev/text.txt"?
It looks like the URI in the 2 examples are the same, your test cases don't match the description of the problem. Can you be more specific?
(In general its easiest to add have a single snippet of example code with the one or two lines that are breaking highlighted.)
How can I implement a FileSystemWatcher for an FTP location (in C#). The idea is whenever anything gets added in the FTP location I wish to copy it to my local machine. Any ideas will be helpful.
This is a follow up of my previous question Selective FTP download using .NET.
You're going to have to implement a polling solution, where you keep asking for the directory content periodically. Compare this to a cached list from the previous call and determine what happened that way.
There's nothing in the FTP protocol that will help you with this unfortunately.
You cannot use the FileSystemWatcher or any other way, because the FTP protocol does not have any API to notify a client about changes in the remote directory.
All you can do is to periodically iterate the remote tree and find changes.
It's actually rather easy to implement, if you use an FTP client library that supports recursive listing of a remote tree. Unfortunately, the built-in .NET FTP client, the FtpWebRequest does not. But for example with WinSCP .NET assembly, you can use the Session.EnumerateRemoteFiles method.
See the article Watching for changes in SFTP/FTP server:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "example.com",
UserName = "user",
Password = "password",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
List<string> prevFiles = null;
while (true)
{
// Collect file list
List<string> files =
session.EnumerateRemoteFiles(
"/remote/path", "*.*", EnumerationOptions.AllDirectories)
.Select(fileInfo => fileInfo.FullName)
.ToList();
if (prevFiles == null)
{
// In the first round, just print number of files found
Console.WriteLine("Found {0} files", files.Count);
}
else
{
// Then look for differences against the previous list
IEnumerable<string> added = files.Except(prevFiles);
if (added.Any())
{
Console.WriteLine("Added files:");
foreach (string path in added)
{
Console.WriteLine(path);
}
}
IEnumerable<string> removed = prevFiles.Except(files);
if (removed.Any())
{
Console.WriteLine("Removed files:");
foreach (string path in removed)
{
Console.WriteLine(path);
}
}
}
prevFiles = files;
Console.WriteLine("Sleeping 10s...");
Thread.Sleep(10000);
}
}
(I'm the author of WinSCP)
Though, if you actually want to just download the changes, it's a way easier. Just use the Session.SynchronizeDirectories in the loop.
session.SynchronizeDirectories(
SynchronizationMode.Local, "/remote/path", #"C:\local\path", true).Check();
See the article Keep local directory up to date (download changed files from remote SFTP/FTP server).
If you do not want to use a 3rd party library, you have to do with limitations of the FtpWebRequest. For an example how to recursively list a remote directory tree with the FtpWebRequest, see my answer to C# Download all files and subdirectories through FTP.
The FileSystemWatcher class works by registering for events with the host Windows operating system. As such, it is limited to working on local paths and UNC paths to directories hosted on Windows systems. The MSDN documentation on FileSystemWatcher explains the paths which you can use and some of the potential problems with using the class.
If you are looking to be alerted to changes on an FTP site, you will have to use a polling mechanism to ask for the current status of files or folders you are interested in monitoring. You will be able to see when files are added and removed by comparing snapshots of the FTP site for changes and raising similar events when you detect changes. Unfortunately you wont be able to detect rename events, but other changes should be simple to monitor this way.
Write a simple service to create FileSystemWatcher, pointing at your ftp location.
Then when a file is uploaded or modified, an event will be fired in your service, which you can then use to copy the file to your local machine.
File.Copy etc.
Hav a look at: this blog
You can monitor the FTP location by following method:
public class FtpFileSystemWatcher
{
public bool IsRunning
{
get;
private set;
}
public string FtpUserName
{
get;
set;
}
public string FtpPassword
{
get;
set;
}
public string FtpLocationToWatch
{
get;
set;
}
public string DownloadTo
{
get;
set;
}
public bool KeepOrignal
{
get;
set;
}
public bool OverwriteExisting
{
get;
set;
}
public int RecheckIntervalInSeconds
{
get;
set;
}
private bool DownloadInprogress
{
get;
set;
}
private System.Timers.Timer JobProcessor;
public FtpFileSystemWatcher(string FtpLocationToWatch = "", string DownloadTo = "", int RecheckIntervalInSeconds = 1, string UserName = "", string Password = "", bool KeepOrignal = false, bool OverwriteExisting = false)
{
this.FtpUserName = UserName;
this.FtpPassword = Password;
this.FtpLocationToWatch = FtpLocationToWatch;
this.DownloadTo = DownloadTo;
this.KeepOrignal = KeepOrignal;
this.RecheckIntervalInSeconds = RecheckIntervalInSeconds;
this.OverwriteExisting = OverwriteExisting;
if (this.RecheckIntervalInSeconds < 1) this.RecheckIntervalInSeconds = 1;
}
public void StartDownloading()
{
JobProcessor = new Timer(this.RecheckIntervalInSeconds * 1000);
JobProcessor.AutoReset = false;
JobProcessor.Enabled = false;
JobProcessor.Elapsed += (sender, e) =>
{
try
{
this.IsRunning = true;
string[] FilesList = GetFilesList(this.FtpLocationToWatch, this.FtpUserName, this.FtpPassword);
if (FilesList == null || FilesList.Length < 1)
{
return;
}
foreach (string FileName in FilesList)
{
if (!string.IsNullOrWhiteSpace(FileName))
{
DownloadFile(this.FtpLocationToWatch, this.DownloadTo, FileName.Trim(), this.FtpUserName, this.FtpPassword, this.OverwriteExisting);
if (!this.KeepOrignal)
{
DeleteFile(Path.Combine(this.FtpLocationToWatch, FileName.Trim()), this.FtpUserName, this.FtpPassword);
}
}
}
this.IsRunning = false;
JobProcessor.Enabled = true;
}
catch (Exception exp)
{
this.IsRunning = false;
JobProcessor.Enabled = true;
Console.WriteLine(exp.Message);
}
};
JobProcessor.Start();
}
public void StopDownloading()
{
try
{
this.JobProcessor.Dispose();
this.IsRunning = false;
}
catch { }
}
private void DeleteFile(string FtpFilePath, string UserName, string Password)
{
FtpWebRequest FtpRequest;
FtpRequest = (FtpWebRequest)FtpWebRequest.Create(new Uri(FtpFilePath));
FtpRequest.UseBinary = true;
FtpRequest.Method = WebRequestMethods.Ftp.DeleteFile;
FtpRequest.Credentials = new NetworkCredential(UserName, Password);
FtpWebResponse response = (FtpWebResponse)FtpRequest.GetResponse();
response.Close();
}
private void DownloadFile(string FtpLocation, string FileSystemLocation, string FileName, string UserName, string Password, bool OverwriteExisting)
{
try
{
const int BufferSize = 2048;
byte[] Buffer = new byte[BufferSize];
FtpWebRequest Request;
FtpWebResponse Response;
if (File.Exists(Path.Combine(FileSystemLocation, FileName)))
{
if (OverwriteExisting)
{
File.Delete(Path.Combine(FileSystemLocation, FileName));
}
else
{
Console.WriteLine(string.Format("File {0} already exist.", FileName));
return;
}
}
Request = (FtpWebRequest)FtpWebRequest.Create(new Uri(Path.Combine(FtpLocation, FileName)));
Request.Credentials = new NetworkCredential(UserName, Password);
Request.Proxy = null;
Request.Method = WebRequestMethods.Ftp.DownloadFile;
Request.UseBinary = true;
Response = (FtpWebResponse)Request.GetResponse();
using (Stream s = Response.GetResponseStream())
{
using (FileStream fs = new FileStream(Path.Combine(FileSystemLocation, FileName), FileMode.CreateNew, FileAccess.ReadWrite))
{
while (s.Read(Buffer, 0, BufferSize) != -1)
{
fs.Write(Buffer, 0, BufferSize);
}
}
}
}
catch { }
}
private string[] GetFilesList(string FtpFolderPath, string UserName, string Password)
{
try
{
FtpWebRequest Request;
FtpWebResponse Response;
Request = (FtpWebRequest)FtpWebRequest.Create(new Uri(FtpFolderPath));
Request.Credentials = new NetworkCredential(UserName, Password);
Request.Proxy = null;
Request.Method = WebRequestMethods.Ftp.ListDirectory;
Request.UseBinary = true;
Response = (FtpWebResponse)Request.GetResponse();
StreamReader reader = new StreamReader(Response.GetResponseStream());
string Data = reader.ReadToEnd();
return Data.Split('\n');
}
catch
{
return null;
}
}
}
The way I handle this is to upload a one element byte array, named ".ftpComplete". The FileSystemWatcher only watches for ".ftpComplete" files, and strips that off the end to know the actual file uploaded. Since the".ftpComplete" file is only 1 byte, it uploads about as fast as it is created on the FTP server, so it can be deleted once you do whatever you need to with the main uploaded file
FtpWebRequest request = (FtpWebRequest)FtpWebRequest.Create(
FTPAddress + "/" + Path.GetFileName(filePath) + ".ftpComplete");
request.Method = WebRequestMethods.Ftp.UploadFile;
request.Credentials = new NetworkCredential(username, password);
request.UsePassive = true;
request.UseBinary = true;
request.KeepAlive = false;
byte[] buffer = new byte[1];
Stream reqStream = request.GetRequestStream();
reqStream.Write(buffer, 0, buffer.Length);
reqStream.Close();
You could use a Robo-FTP script to monitor the FTP site for changes. Here is a link to a sample script that sends an email whenever a change is detected: http://kb.robo-ftp.com/script_library/show/40
I looked at the previous question that you linked. I think you should be able to modify the Robo-FTP sample and use the SETLEFT command with the /split option to make it parse the folder name and ISO file number of the changed file and then move the file to the proper location.
I need to use FtpWebRequest to put a file in a FTP directory. Before the upload, I would first like to know if this file exists.
What method or property should I use to check if this file exists?
var request = (FtpWebRequest)WebRequest.Create
("ftp://ftp.domain.com/doesntexist.txt");
request.Credentials = new NetworkCredential("user", "pass");
request.Method = WebRequestMethods.Ftp.GetFileSize;
try
{
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
}
catch (WebException ex)
{
FtpWebResponse response = (FtpWebResponse)ex.Response;
if (response.StatusCode ==
FtpStatusCode.ActionNotTakenFileUnavailable)
{
//Does not exist
}
}
As a general rule it's a bad idea to use Exceptions for functionality in your code like this, however in this instance I believe it's a win for pragmatism. Calling list on the directory has the potential to be FAR more inefficient than using exceptions in this way.
If you're not, just be aware it's not good practice!
EDIT: "It works for me!"
This appears to work on most ftp servers but not all. Some servers require sending "TYPE I" before the SIZE command will work. One would have thought that the problem should be solved as follows:
request.UseBinary = true;
Unfortunately it is a by design limitation (big fat bug!) that unless FtpWebRequest is either downloading or uploading a file it will NOT send "TYPE I". See discussion and Microsoft response here.
I'd recommend using the following WebRequestMethod instead, this works for me on all servers I tested, even ones which would not return a file size.
WebRequestMethods.Ftp.GetDateTimestamp
Because
request.Method = WebRequestMethods.Ftp.GetFileSize
may fails in some case (550: SIZE not allowed in ASCII mode), you can just check Timestamp instead.
reqFTP.Credentials = new NetworkCredential(inf.LogOn, inf.Password);
reqFTP.UseBinary = true;
reqFTP.Method = WebRequestMethods.Ftp.GetDateTimestamp;
FtpWebRequest (nor any other class in .NET) does not have any explicit method to check a file existence on FTP server. You need to abuse a request like GetFileSize or GetDateTimestamp.
string url = "ftp://ftp.example.com/remote/path/file.txt";
WebRequest request = WebRequest.Create(url);
request.Credentials = new NetworkCredential("username", "password");
request.Method = WebRequestMethods.Ftp.GetFileSize;
try
{
request.GetResponse();
Console.WriteLine("Exists");
}
catch (WebException e)
{
FtpWebResponse response = (FtpWebResponse)e.Response;
if (response.StatusCode == FtpStatusCode.ActionNotTakenFileUnavailable)
{
Console.WriteLine("Does not exist");
}
else
{
Console.WriteLine("Error: " + e.Message);
}
}
If you want a more straightforward code, use some 3rd party FTP library.
For example with WinSCP .NET assembly, you can use its Session.FileExists method:
SessionOptions sessionOptions = new SessionOptions {
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "username",
Password = "password",
};
Session session = new Session();
session.Open(sessionOptions);
if (session.FileExists("/remote/path/file.txt"))
{
Console.WriteLine("Exists");
}
else
{
Console.WriteLine("Does not exist");
}
(I'm the author of WinSCP)
You can use WebRequestMethods.Ftp.ListDirectory to check if a file exist, no need for nasty try catch mechanism.
private static bool ExistFile(string remoteAddress)
{
int pos = remoteAddress.LastIndexOf('/');
string dirPath = remoteAddress.Substring(0, pos); // skip the filename only get the directory
NetworkCredential credentials = new NetworkCredential(FtpUser, FtpPass);
FtpWebRequest listRequest = (FtpWebRequest)WebRequest.Create(dirPath);
listRequest.Method = WebRequestMethods.Ftp.ListDirectory;
listRequest.Credentials = credentials;
using (FtpWebResponse listResponse = (FtpWebResponse)listRequest.GetResponse())
using (Stream listStream = listResponse.GetResponseStream())
using (StreamReader listReader = new StreamReader(listStream))
{
string fileToTest = Path.GetFileName(remoteAddress);
while (!listReader.EndOfStream)
{
string fileName = listReader.ReadLine();
fileName = Path.GetFileName(fileName);
if (fileToTest == fileName)
{
return true;
}
}
}
return false;
}
static void Main(string[] args)
{
bool existFile = ExistFile("ftp://123.456.789.12/test/config.json");
}
I use FTPStatusCode.FileActionOK to check if file exists...
then, in the "else" section, return false.