I need a way to check if multiple files exist in a given FTP path. I will have a list, which lists all the file names to check, and I will need to check if all these exist on the server, and return errors for ones that don't. How easy is this?
Thanks
The safest approach would be to retrieve a list of files/directories per directory and parse that list.
// Get the object used to communicate with the server.
var request = WebRequest.Create(url);
request.Credentials = new NetworkCredential(username, password);
request.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
try
{
using(var response = request.GetResponse())
{
using(var stream = response.GetResponseStream())
{
using(var reader = new StreamReader(stream))
{
while(reader.Peek() >= 0)
{
var line = reader.ReadLine();
// check if this is a file or directory, filter list etc..
}
}
}
}
}
catch
{
}
Another- easier - option would be to try to retrieve the files DateTimestamp and catch the exception if the file doesn't exist. You should check the exception since one could be thrown for a different reason.
var request = WebRequest.Create(url);
request.Credentials = new NetworkCredential(username, password);
request.Method = WebRequestMethods.Ftp.GetDateTimestamp;
try
{
using(var response = (FtpWebResponse)request.GetResponse())
{
// file exists
}
}
catch(WebException e)
{
// file probably doesn't exits
}
Well, if you have access to the server, you could write your script there and then just request that script and thus only have to make one server request. Otherwise, you'll just need to check each file, one by one.
Related
I'm uploading a .txt file with c# and:
client.Credentials = new NetworkCredential(ftpU, ftpP);
client.UploadFile("here ftp server", "STOR", lfilepath);
And sometimes it just throws error like "System Error"
This .txt is just log in information, content of this txt is like User: Name At: 2015/12/12 08:43 AM
Is there any option to eliminate this error? Make ftp upload more effective? Or any idea to save log on informations in Internet.
Try to use FtpWebRequest and WebRequestMethods.Ftp.UploadFile. Here is the piece of code which we use for uplaoding files from ZipArchive to FTP (so there is also an option showing creating a directories if you need it too). From all methods which I've tested it was the most efficient one.
//// Get the object used to communicate with the server.
var request =
(FtpWebRequest)
WebRequest.Create("ftp://" + ftpServer + #"/" + remotePath + #"/" +
entry.FullName.TrimEnd('/'));
//// Determine if we are transferring file or directory
if (string.IsNullOrWhiteSpace(entry.Name) && !string.IsNullOrWhiteSpace(entry.FullName))
request.Method = WebRequestMethods.Ftp.MakeDirectory;
else
request.Method = WebRequestMethods.Ftp.UploadFile;
//// Try to transfer file
try
{
//// This example assumes the FTP site uses anonymous logon.
request.Credentials = new NetworkCredential(user, password);
switch (request.Method)
{
case WebRequestMethods.Ftp.MakeDirectory:
break;
case WebRequestMethods.Ftp.UploadFile:
var buffer = new byte[8192];
using (var rs = request.GetRequestStream())
{
StreamUtils.Copy(entry.Open(), rs, buffer);
}
break;
}
}
catch (Exception ex)
{
//// Handle it!
LogHelper.Error<FtpHelper>("Could not extract file from package.", ex);
}
finally
{
//// Get the response from the FTP server.
var response = (FtpWebResponse) request.GetResponse();
//// Close the connection = Happy a FTP server.
response.Close();
}
First, I did try to search for an answer for my problem but failed to find one.
I've been making a program in which my local text file and online text file, which is located in a FTP server, automatically updates each other. The text file contains list of names. This application was designed to be use by different machines simultaneously and can add and delete names in the list.
With the help of additional algorithm, I am able to merge the local and online text files without causing problems even when used offline and just update when there is an internet connection (using additional local files).
Now for the problem, everything works perfectly 90% of the time. However, sometimes the downloaded text file from the FTP server returns a blank text file even though it is not. I noticed that it is most likely to happen when my application is having a hard time checking for internet connection (Pinging google.com). This will result to a total erasure of the list since my application will interpret it as "The other user using the application deleted the list".
Here is the method I use to download from FTP server:
public Boolean DownloadFile(string uri, string path)
{
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(uri);
request.UsePassive = true;
request.KeepAlive = true;
request.UseBinary = true;
request.Proxy = null;
request.Timeout = 5000;
request.Method = WebRequestMethods.Ftp.DownloadFile;
request.Credentials = new NetworkCredential(this.username, this.password);
try
{
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader reader = new StreamReader(responseStream);
StringBuilder sb = new StringBuilder();
sb.AppendLine(reader.ReadToEnd());
File.WriteAllText(path, sb.ToString());
reader.Close();
response.Close();
return true;
}
catch (Exception e )
{
MessageBox.Show("FTPHANDLER DOWNLOAD FILE:"+e.ToString());
return false;
}
}
The method I used for checking internet connection:
private static bool CheckForInternetConnection()
{
try
{
Ping myPing = new Ping();
String host = "google.com";
byte[] buffer = new byte[32];
int timeout = 15000;
PingOptions pingOptions = new PingOptions();
PingReply reply = myPing.Send(host, timeout, buffer, pingOptions);
return (reply.Status == IPStatus.Success);
}
catch (Exception e)
{
//MessageBox.Show(e.ToString());
return false;
}
}
This is the way I call it. I've provided safety measures for that not to happen but it still occur.
public void Update()
{
if(CheckForInternetConnection()){
if (DownloadFile(uri,path))
{
char[] charsToTrim = { '\r', '\n' };
string onlineFile = File.ReadAllText(path).TrimEnd(charsToTrim);
if (onlineConfigFile.Equals("") || onlineConfigFile == null)
MessageBox.Show("Downloaded file is empty");
//still do stuff here regardless of the message
//If the other user really do intend to delete the list
}
}
}
The update method is executed in a different thread.
As for my first post here, sorry that it turned out to be very long. I did try to make it short but didn't want to miss any details.
Due to the nature of my problem (random), I think the "lock" solution did the fix for me since the problem did not occur after I use it for quite some time now. It was maybe because of the threads, which are spawned every 5secs, that caused the problem where they are processing different parts of the method simultaneously.
Here is the fixed code:
public void Update()
{
lock(lockObject)
{
if(CheckForInternetConnection())
{
if (DownloadFile(uri,path))
{
char[] charsToTrim = { '\r', '\n' };
string onlineFile = File.ReadAllText(path).TrimEnd(charsToTrim);
if (onlineConfigFile.Equals("") || onlineConfigFile == null)
MessageBox.Show("Downloaded file is empty");
//still do stuff here regardless of the message
//If the other user really do intend to delete the list
}
}
}
}
I declared "lockObject" as class level variable with a data type of "Object".
I wrote the following code to detect if a directory exists. The DirectoryExists method accepts either a fully qualified or relative path.
public bool DirectoryExists(string directory)
{
try
{
FtpWebRequest request = GetRequest(directory);
request.Method = WebRequestMethods.Ftp.ListDirectory;
using (FtpWebResponse response = request.GetResponse() as FtpWebResponse)
using (StreamReader sr = new StreamReader(response.GetResponseStream(), System.Text.Encoding.ASCII))
{
sr.ReadToEnd();
}
return true;
}
catch { }
return false;
}
protected FtpWebRequest GetRequest(string filename = "")
{
FtpWebRequest request = WebRequest.Create(_host.GetUrl(filename)) as FtpWebRequest;
request.Credentials = new NetworkCredential(Username, Password);
request.Proxy = null;
request.KeepAlive = false;
return request;
}
Note: _host.GetUrl(filename) returns the fully qualified path of the specified directory or filename. In my case, this is ftp://www.mydomain.com/Articles/controls/precisely-defining-kilobytes-megabytes-and-gigabytes.
This code has worked for many months. But all of a sudden it stopped working. Now, there is no exception raised in DirectoryExists when the directory does not exist. sr.ReadToEnd() simply returns an empty string.
I posted a similar question and it was suggested that I should always append / to the end of my path. I tried that, and thought I got an exception once, but it's not raising an exception now.
I don't know if the behavior changed on the FTP server I'm communicating with or what. Again, this worked fine when I wrote it and now it doesn't.
How can I determine whether or not an FTP directory exists?
EDIT:
Inspecting the response after calling ReadToEnd(), I see:
BannerMessage="220 Microsoft FTP Service\r\n"
ContentLength=-1
ExitMessage="221 Goodbye.\r\n"
StatusCode=ClosingData
StatusDescription="226 Transfer complete.\r\n"
WelcomeMessage="230 User logged in.\r\n"
UPDATE:
Ultimately, I see that most people have been recommending variations of what I was doing originally. This has lent weight to Hans Passant's suggestion that the issue lies with the server. I am attempting to get them to look at this but they seem a little baffled by the entire discussion. I know they are using a Microsoft server, and I'm skeptical I will be able to get a resolution from them.
If all else fails, I think the answer is to do a listing of the parent directory, which does require some extra work to handle cases such as when the directory in question is the root, and also cases when the parent doesn't exist.
FTP servers reply with a 550 message when you ask for a non existing directory/file. This 550 is translated to an exception in .NET.
Regarding the code you presented I don't see the reason to use a StreamReader in production. A simple modification is the following:
public bool DirectoryExists(string directory)
{
try
{
FtpWebRequest request = GetRequest(directory);
request.Method = WebRequestMethods.Ftp.ListDirectory;
return request.GetResponse() != null;
}
catch
{
return false;
}
}
I kept the exception handling as is (missing) but I suggest you to work on it as there are many and different cases to catch that are not related to ftp responses.
request.GetResponse() will generate an exception if the directory does not exist.
I tried it with the following paths for true returns:
ftp://ftp.mozilla.org/
ftp://ftp.mozilla.org/pub/
ftp://ftp.mozilla.org/pub/data/
ftp://ftp.mozilla.org/pub/data/bloat-reports/
The last one is an existing but empty directory
ftp://ftp.mozilla.org/pub/data/bloat-reports2/
returns false
There is a big but regarding trailing / in paths.
mozilla.org is a zero byte length file under pub directory
ftp://ftp.mozilla.org/pub/mozilla.org returns true
ftp://ftp.mozilla.org/pub/mozilla.org/ returned false and then true ?????
This behavior looks like with what you described. You can easily reproduce it on a browser. I suggest you to do the same with your own paths and check the contents of the directories. In case you do have empty files, remove them or replace their contents with a space, and finally check your code to ensure that you will not create new files or recreate them with zero byte length.
I hope that this helps.
UPDATE
No news, bad news! I assume that nothing of the above helped you. Maybe by going one level up in your path will solve the problem. The idea is to get a list of the parent directory and check it for the child name. If the parent path is valid it should always return non empty string. The code is the following:
static bool DirectoryExists(string directory)
{
try
{
directory = GetRequest(directory);
string
parent = directory.Substring(0, directory.TrimEnd('/').LastIndexOf('/') + 1),
child = directory.Substring(directory.TrimEnd('/').LastIndexOf('/') + 1).TrimEnd('/');
FtpWebRequest request = GetRequest(parent);
request.Method = WebRequestMethods.Ftp.ListDirectory;
using (FtpWebResponse response = request.GetResponse() as FtpWebResponse)
{
if (response == null)
return false;
string data = new StreamReader(response.GetResponseStream(), true).ReadToEnd();
return data.IndexOf(child, StringComparison.InvariantCultureIgnoreCase) >= 0;
}
}
catch
{
return false;
}
}
As you can see there is no need to worry about trailing slashes.
Warning: The code will not cover the case of a subdirectory and a filename with same names under the same path.
This might happen because FTP server software installed on your server.
Otherwise, you may want to take a look at response.StatusCode to see if there is any hints.
http://msdn.microsoft.com/en-us/library/system.net.ftpstatuscode(v=vs.110).aspx
Finally, if all of that doesn't work, try to write a dumb text file on that un-existed directory.
I had similar code that would deploy items via FTP to an IIS 6/Windows 2003 Server. This worked fine until I pointed it at an IIS7/Windows 2008 server. I started seeing the exact same behavior you did. I debugged it and ultimately changed my code to the following. This is a direct snippet from my code. I can include the source from the methods it calls if you like, but I think you understand those operations.
try
{
//Get a recursive FTP Listing
items = GetFtpListing(target, true);
}
catch(WebException e)
{
if (string.Equals(e.Message, "The remote server returned an error: (550) File unavailable (e.g., file not found, no access)."))
{
CreateRemoteDirectory(target);
items = GetFtpListing(target, true);
}
else
{
throw e;
}
}
relevant section of GetFtpListing
List<string> ftpRawList = new List<string>();
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(path);
request.Timeout = 60000;
request.ReadWriteTimeout = 60000;
request.Credentials = this.credentials;
request.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
using (FtpWebResponse response = (FtpWebResponse)request.GetResponse())
{
Stream data = response.GetResponseStream();
using (StreamReader reader = new StreamReader(data))
{
string responseString = reader.ReadToEnd();
ftpRawList.AddRange(responseString.Split(new char[2] { '\r', '\n' }, StringSplitOptions.RemoveEmptyEntries));
}
I use the below to check if a ftp connection credentials are valid. You may use the same to check if directory exists. Returns true if url, username, and password are correct.
URL: you specify your directory path in here.
user: ftp username
password: ftp password
public static bool isValidConnection(string url, string user, string password, ref string errorMessage = "")
{
try {
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(url);
request.Method = WebRequestMethods.Ftp.ListDirectory;
request.Credentials = new NetworkCredential(user, password);
request.KeepAlive = false;
request.UsePassive = true;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
response.Close();
} catch (WebException ex) {
errorMessage = ex.Message;
return false;
}
return true;
}
One way is to list the directory content as present in many solutions, but in my case i also have to read data content so the server checks the directory. If the directory is not present, an webException is throw that should be interpreted to filter from unexpected errors.
Here is my solution:
bool directoryExists(string path)
{
bool? exists = null;
FtpWebRequest request = (FtpWebRequest)WebRequest.Create(path);
request.Method = WebRequestMethods.Ftp.ListDirectory;
request.Credentials = new NetworkCredential("username", "*****");
FtpWebResponse response = null;
try
{
response = (FtpWebResponse)request.GetResponse();
using (StreamReader sr = new StreamReader(response.GetResponseStream()))
{
string str = sr.ReadLine(); //Just needs to read one line to fire check on server
sr.Close();
return true;
}
}
catch (WebException ex)
{
var r = (FtpWebResponse)ex.Response;
if (r.StatusCode ==
FtpStatusCode.ActionNotTakenFileUnavailable)
{
//Does not exist
return false;
}
throw; //if error is not related to directory existence then the exception needs to be treated above.
}
finally
{
if (response != null)
response.Close();
}
return false;
}
The FTP server i am targeting i know to be a windows server but i don't have access to the version information.
I'm trying to make a little application where I can select multiple files, set an interval and set the FTP server information. After that it uploads the files at the given interval.
I'm struggling at the FTP part. I could easily copy/paste some parts of codes from here, but I don't think that would help me. Basically I need to check if the file exists, if it does, delete it and upload the new version, if it doesn't exist just upload the file.
How could that be done in the most simple way? I'm not looking for tips on how to do it, i'm looking for the actually code preferable with comments on what does what.
EDIT: Based on the response from here, i think what i'm looking for is a way to override the file. Actually that was my first idea but i couldn't find anything about how to and then i just concluded that i manually needed to check and so on. I'm sorry for the mistake. Can someone give me a hint to what i need for overwriting?
Try something like this
var request = (FtpWebRequest)WebRequest.Create("ftp://ftp.myworld.com/file.txt");
request.Credentials = new NetworkCredential("username", "password");
request.Method = WebRequestMethods.Ftp.GetFileSize;
try
{
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
// To delete file
FtpWebRequest delRequest = (FtpWebRequest)WebRequest.Create(serverUri);
delRequest.Credentials = new NetworkCredential("username", "password");
delRequest.Method = WebRequestMethods.Ftp.DeleteFile;
FtpWebResponse response = (FtpWebResponse) delRequest.GetResponse();
}
catch(Exception e)
{
var response = (FtpWebResponse)ex.Response;
if (response.StatusCode == FtpStatusCode.ActionNotTakenFileUnavailable)
{
//not exist
}
}
In my application I use the WebClient class to download files from a Webserver by simply calling the DownloadFile method. Now I need to check whether a certain file exists prior to downloading it (or in case I just want to make sure that it exists). I've got two questions with that:
What is the best way to check whether a file exists on a server without transfering to much data across the wire? (It's quite a huge number of files I need to check)
Is there a way to get the size of a given remote file without downloading it?
Thanks in advance!
WebClient is fairly limited; if you switch to using WebRequest, then you gain the ability to send an HTTP HEAD request. When you issue the request, you should either get an error (if the file is missing), or a WebResponse with a valid ContentLength property.
Edit: Example code:
WebRequest request = WebRequest.Create(new Uri("http://www.example.com/"));
request.Method = "HEAD";
using(WebResponse response = request.GetResponse()) {
Console.WriteLine("{0} {1}", response.ContentLength, response.ContentType);
}
When you request file using the WebClient Class, the 404 Error (File Not Found) will lead to an exception. Best way is to handle that exception and use a flag which can be set to see if the file exists or not.
The example code goes as follows:
System.Net.HttpWebRequest request = null;
System.Net.HttpWebResponse response = null;
request = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create("www.example.com/somepath");
request.Timeout = 30000;
try
{
response = (System.Net.HttpWebResponse)request.GetResponse();
flag = 1;
}
catch
{
flag = -1;
}
if (flag==1)
{
Console.WriteLine("File Found!!!");
}
else
{
Console.WriteLine("File Not Found!!!");
}
You can put your code in respective if blocks.
Hope it helps!
What is the best way to check whether a file exists on a server
without transfering to much data across the wire?
You can test with WebClient.OpenRead to open the file stream without reading all the file bytes:
using (var client = new WebClient())
{
Stream stream = client.OpenRead(url);
// ^ throws System.Net.WebException: 'Could not find file...' if file is not present
stream.Close();
}
This will indicate if the file exists at the remote location or not.
To fully read the file stream, you would do:
using (var client = new WebClient())
{
Stream stream = client.OpenRead(url);
StreamReader sr = new StreamReader(stream);
Console.WriteLine(sr.ReadToEnd());
stream.Close();
}
In case anyone stuck with ssl certificate issue
ServicePointManager.ServerCertificateValidationCallback = new RemoteCertificateValidationCallback
(
delegate { return true; }
);
WebRequest request = WebRequest.Create(new Uri("http://.com/flower.zip"));
request.Method = "HEAD";
using (WebResponse response = request.GetResponse())
{
Console.WriteLine("{0} {1}", response.ContentLength, response.ContentType);
}