We Keep Coding

Get the list of files from FTP server with specific search pattern

I want to get the list of files from FTP server with specific search pattern ( Ex: get all the files having pattern as "*.txt") and download these files only using C#.net.
Below is the code returning list of files from FTP server, please suggest the addition code required to achieve the required task.
FtpWebRequest reqFTP;
reqFTP = (FtpWebRequest)FtpWebRequest.Create(new Uri("ftp://" + coldata.Host + "/"));
//("ftp://" + coldata.host + "/"));
reqFTP.UseBinary = true;
reqFTP.Credentials = new NetworkCredential(coldata.Uid, coldata.Pwd);
reqFTP.Method = WebRequestMethods.Ftp.ListDirectory;
reqFTP.Timeout = System.Threading.Timeout.Infinite;
reqFTP.Proxy = null;
reqFTP.KeepAlive = true;
reqFTP.UsePassive = true;
FtpWebResponse res = (FtpWebResponse)reqFTP.GetResponse();
response = reqFTP.GetResponse();
reader = new StreamReader(response.GetResponseStream());
string line = reader.ReadLine();
while (line != null)
{
result.Append(line);
result.Append("\n");
line = reader.ReadLine();
}
// to remove the trailing '\n'
result.Remove(result.ToString().LastIndexOf('\n'), 1);
downloadRes = true;
return result.ToString().Split('\n');
Thanks.

You could use System.IO.Path.GetExtension, something like this maybe:
while (line != null)
{
if (System.IO.Path.GetExtension(line) == "txt")
{
result.Append(line);
result.Append("\n");
line = reader.ReadLine();
}
}
Not exactly what you asked for, but you cannot specify search patterns for FTP, see here:
how to fetch a range of files from an FTP server using C#

Listing server folders and sub folders using C# FtpWebRequest

I am trying to get full list of files, directories and sub-directories on tree view using StreamReader. The problem is it took too long and throws *"Operation time out exception" and shows only one level.
Here is my code
public void getServerSubfolder(TreeNode tv, string parentNode) {
string ptNode;
List<string> files = new List<string> ();
try {
FtpWebRequest request = (FtpWebRequest) WebRequest.
Create(parentNode);
request.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
request.Credentials = new NetworkCredential(this.userName, this.Password);
request.UseBinary = true;
request.UsePassive = true;
request.Timeout = 10000;
request.ReadWriteTimeout = 10000;
request.KeepAlive = false;
FtpWebResponse response = (FtpWebResponse) request.GetResponse();
Stream responseStream = response.GetResponseStream();
StreamReader reader = new StreamReader(responseStream);
string fileList;
string[] fileName;
//MessageBox.Show(reader.ReadToEnd().ToString());
while (!reader.EndOfStream) {
fileList = reader.ReadLine();
fileName = fileList.Split(' ');
if (fileName[0] == "drwxr-xr-x") {
// if it is directory
TreeNode tnS = new TreeNode(fileName[fileName.Length - 1]);
tv.Nodes.Add(tnS);
ptNode = parentNode + "/" + fileName[fileName.Length - 1] + "/";
getServerSubfolder(tnS, ptNode);
} else files.Add(fileName[fileName.Length - 1]);
}
reader.Close();
response.Close();
} catch (Exception ex) {
MessageBox.Show("Sub--here " + ex.Message + "----" + ex.StackTrace);
}
}

You have to read (and cache) whole listing before recursing into subdirectories, otherwise the top-level request will timeout before you complete subdirectories listing.
You can keep using ReadLine, no need to use ReadToEnd and separate the lines yourself.
void ListFtpDirectory(
string url, string rootPath, NetworkCredential credentials, List<string> list)
{
FtpWebRequest listRequest = (FtpWebRequest)WebRequest.Create(url + rootPath);
listRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
listRequest.Credentials = credentials;
List<string> lines = new List<string>();
using (FtpWebResponse listResponse = (FtpWebResponse)listRequest.GetResponse())
using (Stream listStream = listResponse.GetResponseStream())
using (StreamReader listReader = new StreamReader(listStream))
{
while (!listReader.EndOfStream)
{
lines.Add(listReader.ReadLine());
}
}
foreach (string line in lines)
{
string[] tokens =
line.Split(new[] { ' ' }, 9, StringSplitOptions.RemoveEmptyEntries);
string name = tokens[8];
string permissions = tokens[0];
string filePath = rootPath + name;
if (permissions[0] == 'd')
{
ListFtpDirectory(url, filePath + "/", credentials, list);
}
else
{
list.Add(filePath);
}
}
}
Use the function like:
List<string> list = new List<string>();
NetworkCredential credentials = new NetworkCredential("user", "mypassword");
string url = "ftp://ftp.example.com/";
ListFtpDirectory(url, "", credentials, list);
A disadvantage of the above approach is that it has to parse server-specific listing to retrieve information about files and folders. The code above expects a common *nix-style listing. But many servers use a different format.
The FtpWebRequest unfortunately does not support the MLSD command, which is the only portable way to retrieve directory listing with file attributes in FTP protocol.
If you want to avoid troubles with parsing the server-specific directory listing formats, use a 3rd party library that supports the MLSD command and/or parsing various LIST listing formats; and recursive downloads.
For example with WinSCP .NET assembly you can list whole directory with a single call to the Session.EnumerateRemoteFiles:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "user",
Password = "mypassword",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
// List files
IEnumerable<string> list =
session.EnumerateRemoteFiles("/", null, EnumerationOptions.AllDirectories).
Select(fileInfo => fileInfo.FullName);
}
Internally, WinSCP uses the MLSD command, if supported by the server. If not, it uses the LIST command and supports dozens of different listing formats.
(I'm the author of WinSCP)

I'm doing similar things, but instead of using StreamReader.ReadLine() for each one, I get it all at once with StreamReader.ReadToEnd(). You don't need ReadLine() to just get a list of directories. Below is my entire code (Entire howto is explained in this tutorial):
FtpWebRequest request=(FtpWebRequest)FtpWebRequest.Create(path);
request.Method=WebRequestMethods.Ftp.ListDirectoryDetails;
List<ftpinfo> files=new List<ftpinfo>();
//request.Proxy = System.Net.WebProxy.GetDefaultProxy();
//request.Proxy.Credentials = CredentialCache.DefaultNetworkCredentials;
request.Credentials = new NetworkCredential(_username, _password);
Stream rs=(Stream)request.GetResponse().GetResponseStream();
OnStatusChange("CONNECTED: " + path, 0, 0);
StreamReader sr = new StreamReader(rs);
string strList = sr.ReadToEnd();
string[] lines=null;
if (strList.Contains("\r\n"))
{
lines=strList.Split(new string[] {"\r\n"},StringSplitOptions.None);
}
else if (strList.Contains("\n"))
{
lines=strList.Split(new string[] {"\n"},StringSplitOptions.None);
}
//now decode this string array
if (lines==null || lines.Length == 0)
return null;
foreach(string line in lines)
{
if (line.Length==0)
continue;
//parse line
Match m= GetMatchingRegex(line);
if (m==null) {
//failed
throw new ApplicationException("Unable to parse line: " + line);
}
ftpinfo item=new ftpinfo();
item.filename = m.Groups["name"].Value.Trim('\r');
item.path = path;
item.size = Convert.ToInt64(m.Groups["size"].Value);
item.permission = m.Groups["permission"].Value;
string _dir = m.Groups["dir"].Value;
if(_dir.Length>0 && _dir != "-")
{
item.fileType = directoryEntryTypes.directory;
}
else
{
item.fileType = directoryEntryTypes.file;
}
try
{
item.fileDateTime = DateTime.Parse(m.Groups["timestamp"].Value);
}
catch
{
item.fileDateTime = DateTime.MinValue; //null;
}
files.Add(item);
}
return files;

How to read files one by one from ftp server using c#?

I am getting files from ftp folder like this.
request = (FtpWebRequest)FtpWebRequest.Create(new Uri("ftp://ipaddress/foldername"));
request.UseBinary = true;
request.Credentials = new NetworkCredential("username", "password");
request.Method = WebRequestMethods.Ftp.ListDirectory;
request.UseBinary = true;
WebResponse response = request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string line = reader.ReadLine();
while (line != null)
{
result.Append(line);
result.Append("\n");
line = reader.ReadLine();
}
// to remove the trailing '\n'
result.Remove(result.ToString().LastIndexOf('\n'), 1);
reader.Close();
response.Close();
string[] results = result.ToString().Split('\n');
foreach (string filename in results)
{
connetionString = "Data Source=testservername;Initial Catalog=testdb;User ID=sa;Password=sa";
connection = new SqlConnection(connetionString);
FtpWebRequest request1 = (FtpWebRequest)FtpWebRequest.Create(new Uri("ftp://servername/" + filename));
request1.Method = WebRequestMethods.Ftp.DownloadFile;
request1.UseBinary = true;
request1.Credentials = new NetworkCredential("username", "password");
responses = (FtpWebResponse)request1.GetResponse();
StreamReader reader1 = new StreamReader(responses.GetResponseStream());
string xml = reader1.ReadToEnd();
ds.ReadXml(new XmlTextReader(new StringReader(xml)));
int i = 0;
connection.Open();
for (i = 0; i <= ds.Tables[1].Rows.Count - 1 ; i++)
{
sql = "insert into Details values('" + ds.Tables[1].Rows[i].ItemArray[0]
+ "')";
command = new SqlCommand(sql, connection);
adpter.InsertCommand = command;
adpter.InsertCommand.ExecuteNonQuery();
}
FtpWebRequest requestDelete = (FtpWebRequest)WebRequest.Create("ftp://sername/" + filename);
requestDelete.Method = WebRequestMethods.Ftp.DeleteFile;
requestDelete.UseBinary = true;
requestDelete.Credentials = new NetworkCredential("username", "password");
responseDelete = (FtpWebResponse)requestDelete.GetResponse();
try
{
string res = responseDelete.StatusDescription;
if (res == "250 Delete operation successful.")
{
MessageBox.Show("Done......");
}
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
}
}
responseDelete.Close();
connection.Close();
responses.Close();
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
}
}
Hi need to get files from ftp folder and read that xml files one by one and save it as records in my Databse after that need to delete that file ftp folder.
if for testing i placed two files in ftp folder first files reading successfully .but getting error for reading second file like "Object Reference is not set instance of object "in the below line.
ds.ReadXml(new XmlTextReader(new StringReader(xml)));
please tell me any thing wrong in my code.i am sending ftp request multiple times it is good or suggest me by seeing above lines of code how to do that?

Downloading multiple files using FTP in C#

I need to write a routines that downloads a given directory on the server completely - all the files and directories in it.
Right now I have a routine that lists dir content
public List<string> GetListOfFiles(string serverPath)
{
List<string> files = new List<string>();
try
{
FtpWebRequest request = (FtpWebRequest)WebRequest.Create("ftp://" + serverPath);
request.Credentials = new NetworkCredential(_domain + "\\" + _username, _password);
request.Method = WebRequestMethods.Ftp.ListDirectory;
FtpWebResponse response = (FtpWebResponse)request.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string line = reader.ReadLine();
while (line != null)
{
files.Add(line);
line = reader.ReadLine();
}
response.Close();
}
catch (WebException ex)
{
FtpWebResponse response = (FtpWebResponse)ex.Response;
string exMsg = string.Empty;
switch (response.StatusCode)
{
case FtpStatusCode.NotLoggedIn:
exMsg = "wrong username/password";
break;
default:
exMsg = "The server is inaccessible or taking too long to respond.";
break;
}
throw new Exception(exMsg);
}
return files;
}
The issue is that I get the list of files and directories...so something like
file1.dll
file2.dll
dir1Name
Is there a way to make a distinction between a file name and a directory name when listing it? Like a flag?

Unfortunately, the returned information is really a function of your FTP server, not the framework.
You can ListDirectoryDetails instead of ListDirectory, which should provide you much more detailed information (including whether each file is a directory or a file), but would require special parsing, as its format, too, is dependent on the FTP Server.

Searching all files for regex matching files on FTP server in C#.NET

I'm creating a program where I need to search an FTP server and download all files which match a given regex. How do I do this? I can connect to the FTP server, but how do I scan all files in the given path for files matching the regex?
I also need to do the same for HTTP servers which I think will be fundamentally more difficult, but I'll stick to doing the FTP server for now.
Thanks

You can use this to get a list
public string[] GetFileList()
{
string[] downloadFiles;
StringBuilder result = new StringBuilder();
FtpWebRequest reqFTP;
try
{
reqFTP = (FtpWebRequest)FtpWebRequest.Create(new Uri("ftp://" + ftpServerIP + "/"));
reqFTP.UseBinary = true;
reqFTP.Credentials = new NetworkCredential(ftpUserID, ftpPassword);
reqFTP.Method = WebRequestMethods.Ftp.ListDirectory;
WebResponse response = reqFTP.GetResponse();
StreamReader reader = new StreamReader(response.GetResponseStream());
string line = reader.ReadLine();
while (line != null)
{
result.Append(line);
result.Append("\n");
line = reader.ReadLine();
}
// to remove the trailing '\n'
result.Remove(result.ToString().LastIndexOf('\n'), 1);
reader.Close();
response.Close();
return result.ToString().Split('\n');
}
catch (Exception ex)
{
System.Windows.Forms.MessageBox.Show(ex.Message);
downloadFiles = null;
return downloadFiles;
}
}
and then maipulate GetFileList Array using regex as per your need

FTP has a list (ls) command - any library you use should have a corresponding method, this will return a listing of files in the current directory.
You can match against this list and only retrieve the files that match.
Without knowing the exact library you are using, it is difficult to get more specific.