We Keep Coding

Compare local and remote files using WinSCP .NET assembly

I'm trying to implement some logic to compare file information between remote server and local server.
I need to compare the file name between local folder and remote folder and download only the new files.
I tried using loading files in a list and use Except function, it didn't work.
Appreciate your help.
Please find one of the scenario I tried.
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
const string remotePath = "/Test";
const string localPath = #"C:\Local";
const string ArchivePath = #"C:\Users\Local\Archive";
System.IO.DirectoryInfo dir2 = new System.IO.DirectoryInfo(ArchivePath);
RemoteDirectoryInfo dir1 = session.ListDirectory(remotePath);
IEnumerable<System.IO.FileInfo> list2 =
dir2.GetFiles("*.*", System.IO.SearchOption.AllDirectories);
IEnumerable<RemoteFileInfo> list1 =
session.EnumerateRemoteFiles(remotePath, "*.csv", EnumerationOptions.None);
var firstNotSecond = list1.Except(list2).ToList();
}
Getting error like
'IEnumerable' does not contain a definition for 'Except' and the best extension method overload 'Queryable.Except(IQueryable, IEnumerable)' requires a receiver of type 'IQueryable'

You will have to compare just the filenames:
var firstNotSecond =
list1.Select(_ => _.Name).Except(list2.Select(_ => _.Name)).ToList();
Though note that WinSCP .NET has this functionality built-in. There's Session.CompareDirectories.
And if you actually want to synchronize the directories, there's Session.SynchronizeDirectories. One method that will do everything for you.
session.SynchronizeDirectories(
SynchronizationMode.Local, localPath, remotePath, false).Check()

Monitor FTP directory in ASP.NET/C#

I have FileSystem watcher for a local directory. It's working fine. I want same to implement for FTP. Is there any way I can achieve it? I have checked many solutions but it's not clear.
Logic: Want to get files from FTP later than some timestamp.
Problem faced: Getting all files from FTP and then filtering the result is hitting the performance (used FtpWebRequest).
Is there any right way to do this? (WinSCP is on hold. Cant use it now.)
FileSystemWatcher oFsWatcher = new FileSystemWatcher();
OFSWatchers.Add(oFsWatcher);
oFsWatcher.Path = sFilePath;
oFsWatcher.Filter = string.IsNullOrWhiteSpace(sFileFilter) ? "*.*" : sFileFilter;
oFsWatcher.NotifyFilter = NotifyFilters.FileName;
oFsWatcher.EnableRaisingEvents = true;
oFsWatcher.IncludeSubdirectories = bIncludeSubdirectories;
oFsWatcher.Created += new FileSystemEventHandler(OFsWatcher_Created);

You cannot use the FileSystemWatcher or any other way, because the FTP protocol does not have any API to notify a client about changes in the remote directory.
All you can do is to periodically iterate the remote tree and find changes.
It's actually rather easy to implement, if you use an FTP client library that supports recursive listing of a remote tree. Unfortunately, the built-in .NET FTP client, the FtpWebRequest does not. But for example with WinSCP .NET assembly, you can use the Session.EnumerateRemoteFiles method.
See the article Watching for changes in SFTP/FTP server:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "example.com",
UserName = "user",
Password = "password",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
List<string> prevFiles = null;
while (true)
{
// Collect file list
List<string> files =
session.EnumerateRemoteFiles(
"/remote/path", "*.*", EnumerationOptions.AllDirectories)
.Select(fileInfo => fileInfo.FullName)
.ToList();
if (prevFiles == null)
{
// In the first round, just print number of files found
Console.WriteLine("Found {0} files", files.Count);
}
else
{
// Then look for differences against the previous list
IEnumerable<string> added = files.Except(prevFiles);
if (added.Any())
{
Console.WriteLine("Added files:");
foreach (string path in added)
{
Console.WriteLine(path);
}
}
IEnumerable<string> removed = prevFiles.Except(files);
if (removed.Any())
{
Console.WriteLine("Removed files:");
foreach (string path in removed)
{
Console.WriteLine(path);
}
}
}
prevFiles = files;
Console.WriteLine("Sleeping 10s...");
Thread.Sleep(10000);
}
}
(I'm the author of WinSCP)
Though, if you actually want to just download the changes, it's a way easier. Just use the Session.SynchronizeDirectories in the loop.
while (true)
{
SynchronizationResult result =
session.SynchronizeDirectories(
SynchronizationMode.Local, "/remote/path", #"C:\local\path", true);
result.Check();
// You can inspect result.Downloads for a list for updated files
Console.WriteLine("Sleeping 10s...");
Thread.Sleep(10000);
}
This will update even modified files, not only new files.
Though using WinSCP .NET assembly from a web application might be problematic. If you do not want to use a 3rd party library, you have to do with limitations of the FtpWebRequest. For an example how to recursively list a remote directory tree with the FtpWebRequest, see my answer to List names of files in FTP directory and its subdirectories.
You have edited your question to say that you have performance problems with the solutions I've suggested. Though you have already asked a new question that covers this:
Get FTP file details based on datetime in C#

Unless you have access to the OS which hosts the service; it will be a bit harder.
FileSystemWatcher places a hook on the filesystem, which will notify your application as soon as something happened.
FTP command specifications does not have such a hook. Besides that it's always initiated by the client.
Therefor, to implement such logic you should periodical perform a NLST to list the FTP-directory contents and track the changes (or hashes, perhaps (MDTM)) yourself.
More info:
FTP return codes
FTP

I have got an alternative solution to do my functionality.
Explanation:
I am downloading the files from FTP (Read permission reqd.) with same folder structure.
So everytime the job/service runs I can check into the physical path same file(Full Path) exists or not If not exists then it can be consider as a new file. And Ii can do some action for the same and download as well.
Its just an alternative solution.
Code Changes:
private static void GetFiles()
{
using (FtpClient conn = new FtpClient())
{
string ftpPath = "ftp://myftp/";
string downloadFileName = #"C:\temp\FTPTest\";
downloadFileName += "\\";
conn.Host = ftpPath;
//conn.Credentials = new NetworkCredential("ftptest", "ftptest");
conn.Connect();
//Get all directories
foreach (FtpListItem item in conn.GetListing(conn.GetWorkingDirectory(),
FtpListOption.Modify | FtpListOption.Recursive))
{
// if this is a file
if (item.Type == FtpFileSystemObjectType.File)
{
string localFilePath = downloadFileName + item.FullName;
//Only newly created files will be downloaded.
if (!File.Exists(localFilePath))
{
conn.DownloadFile(localFilePath, item.FullName);
//Do any action here.
Console.WriteLine(item.FullName);
}
}
}
}
}

List names of files in FTP directory and its subdirectories

I have been searched in the net and I didn't found any result. Actually I want to get the name of all the files that I have in the root and Directory and Sub Directory. I tried the code as bellow but its give me only the files in the root of my FTP.
The folder that I have in the FTP is like as bellow:
/ds/product/Jan/
/ds/subproduct/Jan/
/ds/category/Jan/
The code that I tried:
FtpWebRequest ftpRequest = (FtpWebRequest)WebRequest.Create("ftp://" + FtpIP);
ftpRequest.Credentials = new NetworkCredential(FtpUser, FtpPass);
ftpRequest.Method = WebRequestMethods.Ftp.ListDirectory;
FtpWebResponse response = (FtpWebResponse)ftpRequest.GetResponse();
StreamReader streamReader = new StreamReader(response.GetResponseStream());
List<string> directories = new List<string>();
string line = streamReader.ReadLine();
while (!string.IsNullOrEmpty(line))
{
// directories.Add(line);
line = streamReader.ReadLine().ToString();
MessageBox.Show(line);
}
streamReader.Close();

It is s not easy to implement this without any external library. Unfortunately, neither the .NET Framework nor PowerShell have any explicit support for recursively listing files in an FTP directory.
You have to implement that yourself:
List the remote directory
Iterate the entries, recursing into subdirectories - listing them again, etc.
Tricky part is to identify files from subdirectories. There's no way to do that in a portable way with the .NET Framework (FtpWebRequest). The .NET Framework unfortunately does not support the MLSD command, which is the only portable way to retrieve directory listing with file attributes in FTP protocol. See also Checking if object on FTP server is file or directory.
Your options are:
Do an operation on a file name that is certain to fail for file and succeeds for directories (or vice versa). I.e. you can try to download the "name".
You may be lucky and in your specific case, you can tell a file from a directory by a file name (i.e. all your files have an extension, while subdirectories do not)
You use a long directory listing (LIST command = ListDirectoryDetails method) and try to parse a server-specific listing. Many FTP servers use *nix-style listing, where you identify a directory by the d at the very beginning of the entry. But many servers use a different format. The following example uses this approach (assuming the *nix format)
static void ListFtpDirectory(string url, NetworkCredential credentials)
{
WebRequest listRequest = WebRequest.Create(url);
listRequest.Method = WebRequestMethods.Ftp.ListDirectoryDetails;
listRequest.Credentials = credentials;
List<string> lines = new List<string>();
using (WebResponse listResponse = listRequest.GetResponse())
using (Stream listStream = listResponse.GetResponseStream())
using (StreamReader listReader = new StreamReader(listStream))
{
while (!listReader.EndOfStream)
{
string line = listReader.ReadLine();
lines.Add(line);
}
}
foreach (string line in lines)
{
string[] tokens =
line.Split(new[] { ' ' }, 9, StringSplitOptions.RemoveEmptyEntries);
string name = tokens[8];
string permissions = tokens[0];
if (permissions[0] == 'd')
{
Console.WriteLine($"Directory {name}");
string fileUrl = url + name;
ListFtpDirectory(fileUrl + "/", credentials);
}
else
{
Console.WriteLine($"File {name}");
}
}
}
Use the function like:
NetworkCredential credentials = new NetworkCredential("user", "mypassword");
string url = "ftp://ftp.example.com/directory/to/list/";
ListFtpDirectory(url, credentials);
If you want to avoid troubles with parsing the server-specific directory listing formats, use a 3rd party library that supports the MLSD command and/or parsing various LIST listing formats.
For example with WinSCP .NET assembly you can list whole directory recursively with a single call to Session.EnumerateRemoteFiles:
// Setup session options
var sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "ftp.example.com",
UserName = "user",
Password = "mypassword",
};
using (var session = new Session())
{
// Connect
session.Open(sessionOptions);
// Enumerate files
var options =
EnumerationOptions.EnumerateDirectories |
EnumerationOptions.AllDirectories;
IEnumerable<RemoteFileInfo> fileInfos =
session.EnumerateRemoteFiles("/directory/to/list", null, options);
foreach (var fileInfo in fileInfos)
{
Console.WriteLine(fileInfo.FullName);
}
}
Not only the code is simpler, more robust and platform-independent. It also makes all other file attributes (size, modification time, permissions, ownership) readily available via the RemoteFileInfo class.
Internally, WinSCP uses the MLSD command, if supported by the server. If not, it uses the LIST command and supports dozens of different listing formats.
(I'm the author of WinSCP)

FileSystemWatcher.Changed on FTP

I want to use FileSystemWatcher.Changed on a FTP directory
What do I put in the property FileSystemWatcher.Path ?

You can not do this. A FileSystemWatcher watches the filesystem, not an FTP-folder. So unless you have filesystem access to the FTP-path (UNC paths are supported), you are unable to do this.

You cannot use the FileSystemWatcher or any other way, because the FTP protocol does not have any API to notify a client about changes in the remote directory.
All you can do its to periodically iterate the remote tree and find changes.
It's actually rather easy to implement, if you use an FTP client that supports recursive listing of a remote tree. Unfortunately, the build-in .NET FTP client, the FtpWebRequest does not. For example with WinSCP .NET assembly, you can use the Session.EnumerateRemoteFiles method.
See the article Watching for changes in SFTP/FTP server:
// Setup session options
SessionOptions sessionOptions = new SessionOptions
{
Protocol = Protocol.Ftp,
HostName = "example.com",
UserName = "user",
Password = "password",
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
List<string> prevFiles = null;
while (true)
{
// Collect file list
List<string> files =
session.EnumerateRemoteFiles(
"/remote/path", "*.*", EnumerationOptions.AllDirectories)
.Select(fileInfo => fileInfo.FullName)
.ToList();
if (prevFiles == null)
{
// In the first round, just print number of files found
Console.WriteLine("Found {0} files", files.Count);
}
else
{
// Then look for differences against the previous list
IEnumerable<string> added = files.Except(prevFiles);
if (added.Any())
{
Console.WriteLine("Added files:");
foreach (string path in added)
{
Console.WriteLine(path);
}
}
IEnumerable<string> removed = prevFiles.Except(files);
if (removed.Any())
{
Console.WriteLine("Removed files:");
foreach (string path in removed)
{
Console.WriteLine(path);
}
}
}
prevFiles = files;
Console.WriteLine("Sleeping 10s...");
Thread.Sleep(10000);
}
}
(I'm the author of WinSCP)
Though, if you actually want to just download the changes, it's a way easier. Just use the Session.SynchronizeDirectories in the loop.
session.SynchronizeDirectories(
SynchronizationMode.Local, "/remote/path", #"C:\local\path", true).Check();
If you do not want to use a 3rd party library, you have to do with limitations of the FtpWebRequest. For an example how to recursively list a remote directory tree with the FtpWebRequest, see my answer to C# Download all files and subdirectories through FTP.

How to upload a file in share point server from a local machine connected through lan using c#

I am really new to this share point stuffs.We have a share point server with admin account in it and i was connecting it from my local machine through ip and share point port manually.
Nut i need to write a program which needs to upload the files into the share point server from the local machine to server. Is it possible in using winforms ? or only possible in web services.?
using (SPSite oSite = new SPSite(sharePointSite))
{
using (SPWeb oWeb = oSite.OpenWeb())
{
if (!System.IO.File.Exists(fileToUpload))
throw new FileNotFoundException("File not found.", fileToUpload);
SPFolder myLibrary = oWeb.Folders[documentLibraryName];
// Prepare to upload
Boolean replaceExistingFiles = true;
String fileName = System.IO.Path.GetFileName(fileToUpload);
FileStream fileStream = File.OpenRead(fileToUpload);
// Upload document
SPFile spfile = myLibrary.Files.Add(fileName, fileStream, replaceExistingFiles);
// Commit
myLibrary.Update();
}
}
tried using the above code and i getting error from the following line
using (SPSite oSite = new SPSite(sharePointSite))
and the error was
"The Web application at http://server:port/ could not be found. Verify that you have typed the URL correctly. If the URL should be serving existing content, the system administrator may need to add a new request URL mapping to the intended application"
and am not able to upload the file.
But if i copied and paste the same URL in my local machine i can able to access the sharepoint deployed in server and i can even upload files manually from my local machine.
How to upload a file in sharepoint server from the local machine connected with LAN..??

siteURL = Main URl of the sharepoint (eg) "http://10.0.0.14:48487/";
documentListName = any of the folders in shrepoint (eg) Shared Documents
documentName = name of the file (eg)sampleword.docx , readme.txt etc
documentStream = the byte format of the file which we going to upload.
(eg)byte[] bytefile = System.IO.File.ReadAllBytes(filepath+filename);
public static void UploadDocument(string siteURL, string documentListName, string documentListURL,string documentName, byte[] documentStream = null)
{
try
{
using (SP.ClientContext clientContext = new SP.ClientContext(siteURL))
{
#region"Only if you have credentials"
NetworkCredential Cred = new NetworkCredential("username", "password");
clientContext.Credentials = Cred;
#endregion
SP.List documentsList = clientContext.Web.Lists.GetByTitle(documentListName);
var fileCreationInformation = new SP.FileCreationInformation();
//Assign to content byte[] i.e. documentStream
fileCreationInformation.Content = documentStream;
//Allow owerwrite of document
fileCreationInformation.Overwrite = true;
//Upload URL
fileCreationInformation.Url = documentName;
Microsoft.SharePoint.Client.File uploadFile = documentsList.RootFolder.Files.Add(
fileCreationInformation);
uploadFile.ListItemAllFields.Update();
clientContext.ExecuteQuery();
}
}
catch (Exception ex)
{
}
}
}
This works perfect for me :) :)

make sure your connecting to an existing site collection. the error you have received is quite self explainitory, it cannot find the site collection you have pointed it at. check your string sharePointSite to make sure there isn't a typo and that it's accessing the correct root site collection. Remember with SharePoint SPSite = Site Collection SPWeb = a website in a site collection.
i can't see any glaring errors in your code without running something up myself except to make sure that when you are calling oSite.openweb() it's for a single site within the site collection.