For each run, application (windows forms) extracts output to file (excel/.csv/.pdf), creates current date as folder name and stores it locally. Have to capture those output directly to S3 bucket, successfully uploaded each file as below.
AmazonUploader class
using System;
using System.Collections.Generic;
using System.Text;
using Amazon;
using Amazon.S3;
using Amazon.S3.Transfer;
namespace AwsS3DemoConsoleApp
{
public class AmazonUploader
{
public bool sendMyFileToS3(string localFilePath, string bucketName, string subDirectoryInBucket, string fileNameInS3)
{
SecretConsumer secretConsumer = new SecretConsumer();
IAmazonS3 client = new AmazonS3Client(secretConsumer.AWSAccessKey, secretConsumer.AWSSecretKey, RegionEndpoint.APSouth1);
TransferUtility utility = new TransferUtility(client);
TransferUtilityUploadRequest request = new TransferUtilityUploadRequest();
if (subDirectoryInBucket == "" || subDirectoryInBucket == null)
{
request.BucketName = bucketName; //no subdirectory just bucket name
}
else
{ // subdirectory and bucket name
request.BucketName = bucketName + #"/" + subDirectoryInBucket;
}
request.Key = fileNameInS3;
request.FilePath = localFilePath;
utility.Upload(request);
return true;
}
}
}
Program class
using System;
namespace AwsS3DemoConsoleApp
{
internal class Program
{
static void Main(string[] args)
{
string fileToBackup = #"C:\Users\marvel\Documents\Spiderman\19-05-2022"; // test file path from the local computer
string myBucketName = "cccfilestorage"; // your s3 bucket name goes here
string s3DirectoryName = "dummy_folder"; // the directory path to a sub folder goes here
string s3FileName = "19-05-2022"; // the name of the file when its saved into the S3 buscket
AmazonUploader myUploader = new AmazonUploader();
myUploader.sendMyFileToS3(fileToBackup, myBucketName, s3DirectoryName, s3FileName);
Console.WriteLine("File uploaded to S3");
}
}
}
I want to upload as a complete directory for each run.
current folder structure
19-05-2022(main directory)
sub directory (1)
sub directory (2)
csv
pdf
excel
I'm sure that we can upload folders using transfer utility
Can anyone please guide me how to achieve it?
Related
In my Web.Config file I have add my IP address in file path. when i try to access all files of that folder then it gives an error uri format not supported. but if i give local file path then it works fine.
string pathdata = Utility.GetConfigValue("DevSubmittedStateTaxForms");
string uploadPath = Utility.GetConfigValue("DevUploadFiles");
DirectoryInfo d = new DirectoryInfo(pathdata);
FileInfo[] Files = d.GetFiles("*.pdf");
var status = new List<Object>();
int i = 1;
foreach (FileInfo filename in Files)
{
status.Add(new {ID = i, Name = filename.Name, URL = pathdata + filename.Name + ".pdf" });
i++;
}
using System;
using System.IO;
namespace Test
{
public class Program
{
static void Main(string[] args)
{
string pathdata = #"\\192.168.1.27\Temp\";
var d = new DirectoryInfo(pathdata);
var files = d.GetFiles("*.pdf");
foreach (var filename in files)
{
Console.WriteLine(pathdata + filename.Name + ".pdf");
}
Console.ReadLine();
}
}
}
The above code does work (on my machine - that is my current IP address and a folder I have shared on my machine). Your code likely doesn't work for one of two reasons:
Your path is not valid (e.g. using / instead of \)
CBR folder is not shared
string host = #"ftphost";
string username = "user";
string password = "********";
string localFileName = System.IO.Path.GetFileName(#"localfilename");
string remoteDirectory = "/export/";
using (var sftp = new SftpClient(host, username, password))
{
sftp.Connect();
var files = sftp.ListDirectory(remoteDirectory);
foreach (var file in files)
{
if (!file.Name.StartsWith("."))
{
string remoteFileName = file.Name;
if (file.LastWriteTime.Date == DateTime.Today)
Console.WriteLine(file.FullName);
File.OpenWrite(localFileName);
string sDir = #"localpath";
Stream file1 = File.OpenRead(remoteDirectory + file.Name);
sftp.DownloadFile(remoteDirectory, file1);
}
}
}
I am using SSH.NET (Renci.SshNet) library to work with an SFTP server. What I need to do is grab files from a specific folder on the SFTP server based on today's date. Then copy those files from the SFTP server to a local drive a server of mine.
Above is the code I have but it is not working. Sometimes it says file does not exist but sometimes the files I will be downloading will not be on my local servers but I need to download whatever files were uploaded to the remote folder for that day.
Can someone take a look and see what is wrong? I believe it has something to do with the stream portion. I have worked with FTP much besides uploading files which I took some previous code I had and thought I could reverse the process but that isn't working. The code I used is based off of this example.
A simple working code to download a file with SSH.NET library is:
using (Stream fileStream = File.Create(#"C:\target\local\path\file.zip"))
{
sftp.DownloadFile("/source/remote/path/file.zip", fileStream);
}
See also Downloading a directory using SSH.NET SFTP in C#.
To explain, why your code does not work:
The second parameter of SftpClient.DownloadFile is a stream to write a downloaded contents to.
You are passing in a read stream instead of a write stream. And moreover the path you are opening read stream with is a remote path, what cannot work with File class operating on local files only.
Just discard the File.OpenRead line and use a result of previous File.OpenWrite call instead (that you are not using at all now):
Stream file1 = File.OpenWrite(localFileName);
sftp.DownloadFile(file.FullName, file1);
Or even better, use File.Create to discard any previous contents that the local file may have.
I'm not sure if your localFileName is supposed to hold full path, or just file name. So you may need to add a path too, if necessary (combine localFileName with sDir?)
While the example works, its not the correct way to handle the streams...
You need to ensure the closing of the files/streams with the using clause..
Also, add try/catch to handle IO errors...
public void DownloadAll()
{
string host = #"sftp.domain.com";
string username = "myusername";
string password = "mypassword";
string remoteDirectory = "/RemotePath/";
string localDirectory = #"C:\LocalDriveFolder\Downloaded\";
using (var sftp = new SftpClient(host, username, password))
{
sftp.Connect();
var files = sftp.ListDirectory(remoteDirectory);
foreach (var file in files)
{
string remoteFileName = file.Name;
if ((!file.Name.StartsWith(".")) && ((file.LastWriteTime.Date == DateTime.Today))
using (Stream file1 = File.Create(localDirectory + remoteFileName))
{
sftp.DownloadFile(remoteDirectory + remoteFileName, file1);
}
}
}
}
My version of #Merak Marey's Code. I am checking if files exist already and different download directories for .txt and other files
static void DownloadAll()
{
string host = "xxx.xxx.xxx.xxx";
string username = "###";
string password = "123";string remoteDirectory = "/IN/";
string finalDir = "";
string localDirectory = #"C:\filesDN\";
string localDirectoryZip = #"C:\filesDN\ZIP\";
using (var sftp = new SftpClient(host, username, password))
{
Console.WriteLine("Connecting to " + host + " as " + username);
sftp.Connect();
Console.WriteLine("Connected!");
var files = sftp.ListDirectory(remoteDirectory);
foreach (var file in files)
{
string remoteFileName = file.Name;
if ((!file.Name.StartsWith(".")) && ((file.LastWriteTime.Date == DateTime.Today)))
{
if (!file.Name.Contains(".TXT"))
{
finalDir = localDirectoryZip;
}
else
{
finalDir = localDirectory;
}
if (File.Exists(finalDir + file.Name))
{
Console.WriteLine("File " + file.Name + " Exists");
}else{
Console.WriteLine("Downloading file: " + file.Name);
using (Stream file1 = File.OpenWrite(finalDir + remoteFileName))
{
sftp.DownloadFile(remoteDirectory + remoteFileName, file1);
}
}
}
}
Console.ReadLine();
}
Massimiliano's solution has one problem which will lead to files not being completely downloaded. The FileStream must be closed. This is especially a problem for encrypted files. They won't completely decrypt intermittently without closing the stream.
var files = sftp.ListDirectory(remoteVendorDirectory).Where(f => !f.IsDirectory);
foreach (var file in files)
{
var filename = $"{LocalDirectory}/{file.Name}";
if (!File.Exists(filename))
{
Console.WriteLine("Downloading " + file.FullName);
using (var localFile = File.OpenWrite(filename))
sftp.DownloadFile(file.FullName, localFile);
}
}
This solves the problem on my end.
var files = sftp.ListDirectory(remoteVendorDirectory).Where(f => !f.IsDirectory);
foreach (var file in files)
{
var filename = $"{LocalDirectory}/{file.Name}";
if (!File.Exists(filename))
{
Console.WriteLine("Downloading " + file.FullName);
using (var localFile = File.OpenWrite(filename))
sftp.DownloadFile(file.FullName, localFile);
}
}
Without you providing any specific error message, it's hard to give specific suggestions.
However, I was using the same example and was getting a permissions exception on File.OpenWrite - using the localFileName variable, because using Path.GetFile was pointing to a location that obviously would not have permissions for opening a file > C:\ProgramFiles\IIS(Express)\filename.doc
I found that using System.IO.Path.GetFileName is not correct, use System.IO.Path.GetFullPath instead, point to your file starting with "C:\..."
Also open your solution in FileExplorer and grant permissions to asp.net for the file or any folders holding the file. I was able to download my file at that point.
This question already has an answer here:
Compress a file with RAR
(1 answer)
Closed 9 years ago.
In This Method Finally after saving the File Path I have to Copy the same File and i need to move to another place and i need to make that file as a Rar file How to Do this?
public JsonResult UploadResume(HttpPostedFileBase resume)
{
if (resume != null)
{
string Name = System.IO.Path.GetFileName(resume.FileName);
string path = Server.MapPath("~/Uploads/Resume/0");
string extension = Path.GetExtension(resume.FileName);
DirectoryInfo di = null;
if (!Directory.Exists(path))
{
di = Directory.CreateDirectory(path);
}
var filepath = System.IO.Path.Combine(path, 0 + extension);
resume.SaveAs(filepath);
}
return Json(new { result = true }, JsonRequestBehavior.AllowGet);
}
Try like this
using System;
using System.Collections.Generic;
using System.Text;
using System.IO;
using System.IO.Compression;
namespace ConsoleApplication1
{
class Program
{
static void Main(string[] args)
{
string sFileToZip = #"C:\Documents and Settings\My Documents\Visual Studio 2008\Projects\ConsoleApplication1\bin\Debug\Stuff\text1.txt";
string sZipFile = #"C:\Documents and Settings\My Documents\Visual Studio 2008\Projects\ConsoleApplication1\bin\Debug\Stuff\text1.zip";
using (FileStream __fStream = File.Open(sZipFile, FileMode.Create))
{
GZipStream obj = new GZipStream(__fStream, CompressionMode.Compress);
byte[] bt = File.ReadAllBytes(sFileToZip);
obj.Write(bt, 0, bt.Length);
obj.Close();
obj.Dispose();
}
}
}
}
Source
You need to have some kind of third party library that will compress the files into RAR format, or execute a command line tool (like RAR.EXE) to do it.
Get a rar utility (WinRAR, 7-zip or sth else) and start it from your code
System.Diagnostics.ProcessStart(pathToMyUtility, arguments);
and that's it.
I'm creating C# console app to clean up my download folder in windows
my app work fine for video file and move and and delete it from download folder. but how can I make get get the file in subfolder and add it to my files array?
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
namespace CleanDownloadFolder
{
class Program
{
static void Main(string[] args)
{
string sourcePath = #"C:\Users\___\Downloads";
string targetPath = #"C:\Users\__\Videos";
CopyDirectory(sourcePath, targetPath);
}
private static void CopyDirectory(string sourcePath, string targetPath)
{
// To copy a folder's contents to a new location:
// Create a new target folder, if necessary.
if (!System.IO.Directory.Exists(targetPath))
{
System.IO.Directory.CreateDirectory(targetPath);
}
if (System.IO.Directory.Exists(sourcePath))
{
string[] files = System.IO.Directory.GetFiles(sourcePath);
string fileName = null;
string destFile = null;
// Copy the files and overwrite destination files if they already exist.
foreach (string s in files)
{
// Use static Path methods to extract only the file name from the path.
fileName = System.IO.Path.GetFileName(s);
destFile = System.IO.Path.Combine(targetPath, fileName);
if (Path.GetExtension(fileName) == ".avi")
{
System.IO.File.Copy(s, destFile, true);
System.IO.File.Delete(s);
}
}
}
}
}
}
Directory.GetFiles has an overload that could be used to get the list of files in subdirectories
string[] files = Directory.GetFiles(sourcePath, "*.*", SearchOption.AllDirectories);
The remainder of your code should work as is, however, if your are interested only in the AVI files then you could put that extension directly in the GetFiles call. In that way you get only AVI files and your code could be simplified removing the if
string[] files = Directory.GetFiles(sourcePath. "*.AVI", SearchOption.AllDirectories);
string fileName = null;
string destFile = null;
// Copy the files and overwrite destination files if they already exist.
foreach (string s in files)
{
// Use static Path methods to extract only the file name from the path.
fileName = Path.GetFileName(s);
destFile = Path.Combine(targetPath, fileName);
File.Copy(s, destFile, true);
File.Delete(s);
}
I suggest also to add a using System.IO; at the top of your code file to avoid all of that full namespace typing required without the using
What's the best way to create a file under a nested folder in Sharepoint ?
My current method
public string CreateSPFile(string spServerURL, string spDocumentLibraryURL, string folder, string fileName, Stream fileStream, bool overwrite)
{
if (SPSite.Exists(new Uri(spServerURL)))
{
SPSite site = new SPSite(spServerURL);
SPWeb oWebsite = site.OpenWeb();
oWebsite.AllowUnsafeUpdates = true;
SPFolder spFolder = oWebsite.Folders[spDocumentLibraryURL];
if (!string.IsNullOrEmpty(folder))
{
spFolder.SubFolders[folder].Files.Add(fileName, fileStream, overwrite);
}
else
{
SPFileCollection files = spFolder.Files;
spFolder.Files.Add(fileName, fileStream, overwrite);
}
oWebsite.AllowUnsafeUpdates = false;
site.Close();
}
}
As you can see, if I want to create a file under nested folder, i need to modified my codes.
What will be better way to handle this kind of saving nested folder situation?
According to my project structure, the file can be like /DocumentLibrary/Folder1/Folder2/Folder3/File.txt.
You can load a folder by its server relative URL:
SPFolder folder = web.GetFolder("/DocumentLibrary/Folder1/Folder2/Folder3/");
With this approach you do not have to load folder by folder and your code works with n folder levels.
I've updated your code sample and added some comments regarding SharePoint best practices:
public string CreateSPFile(string spServerURL, string spDocumenttargetUrl, string folder, string fileName, Stream fileStream, bool overwrite)
{
// I suggest skip this pre check since it internally opens a new site object
// If you have to silenlty ignore non-existant SPSite you should catch a FileNotFoundException.
if (SPSite.Exists(new Uri(spServerURL)))
{
// use the using construct to safely dispose the opened SPSite object
using (SPSite site = new SPSite(spServerURL))
{
// SPWeb object opened with SPSite.OpenWeb() have to be disposed as well
using (SPWeb web = site.OpenWeb())
{
web.AllowUnsafeUpdates = true;
string targetUrl = SPUrlUtility.CombineUrl(web.ServerRelativeUrl, spDocumenttargetUrl);
if (!String.IsNullOrEmpty(folder))
{
targetUrl = SPUrlUtility.CombineUrl(targetUrl, folder);
}
SPFolder target = web.GetFolder(target);
SPFileCollection files = target.Files;
target.Files.Add(fileName, fileStream, overwrite);
// no need to revert AllowUnsafeUpdates for newly opened webs
// web.AllowUnsafeUpdates = false;
}
}
}
}
For uploading a file into a nested folder you could consider the following approach:
ensure the target folder exist using the method EnsureFolder provided below
upload a file using SPFileCollection.Add method
How to ensure a nested Folder exist using SharePoint SSOM
internal static class SPWebExtensions
{
/// <summary>
/// Ensure SPFolder
/// </summary>
/// <param name="web"></param>
/// <param name="listTitle"></param>
/// <param name="folderUrl"></param>
/// <returns></returns>
public static SPFolder EnsureFolder(this SPWeb web, string listTitle, string folderUrl)
{
if (string.IsNullOrEmpty(folderUrl))
throw new ArgumentNullException("folderUrl");
var list = web.Lists.TryGetList(listTitle);
return CreateFolderInternal(list, list.RootFolder, folderUrl);
}
private static SPFolder CreateFolderInternal(SPList list, SPFolder parentFolder, string folderUrl)
{
var folderNames = folderUrl.Split(new char[] {'/'}, StringSplitOptions.RemoveEmptyEntries);
var folderName = folderNames[0];
var curFolder =
parentFolder.SubFolders.Cast<SPFolder>().FirstOrDefault( f => System.String.Compare(f.Name, folderName, System.StringComparison.OrdinalIgnoreCase) == 0);
if (curFolder == null)
{
var folderItem = list.Items.Add(parentFolder.ServerRelativeUrl, SPFileSystemObjectType.Folder,
folderName);
folderItem.SystemUpdate();
curFolder = folderItem.Folder;
}
if (folderNames.Length > 1)
{
var subFolderUrl = string.Join("/", folderNames, 1, folderNames.Length - 1);
return CreateFolderInternal(list, curFolder, subFolderUrl);
}
return curFolder;
}
}
Gist: EnsureFolder.cs
The following example demonstrates how to ensure the following folder structure exist under Documents library and upload a file into it:
Orders
|
A --
|
A1
Example:
var targetFolder = web.EnsureFolder("Documents", "Orders3/A/A1");
var fileContent = System.IO.File.ReadAllBytes(fileName);
var fileUrl = Path.GetFileName(fileName);
targetFolder.Files.Add(fileUrl, fileContent);