We Keep Coding

ASP.NET Core ASW S3 - The request signature we calculated does not match the signature you provided

I am getting this error when I try to upload a file to the bucket.
This is my code.
public async Task UploadFileAsync(IFormFile file, string userId)
{
var filePath = Path.GetFullPath(file.FileName);
var bucketName = this.configuration.GetSection("Amazon")["BucketName"];
var accessKey = this.configuration.GetSection("Amazon")["AWSAccessKey"];
var secretKey = this.configuration.GetSection("Amazon")["AWSSecretKey"];
var bucketRegion = RegionEndpoint.EUWest1;
var s3Client = new AmazonS3Client(accessKey, secretKey, bucketRegion);
try
{
var fileTransferUtility =
new TransferUtility(s3Client);
using (var newMemoryStream = new MemoryStream())
{
file.CopyTo(newMemoryStream);
var uploadRequest = new TransferUtilityUploadRequest
{
InputStream = newMemoryStream,
Key = file.FileName,
BucketName = bucketName,
CannedACL = S3CannedACL.PublicRead,
};
await fileTransferUtility.UploadAsync(uploadRequest);
}
await this.filesRepository.AddAsync(new FileBlob
{
Name = file.FileName,
Extension = file.FileName.Split('.')[1],
Size = file.Length,
UserId = userId,
UploadedOn = DateTime.UtcNow,
});
await this.filesRepository.SaveChangesAsync();
}
catch (AmazonS3Exception e)
{
Console.WriteLine(e.Message);
}
catch (Exception e)
{
Console.WriteLine(e.Message);
}
}
Razor: here
It always says "The request signature we calculated does not match the signature you provided. Check your key and signing method."
Any ideas?

The problem was the mistaken accessKey and secretKey. After I got them right all I had to do is change the CannedACL = S3CannedACL.BucketOwnerFullControl. That solved it for me

get url of uploaded s3 object .net

So currently my code uploads an image to S3 but what I want it to now do is to get the URL of the image that has been uploaded, so this URL can be stored in the DB and used later.
I know it's possible, as it was shown in this question here: s3 file upload does not return response (but that is JavaScript and I'm struggling to convert it to c#)
This is my code, it works perfectly, I just need to get the URL of the uploaded object + is there a way to make the object public by default. I tried to console write the response but that was no help
public class AmazonS3Uploader
{
private string bucketName = "cartalkio-image-storage-dev";
private string keyName = DateTime.Now.ToString() + ".png";
public async void UploadFile()
{
byte[] bytes = System.Convert.FromBase64String("iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==");
Stream stream = new MemoryStream(bytes);
var myAwsAccesskey = "*************";
var myAwsSecret = "**************************";
var client = new AmazonS3Client(myAwsAccesskey, myAwsSecret, Amazon.RegionEndpoint.EUWest2);
try
{
PutObjectRequest putRequest = new PutObjectRequest
{
BucketName = bucketName,
Key = keyName,
ContentType = "image/png",
InputStream = stream
};
PutObjectResponse response = await client.PutObjectAsync(putRequest);
// Console.Write(response);
}
catch (AmazonS3Exception amazonS3Exception)
{
if (amazonS3Exception.ErrorCode != null &&
(amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId")
||
amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
{
throw new Exception("Check the provided AWS Credentials.");
}
else
{
throw new Exception("Error occurred: " + amazonS3Exception.Message);
}
}
}

I've searched the documentation and various websites but this was all I found () - I guess there just isn't a URL that is returned. What I've done is changed my code around a bit because you can always predict what the object URL will be based on the name of the object you're uploading. i.e if you're uploading an image called 'test.png', the URL will be this:
https://[Your-Bucket-Name].s3.[S3-Region].amazonaws.com/[test.png]
I tried dependency injection but AWS didn't like that so I've changed my code to this:
(in this example I'm receiving a base64 string, turning it into a BYTE and then into a SYSTEM.IO.Stream)
This is the request I'm sending up:
{ "image":"iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg=="
}
Controller:
public async Task<ActionResult> Index(string image)
{
AmazonS3Uploader amazonS3 = new AmazonS3Uploader();
// This bit creates a random string that will be used as part of the URL
var chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
var stringChars = new char[8];
var random = new Random();
for (int i = 0; i < stringChars.Length; i++)
{
stringChars[i] = chars[random.Next(chars.Length)];
}
var finalString = new String(stringChars);
// This bit adds the '.png' as its an image
var keyName = finalString + ".png";
// This uploads the file to S3, passing through the keyname (which is the end of the URL) and the image string
amazonS3.UploadFile(keyName, image);
// This is what the final URL of the object will be, so you can use this variable later or save it in your database
var itemUrl = "https://[your-bucket-name].s3.[S3-Region].amazonaws.com/" + keyName;
return Ok();
}
AmazonS3Uploader.cs
private string bucketName = "your-bucket-name";
public async void UploadFile(string keyName, string image)
{
byte[] bytes = System.Convert.FromBase64String(image);
Stream stream = new MemoryStream(bytes);
var myAwsAccesskey = "*************";
var myAwsSecret = "**************************";
var client = new AmazonS3Client(myAwsAccesskey, myAwsSecret, Amazon.RegionEndpoint.[S3-Region]);
try
{
PutObjectRequest putRequest = new PutObjectRequest
{
BucketName = bucketName,
Key = keyName,
ContentType = "image/png",
InputStream = stream
};
PutObjectResponse response = await client.PutObjectAsync(putRequest);
}
catch (AmazonS3Exception amazonS3Exception)
{
if (amazonS3Exception.ErrorCode != null &&
(amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId")
||
amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
{
throw new Exception("Check the provided AWS Credentials.");
}
else
{
throw new Exception("Error occurred: " + amazonS3Exception.Message);
}
}
}
The only problem with this is if the file upload to S3 Fails, you might still get the URL, and if you're saving it to your database - you'll save the URL to the database but the object wont exist in the S3 bucket, so it won't lead anywhere. If you implement dependency injection - this shouldn't be an issue (dependency injection not implemented in this example)

How can I get a single object from a particular folder in an Amazon S3 bucket

How can I get a single object from a particular folder in Amazon S3 bucket?
By Using the Asp.Net C# language, I don't know how to retrieve a single object that are in a folder in bucket of Amazon S3 Service?
What I tried so far:
var putRequest1 = new PutObjectRequest { BucketName = bucketName, Key =
"Employee/Word.pdf", FilePath = filePath, }; PutObjectResponse response1 =
await client.PutObjectAsync(putRequest1); var putRequest2 = new
PutObjectRequest { BucketName = bucketName, Key = "Employee/Tenses.txt",
FilePath = filePath, }; PutObjectResponse response2 = await
client.PutObjectAsync(putRequest2);

From AWS Documentation Get an Object Using the AWS SDK for .NET.
using Amazon.S3;
using Amazon.S3.Model;
using System;
using System.IO;
using System.Threading.Tasks;
namespace Amazon.DocSamples.S3
{
class GetObjectTest
{
private const string bucketName = "*** bucket name ***";
private const string keyName = "*** object key ***";
// Specify your bucket region (an example region is shown).
private static readonly RegionEndpoint bucketRegion = RegionEndpoint.USWest2;
private static IAmazonS3 client;
public static void Main()
{
client = new AmazonS3Client(bucketRegion);
ReadObjectDataAsync().Wait();
}
static async Task ReadObjectDataAsync()
{
string responseBody = "";
try
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = bucketName,
Key = keyName
};
using (GetObjectResponse response = await client.GetObjectAsync(request))
using (Stream responseStream = response.ResponseStream)
using (StreamReader reader = new StreamReader(responseStream))
{
string title = response.Metadata["x-amz-meta-title"]; // Assume you have "title" as medata added to the object.
string contentType = response.Headers["Content-Type"];
Console.WriteLine("Object metadata, Title: {0}", title);
Console.WriteLine("Content type: {0}", contentType);
responseBody = reader.ReadToEnd(); // Now you process the response body.
}
}
catch (AmazonS3Exception e)
{
Console.WriteLine("Error encountered ***. Message:'{0}' when writing an object", e.Message);
}
catch (Exception e)
{
Console.WriteLine("Unknown encountered on server. Message:'{0}' when writing an object", e.Message);
}
}
}
}

How to create folders in google drive without duplicate using c#?

I am trying to create a folder on my google drive and check if it already exists. I am new to c#, so I wrote a code to create a specific folder in which I can upload my CSV files, but each time my application runs, it creates a duplicate folder with same name and I am now trying to figure out how to stop it.
class GoogleApi
{
private string[] Scopes = { DriveService.Scope.Drive };
private string ApplicationName;
private string FolderId;
private string FileName;
private string FilePath;
private string ContentType;
public GoogleApi(string fileName, string filePath)
{
ApplicationName = "MantisToDrive";
ContentType = "text/csv";
FileName = fileName;
FilePath = filePath;
}
public void Upload()
{
UserCredential credential = GetCredentials();
DriveService service = GetDriveService(credential);
FolderId = CreateFolderToDrive(service, "MantisData");
UploadFileToDrive(service, FileName, FilePath, ContentType);
}
private DriveService GetDriveService(UserCredential credential)
{
return new DriveService(new BaseClientService.Initializer()
{
HttpClientInitializer = credential,
ApplicationName = ApplicationName,
});
}
//verify User Credentials using client_secret.json file
private UserCredential GetCredentials()
{
UserCredential credential;
using (var stream = new FileStream("client_secret.json", FileMode.Open, FileAccess.Read))
{
string credPath = System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal);
credPath = Path.Combine(credPath, ".credentials/drive-dotnet-quickstart.json");
credential = GoogleWebAuthorizationBroker.AuthorizeAsync(
GoogleClientSecrets.Load(stream).Secrets,
Scopes,
"user",
CancellationToken.None,
new FileDataStore(credPath , true)).Result;
}
return credential;
}
private string UploadFileToDrive(DriveService service, string fileName, string filePath, string contentType)
{
var fileMatadata = new Google.Apis.Drive.v3.Data.File();
fileMatadata.Name = fileName;
fileMatadata.Parents = new List<string> { FolderId };
FilesResource.CreateMediaUpload request;
using (var stream = new FileStream(filePath, FileMode.Open))
{
request = service.Files.Create(fileMatadata, stream, contentType);
//service.Files.Delete(fileName).Execute();
request.Upload();
}
var file = request.ResponseBody;
return file.Id;
}
// creating folder in google drive to store CSV
public static string CreateFolderToDrive(DriveService service, string folderName)
{
bool exists = Exists(service,folderName);
if (exists)
return $"Sorry but the file {folderName} already exists!";
var file = new Google.Apis.Drive.v3.Data.File();
file.Name = folderName;
file.MimeType = "application/vnd.google-apps.folder";
var request = service.Files.Create(file);
request.Fields = "id";
var result = request.Execute();
return result.Id;
}
private static bool Exists(DriveService service, string name)
{
var listRequest = service.Files.List();
listRequest.PageSize = 100;
listRequest.Q = $"trashed = false and name contains '{name}' and 'root' in parents";
listRequest.Fields = "files(name)";
var files = listRequest.Execute().Files;
foreach (var file in files)
{
if (name == file.Name)
return true;
}
return false;
}
}

This should work (this checks whether the folder exists in your root folder. If not, the method will create a folder):
public static string CreateFolder(string folderName)
{
bool exists = Exists(folderName);
if (exists)
return $"Sorry but the file {folderName} already exists!";
var file = new Google.Apis.Drive.v3.Data.File();
file.Name = folderName;
file.MimeType = "application/vnd.google-apps.folder";
var request = service.Files.Create(file);
request.Fields = "id";
return request.Execute().Id;
}
private static bool Exists(string name)
{
var listRequest = service.Files.List();
listRequest.PageSize = 100;
listRequest.Q = $"trashed = false and name contains '{name}' and 'root' in parents";
listRequest.Fields = "files(name)";
var files = listRequest.Execute().Files;
foreach (var file in files)
{
if (name == file.Name)
return true;
}
return false;
}
In case somebody comes here from the future and the pastebin link is dead. This is how you can upload a file to your google drive:
public static async Task UploadFileAsync(string fileName)
{
var file = new Google.Apis.Drive.v3.Data.File();
file.Name = Path.GetFileName(fileName);
file.Parents = new List<string> {folderID};
byte[] byteArray = System.IO.File.ReadAllBytes(fileName);
using (System.IO.MemoryStream stream = new System.IO.MemoryStream(byteArray))
{
await service.Files.Create(file, stream, "some mime type").UploadAsync();
}
}

Download Objects from S3 Bucket using c#

Im trying to download object from S3 bucket facing below issue
The Security token included in the request is Invalid .
Please check and correct where is the mistake.
Below is my code
1. Get Temporary credentails:
main()
{
string path = "http://XXX.XXX.XXX./latest/meta-data/iam/security-credentials/EC2_WLMA_Permissions";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(path);
request.Method = "GET";
request.ContentType = "application/json";
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
string result = string.Empty;
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
result = reader.ReadToEnd();
dynamic metaData = JsonConvert.DeserializeObject(result);
_awsAccessKeyId = metaData.AccessKeyId;
_awsSecretAccessKey = metaData.SecretAccessKey;
}
}
Create SessionAWSCredentials instance:
SessionAWSCredentials tempCredentials =
GetTemporaryCredentials(_awsAccessKeyId, _awsSecretAccessKey);
//GetTemporaryCredentials method:
private static SessionAWSCredentials GetTemporaryCredentials(
string accessKeyId, string secretAccessKeyId)
{
AmazonSecurityTokenServiceClient stsClient =
new AmazonSecurityTokenServiceClient(accessKeyId,
secretAccessKeyId);
Console.WriteLine(stsClient.ToString());
GetSessionTokenRequest getSessionTokenRequest =
new GetSessionTokenRequest();
getSessionTokenRequest.DurationSeconds = 7200; // seconds
GetSessionTokenResponse sessionTokenResponse =
stsClient.GetSessionToken(getSessionTokenRequest);
Console.WriteLine(sessionTokenResponse.ToString());
Credentials credentials = sessionTokenResponse.Credentials;
Console.WriteLine(credentials.ToString());
SessionAWSCredentials sessionCredentials =
new SessionAWSCredentials(credentials.AccessKeyId,
credentials.SecretAccessKey,
credentials.SessionToken);
return sessionCredentials;
}
Get files from S3 using AmazonS3Client:
using (IAmazonS3 client = new AmazonS3Client(tempCredentials,RegionEndpoint.USEast1))
{
GetObjectRequest request = new GetObjectRequest();
request.BucketName = "bucketName" + #"/" + "foldername";
request.Key = "Terms.docx";
GetObjectResponse response = client.GetObject(request);
response.WriteResponseStreamToFile("C:\\MyFile.docx");
}

We do something a little simpler for interfacing with S3 (downloads and uploads)
It looks like you went the more complex approach. You should try just using the TransferUtility instead:
TransferUtility fileTransferUtility =
new TransferUtility(
new AmazonS3Client("ACCESS-KEY-ID", "SECRET-ACCESS-KEY", Amazon.RegionEndpoint.CACentral1));
// Note the 'fileName' is the 'key' of the object in S3 (which is usually just the file name)
fileTransferUtility.Download(filePath, "my-bucket-name", fileName);
NOTE: TransferUtility.Download() returns void because it downloads the file to the path specified in the filePath argument. This may be a little different than what you were expecting but you can still open a FileStream to that path afterwards and manipulate the file all you want. For example:
using (FileStream fileDownloaded = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
// Do stuff with our newly downloaded file
}

Bucketname, Accesskey and secretkey, I took from web config. You could type manually.
public void DownloadObject(string imagename)
{
RegionEndpoint bucketRegion = RegionEndpoint.USEast1;
IAmazonS3 client = new AmazonS3Client(bucketRegion);
string accessKey = System.Configuration.ConfigurationManager.AppSettings["AWSAccessKey"];
string secretKey = System.Configuration.ConfigurationManager.AppSettings["AWSSecretKey"];
AmazonS3Client s3Client = new AmazonS3Client(new BasicAWSCredentials(accessKey, secretKey), Amazon.RegionEndpoint.USEast1);
string objectKey = "EMR" + "/" + imagename;
//EMR is folder name of the image inside the bucket
GetObjectRequest request = new GetObjectRequest();
request.BucketName = System.Configuration.ConfigurationManager.AppSettings["bucketname"];
request.Key = objectKey;
GetObjectResponse response = s3Client.GetObject(request);
response.WriteResponseStreamToFile("D:\\Test\\"+ imagename);
}
//> D:\Test\ is local file path.

Following is how I download it. I need to download only .zip files in this case. Restricting to only required file types (.zip in my case), helped me to avoid errors related to (416) Requested Range Not Satisfiable
public static class MyAWS_S3_Helper
{
static string _S3Key = System.Configuration.ConfigurationManager.ConnectionStrings["S3BucketKey"].ConnectionString;
static string _S3SecretKey = System.Configuration.ConfigurationManager.ConnectionStrings["S3BucketSecretKey"].ConnectionString;
public static void S3Download(string bucketName, string _ObjectKey, string downloadPath)
{
IAmazonS3 _client = new AmazonS3Client(_S3Key, _S3SecretKey, Amazon.RegionEndpoint.USEast1);
TransferUtility fileTransferUtility = new TransferUtility(_client);
fileTransferUtility.Download(downloadPath + "\\" + _ObjectKey, bucketName, _ObjectKey);
_client.Dispose();
}
public static async Task AsyncDownload(string bucketName, string downloadPath, string requiredSunFolder)
{
var bucketRegion = RegionEndpoint.USEast1; //Change it
var credentials = new BasicAWSCredentials(_S3Key, _S3SecretKey);
var client = new AmazonS3Client(credentials, bucketRegion);
var request = new ListObjectsV2Request
{
BucketName = bucketName,
MaxKeys = 1000
};
var response = await client.ListObjectsV2Async(request);
var utility = new TransferUtility(client);
foreach (var obj in response.S3Objects)
{
string currentKey = obj.Key;
double sizeCheck = Convert.ToDouble(obj.Size);
int fileNameLength = currentKey.Length;
Console.WriteLine(currentKey + "---" + fileNameLength.ToString());
if (currentKey.Contains(requiredSunFolder))
{
if (currentKey.Contains(".zip")) //This helps to avoid errors related to (416) Requested Range Not Satisfiable
{
try
{
S3Download(bucketName, currentKey, downloadPath);
}
catch (Exception exTest)
{
string messageTest = currentKey + "-" + exTest;
}
}
}
}
}
}
Here is how it is called
static void Main(string[] args)
{
string downloadPath = #"C:\SourceFiles\TestDownload";
Task awsTask = MyAWS_S3_Helper.AsyncDownload("my-files", downloadPath, "mysubfolder");
awsTask.Wait();
}

Here is what I have done to download the files from S3 bucket,
var AwsImportFilePathParcel = "TEST/TEMP"
IAmazonS3 client = new AmazonS3Client(AwsAccessKey,AwsSecretKey);
S3DirectoryInfo info = new S3DirectoryInfo(client, S3BucketName, AwsImportFilePathParcel);
S3FileInfo[] s3Files = info.GetFiles(pattenForParcel);
now in s3Files, you have all the files which are on provided location, using for each you can save all files in to your system
foreach (var fileInfo in s3Files)
{
var localPath = Path.Combine("C:\TEST\", fileInfo.Name);
var file = fileInfo.CopyToLocal(localPath);
}