We Keep Coding

Upload file to Contabo Object Storage (S3 compatible)

I'm trying to upload a file to the contabo object storage from memory stream using AWSSDK.S3
This is my client configuration.
string accessKey = "xxxxxxxxxxxxxxxxxxxxxxxxxx";
string secretKey = "xxxxxxxxxxxxxxxxxxxxxxxxxx";
Amazon.S3.AmazonS3Config config = new Amazon.S3.AmazonS3Config();
Amazon.S3.AmazonS3Client s3Client;
public BookingsController()
{
config.ServiceURL = "https://eu2.contabostorage.com";
config.DisableHostPrefixInjection = true;
s3Client = new Amazon.S3.AmazonS3Client(
accessKey,
secretKey,
config
);
}
This is the method I'm using:
[HttpPost("/api/Bookings/AddFile")]
public async Task<ActionResult> AddBookingFile([FromForm] IFormFile file)
{
using (var newMemoryStream = new MemoryStream())
{
ListBucketsResponse response = await s3Client.ListBucketsAsync();
file.CopyTo(newMemoryStream);
Amazon.S3.Model.PutObjectRequest request = new Amazon.S3.Model.PutObjectRequest();
request.BucketName = "test-bucket";
request.Key = "recording.wav";
request.ContentType = "audio/wav";
request.InputStream = newMemoryStream;
await s3Client.PutObjectAsync(request);
}
return Ok();
}
The ListBucket-Method works properly. The PutObject Method throws the exception that the Host "Der angegebene Host ist unbekannt. (test-bucket.eu2.contabostorage.com:443)" cannot be found.
Referenced to the Contabo-docs is that correct, because Contabo doesn't support virtual hosted buckets (dns prefix). Reference to contabo docs
I thought, that the following configuration fix this, but that wasn't the solution.
config.DisableHostPrefixInjection = true;
Does anyone has any advise hot to prevent the prefixing of the url?

get url of uploaded s3 object .net

So currently my code uploads an image to S3 but what I want it to now do is to get the URL of the image that has been uploaded, so this URL can be stored in the DB and used later.
I know it's possible, as it was shown in this question here: s3 file upload does not return response (but that is JavaScript and I'm struggling to convert it to c#)
This is my code, it works perfectly, I just need to get the URL of the uploaded object + is there a way to make the object public by default. I tried to console write the response but that was no help
public class AmazonS3Uploader
{
private string bucketName = "cartalkio-image-storage-dev";
private string keyName = DateTime.Now.ToString() + ".png";
public async void UploadFile()
{
byte[] bytes = System.Convert.FromBase64String("iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==");
Stream stream = new MemoryStream(bytes);
var myAwsAccesskey = "*************";
var myAwsSecret = "**************************";
var client = new AmazonS3Client(myAwsAccesskey, myAwsSecret, Amazon.RegionEndpoint.EUWest2);
try
{
PutObjectRequest putRequest = new PutObjectRequest
{
BucketName = bucketName,
Key = keyName,
ContentType = "image/png",
InputStream = stream
};
PutObjectResponse response = await client.PutObjectAsync(putRequest);
// Console.Write(response);
}
catch (AmazonS3Exception amazonS3Exception)
{
if (amazonS3Exception.ErrorCode != null &&
(amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId")
||
amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
{
throw new Exception("Check the provided AWS Credentials.");
}
else
{
throw new Exception("Error occurred: " + amazonS3Exception.Message);
}
}
}

I've searched the documentation and various websites but this was all I found () - I guess there just isn't a URL that is returned. What I've done is changed my code around a bit because you can always predict what the object URL will be based on the name of the object you're uploading. i.e if you're uploading an image called 'test.png', the URL will be this:
https://[Your-Bucket-Name].s3.[S3-Region].amazonaws.com/[test.png]
I tried dependency injection but AWS didn't like that so I've changed my code to this:
(in this example I'm receiving a base64 string, turning it into a BYTE and then into a SYSTEM.IO.Stream)
This is the request I'm sending up:
{ "image":"iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg=="
}
Controller:
public async Task<ActionResult> Index(string image)
{
AmazonS3Uploader amazonS3 = new AmazonS3Uploader();
// This bit creates a random string that will be used as part of the URL
var chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
var stringChars = new char[8];
var random = new Random();
for (int i = 0; i < stringChars.Length; i++)
{
stringChars[i] = chars[random.Next(chars.Length)];
}
var finalString = new String(stringChars);
// This bit adds the '.png' as its an image
var keyName = finalString + ".png";
// This uploads the file to S3, passing through the keyname (which is the end of the URL) and the image string
amazonS3.UploadFile(keyName, image);
// This is what the final URL of the object will be, so you can use this variable later or save it in your database
var itemUrl = "https://[your-bucket-name].s3.[S3-Region].amazonaws.com/" + keyName;
return Ok();
}
AmazonS3Uploader.cs
private string bucketName = "your-bucket-name";
public async void UploadFile(string keyName, string image)
{
byte[] bytes = System.Convert.FromBase64String(image);
Stream stream = new MemoryStream(bytes);
var myAwsAccesskey = "*************";
var myAwsSecret = "**************************";
var client = new AmazonS3Client(myAwsAccesskey, myAwsSecret, Amazon.RegionEndpoint.[S3-Region]);
try
{
PutObjectRequest putRequest = new PutObjectRequest
{
BucketName = bucketName,
Key = keyName,
ContentType = "image/png",
InputStream = stream
};
PutObjectResponse response = await client.PutObjectAsync(putRequest);
}
catch (AmazonS3Exception amazonS3Exception)
{
if (amazonS3Exception.ErrorCode != null &&
(amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId")
||
amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
{
throw new Exception("Check the provided AWS Credentials.");
}
else
{
throw new Exception("Error occurred: " + amazonS3Exception.Message);
}
}
}
The only problem with this is if the file upload to S3 Fails, you might still get the URL, and if you're saving it to your database - you'll save the URL to the database but the object wont exist in the S3 bucket, so it won't lead anywhere. If you implement dependency injection - this shouldn't be an issue (dependency injection not implemented in this example)

AWS S3 transferutility UploadAsync succeeds but does not create file

I got to upload file & streams to S3 periodically and the file size usually under 50MB. What am seeing is TransferUtility.UploadAsync method returns success but the file was never created. This occurs sporadically. Below is the code we use. Any suggestions around this?
public TransferUtilityUploadRequest CreateRequest(string bucket, string path)
{
var request = new TransferUtilityUploadRequest
{
BucketName = bucket,
StorageClass = S3StorageClass.Standard,
Key = path,
AutoCloseStream = true,
CannedACL = S3CannedACL.AuthenticatedRead,
ServerSideEncryptionMethod = ServerSideEncryptionMethod.AES256
};
return request;
}
public async Task CreateS3ObjectFromStreamAsync(MemoryStream memeoryStream, string bucket, string filePath)
{
var ftu = new TransferUtility(client);
var request = CreateRequest(bucket, filePath);
request.InputStream = memeoryStream;
await ftu.UploadAsync(request);
}
public async Task CreateS3ObjectFromFileAsync(string sourceFilePath, string bucketName, string destpath)
{
var request = CreateRequest(bucketName, destpath);
request.FilePath = sourceFilePath;
var ftu = new TransferUtility(client);
await ftu.UploadAsync(request);
}

Please use Upload or UploadAsync.Wait. It worked for me!

Signature does not Match S3

I am getting below error -
The request signature we calculated does not match the signature you provided. Check your Secret Access Key and signing method. For more information, see REST Authentication and SOAP Authentication for details.'
The remote server returned an error: (403) Forbidden.
I have checked my secret key there are no spaces in it.
My key looks like -
TestArea/Destination/SUP000011/ATM-1B4L2KQ0ZE0-0001/SoS_Update_2018_06_04_pram.pptx
Code -
public static Stream GetObjectStream(string keyName)
{
GetObjectRequest request = new GetObjectRequest
{
BucketName = bucketName,
Key = keyName
};
using (AmazonS3Client StorageClient = GetClient())
//This line gives error (getting response)
using (GetObjectResponse response = StorageClient.GetObject(request))
using (Stream responseStream = response.ResponseStream)
{
return responseStream;
}
}
public static Amazon.S3.AmazonS3Client GetClient()
{
AmazonS3Config Config = new AmazonS3Config();
AmazonS3Client StorageClient;
Config.RegionEndpoint = null;
Config.ServiceURL = ConfigurationManager.NGDMSobjECSEndPoint;
Config.AllowAutoRedirect = true;
Config.ForcePathStyle = true;
StorageClient = new AmazonS3Client(ConfigurationManager.NGDMSobjECSUser, ConfigurationManager.NGDMSobjECSKey, Config);
return StorageClient;
}

There was problem with my Secret Key, i created new Secret Key and it worked.

Accessing a bucket at the google S3 api

I am trying to read a bucket at storage.googleapis.com, using the Amazon Web Services .Net SDK in C#.
Can anyone provide a working example of a S3 endpoint Config setup for google, just using the Auth. key/secret pair and a bucket name? Or using any other method to get this working?
According to this tutorial this should be a simple matter, but I get all sorts of exceptions when trying to follow the instructions given. Here is an extract of my current attempt - which throws a TrustFailure exception:
The remote certificate is invalid.
AmazonS3Config conf = new AmazonS3Config();
// Set regionEndpoint to null, or else the serviceURL will be ignored
conf.RegionEndpoint = null;
conf.ServiceURL = "https://s3.storage.googleapis.com";
conf.UseHttp = false;
conf.AuthenticationRegion = null;
conf.UseAccelerateEndpoint = false;
conf.UseDualstackEndpoint = false;
AWSCredentials cred = new BasicAWSCredentials("GOOG3LFXXXXXXXXXXXXX", "BQ6VeMXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
IAmazonS3 client = new AmazonS3Client(cred, conf);
GetBucketVersioningRequest request = new GetBucketVersioningRequest { BucketName = "hisbucket" };
GetBucketVersioningResponse response = client.GetBucketVersioning(request);

I finally got the .NET SDK to upload to Google Cloud Storage with:
AWSConfigsS3.UseSignatureVersion4 = false;
AmazonS3Config config = new AmazonS3Config();
config.ServiceURL = "https://storage.googleapis.com";
config.SignatureVersion = "2";
AmazonS3Client client = new AmazonS3Client(accessKey, secretKey, config);
var transferUtilityConfig = new TransferUtilityConfig
{
ConcurrentServiceRequests = 1,
MinSizeBeforePartUpload = 6291456000,
};
var fileTransferUtilityRequest = new TransferUtilityUploadRequest
{
BucketName = bucketName,
FilePath = filePath,
PartSize = 6291456000,
Key = keyName,
};
TransferUtility fileTransferUtility = new TransferUtility(client, transferUtilityConfig);
fileTransferUtility.Upload(fileTransferUtilityRequest);
fileTransferUtility.Dispose();

You need a Amazon S3 service URL, an access key id, a secret access key id and the bucket name.
var s3Config = new AmazonS3Config
{
ServiceURL = Constants.AmazonS3ServiceUrl,
RegionEndpoint = Amazon.RegionEndpoint.EUWest1
};
string accessKeyId = Constants.AmazonAccessKeyId;
string secretAccessKey = Constants.AmazonSecretAccessKey;
var config = new AwsS3Config(){AmazonS3BucketName = Constants.AmazonS3BucketName};
var client = new AmazonS3Client(accessKeyId, secretAccessKey, s3Config);
Then, you should be able to make calls to the amazon client:
var request = new GetObjectRequest
{
BucketName = _bucketName,
Key = entity.Path
};
var response = _client.GetObjectAsync(request).Result;
The code above works on an S3 account, not particularly storage.googleapis.com, which is your case. Anyway, I hope this helps and answers your question.