So currently my code uploads an image to S3 but what I want it to now do is to get the URL of the image that has been uploaded, so this URL can be stored in the DB and used later.
I know it's possible, as it was shown in this question here: s3 file upload does not return response (but that is JavaScript and I'm struggling to convert it to c#)
This is my code, it works perfectly, I just need to get the URL of the uploaded object + is there a way to make the object public by default. I tried to console write the response but that was no help
public class AmazonS3Uploader
{
private string bucketName = "cartalkio-image-storage-dev";
private string keyName = DateTime.Now.ToString() + ".png";
public async void UploadFile()
{
byte[] bytes = System.Convert.FromBase64String("iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg==");
Stream stream = new MemoryStream(bytes);
var myAwsAccesskey = "*************";
var myAwsSecret = "**************************";
var client = new AmazonS3Client(myAwsAccesskey, myAwsSecret, Amazon.RegionEndpoint.EUWest2);
try
{
PutObjectRequest putRequest = new PutObjectRequest
{
BucketName = bucketName,
Key = keyName,
ContentType = "image/png",
InputStream = stream
};
PutObjectResponse response = await client.PutObjectAsync(putRequest);
// Console.Write(response);
}
catch (AmazonS3Exception amazonS3Exception)
{
if (amazonS3Exception.ErrorCode != null &&
(amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId")
||
amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
{
throw new Exception("Check the provided AWS Credentials.");
}
else
{
throw new Exception("Error occurred: " + amazonS3Exception.Message);
}
}
}
I've searched the documentation and various websites but this was all I found () - I guess there just isn't a URL that is returned. What I've done is changed my code around a bit because you can always predict what the object URL will be based on the name of the object you're uploading. i.e if you're uploading an image called 'test.png', the URL will be this:
https://[Your-Bucket-Name].s3.[S3-Region].amazonaws.com/[test.png]
I tried dependency injection but AWS didn't like that so I've changed my code to this:
(in this example I'm receiving a base64 string, turning it into a BYTE and then into a SYSTEM.IO.Stream)
This is the request I'm sending up:
{ "image":"iVBORw0KGgoAAAANSUhEUgAAAAUAAAAFCAYAAACNbyblAAAAHElEQVQI12P4//8/w38GIAXDIBKE0DHxgljNBAAO9TXL0Y4OHwAAAABJRU5ErkJggg=="
}
Controller:
public async Task<ActionResult> Index(string image)
{
AmazonS3Uploader amazonS3 = new AmazonS3Uploader();
// This bit creates a random string that will be used as part of the URL
var chars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
var stringChars = new char[8];
var random = new Random();
for (int i = 0; i < stringChars.Length; i++)
{
stringChars[i] = chars[random.Next(chars.Length)];
}
var finalString = new String(stringChars);
// This bit adds the '.png' as its an image
var keyName = finalString + ".png";
// This uploads the file to S3, passing through the keyname (which is the end of the URL) and the image string
amazonS3.UploadFile(keyName, image);
// This is what the final URL of the object will be, so you can use this variable later or save it in your database
var itemUrl = "https://[your-bucket-name].s3.[S3-Region].amazonaws.com/" + keyName;
return Ok();
}
AmazonS3Uploader.cs
private string bucketName = "your-bucket-name";
public async void UploadFile(string keyName, string image)
{
byte[] bytes = System.Convert.FromBase64String(image);
Stream stream = new MemoryStream(bytes);
var myAwsAccesskey = "*************";
var myAwsSecret = "**************************";
var client = new AmazonS3Client(myAwsAccesskey, myAwsSecret, Amazon.RegionEndpoint.[S3-Region]);
try
{
PutObjectRequest putRequest = new PutObjectRequest
{
BucketName = bucketName,
Key = keyName,
ContentType = "image/png",
InputStream = stream
};
PutObjectResponse response = await client.PutObjectAsync(putRequest);
}
catch (AmazonS3Exception amazonS3Exception)
{
if (amazonS3Exception.ErrorCode != null &&
(amazonS3Exception.ErrorCode.Equals("InvalidAccessKeyId")
||
amazonS3Exception.ErrorCode.Equals("InvalidSecurity")))
{
throw new Exception("Check the provided AWS Credentials.");
}
else
{
throw new Exception("Error occurred: " + amazonS3Exception.Message);
}
}
}
The only problem with this is if the file upload to S3 Fails, you might still get the URL, and if you're saving it to your database - you'll save the URL to the database but the object wont exist in the S3 bucket, so it won't lead anywhere. If you implement dependency injection - this shouldn't be an issue (dependency injection not implemented in this example)
Related
I am retrieving mp4 clips via a REST API for a security camera. The clips are 2-5 seconds long and are almost always less than 600 KB.
The info for the clips is populated into a DataGridView, and that info includes the URI for the REST API. I get the byte[] array via the API, write it to a file, and then play it through windows media player. About 20% of the clips are corrupt, but they are always exactly 919 bytes when they are. I can't figure out why this is happening.
I know the URI isn't invalid, because Blink.GetClipAsync(uri, user.getToken()); would throw an exception.
Am I doing something wrong that could cause this?
private async void ClipView_CellContentClick(object sender, DataGridViewCellEventArgs e)
{
String uri = "";
int rowIndex = e.RowIndex;
try
{
uri = ClipView.Rows[rowIndex].Tag.ToString();
}
catch (Exception ex)
{
MessageBox.Show(ex.ToString());
}
String fileName = Path.GetFileName(uri);
byte[] mp4 = await Blink.GetClipAsync(uri, user.getToken());
String path = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments);
String pathString = System.IO.Path.Combine(path, "Blink Desktop");
String filePathString = System.IO.Path.Combine(pathString, fileName);
if (!System.IO.Directory.Exists(pathString))
System.IO.Directory.CreateDirectory(pathString);
if (!System.IO.File.Exists(filePathString))
{
File.WriteAllBytes(filePathString, mp4);
}
Properties.Settings.Default.currentClipSelection = filePathString;
var ClipPlayer = new Clip_Player();
ClipPlayer.Show();
}
public async Task<byte[]> GetClipAsync(String url, String token)
{
byte[] response;
using (var httpClient = new HttpClient())
{
using (var request = new HttpRequestMessage(new HttpMethod("GET"), Properties.Settings.Default.RegionAPI + url))
{
request.Headers.TryAddWithoutValidation("token-auth", token);
HttpContent content = httpClient.SendAsync(request).Result.Content;
response = await content.ReadAsByteArrayAsync();
}
}
return response;
}
I got to upload file & streams to S3 periodically and the file size usually under 50MB. What am seeing is TransferUtility.UploadAsync method returns success but the file was never created. This occurs sporadically. Below is the code we use. Any suggestions around this?
public TransferUtilityUploadRequest CreateRequest(string bucket, string path)
{
var request = new TransferUtilityUploadRequest
{
BucketName = bucket,
StorageClass = S3StorageClass.Standard,
Key = path,
AutoCloseStream = true,
CannedACL = S3CannedACL.AuthenticatedRead,
ServerSideEncryptionMethod = ServerSideEncryptionMethod.AES256
};
return request;
}
public async Task CreateS3ObjectFromStreamAsync(MemoryStream memeoryStream, string bucket, string filePath)
{
var ftu = new TransferUtility(client);
var request = CreateRequest(bucket, filePath);
request.InputStream = memeoryStream;
await ftu.UploadAsync(request);
}
public async Task CreateS3ObjectFromFileAsync(string sourceFilePath, string bucketName, string destpath)
{
var request = CreateRequest(bucketName, destpath);
request.FilePath = sourceFilePath;
var ftu = new TransferUtility(client);
await ftu.UploadAsync(request);
}
Please use Upload or UploadAsync.Wait. It worked for me!
Im trying to download object from S3 bucket facing below issue
The Security token included in the request is Invalid .
Please check and correct where is the mistake.
Below is my code
1. Get Temporary credentails:
main()
{
string path = "http://XXX.XXX.XXX./latest/meta-data/iam/security-credentials/EC2_WLMA_Permissions";
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(path);
request.Method = "GET";
request.ContentType = "application/json";
HttpWebResponse response = request.GetResponse() as HttpWebResponse;
string result = string.Empty;
using (StreamReader reader = new StreamReader(response.GetResponseStream()))
{
result = reader.ReadToEnd();
dynamic metaData = JsonConvert.DeserializeObject(result);
_awsAccessKeyId = metaData.AccessKeyId;
_awsSecretAccessKey = metaData.SecretAccessKey;
}
}
Create SessionAWSCredentials instance:
SessionAWSCredentials tempCredentials =
GetTemporaryCredentials(_awsAccessKeyId, _awsSecretAccessKey);
//GetTemporaryCredentials method:
private static SessionAWSCredentials GetTemporaryCredentials(
string accessKeyId, string secretAccessKeyId)
{
AmazonSecurityTokenServiceClient stsClient =
new AmazonSecurityTokenServiceClient(accessKeyId,
secretAccessKeyId);
Console.WriteLine(stsClient.ToString());
GetSessionTokenRequest getSessionTokenRequest =
new GetSessionTokenRequest();
getSessionTokenRequest.DurationSeconds = 7200; // seconds
GetSessionTokenResponse sessionTokenResponse =
stsClient.GetSessionToken(getSessionTokenRequest);
Console.WriteLine(sessionTokenResponse.ToString());
Credentials credentials = sessionTokenResponse.Credentials;
Console.WriteLine(credentials.ToString());
SessionAWSCredentials sessionCredentials =
new SessionAWSCredentials(credentials.AccessKeyId,
credentials.SecretAccessKey,
credentials.SessionToken);
return sessionCredentials;
}
Get files from S3 using AmazonS3Client:
using (IAmazonS3 client = new AmazonS3Client(tempCredentials,RegionEndpoint.USEast1))
{
GetObjectRequest request = new GetObjectRequest();
request.BucketName = "bucketName" + #"/" + "foldername";
request.Key = "Terms.docx";
GetObjectResponse response = client.GetObject(request);
response.WriteResponseStreamToFile("C:\\MyFile.docx");
}
We do something a little simpler for interfacing with S3 (downloads and uploads)
It looks like you went the more complex approach. You should try just using the TransferUtility instead:
TransferUtility fileTransferUtility =
new TransferUtility(
new AmazonS3Client("ACCESS-KEY-ID", "SECRET-ACCESS-KEY", Amazon.RegionEndpoint.CACentral1));
// Note the 'fileName' is the 'key' of the object in S3 (which is usually just the file name)
fileTransferUtility.Download(filePath, "my-bucket-name", fileName);
NOTE: TransferUtility.Download() returns void because it downloads the file to the path specified in the filePath argument. This may be a little different than what you were expecting but you can still open a FileStream to that path afterwards and manipulate the file all you want. For example:
using (FileStream fileDownloaded = new FileStream(filePath, FileMode.Open, FileAccess.Read))
{
// Do stuff with our newly downloaded file
}
Bucketname, Accesskey and secretkey, I took from web config. You could type manually.
public void DownloadObject(string imagename)
{
RegionEndpoint bucketRegion = RegionEndpoint.USEast1;
IAmazonS3 client = new AmazonS3Client(bucketRegion);
string accessKey = System.Configuration.ConfigurationManager.AppSettings["AWSAccessKey"];
string secretKey = System.Configuration.ConfigurationManager.AppSettings["AWSSecretKey"];
AmazonS3Client s3Client = new AmazonS3Client(new BasicAWSCredentials(accessKey, secretKey), Amazon.RegionEndpoint.USEast1);
string objectKey = "EMR" + "/" + imagename;
//EMR is folder name of the image inside the bucket
GetObjectRequest request = new GetObjectRequest();
request.BucketName = System.Configuration.ConfigurationManager.AppSettings["bucketname"];
request.Key = objectKey;
GetObjectResponse response = s3Client.GetObject(request);
response.WriteResponseStreamToFile("D:\\Test\\"+ imagename);
}
//> D:\Test\ is local file path.
Following is how I download it. I need to download only .zip files in this case. Restricting to only required file types (.zip in my case), helped me to avoid errors related to (416) Requested Range Not Satisfiable
public static class MyAWS_S3_Helper
{
static string _S3Key = System.Configuration.ConfigurationManager.ConnectionStrings["S3BucketKey"].ConnectionString;
static string _S3SecretKey = System.Configuration.ConfigurationManager.ConnectionStrings["S3BucketSecretKey"].ConnectionString;
public static void S3Download(string bucketName, string _ObjectKey, string downloadPath)
{
IAmazonS3 _client = new AmazonS3Client(_S3Key, _S3SecretKey, Amazon.RegionEndpoint.USEast1);
TransferUtility fileTransferUtility = new TransferUtility(_client);
fileTransferUtility.Download(downloadPath + "\\" + _ObjectKey, bucketName, _ObjectKey);
_client.Dispose();
}
public static async Task AsyncDownload(string bucketName, string downloadPath, string requiredSunFolder)
{
var bucketRegion = RegionEndpoint.USEast1; //Change it
var credentials = new BasicAWSCredentials(_S3Key, _S3SecretKey);
var client = new AmazonS3Client(credentials, bucketRegion);
var request = new ListObjectsV2Request
{
BucketName = bucketName,
MaxKeys = 1000
};
var response = await client.ListObjectsV2Async(request);
var utility = new TransferUtility(client);
foreach (var obj in response.S3Objects)
{
string currentKey = obj.Key;
double sizeCheck = Convert.ToDouble(obj.Size);
int fileNameLength = currentKey.Length;
Console.WriteLine(currentKey + "---" + fileNameLength.ToString());
if (currentKey.Contains(requiredSunFolder))
{
if (currentKey.Contains(".zip")) //This helps to avoid errors related to (416) Requested Range Not Satisfiable
{
try
{
S3Download(bucketName, currentKey, downloadPath);
}
catch (Exception exTest)
{
string messageTest = currentKey + "-" + exTest;
}
}
}
}
}
}
Here is how it is called
static void Main(string[] args)
{
string downloadPath = #"C:\SourceFiles\TestDownload";
Task awsTask = MyAWS_S3_Helper.AsyncDownload("my-files", downloadPath, "mysubfolder");
awsTask.Wait();
}
Here is what I have done to download the files from S3 bucket,
var AwsImportFilePathParcel = "TEST/TEMP"
IAmazonS3 client = new AmazonS3Client(AwsAccessKey,AwsSecretKey);
S3DirectoryInfo info = new S3DirectoryInfo(client, S3BucketName, AwsImportFilePathParcel);
S3FileInfo[] s3Files = info.GetFiles(pattenForParcel);
now in s3Files, you have all the files which are on provided location, using for each you can save all files in to your system
foreach (var fileInfo in s3Files)
{
var localPath = Path.Combine("C:\TEST\", fileInfo.Name);
var file = fileInfo.CopyToLocal(localPath);
}
I am struggling with being able to create a file with its data based on the byte array returned from the WebAPI. The following is my code for making the call to the web api
using (var http = new WebClient())
{
string url = string.Format("{0}api/FileUpload/FileServe?FileID=" + fileID, webApiUrl);
http.Headers[HttpRequestHeader.ContentType] = "application/octet-stream";
http.Headers[HttpRequestHeader.Authorization] = "Bearer " + authCookie.Value;
http.DownloadDataCompleted += Http_DownloadDataCompleted;
byte[] json = await http.DownloadDataTaskAsync(url);
}
The api code is
[HttpGet]
[Route("FileServe")]
[Authorize(Roles = "Admin,SuperAdmin,Contractor")]
public async Task<HttpResponseMessage> GetFile(int FileID)
{
using (var repo = new MBHDocRepository())
{
var file = await repo.GetSpecificFile(FileID);
if (file == null)
{
throw new HttpResponseException(HttpStatusCode.BadRequest);
}
var stream = File.Open(file.PathLocator, FileMode.Open);
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContent(stream);
response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(file.FileType);
return response;
}
}
I receive a byte array as a response however am unable to create the corresponding file from that byte array. I have no idea how to convert the byte array into the relevant file type (such as jpg, or pdf based on file type in the web api). any help will be appreciated.
Alright so there are a few ways of solving your problem firstly, on the server side of things you can either simply send the content type and leave it at that or you can also send the complete filename which helps you even further.
I have removed the code that is specific to your stuff with basic test code, please just ignore that stuff and use it in terms of your code.
Some design notes here:
[HttpGet]
[Route("FileServe")]
[Authorize(Roles = "Admin,SuperAdmin,Contractor")]
public async Task<HttpResponseMessage> GetFileAsync(int FileID) //<-- If your method returns Task have it be named with Async in it
{
using (var repo = new MBHDocRepository())
{
var file = await repo.GetSpecificFile(FileID);
if (file == null)
{
throw new HttpResponseException(HttpStatusCode.BadRequest);
}
var stream = File.Open(file.PathLocator, FileMode.Open);
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContent(stream);
response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue(file.FileType);
response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment") { FileName=Path.GetFileName(file.PathLocator)};
return response;
}
}
Your client side code has two options here:
static void Main(string[] args)
{
using (var http = new WebClient())
{
string url = string.Format("{0}api/FileUpload/FileServe?FileID={1}",webApiUrl, fileId);
http.Headers[HttpRequestHeader.ContentType] = "application/octet-stream";
http.Headers[HttpRequestHeader.Authorization] = "Bearer " + authCookie.Value;
var response = http.OpenRead(url);
var fs = new FileStream(String.Format(#"C:\Users\Bailey Miller\Downloads\{0}", GetName(http.ResponseHeaders)), FileMode.Create);
response.CopyTo(fs); <-- how to move the stream to the actual file, this is not perfect and there are a lot of better examples
fs.Flush();
fs.Close();
}
}
private static object GetName(WebHeaderCollection responseHeaders)
{
var c_type = responseHeaders.GetValues("Content-Type"); //<-- do a switch on this and return a really weird file name with the correct extension for the mime type.
var cd = responseHeaders.GetValues("Content-Disposition")[0].Replace("\"", ""); <-- this gets the attachment type and filename param, also removes illegal character " from filename if present
return cd.Substring(cd.IndexOf("=")+1); <-- extracts the file name
}
I am writing a Web API service where I want to accept a file (image) and a serialized object (JSON) that contains key information about the image. Not having issues with the image part but when I add string content containing the deserialized object I am having issues in trying to determine which is which and act accordingly.
The client code looks like:
HttpClient client = new HttpClient();
MultipartFormDataContent content = new MultipartFormDataContent();
content.Add(new StreamContent(File.Open("c:\\MyImages\\Image00.jpg", FileMode.Open)), "image_file", "Image00.jpg");
ImageKeys ik = new ImageKeys { ImageId = "12345", Timestamp = DateTime.Now.ToString() };
JavaScriptSerializer js = new JavaScriptSerializer();
if (ik != null)
{
content.Add(new StringContent(js.Serialize(ik), Encoding.UTF8, "application/json"), "image_keys");
}
string uri = "http://localhost/MyAPI/api/MyQuery/TransferFile";
var request = new HttpRequestMessage()
{
RequestUri = new Uri(uri),
Method = HttpMethod.Post
};
request.Content = content;
string responseStr = "";
try
{
HttpResponseMessage result = client.SendAsync(request).Result;
string resultContent = string.Format("{0}:{1}", result.StatusCode, result.ReasonPhrase);
//
// Handle the response
//
responseStr = resultContent;
}
catch (Exception ex)
{
responseStr = ex.Message;
}
listBox1.Items.Add(responseStr);
So I include the image file first followed by a serialized object as StringContent. On the server side I am using the following code to parse the message.
HttpRequestMessage request = this.Request;
HttpResponseMessage ret = new HttpResponseMessage();
//
// Verify that this is an HTML Form file upload request
//
if (!request.Content.IsMimeMultipartContent())
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
string root = "c:\\tmp\\uploads";
if (!Directory.Exists(root))
{
Directory.CreateDirectory(root);
}
//
// Create a stream provider for setting up output streams that saves the output under c:\tmp\uploads
// If you want full control over how the stream is saved then derive from MultipartFormDataStreamProvider
// and override what you need.
//
MultipartFormDataStreamProvider streamProvider = new MultipartFormDataStreamProvider(root);
try
{
await request.Content.ReadAsMultipartAsync(streamProvider);
foreach (var file in streamProvider.Contents)
{
if (file.Headers.ContentDisposition.Name == "image_file")
{
FileInfo finfo = new FileInfo(streamProvider.FileData.First().LocalFileName);
string destFile = Path.Combine(root, streamProvider.FileData.First().Headers.ContentDisposition.FileName.Replace("\"", ""));
//
// File.Move cannot deal with duplicate files
// Ensure that the target does not exist.
//
if (File.Exists(destFile))
{
File.Delete(destFile);
}
File.Move(finfo.FullName, destFile);
}
else if (file.Headers.ContentDisposition.Name == "image_keys")
{
// deserialize key class
string str = file.ReadAsStringAsync().Result;
JavaScriptSerializer js = new JavaScriptSerializer();
ImageKeys ik = js.Deserialize<ImageKeys>(str);
}
}
ret.StatusCode = HttpStatusCode.OK;
ret.Content = new StringContent("File uploaded.");
}
catch (Exception ex)
{
ret.StatusCode = HttpStatusCode.UnsupportedMediaType;
ret.Content = new StringContent("File upload failed.");
}
return ret;
The foreach loop tries to process each item in the multipart content as a file but I want to treat the various content types separately but it is not clear to me how they are delineated.
Thanks
You can cast Content to MultipartFormDataContent and iterate thru it. Based on content type you can read it as a file or string. Example for string content type:
var dataContents = request.Content as MultipartFormDataContent;
foreach (var dataContent in dataContents)
{
var name = dataContent.Headers.ContentDisposition.Name;
var value = dataContent.ReadAsStringAsync().Result;
...
}