Amazon S3 upload with public permissions

Amazon S3 upload with public permissions - c#

I'm using the Amazon C# SDK and trying to upload a file, but by default it has restricted permissions. I would like to make it publicly available, but I can't seem to find out how to do it as part of the upload.
My bucket is public, but when I upload a new file using the code below, the file I upload is not public.
Has anyone had to do this before?
public class S3Uploader
{
private string awsAccessKeyId;
private string awsSecretAccessKey;
private string bucketName;
private Amazon.S3.Transfer.TransferUtility transferUtility;
public S3Uploader(string bucketName)
{
this.bucketName = bucketName;
this.transferUtility = new Amazon.S3.Transfer.TransferUtility("XXXXXXXXXXXXXXXXXX", "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
}
public void UploadFile(string filePath, string toPath)
{
AsyncCallback callback = new AsyncCallback(uploadComplete);
transferUtility.BeginUpload(filePath, bucketName, toPath, callback, null);
}
private void uploadComplete(IAsyncResult result)
{
var x = result;
}
}

The solution by Tahbaza is correct, but it doesn't work in the later versions of AWS SDK (2.1+)...
Consider using the following for 2.1+ versions of SDK:
private void UploadFileToS3(string filePath)
{
var awsAccessKey = ConfigurationManager.AppSettings["AWSAccessKey"];
var awsSecretKey = ConfigurationManager.AppSettings["AWSSecretKey"];
var existingBucketName = ConfigurationManager.AppSettings["AWSBucketName"];
var client = Amazon.AWSClientFactory.CreateAmazonS3Client(awsAccessKey, awsSecretKey,RegionEndpoint.USEast1);
var uploadRequest = new TransferUtilityUploadRequest
{
FilePath = filePath,
BucketName = existingBucketName,
CannedACL = S3CannedACL.PublicRead
};
var fileTransferUtility = new TransferUtility(client);
fileTransferUtility.Upload(uploadRequest);
}

Found it, need to use a TransferUtilityUploadRequest:
public class S3Uploader
{
private string awsAccessKeyId;
private string awsSecretAccessKey;
private string bucketName;
private Amazon.S3.Transfer.TransferUtility transferUtility;
public S3Uploader(string bucketName)
{
this.bucketName = bucketName;
this.transferUtility = new Amazon.S3.Transfer.TransferUtility("XXXXXXXXXXXXXXXXXX", "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX");
}
public void UploadFile(string filePath, string toPath)
{
AsyncCallback callback = new AsyncCallback(uploadComplete);
var uploadRequest = new TransferUtilityUploadRequest();
uploadRequest.FilePath = filePath;
uploadRequest.BucketName = "my_s3_bucket";
uploadRequest.Key = toPath;
uploadRequest.AddHeader("x-amz-acl", "public-read");
transferUtility.BeginUpload(uploadRequest, callback, null);
}
private void uploadComplete(IAsyncResult result)
{
var x = result;
}
}

I have done this before, and in C#, but not with your library (so this may be of limited help).
The idea is to send an ACL header along with the file.
This is one way to do it that should point you in the right direction.
Here's also a link to some relevant AWS docs.
private const string AWS_ACL_HEADER = "x-amz-acl";
private static string ToACLString(S3ACLType acl) {
switch (acl) {
case S3ACLType.AuthenticatedRead:
return "authenticated-read";
case S3ACLType.BucketOwnerFullControl:
return "bucket-owner-full-control";
case S3ACLType.BucketOwnerRead:
return "bucket-owner-read";
case S3ACLType.Private:
return "private";
case S3ACLType.PublicRead:
return "public-read";
case S3ACLType.PublicReadWrite:
return "public-read-write";
default: return "";
}
}
public void Put(string bucketName, string id, byte[] bytes, string contentType, S3ACLType acl) {
string uri = String.Format("https://{0}/{1}", BASE_SERVICE_URL, bucketName.ToLower());
DreamMessage msg = DreamMessage.Ok(MimeType.BINARY, bytes);
msg.Headers[DreamHeaders.CONTENT_TYPE] = contentType;
msg.Headers[DreamHeaders.EXPECT] = "100-continue";
msg.Headers[AWS_ACL_HEADER] = ToACLString(acl);
Plug s3Client = Plug.New(uri).WithPreHandler(S3AuthenticationHeader);
s3Client.At(id).Put(msg);
}

Consider CannedACL approach, shown in the follwing snippet:
string filePath = #"Path\Desert.jpg";
Stream input = new FileStream(filePath, FileMode.Open);
PutObjectRequest request2 = new PutObjectRequest();
request2.WithMetaData("title", "the title")
.WithBucketName(bucketName)
.WithCannedACL(S3CannedACL.PublicRead)
.WithKey(keyName)
.WithInputStream(input);

You can use the property CannedACL of Amazon.S3.Model.PutObjectRequest to upload files to S3 as shown below,
var putObjectRequest = new Amazon.S3.Model.PutObjectRequest
{
BucketName = bucketName,
Key = key,
CannedACL = S3CannedACL.PublicReadWrite
};

Related

AmazonSQSClient not refreshing AWSCredentials when Credentials File is updated

When my AWS Credentials File (see docs) is updated by an external process the AmazonSQSClient doesn't re-read it, SendMessageAsync fails with a security/token error.
We use a custom powershell script to refresh the local AWS cred's file periodically. The script works fine, the file is refreshed prior to the credentials expiring on AWS. However, if my app is running when the file is refreshed the new credentials are not re-read from the file, the "client" will show that the previous credentials are still in use.
The AWS docs list several AWSCredential providers but none of them seem to be the correct choice...I think..
Restarting the app works, the new credentials are read correctly and messages are sent until the next time the cred's file is updated.
using (var client = new AmazonSQSClient(Amazon.RegionEndpoint.EUWest1))
{
return client.SendMessageAsync(request);
}

I don't think there is a way for a running app to pick up the default credentials being refreshed in credentials file. There is a solution for Node.js loading credentials from a JSON file. You can create a similar solution in C#. You can also run a local DB to store credentials so whenever credentials file is updated DB table or JSON file is also updated. You will need to use access key and secret key in your SQS client constructor as opposed to using default credentials.
// Load these from JSON file or DB.
var accessKey = "";
var secretKey = "";
using (var client = new AmazonSQSClient(accessKey, secretKey, Amazon.RegionEndpoint.EUWest1))
{
return client.SendMessageAsync(request);
}

The following works "ok" but I've only tested it with one profile and the file watcher is not as timely as you'd like so I'd recommend you wrap your usage inside a Retry mechanism.
// Usage..
var credentials = new AwsCredentialsFile();
using (var client = new AmazonSQSClient(credentials, Amazon.RegionEndpoint.EUWest1))
{
return client.SendMessageAsync(request);
}
public class AwsCredentialsFile : AWSCredentials
{
// https://docs.aws.amazon.com/sdk-for-net/v2/developer-guide/net-dg-config-creds.html#creds-file
private const string DefaultProfileName = "default";
private static ConcurrentDictionary<string, ImmutableCredentials> _credentials = new ConcurrentDictionary<string, ImmutableCredentials>(StringComparer.OrdinalIgnoreCase);
private static FileSystemWatcher _watcher = BuildFileSystemWatcher();
private readonly System.Text.Encoding _encoding;
private readonly string _profileName;
public AwsCredentialsFile()
: this(AwsCredentialsFile.DefaultProfileName, System.Text.Encoding.UTF8)
{
}
public AwsCredentialsFile(string profileName)
: this(profileName, System.Text.Encoding.UTF8)
{
}
public AwsCredentialsFile(string profileName, System.Text.Encoding encoding)
{
_profileName = profileName;
_encoding = encoding;
}
private static FileSystemWatcher BuildFileSystemWatcher()
{
var watcher = new FileSystemWatcher
{
Path = Path.GetDirectoryName(GetDefaultCredentialsFilePath()),
NotifyFilter = NotifyFilters.LastWrite,
Filter = "credentials"
};
watcher.Changed += (object source, FileSystemEventArgs e) => { _credentials?.Clear(); };
watcher.EnableRaisingEvents = true;
return watcher;
}
public static string GetDefaultCredentialsFilePath()
{
return System.Environment.ExpandEnvironmentVariables(#"C:\Users\%USERNAME%\.aws\credentials");
}
public static (string AccessKey, string SecretAccessKey, string Token) ReadCredentialsFromFile(string profileName, System.Text.Encoding encoding)
{
var profile = $"[{profileName}]";
string awsAccessKeyId = null;
string awsSecretAccessKey = null;
string token = null;
var lines = File.ReadAllLines(GetDefaultCredentialsFilePath(), encoding);
for (int i = 0; i < lines.Length; i++)
{
var text = lines[i];
if (text.Equals(profile, StringComparison.OrdinalIgnoreCase))
{
awsAccessKeyId = lines[i + 1].Replace("aws_access_key_id = ", string.Empty);
awsSecretAccessKey = lines[i + 2].Replace("aws_secret_access_key = ", string.Empty);
if (lines.Length >= i + 3)
{
token = lines[i + 3].Replace("aws_session_token = ", string.Empty);
}
break;
}
}
var result = (AccessKey: awsAccessKeyId, SecretAccessKey: awsSecretAccessKey, Token: token);
return result;
}
public override ImmutableCredentials GetCredentials()
{
if (_credentials.TryGetValue(_profileName, out ImmutableCredentials value))
{
return value;
}
else
{
var (AccessKey, SecretAccessKey, Token) = ReadCredentialsFromFile(_profileName, _encoding);
var credentials = new ImmutableCredentials(AccessKey, SecretAccessKey, Token);
_credentials.TryAdd(_profileName, credentials);
return credentials;
}
}
}

Get Content-Disposition parameters

How do I get Content-Disposition parameters I returned from WebAPI controller using WebClient?
WebApi Controller
[Route("api/mycontroller/GetFile/{fileId}")]
public HttpResponseMessage GetFile(int fileId)
{
try
{
var file = GetSomeFile(fileId)
HttpResponseMessage response = new HttpResponseMessage(HttpStatusCode.OK);
response.Content = new StreamContent(new MemoryStream(file));
response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment");
response.Content.Headers.ContentDisposition.FileName = file.FileOriginalName;
/********* Parameter *************/
response.Content.Headers.ContentDisposition.Parameters.Add(new NameValueHeaderValue("MyParameter", "MyValue"));
return response;
}
catch(Exception ex)
{
return Request.CreateErrorResponse(HttpStatusCode.InternalServerError, ex);
}
}
Client
void DownloadFile()
{
WebClient wc = new WebClient();
wc.DownloadDataCompleted += wc_DownloadDataCompleted;
wc.DownloadDataAsync(new Uri("api/mycontroller/GetFile/18"));
}
void wc_DownloadDataCompleted(object sender, DownloadDataCompletedEventArgs e)
{
WebClient wc=sender as WebClient;
// Try to extract the filename from the Content-Disposition header
if (!String.IsNullOrEmpty(wc.ResponseHeaders["Content-Disposition"]))
{
string fileName = wc.ResponseHeaders["Content-Disposition"].Substring(wc.ResponseHeaders["Content-Disposition"].IndexOf("filename=") + 10).Replace("\"", ""); //FileName ok
/****** How do I get "MyParameter"? **********/
}
var data = e.Result; //File OK
}
I'm returning a file from WebApi controller, I'm attaching the file name in the response content headers, but also I'd like to return an aditional value.
In the client I'm able to get the filename, but how do I get the aditional parameter?

If you are working with .NET 4.5 or later, consider using the System.Net.Mime.ContentDisposition class:
string cpString = wc.ResponseHeaders["Content-Disposition"];
ContentDisposition contentDisposition = new ContentDisposition(cpString);
string filename = contentDisposition.FileName;
StringDictionary parameters = contentDisposition.Parameters;
// You have got parameters now
Edit:
otherwise, you need to parse Content-Disposition header according to it's specification.
Here is a simple class that performs the parsing, close to the specification:
class ContentDisposition {
private static readonly Regex regex = new Regex(
"^([^;]+);(?:\\s*([^=]+)=((?<q>\"?)[^\"]*\\k<q>);?)*$",
RegexOptions.Compiled
);
private readonly string fileName;
private readonly StringDictionary parameters;
private readonly string type;
public ContentDisposition(string s) {
if (string.IsNullOrEmpty(s)) {
throw new ArgumentNullException("s");
}
Match match = regex.Match(s);
if (!match.Success) {
throw new FormatException("input is not a valid content-disposition string.");
}
var typeGroup = match.Groups[1];
var nameGroup = match.Groups[2];
var valueGroup = match.Groups[3];
int groupCount = match.Groups.Count;
int paramCount = nameGroup.Captures.Count;
this.type = typeGroup.Value;
this.parameters = new StringDictionary();
for (int i = 0; i < paramCount; i++ ) {
string name = nameGroup.Captures[i].Value;
string value = valueGroup.Captures[i].Value;
if (name.Equals("filename", StringComparison.InvariantCultureIgnoreCase)) {
this.fileName = value;
}
else {
this.parameters.Add(name, value);
}
}
}
public string FileName {
get {
return this.fileName;
}
}
public StringDictionary Parameters {
get {
return this.parameters;
}
}
public string Type {
get {
return this.type;
}
}
}
Then you can use it in this way:
static void Main() {
string text = "attachment; filename=\"fname.ext\"; param1=\"A\"; param2=\"A\";";
var cp = new ContentDisposition(text);
Console.WriteLine("FileName:" + cp.FileName);
foreach (DictionaryEntry param in cp.Parameters) {
Console.WriteLine("{0} = {1}", param.Key, param.Value);
}
}
// Output:
// FileName:"fname.ext"
// param1 = "A"
// param2 = "A"
The only thing that should be considered when using this class is it does not handle parameters (or filename) without a double quotation.
Edit 2:
It can now handle file names without quotations.

You can parse out the content disposition using the following framework code:
var content = "attachment; filename=myfile.csv";
var disposition = ContentDispositionHeaderValue.Parse(content);
Then just take the pieces off of the disposition instance.
disposition.FileName
disposition.DispositionType

With .NET Core 3.1 and more the most simple solution is:
using var response = await Client.SendAsync(request);
response.Content.Headers.ContentDisposition.FileName

The value is there I just needed to extract it:
The Content-Disposition header is returned like this:
Content-Disposition = attachment; filename="C:\team.jpg"; MyParameter=MyValue
So I just used some string manipulation to get the values:
void wc_DownloadDataCompleted(object sender, DownloadDataCompletedEventArgs e)
{
WebClient wc=sender as WebClient;
// Try to extract the filename from the Content-Disposition header
if (!String.IsNullOrEmpty(wc.ResponseHeaders["Content-Disposition"]))
{
string[] values = wc.ResponseHeaders["Content-Disposition"].Split(';');
string fileName = values.Single(v => v.Contains("filename"))
.Replace("filename=","")
.Replace("\"","");
/********** HERE IS THE PARAMETER ********/
string myParameter = values.Single(v => v.Contains("MyParameter"))
.Replace("MyParameter=", "")
.Replace("\"", "");
}
var data = e.Result; //File ok
}

As #Mehrzad Chehraz said you can use the new ContentDisposition class.
using System.Net.Mime;
// file1 is a HttpResponseMessage
FileName = new ContentDisposition(file1.Content.Headers.ContentDisposition.ToString()).FileName

Including SAML2.0 token in WCF service call without using WIF

I'm trying to set up a WCF service protected by ADFS. I'm currently able to request a token and send it with the request using WIF and Thinktecture IdentityModel 4.5 with the following code:
static SecurityToken GetToken()
{
var factory = new WSTrustChannelFactory(
new UserNameWSTrustBinding(SecurityMode.TransportWithMessageCredential),
"https://fs2.server2012.local/adfs/services/trust/13/usernamemixed")
{
TrustVersion = TrustVersion.WSTrust13
};
if (factory.Credentials != null)
{
factory.Credentials.UserName.UserName = #"username";
factory.Credentials.UserName.Password = "password";
}
var rst = new RequestSecurityToken
{
RequestType = RequestTypes.Issue,
KeyType = KeyTypes.Symmetric,
AppliesTo = new EndpointReference(
"https://wcfservicecertificate/wcfservice/Service.svc/wstrust"),
};
var channel = factory.CreateChannel();
RequestSecurityTokenResponse rstr;
return channel.Issue(rst, out rstr);
}
With this I can call the WCF service by using ChannelFactory.CreateChannelWithIssuedToken:
var factory = new ChannelFactory<IService>(binding,
new EndpointAddress("https://wcfservicecertificate/wcfservice/Service.svc/wstrust"));
if (factory.Credentials != null)
{
factory.Credentials.SupportInteractive = false;
factory.Credentials.UseIdentityConfiguration = true;
}
var proxy = factory.CreateChannelWithIssuedToken(GetToken());
var result= proxy.GetData(2);
This works as expected but can only be used on (mobile) windows platforms. I would also like to be able to use the same principle on iOS and Android. Using this article I was able to request a security token from ADFS using the following code:
const string soapMessage =
#"<s:Envelope xmlns:s=""http://www.w3.org/2003/05/soap-envelope""
xmlns:a=""http://www.w3.org/2005/08/addressing""
xmlns:u=""http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd"">
<s:Header>
<a:Action s:mustUnderstand=""1"">http://docs.oasis-open.org/ws-sx/ws-trust/200512/RST/Issue</a:Action>
<a:To s:mustUnderstand=""1"">https://fs2.server2012.local/adfs/services/trust/13/UsernameMixed</a:To>
<o:Security s:mustUnderstand=""1"" xmlns:o=""http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd"">
<o:UsernameToken u:Id=""uuid-6a13a244-dac6-42c1-84c5-cbb345b0c4c4-1"">
<o:Username>username</o:Username>
<o:Password Type=""http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText"">password</o:Password>
</o:UsernameToken>
</o:Security>
</s:Header>
<s:Body>
<trust:RequestSecurityToken xmlns:trust=""http://docs.oasis-open.org/ws-sx/ws-trust/200512"">
<wsp:AppliesTo xmlns:wsp=""http://schemas.xmlsoap.org/ws/2004/09/policy"">
<a:EndpointReference>
<a:Address>https://wcfservicecertificate/wcfservice/Service.svc/wstrust</a:Address>
</a:EndpointReference>
</wsp:AppliesTo>
<trust:KeyType>http://docs.oasis-open.org/ws-sx/ws-trust/200512/SymmetricKey</trust:KeyType>
<trust:RequestType>http://docs.oasis-open.org/ws-sx/ws-trust/200512/Issue</trust:RequestType>
<trust:TokenType>urn:oasis:names:tc:SAML:2.0:assertion</trust:TokenType>
</trust:RequestSecurityToken>
</s:Body>
</s:Envelope>";
var webClient = new WebClient();
webClient.Headers.Add("Content-Type", "application/soap+xml; charset=utf-8");
var result = webClient.UploadString(
address: "https://fs2.server2012.local/adfs/services/trust/13/UsernameMixed",
method: "POST",
data: soapMessage);
This results in a SAML2.0 token which I would like to send in a request to our WCF service in order to authenticate. There are various sources (including the article mentioned earlier) which state that this should be possible but I've yet to find a solution.
Any help would be appreciated.

You can use one of hybrid solutions which use SAML with OAuth or other authorization technologies. This is more secure against phising techniques. For SAML only approach, you can refer to following link: How to pass security tokenfrom one wcf service to another wcf service. It is said that you need to enable saveBootstrapTokens property on webconfig.
This link can be useful too: Availability of Bootstrap Tokens

This can easily be done without using WIF. Lets completely avoid WIF and the .Net framework and do it in Java for illustration purposes. First make a call to the Security Token Service using the template approach like you have done. You then need to extract the SAML from the response, Base64 encode it and stuff it in the Autorization header of the subsequent request to your protected WCF service. You may also need to do the same with a ProofKey if you are coding for Non-Repudiation. Also I'm only showing authentication using username/password for brevity as Certificate Authentication involves much more work - you have to hash (SHA1 )part of the message then encrypt the hash with the private key of the cert and then add this as a xml element to the original message etc...
Here is the java helper code:
import java.io.*;
import java.io.BufferedReader;
import java.io.DataOutputStream;
import java.io.InputStreamReader;
import java.net.HttpURLConnection;
import java.net.URL;
import java.time.Instant;
import java.util.Map;
import java.util.Properties;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
import java.util.Base64;
import javax.crypto.Mac;
import javax.crypto.spec.SecretKeySpec;
import javax.net.ssl.HttpsURLConnection;
public class SecurityService {
private String _username;
private String _password;
private String _stsUrl;
private String _samlAssertion;
private String _samlEncoded;
private String _binarySecret;
private String _workingDirectory;
private String _platformUrl;
private String _soapBody;
private Integer _responseCode;
private Integer _plaformResponseCode;
private String _response;
private String _platformResponse;
private String _xproofSignature;
private Map<String, String> _headerDictionary;
public void setUsername(String username) {
this._username = username;
}
public void setPassword(String password) {
this._password = password;
}
public void setStsUrl(String stsUrl) {
this._stsUrl = stsUrl;
}
public String getStsUrl() {
return _stsUrl;
}
public void setplatformUrl(String platformUrl) {
this._platformUrl = platformUrl;
}
public String getSamlAssertion() {
return _samlAssertion;
}
public String getSamlEncoded() {
return _samlEncoded;
}
public String getSoapBody() {
return _soapBody;
}
public Integer getResponseCode() {
return _responseCode;
}
public Integer getPlatformResponseCode() {
return _plaformResponseCode;
}
public String getResponse() {
return _response;
}
public String getPlatformResponse() {
return _platformResponse;
}
public String getXProofSignature() {
return _xproofSignature;
}
public String getBinarySecret() {
return _binarySecret;
}
public String gePlatFormUrl() {
return _platformUrl;
}
public void setHeaderDictionary(Map<String, String> headerDictionary){
this._headerDictionary = headerDictionary;
}
public Map<String, String> getHeaderDictionary(){
return _headerDictionary;
}
public SecurityService() throws Exception {
}
public SecurityService(Boolean useConfig) throws Exception {
if (useConfig) {
this._workingDirectory = System.getProperty("user.dir") + "\\app.config";
this.getProperties();
}
}
public void sendAuthenticatedGet() throws Exception {
URL obj = new URL(_platformUrl);
HttpURLConnection con = (HttpURLConnection) obj.openConnection();
// optional default is GET
con.setRequestMethod("GET");
// Add request header
con.setRequestProperty("Authorization", "Saml " + _samlEncoded);
con.setRequestProperty("X-ProofSignature", _xproofSignature);
_plaformResponseCode = con.getResponseCode();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
_platformResponse = response.toString();
}
public void sendAuthenticatedPost(String body) throws Exception {
URL obj = new URL(_platformUrl);
HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
//add request header
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/json");
// Add request header
con.setRequestProperty("Authorization", "Saml " + _samlEncoded);
con.setRequestProperty("X-ProofSignature", _xproofSignature);
// Add Azure Subscription Key using generic Add Headers method
if (_headerDictionary != null) {
for (String key : _headerDictionary.keySet()) {
con.setRequestProperty(key, _headerDictionary.get(key));
}
}
_soapBody = body;
// Send post request
con.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
//wr.writeBytes(urlParameters);
wr.writeBytes(_soapBody);
wr.flush();
wr.close();
_responseCode = con.getResponseCode();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
_response = response.toString();
}
// HTTP POST request
public void sendPostToSts() throws Exception {
URL obj = new URL(_stsUrl);
HttpsURLConnection con = (HttpsURLConnection) obj.openConnection();
//add request header
con.setRequestMethod("POST");
con.setRequestProperty("Content-Type", "application/soap+xml");
String body = getTemplateCertificate();
_soapBody = (((body.replace("[Created]", Instant.now().toString())).replace("[Expires]", Instant.now()
.plusSeconds(300).toString())).replace("[username]", _username)).replace("[password]", _password).replace("[stsUrl]", _stsUrl);
// Send post request
con.setDoOutput(true);
DataOutputStream wr = new DataOutputStream(con.getOutputStream());
//wr.writeBytes(urlParameters);
wr.writeBytes(_soapBody);
wr.flush();
wr.close();
_responseCode = con.getResponseCode();
BufferedReader in = new BufferedReader(new InputStreamReader(con.getInputStream()));
String inputLine;
StringBuffer response = new StringBuffer();
while ((inputLine = in.readLine()) != null) {
response.append(inputLine);
}
in.close();
_response = response.toString();
// Get Binary Secret
// <trust:BinarySecret></trust:BinarySecret>
final Pattern patternBinarySecret = Pattern.compile("<trust:BinarySecret>(.+?)</trust:BinarySecret>");
final Matcher matcherBinarySecret = patternBinarySecret.matcher(response.toString());
matcherBinarySecret.find();
_binarySecret = matcherBinarySecret.group(1);
// Get the SAML Assertion
final Pattern patternEncryptedAssertion = Pattern.compile("<trust:RequestedSecurityToken>(.+?)</trust:RequestedSecurityToken>");
final Matcher matcherEncryptedAssertion = patternEncryptedAssertion.matcher(response.toString());
matcherEncryptedAssertion.find();
_samlAssertion = matcherEncryptedAssertion.group(1);
byte[] proofKeyBytes = _binarySecret.getBytes("UTF-8");
String encoded = Base64.getEncoder().encodeToString(proofKeyBytes);
byte[] decoded = Base64.getDecoder().decode(encoded);
// SAML Stuff - Works beautifully
byte[] samlBytes = _samlAssertion.getBytes("UTF-8");
_samlEncoded = Base64.getEncoder().encodeToString(samlBytes);
_xproofSignature = this.encode(_samlAssertion, _binarySecret);
}
private static String readFile( String file ) throws IOException {
BufferedReader reader = new BufferedReader( new FileReader(file));
String line = null;
StringBuilder stringBuilder = new StringBuilder();
String ls = System.getProperty("line.separator");
try {
while( ( line = reader.readLine() ) != null ) {
stringBuilder.append( line );
stringBuilder.append( ls );
}
return stringBuilder.toString();
} finally {
reader.close();
}
}
// Embedded WS-Trust template for username/password RST
private static String getTemplate () {
return "<s:Envelope xmlns:s=\"http://www.w3.org/2003/05/soap-envelope\" xmlns:a=\"http://www.w3.org/2005/08/addressing\" xmlns:u= \"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd\"><s:Header><a:Action s:mustUnderstand= \"1\">http://docs.oasis-open.org/ws-sx/ws-trust/200512/RST/Issue</a:Action><a:MessageID>urn:uuid:cfea5555-248c-46c3-9b4d- 54936b7f815c</a:MessageID><a:ReplyTo><a:Address>http://www.w3.org/2005/08/addressing/anonymous</a:Address></a:ReplyTo><a:To s:mustUnderstand=\"1\">[stsUrl]</a:To><o:Security s:mustUnderstand=\"1\" xmlns:o=\"http://docs.oasis- open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd\"><u:Timestamp u:Id=\"_0\"><u:Created>[Created] </u:Created><u:Expires>[Expires]</u:Expires></u:Timestamp><o:UsernameToken u:Id=\"uuid-e273c018-1da7-466e-8671-86f6bfe7ce3c- 17\"><o:Username>[username]</o:Username><o:Password Type=\"http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username- token-profile-1.0#PasswordText\">[password] </o:Password></o:UsernameToken></o:Security></s:Header><s:Body><trust:RequestSecurityToken xmlns:trust=\"http://docs.oasis- open.org/ws-sx/ws-trust/200512\"><wsp:AppliesTo xmlns:wsp=\"http://schemas.xmlsoap.org/ws/2004/09/policy \"><wsa:EndpointReference xmlns:wsa=\"http://www.w3.org/2005/08/addressing \"><wsa:Address>https://mbplatform/</wsa:Address></wsa:EndpointReference></wsp:AppliesTo><trust:RequestType>http://docs.oasis- open.org/ws-sx/ws-trust/200512/Issue</trust:RequestType><trust:TokenType>http://docs.oasis-open.org/wss/oasis-wss-saml-token- profile-1.1#SAMLV2.0</trust:TokenType></trust:RequestSecurityToken></s:Body></s:Envelope>";
}
private String encode(String key, String data) throws Exception {
Mac sha256_HMAC = Mac.getInstance("HmacSHA256");
SecretKeySpec secret_key = new SecretKeySpec(key.getBytes("UTF-8"), "HmacSHA256");
sha256_HMAC.init(secret_key);
return Base64.getEncoder().encodeToString(sha256_HMAC.doFinal(data.getBytes("UTF-8")));
}
private void getProperties() throws Exception {
Properties prop = new Properties();
String fileName = _workingDirectory;
InputStream is = new FileInputStream(fileName);
prop.load(is);
_username = prop.getProperty("app.username");
_password = prop.getProperty("app.password");
_platformUrl = prop.getProperty("app.platformUrl");
_stsUrl = prop.getProperty("app.stsUrl");
}
}
and here is example usage:
SecurityService mbss = new SecurityService(true);
mbss.sendPostToSts();
System.out.println("CONTACTING AZURE SECURITY TOKEN SERVICE");
System.out.println("\nSending 'POST' request to URL : " + mbss.getStsUrl());
System.out.println("\nPost parameters : \n" + mbss.getSoapBody());
System.out.println("\nResponse Code : " + mbss.getResponseCode());
System.out.println("\nHERE IS THE SAML RESPONSE\n");
System.out.println(mbss.getResponse());
System.out.println("\nHERE IS THE BINARY SECRET\n");
System.out.println(mbss.getBinarySecret());
System.out.println("\nHERE IS THE SAML ASSERTION\n");
System.out.println(mbss.getSamlAssertion());
System.out.println("\nHERE IS THE ENCODED SAML ASSERTION\n");
System.out.println(mbss.getSamlEncoded());
System.out.println("\nHERE IS THE X-PROOF SIGNATURE\n");
System.out.println(mbss.getXProofSignature());
System.out.println("\nNOW CONTACTING WCF SERVICES WITH SECURITY HEADER\n");
mbss.sendAuthenticatedGet();
System.out.println("\nSending 'GET' request to URL : " + mbss.gePlatFormUrl());
System.out.println("Response Code : " + mbss.getPlatformResponseCode());
System.out.println("\nHERE ARE THE RESULTS FOLKS...ENJOY\n");
System.out.println(mbss.getPlatformResponse());

EntityTooSmall in CompleteMultipartUploadResponse

using .NET SDK v.1.5.21.0
I'm trying to upload a large file (63Mb) and I'm following the example at:
http://docs.aws.amazon.com/AmazonS3/latest/dev/LLuploadFileDotNet.html
But using a helper instead the hole code and using jQuery File Upload
https://github.com/blueimp/jQuery-File-Upload/blob/master/basic-plus.html
what I have is:
string bucket = "mybucket";
long totalSize = long.Parse(context.Request.Headers["X-File-Size"]),
maxChunkSize = long.Parse(context.Request.Headers["X-File-MaxChunkSize"]),
uploadedBytes = long.Parse(context.Request.Headers["X-File-UloadedBytes"]),
partNumber = uploadedBytes / maxChunkSize + 1,
fileSize = partNumber * inputStream.Length;
bool lastPart = inputStream.Length < maxChunkSize;
// http://docs.aws.amazon.com/AmazonS3/latest/dev/LLuploadFileDotNet.html
if (partNumber == 1) // initialize upload
{
iView.Utilities.Amazon_S3.S3MultipartUpload.InitializePartToCloud(fileName, bucket);
}
try
{
// upload part
iView.Utilities.Amazon_S3.S3MultipartUpload.UploadPartToCloud(fs, fileName, bucket, (int)partNumber, uploadedBytes, maxChunkSize);
if (lastPart)
// wrap it up and go home
iView.Utilities.Amazon_S3.S3MultipartUpload.CompletePartToCloud(fileName, bucket);
}
catch (System.Exception ex)
{
// Huston, we have a problem!
//Console.WriteLine("Exception occurred: {0}", exception.Message);
iView.Utilities.Amazon_S3.S3MultipartUpload.AbortPartToCloud(fileName, bucket);
}
and
public static class S3MultipartUpload
{
private static string accessKey = System.Configuration.ConfigurationManager.AppSettings["AWSAccessKey"];
private static string secretAccessKey = System.Configuration.ConfigurationManager.AppSettings["AWSSecretKey"];
private static AmazonS3 client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKey, secretAccessKey);
public static InitiateMultipartUploadResponse initResponse;
public static List<UploadPartResponse> uploadResponses;
public static void InitializePartToCloud(string destinationFilename, string destinationBucket)
{
// 1. Initialize.
uploadResponses = new List<UploadPartResponse>();
InitiateMultipartUploadRequest initRequest =
new InitiateMultipartUploadRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'));
initResponse = client.InitiateMultipartUpload(initRequest);
}
public static void UploadPartToCloud(Stream fileStream, string destinationFilename, string destinationBucket, int partNumber, long uploadedBytes, long maxChunkedBytes)
{
// 2. Upload Parts.
UploadPartRequest request = new UploadPartRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'))
.WithUploadId(initResponse.UploadId)
.WithPartNumber(partNumber)
.WithPartSize(maxChunkedBytes)
.WithFilePosition(uploadedBytes)
.WithInputStream(fileStream) as UploadPartRequest;
uploadResponses.Add(client.UploadPart(request));
}
public static void CompletePartToCloud(string destinationFilename, string destinationBucket)
{
// Step 3: complete.
CompleteMultipartUploadRequest compRequest =
new CompleteMultipartUploadRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'))
.WithUploadId(initResponse.UploadId)
.WithPartETags(uploadResponses);
CompleteMultipartUploadResponse completeUploadResponse =
client.CompleteMultipartUpload(compRequest);
}
public static void AbortPartToCloud(string destinationFilename, string destinationBucket)
{
// abort.
client.AbortMultipartUpload(new AbortMultipartUploadRequest()
.WithBucketName(destinationBucket)
.WithKey(destinationFilename.TrimStart('/'))
.WithUploadId(initResponse.UploadId));
}
}
my maxChunckedSize is 6Mb (6 * (1024*1024)) as I have read that the minimum is 5Mb...
why am I getting "Your proposed upload is smaller than the minimum allowed size" exception? What am I doing wrong?
The error is:
<Error>
<Code>EntityTooSmall</Code>
<Message>Your proposed upload is smaller than the minimum allowed size</Message>
<ETag>d41d8cd98f00b204e9800998ecf8427e</ETag>
<MinSizeAllowed>5242880</MinSizeAllowed>
<ProposedSize>0</ProposedSize>
<RequestId>C70E7A23C87CE5FC</RequestId>
<HostId>pmhuMXdRBSaCDxsQTHzucV5eUNcDORvKY0L4ZLMRBz7Ch1DeMh7BtQ6mmfBCLPM2</HostId>
<PartNumber>1</PartNumber>
</Error>
How can I get ProposedSize if I'm passing the stream and stream length?

Here is a working solution for the latest Amazon SDK (as today: v.1.5.37.0)
Amazon S3 Multipart Upload works like:
Initialize the request using client.InitiateMultipartUpload(initRequest)
Send chunks of the file (loop until the end) using client.UploadPart(request)
Complete the request using client.CompleteMultipartUpload(compRequest)
If anything goes wrong, remember to dispose the client and request, as well fire the abort command using client.AbortMultipartUpload(abortMultipartUploadRequest)
I keep the client in Session as we need this for each chunk upload as well, keep an hold of the ETags that are now used to complete the process.
You can see an example and simple way of doing this in Amazon Docs itself, I ended up having a class to do everything, plus, I have integrated with the lovely jQuery File Upload plugin (Handler code below as well).
The S3MultipartUpload is as follow
public class S3MultipartUpload : IDisposable
{
string accessKey = System.Configuration.ConfigurationManager.AppSettings.Get("AWSAccessKey");
string secretAccessKey = System.Configuration.ConfigurationManager.AppSettings.Get("AWSSecretKey");
AmazonS3 client;
public string OriginalFilename { get; set; }
public string DestinationFilename { get; set; }
public string DestinationBucket { get; set; }
public InitiateMultipartUploadResponse initResponse;
public List<PartETag> uploadPartETags;
public string UploadId { get; private set; }
public S3MultipartUpload(string destinationFilename, string destinationBucket)
{
if (client == null)
{
System.Net.WebRequest.DefaultWebProxy = null; // disable proxy to make upload quicker
client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKey, secretAccessKey, new AmazonS3Config()
{
RegionEndpoint = Amazon.RegionEndpoint.EUWest1,
CommunicationProtocol = Protocol.HTTP
});
this.OriginalFilename = destinationFilename.TrimStart('/');
this.DestinationFilename = string.Format("{0:yyyy}{0:MM}{0:dd}{0:HH}{0:mm}{0:ss}{0:fffff}_{1}", DateTime.UtcNow, this.OriginalFilename);
this.DestinationBucket = destinationBucket;
this.InitializePartToCloud();
}
}
private void InitializePartToCloud()
{
// 1. Initialize.
uploadPartETags = new List<PartETag>();
InitiateMultipartUploadRequest initRequest = new InitiateMultipartUploadRequest();
initRequest.BucketName = this.DestinationBucket;
initRequest.Key = this.DestinationFilename;
// make it public
initRequest.AddHeader("x-amz-acl", "public-read");
initResponse = client.InitiateMultipartUpload(initRequest);
}
public void UploadPartToCloud(Stream fileStream, long uploadedBytes, long maxChunkedBytes)
{
int partNumber = uploadPartETags.Count() + 1; // current part
// 2. Upload Parts.
UploadPartRequest request = new UploadPartRequest();
request.BucketName = this.DestinationBucket;
request.Key = this.DestinationFilename;
request.UploadId = initResponse.UploadId;
request.PartNumber = partNumber;
request.PartSize = fileStream.Length;
//request.FilePosition = uploadedBytes // remove this line?
request.InputStream = fileStream; // as UploadPartRequest;
var up = client.UploadPart(request);
uploadPartETags.Add(new PartETag() { ETag = up.ETag, PartNumber = partNumber });
}
public string CompletePartToCloud()
{
// Step 3: complete.
CompleteMultipartUploadRequest compRequest = new CompleteMultipartUploadRequest();
compRequest.BucketName = this.DestinationBucket;
compRequest.Key = this.DestinationFilename;
compRequest.UploadId = initResponse.UploadId;
compRequest.PartETags = uploadPartETags;
string r = "Something went badly wrong";
using (CompleteMultipartUploadResponse completeUploadResponse = client.CompleteMultipartUpload(compRequest))
r = completeUploadResponse.ResponseXml;
return r;
}
public void AbortPartToCloud()
{
// abort.
client.AbortMultipartUpload(new AbortMultipartUploadRequest()
{
BucketName = this.DestinationBucket,
Key = this.DestinationFilename,
UploadId = initResponse.UploadId
});
}
public void Dispose()
{
if (client != null) client.Dispose();
if (initResponse != null) initResponse.Dispose();
}
}
I use DestinationFilename as the destination file so I can avoid the same name, but I keep the OriginalFilename as I needed later.
Using jQuery File Upload Plugin, all works inside a Generic Handler, and the process is something like this:
// Upload partial file
private void UploadPartialFile(string fileName, HttpContext context, List<FilesStatus> statuses)
{
if (context.Request.Files.Count != 1)
throw new HttpRequestValidationException("Attempt to upload chunked file containing more than one fragment per request");
var inputStream = context.Request.Files[0].InputStream;
string contentRange = context.Request.Headers["Content-Range"]; // "bytes 0-6291455/14130271"
int fileSize = int.Parse(contentRange.Split('/')[1]);,
maxChunkSize = int.Parse(context.Request.Headers["X-Max-Chunk-Size"]),
uploadedBytes = int.Parse(contentRange.Replace("bytes ", "").Split('-')[0]);
iView.Utilities.AWS.S3MultipartUpload s3Upload = null;
try
{
// ######################################################################################
// 1. Initialize Amazon S3 Client
if (uploadedBytes == 0)
{
HttpContext.Current.Session["s3-upload"] = new iView.Utilities.AWS.S3MultipartUpload(fileName, awsBucket);
s3Upload = (iView.Utilities.AWS.S3MultipartUpload)HttpContext.Current.Session["s3-upload"];
string msg = System.String.Format("Upload started: {0} ({1:N0}Mb)", s3Upload.DestinationFilename, (fileSize / 1024));
this.Log(msg);
}
// cast current session object
if (s3Upload == null)
s3Upload = (iView.Utilities.AWS.S3MultipartUpload)HttpContext.Current.Session["s3-upload"];
// ######################################################################################
// 2. Send Chunks
s3Upload.UploadPartToCloud(inputStream, uploadedBytes, maxChunkSize);
// ######################################################################################
// 3. Complete Upload
if (uploadedBytes + maxChunkSize > fileSize)
{
string completeRequest = s3Upload.CompletePartToCloud();
this.Log(completeRequest); // log S3 response
s3Upload.Dispose(); // dispose all objects
HttpContext.Current.Session["s3-upload"] = null; // we don't need this anymore
}
}
catch (System.Exception ex)
{
if (ex.InnerException != null)
while (ex.InnerException != null)
ex = ex.InnerException;
this.Log(string.Format("{0}\n\n{1}", ex.Message, ex.StackTrace)); // log error
s3Upload.AbortPartToCloud(); // abort current upload
s3Upload.Dispose(); // dispose all objects
statuses.Add(new FilesStatus(ex.Message));
return;
}
statuses.Add(new FilesStatus(s3Upload.DestinationFilename, fileSize, ""));
}
Keep in mind that to have a Session object inside a Generic Handler, you need to implement IRequiresSessionState so your handler will look like:
public class UploadHandlerSimple : IHttpHandler, IRequiresSessionState
Inside fileupload.js (under _initXHRData) I have added an extra header called X-Max-Chunk-Size so I can pass this to Amazon and calculate if it's the last part of the uploaded file.
Fell free to comment and make smart edits for everyone to use.

I guess you didn't set the content-length of the part inside the UploadPartToCloud() function.

Determine if an object exists in a S3 bucket based on wildcard

Can someone please show me how to determine if a certain file/object exists in a S3 bucket and display a message if it exists or if it does not exist.
Basically I want it to:
1) Check a bucket on my S3 account such as testbucket
2) Inside of that bucket, look to see if there is a file with the prefix test_ (test_file.txt or test_data.txt).
3) If that file exists, then display a MessageBox (or Console message) that the file exists, or that the file does not exist.
Can someone please show me how to do this?

Using the AWSSDK For .Net I Currently do something along the lines of:
public bool Exists(string fileKey, string bucketName)
{
try
{
response = _s3Client.GetObjectMetadata(new GetObjectMetadataRequest()
.WithBucketName(bucketName)
.WithKey(key));
return true;
}
catch (Amazon.S3.AmazonS3Exception ex)
{
if (ex.StatusCode == System.Net.HttpStatusCode.NotFound)
return false;
//status wasn't not found, so throw the exception
throw;
}
}
It kinda sucks, but it works for now.

Use the S3FileInfo.Exists method:
using (var client = Amazon.AWSClientFactory.CreateAmazonS3Client(accessKey, secretKey))
{
S3FileInfo s3FileInfo = new Amazon.S3.IO.S3FileInfo(client, "your-bucket-name", "your-file-name");
if (s3FileInfo.Exists)
{
// file exists
}
else
{
// file does not exist
}
}

Not sure if this applies to .NET Framework, but the .NET Core version of AWS SDK (v3) only supports async requests, so I had to use a slightly different solution:
/// <summary>
/// Determines whether a file exists within the specified bucket
/// </summary>
/// <param name="bucket">The name of the bucket to search</param>
/// <param name="filePrefix">Match files that begin with this prefix</param>
/// <returns>True if the file exists</returns>
public async Task<bool> FileExists(string bucket, string filePrefix)
{
// Set this to your S3 region (of course)
var region = Amazon.RegionEndpoint.USEast1;
using (var client = new AmazonS3Client(region))
{
var request = new ListObjectsRequest {
BucketName = bucket,
Prefix = filePrefix,
MaxKeys = 1
};
var response = await client.ListObjectsAsync(request, CancellationToken.None);
return response.S3Objects.Any();
}
}
And, if you want to search a folder:
/// <summary>
/// Determines whether a file exists within the specified folder
/// </summary>
/// <param name="bucket">The name of the bucket to search</param>
/// <param name="folder">The name of the folder to search</param>
/// <param name="filePrefix">Match files that begin with this prefix</param>
/// <returns>True if the file exists</returns>
public async Task<bool> FileExists(string bucket, string folder, string filePrefix)
{
return await FileExists(bucket, $"{folder}/{filePrefix}");
}
Usage:
var testExists = await FileExists("testBucket", "test_");
// or...
var testExistsInFolder = await FileExists("testBucket", "testFolder/testSubFolder", "test_");

This solves it:
List the bucket for existing objects and use a prefix like so.
var request = new ListObjectsRequest()
.WithBucketName(_bucketName)
.WithPrefix(keyPrefix);
var response = _amazonS3Client.ListObjects(request);
var exists = response.S3Objects.Count > 0;
foreach (var obj in response.S3Objects) {
// act
}

I know this question is a few years old but the new SDK handles this beautifully. If anyone is still searching this. You are looking for S3DirectoryInfo Class
using (IAmazonS3 s3Client = new AmazonS3Client(accessKey, secretKey))
{
S3DirectoryInfo s3DirectoryInfo = new Amazon.S3.IO.S3DirectoryInfo(s3Client, "testbucket");
if (s3DirectoryInfo.GetFiles("test*").Any())
{
//file exists -- do something
}
else
{
//file doesn't exist -- do something else
}
}

I know this question is a few years old but the new SDK nowdays handles this in an easier manner.
public async Task<bool> ObjectExistsAsync(string prefix)
{
var response = await _amazonS3.GetAllObjectKeysAsync(_awsS3Configuration.BucketName, prefix, null);
return response.Count > 0;
}
Where _amazonS3 is your IAmazonS3 instance and _awsS3Configuration.BucketName is your bucket name.
You can use your complete key as a prefix.

I used the following code in C# with Amazon S3 version 3.1.5(.net 3.5) to check if the bucket exists:
BasicAWSCredentials credentials = new BasicAWSCredentials("accessKey", "secretKey");
AmazonS3Config configurationAmazon = new AmazonS3Config();
configurationAmazon.RegionEndpoint = S3Region.EU; // or you can use ServiceUrl
AmazonS3Client s3Client = new AmazonS3Client(credentials, configurationAmazon);
S3DirectoryInfo directoryInfo = new S3DirectoryInfo(s3Client, bucketName);
bucketExists = directoryInfo.Exists;// true if the bucket exists in other case false.
I used the followings code(in C# with Amazon S3 version 3.1.5 .net 3.5) the file Exists.
Option 1:
S3FileInfo info = new S3FileInfo(s3Client, "butcketName", "key");
bool fileExists = info.Exists; // true if the key Exists in other case false
Option 2:
ListObjectsRequest request = new ListObjectsRequest();
try
{
request.BucketName = "bucketName";
request.Prefix = "prefix"; // or part of the key
request.MaxKeys = 1; // max limit to find objects
ListObjectsResponse response = s3Client .ListObjects(request);
return response.S3Objects.Count > 0;
}

I'm not familiar with C#, but I use this method from Java (conversion to c# is immediate):
public boolean exists(AmazonS3 s3, String bucket, String key) {
ObjectListing list = s3.listObjects(bucket, key);
return list.getObjectSummaries().size() > 0;
}

my 2 cents
public async Task<bool> DoesKeyExistsAsync(string key)
{
ListObjectsResponse response = null;
try
{
response = await _s3Client.ListObjectsAsync(new ListObjectsRequest { BucketName = _settings.BucketName, Prefix = key });
}
catch (Exception ex)
{
_logger.LogError($"Error while checking key {key}");
return false;
}
return (response?.S3Objects?.Count > 0);
}

s3 = new S3(S3KEY, S3SECRET, false);
res = s3->getObjectInfo(bucketName, filename);
It will return array if file exists

try this one:
NameValueCollection appConfig = ConfigurationManager.AppSettings;
AmazonS3 s3Client = AWSClientFactory.CreateAmazonS3Client(
appConfig["AWSAccessKey"],
appConfig["AWSSecretKey"],
Amazon.RegionEndpoint.USEast1
);
S3DirectoryInfo source = new S3DirectoryInfo(s3Client, "BUCKET_NAME", "Key");
if(source.Exist)
{
//do ur stuff
}

using Amazon;
using Amazon.S3;
using Amazon.S3.IO;
using Amazon.S3.Model;
string accessKey = "xxxxx";
string secretKey = "xxxxx";
string regionEndpoint = "EU-WEST-1";
string bucketName = "Bucket1";
string filePath = "https://Bucket1/users/delivery/file.json"
public bool FileExistsOnS3(string filePath)
{
try
{
Uri myUri = new Uri(filePath);
string absolutePath = myUri.AbsolutePath; // /users/delivery/file.json
string key = absolutePath.Substring(1); // users/delivery/file.json
using(var client = AWSClientFactory.CreateAmazonS3Client(accessKey, secretKey, regionEndpoint))
{
S3FileInfo file = new S3FileInfo(client, bucketName, key);
if (file.Exists)
{
return true;
// custom logic
}
else
{
return false;
// custom logic
}
}
}
catch(AmazonS3Exception ex)
{
return false;
}
}

There is an overload for GetFileSystemInfos
Notice this line has filename.*
var files= s3DirectoryInfo.GetFileSystemInfos("filename.*");
public bool Check()
{
var awsCredentials = new Amazon.Runtime.BasicAWSCredentials("AccessKey", "SecretKey");
using (var client = new AmazonS3Client(awsCredentials, Amazon.RegionEndpoint.USEast1))
{
S3DirectoryInfo s3DirectoryInfo = new S3DirectoryInfo(client, bucketName, "YourFilePath");
var files= s3DirectoryInfo.GetFileSystemInfos("filename.*");
if(files.Any())
{
//fles exists
}
}
}

We Keep Coding

C# (C-Sharp) is a programming language developed by Microsoft that runs on the .NET Framework.

Amazon S3 upload with public permissions - c#

You can use the property CannedACL of Amazon.S3.Model.PutObjectRequest to upload files to S3 as shown below, var putObjectRequest = new Amazon.S3.Model.PutObjectRequest { BucketName = bucketName, Key = key, CannedACL = S3CannedACL.PublicReadWrite };

Related

AmazonSQSClient not refreshing AWSCredentials when Credentials File is updated

Get Content-Disposition parameters

Including SAML2.0 token in WCF service call without using WIF

EntityTooSmall in CompleteMultipartUploadResponse

Determine if an object exists in a S3 bucket based on wildcard

Categories

Resources