Currently, I make a GetObjectMetaDataRequest, if the GetObjectMetaDataResponse throw an exception means the object doesn't exist. Is there a better way to check whether the file exists without downloading the file.
you can use S3FileInfo class and Exists method of this class it will hep you to check if file exists without download the file .see the example below I used the AWSSDK 3.1.6 .net(3.5) :
public static bool ExistsFile()
{
BasicAWSCredentials basicCredentials = new BasicAWSCredentials("my access key", "my secretkey");
AmazonS3Config configurationClient = new AmazonS3Config();
configurationClient.RegionEndpoint = RegionEndpoint.EUCentral1;
try
{
using (AmazonS3Client clientConnection = new AmazonS3Client(basicCredentials, configurationClient))
{
S3FileInfo file = new S3FileInfo(clientConnection, "mybucket", "FolderNameUniTest680/FileNameUnitTest680");
return file.Exists;//if the file exists return true, in other case false
}
}
catch(Exception ex)
{
return false;
}
}
If you are in a similar situation as myself and are using .Net Core and don't have access to Amazon.S3.IO (and S3FileInfo method), you can do the following using asynchronous GetObjectMetadataRequest method:
static private AmazonS3Client s3Client = new AmazonS3Client();
public static async Task<bool> FileExistsS3Async(string _bucket, string _key)
{
GetObjectMetadataRequest request = new GetObjectMetadataRequest()
{
BucketName = _bucket,
Key = _key
};
try
{
await s3Client.GetObjectMetadataAsync(request);
return true;
}
catch (AmazonS3Exception exception)
{
return false;
}
}
This function has worked for me when calling within a Unity game. You can also call the above function synchronously using the following:
bool exists = Task.Run(()=>FileExistsS3Async(_bucket, _key)).Result;
Try this solution, it works for me.
AmazonS3Client client = new AmazonS3Client(accessKey, secretKey, regionEndpoint);
S3FileInfo s3FileInfo = new S3FileInfo(client, bucketName, fileName);
return s3FileInfo.Exists;
There is no ListObjectRequest, but instead a ListObjectsRequest where you cannot specify the Key. You then have to go through all the objects to find the one you want. I am currently looking in to it since I seem to get time out errors whilst downloading the file. (If anyone has some idea how to solve that feel free to comment).
You could instead try the List Parts Request if you happen to know the upload id.
Other than that I have no idea. Would like to have a chat with the person who wrote the S3 api...
You're probably going to have to use the REST API yourself, as the method suggested, internally just does exactly the same thing (try...catch on the request)
You can use this code to check whether an object exist in S3 or not:
public class S3CheckFileExists
{
private readonly IAmazonS3 amazonS3;
public S3CheckFileExists(IAmazonS3 amazonS3)
{
this.amazonS3 = amazonS3;
}
public async Task<bool> S3ObjectExists(string bucketName, string keyLocation)
{
var listS3Objects = await this.amazonS3.ListObjectsV2Async(new ListObjectsV2Request
{
BucketName = bucketName,
Prefix = keyLocation, // eg myfolder/myimage.jpg (no / at start)
MaxKeys = 1
});
if (listS3Objects.S3Objects.Any() == false || listS3Objects.S3Objects.All(x => x.Key != keyLocation))
{
// S3 object doesn't exist
return false;
}
// S3 object exists
return true;
}
}
You'll need to register IAmazonS3 in your IoC (aka services) container though:
services.AddAWSService<IAmazonS3>();
Yes.
You can use a ListObjectsRequest. Use the Marker property, and retrieve only 1 element.
Related
I'm wondering whether there is a clean way to create a new file using Octokit in case the file specified doesn't exist, or otherwise create it? I'm currently attempting to do this via a try-catch block. However, there needs to be a more straightforward way to do this except a try-catch block?
private IRepositoryContentsClient Content => _client.Repository.Content;
public void TransmitLog(string logFilePath)
{
var logContent = LogFileToString(logFilePath);
var githubPath = GenerateGithubPath(logFilePath);
try
{
UpdateLog(githubPath, logContent);
}
catch (NotFoundException)
{
CreateLog(githubPath, logContent);
}
catch (AggregateException)
{
CreateLog(githubPath, logContent);
}
}
private void UpdateLog(string githubPath, string logContent)
{
// MY APP FAILS HERE
var existingFile = Content.GetAllContentsByRef(
_owner, _repo, githubPath, _branch).Result;
// update the file
var relevantSha = existingFile.First().Sha;
var updateRequest = new UpdateFileRequest("Log update" + DateTime.UtcNow, logContent, relevantSha, _branch);
var updateChangeSet = Content.UpdateFile(_owner, _repo, githubPath, updateRequest);
}
private void CreateLog(string githubPath, string logFileContent)
{
// if file is not found, create it
var createRequest =
new CreateFileRequest("Log Creation" + DateTime.UtcNow, logFileContent, _branch);
var createChangeSet = Content.CreateFile(_owner, _repo, githubPath, createRequest);
}
EDIT: I initially had both CreateLog and UpdateLog declared as async void, which led to a whole set of other problems. I edited that out since what I'm really interested in is how to avoid this try/catch structure.
This may not be 100% an EPPlus issue, but since it is Blazor WASM it appears I cannot get the file path to a static image in the wwwroot/images folder. I can get the url and paste it into a browser and that works, even adding that same path to the src attribute of an img works, neither of those helps me.
FYI "background" in this context means a watermark.
It appears that the EPPlus dev team only wants a drive path the file (ex. C:\SomeFolder\SomeFile.png), and I am not seeing how to get that within Blazor WASM. I can get the bytes of the file in c# and even a stream, but no direct path.
My code is the following:
using (var package = new ExcelPackage(fileName))
{
var sheet = package.Workbook.Worksheets.Add(exportModel.OSCode);
sheet.BackgroundImage.SetFromFile("https://localhost:44303/images/Draft.png");
...
}
This returns an exception:
Unhandled exception rendering component: Can't find file /https:/localhost:44303/images/Draft.png
Noticing that leading / I even tried:
sheet.BackgroundImage.SetFromFile("images/Draft.png");
Which returned the same error:
Unhandled exception rendering component: Can't find file /images/Draft.png
So, I am perhaps needing 1 of 2 possible answers:
A way to get a local drive path to the file so the .SetFromFile method is not going to error.
To have a way to set that BackgroundImage property with a byte array or stream of the image. There is this property BackgroundImage.Image but it is readonly.
Thanks to a slap in the face from #Panagiotis-Kanavos I wound up taking the processing out of the client and moving it to the server. With that, I was able to use Static Files to add the watermark with relatively little pain.
In case anyone may need the full solution (which I always find helpful) here it is:
Here is the code within the button click on the Blazor component or page:
private async Task GenerateFile(bool isFinal)
{
...
var fileStream = await excelExportService.ProgramMap(exportModel);
var fileName = "SomeFileName.xlsx";
using var streamRef = new DotNetStreamReference(stream: fileStream);
await jsRuntime.InvokeVoidAsync("downloadFileFromStream", fileName, streamRef);
}
That calls a client-side service that really just passes control over to the server:
public class ExcelExportService : IExcelExportService
{
private const string baseUri = "api/excel-export";
private readonly IHttpService httpService;
public ExcelExportService(IHttpService httpService)
{
this.httpService = httpService;
}
public async Task<Stream> ProgramMap(ProgramMapExportModel exportModel)
{
return await httpService.PostAsJsonForStreamAsync<ProgramMapExportModel>($"{baseUri}/program-map", exportModel);
}
}
Here is the server-side controller that catches the call from the client:
[Route("api/excel-export")]
[ApiController]
public class ExcelExportController : ControllerBase
{
private readonly ExcelExportService excelExportService;
public ExcelExportController(ExcelExportService excelExportService)
{
this.excelExportService = excelExportService;
}
[HttpPost]
[Route("program-map")]
public async Task<Stream> ProgramMap([FromBody] ProgramMapExportModel exportModel)
{
return await excelExportService.ProgramMap(exportModel);
}
}
And that in-turn calls the server-side service where the magic happens:
public async Task<Stream> ProgramMap(ProgramMapExportModel exportModel)
{
var result = new MemoryStream();
ExcelPackage.LicenseContext = LicenseContext.Commercial;
var fileName = #$"Gets Overwritten";
using (var package = new ExcelPackage(fileName))
{
var sheet = package.Workbook.Worksheets.Add(exportModel.OSCode);
if (!exportModel.IsFinal)
{
var pathToDraftImage = #$"{Directory.GetCurrentDirectory()}\StaticFiles\Images\Draft.png";
sheet.BackgroundImage.SetFromFile(pathToDraftImage);
}
...
sheet.Cells.AutoFitColumns();
package.SaveAs(result);
}
result.Position = 0; // Without this, data does not get written
return result;
}
For some reason, this next method was not needed when doing this on the client-side but now that it is back here, I had to add a method that returned a stream specifically and used the ReadAsStreamAsync instead of ReadAsJsonAsync:
public async Task<Stream> PostAsJsonForStreamAsync<TValue>(string requestUri, TValue value, CancellationToken cancellationToken = default)
{
Stream result = default;
var responseMessage = await httpClient.PostAsJsonAsync(requestUri, value, cancellationToken);
try
{
result = await responseMessage.Content.ReadAsStreamAsync(cancellationToken: cancellationToken);
}
catch (HttpRequestException e)
{
...
}
return result;
}
Lastly, in order for it to give the end-user a download link, this was used (taken from the Microsoft Docs):
window.downloadFileFromStream = async (fileName, contentStreamReference) => {
const arrayBuffer = await contentStreamReference.arrayBuffer();
const blob = new Blob([arrayBuffer]);
const url = URL.createObjectURL(blob);
const anchorElement = document.createElement("a");
anchorElement.href = url;
anchorElement.download = fileName ?? "";
anchorElement.click();
anchorElement.remove();
URL.revokeObjectURL(url);
}
I am building a ASP.NET Core website, where the user signs in, and is able to save things like the theme of the page, which then triggers some js code:
var data = {
userName: userName,
key: "theme",
value: localStorage.getItem("theme")
}
var pendingRequest;
if (pendingRequest) {
pendingRequest.abort()
pendingRequest = null;
}
pendingRequest = $.post('/Account/setExtension', data, function (data) {
pendingRequest = null;
$('#settingsModal').modal('hide');
});
which calls the controller:
[HttpPost]
public void setExtension(string userName, string key, string value)
{
Dictionary<string, object> addData = new Dictionary<string, object>
{
{key, value }
};
GraphHelper.updateExtension(userName, "com.user.roamingSettings", addData).ConfigureAwait(false);
}
which calls the checks if the extension exists, and then decides to create it, or update the existing extension:
public static async Task<bool> extensionExistsAsync(string userName, string extensionName)
{
try
{
var graphClient = GetAuthenticatedClient();
// if extension doesnt exist
bool extensionFound = false;
var extensions = await graphClient.Users[userName].Extensions.Request().GetAsync();
foreach (var extension in extensions)
{
if (extension.Id == extensionName)
{
extensionFound = true;
}
}
return extensionFound;
}
catch (Exception e)
{
Debug.WriteLine(e.Message.ToString());
throw;
}
}
The problem is, the code just stops running on this line:
var extensions = await graphClient.Users[userName].Extensions.Request().GetAsync();
It doesn't throw or anything. Stepping through it line by line, it returns all the way to the assignment, and the output window is empty when it stops. Why is this? How can I get an extension by name, or get all extensions to see which ones exist, either by graph api, or calls?
When ever you use the below call
var extensions = await graphClient.Users[userName].Extensions.Request().GetAsync();
you will be getting UserExtensionsCollectionPage object which gives the list of extensions of a user.
This page doesn't have Id property, the extension objects that are present in this UserExtensionsCollectionPage object have it. So use the below code to print the id and type of the Extensions.
var extensions = await graphClient.Users["a1ee289f-4cab-4fc3-885f-d4cbefb48895"].Extensions.Request().GetAsync();
foreach(var data in extensions.CurrentPage)
{
Console.WriteLine("Id: " + data.Id + "Type: " + data.ODataType );
}
This will give you the data as below.
I was wondering if there's a way to use a generic Type when you define a table?
The DefineTable method takes your model to define the local table on the mobile phone. The model type is specified using angular brackets. However I am looking to pass a Type object and use this to define the table, however, you cannot use a variable inside angular brackets.
Below are code samples and more explanation of what exactly I am looking for...
This is how it is currently being implemented:
public async Task Init()
{
this.initialized = true;
const string Path = "syncorder.db";
var store = new MobileServiceSQLiteStore(Path);
store.DefineTable<Order>();
}
This is to get an idea of what I am trying to accomplish:
public async Task Init(Type tableType)
{
this.initialized = true;
const string Path = "syncorder.db";
var store = new MobileServiceSQLiteStore(Path);
//This is what i would like to do, but cant since its not possible to put variables inside angular brackets
store.DefineTable<tableType>();
}
I have also tried checking if it is possible to pass in the Type object as a parameter to the DefineTable method (since in some cases this is possible) but there is no overload allowing this.
//This is not possible...
store.DefineTable(tableType);
Anyhow all help is appreciated. Thanks!
So I figured out how to do this in case anyone runs into this issue below is the solution:
public Task Init(Type tableType)
{
try
{
this.initialized = true;
const string Path = "syncorder.db";
var store = new MobileServiceSQLiteStore(Path);
MethodInfo myDefineTable = RuntimeReflectionExtensions.GetRuntimeMethod(typeof(MobileServiceSQLiteStoreExtensions), "DefineTable", new Type[] { typeof(MobileServiceSQLiteStore) });
myDefineTable = myDefineTable.MakeGenericMethod(tableType);
myDefineTable.Invoke(null, new object[] { store });
for the parameters to pass into the method
}
catch (Exception ex)
{
this.initialized = false;
throw ex;
}
}
Hope this helps someone... cheers!
I think that you can just do the following:
public IMobileServiceSyncTable<T> Init<T>()
{
var store = new MobileServiceSQLiteStore(DatabaseName);
store.DefineTable<T>();
var Client = Client = new MobileServiceClient(SystemOptions.AzureStorage);
Client.SyncContext.InitializeAsync(store);
return Client.GetSyncTable<T>();
}
I've got a very simple question (I hope!) - I just want to find out if a blob (with a name I've defined) exists in a particular container. I'll be downloading it if it does exist, and if it doesn't then I'll do something else.
I've done some searching on the intertubes and apparently there used to be a function called DoesExist or something similar... but as with so many of the Azure APIs, this no longer seems to be there (or if it is, has a very cleverly disguised name).
The new API has the .Exists() function call. Just make sure that you use the GetBlockBlobReference, which doesn't perform the call to the server. It makes the function as easy as:
public static bool BlobExistsOnCloud(CloudBlobClient client,
string containerName, string key)
{
return client.GetContainerReference(containerName)
.GetBlockBlobReference(key)
.Exists();
}
Note: This answer is out of date now. Please see Richard's answer for an easy way to check for existence
No, you're not missing something simple... we did a good job of hiding this method in the new StorageClient library. :)
I just wrote a blog post to answer your question: http://blog.smarx.com/posts/testing-existence-of-a-windows-azure-blob.
The short answer is: use CloudBlob.FetchAttributes(), which does a HEAD request against the blob.
Seem lame that you need to catch an exception to test it the blob exists.
public static bool Exists(this CloudBlob blob)
{
try
{
blob.FetchAttributes();
return true;
}
catch (StorageClientException e)
{
if (e.ErrorCode == StorageErrorCode.ResourceNotFound)
{
return false;
}
else
{
throw;
}
}
}
If the blob is public you can, of course, just send an HTTP HEAD request -- from any of the zillions of languages/environments/platforms that know how do that -- and check the response.
The core Azure APIs are RESTful XML-based HTTP interfaces. The StorageClient library is one of many possible wrappers around them. Here's another that Sriram Krishnan did in Python:
http://www.sriramkrishnan.com/blog/2008/11/python-wrapper-for-windows-azure.html
It also shows how to authenticate at the HTTP level.
I've done a similar thing for myself in C#, because I prefer to see Azure through the lens of HTTP/REST rather than through the lens of the StorageClient library. For quite a while I hadn't even bothered to implement an ExistsBlob method. All my blobs were public, and it was trivial to do HTTP HEAD.
The new Windows Azure Storage Library already contains the Exist() method.
It´s in the Microsoft.WindowsAzure.Storage.dll.
Available as NuGet Package
Created by: Microsoft
Id: WindowsAzure.Storage
Version: 2.0.5.1
See also msdn
Here's a different solution if you don't like the other solutions:
I am using version 12.4.1 of the Azure.Storage.Blobs NuGet Package.
I get an Azure.Pageable object which is a list of all of the blobs in a container. I then check if the name of the BlobItem equals to the Name property of each blob inside the container utilizing LINQ. (If everything is valid, of course)
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using System.Linq;
using System.Text.RegularExpressions;
public class AzureBlobStorage
{
private BlobServiceClient _blobServiceClient;
public AzureBlobStorage(string connectionString)
{
this.ConnectionString = connectionString;
_blobServiceClient = new BlobServiceClient(this.ConnectionString);
}
public bool IsContainerNameValid(string name)
{
return Regex.IsMatch(name, "^[a-z0-9](?!.*--)[a-z0-9-]{1,61}[a-z0-9]$", RegexOptions.Singleline | RegexOptions.CultureInvariant);
}
public bool ContainerExists(string name)
{
return (IsContainerNameValid(name) ? _blobServiceClient.GetBlobContainerClient(name).Exists() : false);
}
public Azure.Pageable<BlobItem> GetBlobs(string containerName, string prefix = null)
{
try
{
return (ContainerExists(containerName) ?
_blobServiceClient.GetBlobContainerClient(containerName).GetBlobs(BlobTraits.All, BlobStates.All, prefix, default(System.Threading.CancellationToken))
: null);
}
catch
{
throw;
}
}
public bool BlobExists(string containerName, string blobName)
{
try
{
return (from b in GetBlobs(containerName)
where b.Name == blobName
select b).FirstOrDefault() != null;
}
catch
{
throw;
}
}
}
Hopefully this helps someone in the future.
This is the way I do it. Showing full code for those who need it.
// Parse the connection string and return a reference to the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("AzureBlobConnectionString"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("ContainerName");
// Retrieve reference to a blob named "test.csv"
CloudBlockBlob blockBlob = container.GetBlockBlobReference("test.csv");
if (blockBlob.Exists())
{
//Do your logic here.
}
If your blob is public and you need just metadata:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "HEAD";
string code = "";
try
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
code = response.StatusCode.ToString();
}
catch
{
}
return code; // if "OK" blob exists
If you don't like using the exception method then the basic c# version of what judell suggests is below. Beware though that you really ought to handle other possible responses too.
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
myReq.Method = "HEAD";
HttpWebResponse myResp = (HttpWebResponse)myReq.GetResponse();
if (myResp.StatusCode == HttpStatusCode.OK)
{
return true;
}
else
{
return false;
}
With the updated SDK, once you have the CloudBlobReference you can call Exists() on your reference.
See http://msdn.microsoft.com/en-us/library/microsoft.windowsazure.storage.blob.cloudblockblob.exists.aspx
Although most answers here are technically correct, most code samples are making synchronous/blocking calls. Unless you're bound by a very old platform or code base, HTTP calls should always be done asynchonously, and the SDK fully supports it in this case. Just use ExistsAsync() instead of Exists().
bool exists = await client.GetContainerReference(containerName)
.GetBlockBlobReference(key)
.ExistsAsync();
With Azure Blob storage library v12, you can use BlobBaseClient.Exists()/BlobBaseClient.ExistsAsync()
Answered on another similar question: https://stackoverflow.com/a/63293998/4865541
Java version for the same ( using the new v12 SDK )
This uses the Shared Key Credential authorization (account access key)
public void downloadBlobIfExists(String accountName, String accountKey, String containerName, String blobName) {
// create a storage client using creds
StorageSharedKeyCredential credential = new StorageSharedKeyCredential(accountName, accountKey);
String endpoint = String.format(Locale.ROOT, "https://%s.blob.core.windows.net", accountName);
BlobServiceClient storageClient = new BlobServiceClientBuilder().credential(credential).endpoint(endpoint).buildClient();
BlobContainerClient container = storageClient.getBlobContainerClient(containerName);
BlobClient blob = container.getBlobClient(blobName);
if (blob.exists()) {
// download blob
} else {
// do something else
}
}