We Keep Coding

Getting the latest modified file from Azure Blob

How do I get the latest modified file from a Blob?
Here is the container:
I have the following code where I’m trying to get the latest modified file from the container:
using Azure.Identity;
using Azure.Storage.Blobs;
var storageAccountName = "storageAccountName";
var containerName = "containerName"];
var containerUrl = $"https://{storageAccountName}.blob.core.windows.net/{containerName}";
var containerClient = new BlobContainerClient(new Uri(containerUrl), new DefaultAzureCredential());
public async Task GetBlobItems()
{
foreach (var blobItem in containerClient.GetBlobs())
{
var blobClient = _containerClient.GetBlobClient(blobItem.Name);
var blobExists = await blobClient.ExistsAsync();
if (blobExists)
{
var properties = await blobClient.GetPropertiesAsync();
var lastModified = properties.Value.LastModified;
}
}
}
How do I get the latest modified file from 'cache' blob?
Also, there are 2 types of files in cache blob - one with suffix 'delta' and one without suffix 'delta'. How do I get the latest modified files of both these types?

How to retrieve blobs within a blob directory path using the Azure.Storage.Blobs BlobClient?

I'm not seeing any examples online on how to get all the blobs located inside a certain directory within a BlobContainerClient.
Previously, I was using the Microsoft.Azure.Storage packages, but those have since been deprecated. My old code that was scanning a directory for all blobs was:
public async Task<void> ListAllBlobs(string path)
{
var myContainer = await GetCloudBlobClientAsync();
var directory = myContainer.GetDirectoryReference(path);
var blobs = await directory.ListBlobsSegmentedAsync(true, BlobListingDetails.None,
blobSettings.MaxResult, null, null, null);
var results = blobs.Results;
foreach(CloudBlockBlob b in results)
{
// non-relevant code
}
}
private async Task<CloudBlobContainer> GetCloudBlobClientAsync()
{
var storageAccount = CloudStorageAccount.Parse(azureBlobStorageConnectionString);
var blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference(blobStorageSettings.ContainerName);
if (!await container.ExistsAsync())
{
await container.CreateAsync();
}
return container;
}
Essentially, I'm moving the above code from Microsoft.Azure.Storage over to Azure.Storage.Blobs.
If I were to recreate the ListAllBlobs(string path) function to use Azure.Storage.Blobs, I'm confused on how to setup a container and then access an inner container based on a path that's passed in - then cycle through the blobs that exist within that container. Can anyone help?
Here's what I have so far:
public async Task<void> ListAllBlobs(string path)
{
var myContainer = await GetCloudBlobClientAsync();
var directory = myContainer.GetBlobClient(path);
// This doesn't work because I can't do 'GetBlobs' on the Client, only on the container.
foreach(BlobItem blob in directory.GetBlobs(Blobtraits.None, BlobStates.None, string.Empty))
{
// more non-relevant code
}
}
To clarify, in the above code, it doesn't like that I'm calling GetBlobs on a Client, rather than on the Container, but I can't pass in a path to the container.

You were almost there. You would still use BlobContainerClient and call GetBlobsAsync method on that. What you missed is that you will need to set the prefix parameter's value as the path.
So your code would be something like:
var myContainer = await GetCloudBlobClientAsync();
var blobsListingResult = await myContainer.GetBlobsAsync(prefix=path);
UPDATE
Please try the following code:
var myContainer = await GetCloudBlobClientAsync();
await foreach (BlobItem blob in myContainer.GetBlobsAsync(BlobTraits.None, BlobStates.None, path))
{
names.Add(blob.Name);
}

Try this ...
static async Task GetBlobs()
{
string connectionString = "<connection_string>";
string containerName = "<container_name>";
var blobContainerClient = new BlobContainerClient(connectionString, containerName);
var blobs = blobContainerClient.GetBlobs(Azure.Storage.Blobs.Models.BlobTraits.All, Azure.Storage.Blobs.Models.BlobStates.All,
"YourPrefix");
foreach (var blob in blobs)
{
Console.WriteLine(blob.Name);
}
}
... that worked for me.

Programmatically find Partition Key Path of Cosmos Container

var cosmosClient = new CosmosClient(EndpointUrl, AuthorizationKey,
new CosmosClientOptions() {
AllowBulkExecution = true
});
var database = cosmosClient.GetDatabase(SourceDatabase);
var container = database.GetContainer(SourceContainerName);
I'm looking for an approach to programmatically find out
PartitionKeyPath for the container. Is there no API in the SDK to
obtain it from container object?
P.S. Microsoft displays partition key path in Azure portal.
Was wondering how do they display it?

In Azure SDK for .NET, JSON path used for containers partitioning can be found using
ContainerProperties.PartitionKeyPath Property
PartitionKeyPath defaults to "/PartitionKey". Refer: Ms doc
ContainerProperties cproperties = await container.ReadContainerAsync();
Console.WriteLine(cproperties.PartitionKeyPath);
Additionally, you can go ahead and share this feedback so the CosmosDB team can look into this idea.✌

To retrieve the partition key of a container you have to use the GetContainerQueryIterator() method. Depending on what is already available within your code, you could use something like this:
private static async Task<string> GetPartitionKey(Database database, string containerName)
{
var query = new QueryDefinition("select * from c where c.id = #id")
.WithParameter("#id", containerName);
using var iterator = database.GetContainerQueryIterator<ContainerProperties>(query);
while (iterator.HasMoreResults)
{
foreach (var container in await iterator.ReadNextAsync())
{
return container.PartitionKeyPath;
}
}
return null;
}

In version 3.31.2 you can access the PartitionKeyPath as follows
var containerResponse = await cosmosClient.GetDatabase(options.Value.Database)
.GetContainer(collectionId)
.ReadContainerAsync(cancellationToken: cancellationToken);
Console.WriteLine(containerResponse.Resource.PartitionKeyPath)

How to copy whole Container from one Azure Blob Service to another with Azure Client Library v12

I'm trying my hand at programmatic Azure tooling, specifically trying to copy/clone/move a Container from one Azure Blob Storage account to another one. I'm not seeing a good way to do this though with v12.x of the client library in its docs (so far at least).
For example:
BlobStorage1
|-- SomeContrainer
|-- blob1
|-- blob2
|-- blob3
BlobStorage2
|-- SomeOtherContrainer
|-- otherBlob
I want to programmatically move SomeContainer and all its blobs to BlobStorage2.
What I've tried so far:
docs read/consulated:
StartCopyFrom method
which doesn't seem to show how to go from account1 pushed to account2. You can pull down from account2 into account one but that's the opposite of what I'm after.
code attempt:
var localContainer = blobServiceClient.GetBlobContainerClient(localContainerName);
var blobs = localContainer.GetBlobs();
var remoteClient = new BlobServiceClient(remoteConnectionString);
var remoteContainer = remoteClient.GetBlobContainerClient(localContainerName);
foreach(var blob in blobs)
{
Console.WriteLine($"copying blob: {blob.Name}");
var sourceBlob = localContainer.GetBlobClient(blob.Name);
var remoteBlobClient = remoteContainer.GetBlobClient(blob.Name);
await remoteBlobClient.StartCopyFromUriAsync(sourceBlob.Uri);
}
Problem here is that I could copy from remote to local (via connection string), or within the same account since the URI would be quite similar, but not from the account I'm on to a separate storage account. What would be the recommended way to copy blobs (or containers whole sale if possible) with the client library (v12.x) from one account to a separate one?

Please try the code below:
static async Task CopyContainersAcrossAccounts()
{
var sourceAccountName = "source-account";
var sourceAccountKey = "source-account-key";
var targetAccountName = "target-account";
var targetAccountKey = "target-account-key";
Azure.Storage.StorageSharedKeyCredential sourceCredential = new Azure.Storage.StorageSharedKeyCredential(sourceAccountName, sourceAccountKey);
Azure.Storage.StorageSharedKeyCredential targetCredential = new Azure.Storage.StorageSharedKeyCredential(targetAccountName, targetAccountKey);
var sourceConnectionString = $"DefaultEndpointsProtocol=https;AccountName={sourceAccountName};AccountKey={sourceAccountKey};EndpointSuffix=core.windows.net;";
var sourceContainer = "source-container-name";
var targetConnectionString = $"DefaultEndpointsProtocol=https;AccountName={targetAccountName};AccountKey={targetAccountKey};EndpointSuffix=core.windows.net;";
var targetContainer = "target-container-name";
var sourceBlobContainerClient = new Azure.Storage.Blobs.BlobContainerClient(sourceConnectionString, sourceContainer);
var targetBlobContainerClient = new Azure.Storage.Blobs.BlobContainerClient(targetConnectionString, targetContainer);
var sourceBlobs = sourceBlobContainerClient.GetBlobs();
foreach (var blob in sourceBlobs)
{
//Get shared access signature with "Read" permission in case source container is a private container.
Azure.Storage.Sas.BlobSasBuilder blobSasBuilder = new Azure.Storage.Sas.BlobSasBuilder()
{
BlobContainerName = sourceContainer,
BlobName = blob.Name,
ExpiresOn = DateTime.UtcNow.AddHours(1)
};
blobSasBuilder.SetPermissions(Azure.Storage.Sas.BlobSasPermissions.Read);
var sasToken = blobSasBuilder.ToSasQueryParameters(sourceCredential).ToString();
var sourceBlobClient = sourceBlobContainerClient.GetBlobClient(blob.Name);
var targetBlobClient = targetBlobContainerClient.GetBlobClient(blob.Name);
var sourceBlobUri = new Uri($"{sourceBlobClient.Uri.AbsoluteUri}?{sasToken}");
await targetBlobClient.StartCopyFromUriAsync(sourceBlobUri);
}
}

How to get only folder name in blob using new Azure.Storage.Blobs namespace

I want to get all the folder name under a container using new SDK - Azure.Storage.Blobs

You would want to use GetBlobsByHierarchy method in BlobContainerClient class and pass "/" as prefix. The method will return all the blobs and folders at the root level. Once you get that, you will simply need to filter out the blobs by selecting only the items from the result where IsPrefix property is true (or you could check for Blob property to be null).
Please try something like the following:
var connectionString = "DefaultEndpointsProtocol=https;AccountName=<account-name>;AccountKey=<account-key>;EndpointSuffix=core.windows.net;";
var containerName = "test";
var containerClient = new BlobContainerClient(connectionString, containerName);
var blobFolders = containerClient.GetBlobsByHierarchy(BlobTraits.None, BlobStates.None, "/").Where(b => b.IsPrefix).ToList();
for (var i=0; i<blobFolders.Count; i++)
{
Console.WriteLine("Folder Prefix: " + blobFolders[i].Prefix);
}