We Keep Coding

How to retrieve blobs within a blob directory path using the Azure.Storage.Blobs BlobClient?

I'm not seeing any examples online on how to get all the blobs located inside a certain directory within a BlobContainerClient.
Previously, I was using the Microsoft.Azure.Storage packages, but those have since been deprecated. My old code that was scanning a directory for all blobs was:
public async Task<void> ListAllBlobs(string path)
{
var myContainer = await GetCloudBlobClientAsync();
var directory = myContainer.GetDirectoryReference(path);
var blobs = await directory.ListBlobsSegmentedAsync(true, BlobListingDetails.None,
blobSettings.MaxResult, null, null, null);
var results = blobs.Results;
foreach(CloudBlockBlob b in results)
{
// non-relevant code
}
}
private async Task<CloudBlobContainer> GetCloudBlobClientAsync()
{
var storageAccount = CloudStorageAccount.Parse(azureBlobStorageConnectionString);
var blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference(blobStorageSettings.ContainerName);
if (!await container.ExistsAsync())
{
await container.CreateAsync();
}
return container;
}
Essentially, I'm moving the above code from Microsoft.Azure.Storage over to Azure.Storage.Blobs.
If I were to recreate the ListAllBlobs(string path) function to use Azure.Storage.Blobs, I'm confused on how to setup a container and then access an inner container based on a path that's passed in - then cycle through the blobs that exist within that container. Can anyone help?
Here's what I have so far:
public async Task<void> ListAllBlobs(string path)
{
var myContainer = await GetCloudBlobClientAsync();
var directory = myContainer.GetBlobClient(path);
// This doesn't work because I can't do 'GetBlobs' on the Client, only on the container.
foreach(BlobItem blob in directory.GetBlobs(Blobtraits.None, BlobStates.None, string.Empty))
{
// more non-relevant code
}
}
To clarify, in the above code, it doesn't like that I'm calling GetBlobs on a Client, rather than on the Container, but I can't pass in a path to the container.

You were almost there. You would still use BlobContainerClient and call GetBlobsAsync method on that. What you missed is that you will need to set the prefix parameter's value as the path.
So your code would be something like:
var myContainer = await GetCloudBlobClientAsync();
var blobsListingResult = await myContainer.GetBlobsAsync(prefix=path);
UPDATE
Please try the following code:
var myContainer = await GetCloudBlobClientAsync();
await foreach (BlobItem blob in myContainer.GetBlobsAsync(BlobTraits.None, BlobStates.None, path))
{
names.Add(blob.Name);
}

Try this ...
static async Task GetBlobs()
{
string connectionString = "<connection_string>";
string containerName = "<container_name>";
var blobContainerClient = new BlobContainerClient(connectionString, containerName);
var blobs = blobContainerClient.GetBlobs(Azure.Storage.Blobs.Models.BlobTraits.All, Azure.Storage.Blobs.Models.BlobStates.All,
"YourPrefix");
foreach (var blob in blobs)
{
Console.WriteLine(blob.Name);
}
}
... that worked for me.

Iterating through blobs in Azure SDK v12

I am upgrading a web app to use Azure SDK v 12.10
Before this, I was grabbing all the blobs in a virtual path from the container into a list and iterating through them.
var results = await blobContainer.
ListBlobsSegmentedAsync("publicfiles/images/" + customer.AzureFolderName, true, BlobListingDetails.All, 100, blobContinuationToken, null, null);
// Get the value of the continuation token returned by the listing call.
blobContinuationToken = results.ContinuationToken;
foreach (var item in results.Results)
{
var filename = GetFileNameFromBlobUri(item.Uri, customer.AzureFolderName);
var img = new EvaluationImage
{
ImageUrl = item.Uri.ToString(),
ImageCaption = GetCaptionFromFilename(filename),
IsPosterImage = filename.Contains("poster"),
ImagesForCompany = customer.CompanyName
};
images.Add(img);
}
All I can find from googling how to do so in the new SDK, yields this
await foreach (BlobItem blob in blobContainer.GetBlobsAsync(BlobTraits.None, BlobStates.None, $"publicfiles/images/{customer.AzureFolderName}"))
The problem is the "blob" variable (BlobItem) has minimal useable properties, for example, I need the Uri which was available before. The "blob" variable in prior versions had a multitude of useable properties. Is there some other type (not BlobItem) that has these properties that I'm not finding in my searching?
I don't see anything useable in BlobTraits or BlobStates either that change this.

It's not as straight forward to get the URI of the blob in SDK version 12.x. When you get a BlobItem as part of listing result, you will need to create a BlobClient object out of it and then you will be able to get the URI of the blob.
Here's the sample code to do so:
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(container);
var listingResult = containerClient.GetBlobsAsync();
await foreach (var blobItem in listingResult)
{
BlobClient blobClient = containerClient.GetBlobClient(blobItem.Name);
Uri blobUri = blobClient.Uri;
//Do something with the URI...
}
Code above makes use of Azure.Storage.Blobs (12.9.1) SDK.

Programmatically find Partition Key Path of Cosmos Container

var cosmosClient = new CosmosClient(EndpointUrl, AuthorizationKey,
new CosmosClientOptions() {
AllowBulkExecution = true
});
var database = cosmosClient.GetDatabase(SourceDatabase);
var container = database.GetContainer(SourceContainerName);
I'm looking for an approach to programmatically find out
PartitionKeyPath for the container. Is there no API in the SDK to
obtain it from container object?
P.S. Microsoft displays partition key path in Azure portal.
Was wondering how do they display it?

In Azure SDK for .NET, JSON path used for containers partitioning can be found using
ContainerProperties.PartitionKeyPath Property
PartitionKeyPath defaults to "/PartitionKey". Refer: Ms doc
ContainerProperties cproperties = await container.ReadContainerAsync();
Console.WriteLine(cproperties.PartitionKeyPath);
Additionally, you can go ahead and share this feedback so the CosmosDB team can look into this idea.✌

To retrieve the partition key of a container you have to use the GetContainerQueryIterator() method. Depending on what is already available within your code, you could use something like this:
private static async Task<string> GetPartitionKey(Database database, string containerName)
{
var query = new QueryDefinition("select * from c where c.id = #id")
.WithParameter("#id", containerName);
using var iterator = database.GetContainerQueryIterator<ContainerProperties>(query);
while (iterator.HasMoreResults)
{
foreach (var container in await iterator.ReadNextAsync())
{
return container.PartitionKeyPath;
}
}
return null;
}

In version 3.31.2 you can access the PartitionKeyPath as follows
var containerResponse = await cosmosClient.GetDatabase(options.Value.Database)
.GetContainer(collectionId)
.ReadContainerAsync(cancellationToken: cancellationToken);
Console.WriteLine(containerResponse.Resource.PartitionKeyPath)

How to copy whole Container from one Azure Blob Service to another with Azure Client Library v12

I'm trying my hand at programmatic Azure tooling, specifically trying to copy/clone/move a Container from one Azure Blob Storage account to another one. I'm not seeing a good way to do this though with v12.x of the client library in its docs (so far at least).
For example:
BlobStorage1
|-- SomeContrainer
|-- blob1
|-- blob2
|-- blob3
BlobStorage2
|-- SomeOtherContrainer
|-- otherBlob
I want to programmatically move SomeContainer and all its blobs to BlobStorage2.
What I've tried so far:
docs read/consulated:
StartCopyFrom method
which doesn't seem to show how to go from account1 pushed to account2. You can pull down from account2 into account one but that's the opposite of what I'm after.
code attempt:
var localContainer = blobServiceClient.GetBlobContainerClient(localContainerName);
var blobs = localContainer.GetBlobs();
var remoteClient = new BlobServiceClient(remoteConnectionString);
var remoteContainer = remoteClient.GetBlobContainerClient(localContainerName);
foreach(var blob in blobs)
{
Console.WriteLine($"copying blob: {blob.Name}");
var sourceBlob = localContainer.GetBlobClient(blob.Name);
var remoteBlobClient = remoteContainer.GetBlobClient(blob.Name);
await remoteBlobClient.StartCopyFromUriAsync(sourceBlob.Uri);
}
Problem here is that I could copy from remote to local (via connection string), or within the same account since the URI would be quite similar, but not from the account I'm on to a separate storage account. What would be the recommended way to copy blobs (or containers whole sale if possible) with the client library (v12.x) from one account to a separate one?

Please try the code below:
static async Task CopyContainersAcrossAccounts()
{
var sourceAccountName = "source-account";
var sourceAccountKey = "source-account-key";
var targetAccountName = "target-account";
var targetAccountKey = "target-account-key";
Azure.Storage.StorageSharedKeyCredential sourceCredential = new Azure.Storage.StorageSharedKeyCredential(sourceAccountName, sourceAccountKey);
Azure.Storage.StorageSharedKeyCredential targetCredential = new Azure.Storage.StorageSharedKeyCredential(targetAccountName, targetAccountKey);
var sourceConnectionString = $"DefaultEndpointsProtocol=https;AccountName={sourceAccountName};AccountKey={sourceAccountKey};EndpointSuffix=core.windows.net;";
var sourceContainer = "source-container-name";
var targetConnectionString = $"DefaultEndpointsProtocol=https;AccountName={targetAccountName};AccountKey={targetAccountKey};EndpointSuffix=core.windows.net;";
var targetContainer = "target-container-name";
var sourceBlobContainerClient = new Azure.Storage.Blobs.BlobContainerClient(sourceConnectionString, sourceContainer);
var targetBlobContainerClient = new Azure.Storage.Blobs.BlobContainerClient(targetConnectionString, targetContainer);
var sourceBlobs = sourceBlobContainerClient.GetBlobs();
foreach (var blob in sourceBlobs)
{
//Get shared access signature with "Read" permission in case source container is a private container.
Azure.Storage.Sas.BlobSasBuilder blobSasBuilder = new Azure.Storage.Sas.BlobSasBuilder()
{
BlobContainerName = sourceContainer,
BlobName = blob.Name,
ExpiresOn = DateTime.UtcNow.AddHours(1)
};
blobSasBuilder.SetPermissions(Azure.Storage.Sas.BlobSasPermissions.Read);
var sasToken = blobSasBuilder.ToSasQueryParameters(sourceCredential).ToString();
var sourceBlobClient = sourceBlobContainerClient.GetBlobClient(blob.Name);
var targetBlobClient = targetBlobContainerClient.GetBlobClient(blob.Name);
var sourceBlobUri = new Uri($"{sourceBlobClient.Uri.AbsoluteUri}?{sasToken}");
await targetBlobClient.StartCopyFromUriAsync(sourceBlobUri);
}
}

How to get only folder name in blob using new Azure.Storage.Blobs namespace

I want to get all the folder name under a container using new SDK - Azure.Storage.Blobs

You would want to use GetBlobsByHierarchy method in BlobContainerClient class and pass "/" as prefix. The method will return all the blobs and folders at the root level. Once you get that, you will simply need to filter out the blobs by selecting only the items from the result where IsPrefix property is true (or you could check for Blob property to be null).
Please try something like the following:
var connectionString = "DefaultEndpointsProtocol=https;AccountName=<account-name>;AccountKey=<account-key>;EndpointSuffix=core.windows.net;";
var containerName = "test";
var containerClient = new BlobContainerClient(connectionString, containerName);
var blobFolders = containerClient.GetBlobsByHierarchy(BlobTraits.None, BlobStates.None, "/").Where(b => b.IsPrefix).ToList();
for (var i=0; i<blobFolders.Count; i++)
{
Console.WriteLine("Folder Prefix: " + blobFolders[i].Prefix);
}