We Keep Coding

How do I iterate CloudBlobDirectory to copy the data?

I have written below code to iterate through the Gen2 storage blob
CloudStorageAccount sourceAccount = CloudStorageAccount.Parse(sourceConnection);
CloudStorageAccount destAccount = CloudStorageAccount.Parse(destConnection);
CloudBlobClient sourceClient = sourceAccount.CreateCloudBlobClient();
CloudBlobClient destClient = destAccount.CreateCloudBlobClient();
CloudBlobContainer sourceBlobContainer = sourceClient.GetContainerReference(sourceContainer);
// Find all blobs that haven't changed since the specified date and time
IEnumerable<ICloudBlob> sourceBlobRefs = FindMatchingBlobsAsync(sourceBlobContainer, transferBlobsNotModifiedSince).Result;
private static async Task<IEnumerable<ICloudBlob>> FindMatchingBlobsAsync(CloudBlobContainer blobContainer, DateTime transferBlobsNotModifiedSince)
{
List<ICloudBlob> blobList = new List<ICloudBlob>();
BlobContinuationToken token = null;
// Iterate through the blobs in the source container
do
{
BlobResultSegment segment = await blobContainer.ListBlobsSegmentedAsync(prefix: "", currentToken: token);
foreach (CloudBlobDirectory VARIABLE in segment.Results)
{
BlobResultSegment segment2 = await VARIABLE.ListBlobsSegmentedAsync(currentToken: token);
foreach (CloudBlobDirectory VARIABLE2 in segment2.Results)//Bad coding
{
//how do I get children count ?
}
}
}while (token != null);
}
This will iterate only 2 levels but not dynamically till the inner levels. I have blob in below hierarchy
--Container
--FolderA
--FolderAA
--FolderAA1
--File1.txt
--File2.txt
--FolderAA2
--File1.txt
--File2.txt
--FolderAA3
--FolderAB
--File8.txt
--FolderAC
--File9.txt
This hierarchy is dynamic
How do I loop and copy the blob content.
Note: I do not want to use CLI commands to copy. Because I won't have any control once copy started.
Update
Found some samples here: https://csharp.hotexamples.com/examples/Microsoft.WindowsAzure.Storage.Blob/CloudBlobContainer/ListBlobsSegmented/php-cloudblobcontainer-listblobssegmented-method-examples.html

Please see the sample code below:
class Program
{
static void Main(string[] args)
{
var storageAccount = CloudStorageAccount.Parse("UseDevelopmentStorage=true");
var client = storageAccount.CreateCloudBlobClient();
var container = client.GetContainerReference("test");
var blobs = FindMatchingBlobsAsync(container).GetAwaiter().GetResult();
foreach (var blob in blobs)
{
Console.WriteLine(blob.Name);
}
Console.WriteLine("-------------------------------------");
Console.WriteLine("List of all blobs fetched. Press any key to terminate the application.");
Console.ReadKey();
}
private static async Task<IEnumerable<ICloudBlob>> FindMatchingBlobsAsync(CloudBlobContainer blobContainer)
{
List<ICloudBlob> blobList = new List<ICloudBlob>();
BlobContinuationToken token = null;
// Iterate through the blobs in the source container
do
{
BlobResultSegment segment = await blobContainer.ListBlobsSegmentedAsync(prefix: "", useFlatBlobListing: true, BlobListingDetails.None, 5000, token, new BlobRequestOptions(), new OperationContext());
token = segment.ContinuationToken;
foreach(var item in segment.Results)
{
blobList.Add((ICloudBlob)item);
}
} while (token != null);
return blobList;
}
}

How to get an image from Azure Blob Storage via web api as binary?

I have a webapi method so that users can upload their own profile picture, however in my entity which is saved in CosmosDB I save the profile picture URL.
public async Task<IHttpActionResult> UpdateSuperAdministrator(SuperAdministrator superadministrator)
{
var telemetry = new TelemetryClient();
try
{
var superAdministratorStore = CosmosStoreHolder.Instance.CosmosStoreSuperAdministrator;
//First we validate the model
if (!ModelState.IsValid)
{
return BadRequest(ModelState);
}
//Then we validate the content type
if (!Request.Content.IsMimeMultipartContent("form-data"))
{
throw new HttpResponseException(HttpStatusCode.UnsupportedMediaType);
}
//Initalize configuration settings
var accountName = ConfigurationManager.AppSettings["storage:account:name"];
var accountKey = ConfigurationManager.AppSettings["storage:account:key"];
var profilepicturecontainername = ConfigurationManager.AppSettings["storage:account:profilepicscontainername"];
//Instance objects needed to store the files
var storageAccount = new CloudStorageAccount(new StorageCredentials(accountName, accountKey), true);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer imagesContainer = blobClient.GetContainerReference(profilepicturecontainername);
var provider = new AzureStorageMultipartFormDataStreamProvider(imagesContainer);
foreach (MultipartFileData file in provider.FileData)
{
var fileName = file.Headers.ContentDisposition.FileName.Trim('\"').Trim();
if (fileName.EndsWith(".png"))
{
var img = Image.FromFile(file.LocalFileName);
if (img.Width != 200 && img.Height != 200)
{
string guid = Guid.NewGuid().ToString();
return BadRequest($"Error Lulo. Unsupported extension, only PNG is valid. Or unsuported image dimensions (200px x 200px)");
}
}
}
//Try to upload file
try
{
await Request.Content.ReadAsMultipartAsync(provider);
}
catch (Exception ex)
{
string guid = Guid.NewGuid().ToString();
var dt = new Dictionary<string, string>
{
{ "Error Lulo: ", guid }
};
telemetry.TrackException(ex, dt);
return BadRequest($"Error Lulo. An error has occured. Details: {guid} {ex.Message}: ");
}
// Retrieve the filename of the file you have uploaded
var filename = provider.FileData.FirstOrDefault()?.LocalFileName;
if (string.IsNullOrEmpty(filename))
{
string guid = Guid.NewGuid().ToString();
var dt = new Dictionary<string, string>
{
{ "Error Lulo: ", guid }
};
return BadRequest($"Error Lulo. An error has occured while uploading your file. Please try again.: {guid} ");
}
//Rename file
CloudBlockBlob blobCopy = imagesContainer.GetBlockBlobReference(superadministrator.Id + ".png");
if (!await blobCopy.ExistsAsync())
{
CloudBlockBlob blob = imagesContainer.GetBlockBlobReference(filename);
if (await blob.ExistsAsync())
{
await blobCopy.StartCopyAsync(blob);
await blob.DeleteIfExistsAsync();
}
}
superadministrator.ProfilePictureUrl = blobCopy.Name;
var result = await superAdministratorStore.UpdateAsync(superadministrator);
return Ok(result);
}
catch (Exception ex)
{
string guid = Guid.NewGuid().ToString();
var dt = new Dictionary<string, string>
{
{ "Error Lulo: ", guid }
};
telemetry.TrackException(ex, dt);
return BadRequest("Error Lulo: " + guid);
}
}
Now my question is about the endpoind GET. Because the profile picture needs to be shown in the navigation bar on the app, I need to get it in binary format, if that makes sense?
In other words, I cant return the ProfilePicture URL to the client (a React App), because its a client app which does not have access to the blob storage.
Here is my get.
[HttpGet]
public async Task<IHttpActionResult> GetSuperAdministrator(string email)
{
var telemetry = new TelemetryClient();
try
{
var superAdministratorStore = CosmosStoreHolder.Instance.CosmosStoreSuperAdministrator;
var superadministrator = await superAdministratorStore.Query().FirstOrDefaultAsync(x => x.EmailAddress == email);
if (superadministrator == null)
{
return NotFound();
}
return Ok(superadministrator);
}
catch (Exception ex)
{
string guid = Guid.NewGuid().ToString();
var dt = new Dictionary<string, string>
{
{ "Error Lulo: ", guid }
};
telemetry.TrackException(ex, dt);
return BadRequest("Error Lulo: " + guid);
}
}

Based on my knowledge, you React App will show a html page, which means that there will be a html <img src="url"> in the html. Then the browser will try to get the image from the url.
So, there would be 2 optional solutions:
1.Generate a storage blob url with SAS, and set it as the image's url in <img>. With SAS, a clinet will be able to access the image from storage directly.
2.The url would be the request path of yout Web API. For example: /image/{image_name}. And your web api will return a image:
[HttpGet]
public IActionResult GetImage()
{
// Get the image blob
CloudBlockBlob cloudBlockBlob = ********;
using(MemoryStream ms = new MemoryStream())
{
cloudBlockBlob.DownloadToStream(ms);
return File(ms.ToArray(), "image/jpeg");
}
}
Update:
My code:
public IActionResult Index()
{
string connString = "DefaultEndpointsProtocol=https;AccountName=storagetest789;AccountKey=G36mcmEthM****=;EndpointSuffix=core.windows.net";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(connString);
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference("pub");
CloudBlockBlob cloudBlockBlob = cloudBlobContainer.GetBlockBlobReference("p033tw9j.jpg");
using (MemoryStream ms = new MemoryStream())
{
cloudBlockBlob.DownloadToStream(ms);
return File(ms.ToArray(), "image/jpeg");
}
}
Screenshot:

Method not found: 'Boolean Microsoft.WindowsAzure.Storage.Blob.CloudBlobContainer.CreateIfNotExists

I'm trying to create and delete blobs with the following code. Why am I getting this error? CreateIfNotExists - Method Not Found. I get the same when I try and ListBlobs.
What am I missing? What is the correct version of Microsoft.WindowsAzure.Storage
Here's my code
var storageAccount = CloudStorageAccount.Parse(BlobConnString);
var myClient = storageAccount.CreateCloudBlobClient();
var container = myClient.GetContainerReference(Mycontainer);
container.CreateIfNotExists(BlobContainerPublicAccessType.Blob);
foreach (IListBlobItem blobItem in container.ListBlobs())
{
if (blobItem is CloudBlobDirectory)
{
CloudBlobDirectory directory = (CloudBlobDirectory)blobItem;
if (directory.Uri.AbsoluteUri.Contains(NIPRprocfolder))
{
IEnumerable<IListBlobItem> blobs = directory.ListBlobs(true);
ICloudBlob bi;
foreach (var blob in blobs)
{
if (blob is CloudBlockBlob)
{
bi = blob as CloudBlockBlob;
if (bi.Name.Contains(".xml"))
{
_logger.Info($"Deleting report : {bi.Name} from {NIPRprocfolder}");
bi.DeleteIfExists();
}
}
}
}
}
}
thx

failure deleting blobs from azure storage

I am trying to delete blobs from container. The DeleteIfExits returns true but nothing happens. I check the container using Azure's portal and I can still see the blobs.
What is wrong with my code?
private static void DeleteAllFilesWithSameName(String filePath, String filename, CloudBlobContainer container)
{
String filenameWidthoutExtension = Path.GetFileNameWithoutExtension(filename);
try
{
IEnumerable<IListBlobItem> blobs = container.ListBlobs(filenameWidthoutExtension, true);
if (blobs.Count<IListBlobItem>() > 0)
{
List<string> blobNames = blobs.OfType<CloudBlockBlob>().Select(b => b.Name).ToList();
foreach (String blobName in blobNames)
{
CloudBlockBlob blockBlob = container.GetBlockBlobReference(filename);
bool isDeleted = blockBlob.DeleteIfExists();
}
}
}
catch (Exception e)
{
Console.Write(e.Data);
}
}

I believe the issue is with the logic in your code:
CloudBlockBlob blockBlob = container.GetBlockBlobReference(filename);
Shouldn't the above line be?
CloudBlockBlob blockBlob = container.GetBlockBlobReference(blobName);
Please try by changing your code to:
foreach (String blobName in blobNames)
{
CloudBlockBlob blockBlob = container.GetBlockBlobReference(blobName);
bool isDeleted = blockBlob.DeleteIfExists();
}

how to search for a specific file in blob storage

I am kind of new to Blob storage and i need to access a specific file from blob storage.
i.e when i type in a specific folder it should list out all the blobs underneath it.
Can anyone help me with that
here's the code which i am trying to do.
if (AccountFileTransfer != null)
{
BlobClientFileTransfer = AccountFileTransfer.CreateCloudBlobClient();
ContainerFileTransfer = BlobClientFileTransfer.GetContainerReference(CONTAINER);
CloudBlob blob = ContainerFileTransfer.GetBlobReference(txtFileSearch.Text);
if (blob.Uri == null)
{
System.Windows.Forms.MessageBox.Show("Not a Valid blob search");
}
else
{
lvFileTransfer.Items.Add(blob.Uri);
}
}

Use Azure Search to index and search for files in Blob storage

Try this
if (AccountFileTransfer != null)
{
CloudBlobClient blobClient =
new CloudBlobClient(blobEndpoint,
new StorageCredentialsAccountAndKey(accountName, accountKey));
CloudBlobContainer container = blobClient.GetContainerReference(CONTAINER);
foreach (var blobItem in container .ListBlobs())
{
lvFileTransfer.Items.Add(blobItem .Uri);
}
}

Try this and If the blob in a Directory in container, in that case following format
container.GetBlobReference("Images/" + fileName);
public static bool BlobExists(CloudBlobContainer container, string fileName)
{
var blob = container.GetBlobReference(fileName);
try
{
blob.FetchAttributes();
return true;
}
catch (StorageException e)
{
if (e.RequestInformation.HttpStatusCode == (int)HttpStatusCode.NotFound)
{
return false;
}
}
return false;
}