Copy Azure CloudPageBlob to CloudBlockBlob - c#

Have a load of Azure page blobs in an Azure storage account which need copying to block blobs so they can be set from "cold" to "Archive" storage tier. The page blobs are created by SQL server (pre 2016) BACKUP TO URL.
I spoke to Microsoft about this and they suggested AzCopy with Azure File blob as a stepping stone, e.g. Page blob -> block blob (file storage) -> block blob (Blob storage). This all works fine in AzCopy or GUI based Azure tools but I need to automate it and build in some resilience.
Problem I have is I cannot get a CloudPageBlob to copy to CloudBlockBlob
not matter how I cast it. This is a cut-down example to show (lack of) progress and how I'm going about this. I will wrap this in a Task<> in production but I cannot get the .StartCopy to work. Any ideas?
CredentialEntity ce = Utils.GetBlobCredentials();
StorageCredentials sc = new StorageCredentials(ce.Name, ce.Key);
CloudStorageAccount storageAccount =
new CloudStorageAccount(new StorageCredentials(ce.Name, ce.Key), true);
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer sourceContainer = cloudBlobClient.GetContainerReference(#"archive");
CloudBlobContainer targetContainer = cloudBlobClient.GetContainerReference(#"xfer");
CloudPageBlob source = sourceContainer.GetPageBlobReference("test.bak");
CloudBlockBlob target = targetContainer.GetBlockBlobReference("new.bak");
//target.StartCopy(source);
//target.StartCopyAsync()

Related

Create container in Azure storage with c#

I have the following code:
var cloudStorageAccount = CloudStorageAccount.Parse(this._appSettings.BlobConnectionString);
this._cloudBlobClient = cloudStorageAccount.CreateCloudBlobClient();
CloudBlobContainer container = this._cloudBlobClient.GetContainerReference("dpd/textures/test");
await container.CreateIfNotExistsAsync();
I want to create the "test" directory inside dpd/textures/, but I keep getting this error:
The requested URI does not represent any resource on the server azure
container
What am I doing wrong?
Actually, there is no real directory in blob storage. The directory(not container) is always a part of the blob's name.
So you can just create a container like "dpd", then you can use any upload methods to upload a file like myfile.txt. During uploading file, you can specify your file name as "textures/test/myfile.txt" in your GetBlockBlobReference method. Then the directory "textures/test" is created automatically.
Simple code:
CloudBlobClient cloudBlobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer cloudBlobContainer = cloudBlobClient.GetContainerReference("test1");
var myblob = cloudBlobContainer.GetBlockBlobReference("textures/test/myfile.txt");
myblob.UploadFromFile("your file path");
Test result:
And also note that, if the blob(myfile.txt) is deleted, then the directory "textures/test" will also be removed, since it's a part name of the blob.

c# Azure Cannot set the blob tier

CloudBlockBlob doesn't have any method to set the blob tier to hot/cool/archive. I have also checked the other blob types and they do not have a method that allows this either.
I.E this method: https://learn.microsoft.com/en-us/rest/api/storageservices/set-blob-tier
Is their any way to change the blob tier in code from hot to cold in C# with azure storage?
I think the method is exactly what you need: CloudBlockBlob.SetStandardBlobTier. Maybe you were not checking the latest version of Azure Storage Client Library?
Is their any way to change the blob tier in code from hot to cold in C# with azure storage?
As ZhaoXing Lu mentioned that we could use CloudBlockBlob.SetStandardBlobTier.
Note: The operation is allowed on a page blob in a premium storage account and on a block blob in a blob storage account
The following code works correctly on my side. I use the library WindowsAzure.Storage 9.1.1
var cloudBlobClient = storageAccount.CreateCloudBlobClient();
var container = cloudBlobClient.GetContainerReference("container");
var blob = container.GetBlockBlobReference("blob name");
blob.SetStandardBlobTier(StandardBlobTier.Cool);
blob.FetchAttributes();
var tier = blob.Properties.StandardBlobTier;
Using Azure Blob storage client library v12 for .NET, replace myaccount with the name of your storage account, mycontainer with your container name and myblob with the blob name for which the tier is to be changed:
var sharedKeyCredential = new StorageSharedKeyCredential("myaccount", storageAccountKey);
var baseBlobContainerUrl = string.Format("{0}.blob.core.windows.net", "myaccount");
var blobServiceClient = new BlobServiceClient(new Uri($"https://{baseBlobContainerUrl}"), sharedKeyCredential);
var containerClient = blobServiceClient.GetBlobContainerClient("mycontainer");
BlobClient blobClient = containerClient.GetBlobClient("myblob");
// Set access tier to cool.
await blobClient.SetAccessTierAsync(AccessTier.Cool);
If you are working with Azure Gov, use this url insteadl "{0}.blob.core.usgovcloudapi.net"
Keep in mind, your storage account should support Cool Storage.

Unable to access Azure blob in LocalStorage via its URL

I'm creating a Web Application as part of an assignment I have. The task is to upload an audio clip to an ASP page and then a Web Job will then generate from it a 20 second sample which is then displayed on the same page. I will eventually need to deploy it to Azure but at the moment I'm using local storage with the Azure Storage Emulator.
The page loads correctly and the audio is uploading to the Blob Container okay and the samples are being generated and placed in the correct folder. However the samples aren't loading into the player and I'm not able to access these samples through the browser, even when using the URL I'm reading right off the blobs in storage.
When I enter that URL into the browser (with the Application and Web Job both running) I get this message displayed:
My first instinct was that I didn't have the necessary permissions to access the container. However as far as I can tell the permissions have been configured correctly. Below is my BlobStorageService class that handles the permissions for the containers:
namespace Sampler
{
public class BlobStorageService
{
public CloudBlobContainer getCloudBlobContainer()
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse
(ConfigurationManager.ConnectionStrings["AzureStorage"].ToString());
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("audiostorage");
if (container.CreateIfNotExists())
{
BlobContainerPermissions permissions = container.GetPermissions();
permissions.PublicAccess = BlobContainerPublicAccessType.Container;
container.SetPermissions(permissions);
}
return container;
}
}
So as far as I can tell the permissions should be Public for the whole container and the code I used for it came directly from Microsoft Docs. Any suggestions for next course of action greatly appreciated and happy to post any more code you might need to see.
It seems that there is logic issue in your code. If the container is existed, then the permisson of the container will not be public, default is private. If we try to list the blob in the private container, will get not found issue. Please have a try to use the following code. Or we could use the Microsoft Azure Storage Exploer to set the container public and try it again.
public CloudBlobContainer getCloudBlobContainer()
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(ConfigurationManager.ConnectionStrings["AzureStorage"].ToString());
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("audiostorage");
container.CreateIfNotExists(); // remove the if condition
BlobContainerPermissions permissions = container.GetPermissions();
permissions.PublicAccess = BlobContainerPublicAccessType.Container;
container.SetPermissions(permissions);
return container;
}

Uploading a png file to Azure Blob Storage is losing transparency

We are having some difficulty to figure out how upload png files to Azure Blob Storage without losing image transparency.
We are using the Azure's Storage SDK with the code below:
var storageAccount = CloudStorageAccount.Parse(*blob_connection_string*);
//create client to work with blobs
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
//already created container via azure management portal, set container reference
CloudBlobContainer container = blobClient.GetContainerReference("brand");
foreach (string file in httpRequest.Files)
{
var postedFile = httpRequest.Files[file];
CloudBlockBlob blockBlob = container.GetBlockBlobReference(postedFile.FileName);
blockBlob.UploadFromStream(postedFile.InputStream);
}
We've tried force the ContentType property before upload like this:
blockBlob.Properties.ContentType = "image/png";
But doesn't work neither with or without this property.
Well, for those who are experiencing this issue, we've found a solution:
The problem was with using Microsoft .Net Image object. According MSDN this object work only with PNG 32 bits:
https://msdn.microsoft.com/pt-br/library/1kcb3wy4%28v=vs.110%29.aspx?f=255
We've changed the upload routine to get the posted file binary direct from the input and work like a charm!
Thanks again for the ideas who drive us to this solution!
Hugs

How to upload List of images on azure and retrieve addresses of uploaded images

I am doing Mvc asp.net project. i want to upload four images on azure blob storage, i am uploading one image at a time using following method
public static string UploadToBlob(string fileName, byte[] data)
{
MemoryStream file = new MemoryStream(data);
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(
CloudConfigurationManager.GetSetting("ConnectionSetting"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("mycontainer");
// Create the container if it doesn't already exist.
container.CreateIfNotExists();
CloudBlockBlob blob = container.GetBlockBlobReference(fileName);
blob.Properties.ContentType = "image/jpg";
blob.UploadFromStream(file);
string url = blob.Uri.ToString();
return url;
}
it's takes four call to azure server, so i want do it in one call i.e upload List of four images on azure and retrieve addresses of uploaded images.
You need to make separate service calls to upload separate blobs. However, you should be sharing most of the common code in that method. Namely, everything above actually getting the blob reference. This will save you several service calls by only calling CreateIfNotExists on the container once.

Categories