I have a web api that uses a bunch of appSettings files to load test data.
I want to shift the location of that data to an Azure Blob.
Based on the test infrastructure, I'd like to convert the Blob into an IConfiguration object.
To accomplish this, I wanted to use the AddJsonStream onto a ConfigurationBuilder.
I created this method to go out and grab the blob and convert it to a stream:
public static Stream GetBlobAsStream(Uri blobURI)
{
var storageAccount = CloudStorageAccount.Parse(AZURE_STORAGE_CONNECTION_STRING);
var cloudBlobClient = storageAccount.CreateCloudBlobClient();
var cloudBlobContainer = cloudBlobClient.GetContainerReference(blobContainer);
var cloudBlob = cloudBlobContainer.GetBlockBlobReference(blobName);
var stream = cloudBlob.OpenRead();
return stream;
}
Now this method uses a bunch of hard coded constants - which I'd like to remove.
How can I remove the hard coding, and find the needed azure info based on the Environment in which it's being run?
Or have I programmed myself into a corner here?
You could try to create an instance of CloudBlockBlob using the Blob URI and Blob Client by doing something like:
public static Stream GetBlobAsStream(Uri blobURI)
{
var storageAccount = CloudStorageAccount.Parse(AZURE_STORAGE_CONNECTION_STRING);
var cloudBlobClient = storageAccount.CreateCloudBlobClient();
var cloudBlob = new CloudBlockBlob(blobURI, cloudBlobClient);
var stream = cloudBlob.OpenRead();
return stream;
}
or create an instance of CloudBlockBlob using the Blob URI and Storage Credentials by doing something like:
public static Stream GetBlobAsStream(Uri blobURI)
{
var storageAccount = CloudStorageAccount.Parse(AZURE_STORAGE_CONNECTION_STRING);
var cloudBlob = new CloudBlockBlob(blobURI, storageAccount.Credentials);
var stream = cloudBlob.OpenRead();
return stream;
}
Related
I am trying to upload a new append blob file to a container every time a message comes in from a service bus. I do not want to append to the blob that is already there. I want to create a whole new append blob and add it at the end.
Is this possible?
I was looking at this article but couldn't quiet understand what they meant when they got to the content part: https://azuresdkdocs.blob.core.windows.net/$web/javascript/azure-storage-blob/12.1.1/classes/appendblobclient.html#appendblock
Here is the code that I have so far:
public static async void StoreToBlob(Services service)
{
//Serealize Object
var sender = JsonConvert.SerializeObject(service);
// Create a BlobServiceClient object which will be used to create a container client
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
// Create the container and return a container client object
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);
// Create the container if it doesn't already exist.
await containerClient.CreateIfNotExistsAsync();
//Reference to blob
AppendBlobClient appendBlobClient = containerClient.GetAppendBlobClient("services" + Guid.NewGuid().ToString() + ".json");
// Create the blob.
appendBlobClient.Create();
await appendBlobClient.AppendBlock(sender, sender.Length); //here is where I am having an issue
}
Can you try something like the following (not tested code):
byte[] blockContent = Encoding.UTF8.GetBytes(sender);
using (var ms = new MemoryStream(blockContent))
{
appendBlobClient.AppendBlock(ms, blockContent.Length);
}
Essentially we're converting the string to byte array, creating a stream out of it and then uploading that stream.
I downloaded a file from an FTP server using an azure function and save it in the target that I get from this code:
var target = Path.Combine(context.FunctionAppDirectory, "File.CSV");
Which will be somewhere in "File Shares" that we can see in "Microsoft Azure storage Explorer".
Now my question is about how to copy this file from File Share to Blob container or Directly save it to Blob Storage that azure SQL has access to?
private static void AzureStorageAccountBlob()
{
string filename = "mytestfile.txt";
string fileContents = "some content";
StorageCredentials creds = new StorageCredentials("mystorageaccount2020", "XXXXX");
CloudStorageAccount acct = new CloudStorageAccount(creds, true);
CloudBlobClient client = acct.CreateCloudBlobClient();
CloudBlobContainer container = client.GetContainerReference("myfirstcontainer");
container.CreateIfNotExists();
ICloudBlob blob = container.GetBlockBlobReference(filename);
using (MemoryStream stream = new MemoryStream(Encoding.UTF8.GetBytes(fileContents)))
{
blob.UploadFromStream(stream);
}
}
In my example I have assumed that content already achieved from file. And also one important thing you must create StorageAccount.
Use the below extention to upload to azure:
/// <summary>
/// </summary>
/// <param name="file"></param>
/// <param name="fileName"></param>
/// <param name="connectionString"></param>
/// <param name="containerName"></param>
/// <param name="blobContentType"></param>
/// <returns></returns>
public static async Task<string> AzureUpload(this Stream file, string fileName, string connectionString, string containerName, string blobContentType = null)
{
CloudBlobClient blobClient = CloudStorageAccount.Parse(connectionString).CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
if (await container.CreateIfNotExistsAsync())
{
// Comment this code below if you don't want your files
// to be publicly available. By default, a container is private.
// You can see more on how
// to set different container permissions at:
// https://learn.microsoft.com/en-us/azure/storage/blobs/storage-manage-access-to-resources
await container.SetPermissionsAsync(new BlobContainerPermissions() { PublicAccess = BlobContainerPublicAccessType.Blob });
}
CloudBlockBlob blockBlob = container.GetBlockBlobReference(fileName);
await blockBlob.UploadFromStreamAsync(file);
blobContentType = blobContentType.HasValue() ? blobContentType : getBlobContentType(fileName);
if (blobContentType.HasValue())
{
blockBlob.Properties.ContentType = blobContentType;
await blockBlob.SetPropertiesAsync();
}
return blockBlob.Uri.AbsoluteUri;
}
Do something like this:
var target = Path.Combine(context.FunctionAppDirectory, "File.CSV");
FileStream fileStream = new FileStream(target, FileMode.Open, FileAccess.Read);;
string azureUriForUploadedCSV = await fileStream.AzureUpload(
"File.CSV",
"StorageConnectionString",
"csv-folder",
"application/csv");
Then save azureUriForUploadedCSV into your database...
We can use CloudBlockBlob.StartCopy(CloudFile). You may refer to the code below:
using System;
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Blob;
using Microsoft.Azure.Storage.File;
namespace ConsoleApp3
{
class Program
{
static void Main(string[] args)
{
// Parse the connection string for the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=*************");
// Create a CloudFileClient object for credentialed access to File storage.
CloudFileClient fileClient = storageAccount.CreateCloudFileClient();
// Get a reference to the file share you created previously.
CloudFileShare share = fileClient.GetShareReference("hurytest");
// Get a reference to the file("test.csv") which I have uploaded to the file share("hurytest")
CloudFile sourceFile = share.GetRootDirectoryReference().GetFileReference("test.csv");
// Get a reference to the blob to which the file will be copied.(I have created a container with name of "targetcontainer")
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("targetcontainer");
//container.CreateIfNotExists();
CloudBlockBlob destBlob = container.GetBlockBlobReference("test.csv");
// Create a SAS for the file that's valid for 24 hours.
// Note that when you are copying a file to a blob, or a blob to a file, you must use a SAS
// to authenticate access to the source object, even if you are copying within the same
// storage account.
string fileSas = sourceFile.GetSharedAccessSignature(new SharedAccessFilePolicy()
{
// Only read permissions are required for the source file.
Permissions = SharedAccessFilePermissions.Read,
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(24)
});
// Construct the URI to the source file, including the SAS token.
Uri fileSasUri = new Uri(sourceFile.StorageUri.PrimaryUri.ToString() + fileSas);
// Copy the file to the blob.
destBlob.StartCopy(fileSasUri);
}
}
}
Hope it would be helpful to your problem~
I am trying to download a block blob from Azure using C#. The code I am using is below.
In other tests, I am able to list the blobs in the container but I am unable to download a specific blob. It doesn't give me an exception or error but the file when created locally is empty.
I have cleared out the connection string for obvious reasons.
Does my code look ok?
var containerName = "samples-workitems";
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=xxxxxxxxxxxxxxxxxx.windows.net");
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
try {
CloudBlockBlob blockBlob = container.GetBlockBlobReference("file.png");
var localPath = string.Format("C:\\users\\user\\downloads\\file.png");
blockBlob.DownloadToFileAsync(localPath, FileMode.Create);
catch
{
}
You know what - I have been stuck on this since yesterday and since posting this 2 minutes ago - I removed...
CloudBlockBlob blockBlob = container.GetBlockBlobReference("file.png");
var localPath = string.Format("C:\\users\\user\\downloads\\file.png");
these 2 lines of code from within the try statement and put them above it. It now works?!
Why is that?
I have image files on azure in Blob container. All files have unique names. I nead to search these image files on name without the extentions. For example i have files:
123.PNG
345.jpg
122.JPG
Present code can search if i give complete name of the file such as 123.PNG.
How to make it work with just passing 123.
Code: ID is being passed as a paramenter which is the file name in blob.:
var blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("images");
container.CreateIfNotExists();
var blockBlob = container.GetBlockBlobReference(id);
blockBlob.FetchAttributes();
byte[] downloadedImage = new byte[blockBlob.Properties.Length];
blockBlob.DownloadToByteArray(downloadedImage, 0);
var imageBase64 = Convert.ToBase64String(downloadedImage);
What you could do is use the ListBlobs method that accepts a string prefix parameter like this:
var blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("images");
container.CreateIfNotExists();
var blockBlobs = container.ListBlobs(prefix: "123.").OfType<CloudBlockBlob>();
var blockBlob = blockBlobs.First();
blockBlob.FetchAttributes();
byte[] downloadedImage = new byte[blockBlob.Properties.Length];
blockBlob.DownloadToByteArray(downloadedImage, 0);
var imageBase64 = Convert.ToBase64String(downloadedImage);
The above example will find 123.JPG or 123.PNG (or both)
You will get a list of all blobs that have a name starting with the value of prefix.
For newcomers, you should use like this:
var pagesize = 10;
var resultSegment = blobContainerClient.GetBlobsAsync(prefix: "BlobName")
.AsPages(default, pagesize);
// Enumerate the blobs returned for each page.
await foreach (Azure.Page<BlobItem> blobPage in resultSegment)
{
foreach (BlobItem blobItem in blobPage.Values)
{
Console.WriteLine("Blob name: {0}", blobItem.Name);
}
Console.WriteLine();
}
Ref: MSDN(List blobs with Azure Storage client libraries)
I have an API written in C# that is meant to recieve a file from frontend. As of now it's a byte array and i want to convert this to a .mp4 file and then send it to my azure media service with the blobstorage. I do not want to store it locally and i can't read it from disk either. What is the best approach for this?
I create my CloudBlobClient like so:
private CloudBlobClient CloudBlobClient()
{
var storageAccount = CloudStorageAccount.Parse(ConfigurationManager.ConnectionStrings["StorageConnection"].ConnectionString);
var blobStorage = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobStorage.GetContainerReference(Constants.VideoBlobContainer);
if (container.CreateIfNotExist())
{
var permissions = container.GetPermissions();
permissions.PublicAccess = BlobContainerPublicAccessType.Container;
container.SetPermissions(permissions);
}
return blobStorage;
}
Then I have this method that i've started
private Uri UploadToStorage(CloudBlobClient blobStorage, byte[] video, VideoSize size)
{
var uniqueBlobName = GetVideoUriAsString(VideoId, Type, size);
CloudBlockBlob blob = blobStorage.GetBlockBlobReference(uniqueBlobName);
I'm not sure how to proceede here. I have been looking around a lot on the web for approaches but all I find is example of console applications reading from disk.
Is there anyone familliar with this type of uploading to media serivces?
You're on your way there, although you should just obtain the reference to the blob from the blob container from the first method. Very rough but here you go:
public void uploadBytesToBlobWithMimeAndStorageCreds(string theFolder, string theFileName, byte[] videoBytes, string theMimeType)
{
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(ConfigurationManager.ConnectionStrings["StorageConnection"].ConnectionString);
CloudBlobClient client = storageAccount.CreateCloudBlobClient;
CloudBlobContainer container = client.GetContainerReference(theFolder);
CloudBlob blob = container.GetBlobReference(theFileName);
blob.UploadByteArray(theBytes);
blob.Properties.CacheControl = "max-age=3600, must-revalidate";
blob.Properties.ContentType = theMimeType; // e.g. "video/mp4"
blob.SetProperties();
}