I have image files on azure in Blob container. All files have unique names. I nead to search these image files on name without the extentions. For example i have files:
123.PNG
345.jpg
122.JPG
Present code can search if i give complete name of the file such as 123.PNG.
How to make it work with just passing 123.
Code: ID is being passed as a paramenter which is the file name in blob.:
var blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("images");
container.CreateIfNotExists();
var blockBlob = container.GetBlockBlobReference(id);
blockBlob.FetchAttributes();
byte[] downloadedImage = new byte[blockBlob.Properties.Length];
blockBlob.DownloadToByteArray(downloadedImage, 0);
var imageBase64 = Convert.ToBase64String(downloadedImage);
What you could do is use the ListBlobs method that accepts a string prefix parameter like this:
var blobClient = storageAccount.CreateCloudBlobClient();
var container = blobClient.GetContainerReference("images");
container.CreateIfNotExists();
var blockBlobs = container.ListBlobs(prefix: "123.").OfType<CloudBlockBlob>();
var blockBlob = blockBlobs.First();
blockBlob.FetchAttributes();
byte[] downloadedImage = new byte[blockBlob.Properties.Length];
blockBlob.DownloadToByteArray(downloadedImage, 0);
var imageBase64 = Convert.ToBase64String(downloadedImage);
The above example will find 123.JPG or 123.PNG (or both)
You will get a list of all blobs that have a name starting with the value of prefix.
For newcomers, you should use like this:
var pagesize = 10;
var resultSegment = blobContainerClient.GetBlobsAsync(prefix: "BlobName")
.AsPages(default, pagesize);
// Enumerate the blobs returned for each page.
await foreach (Azure.Page<BlobItem> blobPage in resultSegment)
{
foreach (BlobItem blobItem in blobPage.Values)
{
Console.WriteLine("Blob name: {0}", blobItem.Name);
}
Console.WriteLine();
}
Ref: MSDN(List blobs with Azure Storage client libraries)
Related
I am getting confused with the C# azure sdk.
What I am trying to achieve.
Upload files from my computer to a folder in azure.
For example
Locally
MyFiles
-- Folder1
-- file.txt
-- img.jpg
-- Folder2
-- file2.json
-- test.png
Azure Result
Container
MyFiles
-- Folder1
-- file.txt
-- img.jpg
-- Folder2
-- file2.json
-- test.png
So I want in my container on azure same file structure.
how I am doing it is
var sasCred = new AzureSasCrdentials("sasurl");
var container = new BlobContainerClient(new Uri("containerUrl"), sasCred);
var allFiles = Directory.GetFiles("MyFilesFolderPath", "*", SearchOption.AllDirectories);
foreach(var file in files)
{
var cloudFilePath = file.Replace("MyFilesFolderPath", string.Empty);
var fullPath= $"MyFiles{cloudFilePath};
using(var s = new MemoryStream(File.ReadAllBytes(file))
{
await container.UploadBlobAsync(fullPath,stream);
}
}
This seems to do what I need it to do though I noticed the file type is something like "file octet stream" instead of .json/.png/txt or whatever it should be.
When I search it talks about using BlobCLient to set the file type but now I am sure if I should be using BlobContainerClient or not.
You would need to use both BlobContainerClient and BlobClient in this case. The way you would do it is that you would create an instance of BlobClient (BlockBlobClient specifically) using BlobContainerClient and blob name and use UploadAsync method there.
Your code (untested) would be something like:
var sasCred = new AzureSasCrdentials("sasurl");
var container = new BlobContainerClient(new Uri("containerUrl"), sasCred);
var allFiles = Directory.GetFiles("MyFilesFolderPath", "*", SearchOption.AllDirectories);
foreach(var file in files)
{
var cloudFilePath = file.Replace("MyFilesFolderPath", string.Empty);
var fullPath= $"MyFiles{cloudFilePath};
using (var s = new MemoryStream(File.ReadAllBytes(file))
{
var blockBlob = container.GetBlockBlobClient(fullPath);//Get BlockBlobClient instance
var blobContentType = GetContentTypeFromFileSomehow(file);//Write a helper method to get the content type
var headers = new BlobHttpHeaders() { ContentType = blobContentType};//Set content type header for blob.
var blobUploadOptions = new BlobUploadOptions() { HttpHeaders = headers};
await blockBlob.UploadAsync(s, blobUploadOptions);//Upload blob
}
}
You can specify the ContentType property of the BlobHttpHeaders class as the desired content type.
There are also nuget libraries to get the mime type from the file name extension if they vary by upload.
blobClient.UploadAsync(stream, new BlobHttpHeaders { ContentType = "text/plain" });
I have a web api that uses a bunch of appSettings files to load test data.
I want to shift the location of that data to an Azure Blob.
Based on the test infrastructure, I'd like to convert the Blob into an IConfiguration object.
To accomplish this, I wanted to use the AddJsonStream onto a ConfigurationBuilder.
I created this method to go out and grab the blob and convert it to a stream:
public static Stream GetBlobAsStream(Uri blobURI)
{
var storageAccount = CloudStorageAccount.Parse(AZURE_STORAGE_CONNECTION_STRING);
var cloudBlobClient = storageAccount.CreateCloudBlobClient();
var cloudBlobContainer = cloudBlobClient.GetContainerReference(blobContainer);
var cloudBlob = cloudBlobContainer.GetBlockBlobReference(blobName);
var stream = cloudBlob.OpenRead();
return stream;
}
Now this method uses a bunch of hard coded constants - which I'd like to remove.
How can I remove the hard coding, and find the needed azure info based on the Environment in which it's being run?
Or have I programmed myself into a corner here?
You could try to create an instance of CloudBlockBlob using the Blob URI and Blob Client by doing something like:
public static Stream GetBlobAsStream(Uri blobURI)
{
var storageAccount = CloudStorageAccount.Parse(AZURE_STORAGE_CONNECTION_STRING);
var cloudBlobClient = storageAccount.CreateCloudBlobClient();
var cloudBlob = new CloudBlockBlob(blobURI, cloudBlobClient);
var stream = cloudBlob.OpenRead();
return stream;
}
or create an instance of CloudBlockBlob using the Blob URI and Storage Credentials by doing something like:
public static Stream GetBlobAsStream(Uri blobURI)
{
var storageAccount = CloudStorageAccount.Parse(AZURE_STORAGE_CONNECTION_STRING);
var cloudBlob = new CloudBlockBlob(blobURI, storageAccount.Credentials);
var stream = cloudBlob.OpenRead();
return stream;
}
I downloaded a file from an FTP server using an azure function and save it in the target that I get from this code:
var target = Path.Combine(context.FunctionAppDirectory, "File.CSV");
Which will be somewhere in "File Shares" that we can see in "Microsoft Azure storage Explorer".
Now my question is about how to copy this file from File Share to Blob container or Directly save it to Blob Storage that azure SQL has access to?
private static void AzureStorageAccountBlob()
{
string filename = "mytestfile.txt";
string fileContents = "some content";
StorageCredentials creds = new StorageCredentials("mystorageaccount2020", "XXXXX");
CloudStorageAccount acct = new CloudStorageAccount(creds, true);
CloudBlobClient client = acct.CreateCloudBlobClient();
CloudBlobContainer container = client.GetContainerReference("myfirstcontainer");
container.CreateIfNotExists();
ICloudBlob blob = container.GetBlockBlobReference(filename);
using (MemoryStream stream = new MemoryStream(Encoding.UTF8.GetBytes(fileContents)))
{
blob.UploadFromStream(stream);
}
}
In my example I have assumed that content already achieved from file. And also one important thing you must create StorageAccount.
Use the below extention to upload to azure:
/// <summary>
/// </summary>
/// <param name="file"></param>
/// <param name="fileName"></param>
/// <param name="connectionString"></param>
/// <param name="containerName"></param>
/// <param name="blobContentType"></param>
/// <returns></returns>
public static async Task<string> AzureUpload(this Stream file, string fileName, string connectionString, string containerName, string blobContentType = null)
{
CloudBlobClient blobClient = CloudStorageAccount.Parse(connectionString).CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference(containerName);
if (await container.CreateIfNotExistsAsync())
{
// Comment this code below if you don't want your files
// to be publicly available. By default, a container is private.
// You can see more on how
// to set different container permissions at:
// https://learn.microsoft.com/en-us/azure/storage/blobs/storage-manage-access-to-resources
await container.SetPermissionsAsync(new BlobContainerPermissions() { PublicAccess = BlobContainerPublicAccessType.Blob });
}
CloudBlockBlob blockBlob = container.GetBlockBlobReference(fileName);
await blockBlob.UploadFromStreamAsync(file);
blobContentType = blobContentType.HasValue() ? blobContentType : getBlobContentType(fileName);
if (blobContentType.HasValue())
{
blockBlob.Properties.ContentType = blobContentType;
await blockBlob.SetPropertiesAsync();
}
return blockBlob.Uri.AbsoluteUri;
}
Do something like this:
var target = Path.Combine(context.FunctionAppDirectory, "File.CSV");
FileStream fileStream = new FileStream(target, FileMode.Open, FileAccess.Read);;
string azureUriForUploadedCSV = await fileStream.AzureUpload(
"File.CSV",
"StorageConnectionString",
"csv-folder",
"application/csv");
Then save azureUriForUploadedCSV into your database...
We can use CloudBlockBlob.StartCopy(CloudFile). You may refer to the code below:
using System;
using Microsoft.Azure.Storage;
using Microsoft.Azure.Storage.Blob;
using Microsoft.Azure.Storage.File;
namespace ConsoleApp3
{
class Program
{
static void Main(string[] args)
{
// Parse the connection string for the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=*************");
// Create a CloudFileClient object for credentialed access to File storage.
CloudFileClient fileClient = storageAccount.CreateCloudFileClient();
// Get a reference to the file share you created previously.
CloudFileShare share = fileClient.GetShareReference("hurytest");
// Get a reference to the file("test.csv") which I have uploaded to the file share("hurytest")
CloudFile sourceFile = share.GetRootDirectoryReference().GetFileReference("test.csv");
// Get a reference to the blob to which the file will be copied.(I have created a container with name of "targetcontainer")
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer container = blobClient.GetContainerReference("targetcontainer");
//container.CreateIfNotExists();
CloudBlockBlob destBlob = container.GetBlockBlobReference("test.csv");
// Create a SAS for the file that's valid for 24 hours.
// Note that when you are copying a file to a blob, or a blob to a file, you must use a SAS
// to authenticate access to the source object, even if you are copying within the same
// storage account.
string fileSas = sourceFile.GetSharedAccessSignature(new SharedAccessFilePolicy()
{
// Only read permissions are required for the source file.
Permissions = SharedAccessFilePermissions.Read,
SharedAccessExpiryTime = DateTime.UtcNow.AddHours(24)
});
// Construct the URI to the source file, including the SAS token.
Uri fileSasUri = new Uri(sourceFile.StorageUri.PrimaryUri.ToString() + fileSas);
// Copy the file to the blob.
destBlob.StartCopy(fileSasUri);
}
}
}
Hope it would be helpful to your problem~
I have followed microsoft tutorial about routing in azure IoT Hub, and now all my messages from iot hub are storaged in blobs in container. However each blob is in individual folder. For example messages send 3 nov 2018 19:53 are stored in
xyz/00/2018/11/03/19/53 blob file. How can I get all blobs from all folders in container? I can't use CloudBlobContainer.ListBlobs() because I don't have this method. I have tried:
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(StorageConnectionString);
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
CloudBlobContainer blobContainer = blobClient.GetContainerReference(storageContainerName);
CloudBlob blob = blobContainer.GetBlockBlobReference("xyz/00/2018/11/03/20/15");
MemoryStream stream = new MemoryStream();
await blob.DownloadToStreamAsync(stream);
return Encoding.UTF8.GetString(stream.ToArray());
and this code is returning content of blob located in xyz/00/2018/11/03/20/15
However if I try to iterate over Blobs using below code it returns only "https://{storageName}.blob.core.windows.net/{containerName}/{xyz}/"
List<string> blobs = new List<string>();
BlobResultSegment resultSegment = blobContainer.ListBlobsSegmentedAsync(null).Result;
foreach (IListBlobItem item in resultSegment.Results)
{
if (item.GetType() == typeof(CloudBlockBlob))
{
CloudBlockBlob blob = (CloudBlockBlob)item;
blobs.Add(blob.Name);
}
else if (item.GetType() == typeof(CloudPageBlob))
{
CloudPageBlob blob = (CloudPageBlob)item;
blobs.Add(blob.Name);
}
else if (item.GetType() == typeof(CloudBlobDirectory))
{
CloudBlobDirectory dir = (CloudBlobDirectory)item;
blobs.Add(dir.Uri.ToString());
}
}
var result = String.Join(", ", blobs.ToArray());
return result;
What I am missing here? How can I get every blob from container?
Use an overload of ListBlobsSegmentedAsync that accepts the useFlatBlobListing and set the value of useFlatBlobListing to true:
useFlatBlobListing
A boolean value that specifies whether to list blobs in a flat listing, or whether to list blobs hierarchically, by virtual directory.
(source)
How to create an empty text file ( or text with some message )inside my blob container
var destBlob = blobClient.GetBlobReference(myblob);
something like
https://myxyzstorage.blob.core.windows.net/mycontainer/newfolder/newTextfile.txt
If you are using a newer version of Windows Azure Storage Client Library, you should create a container and then use it to get a blob reference with the path you’d like your blob to have within the container. To create a path similar to the one you posted:
CloudBlobContainer container = blobClient.GetContainerReference(“mycontainer”);
container.CreateIfNotExists();
CloudBlockBlob blob = container.GetBlockBlobReference("newfolder/newTextfile.txt");
blob.UploadText("any_content_you_want");
If you are using .NET standard, this code should work.
CloudBlockBlob blob = blobContainer.GetBlockBlobReference("File Name");
blob.UploadTextAsync("<<File Content here>>");
The following example from here helped me to solve this
public Uri UploadBlob(string path, string fileName, string content)
{
var cloudBlobContainer = cloudBlobClient.GetContainerReference(path);
cloudBlobContainer.CreateIfNotExist();
var blob = cloudBlobContainer.GetBlobReference(fileName);
blob.UploadText(content);
return blob.Uri;
}