We Keep Coding

Creating a container in azure exits the method

I'm learning Azure storage development and I'm trying to create a container and then upload a file to it. I created a console app and am calling the upload method from Main. Here is the first half of the upload method that creates the container.
public async static Task<bool> UploadToAzure()
{
string containerName = "sample-"+Guid.NewGuid().ToString();
var connectionString = "XXXXXXXXXXXXXXXXXXXXX";
var blobServiceClient = new BlobServiceClient(connectionString);
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);
try
{
var test = blobServiceClient.GetBlobContainers();
foreach (var container in test)
{
Console.WriteLine(container.Name);
}
var result = await containerClient.CreateIfNotExistsAsync(Azure.Storage.Blobs.Models.PublicAccessType.BlobContainer);
var containerProp = await containerClient.GetPropertiesAsync();
Console.WriteLine(containerProp.Value);
}
catch (Exception ex)
{
Console.WriteLine(ex.Message+"\n}");
}
I started with just calling the containerclient.CreateIfNotExistsAnsync() and it wouldn't create the container. On a whim, I added the blobServiceClient.GetBlobContainers() and now it will create the container. I can comment out blobServiceClient.GetBlobContainers() and it will not create the container. I can put it after the create method and it still won't work. That's the first issue.
The second one is, it exits the code when inside the containerclient.CreateIfNotExistsAnsync() method. It will create the folder but never hit the next line of code. I get a message in the console that it "exited with code 0. Press any key to close this window...". It doesn't hit the Catch or display any kind of error, it just exists.
I got this pattern from my Udemy course and I've read a number of articles online and these seem to be the right steps. Any suggestions are appreciated.
Thanks

After reproducing from our end, The code you were using was working for us until we found that you have missed using await while calling UploadToAzure().
Below is the code that we used
namespace ConsoleApplication
{
class Program
{
static async Task Main(string[] args)
{
await UploadToAzure();
}
public async static Task<bool> UploadToAzure()
{
string containerName = "sample-" + Guid.NewGuid().ToString();
var connectionString = "<YOUR_COONECTION_STRING>";
var blobServiceClient = new BlobServiceClient(connectionString);
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);
await containerClient.CreateIfNotExistsAsync(Azure.Storage.Blobs.Models.PublicAccessType.BlobContainer);
return true;
}
}
}
else you can completely remove async and have
namespace CreateContainer
{
class Program
{
static async Task Main(string[] args)
{
UploadToAzure();
}
public static bool UploadToAzure()
{
string containerName = "sample-" + Guid.NewGuid().ToString();
var connectionString = "<YOUR_COONECTION_STRING>";
var blobServiceClient = new BlobServiceClient(connectionString);
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(containerName);
containerClient.CreateIfNotExists(Azure.Storage.Blobs.Models.PublicAccessType.BlobContainer);
return true;
}
}
}
NOTE: It is preferable to use the async method since you might face some issues if you are dealing with large volumes of data.

azure function binding causing multiple events fired

Why are we getting duplicate events for the same BlobCreated?
I'm using the following binding in my function app:
[Blob("%Detach:Output%/{file}", Write)] CloudBlockBlob #out,
The only time when I am writing to this output binding is here:
await #out.UploadTextAsync(xml);
I have an event defined like so:
Where Detach:Output env variable is xml-output-with-links-container.
I am consistently getting 2 events for every execution of this function:
The eventTime between the two events are slightly different:
2019-08-05T22:27:06.5279893Z
2019-08-05T22:27:06.5019647Z
We know that they are firing for the same blob because the subject of the event identifies which blob it is:
"subject": "/blobServices/default/containers/xml-output-with-links-container/blobs/tobossthe_awesome_blob.xml",
I've tested this manually by uploading a payload to xml-output-with-links-container and have gotten just 1 event to fire. Yet, when the function is executed, two events are created.
Why are we getting duplicate events?
Here's the entire function:
{
[FunctionName("Detach")]
[StorageAccount("Global:Connection")]
public static async Task Run(
[QueueTrigger("%Detach:Trigger%")] DetachJob detach,
[Blob("%Detach:Input%/{file}", Read)] CloudBlockBlob #in,
[Blob("%Detach:Output%/{file}", Write)] CloudBlockBlob #out,
[Blob("%Detach:Error%/{file}", Write)] CloudBlockBlob fail,
[Blob("%Detach:Error%/{file}_error", Write)] CloudBlockBlob error,
[Blob("%Detach:Attachments%", Write)] CloudBlobContainer attachments)
{
try
{
var xml = await #in.DownloadTextAsync();
var size = ToInt32(GetEnvironmentVariable("Detach:BytesThreshold"));
var bigNodes = GetNodesGreaterThan(size, xml);
foreach (var node in bigNodes)
{
var ext = GetExtension(node.Value);
var path = $"{detach.file}/{node.Key}{ext}.base64";
var attachment = attachments.GetBlockBlobReference(path);
await attachment.UploadTextAsync(node.Value);
xml = xml.Replace(node.Value, path);
}
await #out.UploadTextAsync(xml);
}
catch (Exception e)
{
await error.UploadTextAsync(e.ToString());
await fail.StartCopyAsync(#in);
}
}
}
I was thinking that perhaps the CloudBlockBlob was triggering twice. So I changed the binding to be CloudBlobContainer:
[Blob("%Detach:Output%", Write)] CloudBlobContainer #out,
And updated the respective code:
var shrink = #out.GetBlockBlobReference(file);
await shrink.UploadTextAsync(xml);
Yet the result stayed the same: I still got 2 events.

I was triggering the Detach function by dropping a payload into blob storage using a logic app with the following step:
This step was generating 2 BlobCreated events!
After turning off thunking, the issue has been resolved, and only 1 BlobCreated event is now generating:

Storage Commitment with fo-dicom

I am trying to implement a storage commitment with FO-DICOM framework, but with no result. I am able to create the N-ACTION request. I am able to receive the N-ACTION response. But I don't know how to receive the EVENTREPORT. Anyone can help me and address me to the right way?
private DicomStatus _responseStatus;
public void SendRequestForCommitment(string scImageUid)
{
var client = new DicomClient();
var nAction = new DicomNActionRequest(DicomUID.StorageCommitmentPushModelSOPClass,
new UIDGenerator().PrivatelyDefinedSoapInstanceUid(), 1);
var ds = new DicomDataset();
nAction.Dataset = ds;
nAction.Dataset.Add(DicomTag.TransactionUID, new UIDGenerator().uid);
var sps = new DicomDataset();
nAction.Dataset.Add(new DicomSequence(DicomTag.ReferencedSOPSequence, sps));
sps.Add(DicomTag.ReferencedSOPClassUID, DicomUID.SecondaryCaptureImageStorage);
sps.Add(DicomTag.ReferencedSOPInstanceUID, scImageUid);
DicomNActionRequest.ResponseDelegate nActionResponseDelegate = NActionResponse;
nAction.OnResponseReceived = nActionResponseDelegate;
client.AddRequest(nAction);
client.Send("127.0.0.1", 105, false, "myAE", "DVTK_STRC_SCP");
}
private void NActionResponse(DicomNActionRequest request, DicomNActionResponse response)
{
_responseStatus = response.Status;
}

Disclaimer: I never used FO-DICOM. The code below is just a pseudo code and is NOT FO-DICOM syntax. I hope looking at pseudo code, you will able to figure out exact members (properties, methods, and events) in toolkit.
In your code, you are already building request dataset. Then, you are calling client.AddRequest(nAction); and then client.Send(.......);. I assume this will internally establish a connection, association and will send NAction request.
Then you have subscribed for private void NActionResponse(....) event. I assume this event is being fired and you are getting NAction Response.
Similarly, you should subscribe NEventReport event something (look for exact syntax in toolkit) like following:
private void NEventReportReceived(DicomNEventReport request, ......)
{
//Parse the request here.
//Check what files were archived and what were failed.
//Do your stuff accordingly.
//Send NEventReport response conveying the status.
client.SendReleaseRequest();
}
Subscribe another event to handle release response.
private void ReleaseResponseReceived(......)
{
//Close connection
}
As I said in other answer, your SCU should have ability to process NEventReport. You have added NAction to your client by writing line client.AddRequest(nAction);. Check the toolkit documentation to see if you also need to add similar for NEventReport. I strongly think this should not be needed; you just need to subscribe an event.

SignalR server --> client call not working

I'm currently using SignalR to communicate between a server and multiple separate processes spawned by the server itself.
Both Server & Client are coded in C#. I'm using SignalR 2.2.0.0
On the server side, I use OWIN to run the server.
I am also using LightInject as an IoC container.
Here is my code:
public class AgentManagementStartup
{
public void ConfigurationOwin(IAppBuilder app, IAgentManagerDataStore dataStore)
{
var serializer = new JsonSerializer
{
PreserveReferencesHandling = PreserveReferencesHandling.Objects,
TypeNameHandling = TypeNameHandling.Auto,
TypeNameAssemblyFormat = FormatterAssemblyStyle.Simple
};
var container = new ServiceContainer();
container.RegisterInstance(dataStore);
container.RegisterInstance(serializer);
container.Register<EventHub>();
container.Register<ManagementHub>();
var config = container.EnableSignalR();
app.MapSignalR("", config);
}
}
On the client side, I register this way:
public async Task Connect()
{
try
{
m_hubConnection = new HubConnection(m_serverUrl, false);
m_hubConnection.Closed += OnConnectionClosed;
m_hubConnection.TraceLevel = TraceLevels.All;
m_hubConnection.TraceWriter = Console.Out;
var serializer = m_hubConnection.JsonSerializer;
serializer.TypeNameHandling = TypeNameHandling.Auto;
serializer.PreserveReferencesHandling = PreserveReferencesHandling.Objects;
m_managementHubProxy = m_hubConnection.CreateHubProxy(AgentConstants.ManagementHub.Name);
m_managementHubProxy.On("closeRequested", CloseRequestedCallback);
await m_hubConnection.Start();
}
catch (Exception e)
{
m_logger.Error("Exception encountered in Connect method", e);
}
}
On the server side I send a close request the following way:
var managementHub = GlobalHost.ConnectionManager.GetHubContext<ManagementHub>();
managementHub.Clients.All.closeRequested();
I never receive any callback in CloseRequestedCallback. Neither on the Client side nor on the server side I get any errors in the logs.
What did I do wrong here ?
EDIT 09/10/15
After some research and modifications, I found out it was linked with the replacement of the IoC container. When I removed everything linked to LightInject and used SignalR as is, everything worked. I was surprised about this since LightInject documented their integration with SignalR.
After I found this, I realised that the GlobalHost.DependencyResolver was not the same as the one I was supplying to the HubConfiguration. Once I added
GlobalHost.DependencyResolver = config.Resolver;
before
app.MapSignalR("", config);
I am now receiving callbacks within CloseRequestedCallback. Unfortunately, I get the following error as soon as I call a method from the Client to the Server:
Microsoft.AspNet.SignalR.Client.Infrastructure.SlowCallbackException
Possible deadlock detected. A callback registered with "HubProxy.On"
or "Connection.Received" has been executing for at least 10 seconds.
I am not sure about the fix I found and what impact it could have on the system. Is it OK to replace the GlobalHost.DependencyResolver with my own without registering all of its default content ?
EDIT 2 09/10/15
According to this, changing the GlobalHost.DependencyResolver is the right thing to do. Still left with no explanation for the SlowCallbackException since I do nothing in all my callbacks (yet).

Issue 1: IoC Container + Dependency Injection
If you want to change the IoC for you HubConfiguration, you also need to change the one from the GlobalHost so that returns the same hub when requesting it ouside of context.
Issue 2: Unexpected SlowCallbackException
This exception was caused by the fact that I was using SignalR within a Console Application. The entry point of the app cannot be an async method so to be able to call my initial configuration asynchronously I did as follow:
private static int Main()
{
var t = InitAsync();
t.Wait();
return t.Result;
}
Unfortunately for me, this causes a lot of issues as described here & more in details here.
By starting my InitAsync as follow:
private static int Main()
{
Task.Factory.StartNew(async ()=> await InitAsync());
m_waitInitCompletedRequest.WaitOne(TimeSpan.FromSeconds(30));
return (int)EndpointErrorCode.Ended;
}
Everything now runs fine and I don't get any deadlocks.
For more details on the issues & answers, you may also refer to the edits in my question.

Checking if a blob exists in Azure Storage

I've got a very simple question (I hope!) - I just want to find out if a blob (with a name I've defined) exists in a particular container. I'll be downloading it if it does exist, and if it doesn't then I'll do something else.
I've done some searching on the intertubes and apparently there used to be a function called DoesExist or something similar... but as with so many of the Azure APIs, this no longer seems to be there (or if it is, has a very cleverly disguised name).

The new API has the .Exists() function call. Just make sure that you use the GetBlockBlobReference, which doesn't perform the call to the server. It makes the function as easy as:
public static bool BlobExistsOnCloud(CloudBlobClient client,
string containerName, string key)
{
return client.GetContainerReference(containerName)
.GetBlockBlobReference(key)
.Exists();
}

Note: This answer is out of date now. Please see Richard's answer for an easy way to check for existence
No, you're not missing something simple... we did a good job of hiding this method in the new StorageClient library. :)
I just wrote a blog post to answer your question: http://blog.smarx.com/posts/testing-existence-of-a-windows-azure-blob.
The short answer is: use CloudBlob.FetchAttributes(), which does a HEAD request against the blob.

Seem lame that you need to catch an exception to test it the blob exists.
public static bool Exists(this CloudBlob blob)
{
try
{
blob.FetchAttributes();
return true;
}
catch (StorageClientException e)
{
if (e.ErrorCode == StorageErrorCode.ResourceNotFound)
{
return false;
}
else
{
throw;
}
}
}

If the blob is public you can, of course, just send an HTTP HEAD request -- from any of the zillions of languages/environments/platforms that know how do that -- and check the response.
The core Azure APIs are RESTful XML-based HTTP interfaces. The StorageClient library is one of many possible wrappers around them. Here's another that Sriram Krishnan did in Python:
http://www.sriramkrishnan.com/blog/2008/11/python-wrapper-for-windows-azure.html
It also shows how to authenticate at the HTTP level.
I've done a similar thing for myself in C#, because I prefer to see Azure through the lens of HTTP/REST rather than through the lens of the StorageClient library. For quite a while I hadn't even bothered to implement an ExistsBlob method. All my blobs were public, and it was trivial to do HTTP HEAD.

The new Windows Azure Storage Library already contains the Exist() method.
It´s in the Microsoft.WindowsAzure.Storage.dll.
Available as NuGet Package
Created by: Microsoft
Id: WindowsAzure.Storage
Version: 2.0.5.1
See also msdn

Here's a different solution if you don't like the other solutions:
I am using version 12.4.1 of the Azure.Storage.Blobs NuGet Package.
I get an Azure.Pageable object which is a list of all of the blobs in a container. I then check if the name of the BlobItem equals to the Name property of each blob inside the container utilizing LINQ. (If everything is valid, of course)
using Azure.Storage.Blobs;
using Azure.Storage.Blobs.Models;
using System.Linq;
using System.Text.RegularExpressions;
public class AzureBlobStorage
{
private BlobServiceClient _blobServiceClient;
public AzureBlobStorage(string connectionString)
{
this.ConnectionString = connectionString;
_blobServiceClient = new BlobServiceClient(this.ConnectionString);
}
public bool IsContainerNameValid(string name)
{
return Regex.IsMatch(name, "^[a-z0-9](?!.*--)[a-z0-9-]{1,61}[a-z0-9]$", RegexOptions.Singleline | RegexOptions.CultureInvariant);
}
public bool ContainerExists(string name)
{
return (IsContainerNameValid(name) ? _blobServiceClient.GetBlobContainerClient(name).Exists() : false);
}
public Azure.Pageable<BlobItem> GetBlobs(string containerName, string prefix = null)
{
try
{
return (ContainerExists(containerName) ?
_blobServiceClient.GetBlobContainerClient(containerName).GetBlobs(BlobTraits.All, BlobStates.All, prefix, default(System.Threading.CancellationToken))
: null);
}
catch
{
throw;
}
}
public bool BlobExists(string containerName, string blobName)
{
try
{
return (from b in GetBlobs(containerName)
where b.Name == blobName
select b).FirstOrDefault() != null;
}
catch
{
throw;
}
}
}
Hopefully this helps someone in the future.

This is the way I do it. Showing full code for those who need it.
// Parse the connection string and return a reference to the storage account.
CloudStorageAccount storageAccount = CloudStorageAccount.Parse(CloudConfigurationManager.GetSetting("AzureBlobConnectionString"));
CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();
// Retrieve reference to a previously created container.
CloudBlobContainer container = blobClient.GetContainerReference("ContainerName");
// Retrieve reference to a blob named "test.csv"
CloudBlockBlob blockBlob = container.GetBlockBlobReference("test.csv");
if (blockBlob.Exists())
{
//Do your logic here.
}

If your blob is public and you need just metadata:
HttpWebRequest request = (HttpWebRequest)WebRequest.Create(url);
request.Method = "HEAD";
string code = "";
try
{
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
code = response.StatusCode.ToString();
}
catch
{
}
return code; // if "OK" blob exists

If you don't like using the exception method then the basic c# version of what judell suggests is below. Beware though that you really ought to handle other possible responses too.
HttpWebRequest myReq = (HttpWebRequest)WebRequest.Create(url);
myReq.Method = "HEAD";
HttpWebResponse myResp = (HttpWebResponse)myReq.GetResponse();
if (myResp.StatusCode == HttpStatusCode.OK)
{
return true;
}
else
{
return false;
}

With the updated SDK, once you have the CloudBlobReference you can call Exists() on your reference.
See http://msdn.microsoft.com/en-us/library/microsoft.windowsazure.storage.blob.cloudblockblob.exists.aspx

Although most answers here are technically correct, most code samples are making synchronous/blocking calls. Unless you're bound by a very old platform or code base, HTTP calls should always be done asynchonously, and the SDK fully supports it in this case. Just use ExistsAsync() instead of Exists().
bool exists = await client.GetContainerReference(containerName)
.GetBlockBlobReference(key)
.ExistsAsync();

With Azure Blob storage library v12, you can use BlobBaseClient.Exists()/BlobBaseClient.ExistsAsync()
Answered on another similar question: https://stackoverflow.com/a/63293998/4865541

Java version for the same ( using the new v12 SDK )
This uses the Shared Key Credential authorization (account access key)
public void downloadBlobIfExists(String accountName, String accountKey, String containerName, String blobName) {
// create a storage client using creds
StorageSharedKeyCredential credential = new StorageSharedKeyCredential(accountName, accountKey);
String endpoint = String.format(Locale.ROOT, "https://%s.blob.core.windows.net", accountName);
BlobServiceClient storageClient = new BlobServiceClientBuilder().credential(credential).endpoint(endpoint).buildClient();
BlobContainerClient container = storageClient.getBlobContainerClient(containerName);
BlobClient blob = container.getBlobClient(blobName);
if (blob.exists()) {
// download blob
} else {
// do something else
}
}