I want to upload a file to an Azure Storage account that is automatically generated (As part of a Service Fabric resource group, with a known name), using C#.
I need to upload the file as a blob to allow it to be publicly available.
The tutorial Get started with Azure Blob storage using .NET uses a connection string stored in the App.config file.
Since I want to use the to-be-generated storage account, I can't use such a method.
The prefered method is using the user's AD somehow in order to get the key of the storage account.
This link: Get Storage Account Key shows how to GET it with a Rest request, so I guess there is a way to do it using C# code.
It seems to me, that the solution is using the StorageManagementClient class, which has a StorageAccounts property, though I could not find a way to authenticate it using AzureAd.
I tried using AuthenticationContext.AcquireTokenAsync, and aquiring a token for diffenent resources, for instance: https://management.azure.com/, but when using the token, I get the following error:
Microsoft.WindowsAzure.CloudException: AuthenticationFailed: The JWT token does not contain expected audience uri 'https://management.core.windows.net/'.
When using the resource https://management.core.windows.net/ I get a different error:
Microsoft.WindowsAzure.CloudException: ForbiddenError: The server failed to authenticate the request. Verify that the certificate is valid and is associated with this subscription.
Is there a different resource I should use, different method, or maybe it's impossible?
To use the Storage Service Management REST, we need to specify the resource to https://management.core.windows.net/ instead of https://management.azure.com/. And this is using the operate the classic storage account.
The https://management.azure.com/ is the new endpoint for the Azure REST service. If you want to handle the new storage account, you need to use this resource. And below is a sample using the new Azure REST for your reference:
POST: https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resrouceGroupName}/providers/Microsoft.Storage/storageAccounts/{storageAccountName}/listKeys?api-version=2016-01-01
Authorization: Bearer {token}
Related
I have a function app and need to give it permission to write to blob/table storage. I turned on "System Assigned Managed Identity", and I set the following permissions for it with the scope being the Storage Account I need to access.
Storage Account Contributor
Storage Blob Data Owner
Storage Table Data Contributor
Storage Queue Data Contributor
UPDATE
I remove AZURE_CLIENT_ID, AZURE_CLIENT_SECRET and AZURE_TENANT_ID and then I receive an environment configuration error. I am not running this locally or debugging, I'm triggering it through API Management.
Executed 'Create' (Failed, Duration=1406ms)EnvironmentCredential authentication unavailable. Environment variables are not fully configured. See the troubleshooting guide for more information. https://aka.ms/azsdk/net/identity/environmentcredential/troubleshoot
^ The above link tells me I should add those environment variables back, but it seems like I shouldn't need them according to Peter Bon's answer below
One piece that I find very confusing (because I can't find good documentation on this anywhere) is that I am required by Azure to include three variables
AZURE_CLIENT_ID
AZURE_CLIENT_SECRET
AZURE_TENANT_ID
I am inferring that they are needed to actually allow my function app to access the storage account, which confused me at first because if I just gave it permission explicitly with assigned roles, why do I also need to create something completely outside of my function app to give it permission to do the thing I already gave it permission to do?
My question about this though is "HOW DO I MAKE THIS?" If I need to make an App Registration, what should the callback URL be? What should the API Permissions be? I gave it my best guess and did not get a satisfactory result.
App Registration:
API permissions -> Microsoft.Graph User.Read
Authentication -> https://<mydomain>.onmicrosoft.com/.auth/login/aad/callback
-> ID Tokens
-> Accounts in this organization
Secrets -> Generated new secret to use for AZURE_CLIENT_SECRET
Roles & Admissions -> Cloud application administrator
I then set AZURE_CLIENT_ID to be the app id of this App Registration, AZURE_CLIENT_SECRET to the app registration's secret and AZURE_TENANT_ID to my tenant ID.
Then in my code, I attempt to do the following
var tableUri = new Uri(string.Format("https://{0}.table.core.windows.net/", storageAccountName));
var credential = new DefaultAzureCredential(options);
services.AddScoped(x => new TableServiceClient(tableUri, credential));
And it fails when accessing my table storage with the following error:
Executed 'Create' (Failed, Id=<id>, Duration=2108ms)Server failed to authenticate the request. Please refer to the information in the www-authenticate header.RequestId:<id>Time:2022-10-21T12:15:21.6998519ZStatus: 401 (Server failed to authenticate the request. Please refer to the information in the www-authenticate header.)ErrorCode: InvalidAuthenticationInfoContent:{"odata.error":{"code":"InvalidAuthenticationInfo","message":{"lang":"en-US","value":"Server failed to authenticate the request. Please refer to the information in the www-authenticate header.\nRequestId:<id>\nTime:2022-10-21T12:15:21.6998519Z"}}}Headers:Server: Microsoft-HTTPAPI/2.0x-ms-request-id: <id>x-ms-error-code: REDACTEDWWW-Authenticate: Bearer authorization_uri=https://login.microsoftonline.com/<tenant_id>/oauth2/authorize resource_id=https://storage.azure.comDate: Fri, 21 Oct 2022 12:15:21 GMTContent-Length: 279Content-Type: application/json
and if I update the Authentication redirect to
https://storage.azure.com
then I get the following error:
Executed 'Create' (Failed, Id=<id>, Duration=2349ms)This request is not authorized to perform this operation using this permission.RequestId:<request>Time:2022-10-21T13:14:29.0955823ZStatus: 403 (Forbidden)ErrorCode: AuthorizationPermissionMismatchContent:{"odata.error":{"code":"AuthorizationPermissionMismatch","message":{"lang":"en-US","value":"This request is not authorized to perform this operation using this permission.\nRequestId:<id>\nTime:2022-10-21T13:14:29.0955823Z"}}}Headers:Cache-Control: no-cacheTransfer-Encoding: chunkedServer: Windows-Azure-Table/1.0,Microsoft-HTTPAPI/2.0x-ms-request-id: <id>x-ms-client-request-id: <id>x-ms-version: REDACTEDX-Content-Type-Options: REDACTEDDate: Fri, 21 Oct 2022 13:14:28 GMTContent-Type: application/json; odata=minimalmetadata; streaming=true; charset=utf-8
I am honestly pretty confused at this point, can someone please help me understand how to successfully set up system assigned managed identities?
UPDATE WITH ANSWER
figured it out w/ suggestion from Peter.. so, I'm not a C# developer, but have a programming background & am doing devops for this project, another group is coding the application. I didn't realize that they had specifically used new EnvironmentCredential(); in their code, since they emphasized always needing to use DefaultAzureCredential but the EnvironmentCredential forces the setting of AZURE_CLIENT_ID. This also explains our issue w/ using DefaultAzureCredential because it goes through the list like you linked in the answer, so it sees AZURE_CLIENT_ID is set, and doesn't bother going for MI even though MI has the right permissions.
When using a managed identity you should not provide these env variables. It would also defeat the purpose of not having to store a secret. They are not required in that scenario. It is just one of the many ways DefaultAzureCredential tries to get valid credentials. See this section. If you do not provide them, it will fall back trying to use a managed identity.
The access level of my container is 'Private (no anonymous access)'. I prefer this access level as I only want authorized users to view my content. However, when I try to access a file via the container URL I get a ResourceNotFound error.
I guess there are alternative steps to authenticate the validity of the request. Can someone please help me out by letting me know the steps to get a file displayed.
My front end is Angular/HTML.
This is expected behavior. You can't directly access a blob if it is in a blob container with Private ACL.
What you would need to do is create a Shared Access Signature (SAS) on the blob with at least read permission and use the SAS URL of the blob. You can create a Service SAS on the blob.
The way it normally works is that you would create a SAS on the blob in your backend API and then pass that SAS token/URL to your frontend.
I'm writing a c# program right now that tries to authenticate with Azure to make a generic http request. I finally got the code working and I wanted to test the features but for every request I make I get the following error code in response:
{"error":{"code": "AuthorizationFailed", "message":"The client "(id of the app I registered in AzureAD)" with object id "(same value as before)" does not have authorization to perform action 'Microsoft.Authorization/roleAssignments/read' over scope '/subscriptions/(mysubscriptionid)'."}}.
The thing is ... The account I use to set everything up is a global admin. I checked every permission box in AzureAD I could find...
(that's 8 Application Permissions and 9 Delegated permissions in the Windows Azure Active Directory API and 1 delegated Permission in the Windows Azure Service Management API, though I don't know why there aren't more possible permissions for Azure Service Management)
the relevant code is rather basic but it works so I don't feel like I need post it, I'll just say that I obtain the Token using Microsoft.IdentityModel.Clients.ActiveDirectory.AcquireTokenAsync() with
authorityUri = "https://login.windows.net/(mytenantid)",
string resourceUri = "https://management.azure.com/";
AuthenticationContext authContext = new AuthenticationContext(authorityUri);
var res = authContext.AcquireTokenAsync(resourceUri, new
ClientCredential(clientId,clientSecret));
return res.Result;
and make the Request to
https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Authorization/roleAssignments?api-version=2016-03-01&$filter=atScope()
(as an example, this one is supposed to call the roles).
Tried several different types of GET Requests to different URIs, all give similar errors.
I'm thinking it might not be an issue with the code but a setting in the AzurePortal but everything I found there seems set up right (or rather "universal access").
According to your description, you forget to grant your service principal. You could add it on Azure Portal. You could grant Contributor or Owner role to it.
Please refer to this link:Assign application to role.
I'm trying to use Azure WebJobs SDK to trigger a function when a message is posted on a queue.
This works fine when setting StorageConnectionString to a connection string with the storage account key.
I would like to use a Shared Access Token (SAS) which has access to that queue (and only that) in the StorageConnectionString but getting errors:
Message=Failed to validate Microsoft Azure WebJobs SDK Storage
connection string. The Microsoft Azure Storage account connection
string is not formatted correctly. Please visit
http://msdn.microsoft.com/en-us/library/windowsazure/ee758697.aspx for
details about configuring Microsoft Azure Storage connection strings.
And:
Message=The account credentials for '' are incorrect.
Source=Microsoft.Azure.WebJobs.Host
StackTrace:
at Microsoft.Azure.WebJobs.Host.Executors.DefaultStorageCredentialsValidator.<ValidateCredentialsAsyncCore>d__4.MoveNext()
The connection string I'm using is formatted this way:
BlobEndpoint=https://myaccount.blob.core.windows.net/;QueueEndpoint=https://myaccoount.queue.core.windows.net/queuename;SharedAccessSignature=token
Any chance StorageConnectionString requires access to the whole storage account? If so, do you have an idea what I could do?
Looking at the WebjobSDK code: https://github.com/Azure/azure-webjobs-sdk/tree/dev/src it looks like the exception you are facing is thrown by the storage account parser. Looking at the code, it parses as follows:
public static StorageAccountParseResult TryParseAccount(string connectionString, out CloudStorageAccount account)
{
if (String.IsNullOrEmpty(connectionString))
{
account = null;
return StorageAccountParseResult.MissingOrEmptyConnectionStringError;
}
CloudStorageAccount possibleAccount;
if (!CloudStorageAccount.TryParse(connectionString, out possibleAccount))
{
account = null;
return StorageAccountParseResult.MalformedConnectionStringError;
}
account = possibleAccount;
return StorageAccountParseResult.Success;
}
I checked the formatting you sent using CloudStorageAccount and it seems to pass. Notice that you have an unnecessary '/' after the blob endpoint, maybe you are missing some text and that is causing the parsing to fail.
According to your description, I followed this official document to Configure Azure Storage Connection Strings. Occasionally I encountered the error as you mentioned: The account credentials for '' are incorrect.
As I known, Azure WebJobs SDK would reference the Azure Storage client library, which is a wrapper of Azure Storage Service REST API. For troubleshooting the similar issue, you could leverage Fiddler to capture the network package. Here is the screenshot when I caught the above error via Fiddler:
Any chance StorageConnectionString requires access to the whole storage account? If so, do you have an idea what I could do?
I assumed that there be something wrong with your connection string.
QueueEndpoint=https://myaccoount.queue.core.windows.net/queuename
Here is my connection string that include an account SAS for blob and queue storage within my WebJob project, you could refer to it.
<add name="AzureWebJobsStorage" connectionString="BlobEndpoint=https://brucechen01.blob.core.windows.net/;QueueEndpoint=https://brucechen01.queue.core.windows.net/;SharedAccessSignature=sv=2015-12-11&ss=bq&srt=sco&sp=rwdlacup&se=2016-12-31T18:39:25Z&st=2016-12-25T10:39:25Z&spr=https&sig={signature}" />
Note: If you are specifying a SAS in a connection string in a configuration file, you may need to encode special characters in the URL via Html Encode.
UPDATE:
As you mentioned in the comment below My SAS includes permissions to the queue with the "queuename" only. Since you have configured SAS token for both Blob and Queue, I assumed that you need to create an account SAS token for blob and queue service. You could leverage Microsoft Azure Storage Explorer to create the SAS token as follow:
Choose your storage account, right click and select "Get Shared Access Signature".
Note: When you replace the value of SharedAccessSignature with the generated SAS token, you need to remove the first ? symbol in your SAS token.
I have created a new application in Azure AD using the AAD Graph API. (code)
Unfortunately it doesn't let my client access the requested resources until I have been to the application's configuration page in the Azure management portal and made a cosmetic change, and then saved it. After removing the change and saving again, it still works.
The application manifest files before the change + change back steps and after them are completely identical (as in diff.exe says they are the same).
When comparing the JWT tokens returned when the application authenticates, it shows that the post-change access token includes the "roles" section. The entire "roles" section is not present in the access token returned before saving the application in the management portal.
So it seems the Azure management portal does "something" to the application when saving changes. The question is what it is, and can I do the same using the AAD graph API?
There were several issues. Some bugs in the backend on Azure, which have now been fixed, and also some missing calls to the API which I didn't know were necessary.
Thanks to some very helpful people at MS Support, we were able to get it to work.
When creating an application, you need to do the following:
Create an application object.
Setup the RequiredResourceAccess for the application, ie. which permissions the appliation has to Azure Graph API etc. This is what is configured in the portal's "permissions to other applications" settings. You can get the necessary GUIDs by configuring the permissions manually, and then looking in the application's AAD manifest file.
Create a service principal for the application.
Add AppRoleAssignments to the service principal.
The final part is what I was missing before. Even though you have configured RequiredResourceAccess on the application object, the service principal still needs the AppRoleAssignments to actually have permission to access the resources.
When creating the AppRoleAssignments it is a little bit tricky to figure out which PrincipalId to assign, since that is the AAD ObjectId of the service principal for the other resource.
Here is a snippet for adding the AppRoleAssignment to access the Azure AD Graph API. client is an ActiveDirectoryClient instance, and sp is the ServicePrincipal for my application:
// find the azure ad service principal
var aadsp =
client.ServicePrincipals.Where(csp => csp.AppId == "00000002-0000-0000-c000-000000000000")
.ExecuteSingleAsync().Result;
// create the app role assignment
var azureDirectoryReadAssignment = new AppRoleAssignment
{
PrincipalType = "ServicePrincipal",
PrincipalId = Guid.Parse(sp.ObjectId), //
Id = Guid.Parse("5778995a-e1bf-45b8-affa-663a9f3f4d04"), // id for Directory.Read
// azure active directory resource ID
ResourceId = Guid.Parse(aadsp.ObjectId) // azure active directory resource ID
};
// add it to the service principal
sp.AppRoleAssignments.Add(azureDirectoryReadAssignment);
// update the service principal in AAD
await sp.UpdateAsync();
My experience is that you need to wait a short time, maybe 2-3 minutes, before the newly created objects are valid in AAD, and then you can authenticate using the new application.
Apart from RasmusW's answer above, there a few more things that you might have to do depending on what you are trying to achieve.
If you want delegated permissions to work, you also need to add an Oauth2PermissionGrant into Oauth2PermissionGrants collection at the root level. This should have clientId of caller's SPN ObjectId, ResourceId of called SPN's object Id. The Scope value of the Oauth2PermissionGrant is key. It should have space separated values. Each value here comes from the 'Value' property of the Oauth2Permission object on the target SPN.
Additionally you may also need to be in appropriate DirectoryRole e.g. Directory Readers.