Using C# I can delete Azure Batch Pools and Jobs using Client ID and Client Secret - but currently we want to delete them by using Azure Functions using Managed Identity.
Here is my current code:
internal async Task<string> GetAuthenticationTokenAsync()
{
var authContext = new AuthenticationContext(AuthorityUri);
var authResult = await authContext.AcquireTokenAsync(BatchResourceUri, new ClientCredential(BatchCredentials["ClientId"], BatchCredentials["ClientKey"])).ConfigureAwait(false);
return authResult.AccessToken;
}
Task<string> TokenProvider() => GetAuthenticationTokenAsync();
using (var Batch = BatchClient.Open(new BatchTokenCredentials(BatchCredentials["BatchAccountURL"], TokenProvider)))
{
var CloudPools = Batch.PoolOperations.ListPools();
var JobList = Batch.JobOperations.ListJobs();
foreach (var pool in CloudPools)
{
pool.DeleteAsync();
}
foreach (var job in JobList)
{
job.DeleteAsync();
}
}
I see that in msdn social that there is no support for MSI currently in Azure Batch, so is there any alternative to just delete the Azure Batch Pools and Jobs ?
note: if it is not possible in C#, I am comfortable using Rest API or PowerShell also for deleting the pools and jobs of the batch account
2021-02-17 Updated Answer:
Managed Identity on Batch pools is now in public preview in select regions. Please see this doc.
Original Answer:
Managed Identity is not supported on Azure Batch compute nodes, however, you can use Managed Identities on other Azure resources that support it to authenticate with the Azure Batch resource provider. Please see this doc.
Related
I have been dealing with this issue for which I am not able to find solution online anywhere.
I have a code which connects to AWS DynmoDb and performs read/write operations on one or more tables. This worked fine as long as my code and the DynamoDb table are in the same AWS account. Also the code uses the IAM Role attached to the Web Server. The role as all the necessary permissions assigned to it.
private AmazonDynamoDBClient GetDbClient(int ConnectionTimeOut, int ReadWriteTimeOut, int MaxRetry)
{
AmazonDynamoDBConfig clientConfig = new AmazonDynamoDBConfig
{
Timeout = TimeSpan.FromMilliseconds(ConnectionTimeOut),
ReadWriteTimeout = TimeSpan.FromMilliseconds(ReadWriteTimeOut),
MaxErrorRetry = MaxRetry
};
return new AmazonDynamoDBClient(clientConfig);
}
Recently I need to move my code to different AWS account and things started going crazy.
Following steps I have already taken.
VPC Peering done between the VPC in the old AWS account and the new AWS account.
Cross account permissions on the DynamobDb tables are given to the role which is used by the Web server on the new AWS Account.
With this change, I do not see any more permission errors but the code tries to look for the table on the new AWS account.
It is clear in the code that AWS AccountId is not used anywhere while creating AWS DynamoDb client. So I assume that I should be able to tell the code where to look for DynamoDb table. But the C# SDK of DynamoDb does not have any provision where I can provide AWS AccountId while creating DynamoDb client.
So my issue here is related to C# code to connect to DynamoDb service and not the IAM roles and permissions on AWS (for this I am able to fine plenty of solution).
Found this question aws cross account dynamodb access with IAM role with similar issue but it does not suggest the fix to do in the code.
One way to proceed is to use Security Token Service. First you need to assume a role and get temporary credentials:
Credentials GetCredentials(String roleArn)
{
using (var stsClient = new AmazonSecurityTokenServiceClient())
{
try
{
var response = stsClient.AssumeRole(new AssumeRoleRequest(roleARN));
if (response.HttpStatusCode == System.Net.HttpStatusCode.OK) return response.Credentials;
}
catch (AmazonSecurityTokenServiceException ex)
{
return null;
}
}
}
You can then use the credentials to initiate your DynamoDB client.
See another example here.
The AWS SDK and CLI (whether it's running locally or on (say) an EC2 instance) looks in the following locations for credentials:
Command line options
Environment variables
CLI credentials file
CLI configuration file
Container credentials
Instance profile credentials
If you have a credentials file configured, then, assuming we're running under the default profile, this will indirectly define the account under which it is running via the access key provided.
You can also define AWS-specific environment variables, such as AWS_ACCESS_KEY_ID which take precedence over the credentials file.
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-quickstart.html
I was trying to create a multi node pool in Azure Batch (User Subscription) using C#. I have assigned the RBAC role of "Virtual Machine Contributor" to the Vnet for "Microsoft Azure Batch" along with it's already existing Contributor role. While I was able to create it successfully without Virtual Network Configuration, as soon as I add the property it fails with a Bad Request Exception. This is the snippet of code which i am using.
pool.NetworkConfiguration = new NetworkConfiguration
{
DynamicVNetAssignmentScope = 0,
PublicIPAddressConfiguration = new PublicIPAddressConfiguration(IPAddressProvisioningType.NoPublicIPAddresses),
SubnetId = "/subscriptions/<subscriptionId>/resourceGroups/<ResrouceGroupName>/providers/Microsoft.Network/virtualNetworks/<vnetName>/subnets/<subnetName>",
};
I am trying to programmatically add new MySQL db instance to azure portal.
I looked at this library: https://github.com/Azure/azure-libraries-for-net but i only see an example of how to create a SQL Server and not a MySQL.
var credentials = SdkContext.AzureCredentialsFactory.FromFile(Environment.GetEnvironmentVariable("AZURE_AUTH_LOCATION"));
var azure = Azure
.Configure()
.WithLogLevel(HttpLoggingDelegatingHandler.Level.Basic)
.Authenticate(credentials)
.WithDefaultSubscription();
var sqlServer = azure.SqlServers.Define(sqlServerName)
.WithRegion(Region.USEast)
.WithNewResourceGroup(rgName)
.WithAdministratorLogin(AdministratorLogin)
.WithAdministratorPassword(AdministratorPassword)
.WithNewFirewallRule(FirewallRuleIPAddress)
.WithNewFirewallRule(FirewallRuleStartIPAddress, FirewallRuleEndIPAddress)
.Create();
var database = sqlServer.Databases
.Define(DatabaseName)
.Create();
Any idea if its supported to programmatically create a MySQL server as well?
Looking at the release notes and SDK code, it seems the ability to manage MySQL databases is still not supported (as of version 1.3).
What you could do is consume the REST API for managing Azure MySQL databses directly. For creation of database, please look at Create Or Update Database operation.
To get started with Azure REST API, you may find this link useful: https://learn.microsoft.com/en-us/rest/api/.
Im developing a service fabric Stateless Service App and at some point i need to download a file and save it to my desktop folder, but even if i run as Administrator i get the error:
Access to the path 'C:/Users/me/Desktop/filename' is denied
using (var response = await _httpClient.SendAsync(httpRequestMessage, HttpCompletionOption.ResponseHeadersRead))
{
using (var streamToReadFrom = await response.Content.ReadAsStreamAsync())
{
var fileName = $"C:\\Users\\me\\Desktop\\{name}{DateTime.Now.ToString("yyyyMMdd")}";
using (var streamToWriteTo = File.Create(fileName))
{
await streamToReadFrom.CopyToAsync(streamToWriteTo);
}
}
}
Im testing runnning a local debug cluster but when i deploy it i want to save to a network path.
If you run Task Manager and go to the Details tab, you'll see that Service Fabric's processes (Fabric.exe, FabricGateway.exe, and others) are running as the NETWORK SERVICE user, not your own user account. The NETWORK SERVICE user doesn't have access to your user profile.
It's hard to suggest what you could do without knowing more about why you're trying to save files. But if that's something you're going to continue to do once your service makes it to production, I'd just implement your final solution now. I guess that's probably saving to Blob Storage, or maybe a database of some kind.
I have been attempting to upload files using c# into Hadoop using the WebHDFS REST API.
This code works fine:
using (var client = new System.Net.WebClient())
{
string result = client.DownloadString("http:/ /host:50070/webhdfs/v1/user/myuser/?op=LISTSTATUS");
client.DownloadFile("http:/ /host:50070/webhdfs/v1/user/myuser/tbible.txt?user.name=myuser&op=OPEN","d:\tbible.txt");
}
This code gets a 403 Forbidden:
using (var client = new System.Net.WebClient())
{
client.UploadFile("http:/ /host:50070/webhdfs/v1/user/myuser/?user.name=myuser&op=CREATE", "PUT", "d:\bible.txt");
}
I have tried adding a network credential, with no luck.
How do I authenticate to our Cluster from .NET? The Cluster is Hortonworks HDP1.3 on RHEL5.
(The extra spaces in this post are to keep http:/ / from being a link)
Also, I would have liked to use Microsoft's hadoop SDK, but it is alpha and wont compile in my environment :(
Make sure that you are writing to a directory that is under the group which WebHDFS operates under. By default this is hdfs.
A quick way to check this doing hadoop fs -ls on the directory's parent directory to get the group permission settings (the second column that may look like a username).