var cosmosClient = new CosmosClient(EndpointUrl, AuthorizationKey,
new CosmosClientOptions() {
AllowBulkExecution = true
});
var database = cosmosClient.GetDatabase(SourceDatabase);
var container = database.GetContainer(SourceContainerName);
I'm looking for an approach to programmatically find out
PartitionKeyPath for the container. Is there no API in the SDK to
obtain it from container object?
P.S. Microsoft displays partition key path in Azure portal.
Was wondering how do they display it?
In Azure SDK for .NET, JSON path used for containers partitioning can be found using
ContainerProperties.PartitionKeyPath Property
PartitionKeyPath defaults to "/PartitionKey". Refer: Ms doc
ContainerProperties cproperties = await container.ReadContainerAsync();
Console.WriteLine(cproperties.PartitionKeyPath);
Additionally, you can go ahead and share this feedback so the CosmosDB team can look into this idea.✌
To retrieve the partition key of a container you have to use the GetContainerQueryIterator() method. Depending on what is already available within your code, you could use something like this:
private static async Task<string> GetPartitionKey(Database database, string containerName)
{
var query = new QueryDefinition("select * from c where c.id = #id")
.WithParameter("#id", containerName);
using var iterator = database.GetContainerQueryIterator<ContainerProperties>(query);
while (iterator.HasMoreResults)
{
foreach (var container in await iterator.ReadNextAsync())
{
return container.PartitionKeyPath;
}
}
return null;
}
In version 3.31.2 you can access the PartitionKeyPath as follows
var containerResponse = await cosmosClient.GetDatabase(options.Value.Database)
.GetContainer(collectionId)
.ReadContainerAsync(cancellationToken: cancellationToken);
Console.WriteLine(containerResponse.Resource.PartitionKeyPath)
Related
I am upgrading a web app to use Azure SDK v 12.10
Before this, I was grabbing all the blobs in a virtual path from the container into a list and iterating through them.
var results = await blobContainer.
ListBlobsSegmentedAsync("publicfiles/images/" + customer.AzureFolderName, true, BlobListingDetails.All, 100, blobContinuationToken, null, null);
// Get the value of the continuation token returned by the listing call.
blobContinuationToken = results.ContinuationToken;
foreach (var item in results.Results)
{
var filename = GetFileNameFromBlobUri(item.Uri, customer.AzureFolderName);
var img = new EvaluationImage
{
ImageUrl = item.Uri.ToString(),
ImageCaption = GetCaptionFromFilename(filename),
IsPosterImage = filename.Contains("poster"),
ImagesForCompany = customer.CompanyName
};
images.Add(img);
}
All I can find from googling how to do so in the new SDK, yields this
await foreach (BlobItem blob in blobContainer.GetBlobsAsync(BlobTraits.None, BlobStates.None, $"publicfiles/images/{customer.AzureFolderName}"))
The problem is the "blob" variable (BlobItem) has minimal useable properties, for example, I need the Uri which was available before. The "blob" variable in prior versions had a multitude of useable properties. Is there some other type (not BlobItem) that has these properties that I'm not finding in my searching?
I don't see anything useable in BlobTraits or BlobStates either that change this.
It's not as straight forward to get the URI of the blob in SDK version 12.x. When you get a BlobItem as part of listing result, you will need to create a BlobClient object out of it and then you will be able to get the URI of the blob.
Here's the sample code to do so:
BlobServiceClient blobServiceClient = new BlobServiceClient(connectionString);
BlobContainerClient containerClient = blobServiceClient.GetBlobContainerClient(container);
var listingResult = containerClient.GetBlobsAsync();
await foreach (var blobItem in listingResult)
{
BlobClient blobClient = containerClient.GetBlobClient(blobItem.Name);
Uri blobUri = blobClient.Uri;
//Do something with the URI...
}
Code above makes use of Azure.Storage.Blobs (12.9.1) SDK.
I'm trying my hand at programmatic Azure tooling, specifically trying to copy/clone/move a Container from one Azure Blob Storage account to another one. I'm not seeing a good way to do this though with v12.x of the client library in its docs (so far at least).
For example:
BlobStorage1
|-- SomeContrainer
|-- blob1
|-- blob2
|-- blob3
BlobStorage2
|-- SomeOtherContrainer
|-- otherBlob
I want to programmatically move SomeContainer and all its blobs to BlobStorage2.
What I've tried so far:
docs read/consulated:
StartCopyFrom method
which doesn't seem to show how to go from account1 pushed to account2. You can pull down from account2 into account one but that's the opposite of what I'm after.
code attempt:
var localContainer = blobServiceClient.GetBlobContainerClient(localContainerName);
var blobs = localContainer.GetBlobs();
var remoteClient = new BlobServiceClient(remoteConnectionString);
var remoteContainer = remoteClient.GetBlobContainerClient(localContainerName);
foreach(var blob in blobs)
{
Console.WriteLine($"copying blob: {blob.Name}");
var sourceBlob = localContainer.GetBlobClient(blob.Name);
var remoteBlobClient = remoteContainer.GetBlobClient(blob.Name);
await remoteBlobClient.StartCopyFromUriAsync(sourceBlob.Uri);
}
Problem here is that I could copy from remote to local (via connection string), or within the same account since the URI would be quite similar, but not from the account I'm on to a separate storage account. What would be the recommended way to copy blobs (or containers whole sale if possible) with the client library (v12.x) from one account to a separate one?
Please try the code below:
static async Task CopyContainersAcrossAccounts()
{
var sourceAccountName = "source-account";
var sourceAccountKey = "source-account-key";
var targetAccountName = "target-account";
var targetAccountKey = "target-account-key";
Azure.Storage.StorageSharedKeyCredential sourceCredential = new Azure.Storage.StorageSharedKeyCredential(sourceAccountName, sourceAccountKey);
Azure.Storage.StorageSharedKeyCredential targetCredential = new Azure.Storage.StorageSharedKeyCredential(targetAccountName, targetAccountKey);
var sourceConnectionString = $"DefaultEndpointsProtocol=https;AccountName={sourceAccountName};AccountKey={sourceAccountKey};EndpointSuffix=core.windows.net;";
var sourceContainer = "source-container-name";
var targetConnectionString = $"DefaultEndpointsProtocol=https;AccountName={targetAccountName};AccountKey={targetAccountKey};EndpointSuffix=core.windows.net;";
var targetContainer = "target-container-name";
var sourceBlobContainerClient = new Azure.Storage.Blobs.BlobContainerClient(sourceConnectionString, sourceContainer);
var targetBlobContainerClient = new Azure.Storage.Blobs.BlobContainerClient(targetConnectionString, targetContainer);
var sourceBlobs = sourceBlobContainerClient.GetBlobs();
foreach (var blob in sourceBlobs)
{
//Get shared access signature with "Read" permission in case source container is a private container.
Azure.Storage.Sas.BlobSasBuilder blobSasBuilder = new Azure.Storage.Sas.BlobSasBuilder()
{
BlobContainerName = sourceContainer,
BlobName = blob.Name,
ExpiresOn = DateTime.UtcNow.AddHours(1)
};
blobSasBuilder.SetPermissions(Azure.Storage.Sas.BlobSasPermissions.Read);
var sasToken = blobSasBuilder.ToSasQueryParameters(sourceCredential).ToString();
var sourceBlobClient = sourceBlobContainerClient.GetBlobClient(blob.Name);
var targetBlobClient = targetBlobContainerClient.GetBlobClient(blob.Name);
var sourceBlobUri = new Uri($"{sourceBlobClient.Uri.AbsoluteUri}?{sasToken}");
await targetBlobClient.StartCopyFromUriAsync(sourceBlobUri);
}
}
I'm generating Power BI embed token using C# SDK.
using (var client = new PowerBIClient(new Uri(apiUrl), tokenCredentials))
{
var workspaceId = groupId.ToString();
var report = await client.Reports.GetReportInGroupAsync(workspaceId, reportId);
var generateTokenRequestParameters = new GenerateTokenRequest(accessLevel: "view");
var tokenResponsex = await client.Reports.GenerateTokenAsync(workspaceId, reportId, generateTokenRequestParameters);
result.EmbedToken = tokenResponsex;
result.EmbedUrl = report.EmbedUrl;
result.Id = report.Id;
}
I need to pass a parameter for filtering. But couldn't find a straightforward way to do this.
How do I get this done?
You can use RLS in embedded reports implemented with app owns data scenario (a single master account for authentication) by passing EffectiveIdentity information when generating access token for this report with GenerateTokenInGroup.
To implement RLS in the report itself you need either to use USERPRINCIPALNAME() DAX function to filter the data, or define roles and filter the data based on them. If you implement it with roles, after publishing the report go to dataset's security settings and add users to the roles.
To generate the token by providing an effective identity and roles membership, use code like this:
var credentials = new TokenCredentials(accessToken, "Bearer");
using (var client = new PowerBIClient(new Uri("https://api.powerbi.com"), credentials))
{
var datasets = new List<string>() { datasetId }; // Dataset's GUID as a string
var roles = new List<string>();
roles.Add('ROLE1');
roles.Add('ROLE2');
roles.Add('ROLE3');
var effectiveIdentity = new EffectiveIdentity('user#example.com', datasets, roles);
var r = new GenerateTokenRequest("view", effectiveIdentity);
var token = client.Reports.GenerateTokenInGroup(groupId, reportId, r).Token;
}
I'm using azure log analytics .NET SDK to execute some log analytics queries.
The nugget package I'm using for this SDK is Microsoft.Azure.OperationalInsights.
This allows me to issue some simple queries like in the following code sample :
Authentication & Simple query :
partial class QueryProvider
{
private OperationalInsightsDataClient _operationalInsightsDataClient;
private async void Authenticate()
{
// Retrieving the credentials and settings data from the app settings .
var domain = SettingsHelpers.PullSettingsByKey("domain");
var clientId = SettingsHelpers.PullSettingsByKey("clientId");
var workspaceId = SettingsHelpers.PullSettingsByKey("workspaceId");
var authEndpoint = SettingsHelpers.PullSettingsByKey("authEndpoint");
var clientSecret = SettingsHelpers.PullSettingsByKey("clientSecret");
var tokenAudience = SettingsHelpers.PullSettingsByKey("tokenAudience");
// Authenticating to the azure log analytics service .
var azureActiveDirectorySettings = new ActiveDirectoryServiceSettings
{
AuthenticationEndpoint = new Uri(authEndpoint),
TokenAudience = new Uri(tokenAudience),
ValidateAuthority = true
};
var credentials = await ApplicationTokenProvider.LoginSilentAsync
(
domain
, clientId
, clientSecret
, azureActiveDirectorySettings
);
_operationalInsightsDataClient = new OperationalInsightsDataClient(credentials);
_operationalInsightsDataClient.WorkspaceId = workspaceId;
}
public async Task<string> LogAnalyticsSamleQuery()
{
var query = #"Usage
| where TimeGenerated > ago(3h)
| where DataType == 'Perf'
| where QuantityUnit == 'MBytes'
| summarize avg(Quantity) by Computer
| sort by avg_Quantity desc nulls last";
var jsonResult = await _operationalInsightsDataClient.QueryAsync(query);
return JsonConvert.SerializeObject(jsonResult.Results);
}
}
Now I want to write a method that runs a cross-workspace query , I get the workspaces Ids dynamically and I want to build o query that references all those workspaces .
I did not find any sample in the doc to build such queries .
I found an attribute of OperationalInsightDataClient class called AdditionalWorkspaces but it's unclear how to use it to achieve the goal .
Any help would be very appreciated .
Use the ListWorkspaces method, store workspace Id , CustomerIdor Name in List.
var ws = new List<string>();
foreach (var w in workspaces)
{
ws.Add(w.Id);
}
AdditionalWorkspaces is used to store workspaces you want to query, but has no influence on the query result.
_operationalInsightsDataClient.AdditionalWorkspaces = ws;
To cross-workspace query, add the list of workspace-Id in the query method.
var jsonResult = await _operationalInsightsDataClient.QueryAsync(query,null, _operationalInsightsDataClient.AdditionalWorkspaces);
I've programmatically created a new document collection using the MongoDB C# driver.
At this point I want to create and build indexes programmatically. How can I do that?
Starting from v2.0 of the driver there's a new async-only API. The old API should no longer be used as it's a blocking facade over the new API and is deprecated.
The currently recommended way to create an index is by calling and awaiting CreateOneAsync with an IndexKeysDefinition you get by using Builders.IndexKeys:
static async Task CreateIndexAsync()
{
var client = new MongoClient();
var database = client.GetDatabase("HamsterSchool");
var collection = database.GetCollection<Hamster>("Hamsters");
var indexKeysDefinition = Builders<Hamster>.IndexKeys.Ascending(hamster => hamster.Name);
await collection.Indexes.CreateOneAsync(new CreateIndexModel<Hamster>(indexKeysDefinition));
}
you should use CreateIndex as EnsureIndex is marked obsolete for future compatibility with the next versions of MongoDB:
var client = new MongoClient("mongodb://localhost");
var db = client.GetServer().GetDatabase("db");
var collection = db.GetCollection<Hamster>("Hamsters");
collection.CreateIndex(IndexKeys<Hamster>.Ascending(_ => _.Name));
The overload of CreateOneAsync in the currently accepted answer is now marked as obsolete with the message "Use CreateOneAsync with a CreateIndexModel instead." Here's how you do it:
static async Task CreateIndex(string connectionString)
{
var client = new MongoClient(connectionString);
var database = client.GetDatabase("HamsterSchool");
var collection = database.GetCollection<Hamster>("Hamsters");
var indexOptions = new CreateIndexOptions();
var indexKeys = Builders<Hamster>.IndexKeys.Ascending(hamster => hamster.Name);
var indexModel = new CreateIndexModel<Hamster>(indexKeys, indexOptions);
await collection.Indexes.CreateOneAsync(indexModel);
}
Something like this should do:
var server = MongoServer.Create("mongodb://localhost");
var db = server.GetDatabase("myapp");
var users = db.GetCollection<User>("users");
users.EnsureIndex(new IndexKeysBuilder().Ascending("EmailAddress"));
Please see the following bits in the documentation:
http://api.mongodb.org/csharp/current/html/06bcd201-8844-3df5-d170-15f2b423675c.htm
There is an entire area on Indexing under the Definitions and Builders documentation page:
http://mongodb.github.io/mongo-csharp-driver/2.4/reference/driver/definitions/#index-keys
Example:
IndexKeysDefinition<MyModel> keys = "{ Reference: 1 }";
var indexModel = new CreateIndexModel<MyModel>(keys);
await _context.Indexes.CreateOneAsync(indexModel);
the easiest way to create indexes in c# is by using the driver wrapper library MongoDB.Entities. here's an example of creating a text index:
DB.Index<Author>()
.Key(a => a.Name, Type.Text)
.Key(a => a.Surname, Type.Text)
.Create();
and to do a full-text search, you simply do:
DB.SearchText<Author>("search term");
haven't seen anything else that makes it simpler than that.