I am working with BigQuery. I create a DataSet and I want to define access rights with C# language.
It's not clear to me how to do it.
In GOOGLE web page https://cloud.google.com/bigquery/docs/dataset-access-controls is explained how to do it with some example for Java and Pyton (see below), but no example is provided for c#.
example in pyton:
dataset = client.get_dataset(dataset_id) # Make an API request.
entry = bigquery.AccessEntry(
role="READER",
entity_type="userByEmail",
entity_id="sample.bigquery.dev#gmail.com",
)
entries = list(dataset.access_entries)
entries.append(entry)
dataset.access_entries = entries
dataset = client.update_dataset(dataset, ["access_entries"]) # Make an API request.
full_dataset_id = "{}.{}".format(dataset.project, dataset.dataset_id)
Can anybody help please?
It's probably best to use the BigQueryDataset.Patch method, from the Google.Cloud.BigQuery.V2 package:
// Fetch the existing dataset
var client = BigQueryClient.Create(projectId);
var dataset = client.GetDataset(datasetId);
var accessList = dataset.Resource.Access ?? new List<AccessData>();
accessList.Add(new AccessData
{
Role = "Reader",
UserByEmail = "sample.bigquery.dev#gmail.com"
});
var patchedResource = new Dataset { Access = accessList };
// Push the changes back up to BigQuery
dataset.Patch(patchedResource, matchETag: true);
As an alternative, you can use Update to update the replace the dataset resource completely:
// Fetch the existing dataset
var client = BigQueryClient.Create(projectId);
var dataset = client.GetDataset(datasetId);
// Modify it in memory
var resource = dataset.Resource;
if (resource.Access is null)
{
// If there's no access list yet, create one.
resource.Access = new List<AccessData>();
}
var newAccess = new AccessData
{
Role = "Reader",
UserByEmail = "sample.bigquery.dev#gmail.com"
};
resource.Access.Add(newAccess);
// Push the changes back up to BigQuery
dataset.Update();
At the and I managed to make it work.
I used the same solution suggested by Jon Skeet, using the Patch method.
I attach my code here by.
public static bool GrantDatasetAccess(string dataSetId, bool online, string role, string email,
string bigQueryJsonPath, string bigQueryScope, string projectId, clsLog log)
{
try
{
BigQueryClient client = GetBigQueryClient(online, bigQueryJsonPath, bigQueryScope, projectId);
BigQueryDataset dataset = GetDataSet(online, dataSetId, bigQueryJsonPath, bigQueryScope, projectId, log);
List<AccessData> accessList = new List<AccessData>();
var accessData = new AccessData()
{
Role = role,
GroupByEmail = null,
UserByEmail = email,
SpecialGroup = null,
IamMember = null,
Domain = null,
View = null
};
accessList.Add(accessData);
dataset.Resource.Access = accessList;
dataset.Patch(dataset.Resource, true);
}
catch (Exception e)
{
log.ManageError("Error GetDataSet: {0}\nError: {1}", dataSetId, string.Concat(e.Message, "\n", e.StackTrace));
}
return true;
}
Related
I have implemented the Microsoft Example for Embed for your customers from Github, which works perfectly. Link
I am now extending it which there are articles showing using both V1 and V2 of the API, both result in the same error:
Operation returned an invalid status code 'BadRequest'
at
Microsoft.PowerBI.Api.ReportsOperations.GenerateTokenInGroupWithHttpMessagesAsync(Guid
groupId, Guid reportId, GenerateTokenRequest requestParameters,
Dictionary`2 customHeaders, CancellationToken cancellationToken)
[HttpGet]
public async Task<string> GetEmbedInfo()
{
try
{
// Validate whether all the required configurations are provided in appsettings.json
string configValidationResult = ConfigValidatorService.ValidateConfig(azureAd, powerBI);
if (configValidationResult != null)
{
HttpContext.Response.StatusCode = 400;
return configValidationResult;
}
EmbedParams embedParams = await pbiEmbedService.GetEmbedParams(new Guid(powerBI.Value.WorkspaceId), new Guid(powerBI.Value.ReportId));
//EmbedParams embedParams = await pbiEmbedService.GetEmbedToken4(new Guid(powerBI.Value.WorkspaceId), new Guid(powerBI.Value.ReportId));
return JsonSerializer.Serialize<EmbedParams>(embedParams);
}
catch (Exception ex)
{
HttpContext.Response.StatusCode = 500;
return ex.Message + "\n\n" + ex.StackTrace;
}
}
The above code is getting called and per the demo.
public async Task<EmbedParams> GetEmbedParams(Guid workspaceId, Guid reportId, [Optional] Guid additionalDatasetId)
{
PowerBIClient pbiClient = this.GetPowerBIClient();
// Get report info
var pbiReport = await pbiClient.Reports.GetReportInGroupAsync(workspaceId, reportId);
//var generateTokenRequestParameters = new GenerateTokenRequest("View", null, identities: new List<EffectiveIdentity> { new EffectiveIdentity(username: "**************", roles: new List<string> { "****", "****" }, datasets: new List<string> { "datasetId" }) });
//var tokenResponse = pbiClient.Reports.GenerateTokenInGroupAsync("groupId", "reportId", generateTokenRequestParameters);
// Create list of datasets
var datasetIds = new List<Guid>();
// Add dataset associated to the report
datasetIds.Add(Guid.Parse(pbiReport.DatasetId));
// Append additional dataset to the list to achieve dynamic binding later
if (additionalDatasetId != Guid.Empty)
{
datasetIds.Add(additionalDatasetId);
}
// Add report data for embedding
var embedReports = new List<EmbedReport>() {
new EmbedReport
{
ReportId = pbiReport.Id, ReportName = pbiReport.Name, EmbedUrl = pbiReport.EmbedUrl
}
};
// Get Embed token multiple resources
var embedToken = await GetEmbedToken4(workspaceId, reportId);
// Capture embed params
var embedParams = new EmbedParams
{
EmbedReport = embedReports,
Type = "Report",
EmbedToken = embedToken
};
return embedParams;
}
The above code is per the demo apart from one line, which is calling the next method:
var embedToken = await GetEmbedToken4(workspaceId, reportId);
public EmbedToken GetEmbedToken(Guid reportId, IList<Guid> datasetIds, [Optional] Guid targetWorkspaceId)
{
PowerBIClient pbiClient = this.GetPowerBIClient();
// Create a request for getting Embed token
// This method works only with new Power BI V2 workspace experience
var tokenRequest = new GenerateTokenRequestV2(
reports: new List<GenerateTokenRequestV2Report>() { new GenerateTokenRequestV2Report(reportId) },
datasets: datasetIds.Select(datasetId => new GenerateTokenRequestV2Dataset(datasetId.ToString())).ToList(),
targetWorkspaces: targetWorkspaceId != Guid.Empty ? new List<GenerateTokenRequestV2TargetWorkspace>() { new GenerateTokenRequestV2TargetWorkspace(targetWorkspaceId) } : null
);
// Generate Embed token
var embedToken = pbiClient.EmbedToken.GenerateToken(tokenRequest);
return embedToken;
}
The above code is per the example with no roles being passed in or EffectiveIdentity. This works.
public async Task<EmbedToken> GetEmbedToken4(Guid workspaceId, Guid reportId, string accessLevel = "view")
{
PowerBIClient pbiClient = this.GetPowerBIClient();
var pbiReport = pbiClient.Reports.GetReportInGroup(workspaceId, reportId);
string dataSet = pbiReport.DatasetId.ToString();
// Generate token request for RDL Report
var generateTokenRequestParameters = new GenerateTokenRequest(
accessLevel: accessLevel,
datasetId: dataSet,
identities: new List<EffectiveIdentity> { new EffectiveIdentity(username: "******", roles: new List<string> { "********" }) }
);
// Generate Embed token
var embedToken = pbiClient.Reports.GenerateTokenInGroup(workspaceId, reportId, generateTokenRequestParameters);
return embedToken;
}
This is the method to return the token with the roles and effective Identity. This results in the error, but no message or helpful feedback.
OK, after much research overnight the Bad Request response does hide an English message which is not show in the browser. The debugger doesn't have the symbols for the part that causes the error, but I found it by using Fiddler proxy when the actual API responded to the request. In my case, if you send an ID to enable RLS, but the version of the report on the server doesn't have it, this doesn't ignore it, it refuses to give a token to anything. From reading many posts, the Bad Request is just a poor error message when the actual response from the API itself (not the package or the example code that the sample uses with it presents). Hope this helps someone in the future.
I was having as tough time getting Microsoft.Azure.Cosmos.Table to automatically initialise the SecondaryUri when parsing a connection string that used a SAS token.
So I ended up explicitly specifying the TableSecondaryEndpoint in the connection string, that works but I'm unable to query the secondary because the SDK throws an Exception before even attempting the request.
In my testing, I have identified that this is a regression not present in Microsoft.WindowsAzure.Storage.Table 8.7.0 (The basis for Microsoft.Azure.Cosmos.Table 1.0.6)
Expert opinions very welcome that this point. Thank you.
Project code for this Exception here (also copied below): https://github.com/golfalot/SOshowAzureTableBug
Side issue detailing the SecondaryUri initialisation problem raised here: https://github.com/Azure/azure-cosmos-table-dotnet/issues/36
using System;
using System.Collections.Generic;
using LEGACY_STORAGE = Microsoft.WindowsAzure.Storage;
using LEGACY_RETRY = Microsoft.WindowsAzure.Storage.RetryPolicies;
using LEGACY_TABLE = Microsoft.WindowsAzure.Storage.Table; //8.7.0 because this is the base for 1.0.6
using NEWEST_TABLE = Microsoft.Azure.Cosmos.Table; // version 1.0.6
using Microsoft.Azure.Cosmos.Table; // had to add this to get access CreateCloudTableClient extension method
using System.Diagnostics;
namespace SOshowAzureTableBug
{
class Program
{
// the SAS token is immaterial in reproducing the problem
const string connectionTableSAS = "TableSecondaryEndpoint=http://127.0.0.1:10002/devstoreaccount1-secondary;TableEndpoint=http://127.0.0.1:10002/devstoreaccount1;SharedAccessSignature=immaterial";
static void Main(string[] args)
{
/* Legacy Table SDK */
var storageAccountLegacy = LEGACY_STORAGE.CloudStorageAccount.Parse(connectionTableSAS);
var tableClientLegacy = storageAccountLegacy.CreateCloudTableClient();
Debug.Assert(tableClientLegacy.StorageUri.SecondaryUri != null); // demonstrate SecondaryUri initialised
var tableRequestOptionsLegacy = new LEGACY_TABLE.TableRequestOptions () { LocationMode = LEGACY_RETRY.LocationMode.SecondaryOnly };
tableClientLegacy.DefaultRequestOptions = tableRequestOptionsLegacy;
var tableLegacy = tableClientLegacy.GetTableReference("foo"); // don't need table to exist to show the issue
var retrieveOperation = LEGACY_TABLE.TableOperation.Retrieve(string.Empty, string.Empty, new List<string>() { "bar" });
var tableResult = tableLegacy.Execute(retrieveOperation);
Console.WriteLine("Legacy PASS");
/* Newset Table SDK */
var storageAccountNewest = NEWEST_TABLE.CloudStorageAccount.Parse(connectionTableSAS);
var tableClientNewest = storageAccountNewest.CreateCloudTableClient(new TableClientConfiguration());
Debug.Assert(tableClientNewest.StorageUri.SecondaryUri != null); // demonstrate SecondaryUri initialised
var tableRequestOptionsNewest = new NEWEST_TABLE.TableRequestOptions() { LocationMode = NEWEST_TABLE.LocationMode.SecondaryOnly };
tableClientNewest.DefaultRequestOptions = tableRequestOptionsNewest;
var tableNewset = tableClientNewest.GetTableReference("foo"); // don't need table to exist to show the issue
var retrieveOperationNewset = NEWEST_TABLE.TableOperation.Retrieve(string.Empty, string.Empty, new List<string>() { "bar" });
/* throws Microsoft.Azure.Cosmos.Table.StorageException
* Exception thrown while initializing request: This operation can only be executed against the primary storage location
*/
var tableResultNewset = tableNewset.Execute(retrieveOperationNewset);
Console.WriteLine("Press any key to exit");
Console.Read();
}
}
}
I believe you've encountered a bug with the SDK.
When I try the following code, I get the same error as you:
var account = CloudStorageAccount.Parse(connectionString);
var requestOptions = new TableRequestOptions()
{
LocationMode = LocationMode.SecondaryOnly
};
var client = account.CreateCloudTableClient();
client.DefaultRequestOptions = requestOptions;
var table = client.GetTableReference("myTable");
var op = TableOperation.Retrieve("", "");
var result1 = table.Execute(op);
I decompiled the library code and found the culprit source code:
if (commandLocationMode == CommandLocationMode.PrimaryOnly)
{
if (restCMD.LocationMode == LocationMode.SecondaryOnly)
{
throw new InvalidOperationException("This operation can only be executed against the primary storage location.");//This is the error that gets thrown.
}
Logger.LogInformational(executionState.OperationContext, "This operation can only be executed against the primary storage location.", Array.Empty<object>());
executionState.CurrentLocation = StorageLocation.Primary;
restCMD.LocationMode = LocationMode.PrimaryOnly;
}
However, if I don't set DefaultRequestOptions at client level and specify it below in Execute method, I don't get the error but then it's because the primary endpoint is hit instead of secondary (I checked that in Fiddler).
var account = CloudStorageAccount.Parse(connectionString);
var requestOptions = new TableRequestOptions()
{
LocationMode = LocationMode.SecondaryOnly
};
var client = account.CreateCloudTableClient();
var table = client.GetTableReference("myTable");
var op = TableOperation.Retrieve("", "");
var result1 = table.Execute(op, requestOptions);
Workaround
If your objective is to query entities from secondary location, then you can use ExecuteQuery method on CloudTable like shown below. This works (Again, I checked in Fiddler).
var account = CloudStorageAccount.Parse(connectionString);
var requestOptions = new TableRequestOptions()
{
LocationMode = LocationMode.SecondaryOnly
};
var client = account.CreateCloudTableClient();
client.DefaultRequestOptions = requestOptions;
var table = client.GetTableReference("myTable");
TableQuery query = new TableQuery();
var result = table.ExecuteQuery(query).ToList();
So I finally figured out a way to successfully make detect intent calls and provide an input context. My question is whether or not this is the CORRECT (or best) way to do it:
(And yes, I know you can just call DetectIntent(agent, session, query) but I have to provide a input context(s) depending on the request)
var query = new QueryInput
{
Text = new TextInput
{
Text = model.Content,
LanguageCode = string.IsNullOrEmpty(model.Language) ? "en-us" : model.Language,
}
};
var commonContext = new global::Google.Cloud.Dialogflow.V2.Context
{
ContextName = new ContextName(agent, model.sessionId, "my-input-context-data"),
LifespanCount = 3,
Parameters = new Struct
{
Fields = {
{ "Source", Value.ForString(model.Source) },
{ "UserId" , Value.ForString(model.UserId.ToString())},
{ "Name" , Value.ForString(model.FirstName)}
}
}
};
var request = new DetectIntentRequest
{
SessionAsSessionName = new SessionName(agent, model.sessionId),
QueryParams = new QueryParameters
{
GeoLocation = new LatLng {Latitude = model.Latitude, Longitude = model.Longitude},
TimeZone = model.TimeZone ?? "MST"
},
QueryInput = query
};
request.QueryParams.Contexts.Add(commonContext);
// ------------
var creds = GetGoogleCredentials("myCredentials.json");
var channel = new Grpc.Core.Channel(SessionsClient.DefaultEndpoint.Host, creds.ToChannelCredentials());
var client = SessionsClient.Create(channel);
var response = client.DetectIntent(request);
channel.ShutdownAsync();
return response;
Note: I included the explicit ShutDownAsync (it's not in an async call) because I was getting some file locking issues when attempting to re-deploy the WebAPI project (and only after having executed this code).
Thanks
Chris
Updated 4/25: The most basic way I use this is to integrate the user's name into intent responses:
It can also be read from within the webhook/inline fulfillment index.js:
const name = request.body.queryResult && request.body.queryResult.outputContexts && request.body.queryResult.outputContexts[0].parameters.Name
I trying using Google Analytics with C# to get stats information to display in my webiste
Here is my code
public ActionResult Index()
{
string userName = "admin#email.com";
string passWord = "mypass";
string profileId = "ga:xxxxxxxx";
string key = "2d751338cb092ef8da65f716e37a48604386c9sw";
string dataFeedUrl = "https://www.google.com/analytics/feeds/data"+key;
var service = new AnalyticsService("API Project");
service.setUserCredentials(userName, passWord);
var dataQuery = new DataQuery(dataFeedUrl)
{
Ids = profileId,
Metrics = "ga:pageviews",
Sort = "ga:pageviews",
GAStartDate = new DateTime(2010, 3, 1).ToString("yyyy-MM-dd"),
GAEndDate = DateTime.Now.ToString("yyyy-MM-dd")
};
var dataFeed = service.Query(dataQuery);
var totalEntry = dataFeed.Entries[0];
ViewData["Total"] = ((DataEntry)(totalEntry)).Metrics[0].Value;
dataQuery.GAStartDate = DateTime.Now.AddDays(-1).ToString("yyyy-MM-dd");
dataQuery.GAEndDate = DateTime.Now.AddDays(-1).ToString("yyyy-MM-dd");
dataFeed = service.Query(dataQuery);
var yesterdayEntry = dataFeed.Entries[0];
ViewData["Yesterday"] = ((DataEntry)(yesterdayEntry)).Metrics[0].Value;
dataQuery.GAStartDate = DateTime.Now.ToString("yyyy-MM-dd");
dataQuery.GAEndDate = DateTime.Now.ToString("yyyy-MM-dd");
dataFeed = service.Query(dataQuery);
var todayEntry = dataFeed.Entries[0];
ViewData["Today"] = ((DataEntry)(todayEntry)).Metrics[0].Value;
return View(dataFeed.Entries);
}
But when i run the code it always said "{"Invalid credentials"}"
Not sure why i facing this error while i checked many time about the key,username,password and profileId
Anyone facing this problem,can help me?
Many thanks
I think that your url is wrong. try in this way (you are missing ?key=).
string dataFeedUrl = "https://www.google.com/analytics/feeds/data?key="+key;
refer this google example where there is this example that should help you
public DataFeedExample()
{
// Configure GA API.
AnalyticsService asv = new AnalyticsService("gaExportAPI_acctSample_v2.0");
// Client Login Authorization.
asv.setUserCredentials(CLIENT_USERNAME, CLIENT_PASS);
// GA Data Feed query uri.
String baseUrl = "https://www.google.com/analytics/feeds/data";
DataQuery query = new DataQuery(baseUrl);
query.Ids = TABLE_ID;
query.Dimensions = "ga:source,ga:medium";
query.Metrics = "ga:visits,ga:bounces";
query.Segment = "gaid::-11";
query.Filters = "ga:medium==referral";
query.Sort = "-ga:visits";
query.NumberToRetrieve = 5;
query.GAStartDate = "2010-03-01";
query.GAEndDate = "2010-03-15";
Uri url = query.Uri;
Console.WriteLine("URL: " + url.ToString());
// Send our request to the Analytics API and wait for the results to
// come back.
feed = asv.Query(query);
}
refer also this guide to configure your project
Also follow this guide to use OAuth 2.0
I use DotNetOpenAuth.
So.. I am getting looking good response which has state Authenticated.
That is fine.
Now I want to get user profile info but always getting NULL.
Here is the code.
private ServiceProviderDescription GetServiceDescription()
{
string ValidateTokenEndPoint = ConfigurationManager.AppSettings["identityOAuthValidateTokenEndPointUrl"];
string ValidateAuthorizationHeaderEndPoint = ConfigurationManager.AppSettings["identityOAuthValidateAuthorizationHeaderEndPointUrl"];
string AccessTokenEndPoint = ConfigurationManager.AppSettings["identityOAuthAccessTokenURL"];
bool UseVersion10A = Convert.ToBoolean(ConfigurationManager.AppSettings["identityOAuthUseVersion10a"]);
string RequestTokenStr = ConfigurationManager.AppSettings["identityOAuthRequestTokenURL"];
string UserAuthStr = ConfigurationManager.AppSettings["identityOAuthAuthorizeUserURL"];
string AccessTokenStr = ConfigurationManager.AppSettings["identityOAuthAccessTokenURL"];
string InvalidateTokenStr = ConfigurationManager.AppSettings["identityOAuthRequestInvalidateTokenURL"];
return new ServiceProviderDescription
{
AccessTokenEndpoint = new MessageReceivingEndpoint(AccessTokenStr, HttpDeliveryMethods.PostRequest),
RequestTokenEndpoint = new MessageReceivingEndpoint(RequestTokenStr, HttpDeliveryMethods.PostRequest),
UserAuthorizationEndpoint = new MessageReceivingEndpoint(UserAuthStr, HttpDeliveryMethods.PostRequest),
TamperProtectionElements = new ITamperProtectionChannelBindingElement[] { new HmacSha1SigningBindingElement() },
ProtocolVersion = DotNetOpenAuth.OAuth.ProtocolVersion.V10a
};
}
void GetUserProfile()
{
var tokenManager = TokenManagerFactory.GetTokenManager(TokenManagerType.InMemoryTokenManager);
tokenManager.ConsumerKey = ConfigurationManager.AppSettings["identityOAuthConsumerKey"];
tokenManager.ConsumerSecret = ConfigurationManager.AppSettings["identityOAuthConsumerSecret"];
var serviceDescription = GetServiceDescription();
var consumer = new WebConsumer(serviceDescription, tokenManager);
var result = consumer.ProcessUserAuthorization(response);
if (result != null) // It is always null
{
}
Well I checked 10 times and I am pretty sure that all URLs to create ServiceProviderDescription are correct.
Any clue?
Well
finally check your web.config app keys
add key="identityOAuthConsumerKey" value="put here correct data!!!"
add key="identityOAuthConsumerSecret" value="put here correct data!!!"
and if you use hosts file you have to put correct sitename as well
127.0.0.1 site1.host1.com