We Keep Coding

How to create new AutoML DataSet for simple classification (C#)

As part of ML automation process I want to dynamically create new AutoML model. I'm using C# (.net framework) and Google.Cloud.AutoML.V1.
After trying to run CreateDataSet code:
var autoMlClient = AutoMlClient.Create();
var parent = LocationName.FromProjectLocation(_projectId, _locationId);
var dataset = new Google.Cloud.AutoML.V1.Dataset();
dataset.DisplayName = "NewDataSet";
var response = autoMlClient.CreateDataset(parent, dataset);
I get the following error:
Field: dataset.dataset_metadata; Message: Required field not set
According to this user manual I should set Dataset Metadata Type, but the list contains only specific types of classifications (Translation/ImageClassifications etc.), I can't find a simple classification type.
How do I create a simple classification data set with the API ? in the AutoML UI its just with a simple button click ("NEW DATASET") - and have to provide only name & region - no classification type.
I also tried to set:
dataset.TextClassificationDatasetMetadata =
new TextClassificationDatasetMetadata() { ClassificationType = ClassificationType.Multiclass };
But I was unable to import data to it (got too many errors of invalid inputs from the input CSV file), I guess its related to the reason that the input format is not suitable for Text Classification.
UPDATE
I've just notice that the Nuget works with AutoML v1 but v1 beta does contains TablesDatasetMetadata Dataset Metadata Type for normal classifications. I'm speechless.

I also experienced this scenario today while creating a dataset using the NodeJS client. Since the Google AutoML table service is in the beta level you need to use the beta version of the AutoML client. In the Google cloud documentation they have used the beta client to create a dataset.
In NodeJS importing the beta version require('#google-cloud/automl').v1beta1.AutoMlClient instead of importing the normal version (v1) require('#google-cloud/automl').v1 worked for me to successfully execute the create dataset functionality.
In C# you can achieve the same through a POST request. Hope this helps :)

After #RajithaWarusavitarana comment, and my last question update , below is the code that did the trick. The token is being generated by GoogleClientAPI nuget and AutoML is handled by REST.
string GcpGlobalEndPointUrl = "https://automl.googleapis.com";
string GcpGlobalLocation = "us-central1"; // api "parent" parameter
public string GetToken(string jsonFilePath)
{
var serviceAccountCredentialFileContents = System.IO.File.ReadAllText(jsonFilePath);
var credentialParameters = NewtonsoftJsonSerializer.Instance.Deserialize<JsonCredentialParameters>(serviceAccountCredentialFileContents);
var initializer = new ServiceAccountCredential.Initializer(credentialParameters.ClientEmail)
{
Scopes = new List<string> { "https://www.googleapis.com/auth/cloud-platform" }
};
var cred = new ServiceAccountCredential(initializer.FromPrivateKey(credentialParameters.PrivateKey));
string accessToken = cred.GetAccessTokenForRequestAsync("https://oauth2.googleapis.com/token").Result;
return accessToken;
}
public void GetDataSetList(string projectId, string token)
{
var restClient = new RestClient(GcpGlobalEndPointUrl);
var createDataSetReqUrl = $"v1beta1/projects/{projectId}/locations/{GcpGlobalLocation}/datasets";
var createDataSetReq = new RestRequest(createDataSetReqUrl, Method.GET);
createDataSetReq.AddHeader("Authorization", $"Bearer {token}");
var createDatasetResponse = restClient.Execute(createDataSetReq);
createDatasetResponse.Dump();
}
I took the token generation code from google-api-dotnet-client Test File

Getting files/folders from sharepoint via the api in c#

I'm trying to link a c# application to a sharepoint directory, so I can create folders, download and upload files. However I am strugling with connecting to the correct folder.
I can retrieve the content from allitems.aspx, but I am not sure how to actually get the content from folder.
I have tried using the ClientContext - something like this:
ClientContext cxt = new ClientContext("https://xx.sharepoint.com/sites/");
cxt.Credentials = GetCredentials();
List list = cxt.Web.Lists.GetByTitle("Kontrakter");
var test = list.Views;
var test1 = cxt.Web.Lists;
cxt.Load(test1);
cxt.Load(list);
cxt.Load(test);
var a = 4;
var fullUri = new Uri("https://xx.sharepoint.com/sites/yy/Kontrakter/AllItems.aspx");
//var folder = cxt.Web.GetFolderByServerRelativeUrl(fullUri.AbsolutePath);
using (var rootCtx = new ClientContext(fullUri.GetLeftPart(UriPartial.Authority)))
{
rootCtx.Credentials = GetCredentials();
Uri webUri = Web.WebUrlFromPageUrlDirect(rootCtx, fullUri);
using (var ctx1 = new ClientContext(webUri))
{
ctx1.Credentials = GetCredentials();
var list1 = ctx1.Web.GetList(fullUri.AbsolutePath);
ctx1.Load(list1.RootFolder.Files);
ctx1.ExecuteQuery();
Console.WriteLine(list.RootFolder.Files.Count);
}
}
or via normal api calls like this:
https://xx.sharepoint.com/_api/Web/GetFolderByServerRelativeUrl('Kontrakter/Forms')/Files
The only way I can find some data is if I look into 'Shared documents/Forms'
I'm having problems understanding the directory structure and how I can actually find the content of files/folders.
Thanks in advance :)

Turned out I was missing a /sites in one of my uris.

Get element by key with Firebase in c#

I am coming to a problem where I have a guid style schema in my firebase database, which I want to display a text that is DisplayText, but for some reason my code is not working. I am using a FirebaseDatabase.Net Wrapper. How can I map it in order to read from the database properly using a guid way schema? thanks for the help.
Code:
private async Task ShowQuestion()
{
var firebase = new
FirebaseClient("https://PROJECT_URL.firebaseio.com/");
var dinos = await firebase
.Child("Questions")
.OrderByKey()
.StartAt("DisplayText")
.LimitToFirst(1)
.OnceAsync<GameController>();
foreach (var dino in dinos)
{
Console.WriteLine(dinos);
}
I tried doing:
string page = "https://PROJECT_URL.firebaseio.com/Questions/DisplayText.json?orderBy"DisplayText"&limitToFirst=1";
using (HttpClient client = new HttpClient())
using (HttpResponseMessage response = await client.GetAsync(page))
using (HttpContent content = response.Content)
{
// Reading the string.
string result = await content.ReadAsStringAsync();
Console.WriteLine(result);
// Getting a reference to the text component.
questionDisplayText = GetComponent<Text>();
questionDisplayText.text = result.ToString();
questionDisplayText.text = result.Trim(new char[] {'"'});
}

Firebase queries take a two-step approach:
You order the child nodes on their key, their value, or the value of a property.
You then filter on values of the thing you ordered on.
Since you order by key, the filtering operations like StartAt() compare the key to the value you passed. And since there is no key DisplayText, there are no results.
If you want to read the first question, you shouldn't use a startAt().
FirebaseClient("https://PROJECT_URL.firebaseio.com/");
var dinos = await firebase
.Child("Questions")
.OrderByKey()
.LimitToFirst(1)
If you want to return the results ordered by the value of their DisplayText property, it'd be something like this:
FirebaseClient("https://PROJECT_URL.firebaseio.com/");
var dinos = await firebase
.Child("Questions")
.OrderByChild("DisplayText")
.LimitToFirst(1)
Since you indicated that you want to use the REST API, here's an example of how to do that:
https://stackoverflow.firebaseio.com/59384124.json?orderBy="DisplayText"&startAt="How"
If you want to embed this string in your code, you have a few options. The main ones:
string page = "https://PROJECT_URL.firebaseio.com/Questions/DisplayText.json?orderBy=\"DisplayText\"&startAt=\"How\"&limitToFirst=1";
Or
string page = #"https://PROJECT_URL.firebaseio.com/Questions/DisplayText.json?orderBy=""DisplayText""&startAt=""How""&limitToFirst=1";
And don't forget: in order to be able to filter on the server, you'll need to define an index in your security rules. In my case I did so with:
"59384124": { ".indexOn": "DisplayText" },
Also see:
The documentation for the Firebase REST API
This blog post on embedding quotes in C# strings

How to call google.apis.dialogflow.v2 in C#

I am new to Google APIs. I want to know how to call Google Dialogflow API in C# to get intent form the input text. But I can't find any example to call Dialogflow using C#.
Please provide some example to call Dialogflow from C#.

If I understand your question correctly you want to call the DialogFlow API from within a C# application (rather than writing fulfillment endpoint(s) that are called from DialogFlow. If that's the case here's a sample for making that call:
using Google.Cloud.Dialogflow.V2;
...
...
var query = new QueryInput
{
Text = new TextInput
{
Text = "Something you want to ask a DF agent",
LanguageCode = "en-us"
}
};
var sessionId = "SomeUniqueId";
var agent = "MyAgentName";
var creds = GoogleCredential.FromJson("{ json google credentials file)");
var channel = new Grpc.Core.Channel(SessionsClient.DefaultEndpoint.Host,
creds.ToChannelCredentials());
var client = SessionsClient.Create(channel);
var dialogFlow = client.DetectIntent(
new SessionName(agent, sessionId),
query
);
channel.ShutdownAsync();
In an earlier version of the DialogFlowAPI I was running into file locking issues when trying to re-deploy a web api project which the channel.ShutDownAsync() seemed to solve. I think this has been fixed in a recent release.
This is the simplest version of a DF request I've used. There is a more complicated version that passes in an input context in this post:
Making DialogFlow v2 DetectIntent Calls w/ C# (including input context)

(Nitpicking: I assume you know DialogFlow will call your code as specified/registered in the action at DialogFlow? So your code can only respond to DialogFlow, and not call it.)
Short answer/redirect:
Don't use Google.Apis.Dialogflow.v2 (with GoogleCloudDialogflowV2WebhookRequest and GoogleCloudDialogflowV2WebhookResponse) but use Google.Cloud.Dialogflow.v2 (with WebhookRequest and WebhookResponse) - see this eTag-error. I will also mention some other alternatives underneath.
Google.Cloud.Dialogflow.v2
Using Google.Cloud.Dialogflow.v2 NuGet (Edit: FWIW: this code was written for the beta-preview):
[HttpPost]
public dynamic PostWithCloudResponse([FromBody] WebhookRequest dialogflowRequest)
{
var intentName = dialogflowRequest.QueryResult.Intent.DisplayName;
var actualQuestion = dialogflowRequest.QueryResult.QueryText;
var testAnswer = $"Dialogflow Request for intent '{intentName}' and question '{actualQuestion}'";
var dialogflowResponse = new WebhookResponse
{
FulfillmentText = testAnswer,
FulfillmentMessages =
{ new Intent.Types.Message
{ SimpleResponses = new Intent.Types.Message.Types.SimpleResponses
{ SimpleResponses_ =
{ new Intent.Types.Message.Types.SimpleResponse
{
DisplayText = testAnswer,
TextToSpeech = testAnswer,
//Ssml = $"<speak>{testAnswer}</speak>"
}
}
}
}
}
};
var jsonResponse = dialogflowResponse.ToString();
return new ContentResult { Content = jsonResponse, ContentType = "application/json" }; ;
}
Edit: It turns out that the model binding may not bind all properties from the 'ProtoBuf-json' correctly (e.g. WebhookRequest.outputContexts[N].parameters),
so one should probably use the Google.Protobuf.JsonParser (e.g. see this documentation).
This parser may trip over unknown fields, so one probably also wants to ignore that. So now I use this code (I may one day make the generic method more generic and thus useful, by making HttpContext.Request.InputStream a parameter):
public ActionResult PostWithCloudResponse()
{
var dialogflowRequest = ParseProtobufRequest<WebhookRequest>();
...
var jsonResponse = dialogflowResponse.ToString();
return new ContentResult { Content = jsonResponse, ContentType = "application/json" }; ;
}
private T ParseProtobufRequest<T>() where T : Google.Protobuf.IMessage, new()
{
// parse ProtoBuf (not 'normal' json) with unknown fields, else it may not bind ProtoBuf correctly
// https://github.com/googleapis/google-cloud-dotnet/issues/2425 "ask the Protobuf code to parse the result"
string requestBody;
using (var reader = new StreamReader(HttpContext.Request.InputStream))
{
requestBody = reader.ReadToEnd();
}
var parser = new Google.Protobuf.JsonParser(JsonParser.Settings.Default.WithIgnoreUnknownFields(true));
var typedRequest = parser.Parse<T>(requestBody);
return typedRequest;
}
BTW: This 'ProtoBuf-json' is also the reason to use WebhookResponse.ToString() which in turn uses Google.Protobuf.JsonFormatter.ToDiagnosticString.
Microsoft's BotBuilder
Microsoft's BotBuilder packages and Visual Studio template.
I havent't used it yet, but expect approximately the same code?
Hand written proprietary code
A simple example of incoming request code (called an NLU-Response by Google) is provided by Madoka Chiyoda (Chomado) at Github. The incoming call is simply parsed to her DialogFlowResponseModel:
public static async Task<HttpResponseMessage> Run([...]HttpRequestMessage req, [...]CloudBlockBlob mp3Out, TraceWriter log)
...
var data = await req.Content.ReadAsAsync<Models.DialogFlowResponseModel>();
Gactions
If you plan to work without DialogFlow later on, please note that the interface for Gactions differs significantly from the interface with DialogFlow.
The json-parameters and return-values have some overlap, but nothing gaining you any programming time (probably loosing some time by starting 'over').
However, starting with DialogFlow may gain you some quick dialog-experience (e.g. question & answer design/prototyping).
And the DialogFlow-API does have a NuGet package, where the Gactions-interface does not have a NuGet-package just yet.

Google AdWords library doesn't create an access token

I'm trying to make a targetingIdeaService API call to Google AdWords. This is my code so far:
[HttpGet]
public IEnumerable<string> Get()
{
var user = new AdWordsUser();
using (TargetingIdeaService targetingIdeaService = (TargetingIdeaService)user.GetService(AdWordsService.v201802.TargetingIdeaService))
{
// Create selector.
TargetingIdeaSelector selector = new TargetingIdeaSelector();
selector.requestType = RequestType.IDEAS;
selector.ideaType = IdeaType.KEYWORD;
selector.requestedAttributeTypes = new AttributeType[] {
AttributeType.KEYWORD_TEXT,
AttributeType.SEARCH_VOLUME,
AttributeType.AVERAGE_CPC,
AttributeType.COMPETITION,
AttributeType.CATEGORY_PRODUCTS_AND_SERVICES
};
// Set selector paging (required for targeting idea service).
var paging = Paging.Default;
// Create related to query search parameter.
var relatedToQuerySearchParameter =
new RelatedToQuerySearchParameter
{ queries = new String[] { "bakery", "pastries", "birthday cake" } };
var searchParameters = new List<SearchParameter> { relatedToQuerySearchParameter };
var page = new TargetingIdeaPage();
page = targetingIdeaService.get(selector);
return new string[] { "value1", "value2" };
}
}
it looks ok, compiles at last, and so on. But then I went into debug mode. And I saw this:
So as you can see, the variable doesn't have the access token. The other data comes from app.config file.
I am quite certain the keys passed in are correct.
Then the code throws the famous invalid_grand error. In my case, I believe that's because the access token is not being generated. I'm new to AdWords and ASP.NET, so I probably missed something, but I have no idea what.
I used the
docs,
Code Structure instructions, and
code examples to put it all together.

I had my configuration wrong. I had to recreate all the credentials using incognito window. If you have any issues just open a thread here: https://groups.google.com/forum/#!forum/adwords-api