We Keep Coding

How to create new AutoML DataSet for simple classification (C#)

As part of ML automation process I want to dynamically create new AutoML model. I'm using C# (.net framework) and Google.Cloud.AutoML.V1.
After trying to run CreateDataSet code:
var autoMlClient = AutoMlClient.Create();
var parent = LocationName.FromProjectLocation(_projectId, _locationId);
var dataset = new Google.Cloud.AutoML.V1.Dataset();
dataset.DisplayName = "NewDataSet";
var response = autoMlClient.CreateDataset(parent, dataset);
I get the following error:
Field: dataset.dataset_metadata; Message: Required field not set
According to this user manual I should set Dataset Metadata Type, but the list contains only specific types of classifications (Translation/ImageClassifications etc.), I can't find a simple classification type.
How do I create a simple classification data set with the API ? in the AutoML UI its just with a simple button click ("NEW DATASET") - and have to provide only name & region - no classification type.
I also tried to set:
dataset.TextClassificationDatasetMetadata =
new TextClassificationDatasetMetadata() { ClassificationType = ClassificationType.Multiclass };
But I was unable to import data to it (got too many errors of invalid inputs from the input CSV file), I guess its related to the reason that the input format is not suitable for Text Classification.
UPDATE
I've just notice that the Nuget works with AutoML v1 but v1 beta does contains TablesDatasetMetadata Dataset Metadata Type for normal classifications. I'm speechless.

I also experienced this scenario today while creating a dataset using the NodeJS client. Since the Google AutoML table service is in the beta level you need to use the beta version of the AutoML client. In the Google cloud documentation they have used the beta client to create a dataset.
In NodeJS importing the beta version require('#google-cloud/automl').v1beta1.AutoMlClient instead of importing the normal version (v1) require('#google-cloud/automl').v1 worked for me to successfully execute the create dataset functionality.
In C# you can achieve the same through a POST request. Hope this helps :)

After #RajithaWarusavitarana comment, and my last question update , below is the code that did the trick. The token is being generated by GoogleClientAPI nuget and AutoML is handled by REST.
string GcpGlobalEndPointUrl = "https://automl.googleapis.com";
string GcpGlobalLocation = "us-central1"; // api "parent" parameter
public string GetToken(string jsonFilePath)
{
var serviceAccountCredentialFileContents = System.IO.File.ReadAllText(jsonFilePath);
var credentialParameters = NewtonsoftJsonSerializer.Instance.Deserialize<JsonCredentialParameters>(serviceAccountCredentialFileContents);
var initializer = new ServiceAccountCredential.Initializer(credentialParameters.ClientEmail)
{
Scopes = new List<string> { "https://www.googleapis.com/auth/cloud-platform" }
};
var cred = new ServiceAccountCredential(initializer.FromPrivateKey(credentialParameters.PrivateKey));
string accessToken = cred.GetAccessTokenForRequestAsync("https://oauth2.googleapis.com/token").Result;
return accessToken;
}
public void GetDataSetList(string projectId, string token)
{
var restClient = new RestClient(GcpGlobalEndPointUrl);
var createDataSetReqUrl = $"v1beta1/projects/{projectId}/locations/{GcpGlobalLocation}/datasets";
var createDataSetReq = new RestRequest(createDataSetReqUrl, Method.GET);
createDataSetReq.AddHeader("Authorization", $"Bearer {token}");
var createDatasetResponse = restClient.Execute(createDataSetReq);
createDatasetResponse.Dump();
}
I took the token generation code from google-api-dotnet-client Test File

Google AdWords library doesn't create an access token

I'm trying to make a targetingIdeaService API call to Google AdWords. This is my code so far:
[HttpGet]
public IEnumerable<string> Get()
{
var user = new AdWordsUser();
using (TargetingIdeaService targetingIdeaService = (TargetingIdeaService)user.GetService(AdWordsService.v201802.TargetingIdeaService))
{
// Create selector.
TargetingIdeaSelector selector = new TargetingIdeaSelector();
selector.requestType = RequestType.IDEAS;
selector.ideaType = IdeaType.KEYWORD;
selector.requestedAttributeTypes = new AttributeType[] {
AttributeType.KEYWORD_TEXT,
AttributeType.SEARCH_VOLUME,
AttributeType.AVERAGE_CPC,
AttributeType.COMPETITION,
AttributeType.CATEGORY_PRODUCTS_AND_SERVICES
};
// Set selector paging (required for targeting idea service).
var paging = Paging.Default;
// Create related to query search parameter.
var relatedToQuerySearchParameter =
new RelatedToQuerySearchParameter
{ queries = new String[] { "bakery", "pastries", "birthday cake" } };
var searchParameters = new List<SearchParameter> { relatedToQuerySearchParameter };
var page = new TargetingIdeaPage();
page = targetingIdeaService.get(selector);
return new string[] { "value1", "value2" };
}
}
it looks ok, compiles at last, and so on. But then I went into debug mode. And I saw this:
So as you can see, the variable doesn't have the access token. The other data comes from app.config file.
I am quite certain the keys passed in are correct.
Then the code throws the famous invalid_grand error. In my case, I believe that's because the access token is not being generated. I'm new to AdWords and ASP.NET, so I probably missed something, but I have no idea what.
I used the
docs,
Code Structure instructions, and
code examples to put it all together.

I had my configuration wrong. I had to recreate all the credentials using incognito window. If you have any issues just open a thread here: https://groups.google.com/forum/#!forum/adwords-api

How to retrieve a list of subnets for launching a Fargate task?

I'm using the AWS SDK with C# in Visual Studio 2017, and have a prototype working which launches a Fargate service in ECS. As part of the setup, you instantiate a CreateServiceRequest object which requires a NetworkConfiguration.AwsVpcConfiguration setting with SecurityGroups and Subnets.
var request = new CreateServiceRequest();
request.ServiceName = "myService";
request.TaskDefinition = "myTask"; // family[:revision] of the task definition to use
request.ClientToken = Guid.NewGuid().ToString().Replace("-", ""); // max 32 characters!
request.Cluster = "default";
request.DesiredCount = 1;
request.LaunchType = LaunchType.FARGATE;
request.DeploymentConfiguration = new DeploymentConfiguration
{
MaximumPercent = 100,
MinimumHealthyPercent = 50
};
// configure the network and security groups for the task
List<string> securityGroups = new List<string>();
securityGroups.Add("sg-123456");
List<string> subnets = new List<string>();
subnets.Add("subnet-9b36aa97");
request.NetworkConfiguration = new NetworkConfiguration
{
AwsvpcConfiguration = new AwsVpcConfiguration
{
AssignPublicIp = AssignPublicIp.ENABLED,
SecurityGroups = securityGroups,
Subnets = subnets
}
};
When I configure a service manually via the AWS Console, they display a list of subnets from which to choose. So, I'm wondering how I might programmatically retrieve that list of subnets which are available in our VPC.
I'm searching their SDK documentation for possible solutions, any pointers to their docs or examples is appreciated!

Take a look at EC2Client, you'll find a lot of VPC-related APIs are associated with the EC2 service. Specifically check out AmazonEC2Client.DescribeSubnets(DescribeSubnetsRequest), method documentation here:
Request
Amazon.EC2.Model.DescribeSubnetsRequest
Response
Amazon.EC2.Model.DescribeSubnetsResponse
Response contains a list of Amazon.EC2.Model.Subnet that you will retrieve string property SubnetId from, when deciding which subnet to pass on to your Fargate request.
Example Usage (From the linked documentation):
var response = client.DescribeSubnets(new DescribeSubnetsRequest
{
Filters = new List<filter> {
new Filter {
Name = "vpc-id",
Values = new List<string> {
"vpc-a01106c2"
}
}
}
});
List<subnet> subnets = response.Subnets;
Further Reading
AWS Documentation - EC2Client - Search this document for 'DescribeSubnets' to find async variants of this SDK method.

Moving files with Google Drive API v3

Im trying to move a file from one folder to another using the Google Drive API v3. I found documentation how to this here. I used the .NET sample code from the documentation page and created a method that looks like this:
public ActionResult MoveFile(string fileToMove, string destination)
{
DriveService service = new DriveService(new BaseClientService.Initializer
{
HttpClientInitializer = <USER CREDENTIAL>,
ApplicationName = "APPNAME"
});
var searchFiles = service.Files.List();
searchFiles.Corpus = FilesResource.ListRequest.CorpusEnum.User;
searchFiles.Q = "name = '" + fileToMove + "'";
searchFiles.Fields = "files(*)";
string fileToMoveId = searchFiles.Execute().Files[0].Id;
searchFiles.Q = "name = '" + destination + "'";
string destinationId = searchFiles.Execute().Files[0].Id;
//Code used from documentation
// Retrieve the existing parents to remove
var getRequest = service.Files.Get(fileToMoveId);
getRequest.Fields = "parents";
var file = getRequest.Execute();
var previousParents = String.Join(",", file.Parents);
// Move the file to the new folder
var updateRequest = service.Files.Update(file, fileToMoveId);
updateRequest.Fields = "id, parents";
updateRequest.AddParents = destinationId;
updateRequest.RemoveParents = previousParents;
file = updateRequest.Execute();
return RedirectToAction("Files", new {folderId = destinationId});
}
When I execute this code I get the following error:
The parents field is not directly writable in update requests. Use the
addParents and removeParents parameters instead.
The error doesn't really makes sense to me because this code sample came from the documentation page itself. I can't figure out what other paramters they mean. What addParents and removeParents parameters do they mean? Are updateRequest.AddParents and updateRequest.RemoveParents not the right parameters?

Ok here is the problem.
var updateRequest = service.Files.Update(file, fileToMoveId);
The method is requiring that you send a body of a file to be updated. This normally makes sense as any changes you want to make you can add to the body.
Now the problem you are having is that you got your file from a file.get. Which is totally normal. This is how you should be doing it. THe problem is there are some fields in that file that you cant update. So by sending the full file the API is rejecting your update. If you check Files: update under Request body you will see which fiends are updateable.
Issue:
Now this is either a problem with the client library or the API I am going to have to track down a few people at Google to see which is the case.
Fix:
I did some testing and sending an empty file object as the body works just fine. The file is moved.
var updateRequest = service.Files.Update(new Google.Apis.Drive.v3.Data.File(), fileToMove.Id);
updateRequest.AddParents = directoryToMove.Id;
updateRequest.RemoveParents = fileToMove.Parents[0];
var movedFile = updateRequest.Execute();

This method works well when working in your own drive, but not in a team drive where a file (folder) can only have 1 parent strictly. I do not have the solution in a team drive

using c# to create node in drupal

i need to make a ("webservice") c# app that can create/update/delete nodes for/from drupal 7 using xmlrpc. everytime i run my app i get errors from the xmlrpc files(library). I tried to find code/documentation for C# using xmlrpc to connect to drupal, but in vain.
I would be nice if you could point me in the right direction, or share some c# code with me.
{
[XmlRpcUrl("http://testing/testserver")]
public interface IDrupalServices
{
[XmlRpcMethod("node.get")]
XmlRpcStruct NodeLoad(int nid, string[] field);
[XmlRpcMethod("node.save")]
void NodeSave(XmlRpcStruct node);
}
public partial class Form1 : Form
{
public Form1()
{
InitializeComponent();
IDrupalServices drupal = XmlRpcProxyGen.Create<IDrupalServices>();
int nid = 227;
string[] fields = new string[] { };
XmlRpcStruct node = drupal.NodeLoad(nid, fields);
string teaser = node["teaser"].ToString();
welcomeTxt.Text = teaser;
}
private void button1_Click(object sender, EventArgs e)
{
string title = txtTitle.Text;
string body = txtBody.Text;
IDrupalServices drupal = XmlRpcProxyGen.Create<IDrupalServices>();
XmlRpcStruct node = new XmlRpcStruct();
node["id"] = 1001;
node["title"] = title;
node["body"] = body;
node["teaser"] = body;
node["format"] = 1;
node["type"] = "webservice";
node["promote"] = false;
drupal.NodeSave(node);
MessageBox.Show("The post was been created!");
}
}
}
After i run this i get the error: Server returned a fault exception: [-32601] Server error. Requested method node.get not specified. - in the XmlRpcSerializer.cs
Thank you,
Florin

If you're using Drupal 7 you must be using Services 3 which doesn't have a node.get method (or node.save as it happens). They've been replaced with node.retrieve and node.create & node.update respectively.
You can view all of the available methods in the resources/node_resource.inc file in the Services module folder.
UPDATE
Internally the node is submitted using drupal_execute which is the function used to submit a form. Since the body is a field in Drupal it's expected to be a multi-dimensional array in this format (PHP version):
$data["body"][$language][0]["value"]
The $language will either be the specific language for the node, or und for undefined language (unless you're dealing with a multi-lingual site und is usually the way to go). You'd need to build an array similar to that in your C# code and Drupal should save it.
ANOTHER UPDATE
The Java XML-RPC client example for Services uses a HashMap type to do this so my best guess is you could use a Dictionary (albeit one that seems unnecessarily complicated):
var innerValue = new Dictionary<string, string>();
innerValue.Add("value", txtBody.Text);
var language = new Dictionary<int, Dictionary<string, string>>();
language.Add(0, innerValue);
var body = new Dictionary<string, Dictionary<int, Dictionary<string, string>>>();
body.Add("und", language);
node["body"] = body;
It's been a few years since I've coded in C# so forgive any errors in there. Also I'm pretty sure it could be declared more efficiently but I've forgotten most of the language to be honest!

Jan's answer is not quite right. If you are using the cook xmlrpc library all you would need to do is this:
XmlRpcStruct postStruct = new XmlRpcStruct();
postStruct.Add("type", "article");
postStruct.Add("title", "wohoo another test");
XmlRpcStruct postBodyStructParams = new XmlRpcStruct();
postBodyStructParams.Add("value", "My body yaaay");
postBodyStructParams.Add("format", "filtered_html");
XmlRpcStruct[] postBodyStructParamsArr = new XmlRpcStruct[1];
postBodyStructParamsArr[0] = postBodyStructParams;
XmlRpcStruct postBodyStruct = new XmlRpcStruct();
postBodyStruct.Add("und", postBodyStructParamsArr);
postStruct.Add("body", postBodyStruct);
Unfortunately, the body params need to be an array of struct under the "und" key with only one value. I blame this on the sloppiness of the drupal xmlrpc API.