We Keep Coding

How to create new AutoML DataSet for simple classification (C#)

As part of ML automation process I want to dynamically create new AutoML model. I'm using C# (.net framework) and Google.Cloud.AutoML.V1.
After trying to run CreateDataSet code:
var autoMlClient = AutoMlClient.Create();
var parent = LocationName.FromProjectLocation(_projectId, _locationId);
var dataset = new Google.Cloud.AutoML.V1.Dataset();
dataset.DisplayName = "NewDataSet";
var response = autoMlClient.CreateDataset(parent, dataset);
I get the following error:
Field: dataset.dataset_metadata; Message: Required field not set
According to this user manual I should set Dataset Metadata Type, but the list contains only specific types of classifications (Translation/ImageClassifications etc.), I can't find a simple classification type.
How do I create a simple classification data set with the API ? in the AutoML UI its just with a simple button click ("NEW DATASET") - and have to provide only name & region - no classification type.
I also tried to set:
dataset.TextClassificationDatasetMetadata =
new TextClassificationDatasetMetadata() { ClassificationType = ClassificationType.Multiclass };
But I was unable to import data to it (got too many errors of invalid inputs from the input CSV file), I guess its related to the reason that the input format is not suitable for Text Classification.
UPDATE
I've just notice that the Nuget works with AutoML v1 but v1 beta does contains TablesDatasetMetadata Dataset Metadata Type for normal classifications. I'm speechless.

I also experienced this scenario today while creating a dataset using the NodeJS client. Since the Google AutoML table service is in the beta level you need to use the beta version of the AutoML client. In the Google cloud documentation they have used the beta client to create a dataset.
In NodeJS importing the beta version require('#google-cloud/automl').v1beta1.AutoMlClient instead of importing the normal version (v1) require('#google-cloud/automl').v1 worked for me to successfully execute the create dataset functionality.
In C# you can achieve the same through a POST request. Hope this helps :)

After #RajithaWarusavitarana comment, and my last question update , below is the code that did the trick. The token is being generated by GoogleClientAPI nuget and AutoML is handled by REST.
string GcpGlobalEndPointUrl = "https://automl.googleapis.com";
string GcpGlobalLocation = "us-central1"; // api "parent" parameter
public string GetToken(string jsonFilePath)
{
var serviceAccountCredentialFileContents = System.IO.File.ReadAllText(jsonFilePath);
var credentialParameters = NewtonsoftJsonSerializer.Instance.Deserialize<JsonCredentialParameters>(serviceAccountCredentialFileContents);
var initializer = new ServiceAccountCredential.Initializer(credentialParameters.ClientEmail)
{
Scopes = new List<string> { "https://www.googleapis.com/auth/cloud-platform" }
};
var cred = new ServiceAccountCredential(initializer.FromPrivateKey(credentialParameters.PrivateKey));
string accessToken = cred.GetAccessTokenForRequestAsync("https://oauth2.googleapis.com/token").Result;
return accessToken;
}
public void GetDataSetList(string projectId, string token)
{
var restClient = new RestClient(GcpGlobalEndPointUrl);
var createDataSetReqUrl = $"v1beta1/projects/{projectId}/locations/{GcpGlobalLocation}/datasets";
var createDataSetReq = new RestRequest(createDataSetReqUrl, Method.GET);
createDataSetReq.AddHeader("Authorization", $"Bearer {token}");
var createDatasetResponse = restClient.Execute(createDataSetReq);
createDatasetResponse.Dump();
}
I took the token generation code from google-api-dotnet-client Test File

C# MongoDB driver transactions on a single MongoDB instance

I'm using MongoDB 4.0.8 with C# driver 2.8.1 and I'm trying to implement Transactions in my project.
I copy-pasted the following code sample:
static async Task<bool> UpdateProducts()
{
//Create client connection to our MongoDB database
var client = new MongoClient(MongoDBConnectionString);
//Create a session object that is used when leveraging transactions
var session = client.StartSession();
//Create the collection object that represents the "products" collection
var products = session.Client.GetDatabase("MongoDBStore").GetCollection<Product>("products");
//Clean up the collection if there is data in there
products.Database.DropCollection("products");
//Create some sample data
var TV = new Product { Description = "Television", SKU = 4001, Price = 2000 };
var Book = new Product { Description = "A funny book", SKU = 43221, Price = 19.99 };
var DogBowl = new Product { Description = "Bowl for Fido", SKU = 123, Price = 40.00 };
//Begin transaction
session.StartTransaction(new TransactionOptions(
readConcern: ReadConcern.Snapshot,
writeConcern: WriteConcern.WMajority));
try
{
//Insert the sample data
await products.InsertOneAsync(session, TV);
await products.InsertOneAsync(session, Book);
await products.InsertOneAsync(session, DogBowl);
var filter = new FilterDefinitionBuilder<Product>().Empty;
var results = await products.Find(filter).ToListAsync();
//Increase all the prices by 10% for all products
var update = new UpdateDefinitionBuilder<Product>().Mul<Double>(r => r.Price, 1.1);
await products.UpdateManyAsync(session, filter, update); //,options);
//Made it here without error? Let's commit the transaction
session.CommitTransaction();
//Let's print the new results to the console
Console.WriteLine("Original Prices:\n");
results = await products.Find<Product>(filter).ToListAsync();
foreach (Product d in results)
{
Console.WriteLine(String.Format("Product Name: {0}\tPrice: {1:0.00}", d.Description, d.Price));
}
}
catch (Exception e)
{
Console.WriteLine("Error writing to MongoDB: " + e.Message);
session.AbortTransaction();
}
return true;
}
But in the first Insert command, I'm getting this error:
Command insert failed:
Transaction numbers are only allowed on a replica set member or mongos.
The Documentation says that:
Starting in version 4.0, MongoDB provides the ability to perform multi-document transactions against replica sets.
I don't have replicas in my project, I have only one database instance which is my primary one. If there a solution or a work-around I can use to implement Transactions? I have methods that update more than one collection and I really think it could save me time to use it.

like the documentation says, transactions only work with replica sets. so you need to run your mongodb server as single node replica set. to achieve that, do the following steps...
step 1:
stop the mongodb server.
step 2:
add the replication setting to your mongod.cfg file. here's my own as an example
storage:
dbPath: C:\DATA
directoryPerDB: true
journal:
enabled: true
systemLog:
destination: file
logAppend: true
path: C:\DATA\log\mongod.log
net:
port: 27017
bindIp: 127.0.0.1
replication:
replSetName: MyRepSet
step 3: open up a mongodb shell and issue the following command to initiate the replica set.
rs.initiate()
step 4: restart mongod
on a side-note, if you'd like to write cleaner, more convenient transaction code like the following, check out my library MongoDB.Entities
using (var TN = new Transaction())
{
var author = new Author { Name = "one" };
TN.Save(author);
TN.Delete<Book>(book.ID);
TN.Commit();
}

Microsoft Graph .NET Client Library get verification record

I use Microsoft Graph .NET Client Library ( https://github.com/microsoftgraph/msgraph-sdk-dotnet ).
I want to call this API method:
https://learn.microsoft.com/en-us/graph/api/domain-list-verificationdnsrecords?view=graph-rest-1.0
GET https://graph.microsoft.com/v1.0/domains/{domain-name}/verificationDnsRecords
I try to do the following:
List<QueryOption> options = new List<QueryOption>
{
new QueryOption("domain-name", domain)
};
var r = await graphClient.Domains.Request(options).Select(p => p.VerificationDnsRecords).GetAsync();
But I get an error like "Unknown domain-name" parameter. I tried id instead of domain-name and get the same.
What is wrong and how to do?

Found solution how to set domain name. It should be:
await graphClient.Domains[domain].VerificationDnsRecords.Request().GetAsync();
without any QueryOption

run a primitive mycouch query

I'm new to couchdb and mycouch. I'm trying to implement a very simple query, I just want to get the results of a view and save it into my DTO class.
My couchdb query works, when I query it manually via HTTP:
http://localhost:5984/mydb/_design/tshirts/_view/getAllTshirts
However, when I try running it from my app using mycouch, I can't get to run it. My current query:
using MyCouch.Requests;
using MyCouch.Responses;
// (...)
using (var client = new Client("http://localhost:5984/samples")) {
var query = new QueryViewRequest("getAllTshirts");
ViewQueryResponse<TShirt[]> result = await client.Views.QueryAsync<TShirt[]>(query);
Console.WriteLine (result);
}
For some reason, it won't find the Client class. I found an example where Client is used on github, as you can see, I'm using all the MyCouch related namespaces as in the example.
I also tried using MyCouchStore instead:
using (var store = new MyCouchStore("http://localhost:5984/", "samples")) {
var query = new QueryViewRequest("getAllTshirts");
ViewQueryResponse<TShirt[]> result = await store.Views.QueryAsync<TShirt[]>(query);
Console.WriteLine (result);
}
However, the store doesn't contain any property named Views.
Any ideas how to query my view using MyCouch?

This is what I do, with the MyCouchStore
using (var store = new MyCouchStore("http://user:password#localhost:5984", "samples")) {
var query = new Query("tshirts", "getAllTshirts");
var rows = store.QueryAsync<TShirt>(query).Result;
}

Apparantely, the documentation was not up to date. The constructor requires now 2 arguments, the second being an optional bootstrapper. This worked for me:
var client = new Client("http://localhost:5984/samples", null)

How to create a new Mongo Database programatically?

I'm trying to create a new database programatically, kind'a one database per client.
and using this:
public void CreateNewClientDatabase(Client client)
{
var connectionString = Util.GetClientDatabaseConnectionString(client.DatabaseName);
var mongoClient = new MongoDB.Driver.MongoClient(connectionString);
var server = mongoClient.GetServer();
var db = server.GetDatabase(client.DatabaseName);
db.CreateCollection("DatabaseCreated");
}
The Error I'm getting on CreateCollection is that I do not have the correct credentials, even though that in the connection string, my credentails are correct.
The Exception reads as:
Invalid credentials for database 'client_database_name'.
and the InnerException as:
{"Command 'authenticate' failed: auth fails (response: { \"errmsg\" : \"auth fails\", \"ok\" : 0.0 })"}
The connectionString ends up being this:
mongodb://admin_user:admin_pwd#linus.mongohq.com:10042/client_database_name
What am I missing?
P.S. Using version 1.7 of MongoDB Driver

Bruno ... to do this on a shared service like MongoHQ, you will need to use their API to create new databases programmatically. Docs for the API are located at: http://support.mongohq.com.