We Keep Coding

How to create new AutoML DataSet for simple classification (C#)

As part of ML automation process I want to dynamically create new AutoML model. I'm using C# (.net framework) and Google.Cloud.AutoML.V1.
After trying to run CreateDataSet code:
var autoMlClient = AutoMlClient.Create();
var parent = LocationName.FromProjectLocation(_projectId, _locationId);
var dataset = new Google.Cloud.AutoML.V1.Dataset();
dataset.DisplayName = "NewDataSet";
var response = autoMlClient.CreateDataset(parent, dataset);
I get the following error:
Field: dataset.dataset_metadata; Message: Required field not set
According to this user manual I should set Dataset Metadata Type, but the list contains only specific types of classifications (Translation/ImageClassifications etc.), I can't find a simple classification type.
How do I create a simple classification data set with the API ? in the AutoML UI its just with a simple button click ("NEW DATASET") - and have to provide only name & region - no classification type.
I also tried to set:
dataset.TextClassificationDatasetMetadata =
new TextClassificationDatasetMetadata() { ClassificationType = ClassificationType.Multiclass };
But I was unable to import data to it (got too many errors of invalid inputs from the input CSV file), I guess its related to the reason that the input format is not suitable for Text Classification.
UPDATE
I've just notice that the Nuget works with AutoML v1 but v1 beta does contains TablesDatasetMetadata Dataset Metadata Type for normal classifications. I'm speechless.

I also experienced this scenario today while creating a dataset using the NodeJS client. Since the Google AutoML table service is in the beta level you need to use the beta version of the AutoML client. In the Google cloud documentation they have used the beta client to create a dataset.
In NodeJS importing the beta version require('#google-cloud/automl').v1beta1.AutoMlClient instead of importing the normal version (v1) require('#google-cloud/automl').v1 worked for me to successfully execute the create dataset functionality.
In C# you can achieve the same through a POST request. Hope this helps :)

After #RajithaWarusavitarana comment, and my last question update , below is the code that did the trick. The token is being generated by GoogleClientAPI nuget and AutoML is handled by REST.
string GcpGlobalEndPointUrl = "https://automl.googleapis.com";
string GcpGlobalLocation = "us-central1"; // api "parent" parameter
public string GetToken(string jsonFilePath)
{
var serviceAccountCredentialFileContents = System.IO.File.ReadAllText(jsonFilePath);
var credentialParameters = NewtonsoftJsonSerializer.Instance.Deserialize<JsonCredentialParameters>(serviceAccountCredentialFileContents);
var initializer = new ServiceAccountCredential.Initializer(credentialParameters.ClientEmail)
{
Scopes = new List<string> { "https://www.googleapis.com/auth/cloud-platform" }
};
var cred = new ServiceAccountCredential(initializer.FromPrivateKey(credentialParameters.PrivateKey));
string accessToken = cred.GetAccessTokenForRequestAsync("https://oauth2.googleapis.com/token").Result;
return accessToken;
}
public void GetDataSetList(string projectId, string token)
{
var restClient = new RestClient(GcpGlobalEndPointUrl);
var createDataSetReqUrl = $"v1beta1/projects/{projectId}/locations/{GcpGlobalLocation}/datasets";
var createDataSetReq = new RestRequest(createDataSetReqUrl, Method.GET);
createDataSetReq.AddHeader("Authorization", $"Bearer {token}");
var createDatasetResponse = restClient.Execute(createDataSetReq);
createDatasetResponse.Dump();
}
I took the token generation code from google-api-dotnet-client Test File

C# Unable to Bind EmailMessage

I have a method which converts Email to .eml format.
However, the statement of EmailMessage.Bind suddenly stops working without any error.
Below is my code snippet:
public static string convertEmailToEml(EmailMessage emailMessage, string caseNumber, string EMLSaveFilePath)
{
Console.WriteLine(emailMessage.Subject); //correct value
Console.WriteLine(caseNumber); //correct value
Console.WriteLine(EMLSaveFilePath); //correct value
Console.WriteLine(emailMessage.Id); //correct value
ExchangeService service = new ExchangeService(ExchangeVersion.Exchange2007_SP1);
PropertySet props = new PropertySet(EmailMessageSchema.MimeContent);
var email = EmailMessage.Bind(service, emailMessage.Id, props); //not executed
string emlFullPath = EMLSaveFilePath + caseNumber + ".eml"; //not executed
Console.WriteLine(emlFullPath); //code not reached
Console.ReadKey();
using (FileStream fs = new FileStream(emlFullPath, FileMode.Create, FileAccess.Write))
{
fs.Write(email.MimeContent.Content, 0, email.MimeContent.Content.Length);
}
return emlFullPath;
}
May I know if there are any places that I can look into?
There were warning messages upon solution building as follows, not sure if they are linked:
Found conflicts between different versions of the same dependent
assembly that could not be resolved. These reference conflicts are
listed in the build log when log verbosity is set to detailed.
Exception Thrown during Debug Mode:
'Microsoft.Exchange.WebServices.Data.ServiceLocalException' in
Microsoft.Exchange.WebServices.dll("The Url property of the
ExchangeService object must be set.")
I have verified that service, emailMessage.Id, and props values are not null.

Based on the documentation for `EmailMessage.Bind(...), it states that this method results in a call to Exchange Web Services (EWS).
You create an instance of ExchangeService but you do not provide a URL to it (documentation). This is a requirement. The exception that is thrown is pointing you directly at the missing data.

Error querying the Rally API - DynamicJsonObject does not contain a definition for 'Errors'

I'm trying to query the search API of Rally, here is my c# code:
var searchRequest = new Request()
{
ArtifactName = "search",
Limit = 25,
Project = "/project/" + CurrentProject,
ProjectScopeDown = true,
ProjectScopeUp = true,
PageSize = 25,
Fetch = new List<string>() { "true" }
};
searchRequest.AddParameter("keywords", "foo");
QueryResult queryTaskResult = api.Query(searchRequest);
this works as expected and returns result, however i want to pass a parameter of compact=true, which will return slightly different data (mainly a standard web link to the item).
var searchRequest = new Request()
{
ArtifactName = "search",
Limit = 25,
Project = "/project/" + CurrentProject,
ProjectScopeDown = true,
ProjectScopeUp = true,
PageSize = 25,
Fetch = new List<string>() { "true" }
};
searchRequest.AddParameter("keywords", "foo");
///this is the new item
searchRequest.AddParameter("compact", "true");
QueryResult queryTaskResult = api.Query(searchRequest);
However when i fire this request I get the following error
Rally.RestApi.Json.DynamicJsonObject' does not contain a definition for 'Errors'
However when I try to do this request in the browser it works fine.
Any help as to what I'm doing wrong would be greatly appreciated!
Why do you want to do this?
What I want to do is build up a link to the WEB view of the object, eg:
https://rally1.rallydev.com/#/{CurrentProject}d/detail/{ObjectType}/{ObjectId}
I already know the CurrentProject, I need to know the ObjectType and the ObjectId
I have found, when i pass compact=true, the _ref provides this, '/defect/1234567' but this throws an exception.
If I do not pass compact=true, the _ref returns the API reference 'https://rally1.rallydev.com/slm/webservice/v2.x/defect/1234567'

Unfortunately the compact functionality was added to WSAPI after the .NET toolkit was created and we've never updated it to support it.
I filed a github issue here: https://github.com/RallyTools/RallyRestToolkitFor.NET/issues/37
compact=true was mainly a performance optimization to reduce the size of responses in large result sets.
Other than performance is there a reason you'd like to use it?

AWS Machine Learning RealTimePredictor returns UnknownoperationException in C#

Using Visual Studio, and AWS .NET V 3.0.
I'm trying to perform a real-time Predict operation, and to verify the basic setup works, I first perform a GetMLModel() which works and returns the endpoint (Somewhere in the documentation is was mentioned to use that result as the service endpoint, but it's the same that is listed in the console). Is has status "READY", so far so good.
The exception occurs below on the line below "Prediction P = RTP.Predict(Data)". Data contains a Dictionary with all the prediction values.
Error: Error making request with Error Code UnknownOperationException and Http Status Code BadRequest. No further error information was returned by the service.
public static APIResult GetRealTimePrediction(Dictionary<string, string> Data, string PayloadJSON = null) {
AmazonMachineLearningConfig MLConfig = new AmazonMachineLearningConfig();
MLConfig.RegionEndpoint = Amazon.RegionEndpoint.USEast1;
MLConfig.Validate();
AmazonMachineLearningClient MLClient = new AmazonMachineLearningClient("xxx", "xxx", MLConfig);
GetMLModelResponse MLMOdelResp = MLClient.GetMLModel("xxx"); // <-- WORKS
MLConfig.ServiceURL = MLMOdelResp.EndpointInfo.EndpointUrl;
Console.WriteLine(MLConfig.ServiceURL);
MLConfig.Validate();
Amazon.MachineLearning.Util.RealtimePredictor RTP = new Amazon.MachineLearning.Util.RealtimePredictor(MLClient, "xxx");
Prediction P = RTP.Predict(Data); // <----------------EXCEPTION HERE
}
(Obviously replace xxx with relevant values) :)

It turns out that this line:
MLConfig.ServiceURL = MLMOdelResp.EndpointInfo.EndpointUrl;
cases the MLConfig.RegionEndpoint to be reset. Even though the documentation indicates the RegionEndpoint can be determined from the ServiceURL (I'm pretty sure I read that), the RegionEndpoint needs to be set again before the RTP.Predict(Data) call.
Once I figured that out, I was able to reduce the code to just this, in case anyone else needs help. I guess adding too much information to the Configuration is NOT a good thing, as the AWS. NET library seems to figure all this out on its own.
public static APIResult GetRealTimePrediction(Dictionary<string, string> Data, string PayloadJSON = null) {
AmazonMachineLearningConfig MLConfig = new AmazonMachineLearningConfig();
MLConfig.RegionEndpoint = Amazon.RegionEndpoint.USEast1;
MLConfig.Validate(); // Just in case, not really needed
AmazonMachineLearningClient MLClient = new AmazonMachineLearningClient("xxx", "xxx", MLConfig);
Amazon.MachineLearning.Util.RealtimePredictor RTP = new Amazon.MachineLearning.Util.RealtimePredictor(MLClient, "xxx");
Prediction P = RTP.Predict(Data);
}

How do I programatically change the status of a test in VersionOne?

I am posting this because it might help someone using the VersionOne API SDK Client. I wanted to change the status of a test programmatically, to one of the following categories: Ready, InTesting, Passed, or Failed. I originally tried to change the attribute 'Status.Name' however I would get an error that the attribute is a Read-Only attribute. Another suggestion was to create a new attribute with the same name and that the new attribute would override the previous read-only attribute with the same name. However, it appears that I was looking at it backwards.
internal void TestStatusPassed(string str_TestID)
{
var testId = Oid.FromToken(str_TestID, _context.MetaModel);
var query = new Query(testId);
var testType = _context.MetaModel.GetAssetType("Test");
var sourceAttribute = testType.GetAttributeDefinition("Status.Name");
query.Selection.Add(sourceAttribute);
var result = _context.Services.Retrieve(query);
var test = result.Assets[0];
var oldSource = GetValue(test.GetAttribute(sourceAttribute).Value);
test.SetAttributeValue(sourceAttribute, "Passed");
_context.Services.Save(test);
}
This code will throw an exception "Cannot change a read-only attribute"...

I pulled the XML data for one test from the VersionOne Rest API and noticed a relation named "TestStatus" and then it had a number '9123' assigned to it. So I moved that test manually to 'In Testing' and the "TestStatus" changed to '9121'. Then I moved it to failed and the "TestStatus" changed to '155'. I repeated this with several tests from different testsets and noticed that the numbers for each status were consistent and then changed the code slightly and then I was able to programmatically change the status of each test. I changed "Status.Name" to "Status" and "Passed" to "TestStatus:9123" and now it moves the test into the passed category programmatically.
internal void TestStatusPassed(string str_TestID)
{
var testId = Oid.FromToken(str_TestID, _context.MetaModel);
var query = new Query(testId);
var testType = _context.MetaModel.GetAssetType("Test");
var sourceAttribute = testType.GetAttributeDefinition("Status");
query.Selection.Add(sourceAttribute);
var result = _context.Services.Retrieve(query);
var test = result.Assets[0];
var oldSource = GetValue(test.GetAttribute(sourceAttribute).Value);
test.SetAttributeValue(sourceAttribute, "TestStatus:9123");
_context.Services.Save(test);
}