We Keep Coding

ML.net cant find input column, out of range exception when training algorithm

I'm new with the ML.NET and have a problem which i cannot resolve for few days.
I can train my model, but when i try to CreateEnginePrediciton, i have an error: "Feature column 'Feature' not found"
Here is my code:
var context = new MLContext(seed: 0);
// Create a DataView containing the image paths and labels
var input = LoadLabeledImagesFromPath(_imagePath);
var data = context.Data.LoadFromEnumerable(input);
data = context.Data.ShuffleRows(data);
// Load the images and convert the labels to keys to serve as categorical values
var images = context.Transforms.Conversion.MapValueToKey(inputColumnName: nameof(Input.Label), outputColumnName: _keyColumnName)
.Append(context.Transforms.LoadRawImageBytes(inputColumnName: nameof(Input.ImagePath), outputColumnName: nameof(Input.Image), imageFolder: _imagePath));
var dataPrepModel = images.Fit(data);
var dataPrepDataView = dataPrepModel.Transform(data);
// Split the dataset for training and testing
var trainTestData = context.Data.TrainTestSplit(dataPrepDataView, testFraction: 0.2, seed: 1);
var trainData = trainTestData.TrainSet;
var testData = trainTestData.TestSet;
// Create an image-classification pipeline and train the model
var options = new ImageClassificationTrainer.Options()
{
FeatureColumnName = nameof(Input.Image),
LabelColumnName = _keyColumnName,
ValidationSet = testData,
Arch = ImageClassificationTrainer.Architecture.ResnetV2101, // Pretrained DNN
MetricsCallback = (metrics) => Console.WriteLine(metrics),
TestOnTrainSet = false
};
var pipeline = context.MulticlassClassification.Trainers.ImageClassification(options)
.Append(context.Transforms.Conversion.MapKeyToValue(_predictedLabelColumnName));
Console.WriteLine("Training the model...");
var model = pipeline.Fit(trainData);
// Evaluate the model and show the results
var predictions = model.Transform(trainData);
var metrics = context.MulticlassClassification.Evaluate(predictions, labelColumnName: _keyColumnName, predictedLabelColumnName: _predictedLabelColumnName);
Console.WriteLine();
Console.WriteLine($"Macro accuracy = {metrics.MacroAccuracy:P2}");
Console.WriteLine($"Micro accuracy = {metrics.MicroAccuracy:P2}");
Console.WriteLine(metrics.ConfusionMatrix.GetFormattedConfusionTable());
Console.WriteLine();
// Save the model
Console.WriteLine();
Console.WriteLine("Saving the model...");
context.Model.Save(model, trainData.Schema, _savePath);
}
private static List<Input> LoadLabeledImagesFromPath(string path)
{
var images = new List<Input>();
var directories = Directory.EnumerateDirectories(path);
foreach (var directory in directories)
{
var files = Directory.EnumerateFiles(directory);
images.AddRange(files.Select(x => new Input
{
ImagePath = Path.GetFullPath(x),
Label = Path.GetFileName(directory)
}));
}
return images;
And my classess:
public class Input
{
public byte[] Image;
public string ImagePath;
public string Label;
}
public class Output
{
public float[] Score;
public string PredictedLabel;
}
And code where i try to create prediciton engine:
public static PredictionEngine<ModelInput, ModelOutput> CreatePredictionEngine()
{
// Create new MLContext
MLContext mlContext = new MLContext();
// Load model & create prediction engine
ITransformer mlModel = mlContext.Model.Load(MLNetModelPath, out var modelInputSchema);
var predEngine = mlContext.Model.CreatePredictionEngine<ModelInput, ModelOutput>(mlModel);
return predEngine;
}
Could you help me? Code is mostly from ML.NET tutorials and i can train it but cannot use it :-(

Question about a way to interact with dictionary

Introduction
I built a web crawler for a project. One of the variables parse from the website is the date. I thought that building a dictionary containing the month in French and the int associated with it.
The way that I saw the code going is.
Scrape the data from the website
var moisLettres
Find the variable in the dictionary
Return the number associated with it
So far, I built this code, I searched for a few hours but the concept of comparing a dictionary is a little bit confusing. I'm open to other solutions since I'm here to help. Feel free to correct me.
Dictionary:
IDictionary<string, int> dictMonth = new Dictionary<string, int>();
dictMonth.Add("Janvier",1);
dictMonth.Add("Février",2);
dictMonth.Add("Mars",3);
//Up to 12 months
Examples so far:
foreach (KeyValuePair<string, int> b in dictMonth) //
{
if (b.Value.Equals(e.MonthInLetters, StringComparison.CurrentCultureIgnoreCase))
return /* The TValue associated */
}
Ways I explored:
Using the date.time method to translate automatically the month in string to month in Int. Since the data scraped is in French, the code doesn't work.
FULL CODE 1/2 (Some variables are in French since it is my main language)
`
{
//Variables de bases
const string accountSid = "Private";
const string authToken = "Private";
List<Date> lstDate = new List<Date>();
string sMessage = "";
//Création du dictionnaire
IDictionary<string, int> dictMois = new Dictionary<string, int>();
dictMois.Add("Janvier",1);
dictMois.Add("Février",2);
dictMois.Add("Mars",3);
dictMois.Add("Avril",4);
dictMois.Add("Mai",5);
dictMois.Add("Juin",6);
dictMois.Add("Juillet",7);
dictMois.Add("Aout",8);
dictMois.Add("Septembre",9);
dictMois.Add("Octobre",10);
dictMois.Add("Novembre",11);
dictMois.Add("Décembre",12);
// Initialisation
Date un = new Date("19 Septembre", "Inconnu", "----------- ");
Date deux = new Date("26 Septembre", "Inconnu", "----------");
Date trois = new Date("3 Octobre", "Inconnu", "-----------");
Date quatre = new Date("10 Octobre", "Inconnu", "-----------");
Date cinq = new Date("17 Octobre", "Inconu", "-----------");
Date six = new Date("24 Octobre", "Inconnu", "-----------");
lstDate.Add(un);
lstDate.Add(deux);
lstDate.Add(trois);
lstDate.Add(quatre);
lstDate.Add(cinq);
lstDate.Add(six);
//Captation des dates
foreach (Date d in lstDate)
{
HtmlWeb web = new HtmlWeb();
HtmlDocument doc = web.Load(d.url);
var xpathDate = "-----------";
var locationDate = doc.DocumentNode.SelectSingleNode(xpathDate);
var dateEvenement = locationDate.InnerText;
d.date = dateEvenement;
}
//Vérification si les données sont exactes.
foreach (Date e in lstDate)
{
The way it works
// The data parse is "jeu., 19 septembre 2019"
//un.date = "jeu., 19 septembre 2019"
// un.annee = "jeu., 19 septembre 2019"
// un.annee = (un.date).Substring(un.date.Length -4)
// un.jourMois = "jeu., 19 septembre 2019"
// un.jourMois = (un.date).Subtring(un.date.Length xxx)
//un.moisMots = "jeu., 19 septembre 2019"
//un.moisMots = (un.date).Subtring(un.date.Length xxx)
//un.moisChiffre = (** Here goes the comparaison to the dictionary **)
// At Last
//e.dateReel = (The combinaison of the int of the year, month et day of the month)
//e.dateReel = (e.annee + e.jourMois + e.moisChiffre);
}
//Checking the status of the event if there place it send me a sms message
foreach (Date d in lstDate)
{
HtmlWeb web = new HtmlWeb();
HtmlDocument doc = web.Load(d.url);
var xpathStatut = "//*[#id='event-page']/main/div[1]/div[2]/div/div[2]/div[2]/div/div[2]/div/div/div";
var locationStatut = doc.DocumentNode.SelectSingleNode(xpathStatut);
var statut = locationStatut.InnerText;
if (statut.IndexOf("Complet") <= 0)
{
if (sMessage == "")
sMessage = "Il reste de la place pour";
sMessage += d.date + " ";
}
}
if (sMessage.Length > 0)
{
TwilioClient.Init(accountSid, authToken);
var message = MessageResource.Create(
body: sMessage,
from: new Twilio.Types.PhoneNumber("-------"),
to: new Twilio.Types.PhoneNumber("------------")
);
Console.WriteLine("Message envoyé" + message.Sid);
}
Console.ReadKey();
FULL CODE 2/2 - Classe Date:
class Date
{
public string date;
public string statut;
public string url;
public string moisMots;
public int moisChiffre;
public string jourMois;
public string annee;
public string dateReel;
public Date(string Date, string Statut, string Url)
{
date = Date;
statut = Statut;
url = Url;
}
}

To get the month from a string, DateTime.Parse can be used :
var culture = new System.Globalization.CultureInfo("fr");
var date = DateTime.Parse("jeu., 19 septembre 2019", culture);
int year = date.Year, month = date.Month, day = date.Day; // 2019, 9, 19
For more reliable code, DateTime.TryParse and DateTime.TryParseExact can be used.

It looks like you're looking for a way to lookup a value in a dictionary from a key. You can use the dictionary[key] syntax:
IDictionary<string, int> dictMonth = new Dictionary<string, int>();
dictMonth.Add("Janvier", 1);
dictMonth.Add("Février", 2);
dictMonth.Add("Mars", 3);
string myMonth = "Février"; //comes from your parsed data, for example.
int monthNumber = dictMonth[myMonth]; //lookup value in dictionary
//monthNumber is now equal to 2
You should check that that the key you're looking for exists in the dictionary by using the ContainsKey method, otherwise dictMonth[myMonth] will throw an exception if myMonth is not in dictMonth.

Getting Project List via Rally C# API

I need a way to retrieve a list of projects using C#.
Tried doing something like this:
DynamicJsonObject sub = restApi.GetSubscription("Projects");
//query the project collection
Request wRequest = new Request(sub["Projects"]);
QueryResult queryResult = restApi.Query(wRequest);
return queryResult.Results.Select(result => new Project()
{
Id = result["ObjectID"],
Name = result["Name"]
}).ToList();
unfortunately with no success.
Can anyone help please?

The code below should print workspaces and projects to which the user whose account is used to authenticate the code has access to.
DynamicJsonObject sub = restApi.GetSubscription("Workspaces");
Request wRequest = new Request(sub["Workspaces"]);
wRequest.Limit = 1000;
QueryResult queryResult = restApi.Query(wRequest);
int allProjects = 0;
foreach (var result in queryResult.Results)
{
var workspaceReference = result["_ref"];
var workspaceName = result["Name"];
Console.WriteLine("Workspace: " + workspaceName);
Request projectsRequest = new Request(result["Projects"]);
projectsRequest.Fetch = new List<string>()
{
"Name"
};
projectsRequest.Limit = 10000; //project requests are made per workspace
QueryResult queryProjectResult = restApi.Query(projectsRequest);
int projectsPerWorkspace = 0;
foreach (var p in queryProjectResult.Results)
{
allProjects++;
projectsPerWorkspace++;
Console.WriteLine(projectsPerWorkspace + " Project: " + p["Name"] + " State: " + p["State"]);
}
}
Console.WriteLine("Returned " + allProjects + " projects in the subscription");

How to add/update milestones in a feature using rally rest API and C#?

I am not able to add or update milestones field for the Features in the Rally. If anyone having the code available using C# to update the same, please share with me. I am searching and doing from last one week with no luck.
When I am trying to add/Update milestones in the Features. I am getting the error as "Could not read: Could not read referenced object null". My code is as follows:-
public DynamicJsonObject UpdateFeaturesbyName(string fea, string bFun)
{
//getting list of Feature.
Request feat = new Request("PortfolioItem/Feature");
feat.Query = new Query("Name", Query.Operator.Equals, fea);
QueryResult TCSResults = restApi.Query(feat);
foreach (var res in TCSResults.Results)
{
var steps = res["Milestones"];
Request tsteps = new Request(steps);
QueryResult tstepsResults = restApi.Query(tsteps);
foreach (var item in tstepsResults.Results)
{
}
if (res.Name == fea)
{
var targetFeature = TCSResults.Results.FirstOrDefault();
DynamicJsonObject toUpdate = new DynamicJsonObject();
//toUpdate["Milestones"] = "";
// CreateResult createResult = restApi.Create(steps._ref, toUpdate);
// String contentRef = steps._ref;
//String contentRef = createResult._ref;
string[] value = null;
string AccCri = string.Empty;
if (!string.IsNullOrWhiteSpace(bFun))
{
value = bFun.Split(new string[] { "\r\n", "\n" }, StringSplitOptions.None);
foreach (string item in value)
{
//if (string.IsNullOrWhiteSpace(AccCri))
// AccCri = item;
//else
// AccCri = AccCri + "<br/>" + item;
if (!string.IsNullOrWhiteSpace(item))
{
//Query for Milestone.
Request ms = new Request("Milestone");
ms.Fetch = new List<string>() { "Name", "ObjectID" };
ms.Query = new Query("Name", Query.Operator.Equals, item);
QueryResult msResults = restApi.Query(ms);
var targetMLResult = msResults.Results.FirstOrDefault();
long MLOID = targetMLResult["ObjectID"];
DynamicJsonObject tarML = restApi.GetByReference("Milestone", MLOID, "Name", "_ref", "DisplayColor");
DynamicJsonObject targetML = new DynamicJsonObject();
targetML["Name"] = tarML["Name"];
//targetML["_ref"] = tarML["_ref"];
targetML["_ref"] = "/milestone/" + Convert.ToString(MLOID);
targetML["DisplayColor"] = tarML["DisplayColor"];
// Grab collection of existing Milestones.
var existingMilestones = targetFeature["Milestones"];
long targetOID = targetFeature["ObjectID"];
// Milestones collection on object is expected to be a System.Collections.ArrayList.
var targetMLArray = existingMilestones;
var tagList2 = targetMLArray["_tagsNameArray"];
tagList2.Add(targetML);//
//targetMLArray.Add(targetML);
targetMLArray["_tagsNameArray"] = tagList2;
toUpdate["Milestones"] = targetMLArray;
OperationResult updateResult = restApi.Update(res._ref, toUpdate);
bool resp = updateResult.Success;
}
}
}
//toUpdate["c_AcceptanceCriteria"] = AccCri;
//OperationResult updateResult = restApi.Update(res._ref, toUpdate);
}
}
var features = TCSResults.Results.Where(p => p.Name == fea).FirstOrDefault();
var featuresref = features._ref;
return features;
}

Now that v3.1.1 of the toolkit has been released you can use the AddToCollection method to do this.
Otherwise, you can still always just update the full collection. The value should be an arraylist of objects with _ref properties.
Check out this example (which adds tasks to defects, but should be very similar to what you're doing): https://github.com/RallyCommunity/rally-dot-net-rest-apps/blob/master/UpdateTaskCollectionOnDefect/addTaskOnDefect.cs

How to get all Iteration Paths with a TFS API 2013

I have been using the TFS API libraries for some time and have been using the follwoing code when interfacing with TFS 2010 to get the Iteration Paths
(code from this page)...
public IList<string> GetIterationPaths(Project project)
{
List<string> iterations = new List<string>();
foreach (Node node in project.IterationRootNodes)
AddChildren(string.Empty, node, iterations);
return iterations;
}
private void AddChildren(string prefix, Node node, List<string> items)
{
items.Add(node.Path);
foreach (Node item in node.ChildNodes)
AddChildren(prefix + node.Name + "/", item, items);
}
When I was looking at getting all the iterations within TFS 2013 things changed. In TFS 2013 the concept of iterations changed slightly and the API assemblies for TFS 2013 do not have IterationRootNodes
I have used the following code to get Team Iterations but that is team based and not the full set of Iterations against the project... (currently getting the first team but can be coded to get others)
var cred = new TfsClientCredentials(new WindowsCredential(), true);
TfsTeamProjectCollection coll = new TfsTeamProjectCollection("{URI to SERVER}", cred);
coll.EnsureAuthenticated();
var teamConfig = coll.GetService<TeamSettingsConfigurationService>();
var css = coll.GetService<ICommonStructureService4>();
var project = css.GetProjectFromName(projectInfo.ProjectName);
IEnumerable<TeamConfiguration> configs = teamConfig.GetTeamConfigurationsForUser(new[] { project.Uri.ToString() });
TeamConfiguration team = configs.FirstOrDefault(x => x.ProjectUri == project.Uri.ToString());
var iterations = BuildIterationTree(team, css);
where BuildIterationTree looks like...
private IList<IterationInfo> BuildIterationTree(TeamConfiguration team, ICommonStructureService4 css)
{
string[] paths = team.TeamSettings.IterationPaths;
var result = new List<IterationInfo>();
foreach (string nodePath in paths.OrderBy(x => x))
{
var projectNameIndex = nodePath.IndexOf("\\", 2);
var fullPath = nodePath.Insert(projectNameIndex, "\\Iteration");
var nodeInfo = css.GetNodeFromPath(fullPath);
var name = nodeInfo.Name;
var startDate = nodeInfo.StartDate;
var endDate = nodeInfo.FinishDate;
result.Add(new IterationInfo
{
IterationPath = fullPath.Replace("\\Iteration", ""),
StartDate = startDate,
FinishDate = endDate,
});
}
return result;
}
My question is... How do I get the full iteration tree rather than Team specific Iterations for TFS 2013?
Team specific iterations can be configured to show or not via the Web Portal via a checkbox against the iteration.

Your question answered my question on how to get the team specific iterations so the least I can do is offer this snippet that gets the full iteration hierarchy. I think I originally found this code on this blog:
public static void GetIterations(TfsTeamProjectCollection collection,
ICommonStructureService4 css, ProjectInfo project)
{
TeamSettingsConfigurationService teamConfigService = collection.GetService<TeamSettingsConfigurationService>();
NodeInfo[] structures = css.ListStructures(project.Uri);
NodeInfo iterations = structures.FirstOrDefault(n => n.StructureType.Equals("ProjectLifecycle"));
XmlElement iterationsTree = css.GetNodesXml(new[] { iterations.Uri }, true);
string baseName = project.Name + #"\";
BuildIterationTree(iterationsTree.ChildNodes[0].ChildNodes, baseName);
}
private static void BuildIterationTree(XmlNodeList items, string baseName)
{
foreach (XmlNode node in items)
{
if (node.Attributes["NodeID"] != null &&
node.Attributes["Name"] != null &&
node.Attributes["StartDate"] != null &&
node.Attributes["FinishDate"] != null)
{
string name = node.Attributes["Name"].Value;
DateTime startDate = DateTime.Parse(node.Attributes["StartDate"].Value, CultureInfo.InvariantCulture);
DateTime finishDate = DateTime.Parse(node.Attributes["FinishDate"].Value, CultureInfo.InvariantCulture);
// Found Iteration with start / end dates
}
else if (node.Attributes["Name"] != null)
{
string name = node.Attributes["Name"].Value;
// Found Iteration without start / end dates
}
if (node.ChildNodes.Count > 0)
{
string name = baseName;
if (node.Attributes["Name"] != null)
name += node.Attributes["Name"].Value + #"\";
BuildIterationTree(node.ChildNodes, name);
}
}
}