onnx model in C# - c#

I am trying to run a machine learning model which I built in Python in C# using ml.net. I converted the model to .onnx (ONNX v7) and am following this tutorial almost exactly:
https://learn.microsoft.com/en-us/azure/machine-learning/how-to-use-automl-onnx-model-dotnet
I am getting an CLR/System.ArgumentOutOfRangeException when I try to run a prediction:
var testInput = new OnnxInput { Input4 = new[] { 19f, 1245f, 106f, 87f } };
var prediction = onnxPredictionEngine.Predict(testInput);
When I inspect the onnxPredictionEngine OutputSchema I see below:
[0] [Column] : {input_1: Vector<single>}
[1] [Column] : {input_1: Vector<single>, 1, 30}
The inputs in my model (which I can see using Netron) are below:
name: input_1
type: float32[unk__6,4]
The outputs in my model are below:
name: dense_9
type: float32[unk__7,30]
I expect the input array size of 4 (as shown in the onnx model), but the OutputSchema[0] has size of 0. Is this where the issue is? If so, is there some way to tell the OutputSchema[0] to expect array of size 4?

I needed to add in the VectorType into the class definition.
Below:
class OnnxInput
{
[VectorType(4)]
[ColumnName("input_1")]
public float[]? Input4 { get; set; }
}
class OnnxOutput
{
[VectorType(30)]
[ColumnName("dense_9")]
public float[]? Output30 { get; set; }
}

Related

C# getting MongoDB BsonDocument to return as JSON

I asked a question a couple of days ago to collect data from MongoDB as a tree.
MongoDB create an array within an array
I am a newbie to MongoDB, but have used JSON quite substantially. I thought using a MongoDB to store my JSON would be a great benefit, but I am just experiencing immense frustration.
I am using .NET 4.5.2
I have tried a number of ways to return the output from my aggregate query to my page.
public JsonResult GetFolders()
{
IMongoCollection<BsonDocument> collection = database.GetCollection<BsonDocument>("DataStore");
PipelineDefinition<BsonDocument, BsonDocument> treeDocs = new BsonDocument[]
{
// my query, which is a series of new BsonDocument
}
var documentGroup = collection.Aggregate(treeDocs).ToList();
// Here, I have tried to add it to a JsonResult Data,
// as both documentGroup alone and documentGroup.ToJson()
// Also, loop through and add it to a List and return as a JsonResult
// Also, attempted to serialise, and even change the JsonWriterSettings.
}
When I look in the Immediate Window at documentGroup, it looks exactly like Json, but when I send to browser, it is an escaped string, with \" surrounding all my keys and values.
I have attempted to create a model...
public class FolderTree
{
public string id { get; set; }
public string text { get; set; }
public List<FolderTree> children { get; set; }
}
then loop through the documentGroup
foreach(var docItem in documentGroup)
{
myDocs.Add(BsonSerializer.Deserialize<FolderTree>(docItem));
}
but Bson complains that it cannot convert int to string. (I have to have text and id as a string, as some of the items are strings)
How do I get my MongoDB data output as Json, and delivered to my browser as Json?
Thanks for your assistance.
========= EDIT ===========
I have attempted to follow this answer as suggested by Yong Shun below, https://stackoverflow.com/a/43220477/4541217 but this failed.
I had issues, that the "id" was not all the way through the tree, so I changed the folder tree to be...
public class FolderTree
{
//[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
//public string id { get; set; }
[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
public string text { get; set; }
public List<FolderTreeChildren> children { get; set; }
}
public class FolderTreeChildren
{
[BsonSerializer(typeof(FolderTreeObjectTypeSerializer))]
public string text { get; set; }
public List<FolderTreeChildren> children { get; set; }
}
Now, when I look at documentGroup, I see...
[0]: {Plugins.Models.FolderTree}
[1]: {Plugins.Models.FolderTree}
To be fair to sbc in the comments, I have made so many changes to get this to work, that I can't remember the code I had that generated it.
Because I could not send direct, my json result was handled as...
JsonResult json = new JsonResult();
json.Data = documentGroup;
//json.Data = JsonConvert.SerializeObject(documentGroup);
json.JsonRequestBehavior = JsonRequestBehavior.AllowGet;
return json;
Note, that I also tried to send it as...
json.Data = documentGroup.ToJson();
json.Data = documentGroup.ToList();
json.Data = documentGroup.ToString();
all with varying failures.
If I leave as documentGroup, I get {Current: null, WasFirstBatchEmpty: false, PostBatchResumeToken: null}
If I do .ToJson(), I get "{ \"_t\" : \"AsyncCursor`1\" }"
If I do .ToList(), I get what looks like Json in json.Data, but get an error of Unable to cast object of type 'MongoDB.Bson.BsonInt32' to type 'MongoDB.Bson.BsonBoolean'.
If I do .ToString(), I get "MongoDB.Driver.Core.Operations.AsyncCursor`1[MongoDB.Bson.BsonDocument]"
=========== EDIT 2 =================
As this way of extracting the data from MongoDB doesn't want to work, how else can I make it work?
I am using C# MVC4. (.NET 4.5.2)
I need to deliver json to the browser, hence why I am using a JsonResult return type.
I need to use an aggregate to collect from MongoDB in the format I need it.
My Newtonsoft.Json version is 11.0.2
My MongoDB.Driver is version 2.11.1
My method is the simplest it can be.
What am I missing?

Mongo .Net Driver PipelineStageDefinitionBuilder.Project automatically ignores all Id values with a facet

When using the PipelineStageDefinitionBuilder when creating projection stages for an aggregation pipeline it appears to be always ignoring any Id values in the dataset. I'm using the Mongo .Net driver 2.8 in a .Net Core app. Below are the steps for reproduction.
The same projection worked when using the IAggregateFluent syntax on Aggregate() however I needed to use the builders for a facet. When running the builder against Aggregate it also works, however within a facet it fails to bind any Id values.
Just empty classes with id for testing (Added Type to show normal mapping works):
public class DatabaseModel
{
public Guid Id { get; set; }
public string Type { get; set; }
}
public class ProjectionClass
{
public Guid Id { get; set; }
public string Type { get; set; }
}
When I create the projection with the below, it produces a query sucessfully, however within all models returned the Id value is set to null. The query seems to have a Id_ : 0 value but the same also seems to be produced in normal aggregation so I don't think this is related?
var typeFilter = Builders<DatabaseModel>.Filter.Eq(x => x.Type, "Full");
var aggregationPipeline = new EmptyPipelineDefinition<DatabaseModel>()
.AppendStage(PipelineStageDefinitionBuilder.Match(typeFilter))
.AppendStage(PipelineStageDefinitionBuilder.Project<DatabaseModel, ProjectionClass>(x => new ProjectionClass
{
Id = x.Id,
Type = x.Type,
}));
var normalAggregationResult = await db.Aggregate(aggregationPipeline).ToListAsync();//The id's appear here
var databaseModelsFacet = AggregateFacet.Create("DatabaseModels", aggregationPipeline);
var faucetResult = db.Aggregate().Facet(databaseModelsFacet).SingleOrDefault().Facets;
var projectionModels = faucetResult.
Single(x => x.Name == "DatabaseModels")
.Output<ProjectionClass>();// This results in missing Id's (Including in nested objects with anything named Id)
Resulting mongo query
{[{
"$match" : { "Type" : "Full" } },
{ "$project" : { "Id" : "$_id", "Type" : "$Type", "_id" : 0 }
}]}
Is there any way to be able to run a projection using the pipeline builders with a facet while not ignoring the Id? I have seen examples using similar queries but haven't seen this as an issue. It could be an issue with facet as it only appears to happen when using this.
Thanks!
UPDATE 6/1/2020: Updated question after finding it only seems to occur with facet
It seems to be an Driver issue, (or other issue is when the structure does not match the fields), as Id cant simply be serialized to Id , but if you choose any other value it will work for example
[BsonNoId]
public class DatabaseModel
{
[BsonRepresentation(BsonType.ObjectId)]
public string Identifier { get; set; }
public string Type { get; set; }
}

Asp.net WebApi add one item in nested child list when list is set to null in json

I have the following json Object which i am sending to Web Api controller Using ajax.
var dataObject = {
LanguageId: 1,
SubTenantId: 1,
Object:{"TestApplication":{"Aplication":null}}
}
Controller function RequestObject generic class.
public IHttpActionResult ComplexObject(RequestObject<TestApplicationMain> pubRequestObject)
I am using following Class hierarchy.
public class TestApplicationMain
{
public TestApplicationMain()
{
TestApplication = new TestApplication();
}
public TestApplication TestApplication{get; set;}
}
public class TestApplication
{
public TestApplication()
{
Aplication = new List<ApplicationSearchParam>();
}
public List<ApplicationSearchParam> Aplication { get; set; }
}
public class ApplicationSearchParam
{
public ApplicationSearch ApplicationSearch { get; set; }
public string OrderBy { get; set; }
}
When i send {"TestApplication":{"Aplication":null}} json to the controller. I receive One item Aplication .
Api controller works as expected when i send {"TestApplication":{"Aplication":undefined}} OR {"TestApplication":{"Aplication":[]}}
My Question is Why Asp.net WebApi Controller add one item in nested child list when its set to null ?.
First Problem Spotted:
var dataObject = {
LanguageId: 1,
SubTenantId: 1,
Object:{"TestApplication":{"Aplication":null}}
}
Your JSON is at a level higher than TestApplicationMain
You defined LanguageId and SubTenantId but your receiving class TestApplicationMain does not handle either of the values.
You have not defined Object in TestApplicationMain
The reason your first test scenario {"TestApplication":{"Aplication":null}} works is because TestApplication is a property of TestApplicationmain
When you are automapping JSon Key-Value pairs, the name of the Key needs to match the name of the Property you wish its value to be assigned.
You test JSON should be:
var dataObject = {"TestApplication":{"Aplication":""}}
as this structure would map to:
TestApplicationMain -> TestApplication -> Aplication = ""
My Suggestions:
https://stackoverflow.com/a/31022729/659246
https://www.strathweb.com/2013/06/supporting-only-json-in-asp-net-web-api-the-right-way/

Serializing "string list" to JSON in C#

(I'v restated my question here: Creating class instances based on dynamic item lists)
I'm currently working on a program in Visual Studio 2015 with C#.
I have 5 list strings that contain data that I wish to serialize to a json file.
public List<string> name { get; private set; }
public List<string> userImageURL { get; private set; }
public List<string> nickname { get; private set; }
public List<string> info { get; private set; }
public List<string> available { get; private set; }
An example of the desired json file format is the fallowing:
{
"users" :
[
{
"name" : "name1",
"userImageURL" : "userImageURL1",
"nickname" : "nickname1",
"info" : "info1",
"available" : false,
},
{
"name" : "name2",
"userImageURL" : "userImageURL2",
"nickname" : "nickname2",
"info" : "info2",
"available" : false,
},
{
"name" : "name3",
"userImageURL" : "userImageURL3",
"nickname" : "nickname3",
"info" : "info3",
"available" : false,
},
{
"name" : "name4",
"userImageURL" : "userImageURL4",
"nickname" : "nickname4",
"info" : "info4",
"available" : false,
}
]
}
Note that there might be errors in the json example above.
I've tried combining the 5 lists to create 1 list to serialize it using the following code:
users = new List<string>(name.Count + userImageURL.Count + nickname.Count + info.Count + available.Count);
allPlayers.AddRange(name);
allPlayers.AddRange(userImageURL);
allPlayers.AddRange(nickname);
allPlayers.AddRange(info);
allPlayers.AddRange(available);
Then I serialize the list with the fallowing code:
string data = JsonConvert.SerializeObject(users);
File.WriteAllText("data.json", data);
This just creates an array of unorganized objects. I wish to know how can I organize them as expressed in the format above.
PS: I'm pretty new to coding as you can tell. Sorry if I'm not expressing the question correctly or using the right terminology. Also, this is not the original code. The code creates this lists which I wish to serialize into a json file.
PSS: This data is collected using HtmlAgilityPack. I asked a question yesterday asking how could I parse an html file and serialize it's data to a json file. Using HtmlAgilityPack to get specific data in C# and serialize it to json . As nobody answered, I decided to try and do it myself. The method that I used may not be the best, but it is what I could do with the knowledge that I have.
I would suggest refactoring your code to start with - instead of having 5 "parallel collections", have a single collection of a new type, User:
public class User
{
public string Name { get; set; }
public string ImageUrl { get; set; }
public string NickName { get; set; }
public string Info { get; set; }
public bool Available { get; set; }
}
...
// In your containing type
public List<User> Users { get; set; }
This is likely to make life simpler not just for your JSON, but for the rest of the code too - because you no longer have the possibility of having more nicknames than image URLs, etc. In general, having multiple collections that must be kept in sync with each other is an antipattern. There are times where it's appropriate - typically providing different efficient ways of retrieving the same data - but for something like this it's best avoided.
It is actually what Jon says, but to move from your parallel lists to Jon's single list you need something like (assuming all lists have the same number of elements in the same order):
Users = new List<User>();
for (var i = 0; i < name.Count; i++)
{
Users.Add(new User
{
Available = available[i],
ImageUrl = userImageURL[i],
Info = info[i],
Name = name[i],
NickName = nickname[i]
});
}
And then serialise the Users list.

struct table inside struct - ASP.NET - allegro webAPI

I stopped in one point. I'm writing a small ASP.NET application with WebAPI from polish site allegro.pl (it's similiar to ebay.com)
This API have a method which returns me some data:
(...)
3. searchArray | SearchResponseType[]
Struct table, where are the information about offers that matches the question
(...)
Inside this struct there 28 types of data which are returned, for ex:
1. sItId | long
Offer id
2. sItName | string
Title of offer
etc..
(...)
And on the end there is a struct table with 2 subfields
28. sItAttribsList | AttribStruct[]
Struct table which contains informations about parameters assigned to offer
1. attribName | string
Name of parameter
2. attribValues | string[]
Table of parameter values
Okay - in my code behind I want to retrieve those information and display them on my page.
I have a model which contains fields like the fields which are returned from WebAPI
namespace allegrotest.Models
{
public class SearchArrayModel
{
public long ID { get; set; }
public string Name { get; set; }
(...)
public struct AttribStruct
{
public string AttribName { get; set; }
public string[] AttribValues { get; set; }
}
public AttribStruct[] AttribStructTable { get; set; }
}
}
And in my controller:
foreach (SearchResponseType item in sercharray)
{
SearchArrayModel searchArrMdl = new SearchArradyModel
{
ID = item.sitid,
Name = item.sitname,
(...)
AttribStructTable = new SearchArrayModel.AttribStruct[]
{
//what now? Because I can't enter to the fields from my Model
}
}
The problem is in the line
AttribStructTable = new SearchArrayModel.AttribStruct[]
I don't know is it declared right or maybe I'm making something bad.
How to solve this?
Or maybe I have something wrong in my Model? Maybe AttribStructTable won't be a table? (If AttribStructTable isn't a table I can get to the fields, otherwise I can't)
Maybe - http://allegro.pl/webapi/documentation.php/show/id,116#method-output
It's the link with the information, it's in Polish, so you have to click "Dane zwracane" - it's "Returned data"
It is standard array initialization. The only difference is that you have an array of structs.
Just use:
AttribStructTable = new []
{
new AttribStruct
{
AttribName = "YOUR_NAME",
AttribValues = new [] { "Value1", "Value2" }
},
// There can be n array items
}
Reference to MSDN arrays and MSDN structs

Categories