We Keep Coding

How to serialize references breadth-first with JSON.NET - c#

I am using JSON.NET to serialize and deserialize an object. This object is essentially a grid for a puzzle, and contains 3 main properties. Squares, Vertices and Edges.
public class Puzzle
{
public Size Size { get; set; }
public List<Edge> Edges { get; set; }
public Square[][] Squares { get; set; }
public Vertex[][] Vertices { get; set; }
// ...
}
The thing is, all 3 of these types have references to each other. For example, one Edge remembers which two Vertices it is connecting to each other. And one square knows about his neighbor squares, and the edge that separates it from each of them
public abstract class Edge
{
public Vertex OriginVertex { get; set; }
public Vertex DestinationVertex { get; set; }
public Square LeftSquare { get; set; }
public Square RightSquare { get; set; }
public EdgeDirection Direction { get; set; }
// ...
}
I am using JSON.NET to serialize this puzzle type, and I am using preserving all references, in order for this structure to be able to make the round trip.
private JsonSerializer _serializer = new JsonSerializer()
{
Formatting = Formatting.Indented,
PreserveReferencesHandling = PreserveReferencesHandling.All,
ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
Converters = { new ArrayReferencePreservingConverter()},
TypeNameHandling = TypeNameHandling.Auto
};
This leads to json that has objects with $id properties, and in other places objects that are only defined as $ref to avoid duplication.
The problem is: Since Serialization is Depth-first, this means that as soon as one of the Puzzle Properties is getting serialized, through a large loop of circular dependencies, every single Vertex/Square/Edge gets serialized in there, as sub-elements of whichever property is getting serialized first. This leads to insane depth in the json file, and even if there weren't code problems, it would be quite ugly and very difficult to work with.
Then, when this first property is done serializing, the next two properties are very flat, and only references, because all of their objects have been hit once in the deep spaghetti prior.
What I would like, is for all 3 main properties to Serialize themselves Breadth-First. This would allow every object to appear exactly one-depth below the Puzzle itself, close to the root, and then these objects could refer to each other through the $ref and $id values.
This would significantly reduce the depth of my json file, helping both with code and readability. I essentially have a structure that has a depth of 5 or 6 at most, yet with my current serialization, my depth grows with the size of the puzzle instead of being constant based on my code.
It does not help to re-order the main properties in the Puzzle, as all 3 of them refer to each other and the problem remains the same. Whichever one is serialized first ends up serializing everything inside itself.
One way to solve this problem would be to add an attribute to some properties, that says "only serialize as a $ref, not as the full object", but I was not able to find such an attribute. If this is possible, I would love to know how.
public abstract class Edge
{
[JsonSerializeAsReferenceIfPossible]
public Vertex OriginVertex { get; set; }
[JsonSerializeAsReferenceIfPossible]
public Vertex DestinationVertex { get; set; }
[JsonSerializeAsReferenceIfPossible]
public Square LeftSquare { get; set; }
[JsonSerializeAsReferenceIfPossible]
public Square RightSquare { get; set; }
public EdgeDirection Direction { get; set; }
}
If the above is not possible, I am open to any idea or suggestion as to how to achieve this fixed-depth json that would be ideal both for human and machine readability.

Related

I'm trying to deserialize a JSON file into a generic stack of serialized objects so I can easily make a sort of tree view using a for loop.
I've done a little Googling over the past couple hours but can't seem to find anything that would allow a truly generic approach, as all seem to require using pre-declared variables, setting the max depth of the tree.
There are many online generic JSON viewers (though no code is provided) so I am sure this is possible, I just have no idea how.
The file I'm currently working with as an example is below, as is an example of how I'm trying to get this structured.
{"arcanePrefs":[{"name":"playerWalkingSpeed","category":"player","value":"6"},{"name":"playerRunningSpeed","category":"player","value":"8"},{"name":"playerLookSpeed","category":"player","value":"12"},{"name":"playerHorTurnSpeed","category":"player","value":"2"},{"name":"playerVerTurnSpeed","category":"player","value":"2"},{"name":"playerJumpHeight","category":"player","value":"6"},{"name":"playerVertLimit","category":"player","value":"80"},{"name":"playerAirCtrl","category":"player","value":"True"},{"name":"playerGravMod","category":"player","value":"1.5"},{"name":"playerHBobWlkSpd","category":"player","value":"0.18"},{"name":"playerHBobRunSpd","category":"player","value":"0.35"},{"name":"playerHBobHgt","category":"player","value":"0.4"},{"name":"playerHCant","category":"player","value":"True"},{"name":"playerReloadSpeed","category":"player","value":"1"},{"name":"playerUseRadius","category":"player","value":"0.5"},{"name":"playerWasteHealth","category":"player","value":"False"},{"name":"playerWasteArmor","category":"player","value":"False"},{"name":"playerWasteAmmo","category":"player","value":"False"},{"name":"playerHeight","category":"player","value":"1.6"},{"name":"playerBaseHealth","category":"player","value":"100"},{"name":"playerMaxHealth","category":"player","value":"250"},{"name":"playerbaseArmor","category":"player","value":"100"},{"name":"playerMaxArmor","category":"player","value":"150"},{"name":"playerWeaponSoftDrag","category":"player","value":"True"},{"name":"enemyStunLock","category":"enemy","value":"True"},{"name":"enemyGunshotDetectRaduis","category":"enemy","value":"40"},{"name":"worldWaterFallDmg","category":"world","value":"False"},{"name":"worldCanShootSwitches","category":"world","value":"True"},{"name":"worldLavaDamagePerFrame","category":"world","value":"1"},{"name":"cameraFOV","category":"camera","value":"80"},{"name":"cameraToneMapping","category":"camera","value":"True"},{"name":"cameraCustomPixel","category":"camera","value":"True"},{"name":"cameraPixelRes","category":"camera","value":"res360p"},{"name":"cameraDither","category":"camera","value":"True"},{"name":"cameraDitherStrength","category":"camera","value":"0.95"},{"name":"cameraBloom","category":"camera","value":"True"},{"name":"cameraBloomInt","category":"camera","value":"5"},{"name":"cameraUnderwaterCol1","category":"camera","value":"0.1921569;0.3921569;0.4509804;1"},{"name":"cameraUnderwaterCol2","category":"camera","value":"0.3137255;0.5882353;0.5882353;1"},{"name":"cameraUnderlavaCol1","category":"camera","value":"0.9019608;0.2313726;0;1"},{"name":"cameraUnderlavaCol2","category":"camera","value":"0.9019608;0.6666667;0.3529412;1"},{"name":"cameraCshrCol","category":"camera","value":"0.4901961;0.4901961;0.4901961;1"},{"name":"cameraCshrTgtCol","category":"camera","value":"0.7803922;0;0;1"},{"name":"arcaneLimitFPS","category":"engine","value":"True"},{"name":"arcaneFpsLimit","category":"engine","value":"-1"},{"name":"engineUIScale","category":"engine","value":"0.7"},{"name":"srcBHopping","category":"sem","value":"False"}],"arcaneControls":[{"name":"contForward","key":119},{"name":"contLeft","key":97},{"name":"contBackward","key":115},{"name":"contRight","key":100},{"name":"contRotLeft","key":113},{"name":"contRotRight","key":101},{"name":"contJump","key":32},{"name":"contLookUp","key":280},{"name":"contLookDown","key":281},{"name":"contSprintHold","key":304},{"name":"contSprintToggle","key":301},{"name":"contCrouchHold","key":306},{"name":"contCrouchToggle","key":99},{"name":"contComFire","key":323},{"name":"contComFireAlt","key":305},{"name":"contComSFire","key":324},{"name":"contComSFireAlt","key":303},{"name":"contComReload","key":114},{"name":"contComPrevWeap","key":326},{"name":"contComNextWeap","key":327},{"name":"contComInspectWeap","key":103},{"name":"contDebug","key":96}]}
What I'm trying to do

I do not use Unity, but they seem to provide newtonsoft which is a powerfull serializer / deserializer library for json link
For your use case, you want to convert json to a list of objet I think, so deserialize in this case.
To obtain an object representation of your json string you can go like
string json; // you data here
Arcane arcane=Newtonsoft.Json.JsonConvert.DeserializeObject<Arcane>(json);
public class Arcane
{
public List<ArcanePref> arcanePrefs { get; set; }
public List<ArcaneControl> arcaneControls { get; set; }
}
public class ArcanePref
{
public string name { get; set; }
public string category { get; set; }
public string value { get; set; }
}
public class ArcaneControl
{
public string name { get; set; }
public int key{ get; set; }
}
With the arcane object you can then access the acranePrefs and arcaneControls list.
To represent the json data in a tree view fashion with Unity, I found the tree view api
Which seem the most straight foward way to go for a tree view representation
Once you have your objects, it should be more or less simple.
Concerning riffnl comment, I do not have any idea what the end goal is, or what is the source you are using ^^'
Hope it helps !

I am trying to deserialize the following json string using Newtonsoft Json. I am able to get the string successfully, but when I try using JsonConvert.DeserializeObject<ServerList>(response, settings);, the try catch fails.
[
{"endpoint":"127.0.0.1","id":6,"identifiers":["steam:","license:","xbl:","live:","discord:"],"name":"Blurr","ping":160},
{"endpoint":"127.0.0.1","id":7,"identifiers":["steam:","license:","xbl:","live:","discord:"],"name":"Knight","ping":120}
]
I believe my issue is because the players array being unnamed.
I have tried [JsonProperty("")] and [JsonProperty] for the Users var, I have also tried using List and Array instead of IList.
Here is the object. This may be completely wrong, I have tried many ways of doing this.
public class ServerList
{
// I have tried many ways of doing this array/list. This is just the latest way I tried.
[JsonProperty]
public static IList<Player> Users { get; set; }
}
public class Player
{
[JsonProperty("endpoint")]
public static string Endpoint { get; set; }
[JsonProperty("id")]
public static string ServerId { get; set; }
[JsonProperty("identifiers")]
public static IList<string> Identifiers { get; set; }
[JsonProperty("name")]
public static string Name { get; set; }
[JsonProperty("ping")]
public static int Ping { get; set; }
}
I am expecting to get a 'ServerList' object returned with a list of all the players connected to the server.
Ask any questions you need to, I don't often work with json in this format. Thank you in advance!
ERROR: Error reading JObject from JsonReader. Current JsonReader item is not an object: StartArray. Path '', line 1, position 1.

Simplest way: Your json is an array, so deserialize to an array:
JsonConvert.DeserializeObject<Player[]>(response,settings);
As noted in the comments, properties on the Player should not be static.
If you insist on an object structure similar to what you posted, an object with a Usersproperty, even though it's not present in the JSON, that is also possible, by implementing a custom JSON converter. There's an article with an example of that here: https://www.jerriepelser.com/blog/custom-converters-in-json-net-case-study-1/
HOWEVER; I would recommend sticking to the types present in the json, and perhaps later construct the object that makes sense to the model in your program. This way, what you deserialize are true to the json you are getting, but make no sacrifices on the model you'd like:
var players = JsonConvert.DeserializeObject<Player[]>(response,settings);
var serverList = new ServerList {Users = players};

There are a number of great ways to auto-generate C# code from JSON, such as here and here.
However, the resulting code doesn't include property initializers. For example, the following JSON:
{
"Name" : "Blastoise"
}
gets deserialized to this:
public class RootObject
{
public string Name { get; set; }
}
Presumably this is by design, since the values used in the JSON will probably be overridden anyways, so adding initializers might just annoy people who don't want them.
But what if I want that? Short of manually adding every value by hand, is there a way to deserialize JSON to the following?
public class RootObject
{
public string Name { get; set; } = "Blastoise";
}
Obviously in this case a manual edit is easy, but manual editing becomes tedious for larger JSON objects.

is there a way to deserialize JSON to the following?
Using the source code of the converter you mentioned.
A quick change at the line 204
sw.WriteLine(prefix + "public {0} {1} {{ get; set; }} = {2};", field.Type.GetTypeName(), field.MemberName, field.GetExamplesText());
gives me the result similar to what you described
internal class SampleResponse1
{
[JsonProperty("Name")]
public string Name { get; set; } = "Blastoise";
}

I am retrieving and successfully deserializing an xml string from a database table column for the known element names (these do not change) but there are also some nested XML Elements called "Other Attributes" which are not always going to be known. I'm having some trouble deserialising these unknown element names to a generic list so I can display them in html once deserialised.
The XML is as follows:
<Detail>
<DetailAttributes>
<Name>Name_123</Name>
<Type>Type_123</Type>
</DetailAttributes>
<OtherAttributes>
<SummaryKey AttributeName="SummaryKey">SummaryKey_123</SummaryKey>
<Account AttributeName="Account">Account_123</Account>
</OtherAttributes>
</Detail>
I have no problem deserialising the 'Name' and 'Type' elements and I can deserialise the 'SummaryKey' and 'Account' elements but only if I explicitly specify their element names - which is not the desired approach because the 'OtherAttributes' are subject to change.
My classes are as follows:
[XmlRoot("Detail")]
public class objectDetailsList
{
[XmlElement("DetailAttributes"), Type = typeof(DetailAttribute))]
public DetailAttribute[] detailAttributes { get; set; }
[XmlElement("OtherAttributes")]
public List<OtherAttribute> otherAttributes { get; set; }
public objectDetailsList()
{
}
}
[Serializable]
public class Detail Attribute
{
[XmlElement("Type")]
public string Type { get;set; }
[XmlElement("Name")]
public string Name { get;set; }
public DetailAttribute()
{
}
}
[Serializable]
public class OtherAttribute
{
//The following will deserialise ok
//[XmlElement("SummaryKey")]
//public string sumKey { get; set; }
//[XmlElement("Account")]
//public string acc { get; set; }
//What I want to do below is create a list of all 'other attributes' without known names
[XmlArray("OtherAttributes")]
public List<Element> element { get; set; }
}
[XmlRoot("OtherAttributes")]
public class Element
{
[XmlAttribute("AttributeName")]
public string aName { get; set; }
[XmlText]
public string aValue { get; set; }
}
When I try to retrieve the deserialised list of OtherAttribute elements the count is zero so it's not able to access the elements nested within "Other Attributes".
Can anybody help me with this please?

With concrete classes and dynamic data like this, you won't be able to lean on the standard XmlSerializer to serialize / deserialize for you - as it reflects on your classes, and the properties you want to populate simply don't exist. You could provide a class with all possible properties if your set of 'OtherAttributes' is known and finite, and not subject to future change, but that would give you an ugly bloated class (and I think you've already decided this is not the solution).
Practical options therefore:
Do it manually. Use the XmlDocument class, load your data with .Load(), and iterate the nodes using .SelectNodes() using an XPath query (something like "/Detail/OtherAttributes/*"). You will have to write the lot yourself, but this gives you complete control over the serialization / deserialization. You won't have to cover your code in (arguably superfluous!) attributes either.
Use Json.NET (http://james.newtonking.com/json), it allows for far greater control over serialization and deserialization. It's fast, has good docs and is overall pretty nifty really.

I am trying to build a data structure that can store the trial by trail results of a group of different tests I am running. The test all consist of a number of trails but some of the information that I want to save and later use is different for the different tests. For example, the results of TestA might look like:
Trial(int) WasResponseCorrect(bool) WhichButtonWasPressed(string) SimulusLevel(double)
1 false "Up" 3.5
2 true "Left" 6.5
Where TestB might have different types of result fields:
Trial(int) WasResponseCorrect(bool) ColorPresented(string) LetterPresented(char) LetterGuessed(Char)
1 false green G C
2 false blue H F
I was thinking of creating a dictionary with the field names as the keys (ex. WasResponseCorrect) and an array of the field values as the values of dic. I can't figure out how to do that. Maybe there is a better way to store the information but I can't think of how to do it. I am working with .net (VB and C#) but I think I can understand and convert most any code if you know of examples in other languages. Thanks!

Without knowing more about your requirements (how you are going to store the data, for example), it seems like polymorphism is what you're looking for. That is, you have a superclass (called Trial) and subclasses that represent the specific trial types. For example:
public class Trial {
public int Id { get; set; }
public bool WasResponseCorrect { get; set; } // if this is in every type of trial
// anything else that is common to ALL trial types
}
public class TrialA : Trial {
public string WhichButtonWasPressed { get; set; }
public double SimulusLevel { get; set; }
}
public class TrialB : Trial {
public string ColorPresented { get; set; }
public char LetterPresented { get; set; }
public char LetterGuessed { get; set; }
}
That way you can have a list of Trial objects, but the actual runtime type of those objects can be TrialA or TrialB.