We Keep Coding

How to serialize references breadth-first with JSON.NET

I am using JSON.NET to serialize and deserialize an object. This object is essentially a grid for a puzzle, and contains 3 main properties. Squares, Vertices and Edges.
public class Puzzle
{
public Size Size { get; set; }
public List<Edge> Edges { get; set; }
public Square[][] Squares { get; set; }
public Vertex[][] Vertices { get; set; }
// ...
}
The thing is, all 3 of these types have references to each other. For example, one Edge remembers which two Vertices it is connecting to each other. And one square knows about his neighbor squares, and the edge that separates it from each of them
public abstract class Edge
{
public Vertex OriginVertex { get; set; }
public Vertex DestinationVertex { get; set; }
public Square LeftSquare { get; set; }
public Square RightSquare { get; set; }
public EdgeDirection Direction { get; set; }
// ...
}
I am using JSON.NET to serialize this puzzle type, and I am using preserving all references, in order for this structure to be able to make the round trip.
private JsonSerializer _serializer = new JsonSerializer()
{
Formatting = Formatting.Indented,
PreserveReferencesHandling = PreserveReferencesHandling.All,
ReferenceLoopHandling = ReferenceLoopHandling.Serialize,
Converters = { new ArrayReferencePreservingConverter()},
TypeNameHandling = TypeNameHandling.Auto
};
This leads to json that has objects with $id properties, and in other places objects that are only defined as $ref to avoid duplication.
The problem is: Since Serialization is Depth-first, this means that as soon as one of the Puzzle Properties is getting serialized, through a large loop of circular dependencies, every single Vertex/Square/Edge gets serialized in there, as sub-elements of whichever property is getting serialized first. This leads to insane depth in the json file, and even if there weren't code problems, it would be quite ugly and very difficult to work with.
Then, when this first property is done serializing, the next two properties are very flat, and only references, because all of their objects have been hit once in the deep spaghetti prior.
What I would like, is for all 3 main properties to Serialize themselves Breadth-First. This would allow every object to appear exactly one-depth below the Puzzle itself, close to the root, and then these objects could refer to each other through the $ref and $id values.
This would significantly reduce the depth of my json file, helping both with code and readability. I essentially have a structure that has a depth of 5 or 6 at most, yet with my current serialization, my depth grows with the size of the puzzle instead of being constant based on my code.
It does not help to re-order the main properties in the Puzzle, as all 3 of them refer to each other and the problem remains the same. Whichever one is serialized first ends up serializing everything inside itself.
One way to solve this problem would be to add an attribute to some properties, that says "only serialize as a $ref, not as the full object", but I was not able to find such an attribute. If this is possible, I would love to know how.
public abstract class Edge
{
[JsonSerializeAsReferenceIfPossible]
public Vertex OriginVertex { get; set; }
[JsonSerializeAsReferenceIfPossible]
public Vertex DestinationVertex { get; set; }
[JsonSerializeAsReferenceIfPossible]
public Square LeftSquare { get; set; }
[JsonSerializeAsReferenceIfPossible]
public Square RightSquare { get; set; }
public EdgeDirection Direction { get; set; }
}
If the above is not possible, I am open to any idea or suggestion as to how to achieve this fixed-depth json that would be ideal both for human and machine readability.

How can I deserialize this specific json string?

I am trying to deserialize the following json string using Newtonsoft Json. I am able to get the string successfully, but when I try using JsonConvert.DeserializeObject<ServerList>(response, settings);, the try catch fails.
[
{"endpoint":"127.0.0.1","id":6,"identifiers":["steam:","license:","xbl:","live:","discord:"],"name":"Blurr","ping":160},
{"endpoint":"127.0.0.1","id":7,"identifiers":["steam:","license:","xbl:","live:","discord:"],"name":"Knight","ping":120}
]
I believe my issue is because the players array being unnamed.
I have tried [JsonProperty("")] and [JsonProperty] for the Users var, I have also tried using List and Array instead of IList.
Here is the object. This may be completely wrong, I have tried many ways of doing this.
public class ServerList
{
// I have tried many ways of doing this array/list. This is just the latest way I tried.
[JsonProperty]
public static IList<Player> Users { get; set; }
}
public class Player
{
[JsonProperty("endpoint")]
public static string Endpoint { get; set; }
[JsonProperty("id")]
public static string ServerId { get; set; }
[JsonProperty("identifiers")]
public static IList<string> Identifiers { get; set; }
[JsonProperty("name")]
public static string Name { get; set; }
[JsonProperty("ping")]
public static int Ping { get; set; }
}
I am expecting to get a 'ServerList' object returned with a list of all the players connected to the server.
Ask any questions you need to, I don't often work with json in this format. Thank you in advance!
ERROR: Error reading JObject from JsonReader. Current JsonReader item is not an object: StartArray. Path '', line 1, position 1.

Simplest way: Your json is an array, so deserialize to an array:
JsonConvert.DeserializeObject<Player[]>(response,settings);
As noted in the comments, properties on the Player should not be static.
If you insist on an object structure similar to what you posted, an object with a Usersproperty, even though it's not present in the JSON, that is also possible, by implementing a custom JSON converter. There's an article with an example of that here: https://www.jerriepelser.com/blog/custom-converters-in-json-net-case-study-1/
HOWEVER; I would recommend sticking to the types present in the json, and perhaps later construct the object that makes sense to the model in your program. This way, what you deserialize are true to the json you are getting, but make no sacrifices on the model you'd like:
var players = JsonConvert.DeserializeObject<Player[]>(response,settings);
var serverList = new ServerList {Users = players};

Auto-generate C# classes from JSON, including property initializers

There are a number of great ways to auto-generate C# code from JSON, such as here and here.
However, the resulting code doesn't include property initializers. For example, the following JSON:
{
"Name" : "Blastoise"
}
gets deserialized to this:
public class RootObject
{
public string Name { get; set; }
}
Presumably this is by design, since the values used in the JSON will probably be overridden anyways, so adding initializers might just annoy people who don't want them.
But what if I want that? Short of manually adding every value by hand, is there a way to deserialize JSON to the following?
public class RootObject
{
public string Name { get; set; } = "Blastoise";
}
Obviously in this case a manual edit is easy, but manual editing becomes tedious for larger JSON objects.

is there a way to deserialize JSON to the following?
Using the source code of the converter you mentioned.
A quick change at the line 204
sw.WriteLine(prefix + "public {0} {1} {{ get; set; }} = {2};", field.Type.GetTypeName(), field.MemberName, field.GetExamplesText());
gives me the result similar to what you described
internal class SampleResponse1
{
[JsonProperty("Name")]
public string Name { get; set; } = "Blastoise";
}

How to Deserialize XML elements to generic list with unknown element names

I am retrieving and successfully deserializing an xml string from a database table column for the known element names (these do not change) but there are also some nested XML Elements called "Other Attributes" which are not always going to be known. I'm having some trouble deserialising these unknown element names to a generic list so I can display them in html once deserialised.
The XML is as follows:
<Detail>
<DetailAttributes>
<Name>Name_123</Name>
<Type>Type_123</Type>
</DetailAttributes>
<OtherAttributes>
<SummaryKey AttributeName="SummaryKey">SummaryKey_123</SummaryKey>
<Account AttributeName="Account">Account_123</Account>
</OtherAttributes>
</Detail>
I have no problem deserialising the 'Name' and 'Type' elements and I can deserialise the 'SummaryKey' and 'Account' elements but only if I explicitly specify their element names - which is not the desired approach because the 'OtherAttributes' are subject to change.
My classes are as follows:
[XmlRoot("Detail")]
public class objectDetailsList
{
[XmlElement("DetailAttributes"), Type = typeof(DetailAttribute))]
public DetailAttribute[] detailAttributes { get; set; }
[XmlElement("OtherAttributes")]
public List<OtherAttribute> otherAttributes { get; set; }
public objectDetailsList()
{
}
}
[Serializable]
public class Detail Attribute
{
[XmlElement("Type")]
public string Type { get;set; }
[XmlElement("Name")]
public string Name { get;set; }
public DetailAttribute()
{
}
}
[Serializable]
public class OtherAttribute
{
//The following will deserialise ok
//[XmlElement("SummaryKey")]
//public string sumKey { get; set; }
//[XmlElement("Account")]
//public string acc { get; set; }
//What I want to do below is create a list of all 'other attributes' without known names
[XmlArray("OtherAttributes")]
public List<Element> element { get; set; }
}
[XmlRoot("OtherAttributes")]
public class Element
{
[XmlAttribute("AttributeName")]
public string aName { get; set; }
[XmlText]
public string aValue { get; set; }
}
When I try to retrieve the deserialised list of OtherAttribute elements the count is zero so it's not able to access the elements nested within "Other Attributes".
Can anybody help me with this please?

With concrete classes and dynamic data like this, you won't be able to lean on the standard XmlSerializer to serialize / deserialize for you - as it reflects on your classes, and the properties you want to populate simply don't exist. You could provide a class with all possible properties if your set of 'OtherAttributes' is known and finite, and not subject to future change, but that would give you an ugly bloated class (and I think you've already decided this is not the solution).
Practical options therefore:
Do it manually. Use the XmlDocument class, load your data with .Load(), and iterate the nodes using .SelectNodes() using an XPath query (something like "/Detail/OtherAttributes/*"). You will have to write the lot yourself, but this gives you complete control over the serialization / deserialization. You won't have to cover your code in (arguably superfluous!) attributes either.
Use Json.NET (http://james.newtonking.com/json), it allows for far greater control over serialization and deserialization. It's fast, has good docs and is overall pretty nifty really.

Serializing Name/Value Pairs in a Custom Object via Web Service

This is a very complicated question concerning how to serialize data via a web service call, when the data is not-strongly typed. I'll try to lay it out as best possible.
Sample Storage Object:
[Serializable]
public class StorageObject {
public string Name { get; set; }
public string Birthday { get; set; }
public List<NameValuePairs> OtherInfo { get; set; }
}
[Serializable]
public class NameValuePairs {
public string Name { get; set; }
public string Value { get; set; }
}
Sample Use:
[WebMethod]
public List<StorageObject> GetStorageObjects() {
List<StorageObject> o = new List<StorageObject>() {
new StorageObject() {
Name = "Matthew",
Birthday = "Jan 1st, 2008",
OtherInfo = new List<NameValuePairs>() {
new NameValuePairs() { Name = "Hobbies", Value = "Programming" },
new NameValuePairs() { Name = "Website", Value = "Stackoverflow.com" }
}
},
new StorageObject() {
Name = "Joe",
Birthday = "Jan 10th, 2008",
OtherInfo = new List<NameValuePairs>() {
new NameValuePairs() { Name = "Hobbies", Value = "Programming" },
new NameValuePairs() { Name = "Website", Value = "Stackoverflow.com" }
}
}
};
return o;
}
Return Value from Web Service:
<StorageObject>
<Name>Matthew</Name>
<Birthday>Jan 1st, 2008</Birthday>
<OtherInfo>
<NameValuePairs>
<Name>Hobbies</Name>
<Value>Programming</Value>
</NameValuePairs>
<NameValuePairs>
<Name>Website</Name>
<Value>Stackoverflow.com</Value>
</NameValuePairs>
</OtherInfo>
</StorageObject>
What I want:
<OtherInfo>
<Hobbies>Programming</Hobbies>
<Website>Stackoverflow.com</Website>
</OtherInfo>
The Reason & Other Stuff:
First, I'm sorry for the length of the post, but I wanted to give reproducible code as well.
I want it in this format, because I'm consuming the web services from PHP. I want to easily go:
// THIS IS IMPORANT
In PHP => "$Result["StorageObject"]["OtherInfo"]["Hobbies"]".
If it's in the other format, then there would be no way for me to accomplish that, at all. Additionally, in C# if I am consuming the service, I would also like to be able to do the following:
// THIS IS IMPORANT
In C# => var m = ServiceResult[0].OtherInfo["Hobbies"];
Unfortunately, I'm not sure how to accomplish this. I was able to get it this way, by building a custom Dictionary that implemented IXmlSerializer (see StackOverflow: IXmlSerializer Dictionary), however, it blew the WSDL schema out of the water. It's also much too complicated, and produced horrible results in my WinFormsTester application!
Is there any way to accomplish this ? What type of objects do I need to create ? Is there any way to do this /other than by making a strongly typed collection/ ? Obviously, if I make it strongly typed like this:
public class OtherInfo {
public string Hobbies { get; set; }
public string FavoriteWebsite { get; set; }
}
Then it would work perfectly, I would have no WSDL issues, I would be able to easily access it from PHP, and C# (.OtherInfo.Hobbies).
However, I would completely lose the point of NVP's, in that I would have to know in advance what the list is, and it would be unchangeable.. say, from a Database.
Thanks everyone!! I hope we're able to come up with some sort of solution to this. Here's are the requirements again:
WSDL schema should not break
Name value pairs (NVP's) should be serialized into attribute format
Should be easy to access NVP's in PHP by name ["Hobbies"]
Should be easy to access in C# (and be compatible with it's Proxy generator)
Be easily serializable
Not require me to strongly type the data
Now, I am /completely/ open to input on a better/different way to do this. I'm storing some relatively "static" information (like Name), and a bunch of pieces of data. If there's a better way, I'd love to hear it.

This is like dynamic properties for a object.
C# is not quite a dynamic language unlike javascript or maybe PHP can parse the object properties on the fly. The following two methods are what I can think of. The second one might fit into your requirements.
The KISS Way
The Keep It Simple Stupid way
public class StorageObject {
public string Name { get; set; }
public string Birthday { get; set; }
public List<string> OtherInfo { get; set; }
}
You can have name value pairs which is separated by '|'
OtherInfo = {"Hobbies|Programming", "Website|Stackoverflow.com"}
Serialized forms
<StorageObject>
<Name>Matthew</Name>
<Birthday>Jan 1st, 2008</Birthday>
<OtherInfo>
<string>Hobbies|Programming</string>
<string>Website|Stackoverflow.com</string>
</OtherInfo>
</StorageObject>
The Dynamic Way in C#
Make the name value pair part become an XML element so that you can build it dynamically.
public class StorageObject {
public string Name { get; set; }
public string Birthday { get; set; }
public XElement OtherInfo { get; set; } // XmlElement for dot net 2
}
You can easily build up OtherInfo object as element centric
e.g.
XElement OtherInfo = new XElement("OtherInfo");
OtherInfo.Add( ..Hobbies xelement & text value..);
OtherInfo.Add( ..WebSite xelement & text value..);
The serialized form will be
<OtherInfo>
<Hobbies>Programming</Hobbies>
<Website>Stackoverflow.com</Website>
</OtherInfo>
or build it as attribute centric
XElement OtherInfo = new XElement("OtherInfo");
OtherInfo.Add( ..nvp xattribute Hobbies & value..);
OtherInfo.Add( ..nvp xattribute WebSite & value..);
<OtherInfo>
<nvp n="Hobbies" v="Programming" />
<nvp n="Website" v="Stackoverflow.com" />
</OtherInfo>
For any dynamic language, it can access to the properties directly.
For the rest, they can access the value by read the XML. Reading XML is well supported by most of framework.

This is what I've settled on.
Class Structure:
public class StorageObject {
public string Name { get; set; }
public string Birthday { get; set; }
[XmlAnyElement("Info")] // this prevents double-nodes in the XML
public XElement OtherInfo { get; set; }
}
Usage:
StorageObject o = new StorageObject();
o.OtherInfo.Add(new XElement("Hobbies","Programming");
o.OtherInfo.Add(new XElement("Website","Stackoverflow.com");
Output:
<Info>
<Hobbies>Programming</Hobbies>
<Website>Stackoverflow.com</Website>
</Info>
I would like to thank everyone for their assistance, I really appreciate the help and ideas.

As a completely different take on this, why not think about doing it completely differently. Have one web service method to return the serialized storage object, minus the OtherInfo and another method to return the list of properties (keys) for OtherInfo, and a third to return the list of values for any key. Granted, it will take more round trips to the web service if you want all of the data, but the solution will be much simpler and more flexible.
[Serializable]
public class StorageObject {
public string Name { get; set; }
public string Birthday { get; set; }
[Nonserializable]
public Dictionary<string,List<string>> OtherInfo { get; set; }
}
[WebMethod]
public List<StorageObject> GetStorageObjects() {
// returns list of storage objects from persistent storage or cache
}
[WebMethod]
public List<string> GetStorageObjectAttributes( string name )
{
// find storage object, sObj
return sObj.Keys.ToList();
}
[WebMethod]
public List<string> GetStorageObjectAtributeValues( sting name, string attribute )
{
// find storage object, sObj
return sObj[attribute];
}

Have a look into the System.Xml.Serialization.XmlSerializerAssemblyAttribute attribute. This lets you specify a custom class-level serializer. You'll be able to spit out whatever XML you like.
A quick way to get up to speed on these is to use sgen.exe to generate one and have a peek at it with Reflector.
-Oisin

I'm not sure this would solve your problem (it would in C#, but maybe not in PHP), but try using Dictionary<string,List<string>> OtherInfo instead of List<NameValuePairs>. Then "Hobbies" and "Websites" would be your keys and the values would be the list of hobbies or web sites. I'm not sure how it would serialize, though.
You would be able to reference the lists of hobbies as:
List<string> hobbies = storageObject.OtherInfo["Hobbies"];
[EDIT] See here for a generic XML serializable dictionary. This derived class is the one you would need to use instead of generic Dictionary.