This question already has an answer here:
Why my dull Newtonsoft.Json deserialization code does not work? [duplicate]
(1 answer)
Closed last month.
I am trying to serialize a tree but I only need a tiny part of the data of the object (its a UI tree), so I wrote a custom converter.
The converter simply passes the reader and writer to the object
public override void WriteJson(JsonWriter writer, NavTree value, JsonSerializer serializer)
{
value.SaveAsJson(writer);
}
public override NavTree ReadJson(JsonReader reader, Type objectType, NavTree existingValue, bool hasExistingValue,
JsonSerializer serializer)
{
NavTree tree = hasExistingValue ? existingValue : new NavTree();
tree.LoadFromJson(reader);
return tree;
}
Serialization looks like this
public void SaveAsJson(JsonWriter writer)
{
SerializableTreeItem root = new (this.GetRoot());
JObject.FromObject(root).WriteTo(writer);
}
The object appears to serialize yielding json that looks something like
"NavTree": {
"Id": "All",
"IsCategory": true,
"Children": [
{
"Id": "https://popularresistance.org/feed/",
"IsCategory": false,
"Children": []
},
{
"Id": "https://www.aljazeera.com/xml/rss/all.xml",
"IsCategory": false,
"Children": []
},
... more children
The deserialization looks like:
public void LoadFromJson(JsonReader reader)
{
SerializableTreeItem loaded =
JsonConvert.DeserializeObject<SerializableTreeItem>((string)reader.Value ?? string.Empty);
if (loaded == null) return;
if (this.GetRoot() != null)
{
this.GetRoot().Free();
TreeItem root = this.CreateItem();
root.SetMetadata(0, RootMetaDataId);
}
this.AddItem(loaded, this.GetRoot());
}
Trying to access reader.Value at the start of the function returns null.
Trying to access reader.ReadAsString() at the start results in:
Newtonsoft.Json.JsonReaderException: Unexpected state: ObjectStart. Path 'NavTree', line 669, position 14.
at Newtonsoft.Json.JsonTextReader.ReadStringValue(ReadType readType)
at Newtonsoft.Json.JsonTextReader.ReadAsString()
at Porifera.NavTree.LoadFromJson(JsonReader reader)
Line 669 is the first line of the json posted above. I never made a custom converter before so clearly I messed it up. The question is what did I do wrong? The json looks ok to me and all I really need is for the reader to deliver something and I can reconstruct the object.
You are using SerializableTreeItem as a data transfer object for NavTree:
In the field of programming a data transfer object (DTO) is an object that carries data between processes.
What you should do is to refactor your code to separate the responsibilities for converting from JSON to your DTO, and from your DTO to your NavTree.
First, modify NavTree to remove all references to JsonReader or any other JSON types:
public partial class NavTree
{
public void PopulateFromSerializableTreeItem(SerializableTreeItem loaded)
{
if (loaded == null)
return;
if (this.GetRoot() != null)
{
this.GetRoot().Free();
TreeItem root = this.CreateItem();
root.SetMetadata(0, RootMetaDataId);
}
this.AddItem(loaded, this.GetRoot());
}
public SerializableTreeItem ToSerializableTreeItem()
=> new (this.GetRoot());
}
Now, rewrite your JsonConverter<NavTree> as follows:
public class NavTreeConverter : Newtonsoft.Json.JsonConverter<NavTree>
{
public override void WriteJson(JsonWriter writer, NavTree value, JsonSerializer serializer) =>
serializer.Serialize(writer, value.ToSerializableTreeItem());
public override NavTree ReadJson(JsonReader reader, Type objectType, NavTree existingValue, bool hasExistingValue,
JsonSerializer serializer)
{
var loaded = serializer.Deserialize<SerializableTreeItem>(reader);
// Check for null and return null? Throw an exception?
var tree = hasExistingValue ? existingValue : new NavTree();
tree.PopulateFromSerializableTreeItem(loaded);
return tree;
}
}
And you should be good to go.
Notes:
Your JsonReaderException is caused specifically by the following line:
SerializableTreeItem loaded =
JsonConvert.DeserializeObject<SerializableTreeItem>((string)reader.Value ?? string.Empty);
JsonReader.Value is the value of the current JSON token, but you are using it as if it contained the entire JSON subtree corresponding to your SerializableTreeItem. Instead, use JsonSerializer.Deserialize<T>(JsonReader) to deserialize the JSON subtree anchored by the current JSON token.
When writing, there should be no need to serialize your SerializableTreeItem to a JObject, then write the JObject. Just serialize SerializableTreeItem directly and skip the intermediate JObject representation.
By separating JSON serialization from DTO conversion, you will be able to more easily port your serialization code to System.Text.Json or any other serializer, if you eventually chose to do so.
Related
I am trying to deserialize the following JSON (which validates on https://jsonlint.com/):
{"pandoc-api-version":[1,22],"meta":{"title":{"t":"MetaBlocks","c":[{"t":"Para","c":[{"t":"Str","c":"Dynamic"},{"t":"Space"},{"t":"Str","c":"Language"},{"t":"Space"},{"t":"Str","c":"Runtime"}]},{"t":"Para","c":[]}]}},"blocks":[{"t":"Para","c":[{"t":"Strong","c":[{"t":"Str","c":"Bill"},{"t":"Space"},{"t":"Str","c":"Chiles"},{"t":"Space"},{"t":"Str","c":"and"},{"t":"Space"},{"t":"Str","c":"Alex"},{"t":"Space"},{"t":"Str","c":"Turner"}]}]},{"t":"Para","c":[{"t":"Emph","c":[{"t":"Strong","c":[{"t":"Str","c":"Reading"},{"t":"Space"},{"t":"Str","c":"this"},{"t":"Space"},{"t":"Str","c":"Document:"}]}]}]}]}
into the following classes:
internal record TagContent(string T, OneOf<TagContent[], string>? C);
internal class RawPandoc {
[JsonProperty] public int[] PandocApiVersion = default!;
[JsonProperty] public Dictionary<string, TagContent> Meta = default!;
[JsonProperty] public TagContent[] Blocks = default!;
}
using the following code:
var settings = new JsonSerializerSettings {
ContractResolver = new DefaultContractResolver { NamingStrategy = new KebabCaseNamingStrategy() },
Converters = new JsonConverter[] { new OneOfJsonConverter() }
};
var pandoc = JsonConvert.DeserializeObject<RawPandoc>(s, settings);
and I get the following error:
Unexpected token when deserializing object: StartObject. Path 'meta.title.c[0]', line 1, position 69.
How can I resolve this?
For completeness, here is the current and incomplete code for OneOfJsonConverter. OneOf is a library for union types in C#:
using OneOf;
namespace PandocFilters {
public class OneOfJsonConverter : JsonConverter {
public override void WriteJson(JsonWriter writer, object? value, JsonSerializer serializer) {
if (value is IOneOf of) {
value = of.Value;
}
serializer.Serialize(writer, value);
}
public override object? ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer) {
if (reader.Value is null) { return null; }
// TODO not implemented yet
return reader.Value;
}
public override bool CanConvert(Type objectType) => objectType.UnderlyingIfNullable().GetInterfaces().Contains(typeof(IOneOf));
}
}
Problem is you are not advancing the reader in your ReadJson implementation. You declared your converter can handle IOneOf objects, and so JSON.NET expects your converter to actually read and handle it, however it does nothing as of now. So ReadJson is called (at the start of first array in json which should be deserialized to OneOf), and then after it returns - reader position is still where it was before (at start of array), which is not what JSON.NET expects. Then it fails trying to continue reading next object, because its assumptions are violated. So, just implement ReadJson, and meanwhile you can advance a reader for example like that:
public override object? ReadJson(JsonReader reader, Type objectType, object? existingValue, JsonSerializer serializer) {
// advance reader as expected
var eitherStringOrArray = JObject.ReadFrom(reader);
return reader.Value;
}
https://dotnetfiddle.net/ka6XVw - Fiddle with example type structure
Suppose I have a class that implements IDictionary<string, T>. Json.Net can deserialize such types out of the box, creating an instance of the type and using its indexer to populate the dictionary. The issue is that this class also inherits a string Error property marked with JsonProperty attribute from its base class, and I'd like this property to be populated whenever the input json contains an error field. However, when deserializing an IDictionary Json.Net considers all fields to be dictionary entries and tries to add the value with the error key to the dictionary.
What is the simplest and cleanest way to deserialize the json into a dictionary and the error field into the Error property? Please note that the class is generic, so JsonExtensionData is not an option (without casting its values to the provided type).
Sample valid dictionary json: { 'foo': '1', 'bar': '2' }
Sample error json { 'error': 'blah' }
I've derived a converter solution from this question. Basically, you attach a converter to your DictionaryResponse class, and interpret the incoming JSON yourself. I was lazy enough to use a JObject for parsing:
class DictionaryResponseConverter : JsonConverter<ResponseBase>
{
public override ResponseBase ReadJson(
JsonReader reader, Type objectType,
ResponseBase existingValue, bool hasExistingValue,
JsonSerializer serializer)
{
// find the correct T and call the internal function through reflection
// as DictionaryResponse<T> is sealed, we don't care about inheritance
return (ResponseBase)GetType()
.GetMethod(nameof(InternalReadJson),
BindingFlags.Instance | BindingFlags.NonPublic)
.MakeGenericMethod(objectType.GetGenericArguments()[0])
.Invoke(this, new object[]
{
reader,
existingValue,
hasExistingValue,
serializer
});
}
DictionaryResponse<T> InternalReadJson<T>(
JsonReader reader,
DictionaryResponse<T> existingValue, bool hasExistingValue,
JsonSerializer serializer)
{
if (reader.TokenType == JsonToken.Null)
return null;
var obj = JObject.Load(reader);
var error = (string)obj["error"];
bool hadError = obj.Remove("error");
//var result = new DictionaryResponse<T>();
var result = hasExistingValue ? existingValue : new DictionaryResponse<T>();
foreach (var kvp in obj)
result[kvp.Key] = kvp.Value.ToObject<T>();
if (hadError)
result.Error = error;
return result;
}
public override void WriteJson(
JsonWriter writer, ResponseBase value, JsonSerializer serializer)
{
// don't care about serialization
throw new NotImplementedException();
}
}
[JsonConverter(typeof(DictionaryResponseConverter))]
internal sealed class DictionaryResponse<T> : ResponseBase, IDictionary<string, T>
{
...
Using json.net this test fails by default:
JsonConvert.DeserializeObject(
JsonConvert.SerializeObject(new object()),
typeof(object)
).ShouldBeOfType<object>(); // actual type is JObject
Is there a way to change this behavior, so it deserializes to the actual requested type?
You have a degenerate test case there. If you instruct Json.Net to deserialize into type object, you are telling it that the JSON could represent any possible object. So it will choose to use a JObject in that case, since you were not specific and a JObject can handle any JSON object. It is not expecting that you want to deserialize into a literal empty object instance, because that is not a very useful thing to do. If the JSON contained any data at all, you would not be able to access that data after the deserialization: object has no properties!
You can fix your test by creating an empty class Foo and using that in place of object:
JsonConvert.DeserializeObject(
JsonConvert.SerializeObject(new Foo()), typeof(Foo)
).ShouldBeOfType<Foo>();
If you really do need to force Json.Net to deserialize into an empty object instance whenever object is specified as the type, you can do it using a custom JsonConverter like this:
public class EmptyObjectConverter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return objectType == typeof(object);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
JToken token = JToken.Load(reader); // consume the JSON object from the reader
return token.Type == JTokenType.Null ? null : new object();
}
public override bool CanWrite
{
get { return false; }
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
throw new NotImplementedException();
}
}
Then pass an instance of the converter to JsonConvert.DeserializeObject():
JsonConvert.DeserializeObject(
JsonConvert.SerializeObject(new object()),
typeof(object),
new EmptyObjectConverter()
).ShouldBeOfType<object>();
Fiddle: https://dotnetfiddle.net/7xZ7tm
I'm creating a custom JsonConverter to convert a property "Type" : "A" (or "D", "E", etc) to an enum that contains Article, Disambuigation, etc. Here is the relevant code in the converter class:
public override bool CanConvert(Type objectType) => objectType == typeof(SearchResultType);
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var jsonObject = JObject.Load(reader);
var typeProperty = jsonObject.Properties().FirstOrDefault(p => p.Name == "Type");
switch ((string)typeProperty.Value)
{
case "A": return SearchResultType.Article;
default: return SearchResultType.None;
}
}
My problem is that it always throws an exception with message:
Error reading JObject from JsonReader. Current JsonReader item is not an object: String.
I understand this to mean that it expected something like "Name" : "Hello world!" but got "Name" : 123. However, I know for sure that the properties it's failing to parse are strings. Is there a problem with my converter?
Yes, there is a problem with your converter. Your converter is intended to read a string value from JSON to convert it into an enum value. A string value is not an object in JSON, therefore you cannot load it into a JObject. Instead, you can just get the string value directly from the reader:
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var type = (string)reader.Value;
switch (type)
{
case "A": return SearchResultType.Article;
case "D": return SearchResultType.Disambuigation;
...
default: return SearchResultType.None;
}
}
Fiddle: https://dotnetfiddle.net/tnyZaT
I have a file with some json in it, that was generated by json.net:
[
{
"$type": "Dashboard.Gauges.LabelGaugeSeed, Dashboard",
"Text": "blah",
"LabelColor": {
"X": 1.0,
"Y": 1.0,
"Z": 1.0,
"W": 1.0
},
"Center": {
"X": 0.0,
"Y": 0.0
},
"CharacterWidth": 0.05,
"CharacterHeight": 0.1,
"LineThickness": 0.01,
"TextCentering": 0.5
}
]
Which is giving me the aforementioned error while deserializing. Can anyone find a problem with this json? I ran it through a verifier and it said it was fine.
It errors on the space after "Center": If I change the order of the Center and the LabelColor properties, it then errors in the same manner after "LabelColor":
Here is a dump of the types:
LabelColor is an OpenTK Vector4, Center is an OpenTK Vector2, and LabelGaugeSeed is below:
public class LabelGaugeSeed : IGaugeSeed
{
public IGauge Grow()
{
return new LabelGauge(this);
}
public string Text;
[JsonConverter(typeof(Vector4Converter))]
public Vector4 LabelColor;
[JsonConverter(typeof(Vector2Converter))]
public Vector2 Center;
public float CharacterWidth;
public float CharacterHeight;
public float LineThickness;
public float TextCentering;
}
And here is Vector4Converter:
public class Vector4Converter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return objectType == typeof(Vector2);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
reader.Read();
reader.Read();
var x = serializer.Deserialize<float>(reader);
reader.Read();
reader.Read();
var y = serializer.Deserialize<float>(reader);
reader.Read();
reader.Read();
var z = serializer.Deserialize<float>(reader);
reader.Read();
reader.Read();
var w = serializer.Deserialize<float>(reader);
return new Vector4(x, y, z, w);
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
Vector4 vectorValue = (Vector4)value;
writer.WriteStartObject();
writer.WritePropertyName("X");
writer.WriteValue(vectorValue.X);
writer.WritePropertyName("Y");
writer.WriteValue(vectorValue.Y);
writer.WritePropertyName("Z");
writer.WriteValue(vectorValue.X);
writer.WritePropertyName("W");
writer.WriteValue(vectorValue.Y);
writer.WriteEndObject();
}
}
Vector2Converter is exactly the same, except for it doesn't contain code for the Z and W properties, and the name is different.
The deserialization process will go through the first one successfully, but not even get into the second one.
The Vector classes can be found here: https://github.com/opentk/opentk/tree/develop/Source/OpenTK/Math
Tl;dr - The problem was in your Vector[X]Converters. You read all the properties, but didn't actually navigate to the end of the object. You need an extra reader.Read() right before the last line where you return your concrete object.
The deeper explanation:
JSON.NET is picky about the state of the JsonReader when you are writing a custom converter. You need to traverse all the way to the end of the reader, regardless of whether you actually need the remaining data (in other words, you can't return early).
In this example, you read the value you wanted (W), and then immediately returned a new concrete object since you have all the data you need. However, the JsonReader is still examining the property node, so JSON.NET thinks that there's still data waiting to be deserialized. This is why you get
Additional text found in json string after finishing deserializing object
You can see this yourself if you put a breakpoint inside your JsonConverter and watch the state of the reader object as you advance through the tokens. On the last one, the state is:
...
Path: "LabelColor.W"
TokenType: Float
Value: 1.0
....
If you leave the JsonReader in that state, you'll get an error. But, if you do one last reader.Read(), then the state is:
...
Path: "LabelColor"
TokenType: EndObject
Value: null
...
Now JSON.NET is happy!
The easiest way to do the read is to load into a JToken and access the properties by name, like so:
public class Vector4Converter : JsonConverter
{
public override bool CanConvert(Type objectType)
{
return objectType == typeof(Vector4);
}
public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
{
var token = JToken.Load(reader);
var vec = new Vector4();
if (token["X"] != null)
vec.X = (float)token["X"];
if (token["Y"] != null)
vec.Y = (float)token["Y"];
if (token["Z"] != null)
vec.Z = (float)token["Z"];
if (token["W"] != null)
vec.W = (float)token["W"];
return vec;
}
public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
{
Vector4 vectorValue = (Vector4)value;
writer.WriteStartObject();
writer.WritePropertyName("X");
writer.WriteValue(vectorValue.X);
writer.WritePropertyName("Y");
writer.WriteValue(vectorValue.Y);
writer.WritePropertyName("Z");
writer.WriteValue(vectorValue.Z);
writer.WritePropertyName("W");
writer.WriteValue(vectorValue.W);
writer.WriteEndObject();
}
}
Loading into a JToken and accessing by name also allows the sender to write the JSON properties in any order, which is preferred since the JSON spec states that property name/value pairs are unordered.
Also, notice I fixed a couple mistakes in WriteJson (the code wrote X and Y twice) and a mistake in CanConvert.